Tiny Benchmark Test

[Mike Lobanovsky]

The problem is the way how Aurel gets his scripts parsed into lines.

He creates a hidden richedit20a control and streams the script file into it. Then he reads lines of code from it one by one using SendMessage(EM_GETLINE) and stores them in his own lines[] array - empty lines, rem lines, code lines - all of them without any filtering.

This is a very dirty hack. This control requires initialization before use and its implementation is very platform-specific. What may work for richedit20a under a (Chroatian?) XP Sp2 or a British Vista might not work under a Russian XP Sp3 without proper initialization: half of the lines loaded into the uninitialized control on my machine are simply not read back into the lines[] array.

I think a better design decision would be to discard the richedit20a control altogether and split the file into lines in a custom-coded Split() function based on LF's and having filtered off CR's. Half of modern text editors save their texts in a Unix format without CR (0xD) characters and line parsing based on CRLF's only may simply not work.

Having samples of your code, will help me detect unforseen problems

He-he-he-he-he.... Now I see what you mean.

[Aurel]

This is a very dirty hack

Ok if you say so..
Yes it is not the best way...
But i have found what might be wrong about intialization of richedit control.
In awinh i have a wrong name of dll ....
riched32 =  LoadLibrary "riched32.dll"
but must be ...
riched32 =  LoadLibrary "riched20.dll"
i tried again and it looks that work....
i can only...hope that will work with your xp sp3 ...russian edition. 



.

[Mike Lobanovsky]

Aurel,

1. I got ruben2 running. The offending routine appeared to be ClearGlobalArray(). It must be commented out, otherwise the process memory gets randomly corrupted. These arrays don't require clearing at the current stage of your project development. They are filled up only once per app run and they are effectively cleared when dimensioned at app start.

I do not think it is entirely your fault; it may still be related to string memory leaks in Oxygen (see my message to Charles below).

2. ruben2 is going to be as slow as it is so long as it uses strings for all its operations. You can't improve it radically without minimizing or eliminating string operations from the loop altogether. You're using strings in very many places where you should use numeric tokens and numeric literals. Your tree runs 3.5 seconds on my PC and I think it is close to how fast it can only be until you eliminate string operations in the interpreting loop.

3. Your drawing functions are leaking GDI objects heavily. The tree generates about 7,500 orphaned pens and brushes. Please open up your Task Manager, go to View->Select Columns... and tick the GDI Objects check box. Now you will be able to see a new GDI Objects column in your table and you will be able to control the quality of drawing functions you add to ruben2.

4. Overwrite the respective functions in your ruben2 with the following code. It will prevent GDI leakage - you'll see that your tree needs only about 55 GDI objects to draw itself.

SUB LineXY (wID as INT,byval x as INT,byval y as INT,byval x1 as INT,byval y1 as INT)
    hdc = GetDC(wID)
    GetSize(wID,0,0,ww,hh)

    int np = CreatePen(PS_SOLID,1,fColor)
    int op = SelectObject(hdc, np)

    MoveToEx hdc,x,y,Byval 0
    LineTo hdc,x1,y1
    BitBlt(hDCmem, 0, 0, ww, hh, hdc, 0, 0, SRCCOPY)
   
    DeleteObject(SelectObject(hdc, op))
    ReleaseDC( wID, hdc)
End SUB

SUB Circle (wID as INT, byval cix as INT, byval ciy as INT, byval cra as INT)
    hdc = GetDC(wID)
    GetSize(wID, 0, 0, ww, hh)
   
    int np = CreatePen(PS_SOLID, 1, fColor)
    int op = SelectObject(hdc, np)

    Ellipse hdc, cix-cra, ciy-cra, cra+cix, cra+ciy
    BitBlt(hDCmem, 0, 0, ww, hh, hdc, 0, 0, SRCCOPY)
   
    DeleteObject(SelectObject(hdc, op))
    ReleaseDC( wID, hdc)
End SUB

Sub FillSolidRect(wID as INT, x As Long, Y As Long, cx As Long, cy As Long, bColor as INT)
    Dim hBr As Long ' rc As RECT
    hDC = GetDC(wID)
    rc.Left = x
    rc.Top = Y
    rc.right = x + cx
    rc.bottom = Y + cy
    hBr = CreateSolidBrush(bColor)
   
    FillRect hDC, rc, hBr
    BitBlt(hDCmem, 0, 0, ww, hh, hdc, 0, 0, SRCCOPY)

    DeleteObject(hBr)
    ReleaseDC(wID, hdc)
End Sub

SUB CleanUp
    DeleteObject(SelectObject(hdcMem, oldBrush))
    DeleteObject(SelectObject(hdcMem, oldPen))
    DeleteObject(SelectObject(hdcMem, oldBmp))
    DeleteDC(hdcMem)
End SUB

Hope this helps.

.

[Mike Lobanovsky]

Hi Charles,

I think there are still problems with string garbage collection in Oxygen. While discrete functions seem to be clean enough when used with proper notation, some weird cases may still fool the GC and cause severe memory leaks.

For example, Aurel's tree uses 1.5GB of memory while drawing. Other .RUB examples are not much better in this regard. I didn't manage to check every line of his code for proper string handling but I did test separate basic string functions and they seem to be OK at a first glance.

However despite all the functions being perfectly correct and working, the following queer example of mine (and this is legitimate Oxygen syntax!) crashes in about 2 seconds from app start having exhausted 2GB of XP process memory:

$Filename "test.exe"
Include "RTL32.inc"
#Lookahead

String s = Space(2500)

While 1
   Replace s, "A", "A"
WEnd

Function Replace(String t, w, r) As String
   '=======================================
   '
   Sys a, b, lw, lr
   String s = t
   '
   lw = Len(w)
   lr = Len(r)
   a = 1
   ' 
   Do
      a = InStr(a, s, w)
      If a = 0 Then Exit Do
      s = Left(s, a - 1) + r + Mid(s, a + lw)
      a += lr
   End Do
   Return s
End Function

I realize that this is not an expected way to use Replace() in a BASIC but Oxygen's Functions are not supposed to differ from Subs so in a generic sense, this looks like an obvious bug.

You can replace Replace s, "A", "A" (pun not intended) with Space 2500 in this While/WEnd loop and see similar memory leakage.

I also remember Aurel (I wasn't yet registered on the forum then) complaining that the evaluator code you published seemed to leak memory. This is what I think is still happening now massively in his ruben2 code.

[Charles Pegge]

Many thanks Mike,

I am reviewing the garbage collector's logging-points , and will also trap calls to string functions and float functions, which neglect to assign the returned value to a variable.

[Aurel]

Thank you very much MIke 
Your investigation is great...and yes you have a right about GDI objects...
Infact i have never use this option from taskManager ..    before.

About strings ..hmmm yes oxygen have a real problems with strings but after all my
investigation and trying to do best i know i use tronD( purebasic guy) evaluator which
don't create mem-leaks...
thanks again...

[Peter]

Hello together,

[Mike Lobanovsky]

No Peter,

Your code will leak GDI pens as heavily as Aurel's did before my correction. We discussed this matter with you in the past and I gave you the appropriate links to MSDN.

You cannot delete a GDI object while it is still selected into a device context, and your DeleteObject iPen will simply fail. The iPen object will stay in hdc until the function is called next time. Then it will be replaced with a new iPen in a new call to SelectObject hdc, iPen but the old pen object will stay undeleted. This is because in the new call, iPen already refers to another newly created pen and this call will also fail like the old one.

You must always store the old GDI objects when you make a call to SelectObject() and re-select them afterwards into the device context with another call to this function. The second SelectObject() call will return the de-selected GDI object's handle that may be used to delete this object.

That said, your Sub should be changed to the following in an efficient C-style notation:

Sub LineXY(int hdc, x, y, a, b, color)
    iPen = CreatePen 0,1,color
    int oldPen = SelectObject(hdc, iPen)
    MoveToEx hdc,x,y,NULL
    LineTo hdc,a,b
    DeleteObject(SelectObject(hdc,oldPen)) // SelectObject returns iPen here
End Sub

or atomically in your more common BASIC-style notation:

Sub LineXY(int hdc, x, y, a, b, color)
    iPen = CreatePen 0,1,color
    int oldPen = SelectObject(hdc, iPen)
    MoveToEx hdc,x,y,NULL
    LineTo hdc,a,b
    SelectObject hdc,oldPen
    DeleteObject iPen
End Sub

Please do not argue, Peter. Your code is outright buggy and you are teaching people wrong things again.

[Aurel]

As i say before i have never look in TM GDI objects so i cannot say is Peters functions
leak in GDI.
Mike you say 55 GDI objects ...
on new way i get even less ....48 ..which is good i think 

[Aurel]

2. ruben2 is going to be as slow as it is so long as it uses strings for all its operations. You can't improve it radically without minimizing or eliminating string operations from the loop altogether. You're using strings in very many places where you should use numeric tokens and numeric literals.

Hi Mike
i agree with you in your point that i use lot of string operations
BUT not in main interpreter loop...
Main loop in my case ..as you can see select integer(sys) variable...
not string..right?
BUT this integer is a index of integer array

key = tok[LineNum]

Maybe is that problematic in a terms of speed...
I know that would be far better way to translate to bytecode ..
As i say many times i have code of DLib which compile source program to bytecode file
and then interpreter(VM) execute this file ...but i don't know how to properly translate this
PureBasic code to oxygen which is very complex and PureBasic very specific.
i still don't catch aa time to try translate Ed toy2 interpreter to oxygen that can give me some
directions.

[Mike Lobanovsky]

Hi Aurel,

As i say before i have never look in TM GDI objects so i cannot say is Peters functions leak in GDI.

This is not a question of trial and error here. Any piece of code written like Peter's function must leak GDI objects because it denies the basic principles of working with the Windows device context. I pointed Peter to the MSDN pages and I even sent him a stand-alone Windows SDK help file while you were still discussing publicly "who is a better programmero".

If he still prefers to ignore both MSDN and my recommendations, then let him do it in his own closed-source code. But let's not let him do it publicly. This is because some newbie may copy-paste this "masterpiece" into their own code and then run around the net calling Windows a piece of sh*t because it is Windows that is allegedly leaking tons of its GDI objects. A newbie cannot tell right from wrong at a glance - but I can.

Mike you say 55 GDI objects ...
on new way i get even less ....48 ..which is good i think 

In fact, the actual number is highly platform dependent and what is 48 under your XP may be 55 under mine or even 155 under Peter's Windows 7 or Charles' Vista. But this number must stay invariable for as long as the drawing loop runs.

GDI objects are system objects but even though their number can be very large on a given platform, it is still a finite value. Other concurrent programs may suffer a visible lag or even crash if the system runs out of its GDI memory pool due to some ignorant piece of code running in an endless loop.

[Mike Lobanovsky]

Main loop in my case ..as you can see select integer(sys) variable...
not string..right?
BUT this integer is a index of integer array

But the arguments that your evaluator uses in response to a tokenized command come from the string argX[] arrays.

OTOH such code is a very good test case material for Charles to debug and polish his memory garbage collector.

i still don't catch aa time to try translate Ed toy2 interpreter to oxygen that can give me some
directions.

Where's that interpreter's code, please? But open up a new thread for that on an appropriate board. We already have so much info here that's irrelevant to the topic starter's original message...

[Peter]

Do not tell so much nonsense here!
We select for the Device Contex! Nothing will stored in the Device Context!

Select(Hdc, handle) means:  handle is for the Device Context HDC!
OldHandle = Select(hdc, handle) means: You may restore the previous handle, and nothing more.

And again, do not call me idiot!
Memory leaks are in you, not in my  works.

In other words, our friendship ends now here!

[Mike Lobanovsky]

OldHandle = Select(hdc, handle) means: You may restore the previous handle, and nothing more.

You must restore the old handle to be able to free the new one before you can delete it. If you do not take the new handle out of hdc, you can not delete it because your DeleteObject() call will simply fail silently. You can check it by printing the DeleteObject()'s return value: your calls will return zero which means failure. My calls will return non-zero which means success.

, Peter.

And again, do not call me idiot!
Memory leaks are in you, not in my  works.

Where did you see me calling you an "idiot"?

On the contrary, this is where you're calling me, MSDN, and Microsoft idiots:

Do not tell so much nonsense here!

In other words, our friendship ends now here!

Was there any really?

.

[JRS]

Count to 10 first Peter before getting pissed off and leaving the forum. I will not relink your ghost posts again. No one here is perfect and knows all. (maybe Charles  )