Tiny Benchmark Test

[Arnold]

Hello,


In the Qdepartment forum I found this little code snippet which I tried with Oxygenbasic:

This is the code in C:

Code: [Select]


// gcc -03 prime_test.c -o prime_test.exe
//
#include <stdio.h>
#include <time.h>
 
int main() {
     int n, lim;
     int k, p;
     int pc;

int t1,t2,time_lapsed;
printf ("%s\n","Starting prime numbers:" );
t1=clock();

    pc = 0;
     n  = 1;
     lim = 5000000;
     while (n < lim) {
         k = 3;
         p = 1;
         n = n + 2;
         while (k * k <= n && p) {
             p = n / k * k != n;
             k = k + 2;
         }
         if (p) {
             pc = pc + 1;
         }
     }

t2=clock();
time_lapsed=(t2-t1) / 1000;

     printf("%d,\n", pc);
printf("%d%s\n", time_lapsed," Seconds");
printf("%s", "Enter ...");
getchar();

     return 0;
 }



This is the code in Oxygenbasic:

Code: [Select]


$ filename "prime_test2.exe"

includepath "$/inc/"

'#include "RTL32.inc"
'#include "RTL64.inc"

include "console.inc"


! GetTickCount lib "kernel32.dll" alias "GetTickCount" () as dword
 
sub main() {
     int n, lim;
     int k, p;
     int pc;

int t1,t2, time_lapsed
print "Starting Prime Numbers:" + cr + cr
t1=GetTickCount()

    pc = 0;
     n  = 1;
     lim = 5000000;
     while (n < lim) {
         k = 3;
         p = 1;
         n = n + 2;
         while (k * k <= n && p) {
             p = n / k * k != n;
             k = k + 2;
         }
         if (p) {
             pc = pc + 1;
         }
     }
     print( pc);
     
t2=GetTickCount()
time_lapsed = (t2-t1) / 1000 + cr

print cr + cr + "Time: " +time_lapsed + " Seconds" + cr + cr
print "Enter ..."
GetKey() 
   
     return 0;
end sub

main()


The result is: 348512

I made these observations:
Oxygenbasic has a very flexible syntax, it can handle a subset of C apart from understanding assembly language.
The filesize generated with gcc is about 89 kb. The Filesize generated with Oxygenbasic is about 16 kb.
The execution time on my (old) machine is nearly the same (about 9 to 10 Seconds).

So Oxygenbasic is very fast?


Roland

[Mike Lobanovsky]

Hello Roland,

Both gcc and O2 do exactly the same thing; they compile their respective source scripts into native machine code. The only difference here is that gcc always outputs the code to a disk file for storage and further utilization while O2 can (and very often does) execute it immediately from memory without storing its image to a disk file. So there's no surprise the benchmark results for such a simple and straight-forward integer math task are nearly identical.

OTOH gcc can perform transparently very intricate and clever code optimizations for more elaborate math-intensive scripts.  The -O3 switch you used for gcc compilation enables such optimization to the maximum. The inevitable penalty of optimization is however much, much longer compilation time and also much larger size of resultant disk file which carries in itself many different algorithms and architecture-specific facilities to check for and use in various brands of CPU's your code may run on. Just remove the -O3 switch altogether or change it to -Os to tell gcc to optimize the resultant exe for the smallest size possible. But even then, the resultant exe size will also be largely dependent on the gcc version you're using. The lower (and hence older and simpler) the version, the smaller the output file size.

[Arnold]

Hello Mike,

thank you for your reply. My intention was not to emphasize neither benchmark nor filesize. There is no doubt that C is a mighty language and gcc is a powerful tool which I (unfortunately) will not really ever understand. But I found this little code and wanted to know how it works with O2h. The first surprise for me was that Oxygenbasic can understand some C syntax - I did not realize this until then. The second surprise for me was that the execution time was nearly the same using both gcc and o2h with my notebook. And this seems to be very fast compared with other languages. But currently I cannot verify this.

Roland

[Charles Pegge]

Yes, it generates efficient machine code for integer and float operations. You will often find that there is very little advantage in using assembly code to replace numeric expressions. There are, of course, many opportunities for improvement

I suspect that GCC optimisation involves turning function calls into inline code. This avoids the overhead of function prologs and epilogs, but it will usually make the binary significantly larger.

To achieve in-lining, procedures can be turned into macros quite easily. (Just make sure that expression params are enclosed in brackets)

sub abc(sys a,b,c, string d)
...
end sub

macro(a,b,c,d)
...
end macro

if the procedure uses local variables, then the macro must use a scope to confine them:

macro(a,b,c,d)
  scope
  ...
  end scope
end macro

[Arnold]

This closes a gap in my knowledge and gives new prospects. I found several files which contain macros and wondered about their purpose. But now (sub/func with inline coding) it makes sense.

There is a lot to be discovered in Oxygenbasic. At least for me.

Roland

[Ed Davis]

This is pretty cool, that you can compile using C-like syntax.  I thought I'd give it a try, but I'm running into an error that I can't figure out:

Code: [Select]

$ filename "prime_test2.exe"

includepath "$/inc/"

'#include "RTL32.inc"
'#include "RTL64.inc"

include "console.inc"


! GetTickCount lib "kernel32.dll" alias "GetTickCount" () as dword

sub main() {
    int left_edge, right_edge, top_edge, bottom_edge, max_iter,
    x_step, y_step, y0, x0, x, y, i, x_x, y_y, temp, the_char,
    accum, count;

int t1,t2, time_lapsed
print "Starting Mandel accum:" + cr + cr
t1=GetTickCount()

    accum = 0;
    count = 0;
    while (count < 1545) {
        left_edge   = -420;
        right_edge  =  300;
        top_edge    =  300;
        bottom_edge = -300;
        x_step      =  7;
        y_step      =  15;

        max_iter    =  200;

        y0 = top_edge;
        while (y0 > bottom_edge) {
            x0 = left_edge;
            while (x0 < right_edge) {
                y = 0;
                x = 0;
                the_char = ' ';
                x_x = 0;
                y_y = 0;
                i = 0;
                while (i < max_iter && x_x + y_y <= 800) {
                    x_x = (x * x) / 200;
                    y_y = (y * y) / 200;
                    if (x_x + y_y > 800 ) {
                        the_char = '0' + i;
                        if (i > 9) {
                            the_char = '@';
                        }
                    } else {
                        temp = x_x - y_y + x0;
                        if ((x < 0 && y > 0) || (x > 0 && y < 0)) {
                            y = (-1 * ((-1 * (x * y)) / 100)) + y0;
                        } else {
                            y = x * y / 100 + y0;
                        }
                        x = temp;
                    }

                    i = i + 1;
                }
                accum = accum + the_char;

                x0 = x0 + x_step;
            }
            y0 = y0 - y_step;
        }
        if (count % 300 == 0) {
            print(accum);
        }

        count = count + 1;
    }
    printf(accum);


t2=GetTickCount()
time_lapsed = (t2-t1) / 1000 + cr

print cr + cr + "Time: " +time_lapsed + " Seconds" + cr + cr
print "Enter ..."
GetKey()

     return 0;
end sub

main()

But I get:

exo2 -c mandel.o2bas

ERROR:  `end while` or `wend`  expected

WORD:   }
LINE:   65
FILE:   main source
PASS:   1



I just cut and pasted the C version of this, so not sure what I did wrong.

Thanks for any help!

[Charles Pegge]

Hi Ed,

Curly braces are very hard to follow in multiple nestings, but the problem turned out to be single quotes '0' instead of "0"

Also, printf needs to be print

Code: [Select]


$ filename "prime_test2.exe"

includepath "$/inc/"

'#include "RTL32.inc"
'#include "RTL64.inc"

include "console.inc"


! GetTickCount lib "kernel32.dll" alias "GetTickCount" () as dword

sub main() {
    int left_edge, right_edge, top_edge, bottom_edge, max_iter,
    x_step, y_step, y0, x0, x, y, i, x_x, y_y, temp, the_char,
    accum, count;

int t1,t2, time_lapsed
print "Starting Mandel accum:" + cr + cr
t1=GetTickCount()
    accum = 0;
    count = 0;
    while (count < 1545)
    {
        left_edge   = -420;
        right_edge  =  300;
        top_edge    =  300;
        bottom_edge = -300;
        x_step      =  7;
        y_step      =  15;

        max_iter    =  200;

        y0 = top_edge;
        while (y0 > bottom_edge)
        {
            x0 = left_edge;
            while (x0 < right_edge)
            {
                y = 0;
                x = 0;
                the_char = ""';
                x_x = 0;
                y_y = 0;
                i = 0;
                while (i < max_iter && x_x + y_y <= 800)
                {
                    x_x = (x * x) / 200;
                    y_y = (y * y) / 200;
                    if (x_x + y_y > 800 )
                    {
                        the_char = "0" + i;
                        if (i > 9)
                        {
                            the_char = "@";
                        }
                    }
                    else
                    {
                        temp = x_x - y_y + x0;
                        if ((x < 0 && y > 0) || (x > 0 && y < 0)) {
                            y = (-1 * ((-1 * (x * y)) / 100)) + y0;
                        } else {
                            y = x * y / 100 + y0;
                        }
                        x = temp;
                    }
                    i = i + 1;
                } 'wend
                accum = accum + the_char;
                x0 = x0 + x_step;
            } 'wend
            y0 = y0 - y_step;
        } 'wend
        if (count % 300 == 0) {
            print(accum);
        }

        count = count + 1;
    }'wend
    print accum;


t2=GetTickCount()
time_lapsed = (t2-t1) / 1000 + cr

print cr + cr + "Time: " +time_lapsed + " Seconds" + cr + cr
print "Enter ..."
GetKey()

     return 0;
end sub

main()


[Ed Davis]

Hi Ed,

Curly braces are very hard to follow in multiple nestings, but the problem turned out to be single quotes '0' instead of "0"

Also, printf needs to be print

Thanks for the catch!

re: Curly braces nesting. That is why C programmers depend on the editor matching them when you type one or move the cursor to one.  I'd have a hard time keeping up with them otherwise

Turns out the double quotes don't translate the same way they do in C - so I just used the number, e.g., 32 for space, 48 for '0', and 64 for '@'.

Now I get the result I'm after - 309886830 - and it takes 3 seconds.  The original C version takes one second, so the current result is very good, especially considering how fast Oxygen compiles.

Another question though - I'm not getting the intermediate results - this statement doesn't appear to be correct:

Code: [Select]

        if (count % 300 == 0) {
            print(accum);
        }


[Charles Pegge]

% is used to define equates - so modulus is done like  this:

mod(count,300)

[Mike Lobanovsky]

Hello Ed,

If you want to have a vanilla ANSI C JIT compiler (and also hand-coded JIT Intel-style assembly) within the framework of a BASIC-like interpreter, please have a look at FBSL.

The zip includes raw C and FBSL scripts as well as gcc (both -O3 and unoptimized) and FBSL binaries. Running all the three on the end user's computer may give them a more coherent view of benchmark results. A table put together on a computer of unknown configuration does looks to me a bit like "data in spherical vacuum".

My current config is in my forum signature. The screenshot results were taken under Win XP Sp3 Professional.

.

[Arnold]

Hello,

I tried to transfer the Mandel test to plain basic. If I did it correctly, the code looks like this:

Code: [Select]


$ filename "Mandel2.exe"

includepath "$/inc/"

'#include "RTL32.inc"
'#include "RTL64.inc"

include "console.inc"


! GetTickCount lib "kernel32.dll" alias "GetTickCount" () as dword

sub main()
    int left_edge, right_edge, top_edge, bottom_edge, max_iter,
    x_step, y_step, y0, x0, x, y, i, x_x, y_y, temp, the_char,
    accum, count

    int t1,t2, time_lapsed
    print "Starting Mandel accum:" + cr + cr
    t1=GetTickCount()

    accum = 0
    count = 0
    while count < 1545
   
        left_edge   = -420
        right_edge  =  300
        top_edge    =  300
        bottom_edge = -300
        x_step      =  7
        y_step      =  15

        max_iter    =  200

        y0 = top_edge
        while y0 > bottom_edge       
            x0 = left_edge
            while x0 < right_edge           
                y = 0
                x = 0
                the_char = asc(" ")
                x_x = 0
                y_y = 0
                i = 0
                while (i < max_iter) and (x_x + y_y <= 800)                               
                    x_x = (x * x) / 200
                    y_y = (y * y) / 200
                   
                    if (x_x + y_y) > 800  then
                        the_char = asc("0") + i                       
                        if (i > 9) then                       
                            the_char = asc("@")                       
                        end if
                    else               
                        temp = x_x - y_y + x0
                        if ((x < 0 and y > 0) or (x > 0 and y < 0)) then
                            y = (-1 * ((-1 * (x * y)) / 100)) + y0
                        else
                            y = x * y / 100 + y0
                        end if
                        x = temp
                    end if
                    i = i + 1
                wend
               
                accum = accum + the_char

                x0 = x0 + x_step
            wend
           
            y0 = y0 - y_step
        wend
       
        if mod(count, 300) = 0 then
            print accum + cr
        end if

        count = count + 1
    wend
   
    print accum

    t2=GetTickCount()
    time_lapsed = (t2-t1) / 1000 + cr

    print cr + cr + "Time: " +time_lapsed + " Seconds" + cr + cr
    print "Enter ..."
    GetKey()

end sub

main()


The execution time with my notebook was 5 seconds for gcc, 19 seconds for Mandel_FBSL.exe and 23 seconds for Oxygenbasic. I did not try to optimize anything.

The result shows that my notebook is a lame box, as the test could be run 5 times faster. When I buy a new pc, I will have an usb-stick with this test program with me.

Roland

[Aurel]

Well i don't know how but i get this:
C unopt - 19 sec
FBSL - 24 sec
o2 - 26 sec
 

[Mike Lobanovsky]

Aurel,

I think the only path to knowledge lies in your PC config. Perhaps it's time to upgrade.

Here comes the O2 result on my workstation. The script and binary are attached in the zip.

.

[Charles Pegge]

Oxygen's condition logic is not optimised for speed. It uses the same catch-all strategy for most conditions, and has resisted my attempts to improve it so far. I am looking for ways that do not make compiling excessively complicated.

[JRS]

You would think with all the senseless loop benchmarking going on someone at CERN is holding a contest for the fastest BASIC in town. Who cares if one BASIC runs a second faster than the other? How about posting something of value that can be used for something?