O2 Source ?

[JRS]

This is the definitive reference for x86 programming:

These are good reference guides but what I need is the binary translation of the ASM statements and opt codes.

[JRS]

Here is Charles's contribution from a previous post. I prefixed the ASM instructions with a _ to prevent conflicts with SB.

Code: [Select]

_mov = CHR(0x01)
_inc = CHR(0x05)
_dec = CHR(0x09)
_push = CHR(0x13)
_pop = CHR(0x18)
_add = CHR(0x22)
_sub = CHR(0x26)
_mul = CHR(0x30)
_div = CHR(0x34)
_cmp = CHR(0x38)
_and = CHR(0x42)
_or = CHR(0x46)
_xor = CHR(0x49)
_test = CHR(0x53)
_xchg = CHR(0x58)
_movsx = CHR(0x63)
_movzx = CHR(0x69)

I would love to see a minimal ASM function and the hex display of the binary instructions used that could be called from a C based DLL/so.

[Charles Pegge]

x86 encoding is quite complex and often irregular due to its ancient history and requirements for backward compatibility.

Each Assembler instruction generates several pieces of information, which are used in conjunction with the operands to generate the correct opcodes.

I have tidied up this table. (and fixed 2 bugs relating to 'test' and 'xchg'

Code: OxygenBasic

1.  
2.   'GENERAL ARITHMETIC AND LOGIC
3.  
4.  
5.     m=instr(" mov inc dec push pop add sub mul div cmp and or xor test xchg movsx movzx ",wd)
6.             '01  05  09  13  18  22  26  30  34  38  42   46 49  53   58   63    69    75
7.    if m>0 then
8.       '
9.      select case m
10.       '------------
11.      case 01 : pn=2 : b1=&h88 : im=&hc6 : ic=0 : ima=2 : axdi=&hb0 : axim=&ha0 : goto mos 'mov
12.      case 05 : pn=1 : b1=&hfe : im=0    : c=0  : ima=0 :             goto mos  ' inc    ' sb=&h40 not 64 bit mode
13.      case 09 : pn=1 : b1=&hfe : im=0    : c=1  : ima=0 :             goto mos  ' dec    ' sb=&h48 not 64 bit mode
14.      case 13 : pn=1 : b1=&hff : im=&h68 : c=6  : ima=1 : sb=&h50 : ic=0 : goto mos ' push
15.      case 18 : pn=1 : b1=&h8f : im=0    : c=0  : ima=0 : sb=&h58   : goto mos  ' pop
16.      case 22 : pn=2 : b1=&h00 : im=&h80 : ic=0 : ima=1 :             goto mos  ' add
17.      case 26 : pn=2 : b1=&h28 : im=&h80 : ic=5 : ima=1 :             goto mos  ' sub
18.      case 38 : pn=2 : b1=&h38 : im=&h80 : ic=7 : ima=1 :             goto mos  ' cmp
19.      case 30 : pn=1 : b1=-1   : im=0    : c=4  : ima=1 : axdi=&hf6 : goto mos  ' mul
20.      case 34 : pn=1 : b1=-1   : im=0    : c=6  : ima=1 : axdi=&hf6 : goto mos  ' div
21.      case 42 : pn=2 : b1=&h20 : im=&h80 : ic=4 : ima=1 :             goto mos  ' and
22.      case 46 : pn=2 : b1=&h08 : im=&h80 : ic=1 : ima=1 :             goto mos  ' or
23.      case 49 : pn=2 : b1=&h30 : im=&h80 : ic=6 : ima=1 :             goto mos  ' xor
24.      case 53 : pn=2 : b1=&h84 : im=&hf6 : ic=0 : ima=2 :             goto mos  ' test (no di)
25.      case 58 : pn=2 : b1=&h86 : im=0    :        ima=0 : axdi=&h90 : goto mos  ' XCHG (no di) Exchange Register/Memory with mrm and AL/AX/EAX Register 90+reg
26.      case 63 : pn=3 : b1=&hbe : im=0    :        ima=0 : b0=&h0f   : goto mos  ' MOVSX Move with Sign-Extend
27.      case 69 : pn=3 : b1=&hb6 : im=0    :        ima=0 : b0=&h0f   : goto mos  ' MOVZX Move with Zero-Extend
28.      end select
29.       'ers="Internal error: Misaligned "+wd:ert=19: goto exita
30.    end if
31.  
32.  

Charles

[JRS]

Wow!

I wonder if I'm getting over my head on this project. I may just have to be happy with calling C functions (SB extension modules) and oxygen.dll (Windows only) for ASM speed enhancements.

Thanks for the info and saving me from spinning my wheels. I have a whole new respect for O2 and the difficulty in offering a Basic compiler.

[efgee]

JRS,
if you have time you could take a look at sljit.

It's a jit compiler that takes simplified asm code and is available for the following platforms:
Intel x86-32, AMD x86-64, ARM (Including ARM-v5, ARM-v7 and Thumb2 instruction sets), IBM PowerPC-32, IBM PowerPC-64 and MIPS-32.

Haven't looked into it deeper but maybe it's easier for you to work with this and would give you the benefit that it works on all these platforms.
(As SB obviously works on several OS as well)

bye

[JRS]

sljit compiled fine on my Ubuntu 11.04 64 bit system.

The fun part will be making a SB SLJIT extension module out of this.  

Code: [Select]

jrs@laptop:~/sljit/bin$ ./sljit_test
Generating code for: x86-64
Executable allocator: ok
test1 ok
test2 ok
test3 ok
test4 ok
test5 ok
test6 ok
test7 ok
test8 ok
test9 ok
test10 ok
test11 ok
test12 ok
test13 ok
test14 ok
test15 ok
test16 ok
test17 ok
test18 ok
test19 ok
test20 ok
test21 ok
test22 ok
test23 ok
test24 ok
test25 ok
test26 ok
test27 ok
test28 ok
test29 ok
test30 ok
test31 ok
test32 ok
test33 ok
test34 ok
test35 ok
test36 ok
test37 ok
test38 ok
test39 ok
test40 ok
All tests are passed.
jrs@laptop:~/sljit/bin$

FYI: This test suite executed instantly. Don't blink.  

Thanks for the link!

[Charles Pegge]

Only a small subset of processor opcodes is required for implementing a high level language. One could bypass the assembly code stage and generate the opcodes directly.

This medel provides most of the data you would need to go from operator directly to x86 opcodes. (I have not included floating point)

Code: OxygenBasic

1.  
2.   type coding
3.   string nam 'operator name
4.  sys mod    'encoding mode
5.  sys dir    'direct from mem
6.  sys cdi    'second octal code
7.  sys imm    'immediate literal
8.  sys cim    'second octal code
9.  end type
10.  
11.   coding c[32]
12.  
13.   '=================================================
14.  '        name     mode  direct coder, immed, coder
15.  '        nam      mod , dir  , cdi  , imm  , cim
16.  '=================================================
17.  c[01] <= "load" , 0x1 , 0x8b , -1   , 0xb8 , -1
18.   c[02] <= "stor" , 0x1 , 0x89 , -1   , -1   , -1
19.   c[03] <= "+"    , 0x2 , 0x03 , -1   , 0x81 , 0x0
20.   c[04] <= "-"    , 0x2 , 0x2b , -1   , 0x81 , 0x5
21.   c[05] <= "cmp"  , 0x2 , 0x3b , -1   , 0x81 , 0x7
22.   c[06] <= "*"    , 0x3 , 0xf7 , 0x05 , -1   , -1
23.   c[07] <= "/"    , 0x3 , 0xf7 , 0x07 , -1   , -1
24.   c[08] <= "and"  , 0x4 , 0x23 , -1   , 0x81 , 0x4
25.   c[09] <= "or"   , 0x4 , 0x0b , -1   , 0x81 , 0x1
26.   c[10] <= "xor"  , 0x5 , 0x31 , -1   , 0x81 , 0x6
27.   c[11] <= "++"   , 0x6 , 0xff , 0x0  , -1   , -1
28.   c[12] <= "--"   , 0x6 , 0xff , 0x1  , -1   , -1
29.   c[13] <= "push" , 0x6 , 0x50 , -1   , 0x68 , -1
30.   c[14] <= "pop"  , 0x6 , 0x58 , -1   , -1   , -1
31.   c[15] <= "call" , 0x7 , 0xff , 0x2  , 0xe8 , -1
32.   c[16] <= "jump" , 0x7 , 0xff , 0x5  , 0xe9 , -1
33.   c[17] <= "=="   , 0x8 , -1   , 0x85 , 0x0f , 0x85
34.   c[18] <= "<>"   , 0x8 , -1   , 0x84 , 0x0f , 0x84
35.   c[19] <= ">="   , 0x8 , -1   , 0x82 , 0x0f , 0x8c
36.   c[20] <= "<="   , 0x8 , -1   , 0x87 , 0x0f , 0x8f
37.   c[21] <= ">"    , 0x8 , -1   , 0x86 , 0x0f , 0x8e
38.   c[22] <= "<"    , 0x8 , -1   , 0x83 , 0x0f , 0x8d
39.   '=================================================
40.  
41.  

Charles

[JRS]

I started putting together a SB equivalent to your example.

Code: [Select]

ASM[1]{"nam"} = "load"
ASM[1]{"mod"} = CHR(0x1)
ASM[1]{"dir"} = CHR(0x8b)
ASM[1]{"cdi"} = -1
ASM[1]{"imm"} = CHR(0xb8)
ASM[1]{"cim"} = -1

ASM[2]{"nam"} = "stor"
ASM[2]{"mod"} = CHR(0x1)
ASM[2]{"dir"} = CHR(0x89)
ASM[2]{"cdi"} = -1
ASM[2]{"imm"} = -1
ASM[2]{"cim"} = -1

ASM[3]{"nam"} = "+"
ASM[3]{"mod"} = CHR(0x2)
ASM[3]{"dir"} = CHR(0x03)
ASM[3]{"cdi"} = -1
ASM[3]{"imm"} = CHR(0x81)
ASM[3]{"cim"} = CHR(0x0)

ASM[4]{"nam"} = "-"
ASM[4]{"mod"} = CHR(0x2)
ASM[4]{"dir"} = CHR(0x2b)
ASM[4]{"cdi"} = -1
ASM[4]{"imm"} = CHR(0x81)
ASM[4]{"cim"} = CHR(0x5)

ASM[5]{"nam"} = "cmp"
ASM[5]{"mod"} = CHR(0x2)
ASM[5]{"dir"} = CHR(0x3b)
ASM[5]{"cdi"} = -1
ASM[5]{"imm"} = CHR(0x81)
ASM[5]{"cim"} = CHR(0x7)

ASM[6]{"nam"} = "*"
ASM[6]{"mod"} = CHR(0x3)
ASM[6]{"dir"} = CHR(0xf7)
ASM[6]{"cdi"} = CHR(0x05)
ASM[6]{"imm"} = -1
ASM[6]{"cim"} = -1

ASM[7]{"nam"} = "/"
ASM[7]{"mod"} = CHR(0x3)
ASM[7]{"dir"} = CHR(0xf7)
ASM[7]{"cdi"} = 0x05
ASM[7]{"imm"} = -1
ASM[7]{"cim"} = -1

ASM[8]{"nam"} = "and"
ASM[8]{"mod"} = CHR(0x4)
ASM[8]{"dir"} = CHR(0x23)
ASM[8]{"cdi"} = -1
ASM[8]{"imm"} = CHR(0x81)
ASM[8]{"cim"} = CHR(0x4)


[Charles Pegge]

Hi John,

 I tried each data row into an array then using splita to separate the row data into strings. But these split units would not convert from hexadecimal to numeric values no matter what I tried. So I think setting flag values directly in a case block as I do the the o2 assembler is going to work better.

Charles

[JRS]

I can better understand the problem you may be having with SB arrays if you can show me an example in O2 Basic of using your base table  to generate a binary executable image. It doesn't to do much other than indicate it worked.

Thanks!

[Charles Pegge]

This describes the problem John:

ScriptBasic

Code: OxygenBasic

1.  
2.   ' =================================================
3.  '        name     mode  direct coder, immed, coder  
4.  '        nam      mod , dir  , cdi  , imm  , cim  
5.  ' =================================================
6.  c["load"] =      "0x1 , 0x8b , -1   , 0xb8 , -1"
7.  
8.   cc=c["load"]
9.   splita cc by "," to d
10.   dt=ltrim(rtrim(d[1]))
11.   print dt
12.   v=val(dt)
13.   print "   " & v
14.   line input a
15.  
16.  
17.  

There's too much processing involved here anyway so I favour a more direct approach.

Charles

[JRS]

If I get the big picture here, I need to convert this into a binary string including the -1s to their binary integer format, correct?

 

[Charles Pegge]

Sample Binary Encoding for + and -

SB

Code: [Select]

 
  nam="+"
  vai=-1
  van="a"
  vav=123
  

  function mklong(v)
    h=right("0000000" & hex(v),8)
    'print h
    mklong=chr(asc(mid(h,7,1))*16+asc(mid(h,8,1))) & _
           chr(asc(mid(h,5,1))*16+asc(mid(h,6,1))) & _
           chr(asc(mid(h,3,1))*16+asc(mid(h,4,1))) & _
           chr(asc(mid(h,1,1))*16+asc(mid(h,2,1)))
  end function
  
  
  if nam="+" then
    'direct or immediate
    if vai<>0 then
      cod=chr(0x03) & chr(0x85) & mklong(vai*4)
    else
     cod=chr(0x81) & chr(0*8) & mklong(vav)
    end if
  elseif nam="-" then
    'direct or immediate
    if vai<>0 then
      cod=chr(0x2b) & chr(0x85) & mklong(vai*4)
    else
     cod=chr(0x81) & chr(5*8) & mklong(vav)
    end if
  end if


  print "ok"


  line input a

Can I put comments on the same line as code?
Direct means from a memory location.
Immediate means a number

Charles

[JRS]

Can I put comments on the same line as code?

SB only allows one statement per line. A remark is a statement.

[Charles Pegge]

The -1s indicate code not available but they are not relevant when coding directly. In fact the x86 instruction set is so irregular that it is probably easier to deal with each operator as an individual case and not attempt to formulate generic rules. For instance there is no way of dividing directly by a number. The number has to be preloaded into another register first. Then the edx register has to be set to 0. These things have to be done as a macro before the idiv instruction is given.

Charles