Oxygen Basic

Programming => Problems & Solutions => Topic started by: Aurel on March 02, 2015, 11:08:18 PM

Title: Sequential file reading
Post by: Aurel on March 02, 2015, 11:08:18 PM

Is there any example of that ?
I mean read each line of file and put this text into one array element..
thanks

Title: Re: Sequential file reading
Post by: JRS on March 02, 2015, 11:54:17 PM

In Script BASIC I use SPLITA and use the line ending character as the delimiter.

Code: Script BASIC

1. OPEN "mytextfile" FOR INPUT AS #1
2. text = INPUT(LOF(1), 1)
3. SPLITA text BY "\n" TO txtarray
4.  

Title: Re: Sequential file reading
Post by: Aurel on March 03, 2015, 02:15:30 PM

There is no such a thing in oxygen - splitA

I see that we must detect chr(13) or CRLF and put each line into string array element?

Title: Re: Sequential file reading
Post by: JRS on March 03, 2015, 03:59:52 PM

Quote from: Aurel

Is there any example of that ?

I assumed you were looking for a generic example for direction. Please specify the BASIC next time as there are more than O2 topics discussed here on the forum. Thanks!

Title: Re: Sequential file reading
Post by: Aurel on March 04, 2015, 11:14:47 PM

WOW
look screenshot... :o
I just try o2 example LineSpliter.o2bas ....
I really don't know that print txt, txt as buffer from GetFile() return splited txt file?
I must know how this work because i need this one to avoid function Space().

Charles
where i must look in src of oxygen?

.

Title: Re: Sequential file reading
Post by: Charles Pegge on March 05, 2015, 04:42:42 AM

Hi Aurel,

I forgot all about the line splitter. It was originally written for your benefit.

I have revised it to support the reading of Linux LF terminated lines and Mac CR terminated lines, as well as our native CRLF.

Also cleaner indexing and pointering:

Code: OxygenBasic

1.  
2. '=============
3. 'LINE SPLITTER
4. '=============
5.  
6. indexbase 1
7. string lines[100000]   'static lines array
8. sys i.j              'indexes
9.  
10.  
11. 'data:
12. '=====
13.  
14. 'txt=getfile "s.txt"
15.  
16. 'txt=nuls 100000
17. 'sendMessage richedit1,WM_GETTEXT,100000, strptr txt
18. '
19. txt="11
20. 22
21. 33
22. 44
23. 55
24. 66
25. 77"
26.  
27. 'initial
28. '=======
29. '
30. '
31. b=1  'start of line
32. i=0  'lines array index
33. j=1  'char index
34. '
35. byte a at (strptr txt)
36. '
37. 'splitter loop
38. '=============
39. '
40. do
41.   select a
42.   case 0
43.     if j-b>0 then
44.       i++
45.       lines[i]=mid(txt,b,j-b)
46.     end if
47.     exit do
48.   case 10 'LF TERMINATED LINES
49.    i++
50.     lines[i]=mid(txt,b,j-b)
51.     b=j+1
52.   case 13 'CR AND CRLF TERMINATED LINES
53.    i++
54.     lines[i]=mid(txt,b,j-b)
55.     if a[2]=10 then @a++ : j++ 'SKIP LF
56.    b=j+1
57.   end select
58.   @a++ 'inc pointer for a
59.  j++    
60. end do
61.  
62.  
63. cr=chr(13)+chr(10)
64. print "Lines:  " i cr +
65.       "Line 5: " lines[5] + cr +
66.       "Last:   " lines[i]
67.  

Title: Re: Sequential file reading
Post by: Aurel on March 05, 2015, 12:59:16 PM

Thank you Charles on that
BUT
do you can explain me how command print txt
can split loaded asci file .rub into lines?

here is code:

Code: [Select]

'PRINT SPLITTER (messageBox o2 function)

indexbase 1
string src,source
string txt=nuls 100000 'buffer
string lines[100000]   'lines array
sys pt=strptr txt      'buffer base pointer
sys i.j,p              'indexes and pointer


'test data:
'==========
cr=chr(13)+chr(10)
'lc="abc"+cr
'mid (txt,1)=lc+lc+lc+lc+lc
txt = GetFile "C:\o2Basic\OxygenBasic\array1.rub"
print txt 'look this ????
'IT IS WOW !!! print already split file into lines?
'how is that possible ???


.

Title: Re: Sequential file reading
Post by: Charles Pegge on March 05, 2015, 01:36:52 PM

Hi Aurel,

Print (MessageBox) recognises CRLFs in the text string. The same is also true with console output.

Title: Re: Sequential file reading
Post by: Aurel on March 05, 2015, 02:03:41 PM

Hi Charles
Ok,so do i can see source code of print
or in another words ..is that code or method the same as given function?
I need just exactly what in this case print do
split file into lines ,then each line put into string array element ...
do i must look into o2lexi or somewhere else ?

Title: Re: Sequential file reading
Post by: Charles Pegge on March 05, 2015, 03:24:20 PM

Oxygen does not use line arrays. Instead, it encodes the physical line number into the code source text, which is one large string. This ensures that the physical line number passes through several compiler stages, which have different sets of lines.

Title: Re: Sequential file reading
Post by: Aurel on March 05, 2015, 10:45:11 PM

Quote

Oxygen does not use line arrays. Instead, it encodes the physical line number into the code source text, which is one large string. This ensures that the physical line number passes through several compiler stages, which have different sets of lines.

Ok Charles
But then ..what to use to emulate print spliting?
I am confused this time ...

Title: Re: Sequential file reading
Post by: Charles Pegge on March 06, 2015, 02:06:30 AM

Hi Aurel,

Here is a more familiar, though slightly less efficient line splitter using asc(txt,j), instead of a byte pointer.

Code: OxygenBasic

1.  
2. '=============
3. 'LINE SPLITTER
4. '=============
5.  
6. indexbase 1
7. string lines[100000]   'static lines array
8. sys i.j              'indexes
9.  
10.  
11. 'data:
12. '=====
13.  
14. 'txt=getfile "s.txt"
15.  
16. 'txt=nuls 100000
17. 'sendMessage richedit1,WM_GETTEXT,100000, strptr txt
18. '
19. txt="11
20. 22
21. 33
22. 44
23. 55
24. 66
25. 77
26. "
27.  
28. 'initial
29. '=======
30. '
31. '
32. b=1  'start of line
33. i=0  'lines array index
34. j=1  'char index
35. '
36. '
37. 'splitter loop
38. '=============
39. '
40. do
41.   select asc(txt,j)
42.   case 0
43.     if j-b>0 then
44.       i++
45.       lines[i]=mid(txt,b,j-b)
46.     end if
47.     exit do
48.   case 10 'LF TERMINATED LINES
49.    i++
50.     lines[i]=mid(txt,b,j-b)
51.     b=j+1
52.   case 13 'CR AND CRLF TERMINATED LINES
53.    i++
54.     lines[i]=mid(txt,b,j-b)
55.     if asc(txt,j+1)=10 then j++ 'SKIP LF
56.    b=j+1
57.   end select
58.   j++    
59. end do
60.  
61.  
62. cr=chr(13)+chr(10)
63. print "Lines:  " i cr +
64.       "Line 5: " lines[5] + cr +
65.       "Last:   " lines[i]
66.  

Title: Re: Sequential file reading
Post by: Mike Lobanovsky on March 06, 2015, 02:51:44 AM

Aurel,

Every text file is just one long contiguous string of characters that ends in a unique short sequence of characters (or a single character) that form an operating system-dependent end-of-file (EOF) marker. Under Windows, EOF equals Ctrl+Z (ASCII code decimal 26).

Every file loading function would load this contiguous string of characters into one contiguous memory string of characters with only one substitution: it would change EOF into a zero to mark the end of an ASCIIZ string (Z here stands for "zero-terminated"). Unicode string loading requires more metamorphoses, and I won't discuss it here for simplicity.

In both the file character string and the corresponding memory ASCIIZ string, separate lines of text as we see them in a book are marked with end-of-line (EOL) markers that would be different for different operating systems a particular text file was created in. The EOL marker would be LF ("line feed", ASCII code decimal 10) in Linux, CR ("carriage return" a.k.a. "caret return", ASCII code decimal 13) in Mac OSX, and CRLF (two characters, "carriage return"+"line feed") in Windows. There can be other markers too if the text was created in some rich text editor like, say, MS Word. For example, there may be the so-called "soft line breaks" used for finer line formating; those would usually be denoted with three sequential characters, CRCRLF.

The existing implementations of Windows system services and controls like console windows, message boxes, multiline edit and rich edit controls do not need to split the ASCIIZ text string into any further separate lines of text. They just print the text character by character until they see a CR character that makes them return the caret to the beginning of the line, and a LF character that makes them "feed the line", i.e. scroll the text one line up, and put the caret one line down on a blank line. The caret (a.k.a. "carriage") is the term to denote a point at which the next "printable" character (that is, a character meaningful to the human reader) is about to be printed. In Windows, it is usually denoted with a blinking underline as in the console window, or a "beam-like" (i.e. capital I-shaped) cursor as in edit and rich edit box windows.

If for any reason you would like to split this contiguous ASCIIZ text string into separate text lines, then you will need a function that would scan the text string characters one by one in a loop. Then if the current character is "printable", the function would append it to the current line string in your line array, and if the current character is one of CR and LF characters, the function would stop appending to the current line string, proceed to the next line string in the array, and start appending the incoming "printable" characters to this next line string. The loop runs until the terminating zero character is met in the original ASCIIZ text string.

This is exactly what Charles' LINE SPLITTER function does. There is no other way to implement this functionality. John's SplitA(), my ArrayFromFile(), and all sorts of BASIC line-by-line text file reader functions are built around a similar splitter.

Title: Re: Sequential file reading
Post by: Aurel on March 06, 2015, 04:55:09 AM

Thanks Charles :)

Ok i will try last method even i understand very well method with byte pointer to.
Still ..i have feeling that method how command print split file is a sort of secret
or maybe i am wrong?

here is code which i can copy/paste from my browser:

Code: [Select]

'=============

'LINE SPLITTER

'=============
indexbase 1

string lines[100000]   'static lines array
sys i.j              'indexes
'data:
'=====

 

'txt=getfile "s.txt"
'txt=nuls 100000
'sendMessage richedit1,WM_GETTEXT,100000, strptr txt
'
txt="11
22
33
44
55
66
77
"
'initial
'=======
'
'
b=1  'start of line
i=0  'lines array index
j=1  'char index
'
'
'splitter loop
'=============
'
do
  select asc(txt,j)
  case 0
    if j-b>0 then
      i++
      lines[i]=mid(txt,b,j-b)
    end if
    exit do
  case 10 'LF TERMINATED LINES
    i++
    lines[i]=mid(txt,b,j-b)
    b=j+1
  case 13 'CR AND CRLF TERMINATED LINES
    i++
    lines[i]=mid(txt,b,j-b)
    if asc(txt,j+1)=10 then j++ 'SKIP LF
    b=j+1
  end select
  j++   
end do

 
cr=chr(13)+chr(10)
print "Lines:  " i cr +
      "Line 5: " lines[5] + cr +
      "Last:   " lines[i]

 
Mike
There is no need for such a huge explanation, i am not from yesterday..ok  ;)
but maybe you can show internal code of ArrayfromFile() ...   :D

Title: Re: Sequential file reading
Post by: Mike Lobanovsky on March 06, 2015, 05:48:51 AM

No Aurel,

You are not from yesterday. You must be from Mars if, after my huge explanation, you're still having problems with "how command print split file".


[EDIT]

And oh,

Quote

... maybe you can show internal code of ArrayfromFile() ...

Are you sure you could use one?

Code: C

1. FBSL_FUNCTION(Array_FromFile)
2. {
3.         LPFBVARIANT v = call_get_param_str(0);
4.         int len = call_get_param_int(1);
5.  
6.         FILE *infile = _fsopen( v->strVal, "r", SH_DENYNO );
7.         if (!infile) {
8.                 g_errStr = v->strVal;
9.                 return ERR_CALL_UNABLE_TO_OPEN_FILE;
10.         }
11.  
12.         char *szLine = my_alloc_az(len);
13.  
14.         int j = 0;
15.         while (fgets(szLine, len, infile)) j++;
16.  
17.         static FBARRAYBOUND ab;
18.         ab.LBound = g_optionBase;
19.         ab.UBound = ab.LBound + j - 1;
20.         *pvret = FBSL_newArray_ab(call->script, call->where, 0, &ab, 0, NULL);
21.         LPFBVARIANT pData = (*pvret)->parray->pData;
22.  
23.         FBVARIANT e;
24.         int k = 0;
25.  
26.         rewind(infile);
27.         while( fgets(szLine, len, infile) ) {
28.                 j = strlen(szLine);
29.                 if (j && szLine[j-1] == '\n')
30.                         szLine[--j] = 0;
31.                 mkStrFbVariant(&e, szLine, j);
32.                 copyFbVariant(&pData[k++], &e);
33.         }
34.         fclose(infile);
35.         my_free_ref(szLine);
36.  
37. return 0;
38. }

Title: Re: Sequential file reading
Post by: Peter on March 06, 2015, 05:50:12 AM

Quote

i am not from yesterday..ok  ;)

LOL

Title: Re: Sequential file reading
Post by: Aurel on March 06, 2015, 10:55:21 AM

Quote

Are you sure you could use one?

Hi Mike
Don't worry ...it is perfectly clear to me what is what in your example.
Thank you !
 :)

Title: Re: Sequential file reading
Post by: Aurel on March 06, 2015, 01:48:54 PM

Well..when i remove Space() and fix some other small things
latest method by Charles work fine  :)
Mike way is good to ;)
so i can compile with latest oxygen dll without problem.. :D