how to create csv file from a sequentially created data file with c language-CodePudding

I have a question about :how to create csv file from a sequentially created data file with C language.

With a C program I make several printf of values. The output of the program is redirected to a file by : ./myprog >> file.txt

So the file is like :

0.8952
0.89647
0.3658
!!!
0.258633
0.233655
0.25475
!!!
0.5895
0.54785
0.695555
!!!

etc.

The different dimensions are separeted by "!!!"

The result I would like is :

0.8952;0.258633;0.5895
0.89647;0.233655;0.54785
0.3658;0.25475;0.695555

I tried with a two dimentions array to do so but as i have about 100 000 lines between evevy "!!!" I have a segmentation fault ex. double myTab[100000] [100000].

If you have an idea, thanks a lot. Best regards

CodePudding user response：

Don't try to buffer everything. Just remember where each segment starts, and use fseek() wisely.

Here I use a fixed array presuming up to 10 segments. You may have to either increase this, or possibly make it dynamic and "growable".

(This is just a "rough cut", but may lead you to a solution.)

size_t offsets[ 10 ] = { 0 };
int nStored = 0;
char buf[ 128 ]; // be generous; don't scrimp.

// first pass, just remember position of 1st number
while( fgets( buf, sizeof buf, infp ) )
    if( strncmp( buf, "!!!", 3 ) )
        offsets[ nStored   ] = ftell( infp );

// Now, make 100,000 passes until exhaust (equal sized) sections.
while( true ) {
    for( int i = 0; i < nStored; i   ) {
        fseek( infp, offsets[ i ], SEEK_SET );
        fgets( buf, sizeof buf, infp ) )

        if( strncmp( buf, "!!!", 3 ) ) // start of next section?
            return;

        *strpbrk( buf, "\n" ) = ','; // replace NL with ','
        fprintf( outfp, "%s", buf );
        offsets[ i ] = ftell( infp ); // update for next pass
    }
    fprintf( outfp, "\n" ); // Yeah, trailing comma and null field. Life, eh?
    
}

If the "sections" between "!!!" markers are NOT the same size, then do not "update" the offset value for short sections... When small sections are exhausted, output a "," to the output file to indicate "empty column". Will need a flag to indicate "no new data found during this sweep" and that is the clue that the job is done.

CodePudding user response：

You can do it using simple commands such as :

 cat file.txt| tr '\n' ',' | sed 's/,!!!,/\n/g' | sed 's/,!!!//' > output.csv