Files in c

[Mark Spen]

Hello guys!I want your help if you can.I want to read numbers from a file and store it in a 2d array.My problem is that i do not know how many lines has my file each time,each line has 7 numbers.I imagine that  i should find how many lines has my file while i read the file but i don't have any idea.Can you help me please??

Thank you!

[John Matthews]

Hi Mark - sorry I just posted a silly question; will give a better reply shortly...

[John Matthews]

Are you intending to dynamically allocate the memory for your 2d array using malloc() (or calloc()) once you know the number of lines?

That's fine, but an alternative which avoids having to know the number of lines in advance is to use realloc() to dynamically add another row to your array each time you read a line. It's not something you'd do in time critical code, but as a general solution it's ok.

https://linux.die.net/man/3/realloc

[John Matthews]

But if you do want to count the number of lines first, a simple way is to just read a character at a time using fgetc() until you hit EOF, incrementing your line count if the character is the end-of-line '\n' character. Then, after allocating you array, use rewind() to go back to the beginning of the file and read the numbers.

[Mark Spen]

John Matthews wrote:But if you do want to count the number of lines first, a simple way is to just read a character at a time using fgetc() until you hit EOF, incrementing your line count if the character is the end-of-line '\n' character. Then, after allocating you array, use rewind() to go back to the beginning of the file and read the numbers.

No i don't store dynamically the numbers in array.So i 'll try this solution with count the lines and when this end i'll call the rewind,right??

[Tim Holloway]

Ah yes. In The Good Old Days, you'd pre-allocate a worst-case (largest) table size and define it all statically. And adjust and custom-compile the application, Entire operating systems were designed on that basis.

By the time C came around, it was more likely that you'd allocate an array of pointers, and every time you read a row in, you'd allocate a block of memory to hold that row, then add the pointer to that block to your array of pointers. Here, too, the initial array of pointers might be a fixed worst-case size. We can afford to be even more flexible now, however, though, if you want to avoid fixed allocations entirely.

Basically what you'd do is the same thing that Java does with transparently-expandable resources such as AtrrayLists and StringBuffers. You pre-allocate a nominal pointer array for the incoming rows. If you use it all up. you allocate a new, larger array, copy the contents of the old array into it, and discard (free) the old array. So you need a master pointer that points to the row-pointer array, a count of used slots in that array, and the number of slots.

Somewhere around this point, if you're using C++, the idea may occur to make a class out of all this, although if you're using C++, I'd recommend checking the standard libraries to make sure someone hasn't already provided one, pre-coded and pre-debugged. C originally wasn't supplied with anything near the rich set of libraries that most modern languages have, which meant a lot of re-inventing the wheel.

[John Matthews]

Tim Holloway wrote:Basically what you'd do is the same thing that Java does with transparently-expandable resources such as AtrrayLists and StringBuffers. You pre-allocate a nominal pointer array for the incoming rows. If you use it all up. you allocate a new, larger array, copy the contents of the old array into it, and discard (free) the old array. So you need a master pointer that points to the row-pointer array, a count of used slots in that array, and the number of slots.

Or you just keep it simple and use realloc() to extend the array as suggested, and not bother with arrays of pointers and copying/discarding.

[John Matthews]

Mark Spen wrote:No i don't store dynamically the numbers in array.So i 'll try this solution with count the lines and when this end i'll call the rewind,right??

Sorry, I don't follow - if you aren't dynamically allocating the array, why do you need to know the number of lines in advance?

[Tim Holloway]

John Matthews wrote:

Tim Holloway wrote:Basically what you'd do is the same thing that Java does with transparently-expandable resources such as AtrrayLists and StringBuffers. You pre-allocate a nominal pointer array for the incoming rows. If you use it all up. you allocate a new, larger array, copy the contents of the old array into it, and discard (free) the old array. So you need a master pointer that points to the row-pointer array, a count of used slots in that array, and the number of slots.

Or you just keep it simple and use realloc() to extend the array as suggested, and not bother with arrays of pointers and copying/discarding.

Actually, realloc() does itself do copying and discarding. The advantage of using row pointers is that it's less likely to fail on large arrays, since you don't have to have enough free space for both new and old arrays while the array is being expanded. Additionally, copying the entire array, as opposed to copying just a list of row pointers can be a tremendous overhead for large arrays. So you can save some effort using by realloc() WITH a row pointer array.

[John Matthews]

Tim Holloway wrote:Actually, realloc() does itself do copying and discarding.

Yes, I know how realloc() works, but you gave the impression you weren't aware of it - apologies.

[Matthew Bendford]

Although I don't really know any C yet (but I already started learning the basics) as I like to use linked lists a lot (where they fit if no hgigh-speed random access is required) I guess I may had tried to approach this by a simple single linked list combining a data struct to hold the values and a pointer to the next set. This way one can read each line to the next line terminator, like \n, or until EOF shows up. Then one keeps a pointer to the first struct and the last one is marked by a pointer to null. This way memory gets allocated only when needed and the file has not to be read twice but only once.
I'm not sure if this is a valid idea to approach the requested task, but that's how I would do it in Java.
Even better, as I'm used to design network protocols, is some kind of meta-data, like the file starting with number of rows, or if the lines are equal long just devide the filesize by line length - should get you close even with a of-by-one error.

[John Matthews]

Hi Matthew - I highly recommend looking up realloc(). Linked lists are great if you need them, but there's no point using a sledgehammer to crack a nut.

If you want to dynamically extend a table by one row, you can do it with realloc() in literally one line of code. For example, the OP was reading from a file containing lines of 7 numbers, so the table row type might be:

typedef struct
{
    int val[7];
} Row_t;

So the table could be:

typedef struct
{
    Row_t *rows; // array [0..nRows-1] of dynamically allocated rows
    int nRows;
} Table_t;

To add a row to the end of the table:

    // Define and initialize table.
    Table_t table = {.rows = NULL, .nRows = 0};
    :
    // Add a new (uninitialized) row to the end of the table.
    table.rows = realloc(table.rows, ++table.nRows * sizeof(table.rows[0]));

And that's it. No messing around with arrays of pointers or linked lists etc. Admittedly you should check that realloc() hasn't returned NULL, but you could easily hide that in a wrapper function.

As I said in my first (proper) post, you wouldn't do this in time critical code because the timing is non-deterministic. But I suspect for what the OP is after this is by far the best solution - keep it simple

[John Matthews]

BTW I timed some code using these typedefs to add 1000000 (a million) rows to the table, and it took just under 0.03 seconds, performing 210 actual (re-)allocations. This is running in a linux VirtualBox virtual machine on a Windows i5 4 core laptop.

Interestingly I then compiled (same command line) and ran on our 40 core Xeon native linux work server - the duration approximately halved (presumably it only used one core) but it did 226 allocations. Same compiler version.