Does fs.writeFile perform sequential or random IO?-CodePudding

I am learning about sequential vs random file IO, and it caught my attention that the Node fs documentation does not mention whether fs.writeFile functions do sequential or random IO. Same for functions related to reading files.

Maybe its implicit for the file API for the platform to be sequential. Maybe the abstraction decides between sequential or random based on factors like file size, etc. I dont know. Either way, I am newbie in this matter and would love to get more clarity around this topic.

When a developer new to NodeJS takes the fs code from the getting started page, what is it really doing? Sequential or random IO? Can we opt for one or the other while using the fs API?

const fs = require('fs');
const content = 'Some content!';
fs.writeFile('/Users/joe/test.txt', content, err => {
  if (err) {
    console.error(err);
  }
  // file written successfully
});

CodePudding user response：

I'm assuming you are asking because you want to know whether you can expect sequential write performance or random write performance (random writes are a lot slower). So for that, it matters what is happening at the disk controller level and whether other read/writes to other parts of the disk are being interleaved with your read/writes.

Mostly sequential, but it depends...

If you look internally at the code, fs.writeFile() calls writeFileHandle() which writes the data in max chunk sizes defined by this constant:

const kWriteFileMaxChunkSize = 512 * 1024;

So, for a file that is 512k or less, the file will be written with one call to the OS. Otherwise, it will be broken up into successive 512k writes.

Now, as to whether this is closer to sequential or random IO, that's a bit more complicated. If nothing else is happening on the disk by any other process in the OS and the disk is not fragmented, then these successive 512k writes can very much be sequential writes.

But, if the OS is doing other things on the disk in other processes, then the disk heads may be required to move to handle this other process between the multiple writes of writeFileHandle() and this would probably behave more like random IO because there would be head seeks to a new part of the disk between each write. But, since these are still 512k writes (not super small), there's still an aspect of sequential IO to each 512k write. So, it would probably end up being a combination of mostly sequential, but occasionally random because of other disk activity getting interleaved.

If the disk is heavily fragmented, then all bets are off because even a 512k write may have to put different blocks of that write in non-sequential parts of the disk.