I'm trying to trigger some concurrent conflicts by having several processes writing to the same file, but couldn't:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <fcntl.h>
#include <sys/wait.h>

void concurrent_write()
{
    int create_fd = open("bar.txt", O_CREAT | O_TRUNC, 0644);
    close(create_fd);
    int repeat = 20;
    int num = 4;

    for (int process = 0; process < num; process  )
    {
        int rc = fork();
        if (rc == 0)
        {
            // child
            int write_fd = open("bar.txt", O_WRONLY | O_APPEND, 0644);
            for (int idx = 0; idx < repeat; idx  )
            {
                sleep(1);
                write(write_fd, "child writing\n", strlen("child writing\n"));
            }
            close(write_fd);

            exit(0);
        }
    }

    for (int process = 0; process < num; process  )
    {
        wait(NULL);
        // wait for all children to exits
    }

    printf("write to `bar.txt`\n%d lines written by %d process\n", repeat * num, num);

    printf("wc:");
    if (fork() == 0)
    {
        // child
        char *args[3];
        args[0] = strdup("wc");
        args[1] = strdup("bar.txt");
        args[2] = NULL;
        execvp(args[0], args);
    }
}

int main(int argc, char *argv[])
{
    concurrent_write();

    return 0;
}

This program fork #num children and then have all of them write #repeat lines to a file. But every time (however I change #repeat and #num) I got the same result that the length of bar.txt (output file) matched the number of total written lines. Why is there no concurrent conflicts triggered?

CodePudding user response：

Writing to a file can be divided into a two-step process. First, locate where you want to write. Second, write data into the file. You open a file with flag O_APPEND and it ensures that the two-step process is atomic. So, you can always find the lines of the file as the count you set.

CodePudding user response：

See the open(2) man page:

O_APPEND

The file is opened in append mode. Before each write(2), the file offset is positioned at the end of the file, as if with lseek(2). The modification of the file offset and the write operation are performed as a single atomic step.

In essence, one of the major design features of O_APPEND is precisely to prevent the sort of "concurrent conflicts" you mention. The typical example would be a log file that several processes must write to. Using O_APPEND ensures their messages do not overwrite each other.

Moreover, all data written by a single write call is written atomically, so provided that your write("child writing\n") successfully writes all its bytes (which for a regular file it usually would), they will not be interleaved with the bytes of any other such message.

CodePudding user response：

First, write() calls with the O_APPEND flag should be atomic. Per POSIX write():

If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write and no intervening file modification operation shall occur between changing the file offset and the write operation.

But that's not enough when there are multiple threads or processes making parallel write() calls on the same file - that does not guarantee that parallel write() calls are atomic.

POSIX does guarantee that parallel write() calls are also atomic:

All of the following functions shall be atomic with respect to each other in the effects specified in POSIX.1-2017 when they operate on regular files or symbolic links:

...

write()

...

See also Is file append atomic in UNIX?

Beware, though. Reading that question and its answers shows that Linux filesystems such as ext3 are not POSIX compliant once you get past a relatively small size operation, or possibly if you cross page and/or file system sector boundaries. I suspect XFS and ZFS will support write() atomicity much better given their origins.

And none of this applies to Windows.