for (int t = 0; t < physicalCoreCount; t  ) {
        int pid = fork();
        if (pid==0) {
            // setting up epoll    
            epoll_fd = epoll_create1(0);                
            event.data.fd = listenSocketfd;
            event.events = EPOLLIN | EPOLLET;
            ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listenSocketfd, &event);                
            events = (epoll_event*)calloc(LISTENQ, sizeof(event));
           
            //*****/
            while (1) {
                int ndfs = epoll_wait(epoll_fd, events, curfdCount, -1);
                if (ndfs==-1) {                        
                    continue;
                }
                for (int i=0; i < ndfs; i  ) {
                    if (events[i].data.fd == listenSocketfd) { // original listener
                        int new_connfd = accept(events[i].data.fd, (sockaddr*)&clientaddr, &clientlen);
                        if (new_connfd==-1) {
                            if (errno==EAGAIN || errno==EWOULDBLOCK) {
                                continue;
                            }
                            else exitPerrorLog("accept()");
                        }

                        set_non_block(new_connfd);
                        event.events = EPOLLIN | EPOLLET;
                        event.data.fd = new_connfd;
                        if (epoll_ctl(epoll_fd, EPOLL_CTL_ADD, new_connfd, &event) < 0)
                            exitPerrorLog("epoll_ctl");

                        clientaddrOfFd[new_connfd] = new sockaddr_in();
                        memcpy(clientaddrOfFd[new_connfd], &clientaddr, sizeof(clientaddr));
                        curfdCount  ;
                    }   
                    else {
                        process(events[i].data.fd, clientaddrOfFd[events[i].data.fd]);
                        //epoll_ctl(epoll_fd, EPOLL_CTL_DEL, events[i].data.fd, &event);                           
                        if (curfdCount > 10000) curfdCount = 10000; //curfdCount--;
                    }
                }
            }
        }
    }

The problem appears because I'm trying to implement persistent connection (not closing after response). However, child 0 can accept() a socket file descriptor X, process it, but later child 1 can accept() the same file descriptor. Since I didn't close the connection on child 0 (to implement HTTP1.1 keep-alive), now 2 children are reading/writing to the same file descriptor.

What's a way to prevent this problem? Thank you.

Edit: Important update: so the main problem is that I thought 2 same FD means it's the same connection, I want to avoid that since it would lead to 2 children reading/writing the same thing. If that situation doesn't happen (read/write overlap) then I think the question is solved. Can someone confirm?

CodePudding user response：

You are creating multiple epoll instances and registering for edge-triggered events on the listening socket in each one. Naturally, you will get an event from each one when a new connection becomes available to accept. However, only one process can successfully accept each connection. As observed in comments, two different children might accept connections that are assigned the same file descriptor number in the respective processes, but that doesn't mean they refer to the same socket.

You have several options, but prominent among them are:

• use a single epoll instance, shared by all the processes. You can get this automatically by having the parent create it before forking off any of the children. In this case, only one child will receive each edge-triggered event. Of course, if the children intend to register for events that should be received only by them, then this isn't going to work very well.

• just accept (no pun intended) that multiple processes will receive events when a connection becomes available, and deal with it. That appears to be what you're doing now (by ignoring EAGAIN and EWOULDBLOCK errors from accept()), and I see no particular reason why you shouldn't keep doing it.