Simple way to restart application-CodePudding

Some library has some memory leak, and as a simple solution, I just want to restart my script frequently. My script does some long calculation but I can just save the state and then load it again, so restarting it is not a problem.

For restarting, I was just using os.execv:

os.execv(sys.executable, [sys.executable]   sys.argv)

However, now after a while, I get Too many open files.

Why is that? As far as I know, all file descriptors (fds) should have been closed after exec? I thought we always have the close-on-exec flag set? But maybe not? Or maybe not for all the libraries? Maybe I misunderstood the documentation. Can someone clarify this?

How would I close all fds before the exec? os.closerange? Maybe just:

max_fd = os.sysconf("SC_OPEN_MAX")
os.closerange(3, max_fd)
os.execv(sys.executable, [sys.executable]   sys.argv)

Does that work correctly then?

What are other simple solutions of restarting? I assume fork exec exit will just have the same problem then? Or spawn exit?

CodePudding user response：

os.execv does not by itself close any file descriptors. (Think about it: If the exec system call closed file descriptors, there would be no way for the standard fork exec model to pass on standard input, standard output, and standard error to new processes.) The documentation implicitly mentions this, as it explains that you need to flush any data on any open file handles before calling os.execv.

Like you describe, PEP-446 implemented close-on-exec for most file descriptors created by Python itself. Without access to your code, we can only infer that some of the file descriptors your code creates are outside the scope of the PEP (or created by os.dup2(), where you can control the behavior, but the default is to make the file descriptors inheritable).

Your os.closerange code should work for what you are asking. If you wanted to be more precise, check what files are open in Python has ideas for how to find out which files are open (which could be useful for troubleshooting the root cause, even if you end up using the closerange solution to remediate the immediate problem).