I'm wondering if there is a way to write to a file that was opened in a separate script in Python. For example if the following was run within main.py:

f = open(fname, "w")
writer.write()

Then, within a separate script called writer.py, we have a function write() with the form:

def write()
    get_currently_open_file().write("message")

Without defining f within writer.py. This would be similar to how matplotlib has the method:

pyplot.gca()

Which returns the current axis that's open for plotting. This allows you to plot to an axis defined previously without redefining it within the script you're working in.

I'm trying to write to a file with inputs from many different scripts and it would help a lot to be able to write to a file without reading a file object or filename as an input to each script.

CodePudding user response：

Yes. Python functions have local variables, but those are only the variables that are assigned in the function. Python will look to the containing scope for the others. If you use f, but don't try to assign f, python will find the one you created in the global scope.

def write():
    f.write("text")

fname = "test"
f = open(fname, "w")
write()

This only works if the function is in the same module as the global variable (python "global" is really "module level").

UPDATE

Leveraging a function's global namespace, you could write a module that holds the writing function and a variable holding the file. Every script/module that imports this module could use the write function that gets its file handles from its own module. In this example, filewriter.py is the common place where test.py and somescript.py cooperate on file management.

filewriter.py

def opener(filename, mode="r"):
    global f
    f = open(filename, mode)

def write(text):
    return f.write(text) # uses the `f` in filewriter namespace

test.py

from filewriter import write

def my_test():
    write("THIS IS A TEST\n")

somescript.py

import filewriter
import test
filewriter.opener("test.txt", "w")
test.my_test()
# verify
filewriter.f.seek(0)
assert f.read() == "THIS IS A TEST\n"

CodePudding user response：

Obviously what you're asking for is hacky, but there are semi-standard ways to express the concept "The thing we're currently writing to". sys.stdout is one of those ways, but it's normally sent to the terminal or a specific file chosen outside the program by the user through piping syntax. That said, you can perform temporary replacement of sys.stdout so that it goes to an arbitrary location, and that might satisfy your needs. Specifically, you use contextlib.redirect_stdout in a with statement.

On entering the with, sys.stdout is saved and replaced with an arbitrary open file; while in the with all code (including code called from within the with, not just the code literally shown in the block) that writes to sys.stdout instead writes to the replacement file, and when the with statement ends, the original sys.stdout is restored. Such uses can be nested, effectively creating a stack of sys.stdouts where the top of the stack is the current target for any writes to sys.stdout.

So for your use case, you could write:

import sys

def write():
    sys.stdout.write("message")

and it would, by default, write to sys.stdout. But if you called write() like so:

from contextlib import redirect_stdout

with open(fname, "w") as f, redirect_stdout(f):  # Open a file and redirect stdout to it
    write()

the output would seamlessly go to the file located wherever fname describes.

To be clear, I don't think this is a good idea. I think the correct solution is for the functions in the various scripts to just accept a file-like object as an argument which they will write to ("Explicit is better than implicit", per the Zen of Python). But it's an option.

CodePudding user response：

Writing as a separate answer because it's essentially unrelated to my other answer, the other semi-reasonable solution here is to define a protocol in terms of the contextvars module. In the file containing write, you define:

import contextlib
import io
import sys
from contextvars import ContextVar

outputctx: ContextVar[io.TextIOBase] = ContextVar('outputctx', default=sys.stdout)

@contextlib.contextmanager
def using_output_file(file):
    token = outputctx.set(file)
    try:
        yield
    finally:
        outputctx.reset(token)

Now, your write function gets written as:

def write():
    outputctx.get().write("message")

and when you want to redirect it for a time, the code that wants to do so does:

 with open(fname, "w") as f, using_output_file(f):
     ... do stuff where calling write implicitly uses the newly opened file ...
 ... original file is restored ...

The main differences between this and using sys.stdout with contextlib.redirect_stdout are:

1. It's opt-in, functions have to cooperate to use it (mild negative)
2. It's explicit, so no one gets confused when the code says print or sys.stdout.write and nothing ends up on stdout
3. You don't mess around with sys.stdout (temporarily cutting off sys.stdout from code that doesn't want to be redirected)
4. By using contextvars, it's like thread-local state (where changing it in one thread doesn't change it for other threads, which would cause all sorts of problems if multithreaded code), but moreso; even if you're writing asyncio code (cooperative multitasking of tasks that are all run in the same thread), the context changes won't leak outside the task that makes them, so there's no risk that task A (which wants to be redirected) changes how task B (which does not wish to be redirected) behaves. By contrast, contextlib.redirect_stdout is explicitly making global changes; all threads and tasks see the change, they can interfere with each other, etc. It's madness.