Home > Blockchain >  How can I pass a impl Fn Trait Object into multiple threads, like with a threadpool
How can I pass a impl Fn Trait Object into multiple threads, like with a threadpool

Time:10-31

(Rust Compiler Version 1.64.0) What I am trying to do is: I want to have some kind of world generation process for some minecraftesque world. Therefor, I want to be able to pass a function to some World generator, and execute this function on every block. Up until now this process is singlethreaded, and I want to multithread at least the block generation process, since this requires no synchronization, it is a very multiprocessor-friendly thing to do.

Now, I currently have (something like) a function generate_world(self: &mut Self, generator : impl Fn(isize, isize, isize) ->BlockType) ->(){}

this function has a large for-loop in the middle that calls the function generator everytime it is necessary, single-threadedly.

What I have now decided to do is to use the threadpool crate. This allows me to create a pool: ThreadPool, and call pool.execute(move ||{}); to execute a job (like, e.g.first some preparation, then generating a block, or rather a whole streak at once).

What I have also run into is, that generator is moved into the thread function at the first call, but this has been (apparently) fixed by boxing and Arc-ing it.

Now, I have of course tried to put the generator function in there and was rightfully warned that I should specify Send and Sync on the generator function, which I did without ado. The problem is now I should also specify a lifetime, and what is suggested is: 'static. Unfortunately, my program's structure does not allow me to do this, since some components in the function do have a lifetime limited by a part of the main function.

What can I do to make this work? I know the function will not go out of scope until the whole generation process is finished.

I am pretty new to rust, so I may be (and probably am) oblivious to many concept that are hidden deep in the documentation. I am pretty sure, that there is a very clean and very rust-like way of solving this problem, but my mind is stuck in C thinking. :)

CodePudding user response:

Unfortunately, my program's structure does not allow me to do this, since some components in the function do have a lifetime limited by a part of the main function.

This is the part that you need to change. Change those parts of your program so that nothing is borrowed from main().

  • Use consts or static variables for constant data.

  • Use Arc instead of & to refer to shared data computed at startup.

  • Or, if absolutely necessary (such as when the data is not constant and must be referred to inside a Copy requirement) and you are only going to do it once at startup, use Box::leak(Box::new(data)) to obtain an &'static reference to some data that will never be deallocated afterward. You must not use this for anything but data that's computed only once, because then your program will leak memory in the real sense, continuously growing.

It is possible to pass borrowed data to threads, using scoped threads, but that approach requires special control of the threads and will not work for things like GUI event loops or thread pools that aren't specifically scoped themselves.


Further explanation:

Why do I need to pass something with a static lifetime? The threads don't live that long, everything is joined directly underneath.

Simple: the borrow checker does not understand thread joining. It only looks at very particular compile-time information, and "when this join() function is called, the thread must have exited afterward" is not the kind of reasoning it can perform. Broadly speaking, the borrow checker only cares about things that don't happen ("this reference is not used outside this lifetime") and can't use things that do happen, like "join() was called", as evidence that the program is sound.

The “scoped threads” I mentioned above are a way to make stopping threads understandable to the borrow checker — by making that look like exiting a function scope, which the borrow checker does understand as a reason a borrow has definitely ended.

  • Related