I am having some trouble running mc_replicate (from the mcreplicate library) on Windows (10). I am using R v. 4.1.1 on a machine with 8 cores. When I try to execute this code:

x1 <- mc_replicate(10000, runRandomLot(), mc.cores = 8,
                    envir = environment(), varlist = "runRandomLot",
                    packages = c("dplyr","magrittr"))

I get this error:

Running parallel code on Windows: a parallel socket cluster will be used.

Variables and packages needed for code execution must be explicitely specified.

See the help file for more information and current defaults.


Error in checkForRemoteErrors(val) (ImplantGapRA.R#76): 8 nodes produced errors; first error: could not find function "%>%"

The error suggests that the magrittr package is not being passed, but the docs do not provide clear guidance on how to pass the packages (if that's what the problem is). Any guidance on how to correct my call to mc_replicate would be appreciated.

CodePudding user response：

First, yes, I'd expect specifying packages = "magrittr" would solve this, but I have zero experience with the mcreplicate. Second, and more importantly, I peeked at the code for mcreplicate::mc_replicate() and what's worrying is that it does not set up proper parallel random number generation (RNG), which means the results you get from the parallel workers might be correlated although you'd expect a replicate() call to give you proper random numbers.

(Disclaimer: I'm the author). If you have working code using base's `replicate(), I'd recommend that you replace it as-is with future_replicate() of the future.apply package for parallelization. Something like:

library(future.apply)
plan(multisession, workers = 8)  ## Parallelize on 8 local workers

x1 <- future_replicate(10000, runRandomLot())

That's it.