Run multiple instances of matlab without a parfor loop-CodePudding

I want to run several instances of matlab without running a parfor loop. The structure of my code is the following:

if k == 1
% Set some parameters here
elseif k == 2
% Set some other parameters here 
...
elseif k == 10
%Set some other parameters here
end

Is there an efficient way of opening 10 instances of matlab where each instance will run for a given value of k?

I know that in a cluster with slurm I could use slurm arrays, i.e. I could add this to the beginning of the matlab code:

k = str2num(getenv('SLURM_ARRAY_TASK_ID'));

And then just to a batch submit. Anything similar that I could do on a normal computer?

CodePudding user response：

You can pass arguments to matlab functions from command line.

e.g.

function y = myfunc(k)
switch k:
    case 1:
....

from DOS:

 matlab /r "myfunc(2)"

You can write for loops in windows command line

for /l %x in (1, 1, 100) do echo %x
You can run commands "on the background"

START /B your_command

But I don't understand what you want with this. The overhead of running 10 matlab instances will absolutely undermine any "parallelization" that you think you are achieving. You are running 10 times the engine. Just Observe how much RAM and processing power MATLAB takes when you open it, without running anything.

Note that MATLAB parfor is great at parallelization and that many of the MATLAB functions, if allowed (i.e. if free) will use all available cores to compute the results, maximizing your resources. I'd be surprised if this is anywhere near fast if running in a single PC.

So the above answers how you do it, but the real answer is that you should not do this. Its the worst and slowest way to run multiple instances of a code in MATLAB. A for loop may be faster.

CodePudding user response：

In linux, you could let a bash script write out matlab scripts which can then be executed in parallel. You can just use the ampersand (&) for that after each matlab call, but the GNU parallel software is better: you can then specify how many jobs will run in parallel.

This script

#!/bin/bash

# command line argument: how many scripts (jobs) in parallel?
if [[ ${1} == "" ]]; then
   echo "${0} needs a parameter: N == how many scripts are made 0,1,2 ..."
   exit 1
fi
N=${1}; 
echo "creating and running ${N} scripts ..."

# some constants
c_dir=$(pwd)
ml_ex=$(which matlab)

# create the scripts
for (( i=1; i <= ${N}; i   )); do
cat << EOF > ${c_dir}/script${i}.m
a = ones (${i}) * $i
EOF
done

# list them, then pass this list to parallel
for f in ${c_dir}/script*.m; do
    echo "${ml_ex} < $f" 
done | parallel -j ${N};

# tidy up
rm -f ${c_dir}/script*.m

This bash script makes N matlab scripts (N is the command line parameter) and executes them in matlab in parallel. Each script should show a MxM matrix filled with the number M (for M = 1,2, ... N ). So the command runsN.sh 5 runs 5 copies of matlab at the same time.

This maybe worth trying if you have a number of time-consuming, not very resource-demanding, completely independent jobs.