scp multiple arguments of bash script-CodePudding

I am trying to use scp to copy multiple files from a remote server.

I can successfully do it for a single argument as script file_number

Where script is:

#!/bin/bash
echo Insert your username:
read user
scp $user@server:/path/run$1*.lz4 ./
lz4 -mdv --rm run$1*.lz4

All my files have the format: runXXXXsubY.lz4

This way I can copy all the files that have XXXX==file_number, but I want to pass multiple arguments, and copy all those files e.g. script 1 2 to copy all the files with xxxx==1 and XXXX==2.

I tried with:

scp $user@server:/path/run"$@"*.lz4 ./

But $@ gets expanded without appending the *.lz4

I appreciate any help in the right direction.

Thanks

CodePudding user response：

The basic problem is that when the shell expands $user@server:/path/run"$@"*.lz4, it doesn't copy the $user:... and *.lz4 parts for each argument, it just kind of blindly adds the list of arguments -- including word breaks between arguments -- into the middle. So if the args are 1 and 2, it essentially expands to:

scp $user@server:/path/run"1" "2"*.lz4 ./

...so $user@server:/path/run"1" and "2"*.lz4 are separate arguments to scp, which isn't useful. What you can do is create an array based on the arguments, and then use that as the source list for scp. Something like this:

sources=()
for runNum in "$@"; do
    sources =("$user@server:/path/run${runNum}*.lz4")
done
scp "${sources[@]}" ./

And then use a separate loop for the lz4 command:

for runNum in "$@"; do
    lz4 -mdv --rm run${runNum}*.lz4
done

EDIT: to avoid having to authenticate multiple times, you can open a master SSH connection and let all the scp transfers piggyback on that. Here's an example, based heavily on Felix Rabe's answer here:

# Create the array of source file patterns to fetch
sources=()
for runNum in "$@"; do
    sources =("$user@server:/path/run${runNum}*.lz4")
done

# Open master SSH connection:
# (This is the only time you have to enter the password)
sshSocket=~/"$user@server"
ssh -M -f -N -S "$sshSocket" "$user@server"

# Actually copy the files:
scp -o ControlPath="$sshSocket" "${sources[@]}" ./

# Close master connection:
ssh -S "$sshSocket" -O exit "$user@server"

CodePudding user response：

Your best bet IMO is a simple for loop:

for xxx; do
    scp "$user@server:/path/run${xxx}.lz4" ./ 
done

If you do want to run a single scp (avoiding multiple password prompts is not a motivation for doing this; the way to avoid multiple password prompts is to setup your ssh keys so no password is necessary), you can use xargs:

printf "$user@server:/path/run%s.lz4\0" "$@" | xargs -0 -J % scp % ./

This does not guarantee a single invocation, and xargs will run multiple invocations of scp if enough args are passed that the argument string exceeds certain limits, but this should probably be sufficient.

CodePudding user response：

One parameter substitution idea to build a pattern for the remote file names:

$ fn="$@"
$ echo scp "user@server:/path/run{${fn// /,}}*.lz4" ./

Assuming $@ = 222 333 this generates:

scp user@server:/path/run{222,333}*.lz4 ./

NOTE: once satisified with the format remove the echo