How to copy latest file from sftp to local directory using shell script?-CodePudding

I have multiple file in SFTP server from which I need to copy only latest file. I have written sample code but in that I am passing filename. What logic I need to add that it identify the latest file from sftp and copy it into my local?

In SFTP server -

my_data_20220428.csv
my_data_20220504.csv
my_data_20220501.csv
my_data_20220429.csv

The code which I am running-

datadir="/script/data"
cd ${datadir}
rm -f ${datadir}/my_data*.csv
rm -f ${logfile}
lftp<<END_SCRIPT
open sftp://${sftphost}
user ${sftpuser} ${sftppassword}
cd ${sftpfolder}
lcd $datadir
mget my_data_20220504.csv
bye
END_SCRIPT

what changes I need to do it automatically pick the latest file from server without hardcoding the filename?

CodePudding user response：

You can try this script mainly copied from your sample, so it is expected that the variables have already been created.

#!/usr/bin/env bash

datadir="/script/data"
rm -f "$datadir"/my_data*.csv
rm -f "$logfile"
new=$(echo "ls -halt $sftpfolder" | lftp -u "${sftpuser}","${sftppassword}" sftp://"${sftphost}" | sed -n '/my_data/s/.* \(.*\)/\1/p' | head -1)
lftp -u "${sftpuser}","${sftppassword}" sftp://"${sftphost}" << --EOF--
cd "$sftpfolder"
lcd "$datadir"
get "$new"
bye
--EOF--

CodePudding user response：

You could try:

latest=$(lftp "sftp://$sftpuser:$sftppassword@$sftphost" \
         -e "cd $sftpfolder; glob rels -1t *.csv; bye" |
         head -1)
lftp "sftp://$sftpuser:$sftppassword@myhost" \
-e "cd $sftpfolder; mget $latest; bye"