My perl script, z.pl,
uses
Time::Local qw( timelocal_posix timegm_posix );
and
File::Rsync.
(You will find z.pl at the bottom of this post.)
Currently I work on a macbook pro under macOS Catalina 10.15.7.
But I want my code to be portable,
so that in the future,
if I move my files to a different machine or operating system (or at least another *nix),
the code will not need fixing ---
other than a possible change in the source and destination paths.
To this end, my usual shebang is
#!/usr/bin/env perl.
To have access to
Time::Local qw( timelocal_posix timegm_posix );
I had to install an
up-to-date version of perl.
(For this, I used homebrew.)
This is because the perl that comes loaded on my macbook pro is antiquated.
We see the difference here:
> /usr/bin/perl --version
This is perl 5, version 18, subversion 4 (v5.18.4) built for darwin-thread-multi-2level
(with 2 registered patches, see perl -V for more detail)
Copyright 1987-2013, Larry Wall
> /usr/bin/env perl --version
This is perl 5, version 34, subversion 0 (v5.34.0) built for darwin-thread-multi-2level
Copyright 1987-2021, Larry Wall
> /usr/local/bin/perl --version # same as previous
This is perl 5, version 34, subversion 0 (v5.34.0) built for darwin-thread-multi-2level
And we see here that the older version does not have what I need:
> /usr/bin/env perl -e 'use Time::Local qw( timelocal_posix timegm_posix );'
> /usr/bin/perl -e 'use Time::Local qw( timelocal_posix timegm_posix );'
"timelocal_posix" is not exported by the Time::Local module
"timegm_posix" is not exported by the Time::Local module
Can't continue after import errors at -e line 1.
BEGIN failed--compilation aborted at -e line 1.
I want this code to work once an hour. For this, one could open a command line and type
while [ : ]; do z.pl; sleep 3600; done
and then it runs approximately every hour. But this requires the user to remember to re-launch the command every time the computer is rebooted. It's not automatic.
One would think cron is the answer.
But when I tried to invoke it with cron, the first roadblock was that cron does not know my path.
Thus, my portable shebang points to the antiquated perl.
So, to get z.pl to run under cron, I made a concession to non-portability and hard-coded
the shebang #!/usr/local/bin/perl.
But even with this modification, the rsync part fails. To demonstrate this, first on the command line I typed
~/tmplocal/DUMS> z.pl >> out.txt 2>&1
and subsequently I temporarily modified my crontab to include
00 * * * * $HOME/u/kh/bin/z.pl >> $HOME/tmplocal/DUMS/z.pl/out.txt 2>&1
The resulting out.txt file is below. The two different invocations can be distinguished not only by the timestamp in their header, but by the difference in the PATH variable. Under cron, a script does not know my path.
------------------------2021-12-27 Mon 12:52:22------------------------
HOME: /Users/kpr
USER: kpr
PATH: /Users/kpr/u/kh/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Library/Apple/usr/bin
$VAR1 = {
'err' => '',
'lastcmd' => 'rsync --itemize-changes --update --stats --compress --recursive --times --perms --links --verbose --exclude=.swp --exclude=.swo --exclude=.*.swp --exclude=.*.swo --exclude=JAW.gnucash.LCK --exclude=JAW.gnucash.LNK --exclude=JAW.gnucash*log /Users/kpr/TEMPBACKUP /Volumes/oom/quick',
'out' => 'building file list ... done
.d..t.... TEMPBACKUP/2021/2021d/
>f TEMPBACKUP/2021/2021d/2021-12-27-125000-z.pl
>f TEMPBACKUP/2021/2021d/2021-12-27-125026-z.pl
>f TEMPBACKUP/2021/2021d/2021-12-27-125055-z.pl
>f TEMPBACKUP/2021/2021d/2021-12-27-125211-z.pl
Number of files: 184014
Number of files transferred: 4
Total file size: 60206266413 bytes
Total transferred file size: 7596 bytes
Literal data: 7596 bytes
Matched data: 0 bytes
File list size: 7048863
File list generation time: 16.863 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 7052167
Total bytes received: 114
sent 7052167 bytes received 114 bytes 328013.07 bytes/sec
total size is 60206266413 speedup is 8537.13
',
'realstatus' => 0,
'status' => 0
};
------------------------2021-12-27 Mon 13:00:01------------------------
HOME: /Users/kpr
USER: kpr
PATH: /usr/bin:/bin
$VAR1 = {
'err' => 'rsync: failed to set times on "/Volumes/oom/quick/TEMPBACKUP/2021/2021d": Operation not permitted (1)
rsync: mkstemp "/Volumes/oom/quick/TEMPBACKUP/2021/2021d/.2021-12-27-125615-month060.tex.qZ6XYt" failed: Operation not permitted (1)
rsync: mkstemp "/Volumes/oom/quick/TEMPBACKUP/2021/2021d/.2021-12-27-125720-junk.mvgt5F" failed: Operation not permitted (1)
rsync: failed to set times on "/Volumes/oom/quick/TEMPBACKUP/2021/2021d": Operation not permitted (1)
rsync error: some files could not be transferred (code 23) at /AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/rsync/rsync-54.120.1/rsync/main.c(996) [sender=2.6.9]
',
'lastcmd' => 'rsync --itemize-changes --update --stats --compress --recursive --times --perms --links --verbose --exclude=.swp --exclude=.swo --exclude=.*.swp --exclude=.*.swo --exclude=JAW.gnucash.LCK --exclude=JAW.gnucash.LNK --exclude=JAW.gnucash*log /Users/kpr/TEMPBACKUP /Volumes/oom/quick',
'out' => 'building file list ... done
.d..t.... TEMPBACKUP/2021/2021d/
>f TEMPBACKUP/2021/2021d/2021-12-27-125615-month060.tex
>f TEMPBACKUP/2021/2021d/2021-12-27-125720-junk
Number of files: 184016
Number of files transferred: 2
Total file size: 60206953341 bytes
Total transferred file size: 686928 bytes
Literal data: 686928 bytes
Matched data: 0 bytes
File list size: 7048935
File list generation time: 18.892 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 7272636
Total bytes received: 70
sent 7272636 bytes received 70 bytes 285204.16 bytes/sec
total size is 60206953341 speedup is 8278.48
',
'realstatus' => 5888,
'status' => 23
};
Here is z.pl:
#!/usr/local/bin/perl # 2021v12v26vSunv18h08m02s for cron, which does not know my environment
##!/usr/bin/env perl # does not have the required packages under cron
use strict; use warnings;
use Data::Dumper qw(Dumper); $Data::Dumper::Sortkeys = 1;
use Time::Local qw( timelocal_posix timegm_posix ); use POSIX qw(strftime);
select STDERR; # so that STDERR and STDOUT will appear in the proper order; without this, STDERR comes before STDOUT
my $epoch_s=time;
my $dateJ =(strftime '%Y-%m-%d %a %H:%M:%S', localtime($epoch_s));
print
"\n",
'------------',
'------------',
$dateJ,
'------------',
'------------',
"\n",
;
print 'HOME: ',$ENV{HOME},"\n";
print 'USER: ',$ENV{USER},"\n";
print 'PATH: ',$ENV{PATH},"\n";
my%subvar;
$subvar{src}=join('/',$ENV{HOME},'TEMPBACKUP');
unless(-r$subvar{src} and -w$subvar{src} and -d$subvar{src})
{
die $subvar{src},q#: unless(-r$subvar{src} and -w$subvar{src} and -d$subvar{src})#;
}
$subvar{dest}=q#/Volumes/oom/quick#;
unless(-r$subvar{dest} and -w$subvar{dest} and -d$subvar{dest})
{
die $subvar{dest},q#: unless(-r$subvar{dest} and -w$subvar{dest} and -d$subvar{dest})#;
}
use File::Rsync;
my $FileRsync=File::Rsync->new(
itemize_changes=>1,
delete =>0,
update =>1,
stats =>1,
compress =>1,
recursive =>1,
times =>1,
perms =>1,
links =>1,
verbose =>1,
exclude =>['.swp','.swo','.*.swp','.*.swo','JAW.gnucash.LCK','JAW.gnucash.LNK','JAW.gnucash*log',],
);
$FileRsync->exec(
src=>$subvar{src},
,
dest=>$subvar{dest},
);
$subvar{rsync}{out}='';$subvar{rsync}{out}=join('',@{$FileRsync->out})if(defined$FileRsync->out);
$subvar{rsync}{err}='';$subvar{rsync}{err}=join('',@{$FileRsync->err})if(defined$FileRsync->err);
$subvar{rsync}{status}=$FileRsync->status;
$subvar{rsync}{realstatus}=$FileRsync->realstatus;
$subvar{rsync}{lastcmd}=$FileRsync->lastcmd;
print Dumper$subvar{rsync};
Is there a way to get cron to work with this? Or should I use some tool other than cron?
CodePudding user response：
To launch a shell and have it run its login script, you first need a shell, use the following:
/bin/sh -l -c '...shell cmd...'
You can also set up env vars in the crontab, but this is probably easier.
CodePudding user response：
This answer to my own question merely fills out details from @ikegami's solution. I hope this is the correct protocol, rather than editing @ikegami's work.
First, at https://unix.stackexchange.com/questions/308907/how-to-set-shell-bin-bash-globally-for-cron we find the following:
There is no way to set a global default shell for cron jobs.
which seems to explain why cron "does not know my path." The path I am used to is the interactive path set up automatically by the bash shell. As https://linuxhint.com/path_in_bash/ says:
By default, the PATH variable contains the following locations.
/usr/bin /usr/sbin /usr/local/bin /usr/local/sbin /bin /sbin /snap/bin (if Snap is installed)
But the cron shell is /bin/sh; that is why cron didn't have my usual path.
Now, thanks to @ikegami, the following two solutions worked:
> crontab -l
PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Library/Apple/usr/bin
00 16 * * * $HOME/u/kh/bin/jaw20200410cronmakeccalfile.sh
45 14 * * * $HOME/u/kh/bin/testcron.pl >> $HOME/tmplocal/DUMS/testcron.pl/out.txt 2>&1
>
and the following, which uses one less line of code in the crontab and also avoids hard-coding of the path, which is prone to programmer (my own) error:
> crontab -l
00 16 * * * $HOME/u/kh/bin/jaw20200410cronmakeccalfile.sh
55 14 * * * /bin/bash -l -c '$HOME/u/kh/bin/testcron.pl >> $HOME/tmplocal/DUMS/testcron.pl/out.txt 2>&1'
>
Note that the 2nd part of my question, concerning rsync and permissions, is not answered here. I will have to re-investigate that, now that the path is fixed.