using sed in Makefile inside docker container-CodePudding

I am using a debian-based docker container to build a LaTeX project. The following rule succeeds when run on the host (not inside docker):

.PHONY : timetracking
timetracking:
    $(eval TODAY := $(if $(PAGE),$(PAGE),$(shell TZ=$(TIMEZ) date  %Y-%m-%d)))

    touch $(PAGES)/$(WEEKLY)/$(TODAY).tex
    cat template/page-header-footer/head.tex > $(PAGES)/$(WEEKLY)/$(TODAY).tex;
    cat template/page-header-footer/pagestart.tex >> $(PAGES)/$(WEEKLY)/$(TODAY).tex;
    echo {Week of $(TODAY)} >> $(PAGES)/$(WEEKLY)/$(TODAY).tex;
    cat template/page-header-footer/timetracking.tex >> $(PAGES)/$(WEEKLY)/$(TODAY).tex;
    cat template/page-header-footer/tail.tex >> $(PAGES)/$(WEEKLY)/$(TODAY).tex;
    cat $(PAGES)/$(WEEKLY)/$(TODAY).tex \
        | sed 's/1 January/'"$$(TZ=$(TIMEZ) date  '%d %B')/g" \
        | sed 's/Jan 1/'"$$(TZ=$(TIMEZ) date  '%b %d')/g" \
        | sed 's/Jan 2/'"$$(TZ=$(TIMEZ) date  '%b %d' -d ' 1 days')/g" \
        | sed 's/Jan 3/'"$$(TZ=$(TIMEZ) date  '%b %d' -d ' 2 days')/g" \
        | sed 's/Jan 4/'"$$(TZ=$(TIMEZ) date  '%b %d' -d ' 3 days')/g" \
        | sed 's/Jan 5/'"$$(TZ=$(TIMEZ) date  '%b %d' -d ' 4 days')/g" \
        > $(PAGES)/$(WEEKLY)/$(TODAY).tex;

but when the same rule is run within the docker container, it has variable behavior:

Succeeds (file generated as expected)
Creates a blank file (unexpected)
Creates a file filled with NUL characters (unexpected)

This behavior is a result of the modifications made with sed. The template files have some text containing "January 1" and "Jan 1", "Jan 2", "Jan 3", etc. which are to be replaced.

I would like help understanding:

why does this rule behave erratically inside docker
how can I rewrite the rule to behave reliably with docker

At the moment I can run this rule (and others like it) on the host, so long as I have basic tools like Make and sed installed. But it would be ideal if I could dockerize the entire workflow.

By request, the Dockerfile contents are below. Most of the installation instructions are irrelevant since this question is around make and sed. The tools directory contains a deb file for pandoc, and is also irrelevant to this question.

FROM debian:buster

RUN apt -y update
RUN apt -y install vim 
RUN apt -y install make
RUN apt -y install texlive-full
RUN apt -y install biber
RUN apt -y install bibutils
RUN apt -y install python-pygments
RUN apt -y install cysignals-tools
RUN apt -y install sagemath
RUN apt -y install python-sagetex
RUN apt -y install sagetex

COPY tools /tools
RUN dpkg -i /tools/*deb

WORKDIR /results
ENTRYPOINT ["/usr/bin/make"]

CodePudding user response：

There's a race condition in your shell syntax. When you run

cat file.tex \
  | sed ... \
  > file.tex

first the shell opens the output file for writing (processing the > file.tex), then it creates the various subprocesses and starts them, and then at the end of this cat(1) opens the output file for reading. It's possible, but not guaranteed, that the "open for write" step will truncate the file before the "open for read" step gets any content from it.

The easiest way to get around this is to have sed(1) edit the file in place using its -i option. This isn't a POSIX sed option, but both GNU sed (Debian/Ubuntu images) and BusyBox (Alpine images) support it. sed(1) supports multiple -e options to run multiple expressions, so you can use a single sed command to do this.

# (Bourne shell syntax, not escaped for Make)
sed \
  -e 's/1 January/'"$(TZ=$(TIMEZ) date  '%d %B')/g" \
  -e 's/Jan 1/'"$(TZ=$(TIMEZ) date  '%b %d')/g" \
  -e 's/Jan 2/'"$(TZ=$(TIMEZ) date  '%b %d' -d ' 1 days')/g" \
  -e 's/Jan 3/'"$(TZ=$(TIMEZ) date  '%b %d' -d ' 2 days')/g" \
  -e 's/Jan 4/'"$(TZ=$(TIMEZ) date  '%b %d' -d ' 3 days')/g" \
  -e sed 's/Jan 5/'"$(TZ=$(TIMEZ) date  '%b %d' -d ' 4 days')/g" \
  -i \
  $(PAGES)/$(WEEKLY)/$(TODAY).tex

Be careful with this option, though. In GNU sed, -i optionally takes an extension parameter to keep a backup copy of the file, and the optional parameter can have confusing syntax. In BusyBox sed, -i does not take a parameter. In BSD sed (MacOS hosts) the parameter is required.

If you have to deal with this ambiguity, you can work around it by separately creating and renaming the file.

sed e 's/.../.../g' -e 's/.../.../g' ... \
  $(PAGES)/$(WEEKLY)/$(TODAY).tex \
  > $(PAGES)/$(WEEKLY)/$(TODAY).tex.new
mv $(PAGES)/$(WEEKLY)/$(TODAY).tex.new $(PAGES)/$(WEEKLY)/$(TODAY).tex

In a Make context you might just treat these as separate files.

# lots GNU Make extensions
export TZ=$(TIMEZ)
TODAY := $(if $(PAGE),$(PAGE),$(shell date  %Y-%m-%d))
BASENAME := $(PAGES)/$(WEEKLY)/$(TODAY)

.PHONY: timestamps
timestamps: $(BASENAME).pdf

$(BASENAME).pdf: $(BASENAME).tex
        pdflatex $<

$(BASENAME).tex: $(BASENAME)-original.tex
        sed \
          -e "s/1 January/$$(date  '%d %B')/g" \
          ...
          $< > $@

$(BASENAME)-original.tex: \
                template/page-header-footer/head.tex \
                template/page-header-footer/pagestart.tex \
                template/page-header-footer/timetracking.tex \
                template/page-header-footer/tail.tex
        cat template/page-header-footer/head.tex > $@
        cat template/page-header-footer/pagestart.tex >> $@
        echo {Week of $(TODAY)} >> $@
        cat template/page-header-footer/timetracking.tex >> $@
        cat template/page-header-footer/tail.tex >> $@

I've taken advantage of Make's automatic variables to reduce repetition here: $@ is the current target (on the left-hand side of the rule name, the file we're building) and $< is its first dependency (the first thing after the colon).

You also may consider whether some of this can be done in TeX itself. For example, there are packages to format date stamps and built-in macros to include files. If you can put all of this in the .tex file itself then you don't need the complex Make syntax.