bash: a simple FIFO (first-in first-out) queue-CodePudding

I am looking for a portable bash solution for a simple FIFO queue. My queue input will always be a single line of text.I came up with a simple solution, which works well.

fifo_queue="fifo.txt";
fifo_tmp="fifo.tmp"

# put a new line into FIFO
echo "append to FIFO a line" >> $fifo_stack;

#get line back from FIFO
echo $(head -n1 "$fifo_queue")
tail -n  2 "$fifo_queue" > "$fifo_tmp" && mv "$fifo_tmp" "$fifo_queue"

Now the question: Is there a more elegant (i.e. easy understandable) way to retrieve the FIFO line. For example with a simple one-liner instead of two lines?

CodePudding user response：

One option for retrieving the next FIFO line is to use the POSIX standard ed utility:

printf '%s\n' 1p 1d w | ed -s -- "$fifo_stack"

This invokes ed on the FIFO file and issues 3 commands to it. 1p prints the first line. 1d deletes the first line. w saves the change back to the file.

You may run into problems with this approach if the FIFO file is very large, but you should be OK up to tens of megabytes on a modern machine.

CodePudding user response：

Not easier nor clearer, but portable and safer:

fifo_file=fifo.tmp

rm -f "$fifo_file" && mkfifo "$fifo_file" && exec 3<> "$fifo_file" || exit 1

printf '%s\n' "append to FIFO a line" >&3

IFS='' read -r fifo_line <&3

printf '%s\n' "$fifo_line"

exec 3>&-

CodePudding user response：

you can use a real fifo. Example

#!/bin/bash

#--- setup
FIFO="fifo.tmp"
rm -f "$FIFO"
mkfifo "$FIFO"

#--- sending data in background
for (( i = 0; i < 10; i   )); do echo $i; sleep 1; done >"$FIFO" &

#--- receiving data
echo "start"
while read line
do
    echo "> $line"
done < "$FIFO"
echo "stop"

#--- clean
rm -f "$FIFO"

CodePudding user response：

A simple one-liner is below.

First, let's put some content into our sample file:

FIFO_STACK=fifo.txt
echo -e "1\n2\n3\n" >$FIFO_STACK

Second, let's echo our first line and make sure this line is removed from the file. That can be achieved with sed with update in-place:

echo $(head -n1 "$FIFO_STACK") && sed -i '' '1d' "$FIFO_STACK"
1

The expected result, first line will be displayed, but it will be also removed from the file. Let's verify it:

cat $FIFO_STACK
2
3

I work on Mac, hence I had to add '' after -i, AFAIK on Linux you can skip this part and it should work as well.

CodePudding user response：

For a fifo stack, the modified version of your logic would be something like the below script. Naturally, you would need to modify that to allow passing the desired line as a parameter on the command line (or as redirected input).

As for removing the first/last line from the stack/buffer file from the file in-place, that would be a task for sed.

I was able to identify the logic for deleting the first/last line depending on fifo/lifo mode using very simple sed directives. That is implemented in the below script.

#!/bin/bash

Push=0
Pop=0
fifo=1 ; mode="fifo"    ### DEFAULT
lifo=0

while [ $# -gt 0 ]
do
    case $1 in
        --push ) Push=1 ; Pop=0 ; shift ;;
        --pop ) Pop=1 ; Push=0 ; shift ;;
        --fifo ) fifo=1 ; lifo=0 ; mode="fifo" ; shift ;;   ### Buffer MODE
        --lifo ) lifo=1 ; fifo=0 ; mode="lifo" ; shift ;;   ### Stack MODE
        * ) echo -e "\n Invalid parameter used on command line.  Only valid options: [ --push | --pop ]\n Bye!\n" ; exit 1 ;;
    esac
done

if [ ${Push} -eq 0 -a ${Pop} -eq 0 ]
then
    echo -e "\n ERROR: must specify one of the two stack operations: --push or --pop \n Bye!\n" ; exit 1
fi

stack="${mode}.txt";
tmp="${mode}.tmp"
if [ ! -f "${stack}" ] ; then touch "${stack}" ; fi

i=$(( $(wc -l "${stack}" | awk '{ print $1 }' )   1 ))

if [ ${Push} -eq 1 ]
then
    # put a new line into FIFO/LIFO
    echo "append to FIFO - line ${i}" >> "${stack}"
    wc -l "${stack}"
fi

if [ ${Pop} -eq 1 ]
then
    if [ -s "${stack}" ]
    then
        #get line back from FIFO
        if [ ${fifo} -eq 1 ]
        then
            head -n1 "${stack}"
            sed --in-place '1d' "${stack}"
            wc -l "${stack}"
        else
            tail -n1 "${stack}"
            sed --in-place '$d' "${stack}"
            wc -l "${stack}"
        fi
    else
        echo -e "\t ERROR: attempt to pop line from EMPTY stack.\n" >&2
        exit 1
    fi
fi

Session log is as follows:

me@OasisMega1:/0__WORK$ ./test_129.sh --push -fifo
1 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --push -fifo
2 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --push -fifo
3 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --push -fifo
4 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --pop -fifo
append to FIFO - line 1
3 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --pop -fifo
append to FIFO - line 2
2 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --pop -fifo
append to FIFO - line 3
1 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --pop -fifo
append to FIFO - line 4
0 fifo.txt
me@OasisMega1:/0__WORK$ ./test_129.sh --pop -fifo
     ERROR: attempt to pop line from EMPTY stack.
me@OasisMega1:/0__WORK$

Session log for LIFO mode is as follows:

me@OasisMega1:0__WORK$ ./test_129.sh --push --lifo
1 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --push --lifo
2 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --push --lifo
3 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --push --lifo
4 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --pop --lifo
append to FIFO - line 4
3 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --pop --lifo
append to FIFO - line 3
2 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --pop --lifo
append to FIFO - line 2
1 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --pop --lifo
append to FIFO - line 1
0 lifo.txt
me@OasisMega1:0__WORK$ ./test_129.sh --pop --lifo
     ERROR: attempt to pop line from EMPTY stack.

me@OasisMega1:0__WORK$

This version of the script incorporates providing input for buffer/stack on command line, and option for display of current buffer/stack contents. Also provides return code which might be used for decision on whether contents requiring "processing" are still in buffer or not.

#!/bin/bash
    
Push=0
Pop=0
Display=0
fifo=1 ; mode="FIFO"    ### DEFAULT
lifo=0
dataline=""

while [ $# -gt 0 ]
do
    case $1 in
        --push ) Push=1 ; Pop=0 ; shift ;;
        --pop ) Pop=1 ; Push=0 ; shift ;;
        --show ) Push=0; Pop=0 ; Display=1 ; shift ;;
        --fifo ) fifo=1 ; lifo=0 ; mode="FIFO" ; shift ;;   ### Buffer MODE
        --lifo ) lifo=1 ; fifo=0 ; mode="LIFO" ; shift ;;   ### Stack MODE
        -- ) shift ; dataline=${@} ; break ;;
        * ) echo -e "\n Invalid parameter used on command line.  Only valid options: [ --push | --pop ]\n Bye!\n" ; exit 1 ;;
    esac
done

if [ ${Display} -eq 1 ]
then
    if [ ${fifo} -eq 1 ]
    then
        if [ -f FIFO.txt ]
        then
            if [ -s FIFO.txt ]
            then
                wc -l FIFO.txt >&2
                echo -e "Hit return to view contents ...\c" >&2 ; read k <&2
                more FIFO.txt
                RC=98
            else
                echo -e "\n FIFO buffer is empty at this time.\n" >&2
                RC=0
            fi
        else
            echo -e "\n FIFO buffer does not exist.\n" >&2
            RC=99
        fi
    fi
    if [ ${lifo} -eq 1 ]
    then
        if [ -f LIFO.txt ]
        then
            if [ -s LIFO.txt ]
            then
                wc -l LIFO.txt >&2
                echo -e "Hit return to view contents ...\c" >&2 ; read k <&2
                more LIFO.txt
                RC=98
            else
                echo -e "\n LIFO buffer is empty at this time.\n" >&2
                RC=0
            fi
        else
            echo -e "\n LIFO buffer does not exist.\n" >&2
            RC=99
        fi
    fi
    exit ${RC}
fi

if [ ${Push} -eq 1 -a -z "${dataline}" ]
then
    echo -e "\n ERROR: did not provide data for capture in BUFFER/STACK\n Bye!\n" ; exit 1
fi

if [ ${Push} -eq 0 -a ${Pop} -eq 0 ]
then
    echo -e "\n ERROR: must specify one of the two stack operations: --push or --pop \n Bye!\n" >&2 ; exit 1
fi

stack="${mode}.txt";
tmp="${mode}.tmp"
if [ ! -f "${stack}" ] ; then touch "${stack}" ; fi

if [ ${Push} -eq 1 ]
then
    # put a new line into FIFO/LIFO
    echo "${dataline}" >> "${stack}"
    wc -l "${stack}" >&2
fi

if [ ${Pop} -eq 1 ]
then
    if [ -s "${stack}" ]
    then
        #get line back from FIFO
        if [ ${fifo} -eq 1 ]
        then
            head -n1 "${stack}"
            sed --in-place '1d' "${stack}"
            wc -l "${stack}" >&2
        else
            tail -n1 "${stack}"
            sed --in-place '$d' "${stack}"
            wc -l "${stack}" >&2
        fi
    else
        echo -e "\t ERROR: attempt to pop line from EMPTY stack.\n" >&2
        exit 1
    fi
fi