Confusing behaviour when streaming exec command outputs from c programs and shell scripts-CodePudding

I wrote a test program to analyse the behaviour of a piece of code that caused a bug today, to understand its behaviour better. The opposite happend.

This is the test program. It should execute a testcommand and stream the commands output to stdout.

import (
    "bufio"
    "fmt"
    "io"
    "os/exec"
)

func main() {
    cmd1 := exec.Command("./testcommands/testcommand.sh")
    execCmd(cmd1)
    cmd2 := exec.Command("./testcommands/testcommand")
    execCmd(cmd2)
}

func execCmd(cmd *exec.Cmd) {
    stderr, _ := cmd.StderrPipe()
    stdout, _ := cmd.StdoutPipe()
    multi := io.MultiReader(stdout, stderr)
    scanner := bufio.NewScanner(multi)
    cmd.Start()
    for scanner.Scan() {
        m := scanner.Text()
        fmt.Println(m)
    }
    cmd.Wait()
}

The two testcommands called do basically the same. One is implemented in bash

#!/bin/bash

for i in `seq 1 10` ; do 
    echo "run $i"
    sleep 1
done

The other one in C

#include <stdio.h>
#include <unistd.h>

int main() {
    int i;
    for (i=1; i<=10; i  ) {
        printf("run %d\n", i);
        sleep(1);
    }
    return 0;
}

The output of the shell script does get streamed (1 line per second), however the output of the c program only arrives after the program is finished completely (10 lines at once after 10 seconds).

This goes way over my head. I'm not even sure if this is working as intended and i'm just missing something, or if i should open a bug report - and if so, i'm not even sure if it'll be for bash, golang or c. Or maybe it's some linux thing i don't know about.

CodePudding user response：

When stdout (to which printf writes) is connected to a terminal, then stdout will be line-buffered and output will be flushed (actually written) on each newline.

But when stdout is not connected to a terminal, like for example it's used for redirection or a pipe, then it's fully buffered. Fully buffered means the output only will be written when the buffer becomes full (unlikely in your small example) or when explicitly flushed (for example with fflush(stdout)).

What the Go exec functionality probably does is to create pipes for the programs input and output.

To solve your problem, your C program needs to call fflush after each printf call:

printf("run %d\n", i);
fflush(stdout);