Why does the spawn child process print to stdout only after execution with Python?

Question

I have created the following node.js code with the spawn child process that executes a bash code inside it:

const spawn = require('child_process').spawn

const shellcode = `
file=$(mktemp)
cat > $file <<- "EOF"
echo "Hello World"
sleep 2
echo "Hello World"
sleep 2
echo "Hello World"
sleep 2
echo "Hello World"
EOF

bash $file
`

const child = spawn('/bin/bash', [ '-c',  shellcode ])

child.stdout.on('data', (data) => {
    console.log(data.toString())
})

It works perfectly fine, it executes the bash inside the shellcode variable and it prints Hello World while the node.js code is running.

My problem starts with Python, I've created the following code:

const spawn = require('child_process').spawn

const shellcode = `
file=$(mktemp)
cat > $file <<- "EOF"
import os

print('Hello World')
os.system("sleep 2")
print('Hello World')
os.system("sleep 2")
print('Hello World')
os.system("sleep 2")
print('Hello World')
EOF

python3 $file
`

const child = spawn('/bin/bash', [ '-c',  shellcode ])

child.stdout.on('data', (data) => {
    console.log(data.toString())
})

However, this code doesn't execute as a spawn code. It waits for the process to finish to print the results to stdout. I've also tested the same logic with a node.js code inside the shellcode variable and it works fine just like bash. What am I getting wrong? I'm not sure if it's a Python issue or if it's a node.js issue. Why is the Python code not behaving as a spawn (it's waiting for the process to finish to print its output)?

I'm guessing the problem is that stdout uses [buffered I/o](https://stackoverflow.com/questions/1450551/buffered-vs-unbuffered-io). SUGGESTION: Add either/both a final newline (`\n`) and/or an explicit [flush](https://stackoverflow.com/a/30845184/421195) — paulsm4, Dec 09 '21 at 22:34
try using the options `{stdio:'inherit',env:process.env,cwd:undefined,shell:true}` as a third argument in the `spawn` command — The Bomb Squad, Dec 09 '21 at 23:51
I've tried adding `\n`, an explicit flush with `stdbuf -oL python3 $file` and `stdio:'inherit',env:process.env,cwd:undefined,shell:true}` as a third argument in the `spawn` command as said in the suggestions but none of them worked. The `spawn` is still behaving as an `exec` command when I'm executing `Python`. — raylight, Jan 19 '22 at 13:12

AKX · Accepted Answer · 2022-01-19T13:18:42.577

Python buffers its output by default.

Run python -u ... or set the PYTHONUNBUFFERED environment variable to 1.

As an aside, though, there's no reason to use bash here at all – Python can happily read code from stdin and execute it without a temporary file:

const spawn = require('child_process').spawn

const shellcode = `
import os, time

print('Hello 1')
time.sleep(2)
print('Hello 2')
time.sleep(2)
print('Hello 3')
time.sleep(2)
print('Hello 4')
`.trim();

const child = spawn('/usr/bin/python3', ['-u', '-'], {stdio: ['pipe', 'pipe', 'inherit']});
child.stdin.write(shellcode);
child.stdin.end();
child.stdout.on('data', (data) => {
    console.log(new Date(), data.toString());
});

prints out (at the time of writing):

2022-01-19T13:18:16.084Z Hello 1

2022-01-19T13:18:18.088Z Hello 2

2022-01-19T13:18:20.091Z Hello 3

2022-01-19T13:18:22.093Z Hello 4

Why does the spawn child process print to stdout only after execution with Python?

1 Answers1