Why do shell scripts not behave the same when called from node.js?

Question

My problem

I have a simple sh script that behaves exactly as I want, unless it's called from a node.js script.

What the sh script is supposed to do

• Convert the (potentially binary) data passed via stdin to base64
• Store that base64 string in a variable
• Print the contents of that variable to stdout
• If no data is passed to stdin, exit immediately without printing to stdout

My sh script

/tmp/aaa:

#!/bin/sh
! [ -t 0 ] && stdin_base64=$(base64 -w 0) || stdin_base64=""
echo -n "$stdin_base64"

When called from a terminal it works as expected

Without stdin:

$ /tmp/aaa

With stdin:

$ echo foo | /tmp/aaa
Zm9vCg==

With binary stdin:

$ echo -n -e '\x00\x00\x00' | /tmp/aaa
AAAA

With a ton of binary stdin that takes multiple seconds to be written:

$ dd if=/dev/urandom bs=1 count=10M | /tmp/aaa | rev | head -c 1

When called from node.js it breaks

When the exact same script gets called from node.js using execFile like this:

const { execFile } = require('child_process');

execFile('/tmp/aaa', [], {}, (error, stdout, stderr) => {
  console.log(error, stdout, stderr);
});

it just gets stuck indefinitely, without exiting, no errors and nothing gets printed to stdout or stderr. I assume it just waits for stdin forever because when I change the script to simply echo something, it exits immediately after printing:

#!/bin/sh
echo "test"

What I can/cannot do

• I cannot change the node.js script
• I cannot use Bash (I'm using an Alpine-based Docker image that only supports basic POSIX sh.)
• I cannot install additional software.
• The sh script needs to be changed in a way that it can handle stdin (or the lack thereof) properly, so that I always get the same behavior that I'm seeing when calling the script on the sh terminal directly.
• It must support binary stdin data including null bytes.
• I cannot use something like timeout 1 base64 -w because reading all data from stdin may take well more than 1 second.

Ideas

The closest thing I could come up with was this:

#!/bin/sh
stdin_file=$(mktemp)
cat - | base64 -w 0 > $stdin_file &
base64_pid=$!
timeout 1 sh -c "while [ ! -s $stdin_file ]; do sleep 0.1; done" && wait $base64_pid || kill $base64_pid 2&> /dev/null
stdin_base64=$(cat $stdin_file 2> /dev/null)
rm -f $stdin_file
echo -n "$stdin_base64"

But this would of course always result in a wasted second when called from node.js:

$ node -e "require('child_process').execFile('/tmp/aaa', [], {}, console.log)"

and on the other hand if it takes more than a second for data to arrive on stdin, it will fail, thinking there was no stdin:

$ (sleep 2; echo 123) | /tmp/aaa

Answer 1

What you want to do is peek at stdin and see if any character is being provided. base64 will expect input before exiting so ideally you would want to provide it through the nodejs script (See How to pass STDIN to node.js child process)

As a workaround, you can add a timeout to your bash script to expect stdin to be filled. Here is an example with a timeout of 1 second. /tmp/aaa:

#!/bin/bash
read -t1 input
echo -n $input | base64 -w 0

Answer 2

Not sure if this will fully solve your problen, if you cannot change the existing node.js code, but I suspect that is the problem area.

I seem to remember struggling with this in the past. I believe the call to execFile returns a child process, and you need to listen to events on it, as in this code of mine:

child.stdout.on('data', data =>...);
child.stderr.on('data', data =>...);
child.on('close', code =>...);
child.on('error', e =>...);

You ought to be able to at least debug things a little better using this approach. It seems that right now you are just hanging, and this may get you a little further.

Answer 3

it just gets stuck indefinitely, without exiting, no errors and nothing gets printed to stdout or stderr

This reads like a docker container started with -i but without -t.
Using the same script:

root# cat /tmp/test.sh
#!/bin/sh
! [ -t 0 ] && stdin_base64=$(base64 -w 0) || stdin_base64=""
echo -n "$stdin_base64"

Then running the following combinations work without issues:

root# docker run --rm -it -v /tmp:/host/tmp alpine:latest /host/tmp/test.sh
root# docker run --rm -t -v /tmp:/host/tmp alpine:latest /host/tmp/test.sh
root# docker run --rm -v /tmp:/host/tmp alpine:latest /host/tmp/test.sh

Running this however, results in what you described:

root# docker run --rm -i -v /tmp:/host/tmp alpine:latest /host/tmp/test.sh

Related: When would I use --interactive without --tty in a Docker container?