0

I'm trying to format a string as JSON using jq and I noticed differing behaviors on bash vs zsh; specifically when zsh runs jq directly, the outcome is different than when it runs it in subshell: \n input gets output as \\n in first case, vs as\n in the latter.

I'm puzzled and not sure what's going on there:

  • Is this a known zsh behavior?
  • Is this a jq bug?
  • Or does it work as designed and I'm missing something?

BTW: Use newline with jq suggests to use printf %b to obtain \n instead of \\n, which works for bash,. but discrepancy in zsh between the modes is still there.

$ jq --version
jq-1.6

# ---
# Using \n directly

bash-3.2$        jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'
{"test":"A\\nB"}
bash-3.2$  OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'); echo $OUT
{"test":"A\\nB"}


zsh-5.8.1>       jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'
{"test":"A\\nB"}
zsh-5.8.1> OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test 'A\nB' '{test: $test}'); echo $OUT
{"test":"A\nB"}

# -----
# Using `printf %b` to convert `\n` to real newline

bash-3.2$        jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'
{"test":"A\nB"}
bash-3.2$  OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'); echo $OUT
{"test":"A\nB"}


zsh-5.8.1>       jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'
{"test":"A\nB"}
zsh-5.8.1> OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}'); echo $OUT
{"test":"A
B"}
jakub.g
  • 38,512
  • 12
  • 92
  • 130
  • 1
    `echo` is perhaps the only place I'm aware of where zsh is _more_ compliant with the POSIX specification on default settings than bash is. (Treating backslash-escape expansion as always-on is explicitly permitted; treating it as always-off is also permitted; turning it on only when `-e` is present is not okay, as `-n` is the only thing `echo` is allowed to treat as a flag). – Charles Duffy Sep 29 '22 at 17:01
  • (because POSIX specifies that `echo`'s output becomes unspecified when any backslash is present in input, one can argue that this makes `echo -e`'s behavior permissible in bash... but I'd argue that this makes it permissible _only when there's actually a backslash somewhere in the string_, but bash's `echo` consumes `-e` as an option even when no backslashes are present in later arguments). – Charles Duffy Sep 29 '22 at 17:04

1 Answers1

3

printf behavior is the same between both shells, as are all the shell expansions related to your jq invocation -- but default echo behavior differs.

You can avoid this by switching from echo to printf.

% OUT=$(jq --null-input --compact-output --raw-output --monochrome-output --arg test "$(printf %b 'A\nB')" '{test: $test}')
% printf '%s\n' "$OUT"
{"test":"A\nB"}
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • TIL! Thanks! From my quick test it seems that zsh `echo` works like bash `echo -e` when it comes to `\n` printing. – jakub.g Sep 29 '22 at 17:04
  • 2
    `echo` is a mess. For a fun read, see the APPLICATION USAGE and RATIONALE sections of https://pubs.opengroup.org/onlinepubs/9699919799/utilities/echo.html -- the very standards document defining `echo` recommends using `printf` instead. – Charles Duffy Sep 29 '22 at 17:05