171

It's well known how to pipe the standard ouput of a process into another processes standard input:

proc1 | proc2

But what if I want to send the standard error of proc1 to proc2 and leave the standard output going to its current location? You would think bash would have a command along the lines of:

proc1 2| proc2

But, alas, no. Is there any way to do this?

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • You can do such a simple redirection in `rc`, which is another shell. Eg: `proc1 |[2] proc2`. Isn't it nice? Not in `bash` though. – Rolf Mar 01 '18 at 20:41
  • Related: [Piping both stdout and stderr in bash?](https://stackoverflow.com/q/16497317/4561887). And [here is](https://stackoverflow.com/a/37085215/4561887) the simplest answer to pipe BOTH stdout and stderr. – Gabriel Staples Mar 17 '21 at 00:40

5 Answers5

214

There is also process substitution. Which makes a process substitute for a file.
You can send stderr to a file as follows:

process1 2> file

But you can substitute a process for the file as follows:

process1 2> >(process2)

Here is a concrete example that sends stderr to both the screen and appends to a logfile

sh myscript 2> >(tee -a errlog)
mmlb
  • 877
  • 10
  • 24
Scot
  • 2,156
  • 1
  • 13
  • 2
  • 1
    I tried this. It didn't work ( `weston --help 2> >(less)` ), and it broke my shell, I had to exit and log back in. – Rolf Mar 01 '18 at 20:47
  • 2
    @Rolf if both `weston --help` and `less` are expecting to have keyboard interaction but only 1 of them receives it, then you may be in an awkward situation. Try doing testing with something like `grep` instead. Plus you might find that both mouse/keyboard inputs are going to the 2nd command anyway rather than to weston. – BeowulfNode42 Jun 28 '18 at 08:42
  • 3
    If you want to redirect both stderr and stdout use `|&`, I learnt it from [here](https://stackoverflow.com/questions/16497317/piping-both-stdout-and-stderr-in-bash) – ᐅdevrimbaris Aug 26 '20 at 19:11
105

You can use the following trick to swap stdout and stderr. Then you just use the regular pipe functionality.

( proc1 3>&1 1>&2- 2>&3- ) | proc2

Provided stdout and stderr both pointed to the same place at the start, this will give you what you need.

What the x>&y bit does is to change file handle x so it now sends its data to wherever file handle y currently points. For our specific case:

  • 3>&1 creates a new handle 3 which will output to the current handle 1 (original stdout), just to save it somewhere for the final bullet point below.
  • 1>&2 modifies handle 1 (stdout) to output to the current handle 2 (original stderr).
  • 2>&3- modifies handle 2 (stderr) to output to the current handle 3 (original stdout) then closes handle 3 (via the - at the end).

It's effectively the swap command you see in sorting algorithms:

temp   = value1;
value1 = value2;
value2 = temp;
onlycparra
  • 607
  • 4
  • 22
paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • 5
    What is the value of using `1>&2-` here rather than just `1>&2`? I don't understand why we'd want to close fd `2`, if we're just going to reopen/reassign it immediately. – dubiousjim Oct 20 '12 at 12:00
  • 2
    @dubiousjim, no advantage in that particular case, I suspect I did it just to be consistent - closing file handle 3 is a good idea to free it up. – paxdiablo Oct 20 '12 at 12:21
  • trying to get gcc's make (which is colorized on my system) to work with this "( make 3>&1 1>&2- 2>&3- ) | less -R" whereas "( ls -al 3>&1 1>&2- 2>&3- ) | less -R" works as expected. – unsynchronized Jul 14 '15 at 04:23
  • It seems like your explanations are back to front for the second two redirections. 1>&2- sets file handle 2 (original stderr) to handle 1 (original stdout) 2>&3- sets file handle 3 (copied stdout) to handle 2 (original stderr). Please correct me if I'm wrong though. btw, I would guess that the dash on 2 is to prevent new stderr data from getting sent to this buffer while it is being populated with the data from stdout. – aghsmith Nov 12 '18 at 18:20
  • @aghsmith, that's not the case, the handle to the left is *always* the one being set, the thing on the right is being used to set it (whether handle or file name). – paxdiablo Nov 12 '18 at 21:55
  • @paxdiablo: ah, I think I used confusing language, though I still think the language in your main answer is a bit confusing. I had understood that value on the right was being copied over to the value on the left, which is indicated in the first line you wrote "3>&1 creates a new file handle 3 which is set to teh current 1...". but the language for second and third line eg: "1>&2- sets stdout to go to the current file handle 2" implies a push of information from left to right. – aghsmith Nov 13 '18 at 21:07
  • @aghsmith: okay, let me see if I can clean it up a bit. – paxdiablo Nov 13 '18 at 23:14
88

Bash 4 has this feature:

If `|&' is used, the standard error of command1 is connected to command2's standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error is performed after any redirections specified by the command.

zsh also has this feature.

--

With other/older shells, just enter this explicitly as

FirstCommand 2>&1 | OtherCommand

Lantern Rouge
  • 43
  • 4
Dennis Williamson
  • 346,391
  • 90
  • 374
  • 439
  • 17
    From reading the docs, that does both standard error *and* output as opposed to just stderr, but it's nice to know. Time to start looking at bash 4, I think. – paxdiablo Oct 02 '09 at 07:11
  • The current bash manual reads "If |& is used, command's standard error, **in addition to its standard output,** is connected to command2's standard input". This is explicitly not what the OP wants. – Peter - Reinstate Monica Jun 21 '17 at 14:37
  • @PeterA.Schneider: The OP says "leave the standard output going to its current location" which may be ambiguous. – Dennis Williamson Jun 21 '17 at 14:54
  • I'm failing to see any ambiguity. Your suggestion (1) *conflates* the two streams. (2) `OtherCommand` writes the *combined* data somewhere, possibly somewhere else. So it's not the same data, and it is potentially going somewhere else. That is about the opposite of the OP's wish, isn't it? – Peter - Reinstate Monica Jun 21 '17 at 20:37
  • @PeterA.Schneider: Where else is standard output's current location? If `proc1` outputs to stdout and to stderr and you want stderr to go to the stdin of `proc2` (which is where proc1's stdout is going), then my answer accomplishes that. I gave the OP what he _asked_ for, perhaps not what he _meant_ to ask for. Therein lies the potential ambiguity. The OP accepted the answer which _swaps_ stdout and stderr which is _not_ what he asked for. – Dennis Williamson Jun 21 '17 at 20:59
  • `OtherCommand` may not even write to stdout! It may write the combined data to `/dev/null`. That is *not* "leaving it alone". In the accepted answer, by contrast, the original stdout's data is still available -- separate from the original stderr's data, and unchanged! -- on stderr (because `proc2` doesn't process the new `stderr`, containing the original `stdout` data), so the OP can now handle it any way he wants to. I think that *combining* the two streams is why your solution fails the spec, in any conceivable interpretation. – Peter - Reinstate Monica Jun 22 '17 at 05:33
  • @PeterA.Schneider: No one said anything at all about what `OtherCommand` (or the OP's `proc2`) does except the implication that it accepts input on its stdin. and what the second process does is not specified in the question and is irrelevant. I think you fail to see that the OP originally asked for something different than what he really wanted. I answered the question _as asked_. Continuing this argument is pointless. – Dennis Williamson Jun 22 '17 at 15:03
  • You are right: What the second program does is irrelevant in the OP's scenario with respect to the first program's stdout. That is so because the second program does not see, let alone alter the stdout of the first program (it *does* see and process, in an unknown fashion, stderr). It was the OP's request that the first program's (and hence pipeline's) stdout stay unaltered. Now in your scenario the second program *does* suddenly process the first program's stdout (together with its stderr). So in *your* scenario what the second program does with its input suddenly *is* relevant. – Peter - Reinstate Monica Jun 22 '17 at 17:12
  • Consider proc1: `echo "valuable output"; echo "diagnostic noise" >&2` and proc2: `while read; do true; done`. Now what 's the contents of `file` after `(proc1 2> >(proc2)) > file` vs. your suggestion `(proc1 |& proc2) > file`? I think in the second case it is empty, because that pattern does not "leave the standard output going to its current location". – Peter - Reinstate Monica Jun 22 '17 at 17:20
  • @PeterA.Schneider: You're talking about what `proc2` does with its input _after it receives it_. This is _not at all_ relevant in the OP's question. Also, the subshells in your examples are unnecessary. – Dennis Williamson Jun 22 '17 at 17:33
  • I tried very patiently to explain why it *becomes* relevant in your solution although it wasn't originally. Do you agree that the output in the two examples is different? (I think it is, but as always I may be wrong.) In particular, in your solution there is nothing on stdout (and hence in `file`) any longer, although the OP asked to leave stdout intact. – Peter - Reinstate Monica Jun 22 '17 at 18:28
34

Swapping is great as it solves the problem. Just in case you do not even need the original stdout, you can do it this way:

proc1 2>&1 1>/dev/null | proc2

The order is vital; you would not want:

proc1 >/dev/null 2>&1 | proc1

As this will redirect everything to /dev/null!

Chadwick
  • 12,555
  • 7
  • 49
  • 66
kccqzy
  • 1,538
  • 1
  • 14
  • 22
4

None of these really worked very well. The best way I found do do what you wanted is:

(command < input > output) 2>&1 | less

This only works for cases where command does not need keyboard input. eg:

(gzip -d < file.gz > file) 2>&1 | less

would put gzip errors into less

sbingner
  • 124
  • 7