Why do alternate delimiters not work with sed -e '/pattern/s/a/b/'?

Question

Here is something I have discover that doesn't make sense.

Output of cat config.h:

define somevar foo // Sample variable

This version of the command works to change foo to bar and preserve the comments:

sed -e '/somevar/s/foo/bar/' config.h

This doesn't work:

sed -e '|somevar|s|foo|bar|' config.h

Gives this error:

sed: -e expression #1, char 1: unknown command: `|'

Funnily enough this does work:

sed -e '/somevar/s|foo|bar|' config.h

Perhaps I am missing some part of the documentation. It seems very odd to have two different delimiters in the same sed command.

Bug or feature?

score 33 · Accepted Answer · edited May 23 '17 at 12:10

This will work:

sed -e '\|somevar|s|foo|bar|'

The man page of GNU sed is pretty clear about this:

   /regexp/
          Match lines matching the regular expression regexp.

   \cregexpc
          Match lines matching the regular expression regexp.  The  c  may
          be any character.

That is, the c may be any character, but the starting \ is mandatory.

I don't have a FreeBSD around, but according to @bonsaiviking the man page there is also very clear:

The opening delimiter needs to be preceded by a backslash unless it is a slash.

On the other hand in OSX this is not clear at all:

      In a context address, any character other than a backslash (``\'')
      or newline character may be used to delimit the regular expression.
      Also, putting a backslash character before the delimiting character
      causes the character to be treated literally.  For example, in the
      context address \xabc\xdefx, the RE delimiter is an ``x'' and the
      second ``x'' stands for itself, so that the regular expression is
      ``abcxdef''.

Notice that the example there uses \xpatternx instead of just xpatternx. That's all the clue it gives, it doesn't make it clear that xpatternx won't work.

Based on the argument of @that-other-guy, it makes sense that sed (and other languages like perl as @Birei pointed out) need this extra clue to work correctly.

It's in [GNU sed's man page](http://linux.die.net/man/1/sed), and FreeBSD's man page is clearer: "The opening delimiter needs to be preceded by a backslash unless it is a slash." — bonsaiviking, Dec 27 '13 at 22:06
Thanks @bonsaiviking, I could check on Linux it's much more clear. I improved my answer. — janos, Dec 27 '13 at 22:18

that other guy · Answer 2 · 2013-12-27T22:38:54.423

10

Allowing different delimiters for /pattern/ would introduce parsing ambiguity.

Is ispaghetti supposed to be like /spaghett/, or is it supposed to insert text like i spaghetti?

With s and y there is no such ambiguity. When you see either of those characters, you know the command you're reading, and then you can interpret the next character as a delimiter.

We could resolve this ambiguity for /pattern/ if we started it with a similarly recognizable character, and indeed sed has a separate address specifier for this: backslash, as in \|pattern| (this is not the same as escaping).

We can therefore write \|pattern|s|foo|bar|.

The address and editing commands are separate, so \$pattern$s_foo_bar_ and /pattern/s#foo#bar# work as well.

edited Dec 27 '13 at 22:38

answered Dec 27 '13 at 21:51

that other guy

116,971
11
170
194

2

Good discussion of parsing ambiguity, but sed *does* offer this feature if the first delimiter is preceded with backslash "\". – bonsaiviking Dec 27 '13 at 22:15
1

@bonsaiviking Very good point. – that other guy Dec 27 '13 at 22:39

score 7 · Answer 3 · answered Dec 27 '13 at 21:56

You absolutely can use alternate delimiters for the matching address (/regex/), but you need to tell sed that you intend to do matching with that delimiter. The way you do this is with a leading backslash \. So your command can be:

sed -e '\|somevar|s|foo|bar|' config.h

or just as easily:

sed -e '\%somevar%s|foo|bar|' config.h

Reference: POSIX reference for sed

score 3 · Answer 4 · answered Dec 27 '13 at 21:38

3

My vote is for feature.

They are two different commands, the searching one /.../ and the substitution one, s/.../.../. As far as I know it only lets change the separator of the substitution command.

In perl is similar. You can do a regular expression search with /.../ but if you want to change the separator you have to explicity mark it as a search with the m, like: m|...|.

I guess it will be related with some lexical parsing issue, but I don't know the reason, though.

answered Dec 27 '13 at 21:38

Birei

35,723
2
77
82

1

You're on the right track, but sed uses "\" instead of `m` in this case. – bonsaiviking Dec 27 '13 at 22:00
@bonsaiviking: I didn't know that. I will thank you here and will upvote your answer that clarifies it. – Birei Dec 27 '13 at 22:11

score 2 · Answer 5 · answered Dec 27 '13 at 21:56

Feature. In

 sed -e '/somevar/s/foo/bar/' config.h

the /somevar/ is an address. Addresses must be distinguishable from other functions, like y (yank), i (insert), a (append) and many more. In general, sed commands are parsed as

 [address[,address]]function[arguments]

Why do alternate delimiters not work with sed -e '/pattern/s/a/b/'?

5 Answers5

Linked