74

I have been using d2u to convert line endings. After installing Puppy Linux I noticed that it does not come with d2u, but dos2unix. Then I noticed that Ubuntu is missing both by default.

What is another way to convert line endings?

halfer
  • 19,824
  • 17
  • 99
  • 186
Zombo
  • 1
  • 62
  • 391
  • 407

2 Answers2

122

Some options:

Using tr

tr -d '\15\32' < windows.txt > unix.txt

OR

tr -d '\r' < windows.txt > unix.txt

Using perl

perl -p -e 's/\r$//' < windows.txt > unix.txt

Using sed

sed 's/^M$//' windows.txt > unix.txt

OR

sed 's/\r$//' windows.txt > unix.txt

To obtain ^M, you have to type CTRL-V and then CTRL-M.

B T
  • 57,525
  • 34
  • 189
  • 207
jaypal singh
  • 74,723
  • 23
  • 102
  • 147
  • 8
    on my mac only this works: `tr -d '\r' < windows.txt > unix.txt` – Davoud Taghawi-Nejad Sep 04 '15 at 21:22
  • 2
    I learned on mac you cannot use tr to open and write to the same file. That results in a blank file, but writing to a different name works great! – Loren Jul 28 '16 at 00:59
  • 2
    @Loren i think that should be your assumption with any redirection. The destination file is opened before the reading of the source. Some commands let you do "in-place" like `sed`'s `-i` but use intermediate/backup files anyway – nhed Nov 02 '16 at 20:37
  • These answers are generally correct and you could add `awk '{sub(/\r$/,"")}1' windows.txt > unix.tx` but be aware that the `tr` is deleting **all** `\r`s from the input, not just those that occur at the end of each line as the perl, sed, and now awk scripts would do. – Ed Morton Aug 19 '17 at 13:42
  • 1
    Redirecting stdout and stdin in the same line kinda messes with my head so I used: `cat windows_newlines.txt | tr -d '\r' > unix_newlines.txt` – jocassid Jun 07 '19 at 16:10
  • tr -d '\15\32' working in Ubuntu18 – Samuel Sep 25 '20 at 08:57
73

Doing this with POSIX is tricky:

  • POSIX Sed does not support \r or \15. Even if it did, the in place option -i is not POSIX

  • POSIX Awk does support \r and \15, however the -i inplace option is not POSIX

  • d2u and dos2unix are not POSIX utilities, but ex is

  • POSIX ex does not support \r, \15, \n or \12

To remove carriage returns:

awk 'BEGIN{RS="^$";ORS="";getline;gsub("\r","");print>ARGV[1]}' file

To add carriage returns:

awk 'BEGIN{RS="^$";ORS="";getline;gsub("\n","\r&");print>ARGV[1]}' file
Zombo
  • 1
  • 62
  • 391
  • 407
  • Those awk scripts are GNU awk only due to multi-character RS (more than 1 char in a RS invokes undefined behavior in POSIX so some POSIX awks will silently drop the `$` and retain just the `^`, others can do whatever else they like), they would produce unexpected results when the `getline` fails, they will only operate on the first line of the input, and they will corrupt the input file in some situations and if they were fixed to operate on all lines would cause an infinite loop in others by writing to the input file as it's being read. Do **not** execute those scripts. – Ed Morton Aug 19 '17 at 13:36
  • 3
    This work on the **same file**, ie. it replace line endings in-place. While the `tr` solutions require different file as an output. – PeterM Sep 07 '17 at 10:59
  • 1
    sed or tr is not working for me. – Samuel Sep 25 '20 at 08:44