remove ^M characters from file using sed-CodePudding

I have this line inside a file:

ULNET-PA,client_sgcib,broker_keplersecurities
,KEPLER

I try to get rid of that ^M (carriage return) character so I used:

sed 's/^M//g'

However this does remove everything after ^M:

[root@localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities^M,KEPLER

[root@localhost tmp]# sed 's/^M//g' test
ULNET-PA,client_sgcib,broker_keplersecurities

What I want to obtain is:

[root@localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities,KEPLER

CodePudding user response：

Use tr:

tr -d '^M' < inputfile

(Note that the ^M character can be input using Ctrl VCtrl M)

EDIT: As suggested by Glenn Jackman, if you're using bash, you could also say:

tr -d $'\r' < inputfile

CodePudding user response：

still the same line:

sed -i 's/^M//g' file

when you type the command, for ^M you type Ctrl VCtrl M

actually if you have already opened the file in vim, you can just in vim do:

:%s/^M//g

same, ^M you type Ctrl-V Ctrl-M

CodePudding user response：

You can simply use dos2unix which is available in most Unix/Linux systems. However I found the following sed command to be better as it removed ^M where dos2unix couldn't:

sed 's/\r//g' < input.txt >  output.txt

Hope that helps.

Note: ^M is actually carriage return character which is represented in code as \r What dos2unix does is most likely equivalent to:

sed 's/\r\n/\n/g' < input.txt >  output.txt

It doesn't remove \r when it is not immediately followed by \n and replaces both with just \n. This fails with certain types of files like one I just tested with.

CodePudding user response：

alias dos2unix="sed -i -e 's/'\"\$(printf '\015')\"'//g' "

Usage:

dos2unix file

CodePudding user response：

If Perl is an option:

perl -i -pe 's/\r\n$/\n/g' file

-i makes a .bak version of the input file
\r = carriage return
\n = linefeed
$ = end of line
s/foo/bar/g = globally substitute "foo" with "bar"

CodePudding user response：

This is the better way to achieve

tr -d '\015' < inputfile_name > outputfile_name

Later rename the file to original file name.

CodePudding user response：

I agree with @twalberg (see accepted answer comments, above), dos2unix on Mac OSX covers this, quoting man dos2unix:

To run in Mac mode use the command-line option "-c mac" or use the commands "mac2unix" or "unix2mac"

I settled on 'mac2unix', which got rid of my less-cmd-visible '^M' entries, introduced by an Apple 'Messages' transfer of a bash script between 2 Yosemite (OSX 10.10) Macs!

I installed 'dos2unix', trivially, on Mac OSX using the popular Homebrew package installer, I highly recommend it and it's companion command, Cask.

CodePudding user response：

In awk:

sub(/\r/,"")

If it is in the end of record, sub(/\r/,"",$NF) should suffice. No need to scan the whole record.

CodePudding user response：

This is clean and simple and it works:

sed -i 's/\r//g' file

where \r of course is the equivalent for ^M.

CodePudding user response：

Simply run the following command:

sed -i -e 's/\r$//' input.file

I verified this as valid in Mac OSX Monterey.

CodePudding user response：

remove any \r :

nawk 'NF =OFS=_' FS='\r'
gawk   3  ORS=   RS='\r'

remove end of line \r :

mawk2 8 RS='\r?\n'
mawk -F'\r$' NF=1