Categories
escaping newline replace vi vim

How to replace a character by a newline in Vim

2284

I’m trying to replace each , in the current file by a new line:

:%s/,/\n/g 

But it inserts what looks like a ^@ instead of an actual newline. The file is not in DOS mode or anything.

What should I do?

If you are curious, like me, check the question Why is \r a newline for Vim? as well.

3

2951

Use \r instead of \n.

Substituting by \n inserts a null character into the text. To get a newline, use \r. When searching for a newline, you’d still use \n, however. This asymmetry is due to the fact that \n and \r do slightly different things:

\n matches an end of line (newline), whereas \r matches a carriage return. On the other hand, in substitutions \n inserts a null character whereas \r inserts a newline (more precisely, it’s treated as the input CR). Here’s a small, non-interactive example to illustrate this, using the Vim command line feature (in other words, you can copy and paste the following into a terminal to run it). xxd shows a hexdump of the resulting file.

echo bar > test
(echo 'Before:'; xxd test) > output.txt
vim test '+s/b/\n/' '+s/a/\r/' +wq
(echo 'After:'; xxd test) >> output.txt
more output.txt
Before:
0000000: 6261 720a                                bar.
After:
0000000: 000a 720a                                ..r.

In other words, \n has inserted the byte 0x00 into the text; \r has inserted the byte 0x0a.

17

  • 145

    /r is treated as pressing the Enter/Return key. It works on all platforms.

    Oct 12, 2008 at 11:41

  • 83

    See also Why is \r a newline for Vim?.

    Apr 5, 2012 at 6:14

  • 17

    I wish this worked for classic vi. On AIX v6.1, \r doesn’t work like this. But you can press Ctrl-V Enter in place of typing \r, and it works.

    – eksortso

    Apr 26, 2013 at 19:52

  • 5

    I’m late to the party. If \r inserts <CR> and \n inserts a null, how would I replace something with a carriage return?

    – Mr. Llama

    Jun 3, 2015 at 21:18

  • 5

    @SunnyRaj are you sure about the history of CR and LF? I was under impression that originally LF moved paper one row forward but did not move the printhead, and CR moved the printhead but did not move the paper. As a result, if your OS did not convert the input before printing, you could not just use just LF nor CR to get the correct output. MS DOS used raw printer data as the text file format, Mac OS used CR and converted from that to printer’s raw format and UNIX used LF and converted from that to printer’s raw format.

    Dec 21, 2017 at 7:46

226

Here’s the trick:

First, set your Vi(m) session to allow pattern matching with special characters (i.e.: newline). It’s probably worth putting this line in your .vimrc or .exrc file:

:set magic

Next, do:

:s/,/,^M/g

To get the ^M character, type Ctrl + V and hit Enter. Under Windows, do Ctrl + Q, Enter. The only way I can remember these is by remembering how little sense they make:

A: What would be the worst control-character to use to represent a newline?

B: Either q (because it usually means “Quit”) or v because it would be so easy to type Ctrl + C by mistake and kill the editor.

A: Make it so.

5

  • 9

    I’m using GVim on Windows, and I need neither the :set magic (it’s not in my ~/_vimrc either) or ctrl-q. Just a simple ctrl-v followed by enter creates the ^M character for me just fine.

    Sep 14, 2011 at 21:02

  • 6

    C-v doesn’t represent a newline; it’s the “escape next literal character” command. I dunno what C-v is a mnemonic for either, but there’s a reason it doesn’t mentally map to newline.

    Jun 22, 2012 at 21:26

  • 29

    Ctrl-v is a mnemonic for “verbatim” – i.e. escape next key pressed to its “verbatim” keycode/character. In Windows it’s paste: to keep things familiar. Ctrl-Q is for “(un)Quote” maybe. Quite stupid, anyway – but you can use it in binary files – e.g. to search for Ctrl-A through Ctrl-Z (Ascii 1-26 I guess).

    Nov 6, 2013 at 9:49

  • 1

    Ctrl-C doesn’t actually kill the editor, although it can cancel you back to Normal mode. Ctrl-V means verbatim, and Ctrl-Q means that someone made the mistake of loading the $VIMRUNTIME/mswin.vim configuration file. You don’t need mswin. Just use your own vimrc instead.

    – 00dani

    Oct 12, 2016 at 4:28


  • wow—-this is amazing. I used to do sed -n l till now. Good to know that the same can be achieved with Ctrl-v in vim.

    – arpit

    May 15, 2020 at 4:35

118

In the syntax s/foo/bar, \r and \n have different meanings, depending on context.


Short:

For foo:

\r == “carriage return” (CR / ^M)
\n == matches “line feed” (LF) on Linux/Mac, and CRLF on Windows

For bar:

\r == produces LF on Linux/Mac, CRLF on Windows
\n == “null byte” (NUL / ^@)

When editing files in linux (i.e. on a webserver) that were initially created in a windows environment and uploaded (i.e. FTP/SFTP) – all the ^M‘s you see in vim, are the CR‘s which linux does not translate as it uses only LF‘s to depict a line break.


Longer (with ASCII numbers):

NUL == 0x00 == 0 == Ctrl + @ == ^@ shown in vim
LF == 0x0A == 10 == Ctrl + J
CR == 0x0D == 13 == Ctrl + M == ^M shown in vim

Here is a list of the ASCII control characters. Insert them in Vim via Ctrl + V,Ctrl + ---key---.

In Bash or the other Unix/Linux shells, just type Ctrl + ---key---.

Try Ctrl + M in Bash. It’s the same as hitting Enter, as the shell realizes what is meant, even though Linux systems use line feeds for line delimiting.

To insert literal’s in bash, prepending them with Ctrl + V will also work.

Try in Bash:

echo ^[[33;1mcolored.^[[0mnot colored.

This uses ANSI escape sequences. Insert the two ^[‘s via Ctrl + V, Esc.

You might also try Ctrl + V,Ctrl + M, Enter, which will give you this:

bash: $'\r': command not found

Remember the \r from above? :>

This ASCII control characters list is different from a complete ASCII symbol table, in that the control characters, which are inserted into a console/pseudoterminal/Vim via the Ctrl key (haha), can be found there.

Whereas in C and most other languages, you usually use the octal codes to represent these ‘characters’.

If you really want to know where all this comes from: The TTY demystified. This is the best link you will come across about this topic, but beware: There be dragons.


TL;DR

Usually foo = \n, and bar = \r.

4

  • 1

    So I’m intrigued how you would substitute a character with a carriage return

    – codeshot

    Apr 7, 2015 at 22:25

  • 2

    @codeshot :s/x/^M/g should do. Insert the ^M via ctrl-v followed by ctrl-m.

    – sjas

    Apr 8, 2015 at 8:25


  • 3

    Thanks sjas, You know this question is one of the weirdest of all time. 1008 votes for the answer which basically says nothing more than “vim does what you found. That’s because vim does what you found. Never forget that vim does what you found.” I’d hoped to find a shortlist of codes for interesting characters in the pattern, the replacement and the reason for the weirdness so its easy to remember and predict other similar weirdness. That would have got my vote.

    – codeshot

    Apr 8, 2015 at 11:32


  • @codeshot a list of ascii control characters might help you. See cs.tut.fi/~jkorpela/chars/c0.html for further reference. I will update my answer to include two links.

    – sjas

    Apr 8, 2015 at 14:11