Categories
curl

Automatically add newline at end of curl response body

330

If the HTTP response body for a curl request doesn’t contain a trailing newline, I end up with this really annoying condition where the shell prompt is in the middle of the line, and escaping is messed up enough that when I put the last curl command on the screen, deleting characters from that curl command deletes the wrong characters.

For example:

[[email protected] ~]# curl jsonip.com
{"ip":"10.10.10.10","about":"/about"}[[email protected] ~]#

Is there a trick I can use to automatically add a newline at the end of a curl response, to get the prompt back on the left edge of the screen?

2

  • None of these answers work for me with whatever version of curl my cygwin install has; only wrapping the entire curl command in an echo statement does, e.g. echo "$(curl localhost:8001/api)", re this answer: unix.stackexchange.com/a/217611/110338

    – Alex Hall

    Dec 7, 2017 at 4:59


  • 1

    for json specifically, curl -s https://jsonip.com | jq

    Aug 1, 2020 at 8:28

559

From the man file:

To better allow script programmers to get to know about the progress of
curl, the -w/–write-out option was introduced. Using this, you can specify
what information from the previous transfer you want to extract.

To display the amount of bytes downloaded together with some text and an
ending newline:

curl -w 'We downloaded %{size_download} bytes\n' www.download.com

So try adding the following to your ~/.curlrc file:

-w "\n"

7

  • 74

    You can run the following command that does that for you: echo '-w "\n"' >> ~/.curlrc

    – tbraun

    Aug 9, 2016 at 9:20

  • 13

    I always add -w "\n" manually to every curl command and TIL ~/.curlrc exists!

    Aug 18, 2016 at 4:36

  • 2

    this may not work under git-bash on Windows; I added -w "\nKLJLJKLJ LJKLLKJ" to my ~/.curlrc and ran a curl and got neither a new line or the weird characters.

    – jcollum

    Aug 10, 2017 at 20:57


  • @jcollum Some Windows ports prefer _ over .. Chances are your curl is looking for ~/_curlrc and consequently echo '-w "\n"' >> ~/_curlrc will probably work. It does for me. btw: curl will complain over unescaped whitespace

    Nov 30, 2018 at 20:42


  • There’s value in using the command line with -w "\n" or appending ; echo over modifying your .curlrc. If you modify your curlrc, some scripts may break if they do not handle a newline at the end of the response data. I personally would stick with the additions on the command line over modifying the rc.

    – Jason

    Jan 28, 2020 at 21:52


129

Use this:

curl jsonip.com; echo 

If you need grouping to feed a pipe :

{ curl jsonip.com; echo; } | tee new_file_with_newline

OUTPUT

{"ip":"x.x.x.x","about":"/about"}

This is that simple 😉

(and not limited to curl command but all commands that not finish with a newline)

3

  • 3

    Is there any way to wrap this in a script or bash alias or anything similar? Typing “; echo” at the end of every line can get tedious, and I feel like this should be able to be automated somehow.

    – Matthew G

    Jan 15, 2013 at 0:05

  • On SO, it’s better to provide some search effort. Finding how to put this in an alias or script is obvious.

    Jan 15, 2013 at 6:44

  • 2

    Notice that adding the echo command clears the exit code from curl command. The -w “\n” option to curl suggested in the most popular answer keeps the curl exit code available for inspection.

    – bradoaks

    Oct 4, 2019 at 17:53

20

For more info as well as a clean new line after curl

~/.curlrc

-w "\nstatus=%{http_code} %{redirect_url} size=%{size_download} time=%{time_total} content-type=\"%{content_type}\"\n"

(More options are available here)

redirect_url will be blank if the request doesn’t get redirected or you use -L to follow the redirect.

Example output:

~ ➤  curl https://www.google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>302 Moved</TITLE></HEAD><BODY>
<H1>302 Moved</H1>
The document has moved
<A HREF="https://www.google.co.uk/?gfe_rd=cr&amp;ei=FW">here</A>.
</BODY></HTML>

status=302 https://www.google.co.uk/?gfe_rd=cr&ei=FW size=262 time=0.044209 content-type="text/html; charset=UTF-8"
~ ➤  

Edit, to make things more readable you can add ANSI colours to the -w line, it’s not that easy to write directly, but this script can generate a ~/.curlrc file with colours.

#!/usr/bin/env python3
from pathlib import Path
import click
chunks = [
    ('status=", "blue'),
    ('%{http_code} ', 'green'),
    ('%{redirect_url} ', 'green'),
    ('size=", "blue'),
    ('%{size_download} ', 'green'),
    ('time=", "blue'),
    ('%{time_total} ', 'green'),
    ('content-type=", "blue'),
    ('\\"%{content_type}\\"', 'green'),
]
content="-w "\\n"
for chunk, colour in chunks:
    content += click.style(chunk, fg=colour)
content += '\\n"\n'

path = (Path.home() / '.curlrc').resolve()
print('writing:\n{}to: {}'.format(content, path))
path.write_text(content)