9

I'm experiencing something new with a simple curl request.

The command is a plain call of curl such as this:

curl http://example.com/endpoint/?arg1=val1&arg2=val2

The call is supposed to return a 500+ MB XML file (when ready, I will just -O it to save to a file), but there is nothing strange, really.

What is strange is that curl just print it's pid and the goes into background:

[1] 31562

After a while, the remote file starts to appear into the console and, being in background, I cannot kill it with Ctrl+C.

I never experienced something like this. If I try to download 1.9 GB file:

curl -O https://download.microsoft.com/download/0/A/F/0AFB5316-3062-494A-AB78-7FB0D4461357/7601.17514.101119-1850_Update_Sp_Wave1-GRMSP1.1_DVD.iso

It works normally!

Any idea?

Headers from a regular download:

curl -v -s https://download.microsoft.com/download/0/A/F/0AFB5316-3062-494A-AB78-7FB0D4461357/7601.17514.101119-1850_Update_Sp_Wave1-GRMSP1.1_DVD.iso 1> /dev/null
*   Trying 92.123.112.141:443...
* TCP_NODELAY set
* Connected to download.microsoft.com (92.123.112.141) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
... more HTTPS stuff ...
} [5 bytes data]
> GET /download/0/A/F/0AFB5316-3062-494A-AB78-7FB0D4461357/7601.17514.101119-1850_Update_Sp_Wave1-GRMSP1.1_DVD.iso HTTP/1.1
> Host: download.microsoft.com
> User-Agent: curl/7.65.3
> Accept: */*
> 
{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/octet-stream
< Accept-Ranges: bytes
< Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
< x-ms-blob-content-md5: mw6jWIuMir1+rZG9/mzj+A==
< Last-Modified: Sat, 13 Oct 2018 00:34:54 GMT
< ETag: "0x8D630A3B1CF6138"
< Content-Length: 2048196608
< Date: Mon, 10 Feb 2020 21:11:55 GMT
< Connection: keep-alive
< 
{ [16038 bytes data]
^C

My call:

*   Trying 192.168.0.200:80...
* TCP_NODELAY set
* Connected to example.com port 80 (#0)
> GET /endpoint/?arg1=val1&arg2=val2 HTTP/1.1
> Host: example.com
> User-Agent: curl/7.65.3
> Accept: */*
> 
^C

1 Answers1

20

That URL contains an unquoted & character. This means that your command would start

curl http://example.com/endpoint/?arg1=val1

in the background, and then set the shell variable arg2 to the value val2, since that's what the next bit of the URL means when the shell parses the line.

Use quotes around the URL:

curl 'http://example.com/endpoint/?arg1=val1&arg2=val2'

Even if the URL hadn't contained a &, the ? character is a globbing character (matches any single character in a filename globbing pattern). Leaving the URL unquoted would therefore cause the zsh shell with its default settings to complain about no matches found. It would also cause the bash shell to complain with no match if its failglob shell option was set.

Kusalananda
  • 320,670
  • 36
  • 633
  • 936