1

For $url being the placeholder for an URL.

curl can retrieve headers for an url using 

#> curl -sI -L $url.

wget can do the same using: 

#> wget --server-response --spider $url

fetch is the third command line downloading tool that I am aware of.

Can it also get only headers?

Kusalananda
  • 320,670
  • 36
  • 633
  • 936
Frames Catherine White
  • 573
  • 3
  • 13
  • According to freeBSD man pages ( https://www.freebsd.org/cgi/man.cgi?fetch(1) ), this tool doesn't seem to allow you to get the headers only. –  Jan 16 '18 at 09:06
  • Does it really matter? You already have two tools performing the requested operation, do you need a third one? – Mukesh Sai Kumar Jan 16 '18 at 09:23
  • Because this code in question is for deployment on systems that are guaranteed to have at least one of `wget`, `curl` or `fetch`. – Frames Catherine White Jan 16 '18 at 09:26
  • Considering headers are always separated from body by a single empty line, are you able to pass the tool output through another tool that will drop everything after the headers (e.g. `sed`)? – B Layer Jan 16 '18 at 10:03
  • @BLayer no, because the total response (including the body) might be very large. (where as the headers will be very small) – Frames Catherine White Jan 16 '18 at 10:04
  • The download would be terminated after the headers are seen. – B Layer Jan 16 '18 at 10:05
  • really? Then yes. Other options are doing something like: https://unix.stackexchange.com/a/365365/18637 but substituting "HEAD" for "GET" – Frames Catherine White Jan 16 '18 at 10:07
  • Potential problem with HEAD is that not all servers support it but if you are working with a known, finite number of servers and they all have HEAD then by all means use it. – B Layer Jan 18 '18 at 03:42
  • By any chance do the servers that you have to use `fetch` with have `netcat` (sometimes `nc` or `ncat`) installed? It's a fairly common tool and if it's there I can tell you how to do HEAD with that. `fetch` is looking to be pretty limited. – B Layer Jan 18 '18 at 11:05

1 Answers1

1

If you have a way to fetch the full HTTP page, headers and body, then you could just use this:

fetch ... | sed '/^\r$/q'

Because headers are always separated from body by CRLF this will terminate after retrieving the headers.

B Layer
  • 5,107
  • 1
  • 17
  • 34
  • Thanks. So is it possible to tell `fetch` to use `HEAD` ? It seems like otherwise the first part is not connected by much to the second. – Frames Catherine White Jan 18 '18 at 05:27
  • You said "other options are...substituting HEAD for GET". When someone says something's an "option" that typically means it's possible. I was just trying to move important information out of the comments into an answer where other people are more likely to see it. If HEAD is not _really_ an option then I need to remove it from my answer. – B Layer Jan 18 '18 at 06:56
  • To my knowledge you can't trigger HEAD directly from the command line? the "..." in your quotes covers the part where is links to another post detailing a method where possibly HEAD can be used. This question is about `fetch` however, so if it is not usable with `fetch` then I'm not sure it is worth highlighting. I wouldn't know if it was usable by fetch (thus this question). I assume as the answerer you are more familiar with fetch than I am. – Frames Catherine White Jan 18 '18 at 09:01
  • I don't have access to fetch. I posted my answer as a hopefully helpful suggestion (based on a sound principle) that you could look into since no other answers have been offered. I can't tell from looking at the man page whether full HTTP message is returned. – B Layer Jan 18 '18 at 11:05