30

I would like to check where a single URL redirects. An example of that could be a link from Google's search result page (where a click always goes through Google server).

Can I do that with curl?

Community
  • 1
syntagma
  • 12,091
  • 21
  • 57
  • 74

5 Answers5

26

There is an even easier way

curl -w "%{url_effective}\n" -I -L -s -S $URL -o /dev/null

it would print 

http://raspberrypi.stackexchange.com/questions/1508/how-do-i-access-the-distributions-name-on-the-command-line/1521

for URL

http://raspberrypi.stackexchange.com/a/1521/86
ismail
  • 500
  • 5
  • 7
  • 4
    That uses more time and bandwidth though, since you're downloading the second page as well. – unhammer May 06 '15 at 12:30
  • 2
    @unhammer You are right, updated my answer to do head requests only. – ismail Aug 07 '15 at 08:11
  • This has an issue where on websites with a lot of redirecting this won't work, it'll exit with "too many redirects". How do i make it STOP on the first one? – TheTechRobo the Nerd May 27 '21 at 19:28
  • Update: figured it out thanks to https://unix.stackexchange.com/a/515645/401349. i need to use redirect_url rather than url_effective and programmatically remove the error message (it's being used in a script). then i add `--max-redirs 1` (I am archiving shortURLs). – TheTechRobo the Nerd May 27 '21 at 20:02
21

Try this:

$ LOCATION=`curl -I http://raspberrypi.stackexchange.com/a/1521/86 | perl -n -e '/^Location: (.*)$/ && print "$1\n"'`
$ echo "$LOCATION"
/questions/1508/how-do-i-access-the-distributions-name-on-the-command-line/1521#1521

Google Redirects

Google redirect URLs are slightly different. They return a Javascript redirect, which could easily be processed, but why not process the original URL and for go curl all together?

$ URL="http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CFAQFjAA&url=http%3A%2F%2Fwww.raspberrypi.org%2F&ei=rv8oUODIIMvKswa4xoHQAg&usg=AFQjCNEBMoebclm0Gk0LCZIStJbF04U1cQ"
$ LOCATION=`echo "$URL" | perl -n -e '/url=([a-zA-Z0-9%\.]*)/ && print "$1\n"'`
$ echo "$LOCATION"
http%3A%2F%2Fwww.raspberrypi.org%2F
$ echo "$LOCATION" | perl -pe 's/%([0-9a-f]{2})/sprintf("%s", pack("H2",$1))/eig'
http://www.raspberrypi.org/

Reference

  1. For url decode...
Alex Chamberlain
  • 2,050
  • 3
  • 17
  • 26
  • How about special URLs with special characters (like Google's redirects), e.g.: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CFAQFjAA&url=http%3A%2F%2Fwww.raspberrypi.org%2F&ei=rv8oUODIIMvKswa4xoHQAg&usg=AFQjCNEBMoebclm0Gk0LCZIStJbF04U1cQ – syntagma Aug 13 '12 at 13:24
  • @REACHUS The redirect is encoded into the query string of the URL given. – Alex Chamberlain Aug 13 '12 at 13:28
  • @REACHUS No worries - you can probably combine the 2 perl statements. – Alex Chamberlain Aug 13 '12 at 14:12
  • If there are multiple redirects, this will get you the first one only! This will only show you the location, not check if that is a redirect too, not check if URL is available. You may be better off writing a script with PHP or any other language with a good HTTP request library or using other methods which check the target URL and which follow multiple redirects. – Sybille Peters Aug 04 '21 at 11:14
  • Add --no-progress-meter to curl to turn off the progress meter. – Sybille Peters Aug 04 '21 at 11:14
  • Also, next answer https://unix.stackexchange.com/a/157219/267431 follows multiple redirects. (more intuitive too to use the built in parameters in curl, IMHO). – Sybille Peters Aug 04 '21 at 11:28
9

curl can be configured to follow redirects and to print variables after completion. So what you ask can be achieved with the following command:

curl -Ls -w %{url_effective} -o /dev/null https://google.com

The man page explains the necessary parameters like that:

-L, --location          Follow redirects (H)
-s, --silent            Silent mode (don't output anything)
-w, --write-out FORMAT  Use output FORMAT after completion
-o, --output FILE       Write to FILE instead of stdout
schmijos
  • 427
  • 6
  • 12
5

or try this

curl -s -o /dev/null -I -w "HTTP_CODE: %{http_code}\nREDIRECT_URL: %{redirect_url}\n" http://raspberrypi.stackexchange.com/a/1521/86
user1146332
  • 2,214
  • 13
  • 14
  • How about special URLs with special characters (like Google's redirects), e.g.: http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CFAQFjAA&url=http%3A%2F%2Fwww.raspberrypi.org%2F&ei=rv8oUODIIMvKswa4xoHQAg&usg=AFQjCNEBMoebclm0Gk0LCZIStJbF04U1cQ – syntagma Aug 13 '12 at 13:25
  • just put the url in single quotes, so that your shell ignores the special characters in the link. But the link you have stated does not redirect to another uri, the status code of the response is 200 and not 3xx. The uri you look for is hidden in the uri itself respectively in the content of the response. For further examination you can study the header of the response with `curl -s -I 'http://yoururl'` and the content of the response with `curl -s 'http://yoururl'` (you will see that google use a simple javascript for the redirection). – user1146332 Aug 13 '12 at 14:30
2

The parameters -L (--location) and -I (--head) still doing unnecessary HEAD-request to the location-url.

If you are sure that you will have no more than one redirect, it is better to disable follow location and use a curl-variable %{redirect_url}.

This code do only one HEAD-request to the specified URL and takes redirect_url from location-header:

curl --head --silent --write-out "%{redirect_url}\n" --output /dev/null "https://goo.gl/QeJeQ4"
Geograph
  • 119
  • 2