1

Axel is not resolving URLs that redirect to another URL that has the file as an attachment. As you see, I'm getting 404 when I try to download this URL. (original site is replaced with example.com)

~  ▶ axel https://eusa.example.com/zipstream/1581777.zip?tunnel=1&token=b7385bb62e3111e3ace1002481265109&storage=s09
[2] 767
[3] 768
Initializing download: https://eusa.example.com/zipstream/1581777.zip?tunnel=1
[1]   Exit 1                  axel https://eusa.example.com/zipstream/1581777.zip?tunnel=1
[3]+  Done                    token=b7385bb62e3111e3ace1002481265109
~  ▶ HTTP/1.1 404 Not Found

The website documentations describe URL redirect as following:

First response is always a redirect. Location of the file is in ‘Location’ header. The client must send a new GET request to the new location. Only headers are shown.

HTTP/1.1 302 FOUND
Server: nginx
Date: Thu, 09 Feb 2012 15:58:53 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Keep-Alive: timeout=60
Content-Length: 385
Location: http://s01.example.com/v2/files/26205116/download?oauth_token=4a63387841ce11e1a047001018321b64

goes to:

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 09 Feb 2012 15:58:53 GMT
Content-Type: application/x-rar
Content-Length: 3190802
Last-Modified: Fri, 16 Sep 2011 11:42:13 GMT
Connection: keep-alive
Keep-Alive: timeout=60
Content-Disposition: attachment; filename="file.rar"
Accept-Ranges: bytes

curl is capable of downloading the URL. But axel gives me 404. I want to have Axel's multi-connection feature. How can I use curl to just resolve the URL and give downloading task to axel?

slm
  • 363,520
  • 117
  • 767
  • 871
Mohsen
  • 2,495
  • 4
  • 25
  • 29

1 Answers1

3

Quoting issue?

Try wrapping the URL to axel in single quotes. Bash is attempting to parse the & in the param arguments in the URL as you telling it to background the command.

$ axel 'https://eusa.example.com/zipstream/1581777.zip?tunnel=1&token=b7385bb62e3111e3ace1002481265109&storage=s09'

Axel bug tracker

Going through the list of open bugs for Axel it looks like this issue has been identified previously and the author does not appear to be responding to the issue.

See these issues for more details:

Given the length of time that's passed with no movement on these issues I would consider this project dead. It may still be usable, but there is no one driving it.

This is not that unusual in the open source world and is actually what makes this licensing model unique, in the sense that you can pick up the source code, on a project in this state, and still continue to use it and maintain it yourself.

The source code for this project is easily downloadable via svn. The project's code repo:

Downloading & compiling

You can download a copy of the repository like so:

$ svn co svn://svn.debian.org/svn/axel/

On my Fedora 19 box the code configures and compiles perfectly so this project is actually in a good state, if one were to choose to modify the code base.

configure

$ ./configure
Configuration done:
  Internationalization enabled.
  Debugging disabled.
  Binary stripping enabled.

make

$ make
msgfmt -vo nl.mo nl.po
40 translated messages, 6 fuzzy translations, 4 untranslated messages.
msgfmt -vo de.mo de.po
46 translated messages, 4 fuzzy translations.
msgfmt -vo ru.mo ru.po
46 translated messages, 2 fuzzy translations, 2 untranslated messages.
msgfmt -vo zh_CN.mo zh_CN.po
42 translated messages, 6 fuzzy translations, 2 untranslated messages.
gcc -c axel.c -o axel.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c conf.c -o conf.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c conn.c -o conn.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c ftp.c -o ftp.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c http.c -o http.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c search.c -o search.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c tcp.c -o tcp.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc -c text.c -o text.o -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Os  -Wall
gcc *.o -o axel -pthread
strip axel

I was able to compile it and got a perfectly usable binary of axel.

$ ls -l | grep "axel$"
-rwxrwxr-x. 1 saml saml 35816 Dec 14 21:56 axel

Modifying the code base?

The section of code that I believe you'd want to focus on would be here in the file conn.c:

                conn_exec( conn );
                conn_disconnect( conn );
                /* Code 3xx == redirect                         */
                if( conn->http->status / 100 != 3 )
                        break;
                if( ( t = http_header( conn->http, "location:" ) ) == NULL )
                        return( 0 );
                sscanf( t, "%255s", s );
                if( strstr( s, "://" ) == NULL)
                {
                        sprintf( conn->http->headers, "%s%s",
                                conn_url( conn ), s );
                        strncpy( s, conn->http->headers, MAX_STRING );
                }

C/C++ is not my native "language" so I would have to begin by either running this code through gdb (aka. the GNU Debugger) or insert some printf statements around this section to see what's going on when axel accesses the URLs that you're having issue with.

slm
  • 363,520
  • 117
  • 767
  • 871