4

I am trying to download the contents of a web page using curl but it seems l am missing some content that l suspect to be generated by JavaScript. If the page runs on the browser l am able to get all the contents l require.

Mutai Mwiti
  • 141
  • 1
  • 1
  • 3
  • 1
    See http://superuser.com/questions/448514/command-line-browser-with-js-support – cas Nov 03 '15 at 08:36
  • 1
    Possible duplicate of [Automating web requests using curl?](http://unix.stackexchange.com/questions/11296/automating-web-requests-using-curl) – Gilles 'SO- stop being evil' Nov 03 '15 at 21:14
  • 3
    @Gilles I don't think it's a duplicate. This one here is more complex as it requires Javascript, which `curl` doesn't offer. On the other hand, I would agree it's a duplicate of the [SU question](http://superuser.com/questions/448514/command-line-browser-with-js-support) that [cas](http://unix.stackexchange.com/users/7696/cas) has suggested – roaima Nov 03 '15 at 22:37
  • @roaima Having Javascript was one of the points of that question as well. Both questions are asking for “curl” for the automation plus the trimmings of web browsers such as JS. – Gilles 'SO- stop being evil' Nov 03 '15 at 22:56
  • 1
    @Gilles I see nothing in your suggested duplicate that requires Javascript. (Someone did ask if it was required but there was no reply.) Furthermore the accepted answer offered `curl`, which [mwiti](http://unix.stackexchange.com/users/141397/mwiti) here is already ruling out as inadequate. – roaima Nov 03 '15 at 23:15

1 Answers1

2

You can use PhantomJS to do it.

(From the link, PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.)

roaima
  • 107,089
  • 14
  • 139
  • 261
Tam Nguyen
  • 31
  • 1