I have a url which takes some time to load completely. But when I use curl to download the link as html, it download the html at the real time before loading page completely I mean complete execution of JavaScript. Please help me how to stop the curl until the page is loaded completely or start the download of the page after 1 minute.
Asked
Active
Viewed 1.2k times
2
-
2curl does not understand JavaScript. – Rui F Ribeiro Jun 16 '16 at 07:15
-
1@RuiFRibeiro: You should make that an answer. Although OP may not like it, it's the truth and a good answer which others will upvote. – Julie Pelletier Jun 16 '16 at 07:28
1 Answers
6
curl won't "wait" to interpret your Javascript because curl does not understand JavaScript; JavaScript is always done on the client side, and usually browsers that have JavaScript engines/know how to run JavaScript.
So curl will see it only as block as text, and leave it as it is.
Please do note this behaviour is not unique to curl e.g. in the Linux command line, wget, nc and lynx (and others) won't understand JavaScript too.
links seems to have a buggy JavaScript support (if compiled), however it is a text browser for interactive use.
If you do not mind scripting, have a look at phantomJS
phantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG.
Rui F Ribeiro
- 55,929
- 26
- 146
- 227
-
Please have a look at the related question, that while refers to a Javascript shell, delves around the same theme, http://unix.stackexchange.com/questions/58011/is-there-a-javascript-shell – Rui F Ribeiro Jun 16 '16 at 07:58
-
Thanks for the beautiful and useful info. So is there any other method to download a page without opening browser? – Kiran Kumar Jun 16 '16 at 08:58
-
it is in the answer. phantomJS, but it appears to be scriptable. I think you have a demo script in the page, that may (or not) suit your needs. – Rui F Ribeiro Jun 16 '16 at 09:26