downloading a file behind a link

Question

I want to write a script to download the WEbStorm IDE from Jetbrains and then do further stuffs. But I am unable to download the tar ball.

I am following this, but it doesnt solve my problem. downloading files using wget

The webpage where I want to download the file from is https://www.jetbrains.com/webstorm/download/download-thanks.html?platform=linux

It looks like it is quite simple, but I just get a plain html page all the time when I do a wget. Doing it over the browser address bar or addons like DownThemAll is completely fine.

Following is my command

wget 'https://www.jetbrains.com/webstorm/download/download-thanks.html?platform=linux' -O WebStorm.tar.gz

I can download the file from here:

wget https://download.jetbrains.com/webstorm/WebStorm-2017.2.4.tar.gz

but I could only find this URL after a little bit of digging. Is there a way to simply get the download file url by parsing the Download button?

Is it also possible to do via curl?

Answer 1

This link should work for latest Linux version of WebStorm: https://data.services.jetbrains.com/products/download?code=WS&platform=linux

You can get this address by calling page you wanted to get in a first place and searching for direct link:

      $ curl 'https://www.jetbrains.com/webstorm/download/download-thanks.html?platform=linux' | grep "direct link"
      <p class="sub-title no-margin-bottom">Your download should start shortly. If it doesn't, please use the <a id="download-link" href="//data.services.jetbrains.com/products/download?code=WS" data-release-download-link="">direct link</a>.</p>

Just add &platfor=your_platform and you get latest version

Answer 2

I wrote a short script that will install any Linux-compatible JetBrains IDE, including WebStorm. You can find more details here.

#!/bin/sh

echo "Installing IntelliJ IDEA..."

# We need root to install
[ $(id -u) != "0" ] && exec sudo "$0" "$@"

# Attempt to install a JDK
# apt-get install openjdk-8-jdk
# add-apt-repository ppa:webupd8team/java && apt-get update && apt-get install oracle-java8-installer

# Prompt for edition
while true; do
    read -p "Enter 'U' for Ultimate or 'C' for Community: " ed 
    case $ed in
        [Uu]* ) ed=U; break;;
        [Cc]* ) ed=C; break;;
    esac
done

# Fetch the most recent version
VERSION=$(wget "https://www.jetbrains.com/intellij-repository/releases" -qO- | grep -P -o -m 1 "(?<=https://www.jetbrains.com/intellij-repository/releases/com/jetbrains/intellij/idea/BUILD/)[^/]+(?=/)")

# Prepend base URL for download
URL="https://download.jetbrains.com/idea/ideaI$ed-$VERSION.tar.gz"

echo $URL

# Truncate filename
FILE=$(basename ${URL})

# Set download directory
DEST=~/Downloads/$FILE

echo "Downloading idea-I$ed-$VERSION to $DEST..."

# Download binary
wget -cO ${DEST} ${URL} --read-timeout=5 --tries=0

echo "Download complete!"

# Set directory name
DIR="/opt/idea-I$ed-$VERSION"

echo "Installing to $DIR"

# Untar file
if mkdir ${DIR}; then
    tar -xzf ${DEST} -C ${DIR} --strip-components=1
fi

# Grab executable folder
BIN="$DIR/bin"

# Add permissions to install directory
chmod -R +rwx ${DIR}

# Set desktop shortcut path
DESK=/usr/share/applications/IDEA.desktop

# Add desktop shortcut
echo "[Desktop Entry]\nEncoding=UTF-8\nName=IntelliJ IDEA\nComment=IntelliJ IDEA\nExec=${BIN}/idea.sh\nIcon=${BIN}/idea.png\nTerminal=false\nStartupNotify=true\nType=Application" -e > ${DESK}

# Create symlink entry
ln -sf ${BIN}/idea.sh /usr/local/bin/idea

echo "Done."