Using cURL, jq, and declaration and for loop condition, I tried to download multiple files from a GitLab private repo, but it downloaded only one

Question

I learned from the following sources:

• curl -O: Download a file with curl on Linux / Unix command line
• jq: How to urlencode data for curl command?
• Multiple files and curl -J: download pdf files from using curl
• Condition for loop: Shell: how to use 2 variables with for condition and Unable to download data using curl for loop

Description of the script:

1. Variables, required by GitLab's Repository files API:

   branch="master"
   repo="my-dotfiles"
   private_token="XXY_wwwwwx-qQQQRRSSS"
   username="gusbemacbe"
   

2. I used a declaration for multiple files:

   declare -a context_dirs=(
     "home/.config/Code - Insiders/Preferences"
     "home/.config/Code - Insiders/languagepacks.json"
     "home/.config/Code - Insiders/rapid_render.json"
     "home/.config/Code - Insiders/storage.json"
   )
   

3. I used the condition for loop with jq to convert all files from the declaration context_dirs to encoded URLs:

   for urlencode in "${context_dirs[@]}"; do
     paths=$(jq -nr --arg v "$urlencode" '$v|@uri')
   done
   

4. I used the condition for loop to download with curl multiple files taken from paths converted by jq. It is important that I used -0 and -J to output the file name, and -H for "PRIVATE-TOKEN: $private_token":

   for file in "${paths[@]}"; do 
       curl -sLOJH "PRIVATE-TOKEN: $private_token" "https://gitlab.com/api/v4/projects/$username%2F$repo/repository/files/$file/raw?ref=$branch"
   done
   

Complete source code:

branch="master"
id="1911000X"
repo="my-dotfiles"
private_token="XXY_wwwwwx-qQQQRRSSS"
username="gusbemacbe"

declare -a context_dirs=(
  "home/.config/Code - Insiders/Preferences"
  "home/.config/Code - Insiders/languagepacks.json"
  "home/.config/Code - Insiders/rapid_render.json"
  "home/.config/Code - Insiders/storage.json"
)

for urlencode in "${context_dirs[@]}"; do
  paths=$(jq -nr --arg v "$urlencode" '$v|@uri')
done

for file in "${paths[@]}"; do 
    curl -sLOJH "PRIVATE-TOKEN: $private_token" "https://gitlab.com/api/v4/projects/$username%2F$repo/repository/files/$file/raw?ref=$branch"
done

But the two conditions for loop output only an encoded path and downloaded only a file.

Answer 1

The first loop overwrites the value of the variable paths in each iteration. Since you later expect this to be an array, make sure it is created properly:

paths=()
for urlencode in "${context_dirs[@]}"; do
  paths+=( "$(jq -nr --arg v "$urlencode" '$v|@uri')" )
done

Alternatively, combine the two loops:

for urlencode in "${context_dirs[@]}"; do
  file=$(jq -nr --arg v "$urlencode" '$v|@uri')
  curl -sLOJH "PRIVATE-TOKEN: $private_token" "https://gitlab.com/api/v4/projects/$username%2F$repo/repository/files/$file/raw?ref=$branch"
done