bash shell script errors with: syntax error near unexpected token `$'do\r''

Question

I have written the script below, but when I run it from the directory where it is (and also the files)

bash xmlimport-magento2.sh

I get this error message:

xmlimport-magento2.sh: line 7: syntax error near unexpected token $'do\r''

mlimport-magento2.sh: line 7: do

#!/bin/bash
if exists *.ZIP
then
    # disable autoindexing
    #php -f "$MAGEPATH"shell/indexer.php -- --mode-manual all
    for file in `ls *.ZIP`
    do
        # Backup zip file
        cp $file /backups
        # Extract XML files
        unzip -P web $file
        # Delete zip file
        rm -f $file
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*basic.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*basic.xml" ]
        then
            mv *_BASIC.XML BASIC.XML
            php -f bin/magento import:job:run 1
        fi
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*prices.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*prices.xml" ]
        then
            mv *_PRICES.XML PRICES.XML
            php -f bin/magento import:job:run 4
            php -f bin/magento import:job:run 2
        fi
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*stock.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*stockinfo.xml" ]
        then
            mv *_STOCKINFO.XML STOCKSTOCK.XML
            php -f bin/magento import:job:run 3
        fi
    done
    # verplaats .XML files naar backup
        cp -f $file ../backups
    # Delete XML files
        rm -f *.XML
fi

The script is probably in DOS/Windows format; see [this stackoverflow question](https://stackoverflow.com/questions/39527571/are-shell-scripts-sensitive-to-encoding-and-line-endings). — Gordon Davisson, May 31 '20 at 16:16
Your script is a DOS text file with DOS newlines (carriage-return+linefeed). Convert it with `dos2unix`. You additionally use `ls` needlessly when you could just loop safer over the expansion of a file glob (`for file in *.ZIP`). You have several other errors in your code which would make it fail to do what you want. I suggest rewriting it and testing each piece of code as you add it to the script. Also test what would happen with filenames containing spaces and globbing characters (`touch '* name.ZIP'`; with current code, having a file with this name causes data loss due to `rm -f $file`). — Kusalananda, May 31 '20 at 16:23
In addition to the previous comments, you can also check for syntax errors by pasting your code here: https://www.shellcheck.net/ — Nasir Riley, May 31 '20 at 16:37
`for file in $(ls *.ZIP)` - useless and error prone use of `ls`. — rexkogitans, May 31 '20 at 17:53
i change the script and rewrite it in vi editor and now it is working correct. However it is more easy if i can make script files also on my windows machine. Is there a editor for windows which writes script files in the correct format without the linefeeds? — marcel, May 31 '20 at 21:44

ilkkachu · Accepted Answer · 2020-05-31T17:05:16.900

First, there's the obvious issue with CRLF newlines, the \r in the error message being a tell-tale sign. See e.g. How to bulk convert all the file in a file system branch between Unix and Windows line break format? on how to fix that.

Then, you have

if exists *.ZIP
then

which might also be DOS-ism, at least I've not heard of a utility called exists.

Also here:

for file in `ls *.ZIP`

is a useless use of ls. What happens is that the shell expands the filename pattern *.ZIP, passes the results to ls, which the... just prints the same names out. Except that you run into issues with word splitting of the result.

Instead of those two, use:

shopt -s nullglob
for file in ./*.ZIP; do
    ...
done

First, the shell is quite capable of listing the matching filenames (it's what you did above with exists *.ZIP). Then, Bash's nullglob makes the pattern expand to nothing if there are no matching files. Otherwise you might get errors about a file literally called *.ZIP not existing. Prefixing the pattern with ./ makes some potential problems with filenames starting with dashes go away.

Also, you used unquoted expansions of $file all the way, i.e. rm -f $file, instead of rm -f "$file". That, too, leads to issues with word splitting, see:

Also,

if [ -f "*basic.xml" ]

this doesn't work, because the quotes prevent the pattern from being expanded to actual filenames. It would look for a file literally called *basic.xml. If you're sure there can only be one match, then if [ -f *basic.xml ] could do. If there are more, that would give an error because the test would get too many arguments.

Also, this will probably give an error, unless BASIC.XML is an existing directory.

mv *_BASIC.XML BASIC.XML

Okay, I think the suggestion to write the script in smaller parts, and test them individually (possibly with some debugging output sprinkled in), would be a good idea.

bash shell script errors with: syntax error near unexpected token `$'do\r''

1 Answers1