All files are not getting downloaded by SFTP script

Question

I have a sftp file script with below details,my server folder like BCP11,BCP12,BCP13,BCP14.....BCPXX. In which BCPDUMP folder have files.

Filename : file_sftp.sh

#!/usr/bin/expect
spawn sftp [email protected]
expect "[email protected]'s password:"
send "password\n"
expect "sftp>"
send "get *Backup/GetBackup/BCP*/*BCPDUMP/20150925/20150925_profile*\n"
expect "sftp>"
send "bye\n"

When i am executing this script i am getting only one file while the server have approx 12 files. i am getting below script output.

spawn sftp [email protected]  
Connecting to xx.xxx.x.xxx...  
Password:  
sftp> get *Backup/GetBackup/BCP*/*BCPDUMP/20150925/20150925_profile*  
Couldn't get handle: No such file or directory^M  
Couldn't get handle: No such file or directory^M  
Couldn't get handle: No such file or directory^M  
Fetching   /rsi/Backup/GetBackup/BCP10/BCPDUMP/20150925/20150925_profile_410.list.Z to 20150925_profile_410.list.Z
^M/rsi/Backup/GetBackup/BCP10/BCPDUMP/20150925/20150925_profile_410.list.Z 0%    0     0.0KB/s   --:-- ETA^M/rsi/Backup/GetBackup/BCP10/BCPDUMP/20150925/20150925_profile_410.list.Z                                                           66% 1152KB   1.1MB/s   00:00 ETA^M/rsi/Backup/GetBackup/BCP10/BCPDUMP/20150925/20150925_profile_410.list.Z                                                          100% 1730KB 864.9KB/s   00:02

I ran my script by below command and found my sftp session is closing by 20 secs. and in the mean time sftp downloading the file how much it can download with in 20 sec. please suggest how i can increase sftp session time. -bash-4.1$ date;./file_sftp.sh ;date — user3548033, Sep 25 '15 at 12:32
Are the files all called `20150925_profile_410.list.Z` and located in different directories? Or do they all have distinct names? If you download two files with the same name, the second one will overwrite the first. What does `ls *Backup/GetBackup/BCP*/*BCPDUMP/20150925/20150925_profile*` list? — Gilles 'SO- stop being evil', Sep 25 '15 at 22:40
All the files have different name.the naming convention is following. 20150925_profile_.list.Z. where XYZ is varying in the filenames. — user3548033, Sep 26 '15 at 14:50

score 1 · Answer 1 · answered Sep 25 '15 at 11:31

1

Since you are trying to retrieve directories you should use get -r. Try with that.

answered Sep 25 '15 at 11:31

YoMismo

4,005
1
15
31

Try and add a little more explanation. – X Tian Sep 25 '15 at 11:53
I ran my script by below command and found my sftp session is closing by 20 secs. and in the mean time sftp downloading the file how much it can download with in 20 sec. please suggest how i can increase sftp session time. -bash-4.1$ date;./file_sftp.sh ;date – user3548033 Sep 25 '15 at 14:57
Try running `sftp -vvv` so that you get the output of the communication and see what is going on. Try adding `ServerAliveInterval 120` to your `/etc/ssh/ssh_config` and see if it works. The server could have set that parameter though, so if the server is yours check `/etc/ssh/sshd_config` mind the D in sshD_config. – YoMismo Sep 28 '15 at 06:37

score 1 · Answer 2 · answered Sep 25 '15 at 13:25

Try to use this character for all ( * ) just once per line if you already know the names of the directories BCP* and *BCPDUMP.

If you think that it is not good idea typing the directories names in different lines you can use the ls command for listing all directories and get them into an array, after that you will download all available files for every array value which is different directory or different path.

score 1 · Accepted Answer · edited May 30 '16 at 12:49

1

Try this construct when command is time consuming:

send "command\r"
expect { 
    timeout { 
        puts "Running..." 
        exp_continue 
    } 
    "%PROMPT%" { 
        puts "Finished." 
    } 
}
send "next command\r"

On timeout you will be continuosly waiting with exp_continue command for %PROMPT%.

edited May 30 '16 at 12:49

techraf

5,831
10
33
51

answered Sep 28 '15 at 14:10

user3548033

593
2
12
25

score 0 · Answer 4 · answered Sep 26 '15 at 23:26

0

I don't get why you are using expect.

Use this code segment in your shell script ...

 ftp -i -n ftp.somehost.com  <<-EOF
   user somelogin somepassword
   mget *
   bye
   EOF

Note that each of the lines (user, mget, bye, EOF) start with a "tab" character.

Also note that if you are getting a specific set of files (provided it is greater than two files)

 mget file1 file2 file3 file4 ... filen

answered Sep 26 '15 at 23:26

Don W

1

1

Hi Don W, FTP is not a secure protocol for file transferring in production server environment. That's why i am using sftp protocol.for doing sftp,i required expect. – user3548033 Sep 27 '15 at 04:06

All files are not getting downloaded by SFTP script

4 Answers4