Unix and Linux Stack Exchange
Unix and Linux Stack Exchange

Questions tagged [batch-jobs]

Grid engine batch job processing (qsub, qstat, …)

POSIX batch environment services specifies utilities for batch job processing:

  • qalter — alter a submitted job
  • qdel — cancel a submitted job
  • qhold — suspend a submitted job
  • qmove — move a job to a different queue
  • qmsg — send a message to a batch job's output
  • qrerun — terminate a running job and queue a new instance
  • qrls — release jobs from their hold
  • qselect — list submitted jobs matching certain criteria
  • qsig — send a signal to a running job
  • qstat — show the status of a job, queue or server
  • qsub — submit a job

Oracle Grid Engine is an open source implementation of this specification.

70 questions
114
votes
8 answers

Why is scp so slow and how to make it faster?

I'm trying to copy a batch of files with scp but it is very slow. This is an example with 10 files: $ time scp cap_* user@host:~/dir cap_20151023T113018_704979707.png 100% 413KB 413.2KB/s 00:00 cap_20151023T113019_999990226.png 100% …
laurent
  • 1,868
  • 3
  • 17
  • 22
43
votes
9 answers

How to download files and folders from Onedrive using wget?

How to use wget to download files from Onedrive? (and batch files and entire folders, if possible)
charles
  • 906
  • 2
  • 8
  • 8
19
votes
1 answer

How to batch resize all images in a folder (including subfolders)?

I have huge, 12GB, gallery on the server, full of images in various subfolders. Those files are too big and not used in full resolution. I need to resize all the images down to 820px wide (keeping proportions). So my question is - how can I create…
G-Gore
  • 191
  • 1
  • 1
  • 3
14
votes
2 answers

I don't know how to cancel job

I'm running some simulation by using the Linux cluster but I have some problem in my job. I want to cancel any job(ID : 750, 752, 753 name : gib) but I don't know how to cancel this job. Enclosed file is my job screen. How to kill a jobs by its job…
JINUNG
  • 141
  • 1
  • 1
  • 3
12
votes
1 answer

Is it possible to pause jobs submitted via qsub?

I am running my jobs on a small cluster. I submitted them via qsub. Now my labmates need resources more urgently than me, so I need to either kill my jobs or pause them, if possible. Is there a way of pausing my jobs and releasing the CPU, RAM,…
Sibbs Gambling
  • 1,646
  • 6
  • 20
  • 26
12
votes
2 answers

PBS equivalent of 'top' command: avoid running 'qstat' repeatedly

When I run several jobs on a head node, I like to monitor the progress using the command top. However, when I'm using PBS to run several jobs on a cluster, top will of course not show these jobs, and I have resorted to using 'qstat'. However the…
Nike Dattani
  • 237
  • 2
  • 14
6
votes
4 answers

Replacing counter in a filename for all files in a directory

After importing several 1000 Files from a camera onto a hard drive I realized, that the counter, used in the process of renaming the file - does not start from 0. This leads to file structure like this: My vacation 2018-05-03 2345.jpg My vacation…
karlitos
  • 171
  • 4
6
votes
2 answers

Batch Processing with one task at a time

I'm used to IBM i (AS/400) batch processing with flexible queueing configuration possibilities. I'm searching a similar facility for Linux Batch processing. Most important is that one job at a time being taken from the queue and executed. After…
PoC
  • 163
  • 7
6
votes
4 answers

atd, batch // Setting the load limiting factor

I am launching non interactive jobs using batch, and I would like to increase the load limiting factor in order to use all 8 of my cores. I am on Ubuntu 16.04 LTS. From what I understand, batch uses atd to do the jobs. Jobs start when the load…
ciliou
  • 81
  • 4
4
votes
0 answers

Distinguish between error and "success" in scanimage-batch

I'm running a little script with the scanimage batch-command on a remote server and would like to know if and how the scan has been done the batch. Therefore the script requires a proper "error"-description to handle the next steps. Yet scanimage…
Qohelet
  • 497
  • 1
  • 5
  • 18
4
votes
2 answers

How can I queue processes?

I have a surveillance camera and a program records videos when motion is detected. Basically, this program saves the video in a very heavy format. My solution was to call a script that converts the video using ffmpeg. The script is called each time…
Tomas
  • 1,014
  • 2
  • 12
  • 18
3
votes
0 answers

How to reliably detect stdin pipe in Bash in batch mode?

I need a way to test whether input to my Bash script has been piped to stdin when the script is run in batch mode. The method I use when I run the script interactively (the test [ -p /dev/stdin ]) fails in batch mode. Specifically, it fails in PBS…
Charlie Zender
  • 131
  • 4
3
votes
1 answer

How to set the path of a batch environment to find the location of the "qsub" command

I have a program, called "minvappmpi", which is parallelised and itself calls qsub as a subroutine. When I use the program on the cluster, it throws the error qsub: command not found If I want to submit another job using "qsub", then everything…
KR_Henninger
  • 61
  • 1
  • 5
3
votes
1 answer

How to run batch sas job in unix sas?

I have 5 SAS jobs that I need to run sequentially, one after the other. I typically type in nohup sas filename1.sas & in the command line to run and manually check for progress every few hours. If the 1st job is complete and no error, I then type…
SuperKing
  • 81
  • 1
  • 2
  • 5
3
votes
1 answer

filter document via aspell

I need to get a clean txt document and my first approach is to use aspell. The issue is I need it on batch, no interactive mode. Every txt file is piped to aspell and must be returned a new document with the non-dictionnary words deleted. I've found…
jomaweb
  • 521
  • 1
  • 3
  • 12
1
2 3 4 5