Unix and Linux Stack Exchange
Unix and Linux Stack Exchange

Questions tagged [cluster]

discussion related to cluster mechanisms.

clustering is a way to share ressources between two or more hosts.

Ressources are usually IPs, disks and processes (web server, database)

Failure of a node will bring ressource to another.

213 questions
114
votes
2 answers

How does tcp-keepalive work in ssh?

I am trying to code a shell-script that uses a ssh-connection for doing "heartbeats". I want to terminate the client- and server-side of that connection after a certain timeout (after the connection drops). What I found so far: TCPKeepAlive yes/no…
Nils
  • 18,202
  • 11
  • 46
  • 82
39
votes
2 answers

How do I distribute a large download over multiple computers?

I need to download a large file (1GB). I also have access to multiple computers running Linux, but each is limited to a 50kB/s download speed by an admin policy. How do I distribute downloading this file on several computers and merge them after all…
B Faley
  • 4,213
  • 11
  • 37
  • 48
34
votes
1 answer

How to submit a job to a specific node using Slurm's sbatch command?

Our nodes are named node001 ... node0xx in our cluster. I wonder, is it possible to submit a job to a specific node using Slurm's sbatch command? If so, can someone post an example code for that?
Amir
  • 949
  • 1
  • 7
  • 12
24
votes
4 answers

SLURM: Custom standard output name

When running a SLURM job using sbatch, slurm produces a standard output file which looks like slurm-102432.out (slurm-jobid.out). I would like to customise this to (yyyymmddhhmmss-jobid-jobname.txt). How do I go about doing this? Or more generally,…
mindlessgreen
  • 1,229
  • 4
  • 12
  • 21
17
votes
6 answers

NFS Automatic Fail-over or Load-balanced or Clustering?

I've been using NFS (and it is the only one i know well in Network Files) for a few years. But still can not able to find out .. How to make NFS Servers in Clustered Design (or Load-balanced?) Or at least to having it deployed on MORE-THEN-ONE…
夏期劇場
  • 1,571
  • 9
  • 22
  • 35
14
votes
3 answers

On Ubuntu Linux, is it normal for mandb to run continuously (apparently in the background)?

I have access to a Ubuntu Linux node at my institution. The nodes are shared among the group, but typically I am the only person who uses this particular node. I am running a calculation in parallel on all 8 CPUs on this node. My calculation runs,…
Andrew
  • 16,315
  • 34
  • 73
  • 77
13
votes
3 answers

Small Distributed Computing Cluster

I'm a high school student trying to build a linux cluster for a project (I have a bunch of decent computers slated for re-image this summer, so the tech department basically says as long as I don't physically break them I can do whatever. Anyway, I…
user6026
12
votes
3 answers

How to set up "poor man's" Linux desktop cluster?

There are several small Linux/Android devices on the market with 1 GHz ARM processor, 512 MByte RAM and cca. 4 Gbyte storage for $35-$40. If I buy 5 pcs, it's only around $200. Let's say, we have 5-10 pcs of them, and we've solved basic things, e.g.…
ern0
  • 575
  • 1
  • 5
  • 16
10
votes
5 answers

Directory sync with automatic peer discovery

I'm working with a cluster of Linux servers sitting behind the Amazon cloud autoscaling feature (documentation). There are files that accumulate on each server, they are cache files for the application. I am searching for an automated way to have…
user6394
9
votes
2 answers

How to build a personnal cluster?

I have a debian server running some services (mail, web, git, mysql...). But it is alone, meaning that if it breaks, I have to repair it to access my services. I'd like to put a second server so that if the first one dies the second one is…
Clement Roblot
  • 229
  • 1
  • 8
9
votes
1 answer

How do I download and launch a JNLP file in CentOS?

I'm using Amazon's GPU cluster to experiment with some HPC java code on CentOS. The Java application is in JNLP format and has a GUI. I've followed Amazon's getting started video, but I'm not sure how to use a web browser or otherwise download…
makerofthings7
  • 211
  • 1
  • 2
  • 6
7
votes
2 answers

Straightforward Linux Clustering

We have many unused PC machines and we would like to use them to set up educational lab for high performance computing applications. Which Linux distribution is the most convenient to set up and easy to manage in educational environment? I would be…
Mohammad Al-Turkistany
  • 173
  • 4
7
votes
1 answer

Running parallel bash jobs on a HPC cluster using GNU parallel

On an HPC cluster I am trying to run multiple bash scripts (permute2.sh) from 1 bash script using GNU parallel, however it doesn't complete every job. It randomly completes one job, while it is stuck doing the other. permute1.sh: PROCS=144…
tafelplankje
  • 303
  • 2
  • 6
7
votes
1 answer

Check if a remote machine is in a Veritas cluster

How can I verify that a remote machine is part of a Veritas cluster? I just created the following ssh command to verify if the remote machine is a VRTS cluster or a Linux availability Red Hat cluster LinuxTSR=198.23.12.5 ssh LinuxTSR "ls…
yael
  • 1,511
  • 11
  • 33
  • 57
6
votes
2 answers

Expanding only certain variables inside a heredoc

This is an extension of the question I asked . Passing directory from command line to shell script I have a script which writes another script using a heredoc. I need to be able to write unexpanded variables in the heredoc, so I use single quotes…
Ron
  • 1,027
  • 2
  • 12
  • 21
1
2 3
…
14 15