Site menu:
Announcements
Arcond
a Condor-based front-end for a PC farm
ArCond (ARgonne CONDor) is a package to facilitate running multiple jobs in parallel using a PC farm with local data storage, i.e. when data scattered over many PCs. The package was designed for a Tier-3 type of clusters consisting of a number of independent computer notes (PCs) with a local file storage on each node (i.e. disks), and a common NFS-mounted user (or share) directories. No any server computer is necessary.ArCond is a front-end of the Condor batch system. The main advantage of the ArCond is the possibility to run multiple jobs in parallel using data stored locally on each computing node, rather than using a single file storage on a NFS server (although this is also possible). This means that network is only used during job submission (retrieval), but not during data processing. This leads to a significant performance improvement. ArCond comes with data discovery tools, so it can locate any dataset on local file storage using a static database or build-it on-fly discovery mechanism during job submissions.
Using ArCond, any custom shell command (or a command sequence) defined in a submission script can be executed in parallel. For example, one can run multiple athena option files, perform user-specific tests, file merging etc. directly on the Linux boxes. Also, one can run any custom program defined in a shell script. In addition:
- ArCond has a perfect debugging, which allows to check and resubmit jobs for a particular node, or even run failed jobs on the same node where they fail.
- Combines and merge outputs.
- Contains tools for administrators which allow to monitor a small cluster via ssh tunneling, redistribute data uniformly between different cluster nodes, run any shell command on all cluster nodes.
- Infinitely scalable (since network is not used at runtime)