ANL ASC

ATLAS analysis support center at ANL

Site menu:

• Our mission
• Getting an account
• Working at ASC
• ASC Computing Workbook
• Tier3 Setup and Related
• Meetings
• Useful links
• Getting to ANL ASC
• While at ANL ASC
• Calendar
• Conf. Rm. reservations
• Contact

Arcond

a Condor-based front-end for a PC farm

ArCond (ARgonne CONDor) is a package to facilitate running multiple jobs in parallel using a PC farm with local data storage, i.e. when data scattered over many PCs. The package was designed for a Tier-3 type of clusters consisting of a number of independent computer notes (PCs) with a local file storage on each node (i.e. disks), and a common NFS-mounted user (or share) directories. No any server computer is necessary.

ArCond is a front-end of the Condor batch system. The main advantage of the ArCond is the possibility to run multiple jobs in parallel using data stored locally on each computing node, rather than using a single file storage on a NFS server (although this is also possible). This means that network is only used during job submission (retrieval), but not during data processing. This leads to a significant performance improvement. ArCond comes with data discovery tools, so it can locate any dataset on local file storage using a static database or build-it on-fly discovery mechanism during job submissions.

Using ArCond, any custom shell command (or a command sequence) defined in a submission script can be executed in parallel. For example, one can run multiple athena option files, perform user-specific tests, file merging etc. directly on the Linux boxes. Also, one can run any custom program defined in a shell script. In addition:

• ArCond has a perfect debugging, which allows to check and resubmit jobs for a particular node, or even run failed jobs on the same node where they fail.
• Combines and merge outputs.
• Contains tools for administrators which allow to monitor a small cluster via ssh tunneling, redistribute data uniformly between different cluster nodes, run any shell command on all cluster nodes.
• Infinitely scalable (since network is not used at runtime)

Presently, the ArCond program is used at ANL to process terabytes of data on a daily basis, to generate Monte Carlo models, NLO QCD calculations and more. All of this is done using relatively small Tier3 resources (20%FTE, a few computer boxes).

Further links:

For T3g users and admins:

• ArCond releases and download
• ArCond Manual
• ArCond recent talk

Copyright © 2008 Sergei Chekanov and Rik Yoshida | Design by Andreas Viklund | xNavigation