lcrc:introduction
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
lcrc:introduction [2018/01/09 19:06] – asc | lcrc:introduction [2023/04/27 19:18] – asc | ||
---|---|---|---|
Line 4: | Line 4: | ||
https:// | https:// | ||
The users are usually placed into the " | The users are usually placed into the " | ||
+ | |||
+ | Please also look at the description https:// | ||
+ | |||
The description here uses the bash shell. Please go to https:// | The description here uses the bash shell. Please go to https:// | ||
Line 23: | Line 26: | ||
heplogin2.lcrc.anl.gov | heplogin2.lcrc.anl.gov | ||
</ | </ | ||
- | Each node has 72 CPUs and a lot of memory. After login, you will end up in a rather small " | + | Each node has 72 CPUs and a lot of memory. After login, you will end up in a rather small " |
< | < | ||
- | /blues/gpfs/ | + | / |
</ | </ | ||
- | Apparently, LCRC cannot spend more than 50$/per users home, since they too busy with leadership in computing. But you have other places | + | You can use this location |
< | < | ||
/ | / | ||
</ | </ | ||
- | |||
- | You can go to your " | ||
- | |||
- | <code bash> | ||
- | export TRUEHOME=/ | ||
- | echo " | ||
- | cd $TRUEHOME | ||
- | export HOME=$TRUEHOME | ||
- | </ | ||
- | in the file called " | ||
Line 115: | Line 108: | ||
<code bash> | <code bash> | ||
- | srun --pty -p bdwall | + | srun --pty -p bdwall |
</ | </ | ||
It will allocate a new node (in bash) for 24h. It uses Xeon(R) CPU E5-2695 v4 @ 2.10GHz (36 CPUs per node). | It will allocate a new node (in bash) for 24h. It uses Xeon(R) CPU E5-2695 v4 @ 2.10GHz (36 CPUs per node). | ||
+ | |||
+ | When you use bdwall partition, your jobs will accounted against default CPU allocations (100k per 4 months). Therefore, when possible, please use " | ||
+ | See the next section. | ||
+ | |||
+ | ===== Running Batch job on HEP resources ===== | ||
+ | |||
+ | <code bash> | ||
+ | srun --pty -p hepd -t 24:00:00 /bin/bash | ||
+ | module load StdEnv | ||
+ | </ | ||
+ | |||
+ | Then you can setup root etc as " | ||
+ | |||
+ | SLURM is used as the batch system. | ||
+ | Please see this page for details on how to use SLURM on LCRC http:// | ||
+ | |||
+ | The partion for the HEP nodes is hepd | ||
+ | |||
+ | To run on non HEP nodes use partition bdwall with Account - ATLAS-HEP-group | ||
+ | |||
+ | |||
+ | ==== Using interactive jobs ==== | ||
+ | |||
+ | |||
+ | First, allocate a HEP node: | ||
+ | |||
+ | <code bash> | ||
+ | salloc -N 1 -p hepd -A condo -t 00:30:00 | ||
+ | </ | ||
+ | |||
+ | This allocates it for 30 min, but you can allocate it up to 7 days. | ||
+ | You cam also allocate it on bebob: | ||
+ | |||
+ | <code bash> | ||
+ | salloc -N 1 -p bdwall --account=ATLAS-HEP-group -t 00:30:00 | ||
+ | </ | ||
+ | |||
+ | This does not login you! | ||
+ | Check what node did you allocate | ||
+ | |||
+ | <code bash> | ||
+ | squeue -u user | ||
+ | </ | ||
+ | |||
+ | |||
+ | Now you know the node. Then login to bebob (first!) and then ssh to this node. | ||
+ | |||
+ | Another method is to use | ||
+ | |||
+ | <code bash> | ||
+ | srun --pty -p bdwall | ||
+ | </ | ||
+ | |||
+ | |||
+ | === Running long interactive jobs === | ||
+ | |||
+ | See more description in: https:// | ||
+ | |||
+ | You should be able to do for example: | ||
+ | |||
+ | < | ||
+ | -ssh bebop | ||
+ | -screen | ||
+ | -salloc -N 1 -p hepd -A condo -t 96:00:00 | ||
+ | -ssh < | ||
+ | -Work on interactive job for x amount of time... | ||
+ | -Disconnect from screen (different than exit, see the documentation) | ||
+ | -Logout | ||
+ | </ | ||
+ | |||
+ | < | ||
+ | -Login to the same login node screen was started on | ||
+ | -screen -ls | ||
+ | -Connect to screen session | ||
+ | -Continue where you left off (if they allocation is still active) | ||
+ | </ | ||
+ | |||
+ | See below for more details: | ||
+ | |||
+ | https:// | ||
+ | |||
+ | https:// | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ====== Using Singularity ====== | ||
+ | |||
+ | |||
+ | To run jobs on all LCRC resources using ATLAS analysis base requires Docker/ | ||
+ | | ||
+ | |||
+ | Here are the suggested steps for 21.2.51 release. | ||
+ | |||
+ | < | ||
+ | docker pull atlas/ | ||
+ | </ | ||
+ | |||
+ | Then make singularity image: | ||
+ | <code bash> | ||
+ | docker run -v / | ||
+ | </ | ||
+ | |||
+ | |||
+ | Currently, the image for AtlasBase 2.2.51 located here: | ||
+ | |||
+ | < | ||
+ | / | ||
+ | </ | ||
+ | |||
+ | You can go inside this image as: | ||
+ | |||
+ | < | ||
+ | singularity exec / | ||
+ | </ | ||
+ | |||
+ | ====== Using Singularity for cvmfsexec ====== | ||
+ | |||
+ | One can also setup cvmf on any LCRC nodes as this: | ||
+ | < | ||
+ | source / | ||
+ | </ | ||
+ | |||
+ | Then check: | ||
+ | < | ||
+ | ls /cvmfs/ | ||
+ | </ | ||
+ | |||
+ | You will see the mounted directory (SL7): | ||
+ | < | ||
+ | atlas-condb.cern.ch/ | ||
+ | atlas-nightlies.cern.ch/ | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
--- // | --- // |
lcrc/introduction.txt · Last modified: 2023/06/15 14:05 by rwang