Differences

This shows you the differences between two versions of the page.

--- lcrc:introduction [2019/01/22 15:14] – [Using Singularity] asc
+++ lcrc:introduction [2024/09/19 19:06] (current) – asc
@@ Line 10: / Line 10: @@
 The description here uses the bash shell. Please go to https://accounts.lcrc.anl.gov/account.php to change your shell.
-At this very moment (Jan 2018), this resource cannot replace the ATLAS cluster and have several features:
+At this very moment (Jan 2024), this resource cannot replace the ATLAS cluster and have several features:
   * LCRC resources are under maintenance on Monday (each week?)
@@ Line 27: / Line 27: @@
 </code>
 Each node has 72 CPUs and a lot of memory. After login, you will end up in a rather small "home" space which has a limit of 500 GB.
+You cannot login to these servers directly (from Aug 2024). First login to:
 <code>
+ssh -i $HOME/.ssh/YOURKEY [USER]@bebop.lcrc.anl.gov -X
+</code>
+then ssh to hep1/hep2:
+<code>
+ssh -i  $HOME/.ssh/YOURKEY  heplogin1.lcrc.anl.gov  -X
+</code>
+or
+<code>
+ssh -i  $HOME/.ssh/YOURKEY  heplogin2.lcrc.anl.gov  -X
+</code>
+<code>
 /home/[USER]
 </code>
@@ Line 108: / Line 128: @@
 <code bash>
-srun --pty -p  bdwall  -t 24:00:00 /bin/bash
+srun --pty -p  bdwall   -A condo -t 24:00:00 /bin/bash
 </code>
@@ Line 132: / Line 152: @@
 To run on non HEP nodes use partition bdwall with Account - ATLAS-HEP-group
+==== Using interactive jobs ====
+First, allocate a HEP node:
+<code bash>
+salloc -N 1 -p hepd -A condo  -t 00:30:00
+</code>
+This allocates it for 30 min, but you can allocate it up to 7 days.
+You cam also allocate it on bebob:
+<code bash>
+salloc -N 1 -p bdwall --account=ATLAS-HEP-group -t 00:30:00
+</code>
+This does not login you!
+Check what node did you allocate
+<code bash>
+squeue -u user
+</code>
+Now you know the node. Then login to bebob (first!) and then ssh to this node.
+Another method is to use
+<code bash>
+srun --pty -p  bdwall  --account=ATLAS-HEP-group -t 00:30:00  /bin/bash
+</code>
+=== Running long interactive jobs ===
+See more description in: https://www.lcrc.anl.gov/for-users/using-lcrc/running-jobs/running-jobs-on-bebop/
+You should be able to do for example:
+<code>
+-ssh bebop
+-screen
+-salloc -N 1 -p hepd -A condo -t 96:00:00
+-ssh <nodename>
+-Work on interactive job for x amount of time...
+-Disconnect from screen (different than exit, see the documentation)
+-Logout
+</code>
+<code>
+-Login to the same login node screen was started on
+-screen -ls
+-Connect to screen session
+-Continue where you left off (if they allocation is still active)
+</code>
+See below for more details:
+https://www.gnu.org/software/screen/
+https://www.hamvocke.com/blog/a-quick-and-easy-guide-to-tmux/
+====== CVMFS repositories ======
+Mounted CVMFS repositories on Bebop and Swing computing node.
+<code>
+/cvmfs/atlas.cern.ch
+/cvmfs/atlas-condb.cern.ch
+/cvmfs/grid.cern.ch
+/cvmfs/oasis.opensciencegrid.org
+/cvmfs/sft.cern.ch
+/cvmfs/geant4.cern.ch
+/cvmfs/spt.opensciencegrid.org
+/cvmfs/dune.opensciencegrid.org
+/cvmfs/larsoft.opensciencegrid.org
+/cvmfs/config-osg.opensciencegrid.org
+/cvmfs/fermilab.opensciencegrid.org
+/cvmfs/icarus.opensciencegrid.org
+/cvmfs/sbn.opensciencegrid.org
+/cvmfs/sw.hsf.org
+</code>
+Note, they are not mounted on login nodes
 ====== Using Singularity ======
@@ Line 143: / Line 248: @@
 docker pull atlas/analysisbase:21.2.51
 </code>
+Then make singularity image:
+<code bash>
+docker run -v /var/run/docker.sock:/var/run/docker.sock -v `pwd`:/output --privileged -t --rm singularityware/docker2singularity:v2.3 atlas/analysisbase:21.2.51
+</code>
+Currently, the image for AtlasBase 2.2.51 located here:
+<code>
+/soft/hep/atlas.cern.ch/repo/containers/images/singularity/atlas_analysisbase_21.2.51-2018-11-04-01795eabe66c.img
+</code>
+You can go inside this image as:
+<code>
+singularity exec /soft/hep/atlas.cern.ch/repo/containers/images/singularity/atlas_analysisbase_21.2.51-2018-11-04-01795eabe66c.img bash -l
+</code>
+====== Using Singularity for cvmfsexec ======
+One can also setup cvmf on any LCRC nodes as this:
+<code>
+source /soft/hep/CVMFSexec/setup.sh
+</code>
+Then check:
+<code>
+ls /cvmfs/
+</code>
+You will see the mounted directory (SL7):
+<code>
+atlas-condb.cern.ch/      atlas.cern.ch/  cvmfs-config.cern.ch/  sft-nightlies.cern.ch/  sw.hsf.org/
+atlas-nightlies.cern.ch/  cms.cern.ch/    projects.cern.ch/      sft.cern.ch/            unpacked.cern.ch/
+</code>
- --- //[[Sergei&Doug]] 2018/01/04 13:36//
+ --- //[[Sergei&Doug&Rui]] 2018/01/04 13:36//