Table of Contents

<<back

Getting data to ANL from the grid

Using dq2-get command

For small data samples, use the dq2-get command. dq2-get are set automatically on atlas16-18 if one uses the setup described on the setup page. Read workbook_introduction how to setup atlas software.

One can also setup the dq2-get client using the CVFMS file system

The DQ2 client is described in this HowTo and this manual with further information here

To setup dq2-get using CVFMS file system, use this script:

setup.sh
#!/bin/bash
export ALRB_localConfigDir="/share/sl6/cvmfs/localConfig"
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
localSetupDQ2Client  --skipConfirm
voms-proxy-init -voms atlas -valid 96:00

Put it in a file “setup.sh” and then “source setup.sh”.

Say “yes” and type your password.

The most often used dq2 commands are below dq2-xxx –help will show the options:

In this example, we use the standard (proxy for 3 days): 

grid-proxy-init -valid  96:00

Before start using dq2-get, check the contents of your Grid proxy with the command: 

voms-proxy-info -all

If there is a problem run these commands to debug them: 

setupATLAS
diagnostics
gridCert

Then check the location of your data: 

dq2-ls -r -f user.CCC.0108173.topmix_Egamma.AOD.v2

Then pull the data: 

dq2-get -a  -s <Data location> <Data set name to copy>

Note: if looking for containers (a collection of data sets that end with a /) *dq2-ls* will not work. You need to do *dq2-list-datasets-container*.

Using Pathena commands

Do the setup as above then then type: 

localSetupPandaClient

The general ATLAS instructions for pathena can be found here.

Also look at: Using_PandaClient how to setup Panda. (attention: do not setup dq2-get together with panda!)

Putting files on the grid

(thanks to Taylor for checking)

Setup Atlas release. For SL6, source this code: 

#!/bin/bash
export AVERS=17.7.0
export TEST_AREA=$HOME/testarea
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
asetup --release=$AVERS --cmtconfig=x86_64-slc6-gcc46-opt --testarea=$TEST_AREA
localSetupDQ2Client  --skipConfirm
voms-proxy-init -voms atlas -valid 96:00
localSetupPandaClient --noAthenaCheck

Then put files (assumes MWT2_UC_SCRATCHDISK): 

dq2-put -d -L MWT2_UC_SCRATCHDISK  -s ./<folder of files> user.<username>.<custom name>

Create and register it before doing this:

You can also register it on the grid. 

dq2-register-dataset user.<name>.test1
dq2-register-location user.<name>.test1  MWT2_UC_SCRATCHDISK

Replicate data on the grid

Go to: https://rucio-ui.cern.ch/

Use your X509 certificate go login

(On top click) Data Transfer (R2D2) > Request new rules > click “List of DIDS” > copy your dataset list to the box and click Search > in step 2 use “BNL-OSG2_SCRATCHDISK” or something that you have permission to write > in step 3 you can change lifetime, I suggest to put 365 > then continue and finish the procedure and it will tell you that your new rule is established or something similar.

Using subscription

Read section using data subscription.

— Sergei Chekanov 2013/03/18 09:21