User Tools

Site Tools


asc:workbook_grid

This is an old revision of the document!


<<back

Getting data to ANL from the grid

Using dq2-get command

For small data samples, use the dq2-get command. dq2-get are set automatically on atlas16-18 if one uses the setup described on the setup page. Read workbook_introduction how to setup atlas software.

One can also setup the dq2-get client using the CVFMS file system

The DQ2 client is described in this HowTo and this manual with further information here

To setup dq2-get using CVFMS file system, use this script:

setup.sh
#!/bin/bash
export AVERS=17.0.4
export TEST_AREA=$HOME/testarea
export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase
source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh
asetup --release=$AVERS --testarea=$TEST_AREA

Put it in a file “setup.sh” and then “source setup.sh”.

Then type:

localSetupDQ2Client
voms-proxy-init -voms atlas -valid 96:00

Say “yes” and type your password.

The most often used dq2 commands are below dq2-xxx –help will show the options:

  • dq2-get -copies data from local storage or over the grid
  • dq2-ls -lists datasets matching a given pattern
  • dq2-put -allows the user to create user-defined datasets
  • dq2-ls -P -creates PoolFileCatalog.xml

Before starting using this, type:

voms-proxy-init -voms atlas 

or using the standard (proxy for 3 days):

grid-proxy-init -valid  96:00 

Before start using dq2-get, check the contents of your Grid proxy with the command:

voms-proxy-info -all 

Then check the location of your data:

dq2-ls -r -f user.CCC.0108173.topmix_Egamma.AOD.v2

Then pull the data:

dq2-get -a  -s <Data location> <Data set name to copy>

Note: if looking for containers (a collection of data sets that end with a /) *dq2-ls* will not work. You need to do *dq2-list-datasets-container*.

Using Pathena and pbook, prun commands

Do the setup as above then then type:

localSetupPandaClient

The general ATLAS instructions for pathena can be found here.

Setting up Grid Certificates

  • Request a DOE grid certificate using this site http://www.doegrids.org/pages/cert-request.html and list your Argonne collaborator when it asks for it (you must be using the same computer and browser you want to import the certificate on. Use IE or Firefox .
  • Once your request is approved, you'll be sent a link to retrieve your certificate. At the bottom of that page, there is a link to import your certificate
  • Import your certificate and then you'll want to export it to a .p12 file. Follow the instructions on this page http://www.doegrids.org/pages/cert-request.html . You'll be asked to create a password for the certificate. DO NOT FORGET IT!

In Fuirefox, just go to “Preferences”-“Edit”-“Advanced”-“View Certificates”. Click on most recent one (with expiration 1 year from the time when you have got email) and click “Backup”. Type file name with the extension pt12. You'll be asked to create a password for the certificate. DO NOT FORGET IT!

  • If your on a Linux machine, you'll want to create a directory called
    .globus

    if you don't have it already and you'll need to move your certificate files.

The first time you need to use your certificate, you'll have to run these commands

  openssl pkcs12 -in  YourCert.p12 -clcerts -nokeys -out $HOME/.globus/usercert.pem 
  openssl pkcs12 -in YourCert.p12 -nocerts -out $HOME/.globus/userkey.pem

You will be asked several times for the password. Use the same as that when you made backup copy.

Then do:

chmod 600  $HOME/.globus/userkey.pem
chmod 644  $HOME/.globus/usercert.pem

You can test that the certificate is working correctly by typing “grid-proxy-init”. The command “voms-proxy-init -voms atlas” will not work yet, since you have register again with VO ATLAS (even if you was already VO member - it seems DOE GRID re-registration resets your VO membership! (at least for me, S.C.)

If you are already a member of ATLAS, then you just need to join the Virtual Organization. The instructions can be found here http://www.usatlas.bnl.gov/twiki/bin/view/Support/HowToAtlasVO. Note that even if you were a member and you need just renew this, this is totally different step from DOE grid certificate renewal.

Follow the instructions on that page exactly. When you get to the set where it redirects you to https://lcg-voms.cern.ch:8443/vo/atlas/vomrs you should be using the same browser that you imported the DOE grid certificate or you'll need to import your certificate to the browser you want to use. You can check that it is setup correctly by looking in the bottom left corner and looking for your name.

You will be a candidate until they approve you and then you'll need to go back to https://lcg-voms.cern.ch:8443/vo/atlas/vomrs and complete the phase II registration. Again, follow the instructions on this page http://www.usatlas.bnl.gov/twiki/bin/view/Support/HowToAtlasVO .

Your grid will work only when you will get the email (usually takes 1-3 days):

Dear VO Member,
The membership status has been changed from New to Approved due to the following reason: Approved for your VOMRS atlas account with primary certificate:
DN: /DC=org/DC=doegrids/OU=People/CN=Name   XXXXX
CA: /DC=org/DC=DOEGrids/OU=Certificate Authorities/CN=DOEGrids CA 1
Please contact VO administrator if you have any questions. 

Using subscription

Read section using data subscription.

Sergei Chekanov 2013/03/18 09:21

asc/workbook_grid.1369940005.txt.gz · Last modified: 2013/10/23 15:27 (external edit)