Data Distribution

Currently data is located in /data1 and /data2 on atlas50-65. If a user asks about data distribution, follow these steps:

  1. Copy these 2 scripts to your directory from
    /users/chakanau/public/Arcond/data_distribute
    • hosts.py - keeps host names
    • arc_sync - the actual script to sync data to remote hosts
  2. Look at “arc_sync”. Here you will need to specify 2 variables:
    OriginalLocation - the original location of the data
    USER - your user name

    You will also need to specify the directory structure for this dataset on the farm. For example:

    SetLocation="/data2"
    SetName="mc11_7TeV"
    SetType="113210.HerwigppJetsJ6.merge.NTUP_JETMET"
    SetVersion="e835_s1309_s1300_r3043_r2993_p839"
  3. Run the script. You may need to change the permisions on the remote host to be abale to copy the data.
  4. To remove the data, use the “arc_rm” command (Be sure to specify the location of the data on the farm!).