Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
hepsim:dev_hepsim [2019/04/23 01:30]
hepsim17 [Using Petrel from Globus]
hepsim:dev_hepsim [2020/06/26 21:24] (current)
hepsim17
Line 3: Line 3:
 [[:|<< back to HepSim manual]] [[:|<< back to HepSim manual]]
  
-====== Linking event storages======+====== Linking event storage ======
  
-Your files can be  published on [[http://atlaswww.hep.anl.gov/hepsim/|HepSim]] and you  can maintain your own HepSim repository with PROMC/SLCIO/ROOT files. This is useful if:+Monte Carlo files from any location can be  published on [[https://atlaswww.hep.anl.gov/hepsim/|HepSim]]
 +HepSim is not a storage, but a catalog of files stored in multiple URL locationsIt is your responsibility 
 +to maintain Monte Carlo data visible from HepSim.  HepSim tools allow:
  
-  * You want to share your own filesso they will be available from HepSim +  * to index files so they will be available from HepSim; 
-  * You want to make a  mirror of data on a server close to your location for faster download+  * to make mirrors of data on a server close to your location for faster download;
  
  
 If the dataset is new, you will see a new entry in the HepSim database. Your authorship will be preserved on the HepSim web page. If the dataset is new, you will see a new entry in the HepSim database. Your authorship will be preserved on the HepSim web page.
 If you have made a mirror of the existing data, your server will be added as a mirror for a given dataset. If you have made a mirror of the existing data, your server will be added as a mirror for a given dataset.
 +
 +If you created  Monte Carlo files, you can publish them using hepsim#petrel project, and link such files
 +to HepSim after you index them as explained below. In this case you do not need to maintain  a web server.
  
  
 ====== What do you need ====== ====== What do you need ======
  
-You need the following:+If you decide to publish Monte Carlo files (or make a mirror of the existing files), you need the following:
  
   * A web server that can hold your data (>1 TB for /var/www/html is recommended). It is advisable to create a separate partition with RAID10, say, /data, and link this directory to /var/www/html/   * A web server that can hold your data (>1 TB for /var/www/html is recommended). It is advisable to create a separate partition with RAID10, say, /data, and link this directory to /var/www/html/
Line 25: Line 30:
  
  
-You there is no web server, one can add files to the "petrel#hepsim" project using Globus. See the description latter.  +You there is no web server, one can add files to the "petrel#hepsim" project using [[https://www.globus.org/|Globus]]. See the description later.  
  
  
Line 49: Line 54:
  
 <code bash> <code bash>
-wget http://atlaswww.hep.anl.gov/hepsim/doc/lib/exe/fetch.php?media=hepsim:hepsim_web.tgz -O hepsim_web.tgz+wget https://atlaswww.hep.anl.gov/hepsim/doc/lib/exe/fetch.php?media=hepsim:hepsim_web.tgz -O hepsim_web.tgz
 tar -zvxf hepsim_web.tgz tar -zvxf hepsim_web.tgz
 cd hepsim_web/web_post cd hepsim_web/web_post
-wget http://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;+wget https://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;
 source hs-toolkit/setup.sh # this checks if Java is installed source hs-toolkit/setup.sh # this checks if Java is installed
 </code> </code>
Line 74: Line 79:
   * copy existing files from the HepSim and put them to the correct directory (will be mirror)   * copy existing files from the HepSim and put them to the correct directory (will be mirror)
  
-In both cases, make sure that Linux system administrator set correct permission for the directory "events", so you can copy the files. The directory for indexing should be inside your private directory outside the Web area.+In both cases, make sure that Linux system administrator set the correct permission for the directory "events", so you can copy the files. The directory for indexing should be inside your private directory outside the Web area.
  
 If you have indexed the files successfully, send a request to "[email protected]" (or [email protected]) to include your repository to HepSim database. Please include a short description of your files and your name (it will be shown on the web page). If you have indexed the files successfully, send a request to "[email protected]" (or [email protected]) to include your repository to HepSim database. Please include a short description of your files and your name (it will be shown on the web page).
Line 86: Line 91:
  
 <code bash> <code bash>
-wget http://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;+wget https://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;
 source hs-toolkit/setup.sh # setup HepSim programs source hs-toolkit/setup.sh # setup HepSim programs
 # now copy a dataset from URL to a new location # now copy a dataset from URL to a new location
-SOURCE=http://cepcgit.ihep.ac.cn:81/hepsim/events/ee/250gev/pythia6_higgs_gamgam_test+SOURCE=https://cepcgit.ihep.ac.cn:81/hepsim/events/ee/250gev/pythia6_higgs_gamgam_test
 OUTPUT_DIR=/var/www/html/ OUTPUT_DIR=/var/www/html/
 hs-mirror -i $SOURCE -o $OUTPUT_DIR hs-mirror -i $SOURCE -o $OUTPUT_DIR
Line 110: Line 115:
  
 <code bash> <code bash>
-wget http://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;+wget https://atlaswww.hep.anl.gov/hepsim/soft/hs-toolkit.tgz -O - | tar -xz;
 source hs-toolkit/setup.sh # setup HepSim programs source hs-toolkit/setup.sh # setup HepSim programs
 hs-index events/ep/pythi8_dis hs-index events/ep/pythi8_dis
Line 126: Line 131:
   * "metadata.txt" - with some information from ProMC files   * "metadata.txt" - with some information from ProMC files
   * "dirs.txt" - a list with subdirectories   * "dirs.txt" - a list with subdirectories
- 
- 
  
  
 ====== Using Petrel from Globus====== ====== Using Petrel from Globus======
  
-Monte Carlo files can be added to the "petrel#hepsim" project (https://press3.mcs.anl.gov/petrel/) so hey can be accessed via https. In this case there is no need to run a private web server. The description of Petrel can be found in https://press3.mcs.anl.gov/petrel/documentation/+Monte Carlo files can be added to the "petrel#hepsim" project (https://press3.mcs.anl.gov/petrel/on the 
-You will need a "globusaccount to be able to store files on Petrel.+grid [[https://www.globus.org/|Globus]], so hey can be accessed via the grid and https for public view. In this case there is no need to run a private web server. The description of Petrel can be found in https://press3.mcs.anl.gov/petrel/documentation/
 +You will need a [[https://www.globus.org/|Globus]] account to be able to store files on Petrel.
  
-You can run your personal Petrel projects (with enabled https), or you can be added to the existing petrel#hepsim project. +You can run your personal Petrel projects (with enabled https) with the files with HepSim-supported files, 
-The directory with Monte Carlo files should be properly indexed as described above, and then you can copy such directory to Petrel.+or you can add your files to the existing petrel#hepsim project. 
 +The directory with Monte Carlo files should be properly indexed as described above, and then you can copy such directory to Petrel Web-based storage using the transfer tools.
  
 A typical workflow is following: A typical workflow is following:
Line 144: Line 149:
   * Move the directory with the files to a petrel project (or ask to be included to petrel#hepsim)   * Move the directory with the files to a petrel project (or ask to be included to petrel#hepsim)
   * Send a URL location of your files to [email protected]   * Send a URL location of your files to [email protected]
- 
- 
- 
- 
  
  
Line 154: Line 155:
 When everything is done, this is what you should expect: When everything is done, this is what you should expect:
  
-  * You files will be visible on the main HepSim web page and on the current mirrors (takes ~ few days to propagate the changes)+  * Your files will be visible on the main HepSim web page and on the current mirrors (takes ~ few days to propagate the changes)
   * You can search your files etc. using the main HepSim web page   * You can search your files etc. using the main HepSim web page
   * You can search, list and download files using hs-toolkit commands (hs-find, hs-ls, hs-get etc)   * You can search, list and download files using hs-toolkit commands (hs-find, hs-ls, hs-get etc)