Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
hepsim:usage_download [2017/07/17 14:45]
hepsim17 [File validation]
hepsim:usage_download [2020/05/21 21:45] (current)
hepsim17
Line 6: Line 6:
  
 ====== ​ EVGEN events ====== ====== ​ EVGEN events ======
-Truth-level (EVGEN) data are stored in a platform-independent format called [[http://​atlaswww.hep.anl.gov/​asc/​promc/​ | ProMC]] that allows very effective compression using a variable-byte encoding. ​ This data format is+Truth-level (EVGEN) data are stored in a platform-independent format called [[https://​atlaswww.hep.anl.gov/​asc/​promc/​ | ProMC]] that allows very effective compression using a variable-byte encoding. ​ This data format is
 supported by popular programming languages (C++, Java, Python) on major  operating system (Windows, Mac, Linux, etc.). supported by popular programming languages (C++, Java, Python) on major  operating system (Windows, Mac, Linux, etc.).
 Open access to such files via the http protocol is the central element ​ in the design of HepSim, since Open access to such files via the http protocol is the central element ​ in the design of HepSim, since
Line 20: Line 20:
  
 <code bash> <code bash>
-wget http://​atlaswww.hep.anl.gov/​hepsim/​soft/​hs-toolkit.tgz -O - | tar -xz+wget https://​atlaswww.hep.anl.gov/​hepsim/​soft/​hs-toolkit.tgz -O - | tar -xz
 source hs-toolkit/​setup.sh source hs-toolkit/​setup.sh
 </​code>​ </​code>​
Line 27: Line 27:
  
 <code bash> <code bash>
-curl http://​atlaswww.hep.anl.gov/​hepsim/​soft/​hs-toolkit.tgz | tar -xz+curl https://​atlaswww.hep.anl.gov/​hepsim/​soft/​hs-toolkit.tgz | tar -xz
 source hs-toolkit/​setup.sh source hs-toolkit/​setup.sh
 </​code>​ </​code>​
Line 54: Line 54:
  
 <code bash> <code bash>
-hs-view ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc+hs-view ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc
 </​code>​ </​code>​
 Here we have looked at one file of the [[http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=141 | Pythia8 (QCD) sample]]. ​ Here we have looked at one file of the [[http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=141 | Pythia8 (QCD) sample]]. ​
Line 61: Line 61:
  
 <code bash> <code bash>
-wget http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc+wget https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc
 hs-view tev14_mg5_Httbar_001.promc hs-view tev14_mg5_Httbar_001.promc
 </​code>​ </​code>​
 after you downloaded it. after you downloaded it.
  
-On Windows, download [[http://​atlaswww.hep.anl.gov/​asc/​hepsim/​hepsim.jar| hepsim.jar]] and click the "​hepsim.jar"​ file. Then  open the ProMC file using the "​File"​ menu. +On Windows, download [[https://​atlaswww.hep.anl.gov/​asc/​hepsim/​hepsim.jar| hepsim.jar]] and click the "​hepsim.jar"​ file. Then  open the ProMC file using the "​File"​ menu. 
 You will see a pop-up GUI browser which displays the MC record. You can search for a given particle name, view data layouts and log files using the [Menu]: You will see a pop-up GUI browser which displays the MC record. You can search for a given particle name, view data layouts and log files using the [Menu]:
  
Line 75: Line 75:
 This works for full parton-shower simulations with detailed information on particles. This works for full parton-shower simulations with detailed information on particles.
 Unlike the usual parton shower Monte Carlo, this  browser has a detailed information on event weights, PDF uncertainties and scale uncertainties (in some cases). The browser can show 4-momenta of each event as well as the total cross sections (for NLO, you need to read all events to get an accurate cross section). Look at the [[https://​atlaswww.hep.anl.gov/​asc/​promc/​| ProMC file]] description. Unlike the usual parton shower Monte Carlo, this  browser has a detailed information on event weights, PDF uncertainties and scale uncertainties (in some cases). The browser can show 4-momenta of each event as well as the total cross sections (for NLO, you need to read all events to get an accurate cross section). Look at the [[https://​atlaswww.hep.anl.gov/​asc/​promc/​| ProMC file]] description.
- 
- 
  
 ====== ​ File validation ====== ====== ​ File validation ======
Line 83: Line 81:
  
 <code bash> <code bash>
-hs-info ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc+hs-info ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc
 </​code>​ </​code>​
  
Line 103: Line 101:
 All entries are self-explanatory. Varint units - values used to multiply energy (momenta) to convert to variable-byte integers. All entries are self-explanatory. Varint units - values used to multiply energy (momenta) to convert to variable-byte integers.
 The "​E:​100000"​ means that all px, py, pz, e, mass values are multiplied by 100000, while all distances (x,y,z,t) are multiplied by  1000. The "​E:​100000"​ means that all px, py, pz, e, mass values are multiplied by 100000, while all distances (x,y,z,t) are multiplied by  1000.
-See the [[http://​atlaswww.hep.anl.gov/​asc/​promc/​ | ProMC archive format]].+See the [[https://​atlaswww.hep.anl.gov/​asc/​promc/​ | ProMC archive format]].
  
 One can look at separate events using the above command after passing an integer argument that specifies One can look at separate events using the above command after passing an integer argument that specifies
Line 109: Line 107:
  
 <code bash> <code bash>
-hs-info ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc 100+hs-info ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​14tev/​mg5_httbar/​tev14_mg5_Httbar_001.promc 100
 </​code>​ </​code>​
  
Line 115: Line 113:
  
 Let us show how to find all files associated with a given Monte Carlo event sample. Let us show how to find all files associated with a given Monte Carlo event sample.
-Go to [[http://​atlaswww.hep.anl.gov/​hepsim/​ | HepSim database]]. Look at the links "​Files"​. It list the available files.+Go to [[https://​atlaswww.hep.anl.gov/​hepsim/​ | HepSim database]]. Look at the links "​Files"​. It list the available files.
 Then find the files as:                        Then find the files as:                       
 <code bash> <code bash>
Line 123: Line 121:
 This command shows a table with file names and their sizes. This command shows a table with file names and their sizes.
  
-Here is an example illustrating how to list all files from the [[http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2|Higgs to ttbar]] ​               ​+Here is an example illustrating how to list all files from the [[https://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2|Higgs to ttbar]] ​               ​
 Monte Carlo sample: Monte Carlo sample:
  
Line 145: Line 143:
  
 <code bash> <code bash>
-hs-ls http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2+hs-ls https://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2
 </​code>​ </​code>​
  
Line 151: Line 149:
  
 <code bash> <code bash>
-hs-ls http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5/​+hs-ls https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5/​
 </​code>​ </​code>​
  
Line 222: Line 220:
 hs-get [name] [OUTPUT_DIR] hs-get [name] [OUTPUT_DIR]
 </​code>​ </​code>​
-where [name] is either the name of the dataset, or the URL of Info page [[http://​atlaswww.hep.anl.gov/​hepsim/​ | HepSim repository]],​ or a direct URL pointing to the locations of ProMC files.+where [name] is either the name of the dataset, or the URL of Info page [[https://​atlaswww.hep.anl.gov/​hepsim/​ | HepSim repository]],​ or a direct URL pointing to the locations of ProMC files.
 This example downloads dataset "​tev100pp_higgs_ttbar_mg5"​ to the directory "​data":​ This example downloads dataset "​tev100pp_higgs_ttbar_mg5"​ to the directory "​data":​
 <code bash> <code bash>
Line 231: Line 229:
 Alternatively,​ this example downloads files using the URL of the Info page: Alternatively,​ this example downloads files using the URL of the Info page:
 <code bash> <code bash>
-hs-get ​http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2 data+hs-get ​https://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2 data
 </​code>​ </​code>​
 Or, if you know the download URL with the file locations, use this command: Or, if you know the download URL with the file locations, use this command:
 <code bash> <code bash>
-hs-get ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5 data+hs-get ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5 data
 </​code>​ </​code>​
 All these examples will download all files from the "​tev100pp_higgs_ttbar_mg5"​ event sample. ​ All these examples will download all files from the "​tev100pp_higgs_ttbar_mg5"​ event sample. ​
Line 247: Line 245:
 This example shows how to download 10 files using 3  threads: This example shows how to download 10 files using 3  threads:
 <code bash> <code bash>
-hs-get ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5 higgs_ttbar_mg5 3 10+hs-get ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​higgs_ttbar_mg5 higgs_ttbar_mg5 3 10
 </​code>​ </​code>​
  
Line 255: Line 253:
 Using the URL of the info page: Using the URL of the info page:
 <code bash> <code bash>
-hs-get ​http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2 ​ data 1 5+hs-get ​https://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=2 ​ data 1 5
 </​code>​ </​code>​
 or, when using the dataset name given on the info page: or, when using the dataset name given on the info page:
Line 287: Line 285:
 Reconstructed files are stored inside the directory "​rfastNNN"​ (fast simulation) or "​rfullNNN"​ (full simulation),​ Reconstructed files are stored inside the directory "​rfastNNN"​ (fast simulation) or "​rfullNNN"​ (full simulation),​
 where "​NNN"​ is the version number. For example, where "​NNN"​ is the version number. For example,
-[[http://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=15|tev100pp_ttbar_mg5]] sample includes the link "​rfast001"​ (Delphes+[[https://​atlaswww.hep.anl.gov/​hepsim/​info.php?​item=15|tev100pp_ttbar_mg5]] sample includes the link "​rfast001"​ (Delphes
 fast simulation, version 001). To download the reconstructed events for the reconstruction tag "​rfast001",​ use this syntax: fast simulation, version 001). To download the reconstructed events for the reconstruction tag "​rfast001",​ use this syntax:
  
Line 302: Line 300:
 As before, one can also download the files using the URL approach: As before, one can also download the files using the URL approach:
 <code bash> <code bash>
-hs-ls http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ # list all files +hs-ls https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ # list all files 
-hs-get ​http://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ data+hs-get ​https://​mc.hep.anl.gov/​asc/​hepsim/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ data
 </​code>​ </​code>​
 Note that the reconstruction tag "​rfast001"​ is separated by backslash as for the usual directory. Note that the reconstruction tag "​rfast001"​ is separated by backslash as for the usual directory.
Line 312: Line 310:
 hs-get http://​portal.nersc.gov/​project/​m1758/​data/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ data hs-get http://​portal.nersc.gov/​project/​m1758/​data/​events/​pp/​100tev/​ttbar_mg5/​rfast001/​ data
 </​code>​ </​code>​
- 
- 
  
  
hepsim/usage_download.txt · Last modified: 2020/05/21 21:45 by hepsim17
Back to top
CC Attribution-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0