ATC: ATLAS Center at ANL

User Tools

Site Tools

Trace: • ProMC User's Manual
community:hepsim:usage_truth

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
community:hepsim:usage_truth [2016/04/28 15:17] – created asccommunity:hepsim:usage_truth [2016/11/04 19:23] (current) – [Converting to LCIO] asc
Line 1: Line 1:
 +
 +[[community:hepsim:|<< back to HepSim manual]]
 +
 ======  Plotting distributions ====== ======  Plotting distributions ======
  
Line 39: Line 42:
 </code> </code>
  
 +Look at the "example" directory of the Jas4pp installation. It has a several examples of how to process the ProMC files.
  
 Similarly, you can use a more complex [[http://jwork.org/dmelt/ | DataMelt]]: Similarly, you can use a more complex [[http://jwork.org/dmelt/ | DataMelt]]:
Line 300: Line 303:
      
  
- +Please look at HepSim [[community:hepsim:usage_truth|programming tutorials]] on how to design Jython scripts.
-====== Supported languages ====== +
- +
-  * C++/ROOT +
-  * CPython  (Python implemented in C++) or PyROOT +
-  * Java +
-  * Jython (Python on the Java platform) +
-  * Groovy  (scripting language) +
-  * BeanShell (scripting language) +
-  * JRuby (Ruby on the Java platform) +
- +
- +
-Some variations of the Java languages are also possible (such as Scala). For all Java-derived languages, see the [[http://jwork.org/dmelt/ | DatMelt]] project. +
-The Android platform has a limited support (but the files should be readable on Android).  +
- +
- +
-Although all example scripts are based on the Python language, the actual implementation of the validation online scripts is done in Jython (Java). +
-You can program in Java as well. C++/CPython will be discussed later. +
- +
- +
- +
- +
- +
-===== Using Jython ===== +
- +
-Jython and Java are chosen  to write validation scripts and to run such scripts using web pages. +
-Please look at [[community:refhepsim_analysis| HepSim Analysis Primer]] manual. +
- +
-===== Using C++ ===== +
 Please refer [[asc:promc| ProMC web page]] on how to read/write ProMC files. Please refer [[asc:promc| ProMC web page]] on how to read/write ProMC files.
  
Line 341: Line 315:
 </code> </code>
 Read "README" inside this directory. Do not forget to populate the directory "data" with MC files before running this example. Read "README" inside this directory. Do not forget to populate the directory "data" with MC files before running this example.
- 
-===== Using CPython ===== 
- 
- 
-Use the example above to generate CPython code to read data. 
-This is documentation for CPython approach 
- 
- 
-      * [[http://atlaswww.hep.anl.gov/asc/promc/doc/cpython_promc/html/index.html | annotated classes]] - for complete MC event records 
-      * [[http://atlaswww.hep.anl.gov/asc/promc/doc/cpython_pronlo/html/index.html | annotated classes]] - for NLO event records 
- 
- 
-Read the description of [[http://atlaswww.hep.anl.gov/asc/promc/ | ProMC format]] about how to generate CPython code. 
- 
  
  
Line 371: Line 331:
 </code> </code>
  
 +====== Converting to LCIO ======
  
-====== Fast detector simulation ====== +ProMC files can be converted to LCIO files for full detector simulations.  This is an example of such conversion (it requires Java installed):
- +
-===== On the fly reconstruction ===== +
-HepSim ca be used o create ROOT files after fast detector simulation, or  +
-one can analyse events after the Delphes fast simulation program on the fly. +
-The latter approach allows to make changes to  the detector geometry by the end-users and, at the the same time, perform an analysis. +
-The output ROOT file only includes histograms defined by the user (but you can also add custom ROOT tree).  +
-To do this, use the [[http://atlaswww.hep.anl.gov/asc/hepsim/soft/FastHepSim.tgz| FastHepSim package]] +
-that includes [[http://cp3.irmp.ucl.ac.be/downloads/Delphes-3.2.0.tar.gz | Delphes-3.2.0]] and ProMC 1.5, as well as an example analysis program. +
-You can find a description of the [[https://atlaswww.hep.anl.gov/asc/wikidoc/doku.php?id=vlhc:hcal|current HCAL studies]] that use such an approach. +
- +
-Follow these steps:+
  
 <code bash> <code bash>
-wget http://atlaswww.hep.anl.gov/asc/hepsim/soft/FastHepSim.tgz +wget http://atlaswww.hep.anl.gov/asc/promc/download/current.php -O ProMC.tgz 
-tar -zvxf FastHepSim.tgz +tar -zvxf ProMC.tgz 
-cd FastHepSim/libraries/ +cd examples/promc2lcio
-./install.sh +
-</code> +
- +
-This installs "FastHepSim" which includes Delphes, ProMC and hs-tools to work with HepSim. Then setup the environment  +
- +
-<code bash> +
-cd .. # go to the root directory+
 source setup.sh source setup.sh
 +javac promc2lcio.java
 +java promc2lcio file.promc file.slcio
 </code> </code>
- +The last commends creates  file.slcio with "MCparticle" container. 
-Next, go to the analysis example: +
- +
-<code bash> +
-cd analysis +
-make +
-</code> +
-  +
-This compiles the analysis program (analysis.cc) that fills jetPT and muonPT histograms.  +
-Now we need to bring  data from [[http://atlaswww.hep.anl.gov/hepsim/| HepSim]] and put them somewhere. For this example, +
-we will copy data to the "data" directory: +
- +
-<code bash> +
-hs-get http://mc.hep.anl.gov/asc/hepsim/events/pp/100tev/higgs_ttbar_mg5 data 2 3 +
-</code> +
-This copies 3 files in 2 threads and put them to the directory "data". Now we are ready to run over these 3 files: +
- +
-<code bash> +
-./Make_input data  # creates inputdata.txt +
-./analysis delphes_card_FCC_basic_notau.tcl histo.root inputdata.txt +
-</code> +
- +
-The first command creates a file "inputdata.txt" with input data. Then the program "analysis" reads the configuration file, data and generates "histo.root" +
-with output histograms. This example uses "delphes_card_FCC_basic_notau.tcl" Delphes configuration file describing a basic FCC detector geometry. +
-Note that we have removed  tau tagging since we run over slimmed events with missing mothers of particles. +
- +
-If you want to access other objects (photons, electrons, b-jets),  +
-use [[https://cp3.irmp.ucl.ac.be/projects/delphes/wiki/WorkBook/Arrays | Delphes definitions of arrays]] inside the source code "analysis.cc". +
-You can put external files into the src/ directory where it will be found by Makefile. +
- +
-If you still want to look at the event structure in the form of ROOT tree, run the usual Delphes command: +
-<code bash> +
-../libraries/Delphes/DelphesProMC  delphes_card_FCC_basic_notau.tcl  output.root data/mg5_Httbar_100tev_001.promc +
-</code> +
-where output.root will contain all reconstructed objects. In this case, add "TreeWriter" after "ScalarHT". +
-If the input file contains complete (non-slimmed) record, one can add "tau" reconstruction ("TauTagging" line). +
- +
-Try also more sophisticated detector-geometry cards: +
- +
-  * examples/delphes_card_Snowmass_NoPileUp.tcl - similar to Snowmass 2013. It has CA (fat) jets and B-tagging +
-  * examples/delphes_card_ATLASLIKE.tcl         - similar to ATLAS detector (comes with Delphes) +
- +
-Note that "TreeWriter" module should be enabled when creating ROOT files, and disabled when using analysis mode. +
- +
-===== Creating Delphes files  ===== +
- +
-Here we describe how to make fast detector simulation files  +
-using separate external libraries, without installing FastHepSim. +
-Use the [[https://cp3.irmp.ucl.ac.be/projects/delphes | Delphes]] fast detector simulation program to process the MC events. Delphes can read [[http://atlaswww.hep.anl.gov/asc/promc/ | ProMC]] files directly. First, make sure that ProMC and ROOT library is installed: +
-<code bash> +
-echo $PROMC $ROOTSYS +
-</code> +
-This should point to the installation paths of ProMC and ROOT. +
-  +
-Here are the steps to perform a  fast detector simulation using ProMC files from the HepSim repository: +
- +
-1) Download [[http://cp3.irmp.ucl.ac.be/downloads/Delphes-3.2.0.tar.gz | Delphes-3.2.0.tar.gz]] (or higher) and compile it as:  +
-<code bash> +
-wget http://cp3.irmp.ucl.ac.be/downloads/Delphes-3.2.0.tar.gz +
-tar -zvxf Delphes-3.2.0.tar.gz +
-cd Delphes-3.2.0 +
-./configure +
-make +
-</code> +
-This  creates the converter "DelphesProMC" (among others), if "PROMC" environmental variable is detected.  +
- +
-2)  For FCC studies, copy and modify the detector configuration file "delphes_card_FCC_basic.tcl"  +
-<code bash> +
-cp cards/delphes_card_FCC_basic.tcl delphes_card_FCC_notau.tcl +
-</code> +
-Then remove the line "TauTagging". Do the same when using the card "cards/delphes_card_ATLAS.tcl" (for ATLAS). +
-We do not use  the tau tagging module since it requires complete event records with all mother particles. Since ProMC files are often slimmed by removing some unstable low pT particles and showered partons, Delphes will fail on this line. If you need tau tagging, please use ProMC files with complete particle record. +
- +
-3) Download Monte Carlo files from the [[ http://mc.hep.anl.gov/asc/hepsim/events/?dir=pp/100tev | 100 HepSim repository]]. +
-For example, get a file with 5000 ttbar events generated for a 100 TeV collider: +
-<code bash> +
-wget http://mc.hep.anl.gov/asc/hepsim/events/pp/100tev/ttbar_mg5/mg5_ttbar_100tev_001.promc +
-</code> +
- +
-and then create a ROOT file with reconstructed objects after a  fast detector simulation (FCC detector): +
- +
-<code bash> +
-./DelphesProMC delphes_card_FCC_notau.tcl  mg5_ttbar_100tev_001.root mg5_ttbar_100tev_001.promc +
-</code> +
-The conversion typically takes 30 seconds. +
- +
- +
-====== Full detector simulation  ====== +
- +
-Currently, the full detector simulation can be done using the SLIC software. You will need to convert ProMC files to LCIO files and +
-use these files for the "sclic" Geant4-based program. +
- +
-The physics performance studies are  [[fcs:start|listed here]]. They are based on  [[http://atlaswww.hep.anl.gov/asc/jas4pp|Jas4pp program]]. +
- +
-How to convert ProMC to LCIO files is described in +
-[[community:refhepsim&#converting_to_lcio]] +
- +
-====== Pileups mixing  ====== +
- +
-One can mix events from a signal ProMC file with +
-inelastic (minbias) events using a "pileup" mixing program: +
-<code bash> +
-hs-pileup pN signal.promc minbias.promc output.promc +
-</code> +
-Here "p" indicates that N events from minbias.promc will be mixed with every event from signal.promc using a Poisson distribution. +
-If "p" is missing, then exactly N events from minbias.promc will be mixed with single event from signal.promc. +
-Use large number of events in minbias.promc to minimise reuse of the same events from the minbias.promc. +
-The barcode of particles inside output.promc indicates the event origin. +
- +
- +
-Here is an example to create events with pileup for the paper [[http://arxiv.org/abs/1504.08042]] Phys. Rev. D 91, 114018 (2015): +
-We mix h-> HH signal file with MinBias Pythia8 (A2 tune) sample using <mu>=200 (Poisson mean):  +
-<code bash> +
-wget http://atlaswww.hep.anl.gov/asc/hepsim/soft/hs-toolkit.tgz -O - | tar -xz; +
-source hs-toolkit/setup.sh +
-wget http://mc.hep.anl.gov/asc/hepsim/events/pp/100tev/minbias_a2_pythia8_l3/tev100_pythia8_MinBias_l3__A2_001.promc +
-wget http://mc.hep.anl.gov/asc/hepsim/events/pp/100tev/s0higgshiggs_alltau/tev100_s0higgshiggs_alltau_0001.promc +
-hs-pileup p200 tev100_s0higgshiggs_alltau_0001.promc tev100_pythia8_MinBias_l3__A2_001.promc output.promc  +
-</code> +
-Look at the "output.promc" with the browser to see how events are filled. Now you can use  DELPHES to create fast simulation files using events with pileup. +
- +
-//Note:// pileup mixer is implemented in Java, therefore it uses 64bit zip. For C++ programs, please use ProMCBook("file.promc", "r", true) to  open ProMC  +
-files (notice "true" as a new argument). For example, for the Delphes program, change the line  +
- +
-<code cpp> +
-inputFile = new ProMCBook(argv[i], "r");       # 32bit zip +
-</code> +
-in readers/DelphesProMC.cpp to +
-<code cpp> +
-inputFile = new ProMCBook(argv[i], "r", true); # 64bit zip +
-</code>+
  
  
 +====== Converting to other formats ======
 +Look at other directories in "examples/". You can convert ProMC files to many other formats (most converters require  installation of the ProMC C++ package).
 ====== Extracting events  ====== ====== Extracting events  ======
  
Line 539: Line 356:
 </code> </code>
 where signal.promc is the original file, and N is the number of events to extract. where signal.promc is the original file, and N is the number of events to extract.
- 
- 
- 
- 
  
 ====== Comparing MC and data  ====== ====== Comparing MC and data  ======
Line 555: Line 368:
 the scripts with data from [[http://durpdg.dur.ac.uk/|Durham HepData database]]. Or you can start from a HepData "SCaVis" script and add parts from HepSim validation script that reads data from HepSim. Note that SCaVis and DMelt are equivalent. the scripts with data from [[http://durpdg.dur.ac.uk/|Durham HepData database]]. Or you can start from a HepData "SCaVis" script and add parts from HepSim validation script that reads data from HepSim. Note that SCaVis and DMelt are equivalent.
  
- 
-====== How to create ProMC files  ====== 
- 
-There are several methods to create ProMC files from MC simulations. The most complete description is given [[https://atlaswww.hep.anl.gov/asc/wikidoc/doku.php?id=asc:promc|here]]. Here are several examples: 
- 
-  * If you use Pythia8, look at the example "main46.cc" inside Pythia8 package. 
-  * For other generators, such as Herwig++ and Madgraph, write events in either lhe files or hepmc files. Then  use the converters "stdhep2promc" or "hepmc2promc" that are shipped  with ProMC (inside $PROMC/examples).  
- 
-If are working on a description of how to fill ProMC files from Jetphox or MCFM.  At this moment, contact us to get help. 
- 
-We have very limited support for other formats, since they are typically 1) large files (such as HEPMC); 2) cannot be used for streaming over the network; 3) are not suited in a multi-platform environments (i.e. Windows). 
- 
-  
- 
- 
- 
-====== A note for ANL cluster  ====== 
- 
-For ANL cluster, you do not need to install Delphes. Simply run the reconstruction as: 
-<code bash> 
-source /share/sl6/set_asc.sh 
-$DELPHES/DelphesProMC delphes_card_FCC_notau.tcl mg5_ttbar_100tev_001.root mg5_ttbar_100tev_001.promc 
-</code> 
-The cards are located in $DELPHES/examples. 
- 
-If you want to run over multiple ProMC files without manual download, use this command: 
- 
-<code bash> 
-java -cp hepsim.jar hepsim.Exec DelphesProMC delphes.tcl output.root [URL] [Nfiles] 
-</code> 
-where [URL] is HepSim location of files and  [Nfiles] is the number of files for processing. 
-The output ROOT will be located inside the "hepsim_output" directory. 
-Here is a small example: 
-<code bash> 
-java -cp hepsim.jar hepsim.Exec DelphesProMC delphes.tcl output.root  http://mc.hep.anl.gov/asc/hepsim/events/pp/100tev/higgs_ttbar_mg5 5 
-</code> 
-which processes 5 files from [[http://atlaswww.hep.anl.gov/hepsim/info.php?item=2|Higgs to TTbar sample]].  
-Skip "5" at the end to process all files.  
- 
-====== Single-particle gun ====== 
- 
-ProMC files with single particles can be created using the ProMC. Download the recent ProMC package and 
-look at the directory: 
- 
-<code bash> 
-wget http://atlaswww.hep.anl.gov/asc/promc/download/current.php -O ProMC.tgz 
-tar -zvxf ProMC.tgz 
-cd examples/particle_gun 
-source setup.sh 
-javac promc_gun.java 
-java promc_gun pions.promc 100 211 1000 # E=0-100 GeV, PID=211, events=1000 
-</code> 
-This creates file "pions.promc" with 2 particles per event, with the maximum energy of 100 GeV (randomly distributed from 0 to 100). 
-The particles are pions  (pi+, pid=211). The total number of events 1000. The phi and theta distributions are flat 
-After file creation, look at the events using the file browser. You can also modify the java code to change the single-particle events. 
- 
-====== Converting to LCIO ====== 
- 
-ProMC files can be converted to LCIO files for full detector simulation.  This is an example of such conversions: 
- 
-<code bash> 
-wget http://atlaswww.hep.anl.gov/asc/promc/download/current.php -O ProMC.tgz 
-tar -zvxf ProMC.tgz 
-cd examples/promc2lcio 
-source setup.sh 
-javac promc2lcio.java 
-java promc2lcio file.promc file.slcio 
-</code> 
- 
-Look at other directories in "examples/". You can convert ProMC files to many other formats (most converters require  installation of the ProMC C++ package). 
-====== Record slimming ====== 
-Particle records from the generators based on  LO/NLO+parton showers calculations  
-(PYTHIA, HERWIG, MADGRAPH) are often "slimmed" to reduce file sizes. In the case when 
-records are slimmed, the following algorithm is used: 
- 
-<code python> 
-(status=1 && pT>0.3 GeV ) or   # keep final states with pT>0.3 GeV  
-(PID=5 || PID=6)  or           # keep b or top quark  
-(PID>22 && PID<38)  or         # keep exotics and Higgs 
-(PID>10 && PID<17)  or         # keep all leptons and neutrinos  
-</code> 
-where PID is absolute value of particle codes. Leptons ane neutrinos are also affected by the slimming pT cut. 
-Note: for 100 TeV collisions, the pT cut is increased from 0.3 to 0.4 GeV. 
-For NLO calculations with a few partons + PDF weights, the complete event records are stored. 
- 
-In the case when the slimming is applied, file sizes are reduced by x2 -  x3. In some situation, slimming 
-can affect detector simulation. For example, you should turn of tau reconstruction in Delphes when slimming is used. 
  
  
Line 680: Line 406:
 </code> </code>
 </hidden> </hidden>
 +
 +====== Programming with HepSim ======
 +
 +
 +
 +Please look at the [[community:hepsim:usage_truth|tutorials]] which show how to create Python/Jython scripts to read ProMC files. 
  
  Send comments to:  --- //[[[email protected]|Sergei Chekanov (ANL)]] 2016/02/08 10:26//  Send comments to:  --- //[[[email protected]|Sergei Chekanov (ANL)]] 2016/02/08 10:26//
community/hepsim/usage_truth.1461856649.txt.gz · Last modified: 2016/04/28 15:17 by asc