Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
hepsim:dev_full [2017/08/29 03:28]
hepsim17 [Reconstruction tags]
hepsim:dev_full [2017/09/21 19:18] (current)
hepsim17
Line 1: Line 1:
-{{indexmenu_n>11}}+{{indexmenu_n>13}}
 [[:|<< back to HepSim manual]] [[:|<< back to HepSim manual]]
  
-====== Creating full simulations ====== 
  
 +===== Creating LCIO files =====
  
-===== Reconstruction tags =====+In many cases, HepSim detector simulations were done using ILCSOFT-derived software, such as those used for ILC, CLIC, etc. experiments. 
  
- +To perform Geant4 simulations, you will need to convert PROMC to SLCIO or STDHEP files. You can convert ProMC files with truth-level information to STDHEP or LCIO (MCParticle) with full simulations using promc2stdhep or promc2lcio converters:
-Full detector simulations are created using the concept of tagswhich are +
-small (<3MB) files that include necessary information to convert truth-level files to final reconstructed events. All tags  +
-can be found in [[http://atlaswww.hep.anl.gov/hepsim/recotags.php|this page]].  +
-In many cases we call them "reconstruction" tags, but, generally, include both Geant4  simulation and reconstruction steps. +
- +
-The reconstruction tag is associated with a file with the name "rfull[XXX].tgz", where [XXX] is a number +
-(fast simulations have the files "rfast[XXX].tgz"). +
- +
-Reconstruction tags have unique names. There can be several reconstruction tags corresponding to a single detector, since simulation and reconstruction can be done using different software.  +
-They are small files that are sourced by reconstruction software on grid or HPC. A file "rfull[XXX].tgz" +
-has everything you need to create simulations and reconstruction files. Such files include the detector geometry,  configuration scripts, setup files etc. In most cases, the directory inside rfull[XXX].tgz has this structure (for a detector called "sifcch7"):+
  
 <code bash> <code bash>
-A_RUN     # main script to process events (free form) +wget http://atlaswww.hep.anl.gov/hepsim/soft/promc_convert.tgz 
-TEST      # script for testing (optional) +tar -zvxf promc_convert.tgz 
-source.sh # setup script (optional) +cd promc_convert 
-.....     # other required files if needed +./promc2lcio.sh input.promc output.slcio    # convert to SLCIO 
-sifcch7   # the directory with detector geometry    (always required!) +./promc2stdhep.sh input.promc output.stdhep convert to STDHEP
-    compact.xml     # main detector geometry file (always required!) +
-    - sifcch7.heprep  # some converted files, if needed +
-    - sifcch7.lcdd    # optional file for event displays +
-    - sifcch7.pandora # optional file for Pandora +
-    - sifcch7.json    # optional geometry file +
-    - sifcch7.root    # optional geometry file (use to make 3D display) +
-    - sifcch7.html    # HTML description (required) +
-    - view1.png       # main image to view the detector (440x480 px, Y-Z) +
-    - view2.png       2nd image to view the detector (440x480 px, X-Y) +
-    - some config files+
 </code> </code>
-To see what is inside, download "rful009.tgz" from [[http://atlaswww.hep.anl.gov/hepsim/taginfo.php?id=rfull009]]. 
  
-In addition to the detector geometry, the tag files include all settings needed for successful creation of events. +You can convert all files in a directory with ProMC files as:
-Generally, the form of this file is free, as long as it includes sufficient information  +
-to create reconstructed events. In many cases, the tag file has several scripts A_RUN (to make a complete reconstruction chain) and TEST (to test on a few events). The software to run these scripts should be fully specified, and it is up to the author to make sure that one can use such files to run over events on a dedicated computing resource.+
  
-<note important>Note that the simulation and reconstruction software should not be put inside rfullXXX.tgz files. This file size limit of 3MB is important to keep a low network bandwidth 
-when the tag files are sourced by reconstruction software on each node on HPC or grid,  
-which are assumed to contain the needed software used in combination with rfullXXX.tgz files. 
-</note> 
- 
-You should create the reconstruction tag if: 
- 
-  * detector geometry has changed. In this case, use a different detector name inside rfullXXX.tgz 
-  * detector geometry is the same, but configuration files and setup scripts have changed. 
- 
-HepSim provides a mechanism to upload rfull[XXX].tgz. After upload, HepSim extracts the  
-detector from rfull[XXX].tgz and builds an information detector page. Then you will see two entries: 
- 
-  * [[http://atlaswww.hep.anl.gov/hepsim/detectors.php]] 
-  * [[http://atlaswww.hep.anl.gov/hepsim/recotags.php]] 
- 
-As mentioned before, we leave the authors with the liberty to design rfull[XXX].tgz files. The only strict requirement is that they should have the directory with the detector, its PNG image and the  HTML description, since this populates  the information page with the detector in HepSim.  
-The simulation and reconstruction scripts can include all the required 
-workflow to work together with the installed software (git, makefiles, wget commands etc). 
- 
-<note tip> 
-A tag file should have a directory with the detector name, and the "compact.xml" file inside this directory. This structure helps automatically extract the detector for inclusion in HepSim 
-Please add compact.xml even if it is a dummy file not used for detector description.</note> 
- 
-After a tag file is uploaded, HepSim performs the following operations with this file: 
- 
-  * Moves the  file to the standard location, so it can be used as a source on computing nodes 
-  * Extracts the detector, and makes a zip file for download 
-  * Builds a web page with the detector information (uses included HTML, view1.png, compact.xml and other available files) 
-  * For the SiD derived software, prepares a zip file with detector description for Jas4pp to view the detector inside the Wired4 display 
- 
-  
- 
-===== Detector simulation with FPadSim ===== 
- 
- 
-To create full simulations, you will need to convert ProMC files  to LCIO files. This is described in 
-[[hepsim:usage_truth#converting_to_lcio|LCIO conversions]]. 
- 
-Many samples posted in HepSim are created with ths  package is called "fpadsim" (**F**uture **Pa**rticle **D**etector **Sim**ulator"). This package is under constant development.  
-The contributed authors of this package since 2015 are: 
- 
-  * D.Blyth (HEP/ANL) 
-  * J.MacCormick (SLAC) 
-  * W.Armstrong  (PHYS/HEP) 
-  * S.Chekanov (HEP/ANL) 
-  * A.Kotwal (Duke) 
-  * J.Strube (PNNL) 
-  * J.Marshall (U.Cambridge) 
- 
- 
-The package includes packages from ilcsoft, such as SLIC, PandoraPFA, and LCSIM 
-The simulation and reconstruction proceeds as: SLIC->LCSIM->slicPandora/PandoraPFA. 
-This digram shows the entire chain: 
- 
-{{:hepsim:fpdasim_reco.png?700|}} 
- 
- 
-The input for SLIC are LCIO files with truth-level information ("MCParticle" table). The files 
-are created from ProMC files using the converter promc2lcio (included inside 
-the ProMC packages, in the directory "examples"). 
-This concept originates from the SiD detector(T.Johnson, N.Graf, J.McCormick, J.Strube), 
-and later was re-purposed to be used for generic feature detectors, including 
-proton pp colliders (S.Chekanov, A.Kotwal). The final step 
-of the reconstruction is PandoraPFA (J.Marshall, M.Thomson). 
- 
- 
- 
-To create LCIO files with full simulations and reconstruction, you need: 
- 
-  * ProMC file from HepSim 
-  * Simulation and Reconstruction package called "fpadsim-[version]". 
-  * Tag file which includes detector and the simulation and reconstruction sequence. 
- 
-The fpadsim package is located [[http://atlaswww.hep.anl.gov/hepsim/soft/ |here]]. It is usually designed for the OSG grid. 
-The tag files are also located [[http://atlaswww.hep.anl.gov/hepsim/soft/ |here]].  They have the names "rfull[XXX].tgz". 
-After untaring the file fpadsim.tgz, untar the file "rfull[XXX].tgz". 
-There is a simple script "Test.sh" that creates a few events and illustrate this concept. 
- 
- 
- 
-====== Changes in compact.xml ====== 
- 
- 
-If you want to make changes in the detector design, edit "compact.xml", and then run the converters: 
 <code bash> <code bash>
-JAVA_OPTS="-Xms2048m -Xmx2048m" +./batchconverter stdhep [DIR with ProMC files] convert to STDHEP files 
-GCONVERTER=lcsim/detector-framework/target/lcsim-detector-framework-3.1.6-SNAPSHOT-bin.jar +./batchconverter slcio  [DIR with ProMC files] # convert to LCIO files
-GEOM="mydetector" name of your detector +
-java $JAVA_OPTS -jar $GCONVERTER -o lcdd compact.xml $GEOM.lcdd +
-slic -g ${GEOM}.lcdd  -G ${GEOM}.gdml +
-echo "heprep compact.xml.." +
-java $JAVA_OPTS -jar $GCONVERTER -o heprep compact.xml $GEOM.heprep +
-echo "pandora compact.xml.." +
-java $JAVA_OPTS -jar $GCONVERTER -o pandora compact.xml $GEOM.pandora +
-java $JAVA_OPTS -jar $GCONVERTER -o html compact.xml ${GEOM}.html+
 </code> </code>
-where "${GEOM}" is the name of the detector geometry.  Next, check that you do not have overlapping volumes. Run ROOT with the commands: 
  
-<code cpp> 
-TGeoManager::Import("mydetector.gdml"); 
-gGeoManager->ViewLeaves(true); 
-gGeoManager->ls("All"); 
-gGeoManager->CheckOverlaps(0.1); //0.1 tolerance or better! 
-gGeoManager->PrintOverlaps(); 
-gGeoManager->GetTopVolume()->Draw("ogl"); 
-</code> 
-If you see that some volumes overlap, come back to "compact.xml" and make corrections. 
-====== Pandora PFA ====== 
  
-All information required by Pandora algorithm is stored in the file with the extension ".pandora". This file is created using GeoConvertor, which reads CalorimeterCalibration.properties file. 
-In particular, it reads: 
  
-  * mipEnergy +Files in the LCIO or STDHEP formats can be used as input for DD4HEP or SLIC simulation packages.
-  * mipSigma +
-  * mipCut +
-  * timeCut+
  
-The sampling fractions are taken from the detector  directory called "SamplingFractions" +Many samples of HepSim were created with a dedicated FPadSim packageSee [[hepsim:dev_fpadsim|FPadSim section]].
-Here files look as this: +
- +
-<code> +
-BeamCal.properties* +
-EcalBarrel.properties* +
-EcalEndcap.properties* +
-HcalBarrel.properties* +
-HcalEndcap.properties* +
-LumiCal.properties* +
-MuonBarrel.properties* +
-MuonEndcap.properties* +
-README* +
-</code> +
- +
- +
-Look at the file: +
-<code> +
-lcsim/detector-framework/src/main/java/org/lcsim/geometry/compact/converter/pandora/Main.java +
-</code>+
  
  --- //[[[email protected]|Sergei Chekanov]] 2017/04/12 21:58//  --- //[[[email protected]|Sergei Chekanov]] 2017/04/12 21:58//