HepSim

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
{{indexmenu_n>17}}

[[:|<< back to HepSim manual]]

======  Jas4pp description ======

[[https://atlaswww.hep.anl.gov/asc/jas4pp/ | Jas4PP program]] is 
a data-analysis environment for detector and physics studies of future circular colliders.
Jas4PP is a merge of several open-source Java projects, such as [[http://www.lcsim.org|lcsim.org]] [(Simulator for the linear collider (SLIC): A tool for ILC detector simulations. N. Graf and  J. McCormick, http://inspirehep.net/record/736286?ln=en)], [[http://lcio.desy.de/ |LCIO event model]], [[http://java.freehep.org/|FreeHep]], [[https://datamelt.org|DataMelt]] (community edition) and  [[http://atlaswww.hep.anl.gov/hepsim|hs-tools]] of HepSim. It also supports Java-implemented anti-KT jets. The package is designed to work using the Python language (it includes Jython 2.7, which is compatible with the 2.7 version of CPython), or using the standard Java coding. All HEP Jas plugins are included (and some of them were modified to work with DataMelt).


The installation does not have external dependencies besides Java. Make sure that Oracle Java8 and above is installed (not OpenJDK).
You can download the Jas4pp program using the [[https://atlaswww.hep.anl.gov/asc/jas4pp/download/|download page]].
Then run these commands to install the package using Linux/Mac with the "bash" shell:

<code bash>
wget https://atlaswww.hep.anl.gov/asc/jas4pp/download/current.php -O jas4pp.tgz
tar -zvxf jas4pp.tgz
cd jas4pp
source ./setup.sh # takes 5 sec for first-time optimization
</code>
The last command optimizes Java packages for the Python language.
Now you are ready to run over  any file with truth-level and datector-simulation files, such as LCIO and ProMC. Here are the included commands:



<code bash>
jaspp          # start Jas4pp with preconfigured HEP plugins (Linux/Mac with bash)
jaspp.bat      # same for Windows
fpad           # run Jython scripts in a batch mode (Linux/Mac with bash) 
fpad_edit      # start a minimalistic Python editor "FPAD") and Jython shell
fpad_edit.bat  # same for Windows
gconverter     # geometry converters
gconverter_gui # same in GUI mode
hs-help        # shows commands to access data from the HepSim repository
</code>

Use these programs for:

  * Downloading and searching HepSim data
  * Processing ProMC files from HepSim
  * Processing ROOT files created by the Delphes program for fast simulation
  * Running over SLCIO files with Geant4 simulated / reconstructed events.
  * Data analysis (jets, physics vectors, histogram packages)
  * Visualization of reconstructed events using Wired4 display

You can find more details in [[https://atlaswww.hep.anl.gov/hepsim/doc/doku.php|HepSim manual]].


<note tip>Note that you can also use the [[hepsim:dev_singularity|HepSim Singularity/Docker container]] with included
Jas4pp</note>

====== Examples ======

All examples of Jas4pp are collected in the directory "examples". Run them as:

<code bash>
fpad examples/dmelt/histo1.py                 # DMelt histogram example
fpad examples/jaida/Fit.py                    # Jas/Aida fit example
fpad examples/hepsim/pythia6_zpole_tautau.py  # run over HepSim Monte Carlo data
</code>
The last example from the [[https://atlaswww.hep.anl.gov/hepsim/|HepSim web page]] prints the pT distribution of e, mu, taus.
You can also run these examples as "fpad_edit file.py" or using the Jas-like environment (i.e. jaspp program).


Here is an example of how to compare data points with a histogram using DMelt classes.
Run this script as "fpad file.py", or open it in the Jas4pp editor and run this script.

<hidden Click to show the Python code>
<code python>
from java.awt import *
from java.util import Random
from jhplot  import *

c1 = HPlot()
# c1.doc()  # view documentation
c1.setGTitle("F_{2},  x_{&gamma;}  #bar{p}p F_{2}^{c#bar{c}}") 
c1.visible(1)
c1.setAutoRange()
c1.setMarginLeft(85) # make space for Y label
c1.setNameX("X")
c1.setNameY("Y")

h1 = H1D("MC",25, -2, 2.0)
h1.setPenWidthErr(2) # line width
h2 = H1D("data",25, -2, 2.0)

r=Random()
for i in range(10000):
   h1.fill(r.nextGaussian())      
   h2.fill(r.nextGaussian())          

p1=P1D(h2) # convert to X-Y array 
c1.draw(h1)
c1.draw(p1)
# c1.drawStatBox(h1)

# set HLabel in the normilised coordinate system
lab=HLabel("HLabel in NDC", 0.18, 0.7, "NDC")
lab.setColor(Color.blue)
c1.add(lab)

c1.update()
c1.export("image.eps") 
</code>
</hidden>

Run this example in Jas4pp and look at this image:

<hidden Click to show the image>
{{:asc:histo1.png|Histogram}}
</hidden>


====== Reading LCIO files ======

Jas4pp can be used to process LCIO files. This can be done using Java, or Python (Jython). The examples are given in the directory "example".
<hidden Click to show the Python code>
<code python>
# Example of processing SLCIO files using Jas4pp
# Setup Jas4pp using "source setup.sh".
# Then create the directory "data"  and fill it with SLCIO files using has-get command,
# and run this script as "fpad example.py data"

from org.lcsim.lcio import LCIOReader
from hep.io.sio import SIOReader
from hep.lcio.implementation.sio import SIOLCReader
from hep.lcio.implementation.io import LCFactory
from hep.lcio.event import * 
from hep.lcio.io import *
from jhplot import *
from hephysics.particle import LParticle
import math
import os,sys
from java.lang import System;

# get directory name from the argument
filename = sys.argv[1]

# make list of files..
import glob
files = glob.glob(filename+"/*.slcio")
factory = LCFactory.getInstance()

nEvent=0
for f in files:
    print "Open file=",f
    reader = factory.createLCReader()
    reader.open(f) 
    while(1):
        evt=reader.readNextEvent()
        if (evt == None): break
        nEvent=nEvent+1
        # print " file event: ",evt.getEventNumber(), " run=",evt.getRunNumber()
        if (nEvent%100==0): print "# Event: ",nEvent
        col    = evt.getCollection("MCParticle")
        colPF  = evt.getCollection("PandoraPFOCollection");
        colCl  = evt.getCollection("ReconClusters");
        colTr  = evt.getCollection("Tracks");
        colECB = evt.getCollection("EM_BARREL");
        colHCB = evt.getCollection("HAD_BARREL");

        nMc=col.getNumberOfElements();
        nPF=colPF.getNumberOfElements();
        nCl=colCl.getNumberOfElements();
        nTr=colTr.getNumberOfElements();
        nECB=colECB.getNumberOfElements();
        nHCB=colHCB.getNumberOfElements();
        # now you can access all data here
     
        # manage memory if too much data 
        del col
        del colPF
        del colCl
        del colTr
        del colECB
        del colHCB
        del evt

    reader.close() # close the file
    del reader
    System.gc()    #  force memory cleanup, if needed 
</code>
</hidden>
To process SLCIO files from a directory "data", call this script "example.py as:

<code>
fpad example.py data
</code>
The script open each file, and access every container in the file.
Note that we explicitly mange memory in case of very large files, forcing the garbage collector for each file. In many cases, this is not needed since JVM takes care of memory leakage.

You can find more examples in [[:hepsim:usage_full|this section]] and in the directory "examples" of Jas4pp.

Also, look at concrete example of how to analyze single particles in [[:fcs:eic:intro|this section]].


======  Reading ROOT files ======

Jas4pp natively reads commonly used objects and data structures from [[https://root.cern/ | ROOT files]]  (versions 3, 4, 5 and 6). 
ROOT files can be loaded using the Jas4pp menu [File]-[Open data source]-[Root file] (*.root).

In addition, one can work with ROOT files using Python/Jython or Groovy scripts. 
One can find some examples in the directory "examples/root".  Here is a Jython/Python example showing how to read a ROOT tree:
 
<hidden Show example of a Python code here>
<code python>
from  hep.io.root.interfaces import TTree
from  hep.io.root import RootFileReader

reader = RootFileReader("ntuple_tree.root")

tree = reader.get("tree");
maxevents=tree.getEntries()

leaves = tree.getLeaves()
nrleaves=leaves.size()

print "Nr of events=",maxevents
print "Nr of leaves=",nrleaves

print "Leaves:"
for l in xrange( nrleaves ):
   print "Leaf=",(leaves.get(l)).getName()

print "Run over events"
f0=leaves.get(0);
f1=leaves.get(1);
f2=leaves.get(2);
for i in xrange(tree.getEntries()):
      print f0.getValue(i), f1.getValue(i), f2.getValue(i)

</code>
</hidden>

The example directory also shows how to read histograms.
Similar examples can be made using Java or Groovy scripting.

The supported ROOT interfaces are:

<hidden show more details on ROOT interfaces here>
TArrayC
TArrayD
TArrayF
TArrayI
TArray
TArrayL
TAttAxis
TAttFill
TAttLine
TAttMarker
TAxis
TBasket
TBranchClones
TBranchElement
TBranch
TBranchObject
TClonesArray
TCollection
TDatime
TDirectory
TFile
TGraph
TH1D
TH1F
TH1
TH2D
TH2F
TH2
TKey
TLeafB
TLeafC
TLeafD
TLeafElement
TLeafF
TLeafI
TLeaf
TLeafL
TLeafObject
TLeafO
TLeafS
TList
TMap
TNamed
TObjArray
TObject
TProfile
TSeqCollection
TStreamerBase
TStreamerBasicPointer
TStreamerBasicType
TStreamerElement
TStreamerInfo
TStreamerLoop
TStreamerObjectAny
TStreamerObject
TStreamerObjectPointer
TStreamerString
TString
TTree
</hidden>

One can browser histogram (or ROOT objects) using this:

<code python>
import rootio
rootio.HBrowser("histograms_root5.root") # browser and plot ROOT histograms 
rootio.Browser("histograms_root5.root")  # browser for ROOT objects
</code>

You will see the browsers:

{{:hepsim:browser_histograms.png?300|Histogram browser}}

{{:hepsim:root_browser.png?300|Object browser}}

The script that call these browsers  can be put into Jython or Groovy files and executed as binary programs.


To read ROOT files from Delphes simulations, use 
[[https://atlaswww.hep.anl.gov/asc/jas4pp/api/rootio/Delphes.html|Delphes class]] and its "get" methods,
getFloat, getDouble, getInt, and getBool. But first, explore the Delphes ROOT file using the browser shown above.

Here is an example that uses high-level classes to read transverse momentum (PT) and pseudorapidity (ETA) of Monte Carlo particles:

 
<hidden Click here to show  a Python code to read Delphes files>
<code python>
from  hep.io.root import RootFileReader
from rootio import *

reader = RootFileReader("mg5_ttbar_Njet_001.root" )
tree = reader.get("Delphes");
maxevents=tree.getEntries()
branches=tree.getBranches()
print "Nr of events=",tree.getEntries()
print "Nr of branches=",branches.size()
print "Branches:"
for l in xrange( branches.size()  ):
   print "Branch=",l," name=",(branches.get(l)).getName()

particles=tree.getBranch("Particle") # get  generator particles

ptEvents=Delphes.getFloat(particles,"PT")   # get PT array
etaEvents=Delphes.getFloat(particles,"Eta") # get ETA array

from jhplot import *
h1 = H1D("PT of particles",100, 0, 100)
h2 = H1D("Eta of particles",100, -4, 4)
for i in xrange(tree.getEntries()):
       pt,eta=ptEvents[i],etaEvents[i]
       for j in xrange(len(pt)):
           h1.fill(pt[j])
           h2.fill(eta[j])
c=HPlot("pT",600,300,2,1)
c.visible()
c.setAutoRange()
c.draw(h1)

c.cd(2,1)
c.setAutoRange()
c.draw(h2)
</code>
</hidden>

{{:hepsim:pt_eta_delphes.png?300|}}

<note warning>Reading Delphes trees is an experimental feature. Under study </note>

====== Using GUI mode ======


You can also run Jas4pp in a GUI mode. You can process scripts that read PROMC and LCIO files from these HepSim pages:

  - [[https://atlaswww.hep.anl.gov/hepsim/macrolist.php|EVGEN validation scripts]]
  - [[https://atlaswww.hep.anl.gov/hepsim/macrolist_lcio.php|LCIO validation scripts]]


Download any of these scripts, and open it using the "jaspp" program. Then you can run the script using the right-mouse menu ("run script"). You will see the output below the main editor. Each run creates a new tab with the output. Look at this example:

{{:hepsim:jas4pp_gui.png?400|}}



====== Programming with HepSim ======


Jas4pp can be used to write Jython scripts to validate HepSim ProMC files. Please look at
[[https://atlaswww.hep.anl.gov/asc/promc/|Programming with ProMC files]] section.


====== Java API ======


===== Plots and histograms =====




Jas4pp uses histogram packages supported by both DMelt (community edition) or JAIDA (FreeHep).
DMelt provides programming API similar to PyROOT and with classes named conveniently to reduce code verbosity. 

Here are a few most common classes:

  * [[https://datamelt.org/api/doc.php/jhplot/HPlot|HPlot]] - canvas to show X-Y data and histograms in 2D
  * [[https://datamelt.org/api/doc.php/jhplot/HPlot|HPlot3D]] - canvas to show X-Y-Z data and histograms in 3D
  * [[https://datamelt.org/api/doc.php/jhplot/H1D|H1D]] - 1D histogram
  * [[https://datamelt.org/api/doc.php/jhplot/H1D|H2D]] - 2D histogram
  * [[https://datamelt.org/api/doc.php/jhplot/P1D|P1D]] - X-Y container with support of 2-level errors

The main canvas to show histograms [[http://jwork.org/dmelt/api/doc.php/jhplot/H1D|H1D]] and data points  [[http://jwork.org/dmelt/api/doc.php/jhplot/P1D|P1D]]  is [[http://jwork.org/dmelt/api/doc.php/jhplot/HPlot|HPlot]]. To process scripts in a background without a pop-up [[http://jwork.org/dmelt/api/doc.php/jhplot/HPlot|HPlot]], use method "visible(False)", and set sys.exit(0) at the end of the scripts.  You can also use the JAIDA to make histograms (Histogram1D or Histogram2D). 
In addition, data are saved in the form of XML (with the extension ".jdat") files. Look the manual [[https://handwiki.org/wiki/DMelt:Start]].

===== Lorentz particles and Jets =====

  * [[https://datamelt.org/api/doc.php/hephysics/hepsim/PromcUtil|PromcUtil]] convenient method to fill arrays with particles from ProMC files
  * [[https://datamelt.org/api/doc.php/hephysics/particle/LParticle|LParticle]] a HEP particle with the Lorentz transformations
  * [[https://datamelt.org/api/doc.php/hephysics/jet/FastParticle|FastParticle]] a HEP particles with precomputed Et2,Eta,Phi for jet algorithms. 
  * [[http://java.freehep.org/freehep-physics/apidocs/hep/physics/vec/package-summary.html|Physics vectors]] typical HEP physics vectors the Lorentz transformations
  * [[https://datamelt.org/api/doc.php/hephysics/jet/JetN2|JetN2]] recommended kt-type jet clustering ng algorithms (kT, anti-kT, CA) implemented in Java using N^2 approach, similar to the FastJet algorithm. Recommended for hadron collisions
  * [[https://datamelt.org/api/doc.php/hephysics/jet/SCJet|SCJet]] traditional kt-type jet clustering algorithms (kT, anti-kT) for pp implemented in Java using N^3 approach (slow) 
  * [[http://java.freehep.org/freehep-physics/apidocs/hep/physics/jet/package-summary.html|Jets and event shapes]] traditional jet algorithms and event shapes for e+e- from FreeHep (Geneva, Jade, Durham jets) 
  
  
Java implementation of the longitudinally invariant kT and anti-kT clustering algorithms uses the E-scheme to combine particles (p1+p2) and Eta-Phi space (not Rapidity-Phi). Also, Cambridge/Aachen jets are supported.   [[https://github.com/chekanov/hepjet/|HepJet on github]] shows  benchmarks of the Java implementation of the anti-KT with the original FastJet package.


You can build the standard kt-jets using [[http://datamelt.org/api/doc.php/hephysics/hepsim/PromcUtil | PromcUtil]], followed by the 
[[https://datamelt.org/api/doc.php/hephysics/jet/KTjet | KTjet]] class. 


====== Visualizing a detector Jas+Wired ======

You can run Jas+Wired to visualize the simulated events.  The Wired program is included in Jas4pp, so you simply run it as:
<code bash>
./jaspp
</code>
This will start a Jas3-like environment  with all needed plugins. 
Then copy the detector geometry file from the [[https://atlaswww.hep.anl.gov/hepsim/soft/detectors/| detector repository]].
<code bash>
wget https://atlaswww.hep.anl.gov/hepsim/soft/detectors/sidloi3.tgz -O - | tar -xz;
</code>
This will create a directory "sidloi3". This detector corresponds to "rfull001" tag used for the reconstruction of pythia6_zpole_ee (Z->e+e-). 

Now we can visualize the detector as **[File]-[Open data source]-[HepRep]** XML and select the file "sidloi3.heprep" from sidloi3.tgz:
This is how to do this using the command line:

<code bash>
./jaspp  sidloi3.heprep
</code>
You will see the detector layout:

{{:fcs:detector.png?400|Whired4 event display}}

Now, we will read the event: Open any *.slcio file you copied from HepSim as 
**[File]-[Open data source]-[LCIO]** file.  Then click a small button [Go] (top menu bar). It will process
 events.  Then select again **[File]-[New]-[Wired 4 view]**.  You will get an image in the Wired4 display.
Now press [Go] again to look at next event.
If you want to see how data records are organized inside the slcio file,
do this **[File]-[New]-[LCSim Event browser]**


Here is an example of how to visualize LCIO files from the [[http://atlaswww.hep.anl.gov/hepsim/|HepSim repository]]:

<code bash>
wget https://mc.hep.anl.gov/asc/hepsim/events/misc/pgun/pgun_eta35_muon//rfull009/pgun_muon1024gev_001_pandora.slcio
./jaspp pgun_muon1024gev_001_pandora.slcio
</code>

Click "Go" in the menu. Now you can open the event browser or Wired event display. Look at event record as
"File->New->Event Browser". Similarly, visualize the event as "  File->New->Wired4 View". It shows an black window on the left.
Then go to the next event, click "Go next" in the menu. This image illustrates a single electron event:

{{:hepsim:electron_event.png?400|Whired4 event display}}

Similarly, you can view any events form the HepSim directory. The LCIO event file automatically installs
the required detector geometry, i.e.  when you open a SLCIO file from the HepSim, you do not need to load the geometry file. The geometry file will be downloaded automatically from HepSim and will be put to ".lcsim/cache/" inside your home directory. 



====== Visualizing geometry using ROOT ======
(contribution from N.Nikiforou):

You can also visualize the detector using ROOT. Here are a few steps:
Locate the file detector.gdml inside the file detector.zip (where "detector" shows the name of the detector). 


<hidden show more details here>
This file is created using this command during the detector design stage
<code>
slic -g detector.lcdd -G detector.gdml
</code>
</hidden>

Once you have the GDML file, you can use ROOT to visualize it. You just
need to make sure the ROOT installation has openGL/GDML support (CERN
AFS installations have it for sure). At the ROOT prompt do:

<code cpp>
TGeoManager::Import("detector.gdml");
gGeoManager->GetTopVolume()->Draw("ogl");
</code>
This should popup an OpenGL display with the detector which you can
clip, pan, rotate etc. You will the image as shown here:

{{:asc:image.png?400|}}


====== People ======

  * S.Chekanov (ANL, main developer)
  * E.May (ANL, debugging, Wired4)
  * Gagik Gavalian (JLab)


====== Additional help ======


  * [[http://jas.freehep.org/jas3/]] - JAS3
  * [[http://java.freehep.org/]] - FreeHep
  * [[http://datamelt.org]] - DataMelt
  * [[https://confluence.slac.stanford.edu/display/ilc/ILC+Detector+Simulation+FAQ]] - SLIC
  * [[https://confluence.slac.stanford.edu/display/ilc/LCSim+Tutorials]] - SLIC
  * [[https://atlaswww.hep.anl.gov/asc/promc/]] ProMC file format
  * [[http://lcio.desy.de/]] LCIO file format



 --- //[[chekanov@anl.gov|Sergei Chekanov]] 2016/03/23 21:53//