====== Sherpa Notes ====== [[ https://sherpa.hepforge.org | Sherpa Homepage ]] ''wget http://www.hepforge.org/archive/sherpa/SHERPA-MC-2.1.1.tar.gz'' In most cases opening ''Makefile'' and commenting out line 470/471 which have the words ''Manual'' and ''Examples'' on it will save you some time. <<< I frequently have to do 'autoreconf' before 'make' when doing a fresh install. Not sure why, just do. >>> ===== Some Notes ===== * The code generation step of Sherpa produces a ''makelibs'' compile script and many other ''Makefiles.in'' in the ''Process'' subdirectory that has full absolute path inside that points to the local Sherpa installation. Many of the ''configure*'' and ''automake'' files contain absolute paths to the directory in which the code generation step was executed. This causes problems if one tries to move the code generation to another system or folder. * 200900 = W+jets, 200901 = Z+jets, 200903 = Drell-Yan, 200910 = Multi-jets * During a run Sherpa dlopen() calls the following libraries: * LibCT10Sherpa.so (or whatever PDF you are using) * libSherpaOpenLoops.so * libProc2_2... * libopenloops.so * openloops/proclib/*.so * libSherpaHepMcOutput.so * Runtime Errors and Possible Meanings: * 'Couldn't find virtual ME for this process' -> look that it has the libSherpaOpenLoops.so and the proper openloops libraries. * 'Process not found' -> make sure Run.dat card is found, and if so, make sure Processes are defined in the card properly. ===== Compilation and Installation at Edison@NERSC ===== === Local Environment === Install [[hpc:hepmc | HepMC]]. After logging into a clean shell run module unload altd module unload darshan module unload PrgEnv-intel module load PrgEnv-gnu module unload cray-shmem module load scons module load mercurial module load cmake module load boost module load gsl === Compilation === Unpack Sherpa tarball and enter sherpa main directory. After configure comment out line 470 of the Makefile for Sherpa which has the word "Manual" on it. You can create a mostly-static or dynamic executable by using the '-static' and '--enable-static=yes', or '-dynamic' flags, respectively. Then run: ./configure --enable-mpi --enable-gzip --with-sqlite3=/usr --enable-hepmc2= --enable-openloops= --prefix=$PWD/install CC=cc CXX=CC FC='ftn -fPIC -funderscoring' make -j 10 make install If you go for the static flags the make will fail just before the executable is built so then do this: ./configure --enable-mpi --enable-hepmc2=/project/projectdirs/m2015/tools/HepMC-2.06.08/local/ --with-pic LDFLAGS=-static MPICXX=CC CC=cc CXX=CC --prefix=$PWD/install --enable-openloops=/project/projectdirs/m2015/tools/OpenLoops/ --enable-gzip --enable-static=yes --enable-shared=yes make -j 10 make -j 10 install cd SHERPA/Run ../../libtool --tag=CXX --mode=link CC -Wl,--no-as-needed -g -O2 -rdynamic -Wl,--no-as-needed -o Sherpa ../../*/*/*.o ../../*/*/*/*.o /project/projectdirs/m2015/tools/HepMC-2.06.08/src/*.o -lgfortran /usr/lib64/libz.a -lsqlite3 > libtool.log # this will fail with errors then run this sed s/"libtool: link: "//g libtool.log | sed s/"..\/..\/PDF\/CT10\/CT10_Fortran_Interface.o"//g | sed s/"..\/..\/PDF\/CT10\/CT10Pdf.o "//g | sed s/"..\/..\/PDF\/CT10\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12Pdf.o "//g | sed s/"..\/..\/PDF\/CT12\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CTEQ\/CTEQ6_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CTEQ\/Cteq6Pdf-2008.o "//g | sed s/"..\/..\/PDF\/CTEQ\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/Electron\/PDF_Electron.o "//g | sed s/"..\/..\/PDF\/Electron\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/GRV\/GRVph_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/GRV\/grvphoton.o "//g | sed s/"..\/..\/PDF\/GRV\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2001lo.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2004qed.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst99.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST01LO.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST04QED.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST99.o "//g | sed s/"..\/..\/PDF\/MRST\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MSTW\/mstwpdf.o "//g | sed s/"..\/..\/PDF\/MSTW\/PDF_MSTW.o "//g | sed s/"..\/..\/PDF\/MSTW\/SVN_Info.o "//g | sed s/"\/opt\/gcc\/4.9.2\/snos\/lib\/..\/lib64\/libquadmath.so"/"-lquadmath"/g | sed s/"\/opt\/gcc\/4.9.2\/snos\/lib\/..\/lib64\/libgfortran.so"/"-lgfortran"/g | sed s/"\/usr\/lib64\/libsqlite3.so"/"-lsqlite3"/g > compile_Sherpa.sh sh compile_Sherpa.sh cp Sherpa ../../install/bin/ === Test Run === * Ensure proper modules are loaded as listed above. * Create directory in ''Examples/test''. * Change to this folder and create the file ''Run.dat'' and fill it with this text: (run){ % general setting EVENTS 1; ERROR 0.99; EVENT_OUTPUT=HEPEVT[specialtest]; % scales, tags for scale variations SP_NLOCT 1; FSF:=1.; RSF:=1.; QSF:=1.; SCALES METS{FSF*MU_F2}{RSF*MU_R2}{QSF*MU_Q2}; % tags for process setup NJET:=2; LJET:=2; QCUT:=20.; % me generator settings ME_SIGNAL_GENERATOR Comix Amegic LOOPGEN; EVENT_GENERATION_MODE Weighted; LOOPGEN:=Internal; % exclude tau from lepton container MASSIVE[15] 1; % collider setup BEAM_1 2212; BEAM_ENERGY_1 = 4000.; BEAM_2 2212; BEAM_ENERGY_2 = 4000.; }(run) (processes){ Process 93 93 -> 90 91 93{NJET}; Order_EW 2; CKKW sqr(QCUT/E_CMS); NLO_QCD_Mode MC@NLO {LJET}; ME_Generator Amegic {LJET}; Loop_Generator LOOPGEN {LJET}; Scales LOOSE_METS{FSF*MU_F2}{RSF*MU_R2}{QSF*MU_Q2} {7,8}; End process; }(processes) (selector){ Mass 11 -12 5. E_CMS Mass 13 -14 5. E_CMS Mass -11 12 5. E_CMS Mass -13 14 5. E_CMS }(selector) }(run) * Start an interactive session: ''qsub -I -q default -l mppwidth=24'' * Once this session begins you'll be back in your home directory on the worker node. * Change directories back: ''cd $PBS_O_WORKDIR''. * Next run the initialization step of Sherpa which produces the makelibs file: ''aprun -n 1 -N 1 ../../install/bin/Sherpa'' * Once complete, exit the interactive session and run ''./makelibs'' ===== Compilation and Installation at Cooley@ALCF ===== * ''soft add +mvapich2'' * ''soft add +gcc-4.8.1'' * Unpack tarball and enter main sherpa directory. mkdir install ./configure --enable-gzip --enable-openloops=/projects/EnergyFEC/anl/cooley/tools/OpenLoops-r1354 --enable-hepmc2=/projects/EnergyFEC/anl/cooley/tools/HepMC-2.06.09/install --with-sqlite3=/projects/EnergyFEC/anl/cooley/tools/sqlite-autoconf-3080200/install --enable-mpi --enable-static=yes --prefix=$PWD/install * In the ''Makefile'' remove line 472 because tukey doesn't have the 'texi2html' app installed needed to compile the manual. SUBDIRS = \ ATOOLS \ BEAM \ PDF \ MODEL \ METOOLS \ PHASIC++ \ MCATNLO \ EXTRA_XS \ AMEGIC++ \ COMIX \ CSSHOWER++ \ AMISIC++ \ AHADIC++ \ HADRONS++ \ PHOTONS++ \ SHRiMPS \ SHERPA \ AddOns \ Manual \ # <---- this line Examples * Make & Install: make make install * Install [[http://www.scons.org/download.php|Scons]]. If you've installed OpenLoops, scons is in ''/path/to/OpenLoops/scons-local''. Scons is used to compile the process libraries, the code for which is written when you first call Sherpa with your run card. If you don't have scons, it tries to use ''autoconfig'' which can have difficulties sometimes whereas Scons does not. wget http://prdownloads.sourceforge.net/scons/scons-2.3.4.tar.gz # Unpack tarball and enter main scons directory mkdir install python setup.py install --prefix=$PWD/install export PATH=$PATH:$PWD/install/bin ===== Compiling on Mira/Cetus@ALCF ===== * soft add +mpiwrapper-gcc soft add +bgqtoolchain-gcc484 soft add +python-2.7 svn export svn+ssh://atlas1.hep.anl.gov/users/svn/hpc/generators/sherpa/v220/dev/tags/01-00-00 01-00-00 cd 01-00-00 ./configure --enable-mpi --with-pic --enable-gzip=/soft/libraries/alcf/current/gcc/ZLIB --enable-openloops= --with-sqlite3=install --enable-hepmc2= --host=powerpc64-bgq-linux CC=mpicc CFLAGS=-g CPPFLAGS=-g CXXFLAGS=-g MPICXX=mpicxx CXX=mpicxx FC='mpif90 -fPIC -funderscoring' FCFLAGS=-g LDFLAGS=-dynamic --prefix=$PWD/install # build and install sqlite3 # rerun configure with: --with-sqlite3= make -j 20 install * Hopefully, now everything is working. * You may need to copy the HepMC and sqlite3 libraries to the installed sherpa library directory. * I have run into these problems in the past: * /projects/EnergyFEC/anl/generators/sherpa/v220/rel-2-2-0_anl/SHERPA/PerturbativePhysics/.libs/libSherpaPerturbativePhysics.so: undefined reference to `std::__detail::_List_node_base::_M_hook(std::__detail::_List_node_base*)' /projects/EnergyFEC/anl/generators/sherpa/v220/rel-2-2-0_anl/SHERPA/PerturbativePhysics/.libs/libSherpaPerturbativePhysics.so: undefined reference to `std::__detail::_List_node_base::_M_unhook()' /projects/EnergyFEC/anl/generators/sherpa/v220/rel-2-2-0_anl/SHERPA/PerturbativePhysics/.libs/libSherpaPerturbativePhysics.so: undefined reference to `std::__detail::_List_node_base::_M_transfer(std::__detail::_List_node_base*, std::__detail::_List_node_base*)' /projects/EnergyFEC/anl/generators/sherpa/v220/rel-2-2-0_anl/AHADIC++/Formation/.libs/libAhadicFormation.so: undefined reference to `std::__detail::_List_node_base::_M_reverse()' ===== Compiling on Vesta@ALCF ===== svn export svn+ssh://atlas1.hep.anl.gov/users/svn/hpc/generators/sherpa/v211/bgq/tags/02-01-00 sherpa-02-01-00 ./configure --host=powerpc64-unknown-linux-gnu --with-sqlite3=install --enable-gzip=/soft/libraries/alcf/current/gcc/ZLIB --enable-openloops=/projects/ATLASQ/ol-mpi --enable-hepmc2=/home/projects/ATLASQ/tools/HepMC-2.06.08/install --enable-mpi --enable-shared=yes --prefix=$PWD/install --with-pic CC=mpicc CFLAGS=-pg LDFLAGS=-pg CPPFLAGS=-pg FC=mpif90 FCFLAGS=-pg MPICXX=mpicxx CXX=mpicxx CXXFLAGS=-pg This step will fail because it cannot compile sqlite3 without some help. So go in the sqlite3 directory and run the following. mkdir install ./configure --enable-static=yes --prefix=$PWD/install make make install Then move back to Sherpa root path and rerun the configure step. ./configure --host=ppc64 --enable-gzip=/soft/libraries/alcf/current/gcc/ZLIB --enable-openloops=/projects/ATLASQ/ol-mpi --enable-hepmc2=/home/projects/ATLASQ/tools/HepMC-2.06.08/install --enable-mpi --enable-shared=yes --prefix=$PWD/install --with-pic CC=mpicc CFLAGS=-pg LDFLAGS=-pg CPPFLAGS=-pg FC=mpif90 FCFLAGS=-pg MPICXX=mpicxx CXX=mpicxx CXXFLAGS=-pg make V=1 -j 10 > make.log 2>&1 & make -j 10 install For static build: /bin/sh ../../libtool --tag=CXX --mode=link mpicxx -Wl,--no-as-needed -g -O2 -pg -pg -dynamic -o Sherpa ../../*/*/*.o ../../*/*/*/*.o /home/projects/ATLASQ/tools/HepMC-2.06.08/src/*.o /home/projects/ATLASQ/generators/sherpa-02-01-00/sqlite-autoconf-3080200/install/lib/libsqlite3.a /soft/libraries/alcf/current/gcc/ZLIB/lib/libz.a -L/projects/ATLASQ/ol-mpi/lib -lopenloops -lcoli > libtool.log This will fail because the PDF libraries contain overloaded functions and cannot be all loaded together. Sherpa loads these dynamically so the ''sed'' command below removes all the PDF object files. sed s/"libtool: link: "//g libtool.log | sed s/"..\/..\/PDF\/CT10\/CT10_Fortran_Interface.o"//g | sed s/"..\/..\/PDF\/CT10\/CT10Pdf.o "//g | sed s/"..\/..\/PDF\/CT10\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12Pdf.o "//g | sed s/"..\/..\/PDF\/CT12\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CTEQ\/CTEQ6_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CTEQ\/Cteq6Pdf-2008.o "//g | sed s/"..\/..\/PDF\/CTEQ\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/Electron\/PDF_Electron.o "//g | sed s/"..\/..\/PDF\/Electron\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/GRV\/GRVph_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/GRV\/grvphoton.o "//g | sed s/"..\/..\/PDF\/GRV\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2001lo.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2004qed.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst99.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST01LO.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST04QED.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST99.o "//g | sed s/"..\/..\/PDF\/MRST\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MSTW\/mstwpdf.o "//g | sed s/"..\/..\/PDF\/MSTW\/PDF_MSTW.o "//g | sed s/"..\/..\/PDF\/MSTW\/SVN_Info.o "//g > compile_Sherpa.sh sh compile_Sherpa.sh Now to run a test. qsub -t 20 -n 1 --mode=c1 Sherpa -l run.codegen.log INIT_ONLY=1 qsub -t 20 -n 1 --mode=c4 Sherpa -l run.integration.log SQLITE_OPEN_FLAG=unix-none The SQLITE_OPEN_FLAG is explained here: https://sherpa.hepforge.org/trac/ticket/263?cversion=3&cnum_hist=2 ===== Compiling on Ascinode ===== tar xzf SHERPA-MC-2.1.1.tar.gz cd SHERPA-MC-2.1.1 ./configure --enable-gzip --enable-openloops=/users/hpcusers/tools/OpenLoops --enable-hepmc2=/users/hpcusers/svn/tools/HepMC-2.06.08/local --enable-mpi --enable-static=yes --enable-shared=yes --prefix=$PWD/install --with-pic make V=1 -j 10 > make.log 2>&1 & make install This will create the Sherpa executable but will have a dynamically linked executable. As seen here: ldd SHERPA/Run/.libs/Sherpa linux-vdso.so.1 => (0x00007ffff4bff000) libSherpaMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaMain.so.0 (0x00007f83841ea000) libSherpaInitialization.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaInitialization.so.0 (0x00007f8383fbe000) libSherpaPerturbativePhysics.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaPerturbativePhysics.so.0 (0x00007f8383d62000) libSherpaSoftPhysics.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaSoftPhysics.so.0 (0x00007f8383b36000) libLundTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libLundTools.so.0 (0x00007f8383935000) libSherpaTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaTools.so.0 (0x00007f838369b000) libAmisic.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmisic.so.0 (0x00007f838348e000) libAmisicModel.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmisicModel.so.0 (0x00007f8383251000) libAmisicTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmisicTools.so.0 (0x00007f838303c000) libCSTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libCSTools.so.0 (0x00007f8382e30000) libCSCalculators.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libCSCalculators.so.0 (0x00007f8382c04000) libCSShowers.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libCSShowers.so.0 (0x00007f83829d2000) libCSMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libCSMain.so.0 (0x00007f83827af000) libAhadicMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAhadicMain.so.0 (0x00007f83825a6000) libAhadicTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAhadicTools.so.0 (0x00007f8382326000) libAhadicFormation.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAhadicFormation.so.0 (0x00007f8382114000) libAhadicDecays.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAhadicDecays.so.0 (0x00007f8381f10000) libShrimpsMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsMain.so.0 (0x00007f8381d01000) libShrimpsEvents.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsEvents.so.0 (0x00007f8381ac1000) libShrimpsBeamRemnants.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsBeamRemnants.so.0 (0x00007f8381894000) libShrimpsXsecs.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsXsecs.so.0 (0x00007f8381674000) libShrimpsEikonals.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsEikonals.so.0 (0x00007f838144c000) libShrimpsTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libShrimpsTools.so.0 (0x00007f838123d000) libBeam.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libBeam.so.0 (0x00007f838101b000) libComixAmplitude.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libComixAmplitude.so.0 (0x00007f8380dd3000) libComixPhasespace.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libComixPhasespace.so.0 (0x00007f8380b8b000) libComix.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libComix.so.0 (0x00007f8380952000) libComixCluster.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libComixCluster.so.0 (0x00007f8380738000) libAmegic.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmegic.so.0 (0x00007f83804cc000) libDipoleSubtraction.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libDipoleSubtraction.so.0 (0x00007f838024e000) libAmplitude.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmplitude.so.0 (0x00007f837ffc6000) libAmegicPSGen.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmegicPSGen.so.0 (0x00007f837fd1d000) libZfunctions.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libZfunctions.so.0 (0x00007f837fa76000) libString.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libString.so.0 (0x00007f837f83e000) libAmegicCluster.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libAmegicCluster.so.0 (0x00007f837f61f000) libModelMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libModelMain.so.0 (0x00007f837f38c000) libModelInteractions.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libModelInteractions.so.0 (0x00007f837efc4000) libPDF.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPDF.so.0 (0x00007f837eda6000) libRemnant.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libRemnant.so.0 (0x00007f837eb8e000) libMCatNLOTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMCatNLOTools.so.0 (0x00007f837e988000) libMCatNLOCalculators.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMCatNLOCalculators.so.0 (0x00007f837e771000) libMCatNLOShowers.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMCatNLOShowers.so.0 (0x00007f837e548000) libMCatNLOMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMCatNLOMain.so.0 (0x00007f837e329000) libPhasicChannels.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicChannels.so.0 (0x00007f837e05e000) libPhasicMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicMain.so.0 (0x00007f837de0b000) libPhasicProcess.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicProcess.so.0 (0x00007f837db7e000) libPhasicSelectors.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicSelectors.so.0 (0x00007f837d8e3000) libPhasicScales.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicScales.so.0 (0x00007f837d694000) libPhasicEnhance.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicEnhance.so.0 (0x00007f837d48a000) libPhasicDecays.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhasicDecays.so.0 (0x00007f837d271000) libMEToolsExplicit.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsExplicit.so.0 (0x00007f837d046000) libMEToolsCurrents.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsCurrents.so.0 (0x00007f837ce16000) libMEToolsVertices.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsVertices.so.0 (0x00007f837cbea000) libMEToolsColors.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsColors.so.0 (0x00007f837c9db000) libMEToolsSpinCorrelations.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsSpinCorrelations.so.0 (0x00007f837c7ce000) libMEToolsLoops.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsLoops.so.0 (0x00007f837c5ab000) libMEToolsMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libMEToolsMain.so.0 (0x00007f837c378000) libHadronsMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libHadronsMain.so.0 (0x00007f837c148000) libHadronsMEs.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libHadronsMEs.so.0 (0x00007f837bf0a000) libHadronsCurrents.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libHadronsCurrents.so.0 (0x00007f837bc7e000) libHadronsPSs.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libHadronsPSs.so.0 (0x00007f837ba68000) libPhotonsMain.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhotonsMain.so.0 (0x00007f837b85d000) libPhotonsTools.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhotonsTools.so.0 (0x00007f837b63b000) libPhotonsPhaseSpace.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhotonsPhaseSpace.so.0 (0x00007f837b430000) libPhotonsMEs.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libPhotonsMEs.so.0 (0x00007f837b1ee000) libExtraXS.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXS.so.0 (0x00007f837afdb000) libExtraXS2_2.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXS2_2.so.0 (0x00007f837ad95000) libExtraXS1_2.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXS1_2.so.0 (0x00007f837ab8f000) libExtraXS1_3.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXS1_3.so.0 (0x00007f837a984000) libExtraXSCluster.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXSCluster.so.0 (0x00007f837a781000) libExtraXSNLO.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libExtraXSNLO.so.0 (0x00007f837a55b000) libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f837a24e000) libdl.so.2 => /lib64/libdl.so.2 (0x00000031bde00000) libSherpaSingleEvents.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libSherpaSingleEvents.so.0 (0x00007f8379feb000) libToolsPhys.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libToolsPhys.so.0 (0x00007f8379d8f000) libToolsMath.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libToolsMath.so.0 (0x00007f8379ada000) libToolsOrg.so.0 => /users/hpcusers/svn/generators/sherpa/v211/test/SHERPA-MC-2.1.1/install/lib/SHERPA-MC/libToolsOrg.so.0 (0x00007f837983d000) libz.so.1 => /lib64/libz.so.1 (0x00000031bea00000) libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00000031cf600000) libmpi_cxx.so.1 => /usr/lib64/openmpi/lib/libmpi_cxx.so.1 (0x00007f8379622000) libmpi.so.1 => /usr/lib64/openmpi/lib/libmpi.so.1 (0x00000031bf600000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000031c0e00000) libm.so.6 => /lib64/libm.so.6 (0x00000031be200000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000031c0600000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000031bda00000) libc.so.6 => /lib64/libc.so.6 (0x00000031bd600000) /lib64/ld-linux-x86-64.so.2 (0x00000031bd200000) libopen-rte.so.7 => /usr/lib64/openmpi/lib/libopen-rte.so.7 (0x00007f83793a6000) libopen-pal.so.6 => /usr/lib64/openmpi/lib/libopen-pal.so.6 (0x00000031bf200000) librt.so.1 => /lib64/librt.so.1 (0x00000031be600000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00000031c1a00000) libutil.so.1 => /lib64/libutil.so.1 (0x00000031cd600000) libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x00000031c6200000) To build a more statically linked executable do the following steps. cd SHERPA/Run ../../libtool --tag=CXX --mode=link mpic++ -Wl,--no-as-needed -g -O2 -rdynamic -Wl,--no-as-needed -o Sherpa ../../*/*/*.o ../../*/*/*/*.o /users/hpcusers/svn/tools/HepMC-2.06.08/src/*.o -lgfortran /usr/lib64/libz.a -lsqlite3 > libtool.log sed s/"libtool: link: "//g libtool.log | sed s/"..\/..\/PDF\/CT10\/CT10_Fortran_Interface.o"//g | sed s/"..\/..\/PDF\/CT10\/CT10Pdf.o "//g | sed s/"..\/..\/PDF\/CT10\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CT12\/CT12Pdf.o "//g | sed s/"..\/..\/PDF\/CT12\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/CTEQ\/CTEQ6_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/CTEQ\/Cteq6Pdf-2008.o "//g | sed s/"..\/..\/PDF\/CTEQ\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/Electron\/PDF_Electron.o "//g | sed s/"..\/..\/PDF\/Electron\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/GRV\/GRVph_Fortran_Interface.o "//g | sed s/"..\/..\/PDF\/GRV\/grvphoton.o "//g | sed s/"..\/..\/PDF\/GRV\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2001lo.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst2004qed.o "//g | sed s/"..\/..\/PDF\/MRST\/mrst99.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST01LO.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST04QED.o "//g | sed s/"..\/..\/PDF\/MRST\/PDF_MRST99.o "//g | sed s/"..\/..\/PDF\/MRST\/SVN_Info.o "//g | sed s/"..\/..\/PDF\/MSTW\/mstwpdf.o "//g | sed s/"..\/..\/PDF\/MSTW\/PDF_MSTW.o "//g | sed s/"..\/..\/PDF\/MSTW\/SVN_Info.o "//g > compile_Sherpa.sh sh compile_Sherpa.sh cp Sherpa ../../install/bin/ Now you have: ldd install/bin/Sherpa linux-vdso.so.1 => (0x00007fff8e5ff000) libgfortran.so.3 => /usr/lib64/libgfortran.so.3 (0x00007f63da4cc000) libsqlite3.so.0 => /usr/lib64/libsqlite3.so.0 (0x00000031cf600000) libmpi_cxx.so.1 => /usr/lib64/openmpi/lib/libmpi_cxx.so.1 (0x00007f63da2b1000) libmpi.so.1 => /usr/lib64/openmpi/lib/libmpi.so.1 (0x00000031bf600000) libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000031c0e00000) libm.so.6 => /lib64/libm.so.6 (0x00000031be200000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000031c0600000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00000031bda00000) libc.so.6 => /lib64/libc.so.6 (0x00000031bd600000) libdl.so.2 => /lib64/libdl.so.2 (0x00000031bde00000) libopen-rte.so.7 => /usr/lib64/openmpi/lib/libopen-rte.so.7 (0x00000031bea00000) libopen-pal.so.6 => /usr/lib64/openmpi/lib/libopen-pal.so.6 (0x00000031bf200000) librt.so.1 => /lib64/librt.so.1 (0x00000031be600000) libnsl.so.1 => /lib64/libnsl.so.1 (0x00000031c1a00000) libutil.so.1 => /lib64/libutil.so.1 (0x00000031cd600000) libltdl.so.7 => /usr/lib64/libltdl.so.7 (0x00000031c6200000) /lib64/ld-linux-x86-64.so.2 (0x00000031bd200000) ===== Patch Notes ===== Apply in this order: * gzip_2.patch * mpi_seed_mode_1.patch * one_lib_1.patch * remove_summarize_1.patch * resdir_1.patch Sometimes I have trouble applying these so just use a text editor to see what changes need to be applied. ===== Converting Sherpa HepMC_GenEvent Format to POOL EVNT Format ===== === Hack to Athena Algs === It was found that certain Truth containers are not created in Athena Reco transforms if the HepMC::GenEvent::signal_process_id is not greater than zero. Therefore the following hack describes a way to introduce this. I was using the following commands on ''ascinode.hep.anl.gov'' with Athena setup with the environment ''asetup 17.2.12.7.10,MCProd,64,slc5'', though this should work for other releases as well. Then do the following from your Athena home directory where the ''InstallArea'' will be created cmt show versions PhysicsAnalysis/TruthParticleID/McParticleTools Which ever version is shown for your relase do the following: cmt co McParticleTools-00-19-00 PhysicsAnalysis/TruthParticleID/McParticleTools cd McParticleTools-00-19-00 PhysicsAnalysis/TruthParticleID/McParticleTools/src/ Edit the file named ''HepMcReaderTool.cxx'': StatusCode HepMcReaderTool::read( HepMC::GenEvent* evt ) { m_ioFrontend->fill_next_event(evt); ////////// BEGIN NEW PARTS /////////////////// // required to be non-zero for TruthParticleContainer creation evt->set_signal_process_id(1); // taken from Generators/Sherpa_i/trunk/src/Sherpa_i.cxx:115 // ATLAS renormalizes the output Sherpa event weights if (evt->weights().size()>2) { evt->weights()[0]/=evt->weights()[2]; while (evt->weights().size()>1) evt->weights().pop_back(); } // Sherpa uses GeV, ATLAS uses MeV so convert all the momenta and energies. for (HepMC::GenEvent::particle_iterator p = evt->particles_begin(); p != evt->particles_end(); ++p) { // ATH_MSG_DEBUG("PDG, BAR " << (*p)->pdg_id() << " " << (*p)->barcode()); const HepMC::FourVector fv((*p)->momentum().px() * 1000, (*p)->momentum().py() * 1000, (*p)->momentum().pz() * 1000, (*p)->momentum().e() * 1000); (*p)->set_momentum(fv); (*p)->set_generated_mass(1000 * (*p)->generated_mass()); } ////////// END NEW PARTS /////////////////// return StatusCode::SUCCESS; } Then run: cd ../cmt gmake gmake install You should now have this installed in your ''InstallArea'' directory. === Job Options for converting === ===== Installing OpenLoops ===== On Edison@NERSC: module unload altd module unload darshan module unload PrgEnv-intel module load PrgEnv-gnu module unload cray-shmem module load scons module load mercurial module load cmake module load boost module load gsl ./scons -j 10 ./scons auto=all/ -j 15 On Mira/Cetus@ALCF: * soft add +bgqtoolchain-gcc484 soft add +mpiwrapper-gcc soft add +python2.7.9 * Checkout from ATLAS: svn co --username atlas https://openloops.hepforge.org/svn/OpenLoops/branches/public_collier atlas_ol * ''pmg4711'' * cd atlas_ol * Make these changes:Index: pyol/config/default.cfg =================================================================== --- pyol/config/default.cfg (revision 1831) +++ pyol/config/default.cfg (working copy) @@ -8,8 +8,8 @@ [OpenLoops] -supported_compilers = gfortran ifort -fortran_compiler = gfortran +supported_compilers = gfortran ifort mpif90 +fortran_compiler = mpif90 fortran_tool = auto cc = gcc cxx = g++ Index: lib_src/olcommon/src/kind_types.F90 =================================================================== --- lib_src/olcommon/src/kind_types.F90 (revision 1831) +++ lib_src/olcommon/src/kind_types.F90 (working copy) @@ -18,6 +18,7 @@ module kind_types + use ISO_FORTRAN_ENV implicit none integer, parameter :: intkind1 = selected_int_kind(2) @@ -35,7 +36,8 @@ complex(ep), parameter :: cI_ep = (0._ep, 1._ep) #endif ! #ifdef USE_qp - integer, parameter :: qp = selected_real_kind(33) +! integer, parameter :: qp = selected_real_kind(33) + integer, parameter :: qp = REAL128 complex(qp), parameter :: cI_qp = (0._qp, 1._qp) ! #endif Index: lib_src/openloops/src/ol_interface.F90 =================================================================== --- lib_src/openloops/src/ol_interface.F90 (revision 1831) +++ lib_src/openloops/src/ol_interface.F90 (working copy) @@ -1108,7 +1108,7 @@ if (index(trim(infofilename),"."//trim(info_file_suffix)) == 0) then cycle else - infofilename = trim(install_path) // "/proclib/" // trim(infofilename) + infofilename = trim(install_path) // "/proclib/" // infofilename(2:len(infofilename)) end if inquire(gf_info, opened=iqopen) * To build: ./scons -j 20 * To download and build all processes: ./scons proc=all/ -j 20 Setup: * On Tukey only soft add +gcc-4.8.1 * On SLC6 machine: scons needs gcc-4.6+ so use ATLAS compiler: * export PATH=/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/462/x86_64/slc6/x86_64-slc6-gcc46-opt/bin:$PATH * export LD_LIBRARY_PATH=/cvmfs/atlas.cern.ch/repo/sw/atlas-gcc/462/x86_64/slc6/x86_64-slc6-gcc46-opt/lib:$LD_LIBRARY_PATH - Get OpenLoops: svn checkout http://openloops.hepforge.org/svn/OpenLoops/branches/public ./OpenLoops - Run Scons: cd OpenLoops ./scons auto=all/ This downloads all the process libraries and takes some time. List of libraries.. - To compile OpenLoops as a static library, create the file openloops.cfg in the top-level directory. [OpenLoops] shared_libraries=0