hepsim:dev_hepsim
Differences
This shows you the differences between two versions of the page.
— | hepsim:dev_hepsim [2024/07/01 21:25] (current) – created - external edit 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | {{indexmenu_n> | ||
+ | [[: | ||
+ | |||
+ | ====== Linking event storage ====== | ||
+ | |||
+ | Monte Carlo files from any location can be published on [[https:// | ||
+ | HepSim is not a storage, but a catalog of files stored in multiple URL locations. It is your responsibility | ||
+ | to maintain Monte Carlo data visible from HepSim. | ||
+ | |||
+ | * to index files so they will be available from HepSim; | ||
+ | * to make mirrors of data on a server close to your location for faster download; | ||
+ | |||
+ | |||
+ | If the dataset is new, you will see a new entry in the HepSim database. Your authorship will be preserved on the HepSim web page. | ||
+ | If you have made a mirror of the existing data, your server will be added as a mirror for a given dataset. | ||
+ | |||
+ | If you created | ||
+ | to HepSim after you index them as explained below. In this case you do not need to maintain | ||
+ | |||
+ | |||
+ | ====== What do you need ====== | ||
+ | |||
+ | If you decide to publish Monte Carlo files (or make a mirror of the existing files), you need the following: | ||
+ | |||
+ | * A web server that can hold your data (>1 TB for / | ||
+ | * Linux OS (any flavor) | ||
+ | * Apache2 with PHP module that can serve/ | ||
+ | * Java 8 JDK or JRE (optionally, | ||
+ | |||
+ | |||
+ | You there is no web server, one can add files to the " | ||
+ | |||
+ | |||
+ | The root directory of a typical HepSim repository has the name " | ||
+ | be served by Apache, i.e. it should be visible from / | ||
+ | |||
+ | <code bash> | ||
+ | |-events | ||
+ | | |-pp # process type: pp, ee, ep, misc (single particles) | ||
+ | | | |-14tev | ||
+ | | | | |-qcd_pythia8 # this directory contains ProMC files for EVGEN | ||
+ | | | | | |-rfast001 # subdirectory with ROOT/ | ||
+ | | | | | |-rfull001 # subdirectory with SLCIO files | ||
+ | | | | | |-macros | ||
+ | </ | ||
+ | (it shows a data sample for 14 TeV pp with the name " | ||
+ | The directory " | ||
+ | |||
+ | |||
+ | After populating the directory " | ||
+ | Download the example from {{: | ||
+ | Assuming that you are in the directory above the directory " | ||
+ | |||
+ | <code bash> | ||
+ | wget https:// | ||
+ | tar -zvxf hepsim_web.tgz | ||
+ | cd hepsim_web/ | ||
+ | wget https:// | ||
+ | source hs-toolkit/ | ||
+ | </ | ||
+ | |||
+ | This package has 2 directories: | ||
+ | * **events** directory | ||
+ | * ** web_post** directory should be outside the Apache area. This directory is used to index your files in the directory " | ||
+ | |||
+ | This package has 2 example files in " | ||
+ | |||
+ | |||
+ | Now, index all ProMC/ | ||
+ | Then run "bash ./ | ||
+ | It will process the directory " | ||
+ | |||
+ | To make sure that everything works, look with the browser in the HTTP link " | ||
+ | |||
+ | Now you can: | ||
+ | |||
+ | * populate the directory tree with your files (for a new dataset). ProMC is used for EVGEN files, abd SLCIO/ROOT files for directories " | ||
+ | * copy existing files from the HepSim and put them to the correct directory (will be mirror) | ||
+ | |||
+ | In both cases, make sure that Linux system administrator set the correct permission for the directory " | ||
+ | |||
+ | If you have indexed the files successfully, | ||
+ | |||
+ | |||
+ | ====== How to mirror entire dataset ====== | ||
+ | |||
+ | You can mirror an entire dataset by copying HepSim files to your mirror web server (or local computer). | ||
+ | You can use the command " | ||
+ | to mirror a dataset with known URL, use this example: | ||
+ | |||
+ | <code bash> | ||
+ | wget https:// | ||
+ | source hs-toolkit/ | ||
+ | # now copy a dataset from URL to a new location | ||
+ | SOURCE=https:// | ||
+ | OUTPUT_DIR=/ | ||
+ | hs-mirror -i $SOURCE -o $OUTPUT_DIR | ||
+ | </ | ||
+ | This example creates the directory "/ | ||
+ | URL. The URL can be found using the HepSim web page. | ||
+ | |||
+ | If you want to download only EVGEN files, without reconstructed events, use this command: | ||
+ | <code bash> | ||
+ | hs-mirror -i $SOURCE -o $OUTPUT_DIR -t evgen | ||
+ | </ | ||
+ | |||
+ | |||
+ | Generally, you do not need to index files in the mirror directory. | ||
+ | |||
+ | ====== How to index directories | ||
+ | |||
+ | If you have made new files for HepSim (and added new file to the mirrored directory), you will need to index all files | ||
+ | Assume your files are located inside the directory " | ||
+ | |||
+ | <code bash> | ||
+ | wget https:// | ||
+ | source hs-toolkit/ | ||
+ | hs-index events/ | ||
+ | </ | ||
+ | |||
+ | If you need to re-index all subdirectories inside the directory " | ||
+ | |||
+ | <code bash> | ||
+ | hs-index events | ||
+ | </ | ||
+ | |||
+ | This script creates: | ||
+ | |||
+ | * " | ||
+ | * " | ||
+ | * " | ||
+ | |||
+ | |||
+ | ====== Using Petrel from Globus====== | ||
+ | |||
+ | Monte Carlo files can be added to the " | ||
+ | grid [[https:// | ||
+ | You will need a [[https:// | ||
+ | |||
+ | You can run your personal Petrel projects (with enabled https) with the files with HepSim-supported files, | ||
+ | or you can add your files to the existing petrel# | ||
+ | The directory with Monte Carlo files should be properly indexed as described above, and then you can copy such directory to Petrel Web-based storage using the transfer tools. | ||
+ | |||
+ | A typical workflow is following: | ||
+ | |||
+ | * create directory with MC files on OSG or other resources | ||
+ | * index files using the tools described above | ||
+ | * Move the directory with the files to a petrel project (or ask to be included to petrel# | ||
+ | * Send a URL location of your files to [email protected] | ||
+ | |||
+ | |||
+ | ====== Summary ====== | ||
+ | |||
+ | When everything is done, this is what you should expect: | ||
+ | |||
+ | * Your files will be visible on the main HepSim web page and on the current mirrors (takes ~ few days to propagate the changes) | ||
+ | * You can search your files etc. using the main HepSim web page | ||
+ | * You can search, list and download files using hs-toolkit commands (hs-find, hs-ls, hs-get etc) | ||
+ | * Since your files are public, there is good chance that somebody will make a mirror to reduce the load on your server | ||
+ | * You server will be listed on the page [[http:// | ||
+ | |||
+ | |||
+ | |||
+ | --- // | ||
+ | --- // |