ATLAS Support Center

User Tools

Site Tools

Trace: • agence_web_la_rochelle • vlhc • sherpa • rabbitmq • workbook_svn • kylespurgeon • dev_fast
hpc:alpgenbuildingnotes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
hpc:alpgenbuildingnotes [2014/01/09 15:58]
jchilders 		hpc:alpgenbuildingnotes [2014/03/24 13:08] (current)
jchilders
Line 1: Line 1:
 ===== Building ALPGEN ===== ===== Building ALPGEN =====
  
- +==== Machine dependent compile time changes ====
-==== Compile and Fix Asinh Bug ==== +
------ +
-=== For BG/P & BG/Q ===  +
-  +
-For now, use the IBM compiler. That requires <fontcolor red>''compile.mk''</fontcolor> +
-to have the following changes +
-<code fortran> +
-ifeq ($(shell uname),Linux) +
-#  FFF = gfortran -fno-automatic +
-#  FF90 = gfortran -fno-automatic +
-  FFF  = mpixlf77  -fno-automatic +
-  FF90 =  mpixlf77  -fno-automatic +
-#FFF = g77 -O1 -Wall -fno-automatic -Wno-globals -fno-backslash \ +
-#        -ffast-math +
-endif +
-</code> +
-After a  +
-<code>make gen</code> +
-or +
-<code>make gen90</code> +
-fails, one needs to do the following: +
-<code> +
-mpixlf77  -o wqqgen ./wqqusr.o ../wqqlib/wqq.o \ +
-../alplib/alpgen.o ../alplib/alputi.o ../alplib/alppdf.o \ +
-../alplib/Acp.o ../alplib/Asu3.o ../alplib/Aint.o +
-</code> +
-(or similar) to pass the linking phase.+
  
  
 ===For BG/Q=== ===For BG/Q===
- +---- 
-Run on the command line: +On the command line, run \\ 
-<code>soft add  +mpiwrapper-xl</code> +''soft add  +mpiwrapper-xl'' \\ 
-and change +and change the following in the file ''compile.mk'' \\
-<code>compile.mk</code> +
-to+
 <code fortran> <code fortran>
 ifeq ($(shell uname),Linux) ifeq ($(shell uname),Linux)
Line 44: Line 15:
   FFF = mpif77 -fno-automatic   FFF = mpif77 -fno-automatic
   FF90 = mpif90 -fno-automatic   FF90 = mpif90 -fno-automatic
 +# Or if you want optimization use this
 +#  FFF = mpixlf77 -O5 -qarch=qp -qtune=qp -qsave -qsimd=auto -qhot=level=1 -qprefetch -qunroll=yes -qnoipa -qfloat=rsqrt:hssngl:fltint
 +#  FF90 = mpixlf90 -O5 -qarch=qp -qtune=qp -qsave -qsimd=auto -qhot=level=1 -qprefetch -qunroll=yes -qnoipa -qfloat=rsqrt:hssngl:fltint
 #FFF = g77 -O1 -Wall -fno-automatic -Wno-globals -fno-backslash \ #FFF = g77 -O1 -Wall -fno-automatic -Wno-globals -fno-backslash \
 #        -ffast-math #        -ffast-math
 endif endif
 </code> </code>
 +
 On line 182 of the file ''alplib/alpgen.f'' change \\ On line 182 of the file ''alplib/alpgen.f'' change \\
-''access='append' '' to ''position='append' ''+''access='append' '' \\ 
 +to \\ 
 +''position='append' '' \\
  
-There's no getting around the bug by switching compilers on the Q. +Now, Alpgen creates its own ''asinh()'' function. Alternating versions of Fortran have an asinh() function or does not have it. The version installed on the BGP does not have it, so there is no conflict. However, the BGQ version has the ''asinh()'' defined so we need to replace every mention of ''asinh()'' in Alpgen to some unique name. In this case, we chose ''argsignh()''. Use this command: 
-Replace asinh with argsinh whenever it appears: +<code bash>
-  - alpgen.f +
-  - zjet.f +
-can use this command:+
 find . -exec grep -l "asinh" {} \; | xargs -I GGGG sh -c 'sed "s/asinh/argsinh/g" GGGG > GGGG.tmp; mv GGGG.tmp GGGG' find . -exec grep -l "asinh" {} \; | xargs -I GGGG sh -c 'sed "s/asinh/argsinh/g" GGGG > GGGG.tmp; mv GGGG.tmp GGGG'
 +</code>
  
-ran into error: +You will get error: \\ 
-ld: -f may not be used without -shared +''ld: -f may not be used without -shared'' \\ 
-to fix it one needs to do the following: +to fix it one needs to do the following: \\ 
 +<code bash>
 mpixlf77  -o wqqgen ./wqqusr.o ../wqqlib/wqq.o \ mpixlf77  -o wqqgen ./wqqusr.o ../wqqlib/wqq.o \
 ../alplib/alpgen.o ../alplib/alputi.o ../alplib/alppdf.o \ ../alplib/alpgen.o ../alplib/alputi.o ../alplib/alppdf.o \
 ../alplib/Acp.o ../alplib/Asu3.o ../alplib/Aint.o ../alplib/Acp.o ../alplib/Asu3.o ../alplib/Aint.o
 +</code>
 (or similar) to pass the linking phase. (or similar) to pass the linking phase.
  
- +=== Carver === 
-Carver:  +---- 
-Change compile.mk to have the following changes: +Change ''compile.mk'' to have the following changes: 
 +<code make>
 module swap pgi gcc module swap pgi gcc
 module swap openmpi openmpi-gcc module swap openmpi openmpi-gcc
 +</code>
 Then everything is fine. Then everything is fine.
  
-Hopper: +=== Hopper === 
- +----
-module swap PrgEnv-pgi PrgEnv-gnu +
- +
-That requires compile.mk to have the following changes+
  
 +Run \\
 +''module swap PrgEnv-pgi PrgEnv-gnu'' \\
 +That requires ''compile.mk'' to have the following changes
 +<code make>
 ifeq ($(shell uname),Linux) ifeq ($(shell uname),Linux)
 #  FFF = gfortran -fno-automatic #  FFF = gfortran -fno-automatic
Line 91: Line 67:
 #        -ffast-math #        -ffast-math
 endif endif
 +</code>
  
- +=== Edison === 
-Edison +---- 
- +Run \\ 
-module swap PrgEnv-intel PrgEnv-gnu +''module swap PrgEnv-intel PrgEnv-gnu'' \\ 
- +That requires ''compile.mk'' to have the following changes 
-That requires compile.mk to have the following changes +<code make>
 ifeq ($(shell uname),Linux) ifeq ($(shell uname),Linux)
 #  FFF = gfortran -fno-automatic #  FFF = gfortran -fno-automatic
Line 107: Line 83:
 #        -ffast-math #        -ffast-math
 endif endif
 +</code>
  
 +==== Introduce MPI to Alpgen ====
  
-2) Deal with MPI +=== Edits to ''alplib/alpgen.f'' === 
- +----
-2.1) Add the lines+
  
 +In subroutine ''alsdef'' in file ''alplib/alpgen.f'' near line 209, just after \\
 +''c common declarations'' \\
 +add/merge these lines
 +<code fortran>
       integer mpirank       integer mpirank
       character*5 rankstring       character*5 rankstring
       common/mpi/mpirank,rankstring       common/mpi/mpirank,rankstring
- +       
-Just after the line "c common declarations" in subroutine alsdef in file alpgen.f +c local variables 
-It will be near line 209 +      double precision dummy,ranset,ranset2 
- +      integer i,retval,chdir 
-2.2 Add the lines +c 
 +</code> 
 +---- 
 +Near line 230, just before \\ 
 +''c list processes'' \\ 
 +add the lines 
 +<code fortran>
 C MPI variables C MPI variables
       include 'mpif.h'       include 'mpif.h'
       integer rank, size, ierror, tag, status(MPI_STATUS_SIZE)       integer rank, size, ierror, tag, status(MPI_STATUS_SIZE)
- +</code> 
-Just before the line "c list processes".  This will be near line 230 +Following the remaining variable definitions, add this code 
 +<code fortran>
 c beginning of MPI block c beginning of MPI block
 +c  retrieve MPI rank information
       call MPI_INIT(ierror)       call MPI_INIT(ierror)
       call MPI_COMM_SIZE(MPI_COMM_WORLD, size, ierror)       call MPI_COMM_SIZE(MPI_COMM_WORLD, size, ierror)
Line 135: Line 122:
       call MPI_FINALIZE(ierror)       call MPI_FINALIZE(ierror)
       write(6,*) 'MPI finalized: rank ', mpirank       write(6,*) 'MPI finalized: rank ', mpirank
-      write( rankstring, '(i5.5)' )  mpirank+      write( rankstring, '(i8.8)' )  mpirank 
 +c work in a unique directory ./work/rankstring 
 +      retval = chdir('work/'//rankstring//CHAR(0)) 
 +      if(retval.ne.0) then 
 +         write(6,*) 'Error changing directory to "work/rankstring" ' 
 +      endif
 c end of MPI block c end of MPI block
- +</code> 
- +---- 
-2.3) Add the lines +Just after the lines (near line 338) 
- +<code fortran>
-c work in a unique directory ./work/rankstring +
-      call chdir("work") +
-      call chdir(rankstring) +
- +
-Immediately after the end of the MPI block +
- +
-2.4) Just after the lines: +
- +
 c     save parameter values to relative variables in common blocks c     save parameter values to relative variables in common blocks
       call alspar       call alspar
- +</code>
-This will be near line 338: +
 Add the lines Add the lines
 +<code fortran>
 c beginning of MPI block c beginning of MPI block
       iseed(1)=iseed(1)+mod(mpirank,65536)       iseed(1)=iseed(1)+mod(mpirank,65536)
Line 162: Line 143:
       print*, 'Seeds and mpirank', iseed(1),iseed(2),mpirank       print*, 'Seeds and mpirank', iseed(1),iseed(2),mpirank
 c end of MPI block c end of MPI block
- +</code> 
- +---- 
- +In subroutine ''alstio'', just after 
- +<code fortran> 
- +      include 'alpgen.inc' 
-2.3 In subroutine alstio, just after  include 'alpgen.inc'+</code>
 (near line 4584) add the lines: (near line 4584) add the lines:
 +<code fortran>
       integer mpirank       integer mpirank
       character*5 rankstring       character*5 rankstring
       common/mpi/mpirank,rankstring       common/mpi/mpirank,rankstring
- +</code> 
- +---- 
-2.3 In subroutine alstio, replace +Near line 4588, replace 
-  +<code fortran> 
-call alustc(fname,'.wgt',tmpstr) +      call alustc(fname,'.wgt',tmpstr) 
 +</code>
 with  with 
- +<code fortran> 
-call alustc('../../'//fname,'.'//rankstring//'.wgt',tmpstr) +      call alustc('../../'//fname,'.'//rankstring//'.wgt',tmpstr) 
 +</code>
 immediately after immediately after
- +<code fortan> 
-         write(6,*) 'Run parameters and diagnostics written to '+      write(6,*) 'Run parameters and diagnostics written to '
      $        ,tmpstr(1:l)      $        ,tmpstr(1:l)
- +</code> 
- +This outputs the weighted event files in a common directory 
-Near line 4588. This outputs the weighted event files in a common directory +---- 
- +Replace unit ''5'' with unit ''9'' for input near line 306, after \\ 
-3) Replace unit 5 with unit 9 for input +''C MANDATORY INPUTS add'' \\ 
- +For BlueGene 
-Near line 306, after C MANDATORY INPUTS add +<code fortran>
-for BlueGene+
       open(9,file='../../input.1',action='read')        open(9,file='../../input.1',action='read') 
-for Carver+</code> 
 +For Carver 
 +<code fortran>
       open(9,file='../../input.1',access='read')        open(9,file='../../input.1',access='read') 
- +</code> 
-This reads the input configuration from file, instead of from the command line input via a redirect as in the tutorial +This reads the input configuration from file, instead of from the command line input via a redirect as in the tutorial. 
- +Then replace all \\ 
-Then replace all READ(5, with READ(9, +''READ(5,'' \\ 
- +with \\ 
- +''READ(9,'' 
- +This ensures everywhere is reading from the input file instead of ''stdin''. Only change this in the ''alplib/alpgen.f'' file.
  
hpc/alpgenbuildingnotes.1389283092.txt.gz · Last modified: 2014/01/09 15:58 by jchilders

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: Public Domain
Public Domain Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki