Differences

This shows you the differences between two versions of the page.

--- hpc:alpgenbuildingnotes [2014/01/09 15:58] – jchilders
+++ hpc:alpgenbuildingnotes [2014/03/24 13:08] (current) – jchilders
@@ Line 1: / Line 1: @@
 ===== Building ALPGEN =====
+==== Machine dependent compile time changes ====
-==== Compile and Fix Asinh Bug ====
------
-=== For BG/P & BG/Q ===
-For now, use the IBM compiler. That requires <fontcolor red>''compile.mk''</fontcolor>
-to have the following changes
-<code fortran>
-ifeq ($(shell uname),Linux)
-#  FFF = gfortran -fno-automatic
-#  FF90 = gfortran -fno-automatic
-  FFF  = mpixlf77  -fno-automatic
-  FF90 =  mpixlf77  -fno-automatic
-#FFF = g77 -O1 -Wall -fno-automatic -Wno-globals -fno-backslash \
-#        -ffast-math
-endif
-</code>
-After a
-<code>make gen</code>
-or
-<code>make gen90</code>
-fails, one needs to do the following:
-<code>
-mpixlf77  -o wqqgen ./wqqusr.o ../wqqlib/wqq.o \
-../alplib/alpgen.o ../alplib/alputi.o ../alplib/alppdf.o \
-../alplib/Acp.o ../alplib/Asu3.o ../alplib/Aint.o
-</code>
-(or similar) to pass the linking phase.
 ===For BG/Q===
+----
-Run on the command line:
+On the command line, run \\
-<code>soft add  +mpiwrapper-xl</code>
+''soft add  +mpiwrapper-xl'' \\
-and change
+and change the following in the file ''compile.mk'' \\
-<code>compile.mk</code>
-to
 <code fortran>
 ifeq ($(shell uname),Linux)
@@ Line 44: / Line 15: @@
   FFF = mpif77 -fno-automatic
   FF90 = mpif90 -fno-automatic
+# Or if you want optimization use this
+#  FFF = mpixlf77 -O5 -qarch=qp -qtune=qp -qsave -qsimd=auto -qhot=level=1 -qprefetch -qunroll=yes -qnoipa -qfloat=rsqrt:hssngl:fltint
+#  FF90 = mpixlf90 -O5 -qarch=qp -qtune=qp -qsave -qsimd=auto -qhot=level=1 -qprefetch -qunroll=yes -qnoipa -qfloat=rsqrt:hssngl:fltint
 #FFF = g77 -O1 -Wall -fno-automatic -Wno-globals -fno-backslash \
 #        -ffast-math
 endif
 </code>
 On line 182 of the file ''alplib/alpgen.f'' change \\
-''access='append' '' to ''position='append' ''
+''access='append' '' \\
+to \\
+''position='append' '' \\
-There's no getting around the bug by switching compilers on the Q.
+Now, Alpgen creates its own ''asinh()'' function. Alternating versions of Fortran have an asinh() function or does not have it. The version installed on the BGP does not have it, so there is no conflict. However, the BGQ version has the ''asinh()'' defined so we need to replace every mention of ''asinh()'' in Alpgen to some unique name. In this case, we chose ''argsignh()''. Use this command:
-Replace asinh with argsinh whenever it appears:
+<code bash>
-  - alpgen.f
-  - zjet.f
-can use this command:
 find . -exec grep -l "asinh" {} \; | xargs -I GGGG sh -c 'sed "s/asinh/argsinh/g" GGGG > GGGG.tmp; mv GGGG.tmp GGGG'
+</code>
-ran into error:
+You will get error: \\
-ld: -f may not be used without -shared
+''ld: -f may not be used without -shared'' \\
-to fix it one needs to do the following:
+to fix it one needs to do the following: \\
+<code bash>
 mpixlf77  -o wqqgen ./wqqusr.o ../wqqlib/wqq.o \
 ../alplib/alpgen.o ../alplib/alputi.o ../alplib/alppdf.o \
 ../alplib/Acp.o ../alplib/Asu3.o ../alplib/Aint.o
+</code>
 (or similar) to pass the linking phase.
+=== Carver ===
-Carver:
+----
-Change compile.mk to have the following changes:
+Change ''compile.mk'' to have the following changes:
+<code make>
 module swap pgi gcc
 module swap openmpi openmpi-gcc
+</code>
 Then everything is fine.
-Hopper:
+=== Hopper ===
+----
-module swap PrgEnv-pgi PrgEnv-gnu
-That requires compile.mk to have the following changes
+Run \\
+''module swap PrgEnv-pgi PrgEnv-gnu'' \\
+That requires ''compile.mk'' to have the following changes
+<code make>
 ifeq ($(shell uname),Linux)
 #  FFF = gfortran -fno-automatic
@@ Line 91: / Line 67: @@
 #        -ffast-math
 endif
+</code>
+=== Edison ===
-Edison
+----
+Run \\
-module swap PrgEnv-intel PrgEnv-gnu
+''module swap PrgEnv-intel PrgEnv-gnu'' \\
+That requires ''compile.mk'' to have the following changes
-That requires compile.mk to have the following changes
+<code make>
 ifeq ($(shell uname),Linux)
 #  FFF = gfortran -fno-automatic
@@ Line 107: / Line 83: @@
 #        -ffast-math
 endif
+</code>
+==== Introduce MPI to Alpgen ====
-) Deal with MPI
+=== Edits to ''alplib/alpgen.f'' ===
+----
-.1) Add the lines
+In subroutine ''alsdef'' in file ''alplib/alpgen.f'' near line 209, just after \\
+''c common declarations'' \\
+add/merge these lines
+<code fortran>
       integer mpirank
       character*5 rankstring
       common/mpi/mpirank,rankstring
-Just after the line "c common declarations" in subroutine alsdef in file alpgen.f
+c local variables
-It will be near line 209
+      double precision dummy,ranset,ranset2
+      integer i,retval,chdir
-.2 Add the lines
+c
+</code>
+----
+Near line 230, just before \\
+''c list processes'' \\
+add the lines
+<code fortran>
 C MPI variables
       include 'mpif.h'
       integer rank, size, ierror, tag, status(MPI_STATUS_SIZE)
+</code>
-Just before the line "c list processes".  This will be near line 230
+Following the remaining variable definitions, add this code
+<code fortran>
 c beginning of MPI block
+c  retrieve MPI rank information
       call MPI_INIT(ierror)
       call MPI_COMM_SIZE(MPI_COMM_WORLD, size, ierror)
@@ Line 135: / Line 122: @@
       call MPI_FINALIZE(ierror)
       write(6,*) 'MPI finalized: rank ', mpirank
-      write( rankstring, '(i5.5)' )  mpirank
+      write( rankstring, '(i8.8)' )  mpirank
+c work in a unique directory ./work/rankstring
+      retval = chdir('work/'//rankstring//CHAR(0))
+      if(retval.ne.0) then
+         write(6,*) 'Error changing directory to "work/rankstring" '
+      endif
 c end of MPI block
+</code>
+----
-.3) Add the lines
+Just after the lines (near line 338)
+<code fortran>
-c work in a unique directory ./work/rankstring
-      call chdir("work")
-      call chdir(rankstring)
-Immediately after the end of the MPI block
-.4) Just after the lines:
 c     save parameter values to relative variables in common blocks
       call alspar
+</code>
-This will be near line 338:
 Add the lines
+<code fortran>
 c beginning of MPI block
       iseed(1)=iseed(1)+mod(mpirank,65536)
@@ Line 162: / Line 143: @@
       print*, 'Seeds and mpirank', iseed(1),iseed(2),mpirank
 c end of MPI block
+</code>
+----
+In subroutine ''alstio'', just after
+<code fortran>
+      include 'alpgen.inc'
-.3 In subroutine alstio, just after  include 'alpgen.inc'
+</code>
 (near line 4584) add the lines:
+<code fortran>
       integer mpirank
       character*5 rankstring
       common/mpi/mpirank,rankstring
+</code>
+----
-.3 In subroutine alstio, replace
+Near line 4588, replace
+<code fortran>
-call alustc(fname,'.wgt',tmpstr)
+      call alustc(fname,'.wgt',tmpstr)
+</code>
 with
+<code fortran>
-call alustc('../../'//fname,'.'//rankstring//'.wgt',tmpstr)
+      call alustc('../../'//fname,'.'//rankstring//'.wgt',tmpstr)
+</code>
 immediately after
+<code fortan>
-         write(6,*) 'Run parameters and diagnostics written to '
+      write(6,*) 'Run parameters and diagnostics written to '
      $        ,tmpstr(1:l)
+</code>
+This outputs the weighted event files in a common directory
-Near line 4588. This outputs the weighted event files in a common directory
+----
+Replace unit ''5'' with unit ''9'' for input near line 306, after \\
-) Replace unit 5 with unit 9 for input
+''C MANDATORY INPUTS add'' \\
+For BlueGene
-Near line 306, after C MANDATORY INPUTS add
+<code fortran>
-for BlueGene
       open(9,file='../../input.1',action='read')
-for Carver
+</code>
+For Carver
+<code fortran>
       open(9,file='../../input.1',access='read')
+</code>
-This reads the input configuration from file, instead of from the command line input via a redirect as in the tutorial
+This reads the input configuration from file, instead of from the command line input via a redirect as in the tutorial.
+Then replace all \\
-Then replace all READ(5, with READ(9,
+''READ(5,'' \\
+with \\
+''READ(9,''
+This ensures everywhere is reading from the input file instead of ''stdin''. Only change this in the ''alplib/alpgen.f'' file.