NCSA wiki will be offline Friday, Apr 19, 2024, from 1700 hours until Sun, Apr 21, 2024 in order to upgrade Confluence.
mpiP lightweight scalable MPI profiling is available from mpip.sourceforge.net .
To setup mpiP in your $HOME (so that it goes with the mpi implementation you prefer), follow steps similar to the ones below:
./configure --with-cc=mpiicc --with-f77=mpiifort --disable-libunwind --prefix=$HOME/mpip_3.3
# intel compiler versions of mpicc and mpif77 contain an extra "i" in them on iforge
make
make install
Then compile/link with mpiP (include -g for source line number information in the mpiP report ):
mpiicc -g cpi-sync.c -L ~/mpip_3.3/lib -lmpiP -lbfd
Run the code as usual. This example was run from within an interactive PBS batch job on iforge [ qsub -I -V -lnodes=2:ppn=4,walltime=00:30:00 ]:
[arnoldg@iforge116 src]$ mpirun -np 8 ./a.out mpiP: mpiP: mpiP: mpiP V3.3.0 (Build May 29 2012/12:57:33) mpiP: Direct questions and errors to mpip-help@lists.sourceforge.net mpiP: ... Process 6 received "Hi, how are you?" PI is approximately 3.1415926535896133 mpiP: mpiP: Storing mpiP output in [./a.out.8.11834.1.mpiP]. mpiP:
View the resulting text file report. The default format is shown, but there are many options to mpiP profiling available through the MPIP environment variable . Note that the ID number under the callsites will be the Site number in all the report sections that follow. Also, there are measurements for each rank (task) and averages are denoted by '*' .
@ mpiP @ Command : ./a.out @ Version : 3.3.0 @ MPIP Build date : May 29 2012, 12:57:33 @ Start time : 2012 05 29 13:47:20 @ Stop time : 2012 05 29 13:47:21 @ Timer Used : PMPI_Wtime @ MPIP env var : [null] @ Collector Rank : 0 @ Collector PID : 11834 @ Final Output Dir : . @ Report generation : Single collector task @ MPI Task Assignment : 0 iforge116 @ MPI Task Assignment : 1 iforge116 @ MPI Task Assignment : 2 iforge116 @ MPI Task Assignment : 3 iforge116 @ MPI Task Assignment : 4 iforge115 @ MPI Task Assignment : 5 iforge115 @ MPI Task Assignment : 6 iforge115 @ MPI Task Assignment : 7 iforge115 --------------------------------------------------------------------------- @--- MPI Time (seconds) --------------------------------------------------- --------------------------------------------------------------------------- Task AppTime MPITime MPI% 0 0.418 0.00967 2.31 1 0.41 0.00196 0.48 2 0.413 0.00436 1.05 3 0.413 0.0047 1.14 4 0.413 0.00386 0.94 5 0.413 0.00376 0.91 6 0.413 0.00365 0.89 7 0.418 0.00348 0.83 * 3.31 0.0354 1.07 --------------------------------------------------------------------------- @--- Callsites: 5 --------------------------------------------------------- --------------------------------------------------------------------------- ID Lev File/Address Line Parent_Funct MPI_Call 1 0 cpi-sync.c 31 calc_pi Bcast 2 0 cpi-sync.c 93 main Bcast 3 0 cpi-sync.c 84 main Recv 4 0 cpi-sync.c 46 calc_pi Reduce 5 0 cpi-sync.c 79 main Send --------------------------------------------------------------------------- @--- Aggregate Time (top twenty, descending, milliseconds) ---------------- --------------------------------------------------------------------------- Call Site Time App% MPI% COV Bcast 2 18.5 0.56 52.31 0.52 Reduce 4 8.57 0.26 24.19 2.45 Send 5 5.96 0.18 16.82 0.60 Recv 3 1.98 0.06 5.58 0.00 Bcast 1 0.39 0.01 1.10 0.50 --------------------------------------------------------------------------- @--- Aggregate Sent Message Size (top twenty, descending, bytes) ---------- --------------------------------------------------------------------------- Call Site Count Total Avrg Sent% Bcast 2 8 199 24.9 41.72 Send 5 7 182 26 38.16 Reduce 4 8 64 8 13.42 Bcast 1 8 32 4 6.71 --------------------------------------------------------------------------- @--- Callsite Time statistics (all, milliseconds): 32 --------------------- --------------------------------------------------------------------------- Name Site Rank Count Max Mean Min App% MPI% Bcast 1 0 1 0.072 0.072 0.072 0.02 0.74 Bcast 1 1 1 0.055 0.055 0.055 0.01 2.80 Bcast 1 2 1 0.044 0.044 0.044 0.01 1.01 Bcast 1 3 1 0.02 0.02 0.02 0.00 0.43 Bcast 1 4 1 0.082 0.082 0.082 0.02 2.12 Bcast 1 5 1 0.066 0.066 0.066 0.02 1.76 Bcast 1 6 1 0.038 0.038 0.038 0.01 1.04 Bcast 1 7 1 0.013 0.013 0.013 0.00 0.37 Bcast 1 * 8 0.082 0.0488 0.013 0.01 1.10 Bcast 2 0 1 0.042 0.042 0.042 0.01 0.43 Bcast 2 1 1 1.72 1.72 1.72 0.42 87.82 Bcast 2 2 1 3.74 3.74 3.74 0.91 85.91 Bcast 2 3 1 3.81 3.81 3.81 0.92 81.09 Bcast 2 4 1 2.15 2.15 2.15 0.52 55.58 Bcast 2 5 1 2 2 2 0.48 53.18 Bcast 2 6 1 2.59 2.59 2.59 0.63 71.03 Bcast 2 7 1 2.48 2.48 2.48 0.59 71.30 Bcast 2 * 8 3.81 2.32 0.042 0.56 52.31 Recv 3 0 7 1.45 0.282 0.006 0.47 20.44 Recv 3 * 7 1.45 0.282 0.006 0.06 5.58 Reduce 4 0 1 7.58 7.58 7.58 1.81 78.38 Reduce 4 1 1 0.095 0.095 0.095 0.02 4.84 Reduce 4 2 1 0.051 0.051 0.051 0.01 1.17 Reduce 4 3 1 0.333 0.333 0.333 0.08 7.09 Reduce 4 4 1 0.18 0.18 0.18 0.04 4.66 Reduce 4 5 1 0.223 0.223 0.223 0.05 5.93 Reduce 4 6 1 0.08 0.08 0.08 0.02 2.19 Reduce 4 7 1 0.035 0.035 0.035 0.01 1.00 Reduce 4 * 8 7.58 1.07 0.035 0.26 24.19 Send 5 1 1 0.089 0.089 0.089 0.02 4.53 Send 5 2 1 0.519 0.519 0.519 0.13 11.91 Send 5 3 1 0.535 0.535 0.535 0.13 11.40 Send 5 4 1 1.45 1.45 1.45 0.35 37.64 Send 5 5 1 1.47 1.47 1.47 0.36 39.13 Send 5 6 1 0.94 0.94 0.94 0.23 25.74 Send 5 7 1 0.952 0.952 0.952 0.23 27.32 Send 5 * 7 1.47 0.851 0.089 0.18 16.82 --------------------------------------------------------------------------- @--- Callsite Message Sent statistics (all, sent bytes) ------------------- --------------------------------------------------------------------------- Name Site Rank Count Max Mean Min Sum Bcast 1 0 1 4 4 4 4 Bcast 1 1 1 4 4 4 4 Bcast 1 2 1 4 4 4 4 Bcast 1 3 1 4 4 4 4 Bcast 1 4 1 4 4 4 4 Bcast 1 5 1 4 4 4 4 Bcast 1 6 1 4 4 4 4 Bcast 1 7 1 4 4 4 4 Bcast 1 * 8 4 4 4 32 Bcast 2 0 1 17 17 17 17 Bcast 2 1 1 26 26 26 26 Bcast 2 2 1 26 26 26 26 Bcast 2 3 1 26 26 26 26 Bcast 2 4 1 26 26 26 26 Bcast 2 5 1 26 26 26 26 Bcast 2 6 1 26 26 26 26 Bcast 2 7 1 26 26 26 26 Bcast 2 * 8 26 24.88 17 199 Reduce 4 0 1 8 8 8 8 Reduce 4 1 1 8 8 8 8 Reduce 4 2 1 8 8 8 8 Reduce 4 3 1 8 8 8 8 Reduce 4 4 1 8 8 8 8 Reduce 4 5 1 8 8 8 8 Reduce 4 6 1 8 8 8 8 Reduce 4 7 1 8 8 8 8 Reduce 4 * 8 8 8 8 64 Send 5 1 1 26 26 26 26 Send 5 2 1 26 26 26 26 Send 5 3 1 26 26 26 26 Send 5 4 1 26 26 26 26 Send 5 5 1 26 26 26 26 Send 5 6 1 26 26 26 26 Send 5 7 1 26 26 26 26 Send 5 * 7 26 26 26 182 --------------------------------------------------------------------------- @--- End of Report -------------------------------------------------------- ---------------------------------------------------------------------------