| SP Parallel Programming Workshop |
| v i s u a l i z a t i o n t o o l ( v t ) |
| What is the Visualization Tool? |
| What Does VT Look Like? |
| Trace Visualization Window
| VT View Selector Window
|
This is the "main" VT window. Its primary purpose is for monitoring and controlling the playback of trace files.
Several sets of analog (VCR-like) controls and two pulldown menus are used to:
Used to select the desired views for both trace visualization and performance monitoring.
Both an individual view or, an entire view group can be selected by a single mouse click on the desired icon. An entire view group can be "deselected" by a single click also.
If performance monitor is selected (discussed later), a second View Selector window will appear. Use it instead of the initial one.
| Visualization Tool Views |
| Using VT for Trace Visualization |
| About Trace Records and Files |
| Using VT for Trace Visualization |
| How to Generate Trace Files |
The table below describes run time events which can be traced, and the corresponding flag which you will need to use to set VT's trace level.
| Flag | Message Passing | Collective Communication | AIX Kernel Statistics | PM Array |
|---|---|---|---|---|
| 0 | ||||
| 1 | 
| |||
| 2 |
|
| ||
| 3 |
|
|
| |
| 9 |
|
|
|
|
| This step is only needed if you want to select specific areas within your program for tracing. It can be skipped if you want the entire program to be automatically traced. |
Use the VT trace "start" and "stop" routines shown below to turn on and turn off trace record generation anywhere within your program. The flag argument matches what you selected in the previous step. For example:
| Fortran |
INTEGER FLAG, RC
C Turn tracing on
FLAG = 3
CALL VT_TRC_START(FLAG,RC)
..... program execution ......
C Turn tracing off
CALL VT_TRC_STOP(RC)
|
| C Language |
#include <VT_trc.h>
int flag, rc;
/* Turn tracing on */
flag = 3;
rc = VT_trc_start_c(flag);
..... program execution ......
/* Turn tracing off */
rc = VT_trc_stop_c();
|
Use either mpcc or mpxlf to compile your program. Note: If you plan on using the "Source Code" view, you will need the -g option.
Examples:
mpcc myprogram.c -o myprogram
mpcc -g myprogram.c -o myprogram
mpxlf myprogram.f -o myprogram
mpxlf -g myprogram.f -o myprogram
Minimally, you must set the MP_TRACELEVEL environment variable. By default, tracing is turned off within POE. To generate a trace file, you need to run your program with tracing turned on.
Examples:
setenv MP_TRACELEVEL 3 - csh, tcsh shells
export MP_TRACELEVEL=9 - sh, bsh, ksh shells
If you are controlling trace record generation from within your program, the integer you use with the MP_TRACELEVEL environment variable and its associated flags are the same ones you use on the VT_trc_start statement within your programs - if they do not match, the lesser value is used.
If you are NOT controlling trace record generation from within your program, then just setting MP_TRACELEVEL is adequate.
Other VT environment variables may also be set to specify temporary file sizes, name, etc. These are discussed later.
As your program executes on each SP node, trace records are created for execution events on that node. The trace level setting determines which events cause trace record generation.
Three-tiered memory buffering approach is used for storing trace files
Trace files from all nodes are merged after execution into a single serialized collection on your home (login) node. This is the file which VT can use to display your application's execution events.
By default, VT will name your trace file the same as the program name with the suffix .trc added.
Optional: When running your program, you can specify a command-line flag (such as -tlevel) to override its associated environment variable.
| Using VT for Trace Visualization |
| Additional Options |
To specify a trace file name other than the default, you can set the environment variables MP_TRACEDIR and MP_TRACEFILE or use the -tracedir/tdir and -tracefile/-tfile flags when invoking the program. As with all PE command-line flags, they temporarily override their associated environment variable. Note that the default value of MP_TRACEDIR is the current directory and that you are required to set this variable if using a different directory.
Examples:
setenv MP_TRACEDIR /u/user3/test1
setenv MP_TRACEFILE tfile
-or-
myprogram -tracedir /u/user3/test1 -tracefile tfile
By default, VT writes the memory buffer to a file /tmp/username. You can override this by setting the MP_TMPDIR environment variable or using the -tmpdir flag when invoking the program. Note that the -tmpdir flag temporarily overrides its associated environment variable.
Examples:
setenv MP_TMPDIR /u/user4/mytempdir
-or-
myprogram -tmpdir /u/user4/mytempdir
If you are generating trace records for AIX Kernel Statistics, VT gets a sampling of those statistics at set intervals. By default, the set interval is every ten milliseconds.
You can change the sampling interval by setting the environment variable MP_SAMPLEFREQ, or using the -samplefreq or -sfreq flag when invoking the program. Note that the -samplefreq and -sfreq temporarily override their associated environment variable.
Examples:
setenv MP_SAMPLEFREQ 50
-or-
myprogram -sfreq 50
More frequent sampling is more detailed, but requires more storage.
If storage is a consideration, set the sampling frequency to a higher
value.
By default, the system uses the following three-tiered approach to manage the storage of trace files:
You have several options regarding the storage of trace files. You can specify:
Examples:
setenv MP_TBUFFSIZE 5M
setenv MP_TTEMPSIZE 20M
setenv MP_TPERMSIZE 300M
-or-
myprogram -tbuffsize 20M -ttempsize 50M -tpermsize 300M
Examples:
setenv MP_TBUFFWRAP yes
-or-
myprogram -tbuffwrap yes
You may use this VT tracing routine (either C or Fortran version) within your application code, instead of environment variables and command-line flags, to accomplish all of the options just discussed. Please see the "IBM AIX Parallel Environment Operation and Use" manual for more information.
| Using VT for Trace Visualization |
| Playing Trace Files |
You can start VT with the name of your trace file to have it automatically load that file for playback. For example:
vt -tfile mytrace.file
Alternately, if you start VT without specifying a tracefile, you can use the Trace Visualization window's "File" pull-down menu or the trace file "Select" button to select a trace file.
Keep in mind that the time is the original processing time, not the current trace file's playback time.
As you play the trace file, a Playback Position Line moves from left to right to show your current playback position.
You can change the current playback position by dragging the small triangle at the base of the Playback Position Line to a new position within the Trace File Time Control.
Two range markers allow you to set the range of playback within your trace file.
| Using VT for Performance Monitoring |
Select "Performance Monitor" from the "File" pull-down menu of the Trace Visualization window. This will cause the Performance Monitor window to appear.
A second window will also appear, called the Performance Monitor View Selector. This window is very similar to the VT View Selector window in appearance and function. You may want to close the original View Selector window to avoid confusion. (See Visualization Tool Views to review the description on VT views.)
Performance Monitor
| Performance Monitor View Selector
|
|
Note: the meaning of these colors/symbols can be displayed at any time by selecting "Legends" from the Performance Monitor's "File" pull-down menu. |
First, select one or more views from the VT Performance Monitor View Selector window. This is required before monitoring can begin.
Then, from the "File" pull-down menu of the Performance Monitor, select "Toggle Monitoring". The Statistics Collector daemon(s) on the selected processor node(s) will begin sending samplings of AIX kernel statistics to VT. All open views will begin visualizing these statistics.
If communication between VT and a selected node cannot be established by the network, it may take a few minutes for the network to respond with an error.
If you select additional processor nodes or additional views after you have already started monitoring, you must stop and restart monitoring. Use "Toggle Monitoring" from the "File" pull-down menu.
The default sampling frequency interval, expressed in seconds, appears in the "Frequency" box at the bottom of the Performance Monitor. You can change this by simply entering a new value into the small box.
For the new value to have effect, you must stop and restart monitoring. Use "Toggle Monitoring" from the "File" pull-down menu.
Select "Close Performance Monitor" from the Performance Monitor's "File" pull-down menu. The Performance Monitor View Selector window can be closed by selecting "Done" from its "Actions" pull-down menu.
| Adjusting a View |
Place the mouse cursor in any open view and press the right mouse button. A pop-up menu will appear showing you which parameters and configuration settings you may change.
When doing trace visualization or performance monitoring, views redisplay using new information from VT after set intervals. By default, this interval is one second.
To change this rate, select the "Config" option from the pop-up menu. The view's Time Resolution window will open.
Enter the new display rate and press "OK". The view will now display itself at the new display rate.
To change a view's colors, select the "Parameters" option from the pop-up menu. The view's Key Spectrum window will open.
Click on the spectrum to see what other spectrums are available. When the desired spectrum is displayed, press "OK" or "APPLY". The view will now display itself with the new colors.
Note that the colors used in the VT views are requested from the default X-Windows color map. The appearance of your views could be changed if the default color map is changed. To avoid this, instruct VT to create a private copy of the color map using the -cmap flag on the vt command when starting VT.
These will differ from view to view and are not discussed here.
| Creating and Using Your Own VT Configuration File |
Enter the name you want to give the configuration file. VT will save the file as specified.
Locate and then select the name of the desired configuration file. VT will load the specified file.
You can also load a configuration file when starting a VT session by using the -cfile or -configfile flags with the vt command.
| Index of VT Views |
The following index is arranged according to VT view groups. A graphical example of the group views can be obtained by clicking on the group name. Text descriptions of the individual views can be obtained by clicking on the view name.
| References and More Information |