mcpgath
NAME
mcpgath - Takes files from each task of tasks 0 through task N and copies
them back in sequence to task 0.
SYNOPSIS
mcpgath [-ai] source ... destination [POE options]
Source is one of the following:
- one or more existing file names - files will be copied with the same
names to the destination directory on task 0. Each file name specified
must exist on all tasks involved in the copy.
- a directory name - all files in that directory on each task are copied
with the same names to the destination directory on task 0.
- an expansion of file names, using wildcards - files are copied with
the same names to the destination directory. All wildcarded input
strings must be enclosed in double quotes.
Destination is an existing destination directory name to where the data
will be copied. The destination directory must be the last item
specified before any POE flags.
FLAGS
-a An optional flag that appends the task number to the end of the file
name when it is copied to task 0. This is for task identification
purposes, to know where the data came from. The -a and -i flags can
be combined to check for existing files appended with the task number.
-i An optional flag that checks for duplicate or existing files of the
same name, and does not replace any existing file found. Instead,
issues an error message and continues with the remaining files to
be copied. The -a and -i flags can be combined to check for existing
files appended with the task number.
DESCRIPTION
The mcpgath function determines the list of files to be gathered on each
task. This function also resolves the source file, destination directory,
and path names with any meta characters, wildcard expansions, etc. to
come up with valid file names. Wildcards should be enclosed in double
quotes, otherwise they will be expanded locally on the task from where
the command is issued, which may not produce the intended file name
resolution.
mcpgath is a POE program and, therefore, all POE options are available.
You can set POE options with either command line flags or environment
variables. The number of nodes to copy the file to (-procs), and the
message passing protocol used to copy the file (-euilib) are the POE
options of most interest.
Note:
A default of 100K data block size is used for copying the data. This can
be changed by updating the source found in /usr/lpp/ppe.poe/samples/mpi,
and compiling it with the mpcc command.
Return codes are:
-1 invalid number of arguments specified
-2 invalid option flag specified
-3 unable to resolve input file name(s)
-4 could not open input file for read
-5 no room on destination node's file system
-6 error opening file output file
-7 error creating output file
-8 error writing to output file
-9 MPI_Send of data failed
-10 final MPI_Send failed
-11 MPI_Recv failed
EXAMPLES
1. You can copy a single file from all tasks into the destination directory.
For example, enter:
mcpgath -a hello_world /tmp -procs 4
This will copy the file hello_world (assuming it is a file and not a
directory) from tasks 0 through 3 as to task 0:
From task 0: /tmp/hello_world.0
From task 1: /tmp/hello_world.1
From task 2: /tmp/hello_world.2
From task 3: /tmp/hello_world.3
2. You can specify any number of files as source files. The destination
directory must be the last item specified before any POE flags. For
example:
mcpgath -a file1.a file2.a file3.a file4.a file5.a /tmp -procs 4
will take file1.a through file5.a from the local directory on each
task and copy them back to task 0. All files specified must exist on
all tasks involved. The file distribution will be as follows:
From Task 0: /tmp/file1.a.0
From Task 1: /tmp/file1.a.1
From Task 2: /tmp/file1.a.2
From Task 3: /tmp/file1.a.3
From Task 0: /tmp/file2.a.0
From Task 1: /tmp/file2.a.1
From Task 2: /tmp/file2.a.2
From Task 3: /tmp/file2.a.3
From Task 0: /tmp/file3.a.0
From Task 1: /tmp/file3.a.1
From Task 2: /tmp/file3.a.2
From Task 3: /tmp/file3.a.3
From Task 0: /tmp/file4.a.0
From Task 1: /tmp/file4.a.1
From Task 2: /tmp/file4.a.2
From Task 3: /tmp/file4.a.3
From Task 0: /tmp/file5.a.0
From Task 1: /tmp/file5.a.1
From Task 2: /tmp/file5.a.2
From Task 3: /tmp/file5.a.3
3. You can specify wildcard values to expand into a list of files to be
gathered. For this example, assume the following distribution of files
before calling mcpgath:
Task 0 contains file1.a and file2.a
Task 1 contains file1.a only
Task 2 contains file1.a, file2.a, and file3.a
Task 3 contains file4.a, file5.a, and file6.a
Enter:
mcpgath -a "file*.a" /tmp -procs 4
This will pass the wildcard expansion to each task, which will
resolve into the list of locally existing files to be copied. This
will result in the following distribution of files on task 0:
From Task 0: /tmp/file1.a.0
From Task 0: /tmp/file2.a.0
From Task 1: /tmp/file1.a.1
From Task 2: /tmp/file1.a.2
From Task 2: /tmp/file2.a.2
From Task 2: /tmp/file3.a.2
From Task 3: /tmp/file4.a.3
From Task 3: /tmp/file5.a.3
From Task 3: /tmp/file6.a.3
4. You can specify a directory name as the source, from which the files to
be gathered are found. For this example, assume the following
distribution of files before calling mcpgath:
Task 0 /test contains file1.a and file2.a
Task 1 /test contains file1.a only
Task 2 /test contains file1.a and file3.a
Task 3 /test contains file2.a, file4.a, and file5.a
Enter:
mcpgath -a /test /tmp -procs 4
This results in the following file distribution:
From Task 0: /tmp/file1.a.0
From Task 0: /tmp/file2.a.0
From Task 1: /tmp/file1.a.1
From Task 2: /tmp/file1.a.2
From Task 2: /tmp/file3.a.2
From Task 3: /tmp/file2.a.3
From Task 3: /tmp/file4.a.3
From Task 3: /tmp/file5.a.3