Platform configuration

By default, vvtest assumes the machine on which it is running is a multi-core workstation. The np parameter in each test is used to determine the number of cores per test to occupy, and defaults to one if not specified.

However, for machines with GPUs or cluster machines using a batch system, vvtest must know a few things about the machine in order to identify resources and execute each test with some amount of concurrency.

The platform attributes are

• plat: the platform name
• numcores: number of cores on which to run concurrently (only applies to workstation mode)
• numdevices: number of devices on which to run concurrently (workstation mode)
• maxcores: maximum number of cores allowed for any single test
• maxdevices: maximum number of devices allowed for any single test
• batchsys: the batch system name, such as "slurm", "lsf", "pbs" (required for batch mode)
• cores_per_node: number of cores per compute node on cluster machines (required for batch mode)
• devices_per_node: number of devices per compute node on cluster machines
• queue: the queue name (a.k.a. partition name) for submitting batch jobs
• account: the account name for submitting batch jobs
• walltime: number of seconds to request for each batch job
• maxsubs: maximum number of concurrent batch jobs submitted to the queue
• maxqtime: maximum seconds allowed for each batch job submission
• submit_flags: arbitrary command line options passed through to the batch submit command

At this time, plat, numcores and numdevices must be given on the vvtest command line (using --plat, -n and --devices options). The rest can be given on the command line, specified by environment variables, or specified in the idplatform.py plugin file.

Specifying platform attributes on the command line

Here are the platform attributes that can be given directly to the vvtest command line.

direct option
--plat NAME	sets the platform name, plat
-n NUMCORES	sets numcores
-N MAXCORES	set maxcores
--devices NUMDEVICES	sets numdevices
--max-devices MAXDEVICES	sets maxdevices

All platform attributes except plat, numcores and numdevices can be specified using the option --platopt NAME=VALUE. Examples,

example
--platopt batchsys=slurm	sets batchsys to "slurm"
--platopt batchsys=slurm --platopt cores_per_node=36	sets batchsys to "slurm" and cores_per_node to 36
--platopt "submit_flags=-V --licenses=gpf1"	sets the submit_flags to "-V --licenses=gpfs1"

Multiple --platopt options are allowed and the values are accumulated.

Specifying platform attributes using environment variables

The platform name can be set using the VVTEST_PLATFORM environment variable. The platform attributes can be set using the VVTEST_PLATFORM_SPECS environment variable, whose value takes the same form as a --platopt value. For example,

$ export VVTEST_PLATFORM=Beowulf
$ export VVTEST_PLATFORM_SPECS="batchsys=pbs,cores_per_node=44,devices_per_node=4"
$ vvtest --batch --platopt queue=short ...

When both command line --platopt options and an environment variable is used, the attributes given on the command line take precedence.

A common use of these environment variables is for a project to define VVTEST_PLATFORM and VVTEST_PLATFORM_SPECS in an environment module for a given platform. Then developers and users get these variables defined correctly for each machine supported by the project. For example,

$ ssh supercluster
$ module load myapp
$ vvtest --batch ...

Specifying platform attributes in the idplatform.py plugin

When vvtest starts up, it looks in the configdir for a file named idplatform.py. If found, vvtest loads it and calls functions for determining the platform name, the compiler name and for setting platform attributes.

An idplatform.py file should take the form

def get_platform():
    return "ATS4"

def get_compiler( platname, options ):
    pass

def load_specifications( specs, platname, cplrname, options ):
    specs['batchsys'] = 'lsf'
    specs['cores_per_node'] = 32

where the 'options' argument is a list of the command line -o options, and 'specs' is a dictionary-like object used to hold the platform attributes.

• Not all three functions need to be provided. If a function is not provided, default handling will be performed instead.
• The get_platform() function should return the name of the platform, or None. A value of None causes vvtest to use its builtin logic to determine the platform name. The platform name is an arbitrary string, but the value can be referenced in individual tests for filtering purposes.
• The get_compiler() function should return the name of the compiler (an arbitrary string) or None. The resulting value is injected into the -o command line options and made available to each test. No default value is provided by vvtest.
• The load_specifications() function should fill the 'specs' object with platform attributes for the given platform name, 'platname'. If no modifications are made to 'specs', then vvtest will fill with defaults (if any exist for the platform).
• Command line platform attribute settings take precedence over the values specified in the idplatform.py file. Command line platform attributes are set in the 'specs' object prior to the load_specifications() call, so they are available. However, they are not overwritten (the first value set for an attribute will stick). To overwrite a value, use the specs.overwrite(key,value) method.
• The builtin platform attribute handling is in the config/idplatform.py file in the vvtest install location. That is where the SNL known platform attributes are located, as well as various ATS machine attributes.

To be clear, command line attributes take precedence, followed by environment variables, followed by values in the idplatform.py file.