ReFrame’s System Configuration File

feelpp.benchmarking makes use of ReFrame’s Configuration files to set up multiple HPC systems.

These files describe the system’s architecture, as well as the necessary modules, partitions, commands and environments for your applications to run as expected.

System configuration files can be provided as a JSON file, or as a Python script where the configuration is stored as a dictionary in a variable named site_configuration.

Very simple configuration file
site_configuration = {
    'systems': [
        {
            'name': 'my_system',
            'descr': 'Very simple example system',
            'hostnames':['py::socket.gethostname'],
            'partitions': [
                {
                    'name': 'my_partition',
                    'scheduler': 'local',
                    'launcher': 'mpiexec',
                    'environs': ['my_environment'],
                    'processor': { 'num_cpus': 4 }
                }
            ]
        }
    ],
    'environments': [
        {
            'name': 'my_environment',
            'modules': [],
            'target_systems': ['my_system:my_partition']
        }
    ]
}
Built-in supercomputers config

  • Discoverer. Sofia, Bulgaria.

  • Vega. Maribor, Slovenia.

  • MeluXina. Bissen, Luxembourg.

  • Karolina. Ostrava, Czechia.

  • Gaya. Strasbourg, France.

  • LUMI. Kajaani, Finland. [SOON]

There is no need to write system configurations for built-in systems. Users can specify them directly on the machine specification JSON.

1. System partitions and Environments

According to ReFrame’s documentation, "a system is an abstraction of an HPC system that is managed by a workload manager. A system can comprise multiple partitions, which are collection of nodes with similar characteristics". And "an environment is an abstraction of the environment where a test will run and it is a collection of environment variables, environment modules and compiler definitions".

ReFrame system architecture
Example 1. Karolina system

Karolina has defined a qcpu partition consisting of 720, and a qgpu partition consisting of 72 nodes equiped with GPU a accelerator. The entire list of Karolina’s partitions can be found here.

Now, a user might define a programming environment that uses Python3.8, and another environment that uses Python3.13.

2. System specific parameters

It can be useful to specify some configurations, like the maximum number of jobs that can be submitted asynchronously, additional launcher options, or the available memory per node.

Specify custom launcher options
{
    "systems":[{
        "partitions":[{
            "resources": [{
                "name":"launcher_options",
                "options":["-bind-to","core"]
            }]
        }]
    }]
}

feelpp.benchmarking requires to set the systems.partitions.processor.num_cpus value for each provided partition to indicate the maximum number of logical CPUs per partition’s node.
Karolina’s Configuration File
{
    "systems": [
        {
            "name": "karolina",
            "descr": "karolina",
            "hostnames": ["login\d+.karolina.it4i.cz","cn\d+.karolina.it4i.cz"],
            "modules_system": "lmod",
            "partitions": [
                {
                    "name": "qcpu",
                    "scheduler": "slurm",
                    "launcher": "srun",
                    "max_jobs": 8,
                    "access": ["--partition=qcpu"],
                    "environs": ["default"],
                    "processor": {
                        "num_cpus": 128
                    },
                    "devices": [
                        {
                            "type": "cpu",
                            "num_devices":829
                        }
                    ],
                    "container_platforms":[
                        {
                            "type": "Singularity"
                        }
                    ],
                    "extras":{
                        "memory_per_node":256
                    }
                }
            ]
        }
    ],
    "environments": [
        {
            "name": "default",
            "modules": ["OpenMPI/4.1.4-GCC-12.2.0","apptainer"],
            "target_systems": ["karolina:qcpu"]
        }
    ]
}