JsonReport Schema

The root of the configuration file is the JsonReportSchema. It defines global metadata and the main content structure.

1. Core Fields

Field Type Description Default Value

title

string

The main title of the document.

null

datetime

string

A timestamp for the report generation.

Current date and time (YYYY-MM-DD HH:MM:SS)

data

array of objects

A list of external or internal data files to load (see Including Data).

[]

contents

array of objects

The ordered list of content nodes that make up the report (see Allowed Content Nodes).

[]

1.1. Example

{
  "title": "My Project Benchmarking Report",
  "datetime": "2025-11-28 10:00:00",
  "data": [
    // ... DataFile definitions ...
  ],
  "contents": [
    // ... Content Node definitions (Sections, Tables, Plots, Text) ...
  ]
}

2. Report Structure

The contents field defines the hierarchical structure and entire narrative flow of the final document. It is an ordered list of Content Nodes.

A report is primarily built using the Section Node, which contains a title and an array of nested content, allowing you to create chapters, sub-sections, and detailed analysis blocks.

2.1. Allowed Content Nodes

The engine supports the following node types, which can be placed directly in the root contents array or nested within a SectionNode:

Node Type Purpose See Documentation

section

Used for creating hierarchical structure (chapters, sections, subsections).

Section Node

grid

Layout node type used to structure elements into columns.

[Grid Node]

text

For static narrative text or embedding dynamic data values into sentences.

Text Node

table

Defines how to extract, process, and render a data file into a structured table.

Table Node

plot

Defines the parameters for generating a data visualization (figure).

Plot Node

image

For embedding pre-rendered images with captions.

Image Node

latex

Allows direct insertion of raw LaTeX mathematical expressions or blocks.

LaTeX Node

itemize

Defines an ordered list of text items using bullet points.

List Node

3. Including Data

The data field in the root report schema is an array of DataField derived objects, which instruct the renderer how to load data from different sources and make it available for content generation. Data can be specified inline, can come from files that can be located on the disk or on remote locations.

Remote file handling is not yet implemented. Remote locations coming soon : Girder, CKAN, github

3.1. Base DataField Schema

The base schema for loading data is defined in the following table.

Field Type Description Default Value

name

string

A unique identifier used to reference this data in content nodes (e.g., in a table or plot).

Required

type

string("DataTable","Object","Raw")

The type for how to load and process the data.

It is inferred from the format or the schema. Only provide it when you need custom casting (e.g. load a json as a Table)

preprocessor

string or object (“module``:`my.function”)

Defines a Python function to preprocess the loaded data.

null

3.2. Loading Inline Data

It is possible to specify the data directly on the JSON configuration, allowing users to refactor and mantain their configuration files clean.

Three inline types are supported at the moment:

  • Raw : Treated as a string

  • Object : Treated as a python dictionary

  • DataTable : Treated as a pandas dataframe

3.2.1. Raw Inline data

Inline data can be loaded by using the value field.

Setting inline text in data
{
  "data":[
    {"name":"myCustomText", "value":" Some text I want to access across my report."}
  ]
}

Later, users can reference this text with :

{
  "contents":[
    {
      "type":"text", "ref":"myCustomText",
      "text":" Here, the text defined in the data section will appear : @{myCustomText}@"
    }
  ]
}

3.2.2. Object Inline data

Object (dict) data can be loaded by using the object field.

Setting inline objects in data
{
  "data":[{
    "name":"myCustomObject",
    "object":{
      "students":{
        "Robert" : { "score":"50", "grade":"D" },
        "John": { "score":"100", "grade":"A" }
      }
    }
  }]
}

Later, users can reference this object with :

{
  "contents":[{
    "type":"text", "ref":"myCustomObject",
    "itemize":[
      "Robert had a score of @{students.Robert.score}@ and a grade @{students.Robert.grade}@",
      "John had a score of @{students.John.score}@ and a grade @{students.John.grade}@"
    ]
  }]
}

3.2.3. Table Inline data

Inline tables can be loaded using the columns field, which is a list of objects following the format

{ "columns":[ { "name":"", "values":[] } ] }
Setting inline tables in data
{
  "data":[{
    "name":"myInlineTable",
    "columns":[
      { "name":"time", "values":[1.2,2.3,3.4,4.5] },
      { "name":"power", "values":[10,20,30,40] },
      { "name":"temperature", "values":[22.5,23.0,21.3,22.1] },
      { "name":"comfort", "values":[0.5,0.6,0.7,0.8] }
    ],
  }]
}

Later, users can render this table with its reference:

{
  "contents":[
    { "type":"table", "ref":"myInlineTable" }
  ]
}
When providing inline tables, the values for each column should have exactly the same length. An exception will be raised if not the case.
Multiple options can be provided to table objects when loading to perform some preprocessing. See the Table options section for more information.

3.3. Loading File Data

Files can be loaded into the report by providing filepath key in the Data Object. It can either be an absolute path, or a path relative to the report file location. Additionally, the format can be specified, otherwise it will be inferred from the file extension.

The fields are also specified in this table.

Field Type Description Default Value

filepath

string

The path to the data file. Relative paths are resolved relative to the JSON report file.

Required

format

string ("json", "csv", "raw")

The format of the file. If omitted, it is inferred from the file extension (.json, .csv).

Inferred from filepath
Any file extension other than json and csv will be treated as raw data.
Loading files on the disk
{
  "data":[
    { "name":"my_table", "filepath":"tabular_data.csv" },
    { "name":"my_json_data", "filepath":"my_data.json" },
    { "name":"my_error_log", "filepath":"./logs/log.err" } //The err extension will default to raw text
  ]
}

3.4. Table options

Table options can optionally be provided to data objects of type DataTable, using the table_options field. It does not matter if data is loaded from a file or provided inline. It is used to preprocess the data in a sequential pipeline, just after loading.

{
  "name":"my_processed_table", "filepath":"my_data.csv",
  "table_options":{
    "filter":[...],
    "computed_columns":{...},
    "group_by":{...},
    "pivot":{...},
    "sort":[...],
    "format":{...},
  }
}
Validation Rules
The engine enforces validation rules: pivot and group_by are mutually exclusive, and sorting is discouraged after pivoting due to ambiguous results.

Operations are executed in this strict order:

  1. Filter: Row reduction based on conditions.

  2. Computed Columns: Creation of new columns via Python expressions.

  3. Group By / Pivot: Data aggregation or structural reshaping (mutually exclusive).

  4. Sort: Ordering of final rows.

  5. Format: Value presentation (e.g., decimal precision, value substitution).

3.4.1. Filtering

The filter field is a list of conditions used to exclude rows from the initial dataset.

Condition Field Type Description

column

string

The column name to apply the condition to.

op

string

The comparison operator. Supported: ==, !=, >, <, >=, ⇐, in, not in.

value

object

The value(s) to compare against. Use an array for in/not in operators.
Filtering a table
"filter": [
  {"column": "Time", "op": "<", "value": 50.0},
  {"column": "Method", "op": "in", "value": ["A", "B"]}
]

3.4.2. Creating New Columns

You can define new columns based on existing row values using Python expressions.

Creating new columns
"computed_columns": {
  "Speedup": "row['Time_Ref'] / row['Time']",
  "Max_Memory_KB": "row['Memory_MB'] * 1024"
}

3.4.3. GroupBy

Used to group rows by categorical columns and apply aggregation functions to columns.

Field Type Description

columns

array of strings

The columns to group the data by.

agg

object or string

The aggregation function (e.g., "mean", "sum", "max", "count"). Can be a single string for all columns, or a map like {"Time_s": "mean"}.

3.4.4. Pivoting

This option is used to create cross-tabulation tables by transforming unique column values into new column headers.

Field Type Description

index

array of strings

The column(s) for the new row headers.

columns

array of strings

The column(s) whose unique values will become the new column headers.

values

string

The column providing the data for the new cell values.

agg

string

The aggregation function applied to cell values (e.g., "mean").

3.4.5. Sorting

The sort field is a list of instructions defining the final order of rows.

Instruction Field Type Description Default Value

column

string

The column name to sort by.

ascending

boolean

If true, sorts ascending (A-Z, 0-9). If false, sorts descending.

true
Sorting columns
"sort": [
  {"column": "Configuration", "ascending": true},
  {"column": "Time", "ascending": false}
]

3.4.6. Formatting

The format field is used to parse the content columns. Specific values can also be subsituted.

Formating Time and result columns
"format":{
  "Time": "%.3f", //Cast to string and format the number to 3 decimal places
  "result":{ //Will look for 'true' and 'false" in the "result" column and will replace those values by Success and Failure
    "true":"Success",
    "false":"Failure"
  }
}

3.5. Using a preprocessor

The preprocessor field allows for custom data manipulation after loading. It can be a string in the format "module_name:function_name" or an object {"module":"module_name", "function":"function_name"}

Function names can be splitted with a dot (.) to access specific methods. For example "builtins:str.lower" will access the lower() method of the str class in the builtins module.

This field should only be used to perform complex processing that cannot be done using the available fields described in Table options.

Security Implication
The preprocessor feature involves dynamic module loading and function execution. Ensure that preprocessors originate from a trusted source.
Using a preprocessor
{
  "data": [
    {
      "name": "benchmark_results",
      "filepath": "data/timing_data.csv",
      "preprocessor": "my_data_utils:clean_data"
    },
  ]
}

In this example above, my_data_utils.py must contain a function clean_data(df: pandas.DataFrame) → pandas.DataFrame.

Note that it can also be used to cast your data to other types (e.g. you want to transform a nested JSON into a dataframe).

3.6. Data Referencing

The report schema supports referencing one data object from another. This allows you to create derived data fields or avoid duplicating data definitions. References are specified using the ref field inside a DataField object.

3.6.1. Creating a reference

To reference an existing data field, define a new DataField with the ref key pointing to the name of another data object:

{
  "data":[
    { "name":"raw_results", "filepath":"results.csv" },
    { "name":"processed_results", "ref":"raw_results", "table_options":{...} }
  ]
}

In this example:

  • processed_results will perform some preprocessing using "table_options" to the data loaded from raw_results, without modifying the actual "raw_results" field.

  • Users can reference these two data fields in the content independently.

  • The type of the reference is automatically inferred from the parent (raw_results).

References can be chained: a data field can reference another reference.
All processing from the referenced data field is performed before, preserving the logical order.

3.6.2. Type Resolution

  • If the type is not explicitly defined, it is inferred from the referenced field.

  • The system supports DataTable, Object, and Raw types.

  • The reference inherits the type and can be cast automatically to the correct subclass (DataTable, Object, or Raw) after resolution.

3.6.3. Caching and Reuse

  • Resolved references are cached in the dependency graph to avoid redundant computation.

  • This ensures efficiency when multiple fields reference the same source data.

  • Circular references are detected and will raise a ReferenceError.

4. Section Node

The Section Node is the fundamental building block for structuring your report. It defines a titled, hierarchical container that organizes other content nodes into logical chapters and subsections.

Field Type Description Default Value

type

string

Must be set to "section".

"section"

title

string

The title of the section.

Required

contents

array of objects

A recursive list of other content nodes, including nested `SectionNode`s.

[]

4.1. Example

{
  "contents": [
    {
      "type": "section",
      "title": "Introduction",
      "contents": [
        // Text, Tables, Images, etc.
      ]
    },
    {
      "type": "section",
      "title": "Detailed Results",
      "contents": [
        {
          "type": "section",
          "title": "Timing Analysis",
          "contents": [
            // Plots and Tables
          ]
        }
      ]
    }
  ]
}

5. Section Node

The Section Node is the fundamental building block for structuring your report. It defines a titled, hierarchical container that organizes other content nodes into logical chapters and subsections.

Field Type Description Default Value

type

string

Must be set to "section".

"section"

title

string

The title of the section.

Required

contents

array of objects

A recursive list of other content nodes, including nested `SectionNode`s.

[]

5.1. Example

{
  "contents": [
    {
      "type": "section",
      "title": "Introduction",
      "contents": [
        // Text, Tables, Images, etc.
      ]
    },
    {
      "type": "section",
      "title": "Detailed Results",
      "contents": [
        {
          "type": "section",
          "title": "Timing Analysis",
          "contents": [
            // Plots and Tables
          ]
        }
      ]
    }
  ]
}

6. Text Node

The Text Node is used to insert narrative content into your report. It supports both fixed, static text blocks and dynamic content that integrates processed values directly from loaded data files.

Field Type Description Default Value

type

string

Must be set to "text".

"text"

text

string or object

The text content or a nested Text object.

Required

ref

string

The name of the data field to reference for dynamic placeholder resolution.

null

6.1. Static Text

Static content is the simplest form of a content node and does not require the ref field.

If the text field is defined as a raw string, the engine automatically treats it as static content unless placeholders are detected. 
{
  "type":"text",
  "text":"My custom static text!"
}

Dynamic text uses placeholders (@{…​}@) to inject data values. The content inside the placeholders follows a simple path and operator syntax.

6.1.1. Path Syntax

Use dot (.) notation to traverse nested data structures within the data source referenced by the data field.

  • Dictionary/JSON: Use keys: @{data_name.key.subkey}@

  • Lists/Arrays: Use numerical indices: @{data_list.0.item_name}@

6.1.2. Value Operators

Simple operators can be applied to the retrieved value using the pipe (|) syntax to perform common transformations directly at render-time.

Operator Description Example

length

Returns the size of a list, array, or string.

@{list_data | length}@
Rendering dynamic text

Assume a data file named "results" was loaded, containing the data: {"version": "1.2.3", "timings": [1, 2, 3]}.

{
  "type": "text", "ref": "results",
  "text": "The report was generated for version @{results.version}@. We analyzed @{results.timings | length}@ samples."
}
// Renders as: "The report was generated for version 1.2.3. We analyzed 3 samples."

6.2. Text Configuration (Nested Object)

If you need to override the default dynamic behavior (e.g., change the placeholder delimiters), you can use the detailed configuration object instead of a raw string for the text field.

Field Type Description Default Value

content

string

The text string containing the narrative and any placeholders.

Required

mode

string ("static", "dynamic")

Explicitly controls processing. If omitted, the mode is inferred based on whether placeholders are detected in the content.

Inferred

placeholder_expr

string

The regular expression used to identify placeholders in the text.

@{([^}]+)}@
Overriding dynamic behaviour

Assume a data file named "results" was loaded, containing the data: {"version": "1.2.3", "timings": [1, 2, 3]}.

{
  "type": "text", "ref": "results",
  "text":{
    "placeholder":"[[([^}]+)]]",
    "content":"The report was generated for version [[results.version]]. We analyzed [[results.timings | length]] samples.",
  }
}
// Renders as: "The report was generated for version 1.2.3. We analyzed 3 samples."

7. LaTeX Node

The LaTeX Node is designed for the direct embedding of mathematical formulas, equations, or complex typesetting commands into the final report.

Field Type Description Default Value

type

string

Must be set to "latex".

"latex"

latex

string

The raw LaTeX content (e.g., a mathematical equation or a custom LaTeX block).

Required

ref

string

The name of the data field to reference for dynamic placeholder resolution.

null

7.1. Example

This example demonstrates including a multi-line equation and an inline mathematical expression.

{
  "type": "latex",
  "latex": "\\begin{equation*}\n  \\nabla \\cdot \\mathbf{E} = \\frac{\\rho}{\\epsilon_0}\n\\end{equation*}\n\nAn inline expression is also supported, such as $E=mc^2$."
}

8. List Node

The List Node is used to render ordered lists. Its primary strength is that every item in the list is automatically processed as a potential Dynamic Text Node, allowing placeholders to be used in any list item.

8.1. ListNode Schema

Field Type Description Default Value

type

string

Must be set to "itemize".

"itemize"

items

array of objects/strings

The contents of the list. Each entry can be a simple string, a full Text Node object, or a Text Configuration object.

Required

ref

string

The name of the data field to reference for dynamic placeholder resolution within all list items.

null

8.2. Item Coercion and Dynamic Text

For simplicity, the engine automatically coerces simple strings and Text objects provided in the items array into full Text Node objects.

  • If an item is a simple string, it can contain placeholders (@{…​}@).

  • The ref specified on the ListNode container is automatically inherited by all items, enabling dynamic text substitution for the entire list based on a single data source.

8.2.1. Example: Dynamic List

Assume a data file named "config" was loaded, containing: {"user": "Alice", "tasks": ["compute", "fill", "finalize"]}.

{
  "type": "itemize",
  "ref": "config",
  "items": [
    "Report generated for user: @{user}@",
    "Total tasks found: @{tasks | length}@",
    "First task: @{tasks.0}@"
  ]
}

When rendered, the output would be:

- Report generated for user: Alice
- Total tasks found: 3
- First task: compute

9. Image Node

The Image Node is used to embed static, pre-rendered image files into the report. It translates directly to the AsciiDoc image macro, supporting essential attributes like source, caption, and alternative text.

Path Resolution
The src file path is typically resolved relative to the passed configuration file.

9.1. ImageNode Schema

Field Type Description Default Value

type

string

Must be set to "image".

"image"

src

string

The file path or URL of the image to embed.

Required

caption

string

An optional caption displayed beneath the image. If provided, the image is rendered using the title attribute in AsciiDoc.

null

alt

string

Alternative text for the image, used for accessibility and when the image cannot be displayed.

null

styles

List[strin]

List of classnames to style the image. See Feel++ Antora UI for the complete list.

["img-fluid"]

9.2. Example

{
  "type": "image",
  "src": "figures/my_simulation_output.png",
  "caption": "Figure 3. Simulation domain with boundary conditions.",
  "alt": "Diagram of the domain geometry."
}

10. Table Node

The Table Node is a powerful mechanism for defining, processing, and styling tabular data extracted from a loaded data source. It leverages a comprehensive pipeline of data transformations (filtering, grouping, pivoting) before the data is rendered.

The structure is composed of the TableNode container and the nested Table configuration object.

10.1. TableNode Schema (Container)

Field Type Description Default Value

type

string

Must be set to "table".

"table"

ref

string

The unique reference of the data file (from the root data list) containing the table to be rendered.

Required

caption

string

Optional. Table caption.

null

layout

object

Optional. A layout configuration object defining renaming and ordering operations.

{}

style

object

Optional. A style configuration object allowing column alignment, resizing and custom styling

{}

filter

object

Optional. Configuration for an interactive filter input element added above the table (if supported by the target format/template).

null

10.2. Table layout

The layout field allows users to rename table columns when rendering, as well as change the column order.

Field Type Description Default

rename

Dict[str,str]

Map of {"old_col_name": "new_col_name"}

{}

column_order

List[str]

List of column names to render respecting the order. If not provided, all columns will be shown.

null

10.2.1. Renaming

To rename columns, pass a dictionary containing the original column names as keys and the new names as values.

"layout":{
  "rename":{
    "ntask":"Number of Tasks (x10)",
    "avg_v":"Average Computation Load",
    "datetime":""
  }
}
Columns can be renamed into an empty string "" to hide the header name.

10.2.2. Column ordering

To order the way the columns appear on the rendered table, you can use the column_order field.

This field also controls which columns appear and which are hidden on the tables.

By default, all columns are shown.

It follows the conventions:

  • If column_order is not provided, all columns are rendered in the same way they are loaded.

  • If column_order is []. No column will be shown on the table.

  • If column_order is a non-empty list, only the provided columns will be rendered, in the same order as in the list.

10.3. Styling and Interactivity

The style object defines presentation attributes that are converted into AsciiDoc table attributes and HTML class names for dynamic behavior.

Field Type Description

column_align

object

Map of {"column_name": "alignment"} where alignment is "left", "center", or "right". Used to build the AsciiDoc cols attribute.

column_width

object

Map of {"column_name": integer} defining the relative width of columns (used in the AsciiDoc cols attribute).

classnames

List[str]

List of class names to apply to the table

10.3.1. Interactive Classnames

The classnames field applies CSS classes to the table, enabling client-side interactive features when the output format supports it (e.g., HTML/Antora with JavaScript).

Users can customize their own classes, but the framework provides the following built-in functionalities.

Classname Purpose

sortable

Enables client-side sorting by clicking on column headers.

filterable

Enables client-side filtering or searching over the table content.

 
"style": {
    "column_align": {"Time": "right", "Config": "center"},
    "classnames": ["sortable", "filterable", "grid"]
}

10.4. Searching and filtering

If using the filterable classname, users can provide the filter field to customize the text input appearing on the bottom of the table.

Field Type Description Default

placeholder

str

The placeholder for the text input

"Filter…​"

style

str

Optional inline css to include in the <input/> object.

margin-bottom:0.5em;padding:0.3em;width:50%;

10.5. Example: renaming, ordering, interactivity and custom style

The following example assume a data file was loaded with the name "batch_data".

This example renames the Time column, and orders the column to appear in the order Method, "Time", "Status" from left to right.

It also enables interactive sorting and filtering for the final HTML output.

{
  "type": "table", "ref": "batch_data",
  "layout": {
    "column_order": ["Method", "Time", "Status"],
    "rename": { "Time": "Execution Time (s)" },
  },
  "style": {
    "column_align": { "Time": "right" },
    "classnames": ["sortable", "filterable"]
  }
}

11. Plot Node

The Plot Node defines a data visualization within the report. It specifies the data to be used, how that data should be mapped to the figure’s axes and dimensions, and what transformations should be applied before rendering.

11.1. PlotNode Schema (Container)

Field Type Description Default Value

type

string

Must be set to "plot".

"plot"

ref

string

The unique name of the data file (from the root data list) to reference for building the figure.

Required

plot

object

The nested configuration object (Plot schema) defining the figure’s dimensions, type, and transformation.

Required

11.2. Plot Configuration (plot Field)

The core configuration is provided here, defining the geometry and data logic of the figure.

Field Type Description Default Value

title

string

The title displayed at the top of the figure.

Required

plot_types

array of strings

A list of one or more visualization types to generate from the same configuration (e.g., ["scatter", "table"]).

Required

transformation

string

Defines how the input data should be processed and normalized before plotting.

"performance"

aggregations

array of objects

A sequential list of instructions to filter and summarize the dataset before mapping to axes.

null

xaxis

object

The configuration for the horizontal axis (PlotAxis schema).

Required

yaxis

object

The configuration for the vertical axis (PlotAxis schema).

Required

secondary_axis

object

A second independent axis used for animation (sliders) or defining a third dimension.

null

color_axis

object

The parameter used to define distinct lines or groups, mapping data values to the trace color/legend.

null

extra_axes

array of objects

Additional parameters used for filtering, grouping, or serving as a fourth dimension in complex plots (e.g., 3D or parallel coordinates).

[]

layout_modifiers

object

Allows passing raw configuration options to the underlying plotting library (Plotly) for fine-grained control (e.g., log scale).

{}

11.3. Dimensions and Axes (PlotAxis Schema)

All axis fields (xaxis, yaxis, secondary_axis, color_axis, extra_axes) use the PlotAxis schema to define which data column maps to which dimension.

Field Type Description Default Value

parameter

string

The name of the data column to be mapped to this axis. Use dot notation (parameter.subparameter) for nested data.

Required

label

string

The display label for the axis in the final figure.

Inferred from parameter

filter

string, array, or object

Allows explicit filtering or renaming of values within this dimension. (See Filtering section below.)

[]

11.3.1. Dimension Filtering and Renaming

The filter field within an axis allows you to select specific values from that column to include in the plot, and optionally rename them for display.

Syntax Action Example

Single string

Filters by the value, keeps the value as the label.

"filter": "ValueA"

Array of strings

Filters by multiple values, keeps values as labels.

"filter": ["ValueA", "ValueB"]

Map of strings

Filters by keys, renames to values.

"filter": {"old_name": "New Label"}

 
"xaxis": {
    "parameter": "System",
    "filter": {
        "cluster_a": "Cluster A (Production)",
        "dev_box": "Development System"
    }
}

11.4. Data Transformations

The transformation field defines how the data is restructured and normalized before being plotted.

Transformation Purpose Output Focus

"performance"

The base and default strategy. Data is restructured (pivoted) using the axes dimensions but values remain absolute.

Absolute Values

"relative_performance"

Data is restructured, and each value is normalized by the total sum of its row (e.g., component percentages of total time).

Percentage/Proportion

"speedup"

Computes the speedup relative to the smallest value found along the X-axis dimension (often used to show parallel scaling).

Scaling Factors

To explain how transformation and plot types work, we can consider the following example.

records = [{"perfvalue": "elapsed_fill", "value": 28.9239, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 10000.0}, {"perfvalue": "elapsed_compute", "value": 1.92806, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 10000.0}, {"perfvalue": "elapsed_fill", "value": 63.0273, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 20000.0}, {"perfvalue": "elapsed_compute", "value": 3.74504, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 20000.0}, {"perfvalue": "elapsed_fill", "value": 92.358, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 30000.0}, {"perfvalue": "elapsed_compute", "value": 3.29597, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 30000.0}, {"perfvalue": "elapsed_fill", "value": 99.9142, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 40000.0}, {"perfvalue": "elapsed_compute", "value": 2.37914, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 1.0, "elements": 40000.0}, {"perfvalue": "elapsed_fill", "value": 24.8636, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 10000.0}, {"perfvalue": "elapsed_compute", "value": 0.629236, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 10000.0}, {"perfvalue": "elapsed_fill", "value": 14.9139, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 20000.0}, {"perfvalue": "elapsed_compute", "value": 0.498528, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 20000.0}, {"perfvalue": "elapsed_fill", "value": 98.9536, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 30000.0}, {"perfvalue": "elapsed_compute", "value": 2.3708, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 30000.0}, {"perfvalue": "elapsed_fill", "value": 45.4764, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 40000.0}, {"perfvalue": "elapsed_compute", "value": 1.56548, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 2.0, "elements": 40000.0}, {"perfvalue": "elapsed_fill", "value": 2.89716, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 10000.0}, {"perfvalue": "elapsed_compute", "value": 0.0434488, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 10000.0}, {"perfvalue": "elapsed_fill", "value": 36.537, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 20000.0}, {"perfvalue": "elapsed_compute", "value": 0.519222, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 20000.0}, {"perfvalue": "elapsed_fill", "value": 52.302, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 30000.0}, {"perfvalue": "elapsed_compute", "value": 1.23368, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 30000.0}, {"perfvalue": "elapsed_fill", "value": 98.397, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 40000.0}, {"perfvalue": "elapsed_compute", "value": 1.55742, "unit": "s", "result": "pass", "system": "gaya", "partition": "public", "tasks": 4.0, "elements": 40000.0}]














import pandas as pd

master_df = pd.DataFrame.from_dict(records)





























master_df.head(5)















Out[0]:
         perfvalue     value unit result system partition  tasks  elements
0     elapsed_fill  28.92390    s   pass   gaya    public    1.0   10000.0
1  elapsed_compute   1.92806    s   pass   gaya    public    1.0   10000.0
2     elapsed_fill  63.02730    s   pass   gaya    public    1.0   20000.0
3  elapsed_compute   3.74504    s   pass   gaya    public    1.0   20000.0
4     elapsed_fill  92.35800    s   pass   gaya    public    1.0   30000.0












We can see that this dataframe contains the parameters:
- system
- result
- tasks
- elements
- value
- perfvalue
- unit






By having this common structure, we can make use of transformation strategies to manipulate values depending on the desired output.






Strategies will depend on the figure axis. All strategies will create a pivot dataframe that will contain the parameter specified as color_axis as columns, xaxis as first level index and secondary_axis as second level index. Values of the dataframe will always be the values of the master dataframe.






As an example, we will consider the following axis definitions:








"xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
"yaxis":{ "parameter":"value", "label":"Execution time (s)" },
"secondary_axis":{ "parameter":"elements", "label":"N" },
"color_axis":{ "parameter":"perfvalue", "label":"Performance variable" }








performance






This strategy should be seen as the "base" strategy. No transformation, other that a pivot, is done.
For the given example, it produces the following dataframe








from feelpp.benchmarking.json_report.figures.transformationFactory import TransformationStrategyFactory
from feelpp.benchmarking.json_report.figures.schemas.plot import Plot
plot_config = Plot(**{
    "title": "Absolute performance",
    "plot_types": [ "stacked_bar", "grouped_bar" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)"},
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{"parameter":"elements", "label":"N"}
})
strategy = TransformationStrategyFactory.create(plot_config)
df = strategy.calculate(master_df)
print(df)















perfvalue       elapsed_compute  elapsed_fill
elements tasks
10000.0  1.0           1.928060      28.92390
         2.0           0.629236      24.86360
         4.0           0.043449       2.89716
20000.0  1.0           3.745040      63.02730
         2.0           0.498528      14.91390
         4.0           0.519222      36.53700
30000.0  1.0           3.295970      92.35800
         2.0           2.370800      98.95360
         4.0           1.233680      52.30200
40000.0  1.0           2.379140      99.91420
         2.0           1.565480      45.47640
         4.0           1.557420      98.39700












relative_performance






The relative performance strategy computes the proportion of the time that a a color_axis variable takes with regards of the total.








plot_config = Plot(**{
    "title": "Absolute performance",
    "plot_types": [ "stacked_bar", "grouped_bar" ],
    "transformation": "relative_performance",
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{"parameter":"value", "label":"Execution time (s)"},
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N"
    }
})
strategy = TransformationStrategyFactory.create(plot_config)
df = strategy.calculate(master_df)
print(df)















perfvalue       elapsed_compute  elapsed_fill
elements tasks
10000.0  1.0           6.249392     93.750608
         2.0           2.468286     97.531714
         4.0           1.477544     98.522456
20000.0  1.0           5.608670     94.391330
         2.0           3.234584     96.765416
         4.0           1.401174     98.598826
30000.0  1.0           3.445722     96.554278
         2.0           2.339812     97.660188
         4.0           2.304407     97.695593
40000.0  1.0           2.325801     97.674199
         2.0           3.327843     96.672157
         4.0           1.558130     98.441870












The sum along the column axis will always be equal to 100.






speedup






The speedup strategy computes the speedup of the color_axis variables. The minimum of the xaxis values is taken as the base of the speedup.
For the example, this strategy will produce the following.








plot_config = Plot(**{
    "title": "Absolute performance",
    "plot_types": [ "stacked_bar", "grouped_bar" ],
    "transformation": "speedup",
    "xaxis":{"parameter":"tasks", "label":"Number of tasks"},
    "yaxis":{"parameter":"value", "label":"Execution time (s)"},
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{"parameter":"elements", "label":"N"}
})
strategy = TransformationStrategyFactory.create(plot_config)
df = strategy.calculate(master_df)
print(df)















perfvalue       elapsed_compute  elapsed_fill  optimal  half-optimal
elements tasks
10000.0  1.0           1.000000      1.000000      1.0           1.0
         2.0           3.064129      1.163303      2.0           1.5
         4.0          44.375449      9.983536      4.0           2.5
20000.0  1.0           1.000000      1.000000      1.0           1.0
         2.0           7.512196      4.226078      2.0           1.5
         4.0           7.212791      1.725027      4.0           2.5
30000.0  1.0           1.000000      1.000000      1.0           1.0
         2.0           1.390235      0.933347      2.0           1.5
         4.0           2.671657      1.765860      4.0           2.5
40000.0  1.0           1.000000      1.000000      1.0           1.0
         2.0           1.519751      2.197056      2.0           1.5
         4.0           1.527616      1.015419      4.0           2.5
















11.5. Aggregations




The optional aggregations field allows for data reduction and filtering across the dataset before the final pivot structure is created. The order of these instructions matters.













Aggregation Field
Type
Description







column


string


The column to apply the aggregation or filter to.






agg


string


The aggregation function or filter instruction.









11.5.1. Available Aggregation Functions










Function
Action







mean


Computes the arithmetic average of the column.






sum


Computes the total sum of the column.






max, min


Finds the maximum or minimum value in the column.






count, nunique


Counts the number of values or unique values.






filter:VALUE


Filters the column to keep only rows where the column value matches VALUE.











"aggregations":[
    {"column":"Date","agg":"max"}, // Keep only records with the latest date
    {"column":"System","agg":"filter:gaya"}, // Filter to only keep the 'gaya' system
    {"column":"Time","agg":"mean"} // Calculate the mean execution time
]














11.6. Plot Types




The engine supports a diverse range of visualizations, grouped into 2D, 3D, and specialized formats. The definition of each axis depends on the plot type.






11.6.1. 2D Visualizations




These are standard 2D plots that support the xaxis, yaxis, and often the color_axis and secondary_axis (for animation/faceting).






For examples in this section, we will define the axis as:








"xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
"yaxis":{ "parameter":"value", "label":"Execution time (s)" },
"color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
"secondary_axis":{ "parameter":"elements", "label":"N" }








scatter: Standard scatter plot, typically showing performance trends.






Axis definition:






    

  • 

    secondary_axis : represents the figure slider
    

    • 









from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Scatter Plot",
    "plot_types": [ "scatter" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















marked_scatter: Enhanced scatter plot where the symbol/mark shape can be mapped to an additional dimension.






Axis definition:






    

  • 

    secondary_axis : represents the marks
    

    • 









from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Marked Scatter Plot",
    "plot_types": [ "marked_scatter" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















stacked_bar: Displays cumulative values, useful for showing the proportion of components (e.g., relative_performance transformation).






Axis definition:






    

  • 

    xaxis : represents the subplot dimension. A subplot is created for each value in the x axis.
    

    • 

  • 

    secondary_axis : represents the x axis of each subplot.
    

    • 









from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Stacked Bar Plot",
    "plot_types": [ "stacked_bar" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















grouped_bar: Displays values side-by-side for easy comparison.






Axis definition:






    

  • 

    secondary_axis : represents the figure slider
    

    • 









from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Grouped Bar Plot",
    "plot_types": [ "grouped_bar" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















heatmap: Displays data values as colors on a 2D grid defined by the X and Color axes.






Axis definition:






    

  • 

    yaxis: Is given by the color scale (I know it is counter-intuitive).
    

    • 

  • 

    xaxis: Represents the columns of the heatmap
    

    • 

  • 

    color_axis: Represents the rows of the heatmap
    

    • 

  • 

    secondary_axis : Represents the figure slider
    

    • 









from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Heatmap",
    "plot_types": [ "heatmap" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    

















11.6.2. Specialized Visualizations




table: Renders the processed data directly into a styled, tabular format (similar to the Table Node, but generated through the figure pipeline).






Axis definition: Columns are displayed in the following order, from left to right: [extra_axes] → secondary_axis → xaxis →  [all values in color_axis].






Cell values correspond to yaxis.








from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Table",
    "plot_types": [ "table" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















sunburst: Displays hierarchical data as a multi-layered ring chart, useful for showing partitions.






Dimensions are mapped to the rings: From inner to outer:  secondary_axis → xaxis→ [extra_axes] → color_axis (outer).








from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Sunburst",
    "plot_types": [ "sunburst" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















parallelcoordinates: Displays multidimensional data where each variable is represented by a vertical axis, and data points are drawn as colored lines connecting the axes.






Axes will be shown on the following order from left to right:
secondary_axis → xaxis, [extra_axes] → color_axis.
The yaxis will be shown in the line color.








from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Parallel Coordinates",
    "plot_types": [ "parallelcoordinates" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    

















11.6.3. 3D Visualizations




These plots are used for visualizing three or four variables. At least three dimensions are required (xaxis, yaxis, and secondary_axis).






    

  • 

    scatter3d: Displays individual data points in three dimensions.
    

    • 

  • 

    surface3d: Displays values as a 3D surface grid.
    

    • 














3D Axis


Mapped Parameter






X-axis


xaxis parameter






Y-axis


secondary_axis parameter (or first extra_axes element if provided)






Z-axis


yaxis parameter (the measured value)






Color/Trace


color_axis parameter











scatter3d








from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Scatter 3D",
    "plot_types": [ "scatter3d" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    















surface3d








from feelpp.benchmarking.json_report.figures.figureFactory import FigureFactory
figures = FigureFactory.create(Plot(**{
    "title": "Absolute performance - Surface 3D",
    "plot_types": [ "surface3d" ],
    "xaxis":{ "parameter":"tasks", "label":"Number of tasks" },
    "yaxis":{ "parameter":"value", "label":"Execution time (s)" },
    "color_axis":{ "parameter":"perfvalue", "label":"Performance variable" },
    "secondary_axis":{ "parameter":"elements", "label":"N" }
}))
fig = figures[0].createFigure(master_df)
fig.show()


















    
                            
                    



















11.7. Custom Layouts (layout_modifiers)




The layout_modifiers field allows expert users to pass raw configuration dictionaries to the underlying plotting engine (Plotly). This is useful for advanced styling not covered by the standard schema.






These options correspond to the accepted layout reference for Plotly: Plotly layout reference
It accepts a nested dictionnary just as Plotly does.






For example, we could customize a figure to have have its x-axis on a logscale.








"layout_modifiers":{
    "xaxis":{
        "type":"log"
    }
}