metric

Metrics to use for training or explanations.

Classes:

Name	Description
Metric	Represents a metric for the explainable model.
TorchGlobalMetric	Represent a metric to be used in the training or evaluation process for the whole model (global metric).
TorchLeafMetric	Represent a metric to be used during the training or the evaluation process, per leaf.
DictMetrics	Dictionary of metrics (torch eval etc...) used in training or explain stage.

Metric

Represents a metric for the explainable model.

Parameters:

Name	Type	Description	Default
metric	partial[Metric]	The torchmetrics metric to use, as partial.	required
on_raw_data	bool	If set, value are compute from raw data and not preprocessed data (inputs or targets).	False
reduced_dimensions	tuple[int, ...]	The dimensions to reduce in the statistics. For instance, given a batch with dimensions (batch size x num_timestamps x num_channels), and a reduced_dimensions=(0,2), it returns a per timestamp statistic, averaged over the batch and channels. Default (0,), meaning the statistic is reduced over the batch only.	(0, )
target_as_indexes	bool	For a classification task, torchmetrics expects targets as indexes and not onehot vectors. If True, will transform targets from onehot vectors to indexes prior to the metric computation.	False

Methods:

Name	Description
to_model	Parse the object to a RequestBody for the backend.

Attributes:

Name	Type	Description
metric	partial[Metric]
on_raw_data	bool
reduced_dimensions	tuple[int, ...]
target_as_indexes	bool

metric: partial[torchmetrics.metric.Metric]

on_raw_data: bool = False

reduced_dimensions: tuple[int, ...] = (0,)

target_as_indexes: bool = False

to_model() -> TrustedObjectInput

Parse the object to a RequestBody for the backend.

Source code in src/xpdeep/metric.py

def to_model(self) -> TrustedObjectInput:
    """Parse the object to a RequestBody for the backend."""
    raise NotImplementedError

TorchGlobalMetric

Represent a metric to be used in the training or evaluation process for the whole model (global metric).

Parameters:

Name	Type	Description	Default
metric	partial[Metric]		required
on_raw_data	bool		False
reduced_dimensions	tuple[int, ...]		(0,)
target_as_indexes	bool		False

Methods:

Name	Description
to_model	Parse the object to a RequestBody for the backend.

to_model() -> TrustedObjectInput

Parse the object to a RequestBody for the backend.

Source code in src/xpdeep/metric.py

def to_model(self) -> TrustedObjectInput:
    """Parse the object to a RequestBody for the backend."""
    return TrustedObjectConverter.to_model(
        "xpdeep_core.node_stat.stats_zoo.metrics",
        "RootMetric",
        reconstructor="partial",
        on_raw_data=self.on_raw_data,
        metric=self.metric,
        reduced_dimensions=self.reduced_dimensions,
        target_as_indexes=self.target_as_indexes,
    )

TorchLeafMetric

Represent a metric to be used during the training or the evaluation process, per leaf.

Parameters:

Name	Type	Description	Default
metric	partial[Metric]		required
on_raw_data	bool		False
reduced_dimensions	tuple[int, ...]		(0,)
target_as_indexes	bool		False

Methods:

Name	Description
to_model	Parse the object to a RequestBody for the backend.

to_model() -> TrustedObjectInput

Parse the object to a RequestBody for the backend.

Source code in src/xpdeep/metric.py

def to_model(self) -> TrustedObjectInput:
    """Parse the object to a RequestBody for the backend."""
    return TrustedObjectConverter.to_model(
        "xpdeep_core.node_stat.stats_zoo.metrics",
        "LeafMetric",
        reconstructor="partial",
        on_raw_data=self.on_raw_data,
        metric=self.metric,
        reduced_dimensions=self.reduced_dimensions,
        target_as_indexes=self.target_as_indexes,
    )

DictMetrics

Dictionary of metrics (torch eval etc...) used in training or explain stage.

If a metric is: - from torchmetrics, it will be computed globally (for the whole model) and per leaf. - a TorchGlobalMetric, it will be computed globally only. - a TorchLeafMetric, it will be computed per leaf only.

In the training stage, metrics help to monitor the training. In the explain stage, metrics are displayed in XpViz along with explanations. Dictionary keys are will be used as metric names in XpViz.

Methods:

Name	Description
to_model_train_pipeline_input	Convert to TrainPipelineInputMetricsType0 instance, required for training pipelines.
to_model	Convert to GlobalExplainPipelineInputMetrics instance, required for explanation pipelines.

to_model_train_pipeline_input() -> TrainPipelineInputMetricsType0

Convert to TrainPipelineInputMetricsType0 instance, required for training pipelines.

TODO https://gitlab.xpdeep.com/xpdeep/xpdeep-client/-/issues/279

Source code in src/xpdeep/metric.py

def to_model_train_pipeline_input(self) -> TrainPipelineInputMetricsType0:
    """Convert to TrainPipelineInputMetricsType0 instance, required for training pipelines.

    # TODO https://gitlab.xpdeep.com/xpdeep/xpdeep-client/-/issues/279
    """
    return TrainPipelineInputMetricsType0.from_dict({
        metric_name: metric_value.to_model().to_dict() for metric_name, metric_value in self.items()
    })

to_model() -> GlobalExplainPipelineInputMetrics

Convert to GlobalExplainPipelineInputMetrics instance, required for explanation pipelines.

xpdeep-api-client generates two classes for a dictionary, to avoid redundancy we use the same class, for GlobalExplainPipelineInputMetrics, LocalExplainPipelineInputMetrics, and TrainPipelineInputMetricsType0.

TODO https://gitlab.xpdeep.com/xpdeep/xpdeep-client/-/issues/279

Source code in src/xpdeep/metric.py

def to_model(self) -> GlobalExplainPipelineInputMetrics:
    """Convert to GlobalExplainPipelineInputMetrics instance, required for explanation pipelines.

    xpdeep-api-client generates two classes for a dictionary, to avoid redundancy we use the same class,
    for GlobalExplainPipelineInputMetrics, LocalExplainPipelineInputMetrics, and TrainPipelineInputMetricsType0.
    # TODO https://gitlab.xpdeep.com/xpdeep/xpdeep-client/-/issues/279
    """
    return GlobalExplainPipelineInputMetrics.from_dict({
        metric_name: metric_value.to_model().to_dict() for metric_name, metric_value in self.items()
    })