Skip to content

Explain an Existing Model#

Explain an Existing Model vs Learn a Self-Explainable Model#

Two main approaches to explainability can be distinguished:

  1. Learning a self-explainable model: in this case, explanations are generated as part of the training process itself. The model is designed to be intrinsically interpretable.

  2. Explaining an existing model: here, explainability is applied a posteriori using methods that generate explanations from a pre-trained (frozen) model.

Xpdeep provides both types of explanations. While the self-explainable approach was previously presented in the explain section, the approach explaining an existing model is detailed below.

Build the model#

The first step consists in designing a model with frozen weights. We will follow the previous tutorial on how to create an explainable model, and focus on the differences between explaining an existing model and learning a self-explainable model.

Similarly to the model creation designed to be trained and compute explanations, the explanation of an existing model requires your original model to be converted into an explainable model, please follow how to convert your original model to an XpdeepModel.

Info

Your model weights are conserved, therefore you will get the same performance on your original model and on your converted explainable model.

Model Specifications#

Model specification of an existing model needs to be slightly revisited, as we don't require the model itself to be trained but only its explanations.

Set the frozen_model parameter to True to specify the context of explaining an existing model.

from xpdeep.model.model_parameters import ModelDecisionGraphParameters
from xpdeep.model.feature_extraction_output_type import FeatureExtractionOutputType

build_configuration = ModelDecisionGraphParameters(
  graph_depth=3,
  target_homogeneity_weight=0.2,
  discrimination_weight=0.2,
  balancing_weight=0.2,
  target_homogeneity_pruning_threshold=0.9,
  population_pruning_threshold=0.1,
  prune_step=5,
  internal_model_complexity=1,
  feature_extraction_output_type=FeatureExtractionOutputType.VECTOR,
  frozen_model=True
)

Train the Explanations#

The training process remains similar to that of a self-explainable model. The Trainer object requires specifying the max_epochs parameter, which represents the number of epochs dedicated to learning the explanations of the existing model. You should use the FrozenModelTrainer.

Internally, Xpdeep uses its own internal algorithm to compute and train explanations while conserving the original model parameters and performances intact.

from xpdeep.trainer.trainer import FrozenModelTrainer

trainer = FrozenModelTrainer(max_epochs=5)

Get the Explanations#

Finally, once trained, the explanations can be computed and displayed using the exact same process as for the self-explainable model.

You can visualize, analyze, and interpret the explanations of an existing model in the same way as those of a self-explainable model.