Interpretation

BaseModel can explain why a scenario model makes specific predictions. It computes attribution scores — numeric measures of how much each input contributes to a prediction — using gradient-based attribution methods: Integrated Gradients (deterministic, default) or GradientSHAP (stochastic, typically faster). Both operate on the input-space Cleora/EMDE sketches the model consumes. Attributions are produced at three levels — data sources, features, and feature values — so you can trace a prediction from the broadest signal (which data source matters most) down to individual values (which brand or price range pushed the score).

In This Section

Prediction Drivers

Generating Attributions

Point interpret() at a predictions file and the checkpoint that produced it. The function computes attributions and writes plots and JSON summaries to the output directory.

from pathlib import Path
from datetime import datetime
from monad.interpretability import interpret

interpret(
    predictions_path=Path("./predictions.tsv"),
    output_path=Path("./interpretations"),
    checkpoint_path=Path("./my_model"),
    device="cuda",
    prediction_date=datetime(2024, 6, 1),
)

You need to provide the paths to your predictions file, output directory, and model checkpoint. For classification models, set target_index to choose which output to explain (0 for binary classification); without it, the call raises an error.

For the full parameter list, see Reference: Interpretability.

Choosing an Attribution Method

interpret() defaults to Integrated Gradients (method="integrated_gradients") — deterministic, reproducible run-to-run, and the right choice for regulated reporting. For production-scale attribution or downstream SHAP-style plots, switch to GradientSHAP, which averages gradients across stochastic baselines and is typically ~2× faster with denser attributions:

interpret(
    predictions_path=Path("./predictions.tsv"),
    output_path=Path("./interpretations"),
    checkpoint_path=Path("./my_model"),
    device="cuda",
    method="gradient_shap",
)

Both methods write the same source_importance.json / per-feature / per-value structure described below — only the numeric attribution values differ. See Attribution Methods for the tradeoff table.

Both methods measure attributions against a reference drawn from your predict set: Integrated Gradients against the average observation, GradientSHAP against a random sample of real observations. A positive attribution therefore means "pushes the prediction up compared with the reference". See Reference Point. If a run does not fit in device memory, see Controlling Memory.

Understanding the Output

The output directory mirrors the data-source / feature / value hierarchy:

output_path/
├── run.json                        # Method, reference, and observation count of the run
├── source_importance.json          # Attribution scores per data source
├── source_importance.png           # Bar chart of data source importance
├── top_features.json               # Features ranked across all data sources
├── top_features.png
└── {data_source}/                  # One directory per data source
    ├── feature_importance.json     # Attribution scores per feature
    ├── feature_importance.png      # Bar chart of feature importance
    ├── Events frequency/           # Only when event counts span several time buckets
    │   └── buckets_importance.png
    └── {feature}/                  # One directory per feature
        ├── values_importance.json
        ├── values_collision.json   # Categorical and recency sketches only
        ├── values_highest_importance.png
        └── values_lowest_importance.png

Each level contains a JSON file with numeric scores and a PNG bar chart. You can consume the JSON programmatically or explore the results interactively with Treemaps.

Each chart's subtitle names the method, the reference it was measured against, and the number of observations, and run.json records the same. Bar labels show the value and its net share, the fraction of the magnitude that points in one direction. Magnitudes are comparable across runs and methods; directions are comparable only between runs with the same reference. values_collision.json tells you how many of a feature's values share sketch positions, and therefore how far its per-value scores describe groups of values rather than single ones. See Output Files for details.