innvestigate.analyzer¶

Interface and base code¶

exception innvestigate.analyzer.base.NotAnalyzeableModelException¶: Indicates that the model cannot be analyzed by an analyzer.

class innvestigate.analyzer.base.AnalyzerBase(model, disable_model_checks=False)¶

The basic interface of an iNNvestigate analyzer.

This class defines the basic interface for analyzers:

>>> model = create_keras_model()
>>> a = Analyzer(model)
>>> a.fit(X_train)  # If analyzer needs training.
>>> analysis = a.analyze(X_test)
>>>
>>> state = a.save()
>>> a_new = A.load(*state)
>>> analysis = a_new.analyze(X_test)

Parameters:	model – A Keras model. disable_model_checks – Do not execute model checks that enforce compatibility of analyzer and model.

Note

To develop a new analyzer derive from AnalyzerNetworkBase.

analyze(X)¶

Analyze the behavior of model on input X.

Parameters:	X – Input as expected by model.

fit(*args, **kwargs)¶

Stub that eats arguments. If an analyzer needs training include TrainerMixin.

Parameters:	disable_no_training_warning – Do not warn if this function is called despite no training is needed.

fit_generator(*args, **kwargs)¶

Stub that eats arguments. If an analyzer needs training include TrainerMixin.

Parameters:	disable_no_training_warning – Do not warn if this function is called despite no training is needed.

static load(class_name, state)¶

Resembles an analyzer from the state created by analyzer.save().

Parameters:	class_name – The analyzer’s class name. state – The analyzer’s state.

static load_npz(fname)¶

Resembles an analyzer from the file created by analyzer.save_npz().

Parameters:	fname – The file’s name.

save()¶

Save state of analyzer, can be passed to Analyzer.load() to resemble the analyzer.

Returns:	The class name and the state.

save_npz(fname)¶

Save state of analyzer, can be passed to Analyzer.load_npz() to resemble the analyzer.

Parameters:	fname – The file’s name.

class innvestigate.analyzer.base.TrainerMixin¶

Mixin for analyzer that adapt to data.

This convenience interface exposes a Keras like training routing to the user.

fit(X=None, batch_size=32, **kwargs)¶: Takes the same parameters as Keras’s model.fit() function.

fit_generator(*args, **kwargs)¶: Takes the same parameters as Keras’s model.fit_generator() function.

class innvestigate.analyzer.base.OneEpochTrainerMixin¶

Exposes the same interface and functionality as TrainerMixin except that the training is limited to one epoch.

fit(*args, **kwargs)¶: Same interface as fit() of TrainerMixin except that the parameter epoch is fixed to 1.

fit_generator(*args, **kwargs)¶: Same interface as fit_generator() of TrainerMixin except that the parameter epoch is fixed to 1.

class innvestigate.analyzer.base.AnalyzerNetworkBase(model, neuron_selection_mode='max_activation', allow_lambda_layers=False, **kwargs)¶

Convenience interface for analyzers.

This class provides helpful functionality to create analyzer’s. Basically it:

takes the input model and adds a layer that selects the desired output neuron to analyze.
passes the new model to _create_analysis() which should return the analysis as Keras tensors.
compiles the function and serves the output to analyze() calls.
allows _create_analysis() to return tensors that are intercept for debugging purposes.

Parameters:	neuron_selection_mode – How to select the neuron to analyze. Possible values are ‘max_activation’, ‘index’ for the neuron (expects indices at `analyze()` calls), ‘all’ take all neurons. allow_lambda_layers – Allow the model to contain lambda layers.

analyze(X, neuron_selection=None)¶

Same interface as Analyzer besides

Parameters:	neuron_selection – If neuron_selection_mode is ‘index’ this should be an integer with the index for the chosen neuron.

create_analyzer_model()¶: Creates the analyze functionality. If not called beforehand it will be called by analyze().

class innvestigate.analyzer.base.ReverseAnalyzerBase(model, reverse_verbose=False, reverse_clip_values=False, reverse_project_bottleneck_layers=False, reverse_check_min_max_values=False, reverse_check_finite=False, reverse_keep_tensors=False, reverse_reapply_on_copied_layers=False, **kwargs)¶

Convenience class for analyzers that revert the model’s structure.

This class contains many helper functions around the graph reverse function innvestigate.utils.keras.graph.reverse_model().

The deriving classes should specify how the graph should be reverted by implementing the following functions:

_reverse_mapping(layer)() given a layer this function returns a reverse mapping for the layer as specified in innvestigate.utils.keras.graph.reverse_model() or None.

This function can be implemented, but it is encouraged to implement a default mapping and add additional changes with the function _add_conditional_reverse_mapping() (see below).

The default behavior is finding a conditional mapping (see below), if none is found, _default_reverse_mapping() is applied.
_default_reverse_mapping() defines the default reverse mapping.
_head_mapping() defines how the outputs of the model should be instantiated before the are passed to the reversed network.

Furthermore other parameters of the function innvestigate.utils.keras.graph.reverse_model() can be changed by setting the according parameters of the init function:

Parameters:

reverse_verbose – Print information on the reverse process.
reverse_clip_values – Clip the values that are passed along the reverted network. Expects tuple (min, max).
reverse_project_bottleneck_layers – Project the value range of bottleneck tensors in the reverse network into another range.
reverse_check_min_max_values – Print the min/max values observed in each tensor along the reverse network whenever analyze() is called.
reverse_check_finite – Check if values passed along the reverse network are finite.
reverse_keep_tensors – Keeps the tensors created in the backward pass and stores them in the attribute _reversed_tensors.
reverse_reapply_on_copied_layers – See innvestigate.utils.keras.graph.reverse_model().

Gradient methods¶

class innvestigate.analyzer.gradient_based.BaselineGradient(model, postprocess=None, **kwargs)¶

Gradient analyzer based on build-in gradient.

Returns as analysis the function value with respect to the input. The gradient is computed via the build in function. Is mainly used for debugging purposes.

Parameters:	model – A Keras model.

class innvestigate.analyzer.gradient_based.Gradient(model, postprocess=None, **kwargs)¶

Gradient analyzer.

Returns as analysis the function value with respect to the input. The gradient is computed via the librarie’s network reverting.

Parameters:	model – A Keras model.

class innvestigate.analyzer.gradient_based.InputTimesGradient(model, **kwargs)¶

Input*Gradient analyzer.

Parameters:	model – A Keras model.

class innvestigate.analyzer.gradient_based.Deconvnet(model, **kwargs)¶

Deconvnet analyzer.

Applies the “deconvnet” algorithm to analyze the model.

Parameters:	model – A Keras model.

class innvestigate.analyzer.gradient_based.GuidedBackprop(model, **kwargs)¶

Guided backprop analyzer.

Applies the “guided backprop” algorithm to analyze the model.

Parameters:	model – A Keras model.

class innvestigate.analyzer.gradient_based.IntegratedGradients(model, steps=64, **kwargs)¶

Integrated gradient analyzer.

Applies the “integrated gradient” algorithm to analyze the model.

Parameters:	model – A Keras model. steps – Number of steps to use average along integration path.

class innvestigate.analyzer.gradient_based.SmoothGrad(model, augment_by_n=64, **kwargs)¶

Smooth grad analyzer.

Applies the “smooth grad” algorithm to analyze the model.

Parameters:	model – A Keras model. augment_by_n – Number of distortions to average for smoothing.

Layer-wise relevance propagation¶

class innvestigate.analyzer.relevance_based.relevance_analyzer.BaselineLRPZ(model, **kwargs)¶

LRPZ analyzer - for testing purpose only.

Applies the “LRP-Z” algorithm to analyze the model. Based on the gradient times the input formula. This formula holds only for ReLU/MaxPooling networks, for which LRP-Z collapses into the stated formula.

Parameters:	model – A Keras model.

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRP(model, *args, **kwargs)¶

Base class for LRP-based model analyzers

Parameters:	model – A Keras model. rule – A rule can be a string or a Rule object, lists thereof or a list of conditions [(Condition, Rule), … ] gradient. input_layer_rule – either a Rule object, atuple of (low, high) the min/max pixel values of the inputs

create_rule_mapping(layer, reverse_state)¶

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPZ(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-Z rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPZIgnoreBias(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-Z-ignore-bias rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPEpsilon(model, epsilon=1e-07, bias=True, *args, **kwargs)¶: LRP-analyzer that uses the LRP-Epsilon rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPEpsilonIgnoreBias(model, epsilon=1e-07, *args, **kwargs)¶: LRP-analyzer that uses the LRP-Epsilon-ignore-bias rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPWSquare(model, *args, **kwargs)¶: LRP-analyzer that uses the DeepTaylor W**2 rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPFlat(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-Flat rule

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPAlphaBeta(model, alpha=None, beta=None, bias=True, *args, **kwargs)¶: Base class for LRP AlphaBeta

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPAlpha2Beta1(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-alpha-beta rule with a=2,b=1

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPAlpha2Beta1IgnoreBias(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-alpha-beta-ignbias rule with a=2,b=1

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPAlpha1Beta0(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-alpha-beta rule with a=1,b=0

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPAlpha1Beta0IgnoreBias(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-alpha-beta-ignbias rule with a=1,b=0

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPZPlus(model, *args, **kwargs)¶: LRP-analyzer that uses the LRP-alpha-beta rule with a=1,b=0

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPZPlusFast(model, *args, **kwargs)¶: The ZPlus rule is a special case of the AlphaBetaRule for alpha=1, beta=0 and assumes inputs x >= 0.

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPSequentialPresetA(model, epsilon=0.1, *args, **kwargs)¶: Special LRP-configuration for ConvNets

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPSequentialPresetB(model, epsilon=0.1, *args, **kwargs)¶: Special LRP-configuration for ConvNets

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPSequentialPresetAFlat(model, *args, **kwargs)¶: Special LRP-configuration for ConvNets

class innvestigate.analyzer.relevance_based.relevance_analyzer.LRPSequentialPresetBFlat(model, *args, **kwargs)¶: Special LRP-configuration for ConvNets

Pattern methods¶

class innvestigate.analyzer.pattern_based.PatternNet(model, patterns=None, pattern_type=None, **kwargs)¶

PatternNet analyzer.

Applies the “PatternNet” algorithm to analyze the model’s predictions.

Parameters:	model – A Keras model. patterns – Pattern computed by `innvestigate.tools.PatternComputer`. If None `fit()` needs to be called. allow_lambda_layers – Approximate lambda layers with the gradient. reverse_project_bottleneck_layers – Project the analysis vector into range [-1, +1]. (default: True)

class innvestigate.analyzer.pattern_based.PatternAttribution(model, patterns=None, pattern_type=None, **kwargs)¶

PatternAttribution analyzer.

Applies the “PatternNet” algorithm to analyze the model’s predictions.

Parameters:	model – A Keras model. patterns – Pattern computed by `innvestigate.tools.PatternComputer`. If None `fit()` needs to be called. allow_lambda_layers – Approximate lambda layers with the gradient. reverse_project_bottleneck_layers – Project the analysis vector into range [-1, +1]. (default: True)

Deep Taylor¶

class innvestigate.analyzer.deeptaylor.DeepTaylor(model, *args, **kwargs)¶

DeepTaylor for ReLU-networks with unbounded input

This class implements the DeepTaylor algorithm for neural networks with ReLU activation and unbounded input ranges.

Parameters:	model – A Keras model.

class innvestigate.analyzer.deeptaylor.BoundedDeepTaylor(model, low=None, high=None, **kwargs)¶

DeepTaylor for ReLU-networks with bounded input

This class implements the DeepTaylor algorithm for neural networks with ReLU activation and bounded input ranges.

Parameters:	model – A Keras model. low – Lowest value of the input range. See Z_B rule. high – Highest value of the input range. See Z_B rule.

DeepLIFT¶

class innvestigate.analyzer.deeplift.DeepLIFTWrapper(model, **kwargs)¶

Wrapper around DeepLIFT package

This class wraps the DeepLIFT package. For further explanation of the parameters check out: https://github.com/kundajelab/deeplift

Parameters:	model – A Keras model. nonlinear_mode – The nonlinear mode parameter. reference_inputs – The reference input used for DeepLIFT. verbose – Verbosity of the DeepLIFT package.
Note:	Requires the deeplift package.

analyze(X, neuron_selection=None)¶

Same interface as Analyzer besides

Parameters:	neuron_selection – If neuron_selection_mode is ‘index’ this should be an integer with the index for the chosen neuron.

Misc¶

class innvestigate.analyzer.misc.Random(model, stddev=1, **kwargs)¶

Returns noise.

Returns the Gaussian noise as analysis.

Parameters:	model – A Keras model. stddev – The standard deviation of the noise.

class innvestigate.analyzer.misc.Input(model, neuron_selection_mode='max_activation', allow_lambda_layers=False, **kwargs)¶

Returns the input.

Returns the input as analysis.

Parameters:	model – A Keras model.

innvestigate.analyzer¶

Interface and base code¶

Gradient methods¶

Layer-wise relevance propagation¶

Pattern methods¶

Deep Taylor¶

DeepLIFT¶

Misc¶

iNNvestigate

Navigation

Related Topics