neural_trees.DistilledForest#

class neural_trees.DistilledForest(teacher=None, student=None, n_trees: int = 3, alpha: float = 0.0, threshold: float = 0.01, synthetic: int = 10, random_state: int | None = None)[source]#

Bases: DistilledTree

A few soft trees trained on what a strong model learned: the same teacher and transfer rows as DistilledTree, then n_trees students, each on its own bootstrap sample of those rows. The group predicts with the sum of the trees’ log probabilities.

Use it when accuracy matters more than having one tree to read. Each tree is still a SoftDecisionTree and can be explained and exported on its own (estimators_), but the prediction is the group’s.

Measured with an XGBoost teacher (300 trees of depth 6), ten MUNGE passes and depth-8 students, on the 11 cells of six OpenML tables up to 20 000 rows (benchmarks/grup.py with MUNGE_PASSES=10): three trees were 0.38 points above one distilled tree (10 cells higher, none lower, p=0.001) and 0.9 below the teacher (p=0.12). With about 450 splits in all, the group was 1.2 points above XGBoost with 10 trees (521 splits; 10 of 11 cells, p=0.003) and level with XGBoost with 100 trees (4 159 splits; -0.4, p=0.58). Five trees were no better than three, and depth-6 students no better than one depth-8 tree. It fits in about three times the time of DistilledTree.

Parameters are those of DistilledTree, plus

n_treesint, default=3

Number of student trees.

Attributes:
teacher_the fitted teacher
estimators_list of the fitted student trees
classes_, n_features_in_
explain(X, **kwargs)[source]#

The student tree’s explanation; see SoftDecisionTree.explain.

fidelity(X) → float[source]#

Share of rows of X where the group predicts the teacher’s class.

fit(X, y, sample_weight=None, X_unlabeled=None) → DistilledForest[source]#

Fit the teacher, then n_trees students on bootstrap samples of the transfer rows.

set_fit_request(*, X_unlabeled: bool | None | str = '$UNCHANGED$', sample_weight: bool | None | str = '$UNCHANGED$') → DistilledForest#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
X_unlabeledstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for X_unlabeled parameter in fit.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns:
selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → DistilledForest#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:
selfobject

The updated object.

to_numpy(**kwargs)[source]#

One NumpySoftTree per tree; sum their predict_log_proba to predict like the group.