neural_trees.SoftDecisionTreeRegressor#

class neural_trees.SoftDecisionTreeRegressor(depth: int = 4, max_epochs: int = 100, learning_rate: float = 0.01, batch_size: int = 64, penalty_coef: float = 0.001, device: str = 'cpu', random_state: int | None = None, early_stopping: bool = False, validation_fraction: float = 0.1, n_iter_no_change: int = 10, verbose: bool = False, growth: str = 'none', growth_init: str = 'residual', growth_jitter: float = 0.2, growth_budget: str = 'split')[source]#

Bases: RegressorMixin, BaseEstimator

Soft decision tree for regression, scikit-learn compatible.

Parameters:

depthint, default=4: Depth of the complete tree; 2**depth leaves.
max_epochsint, default=100
learning_ratefloat, default=0.01
batch_sizeint, default=64
penalty_coeffloat, default=1e-3: Weight of the balanced-routing penalty, the same regulariser the classifier uses; it discourages gates from sending everything one way.
devicestr, default=”cpu”: "auto" picks CUDA, then MPS, then CPU.
random_stateint or None, default=None
growth{“none”, “incremental”, “per_leaf”}, default=”none”: As on the classifier. "incremental" grows one level at a time and keeps a level only if held-out squared error improves; "per_leaf" splits the leaf carrying the most weighted squared error, one at a time, and stops at the first split that does not pay off. Either one holds out validation_fraction of the training data to decide, so the fixed tree trains on more rows; the classifier’s measurements of that cost apply here too. Without enough rows for a split (fewer than ten) the fit falls back to the full depth and growth_ says so.
growth_init{“random”, “residual”, “residual_gate”}, default=”residual”: How a split’s two children are told apart. Children start at the parent’s value and are pushed apart by growth_jitter: at random, or along the leaf’s residual direction (the mean of y - prediction over the samples that reach it), optionally with the new gate pointed at the samples pulling towards the right child. Identical children would leave the new gate with a zero gradient, as the paper shows for the classifier; the same argument holds for a sum of values.
growth_jitterfloat, default=0.2: Size of that push, in standardised target units.
growth_budget{“split”, “full”}, default=”split”: Whether max_epochs is divided across growth rounds or given to every round.
early_stoppingbool, default=False: Hold out validation_fraction of the training data and stop when the validation loss has not improved for n_iter_no_change epochs, restoring the best parameters.
validation_fractionfloat, default=0.1
n_iter_no_changeint, default=10
verbosebool, default=False

Attributes:

model_the fitted module
n_features_in_int
n_outputs_int
n_iter_int: Epochs actually run.
growth_str: The growth that ran: the growth setting, or "none" when the data could not spare a validation split.
training_history_list of dict: Per-epoch training loss and, with early stopping, validation loss.

Notes

Targets are centred and scaled internally by their (weighted) mean and standard deviation for the optimiser’s sake; predictions are returned in the original units. Leaves start at the target mean, so an untrained tree predicts the mean everywhere, which is the right place to start from.

get_leaf_values() → ndarray[source]#: Leaf values in the original target units, shape (n_leaves, n_outputs).

get_split_weights()[source]#: Gate weight vectors, one per internal node, in feature units.

save_onnx(path: str, feature_names=None, opset: int = 17) → None[source]#: Write to_onnx() to a file.

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SoftDecisionTreeRegressor#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight parameter in fit.

Returns:

selfobject: The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → SoftDecisionTreeRegressor#

Configure whether metadata should be requested to be passed to the score method.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight parameter in score.

Returns:

selfobject: The updated object.

to_hard_tree()[source]#: The gates read as hard decisions and one value per leaf, in target units: a HardRegressionTree that prints its rules. A different model from the soft tree, as for the classifier; measure its agreement before relying on it.

to_numpy(feature_names=None)[source]#: The fitted tree as a NumpySoftTree of kind "regressor": same model, no torch in the prediction path, JSON round trip. Predictions agree with predict() to float32 precision.

to_onnx(feature_names=None, opset: int = 17)[source]#

The fitted tree as an onnx.ModelProto (pip install neural-trees[onnx]).

Standard operators only, so it runs wherever ONNX runs, without torch or this library, and predicts the same thing as predict() to float32 precision. Input X is float32 (n, n_features).