neural_trees.OmnivariateDecisionTree#

class neural_trees.OmnivariateDecisionTree(max_depth: int = 5, min_samples_split: int = 10, cv_folds: int = 3, selection: str = 'accuracy', alpha: float = 0.05, min_samples_test: int = 50, random_state: int | None = None, n_jobs: int | None = None, class_weight=None, min_weight_fraction_leaf: float = 0.0)[source]#

Bases: ClassifierMixin, BaseEstimator

Omnivariate Decision Tree Classifier (sklearn-compatible).

At each node, automatically selects the best split type from: univariate (single feature), linear (LDA), or nonlinear (MLP) splits, chosen by cross-validation.

Parameters:
max_depthint, default=5

Maximum depth of the tree.

min_samples_splitint, default=10

Minimum number of samples required to split a node.

cv_foldsint, default=3

Number of cross-validation folds used to score split types at a node.

selection{“accuracy”, “test”}, default=”accuracy”

How a node picks its split type.

  • "test" keeps the simplest type that is not significantly worse than the best one, judged by the combined 5x2cv F test this library ships. Simplicity runs univariate, then linear, then nonlinear.

  • "accuracy" takes whichever type scored highest on the folds. That is the ad hoc accuracy comparison the README argues against, and it biases toward the most flexible candidate, since noise helps whoever has the most capacity to exploit it.

"accuracy" is still the default, because the principled rule costs accuracy here. On Breast Cancer the accuracy rule picks a nonlinear split at 15 of 21 nodes and scores 0.971; the test finds those MLPs no better than a univariate split at the 0.05 level, picks univariate, and scores 0.959. Significance at a node does not compose into performance of the tree, and this library would rather say that than pick the answer that sounds better.

alphafloat, default=0.05

Significance level for the test under selection="test".

min_samples_testint, default=50

Smallest group size at a node that still gets a hypothesis test. The 5x2cv F test needs each half of a 2-fold split to hold both groups, five times over; below this the node falls back to the accuracy rule rather than treating a test with no power as evidence of no difference. The default matters: at 20 the test fires on nodes too small to resolve anything and Wine gives 0.959 with 9.3 nodes, while at 50 it gives 0.987 with 5.1 (three seeds of five-fold cross-validation, depth 3).

random_stateint or None, default=None

Seed for everything stochastic in the fit: the k-means that pairs classes into two groups at each node, the stump and MLP candidates, and the fold assignment of the F test. Same seed, same tree. The seed moves the tree more than usual here because the type of split chosen at a node can flip: on Breast Cancer with selection="test" and max_depth=3, five seeds gave (univariate, linear, nonlinear) counts of (2, 1, 1), (0, 0, 1), (0, 0, 1), (1, 1, 2) and (2, 1, 2). Until this parameter existed a fixed seed of 42 hid that.

n_jobsint or None, default=None

Processes for the per-node model selection, scikit-learn convention: None is one, -1 is every core. The three candidate split types at a node are cross-validated in parallel; the tree is identical for every n_jobs on a fixed random_state. The candidates are few and the nodes small, so the gain is bounded by the MLP’s share of the work. Measured on Breast Cancer at max_depth=3: with selection="accuracy" 0.82 s sequential against 0.68 s with three workers; with selection="test" 2.72 s against 2.54 s, because the F tests that follow the selection stay sequential. Never more workers than candidates are started, so -1 cannot be slower than 1.

class_weightdict, “balanced” or None, default=None

Weights per class, combined multiplicatively with sample_weight. “balanced” uses n_samples / (n_classes * bincount(y)).

min_weight_fraction_leaffloat, default=0.0

Smallest fraction of the total sample weight a child may hold, as in scikit-learn’s trees. Stops a split from isolating a region whose weight is negligible, which is how a down-weighted class would otherwise keep leaves of its own.

References

Yıldız, O. T., & Alpaydın, E. (2001). Omnivariate Decision Trees. IEEE Transactions on Neural Networks, 12(6), 1539-1546.

Examples

>>> from neural_trees import OmnivariateDecisionTree
>>> from sklearn.datasets import load_wine
>>> X, y = load_wine(return_X_y=True)
>>> odt = OmnivariateDecisionTree(max_depth=4)
>>> odt.fit(X, y)
>>> odt.score(X, y)
fit(X, y, sample_weight=None) → OmnivariateDecisionTree[source]#

Fit the tree.

sample_weight reaches the two-group construction (weighted class centroids), the model selection (weighted fold accuracy, and the candidate’s own fit where it accepts weights: the stump does, scikit-learn’s LDA does not, its MLP only from 1.7) and the leaf distributions. The 5x2cv F test under selection=”test” compares unweighted accuracies. Size limits count rows. Without weights the fit is unchanged.

get_split_type_distribution() → Dict[str, int][source]#

Count how many nodes use each split type.

predict_proba(X) → ndarray[source]#

Predict class probabilities from the reached leaf’s class distribution.

Parameters:
Xarray-like of shape (n_samples, n_features)
Returns:
probandarray of shape (n_samples, n_classes)

Class probabilities in the order of self.classes_, each row summing to 1.

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → OmnivariateDecisionTree#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns:
selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → OmnivariateDecisionTree#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:
selfobject

The updated object.