neural_trees.OmnivariateDecisionTree#

class neural_trees.OmnivariateDecisionTree(max_depth: int = 5, min_samples_split: int = 10, cv_folds: int = 3, selection: str = 'accuracy', alpha: float = 0.05, min_samples_test: int = 50)[source]#

Bases: ClassifierMixin, BaseEstimator

Omnivariate Decision Tree Classifier (sklearn-compatible).

At each node, automatically selects the best split type from: univariate (single feature), linear (LDA), or nonlinear (MLP) splits, chosen by cross-validation.

Parameters:
max_depthint, default=5

Maximum depth of the tree.

min_samples_splitint, default=10

Minimum number of samples required to split a node.

cv_foldsint, default=3

Number of cross-validation folds used to score split types at a node.

selection{“accuracy”, “test”}, default=”accuracy”

How a node picks its split type.

  • "test" keeps the simplest type that is not significantly worse than the best one, judged by the combined 5x2cv F test this library ships. Simplicity runs univariate, then linear, then nonlinear.

  • "accuracy" takes whichever type scored highest on the folds. That is the ad hoc accuracy comparison the README argues against, and it biases toward the most flexible candidate, since noise helps whoever has the most capacity to exploit it.

"accuracy" is still the default, because the principled rule costs accuracy here. On Breast Cancer the accuracy rule picks a nonlinear split at 15 of 21 nodes and scores 0.971; the test finds those MLPs no better than a univariate split at the 0.05 level, picks univariate, and scores 0.959. Significance at a node does not compose into performance of the tree, and this library would rather say that than pick the answer that sounds better.

alphafloat, default=0.05

Significance level for the test under selection="test".

min_samples_testint, default=50

Smallest group size at a node that still gets a hypothesis test. The 5x2cv F test needs each half of a 2-fold split to hold both groups, five times over; below this the node falls back to the accuracy rule rather than treating a test with no power as evidence of no difference. The default matters: at 20 the test fires on nodes too small to resolve anything and Wine drops from 0.977 to 0.961, while at 50 it recovers completely.

References

Yıldız, O. T., & Alpaydın, E. (2001). Omnivariate Decision Trees. IEEE Transactions on Neural Networks, 12(6), 1539-1546.

Examples

>>> from neural_trees import OmnivariateDecisionTree
>>> from sklearn.datasets import load_wine
>>> X, y = load_wine(return_X_y=True)
>>> odt = OmnivariateDecisionTree(max_depth=4)
>>> odt.fit(X, y)
>>> odt.score(X, y)
get_split_type_distribution() → Dict[str, int][source]#

Count how many nodes use each split type.

predict_proba(X) → ndarray[source]#

Predict class probabilities from the reached leaf’s class distribution.

Parameters:
Xarray-like of shape (n_samples, n_features)
Returns:
probandarray of shape (n_samples, n_classes)

Class probabilities in the order of self.classes_, each row summing to 1.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → OmnivariateDecisionTree#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:
selfobject

The updated object.