September 28, 2026

I Built neural-trees. It Is Ready for Other People. Here Is What It Does and What It Does Not.

I have been building neural-trees for the past few months and it is now at a point where I would put it in front of other people.

It is a scikit-learn compatible library of tree-shaped models that train by gradient descent: soft decision trees, multivariate and omnivariate trees, a hierarchical mixture of experts, and a network that grows while it learns. The algorithms come from Ethem Alpaydin's group; the implementations, the fixes and the measurements are mine.

Three decision boundaries on the same two-moons data: a hard CART tree with axis-aligned steps, a multivariate tree with straight cuts, and a soft tree with a smooth boundary
The same 400 points, three kinds of tree, default settings. A hard tree can only cut along one feature at a time, so it builds a staircase and fits the training points best. A multivariate tree cuts with a weighted sum. A soft tree replaces each cut with a sigmoid gate, so every point flows down both sides in proportion and the boundary comes out smooth. The training accuracies are printed above each panel; the staircase wins on training data, which is exactly the kind of number the rest of this post is careful about.

Two things I care about more than the model list

First, every claim is measured. The README benchmark table is regenerated in CI and the build fails if a cell drifts. The design choices the papers leave open (how a soft tree should grow, whether a new unit should start from the residual, when to stop) are each settled by a measurement, not an opinion, and the ones that went against me are documented too.

Second, the model can explain itself. explain() gives, for any single prediction, the path through the tree, the features that decided each gate, and the smallest change that would flip the answer, checked by re-predicting. The fitted tree exports to a JSON file that predicts with numpy alone, and to ONNX, where a prediction takes about 11 microseconds and needs neither torch nor the library.

The structure of a soft decision tree fitted on the two-moons data: seven gates with their two largest weights and the share of data reaching them, edges labelled with branch probabilities, leaves coloured by class
What the soft tree above actually is. Each box is a gate: the two features that weigh most in it and the share of the data that reaches it. The number on an edge is the mean probability of taking that branch, and the edge is drawn thicker the more data flows through it. Leaves are coloured by the class they predict. This is the drawing the playground shows for any model on any of its datasets, and the same structure the explanation walks through for a single prediction.

One honest result

On 24 datasets against XGBoost, LightGBM, GRANDE and NODE, all untuned, the soft models win on small tables (a few hundred rows) and lose above a thousand. If you have thousands of rows and only care about accuracy, use LightGBM. If you have a few hundred and have to explain every decision, this is worth ten minutes.

The first complete case study is on the public German Credit data, 1 000 loan applicants with the dataset's own cost matrix. The soft tree ties logistic regression on accuracy, AUC and cost, and adds a small tree of rules and a per-applicant path with a counterfactual a real applicant could reach. The untuned boosted models match it on accuracy and lose on cost, because their probabilities are not calibrated and a cost-weighted decision is exactly where calibration is paid for. The whole package, from the fold-level numbers to the model document, regenerates from one script: cases/german-credit.

Where to find it

You can try it on your own CSV or Excel file in the browser, no install: neural-trees.streamlit.app.

If you have a table of a few hundred to a few thousand rows where every decision has to be explained, I will run a free case study on it within two weeks; the German Credit package above is what you would get. Issues and pull requests are welcome; the open ones so far all came with a measured report, and I would like to keep it that way.

← Back to all posts