neural_trees.distillation.ThresholdBasis#
- class neural_trees.distillation.ThresholdBasis(teacher, n_features: int, k: int = 512)[source]#
Bases:
objectThe inputs a threshold-basis student sees: the original features, then one 0/1 input per (feature, threshold) pair among the teacher’s k splits with the largest total gain, 1 where the row goes right at that split.
A depth-8 soft tree learns its own thresholds slowly and coarsely; the teacher has already found where each feature matters. With these inputs a gate can pick up a boosted model’s cut points directly. On the nine tables of benchmarks/kanit.py (three seeds, k = 512, the tuned XGBoost as teacher; benchmarks/esik.py) one tree was 0.70 points under tuned XGBoost at 20 000 rows with 14 to 99 splits, against XGBoost’s 4 031 to 115 351; with 500 to 5 000 rows (rivals re-tuned at each size) the three-tree group was level with tuned XGBoost and LightGBM, and 0.40 points above XGBoost at 500 rows (18 of 27 cells higher, Wilcoxon p = 0.012). These numbers were measured with the development script this class was taken from; the library code gave the same splits and accuracy within 0.1 point on the cells checked.