feature-importance-financiallisted

Rank features without believing MDI - it is in-sample, it favours columns with many distinct values, and it splits credit between substitutable features; MDA under-states collinear pairs and leaks outright on a shuffled k-fold. TRIGGER - feature_importances_, feature importance, MDI, mean decrease impurity, Gini importance, MDA, mean decrease accuracy, permutation importance, permutation_importance, single feature importance, SFI, clustered feature importance, "which features matter", "my random forest says this random column is important", "importance changes every run", correlated features importance, substitution effect, Lopez de Prado chapter 8, AFML feature importance. SKIP for purged and embargoed cross-validation itself (lib-purgedcv), for the overlapping-label weights (sample-weights-and-uniqueness), for feature construction and causality (signal-construction), for factor-return attribution (factor-models), and for the trials a feature search spends (backtest-validation).
howard-lynn-ye/fin-skills · ★ 1 · AI & Automation · score 77

Install: claude install-skill howard-lynn-ye/fin-skills

# Feature importance for financial data **Three methods, three different lies, and the worst of them is the default.** Advances in Financial Machine Learning (Lopez de Prado 2018), chapter 8. `feature_importances_` is one attribute access away and it is in-sample, biased toward high-cardinality columns, and split between features that substitute for each other. Permutation importance fixes the first two and not the third. And every one of them is meaningless if the fold was shuffled. Every number below is printed by `scripts/feature_importance.py` (numpy, seed 0, **23 s**, `scikit-learn` optional — the script carries its own CART and bagged forest, so the claims hold on a bare install and are *checked against* sklearn when it is present). ## 1. The design, so "wrong" is measurable n = 1,500. `y = 1.00 * x0 + 0.60 * x2 + e`, where `e` is a rolling mean of 25 shocks — each row's target is fully explained by its own features, and neighbouring rows share most of their *residual*. That is what an overlapping label horizon does, and it is what makes section 5 happen. | feature | distinct values | corr with x0 | corr with y | truth | |---|---|---|---|---| | **x0** | 1,500 | 1.000 | 0.425 | informative, coefficient 1.00 | | **x1** | 1,500 | **0.995** | 0.426 | a noisy copy of x0 — **no independent information** | | **x2** | 1,500 | 0.005 | 0.293 | informative, coefficient 0.60 | | **x3** | **1,500** | 0.032 | 0.016 | **irrelevant**, continuous | | **x4** | **2** | 0.007 | 0.011