pyfixest-cupy64-absorbed-regressors

Featured

pyfixest demeaner_backend="cupy64" (including its CPU fallback when cupy is absent) is NOT numerically identical to the default numba backend and does NOT drop fully-absorbed/collinear regressors the same way. Use when: (1) adding demeaner_backend="cupy64" to existing pf.feols/fepois calls changes the printed coefficient table, (2) a regression report suddenly gains rows with absurd estimates (e.g. coef 435.8, SE 7106) for controls absorbed by the fixed effects, (3) diffing outputs before/after a backend change, or (4) anything parses a pyfixest text report by line position.

Data & Documents 28 stars 0 forks Updated 6 days ago MIT

Install

View on GitHub

Quality Score: 89/100

Stars 20%
49
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# pyfixest cupy64 backend: absorbed regressors survive, reports change shape ## Problem Adding `demeaner_backend="cupy64"` to an existing `pf.feols()` call is treated as a pure performance switch, but it changes the output: regressors with zero within-FE identifying variation (fully absorbed by the fixed effects) that the default numba backend silently drops are RETAINED by the cupy64 path (also in its scipy/CPU fallback when cupy is not installed). They appear in the coefficient table as non-identified garbage (huge coefficient, huge SE). ## Context / Trigger Conditions Observed 2026-07-18 (Specialist Directors US, H5 re-baseline audit): adding the kwarg to 4 feols sites left the headline triple stable to 4 decimals (B1 diff 1.45e-6, SE diff 9.9e-6) but the text report grew from 366 to 390 lines — 24 new rows for absorbed controls (e.g. `event_x_lrisk = 435.8059`, SE `7106.3365`) under firm-year + director FE, where firm-year-level variables have no identifying variation. ## Solution 1. Treat a backend change as a POTENTIALLY OUTPUT-CHANGING edit: diff the report and expect schema changes, not byte equality. Verify the coefficients of interest at ~4-decimal precision instead. 2. Never interpret retained absorbed-regressor rows as estimates; check within-FE variation before reading nuisance coefficients. 3. Never parse pyfixest text reports by line position; anchor on the variable name of the coefficient you need. 4. When byte-stable reports matter (regression-...

Details

Author
kennethkhoocy
Repository
kennethkhoocy/applied-micro-skills
Created
1 months ago
Last Updated
6 days ago
Language
Python
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

pyfixest-grid-sharding

Diagnose and fix slow pyfixest regression GRIDS (many feols/fepois calls run sequentially) that stay slow despite demeaner_backend="cupy64" and an idle GPU. Use when: (1) a script looping dozens of pf.feols models on a 100k+ row panel takes ~1 min/model, (2) process inspection shows ~1-1.5 cores busy and nvidia-smi shows ~0% GPU utilization with a resident cupy context, (3) planning any worker prompt that will run a model grid (robustness variants x FE structures x domains). Root cause: per-model CPU-side single-threaded fixed costs (formulaic model-matrix build, interaction construction, singleton detection, cluster vcov) dominate wall time; GPU demeaning is a small slice. Fix: shard the model grid across OS processes and/or use pyfixest multiple-estimation syntax; mandate this IN THE WORKER PROMPT.

28 Updated 6 days ago
kennethkhoocy
AI & Automation Listed

ck-defix

Adversarial anti-fixation pass over a C-K tree - detect where the design reasoning has collapsed onto obvious answers and force alternative partitions. Use when the expansive ratio is low, a branch has no expansive step, the tree looks like a list of existing products, or the user feels stuck or wants to be pushed.

0 Updated 1 months ago
dbbz
AI & Automation Listed

lib-purgedcv

The only genuinely sklearn-protocol-compliant purged and embargoed splitter, and the one that refuses to run until you state when each label resolved - understate evaluation_times and it silently reintroduces the leak while reporting clean folds. TRIGGER - purgedcv, PurgedKFold, PBO, CPCV, probability of backtest overfitting, CombinatorialPurgedCV, PurgedGroupKFold, WalkForwardSplit, prediction_times, evaluation_times, purge_horizon, embargo, audit_splitter, reconstruct_paths, path_metrics, overlapping labels, triple-barrier touch time, mlfinlab.cross_validation. Memory is stale or absent - this package first shipped 2026-05-16 and is at 0.1.6. SKIP for CPCV over portfolio models (lib-skfolio) and for SPA/StepM/MCS (lib-arch). SKIP when the question is WHICH library to choose, or names no library at all - both belong to the domain skill.

1 Updated today
howard-lynn-ye