chore: alpha cleanup and beta prep - #2
Merged
Merged
Conversation
Dead code removal (~2,500 lines): - Deleted selection/, validation/, visualization/, pipeline/ modules - Deleted config/feature_config.py, config/validation.py, core/deprecation.py - Deleted labeling backward-compat shims (core.py, barriers.py, barrier_utils.py) - Removed all __getattr__ error traps, D06 aliases, deprecated params - Cleaned bar samplers: removed initial_expectation/initial_run_expectation - Cleaned mom(): timeperiod → period New tests: - +58 comprehensive volatility tests (all 11 non-TA-Lib estimators + Bollinger) CI improvements: - ta-lib separated from dev deps, test job uses --extra ta - 57 perf tests marked @pytest.mark.perf, excluded from default runs - Removed 6 calendar test deselections (bugs fixed) - Removed mypy config block and optional extra Bug fixes: - EquityCalendar._next_basic_open: returned past time during market hours - EquityCalendar._previous_basic_close: returned future time during market hours Docs: - Created CHANGELOG.md (full alpha history a3-a11) - Updated user guides for features, labeling, bars - Added dataset-builder, discovery, preprocessing, fractional-differencing guides
There was a problem hiding this comment.
Pull request overview
This PR performs a broad “alpha cleanup / beta prep” refactor: removing deprecated/dead modules and shims, standardizing test markers/deps, fixing EquityCalendar edge behavior during market hours, and expanding documentation and volatility test coverage.
Changes:
- Removed deprecated/dead packages and compatibility shims (
selection/,validation/,visualization/,pipeline/, labeling shims, deprecation helpers). - Added comprehensive volatility test suite and standardized performance tests under
@pytest.mark.perf(excluded by default). - Updated calendar logic/tests and refreshed user-guide documentation; adjusted CI to install TA-Lib only for the test job.
Reviewed changes
Copilot reviewed 87 out of 88 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/visualization/test_summary.py | Removed visualization export_plot tests (module removed). |
| tests/visualization/init.py | Removed visualization test package init (module removed). |
| tests/test_wclprice.py | Marked benchmark as perf. |
| tests/test_var.py | Marked benchmark as perf. |
| tests/test_ultosc.py | Marked benchmark as perf. |
| tests/test_typprice.py | Marked benchmark as perf. |
| tests/test_tsf.py | Marked benchmark as perf. |
| tests/test_trix.py | Marked benchmark as perf. |
| tests/test_trange.py | Marked benchmark as perf. |
| tests/test_timezone_handling.py | Updated expectations for basic open/close helpers during session. |
| tests/test_talib_p0_indicators.py | Updated MOM API usage (period=), marked perf benchmarks. |
| tests/test_talib_accuracy.py | Marked select performance tests as perf. |
| tests/test_t3.py | Marked benchmark as perf. |
| tests/test_stochf.py | Marked benchmark as perf. |
| tests/test_roc_variants.py | Marked benchmark as perf. |
| tests/test_risk.py | Marked performance test as perf. |
| tests/test_pipeline_engine.py | Removed pipeline engine tests (pipeline removed). |
| tests/test_performance.py | Marked performance tests as perf. |
| tests/test_optimized_indicators.py | Marked performance comparison as perf. |
| tests/test_new_indicators.py | Updated MOM API usage; renamed performance marker to perf. |
| tests/test_natr.py | Marked benchmark as perf. |
| tests/test_midprice.py | Marked benchmark as perf. |
| tests/test_midpoint.py | Marked benchmark as perf. |
| tests/test_medprice.py | Marked benchmark as perf. |
| tests/test_math_operators.py | Marked benchmark as perf. |
| tests/test_linearreg_family.py | Marked benchmark as perf. |
| tests/test_linearreg.py | Marked benchmark as perf. |
| tests/test_labeling.py | Removed tests for legacy import shims; marked perf benchmarks as perf. |
| tests/test_kama.py | Marked benchmark as perf. |
| tests/test_integration_pipeline.py | Removed end-to-end pipeline integration tests (pipeline removed). |
| tests/test_imi.py | Marked benchmark as perf. |
| tests/test_fdiff.py | Removed pipeline integration test referencing removed pipeline. |
| tests/test_dm_indicators.py | Marked benchmark as perf. |
| tests/test_directional_indicators.py | Marked benchmark as perf. |
| tests/test_config_system.py | Removed tests tied to removed feature_config validators. |
| tests/test_cmo.py | Marked benchmark as perf. |
| tests/test_bars_specialized.py | Updated sampler tests for removed deprecated params; clarified dynamic threshold behavior. |
| tests/test_bars.py | Updated sampler construction for removed deprecated params; removed pipeline integration tests. |
| tests/test_avgprice.py | Marked benchmark as perf. |
| tests/test_avgdev.py | Marked benchmark as perf. |
| tests/test_adxr.py | Marked benchmark as perf. |
| tests/test_adosc.py | Marked benchmark as perf. |
| tests/test_ad.py | Marked benchmark as perf. |
| tests/features/volatility/test_volatility_comprehensive.py | Added comprehensive non-TA-Lib volatility estimator tests + Bollinger enhancements. |
| tests/core/test_calendars.py | Updated calendar tests to reflect corrected “during session” open/close behavior and avoid holiday ambiguity. |
| tests/bars/test_vectorized_bars.py | Updated AFML parameter estimation tests; removed deprecated-param test. |
| tests/bars/test_run_bars.py | Removed deprecated initial_run_expectation warning tests. |
| tests/bars/test_imbalance_bars.py | Removed deprecated initial_expectation warning tests. |
| src/ml4t/engineer/visualization/summary.py | Removed plot export utility module. |
| src/ml4t/engineer/visualization/init.py | Removed visualization package exports. |
| src/ml4t/engineer/validation/cv.py | Removed validation stub module. |
| src/ml4t/engineer/validation/init.py | Removed validation package init. |
| src/ml4t/engineer/validation/README.md | Removed validation README (module removed). |
| src/ml4t/engineer/selection/init.py | Removed selection shim with custom __getattr__ ImportErrors. |
| src/ml4t/engineer/preprocessing.py | Updated docstrings to reference ml4t-diagnostic recommendation source. |
| src/ml4t/engineer/pipeline/engine.py | Removed DAG pipeline engine implementation. |
| src/ml4t/engineer/pipeline/init.py | Removed pipeline package exports. |
| src/ml4t/engineer/labeling/core.py | Removed compatibility shim that raised custom ImportError. |
| src/ml4t/engineer/labeling/barriers.py | Removed compatibility shim that raised custom ImportError. |
| src/ml4t/engineer/labeling/barrier_utils.py | Removed compatibility shim that raised custom ImportError. |
| src/ml4t/engineer/labeling/init.py | Removed custom __getattr__ trap for removed exports. |
| src/ml4t/engineer/features/momentum/mom.py | Removed deprecated timeperiod alias + deprecation resolver; default period=10 when None. |
| src/ml4t/engineer/core/deprecation.py | Removed deprecation helper utilities (no longer used). |
| src/ml4t/engineer/core/calendars/equity.py | Fixed basic next-open / previous-close behavior during market hours. |
| src/ml4t/engineer/config/validation.py | Removed feature-config validation types/enums (moved out). |
| src/ml4t/engineer/config/labeling.py | Removed custom __getattr__ trap for removed exports. |
| src/ml4t/engineer/config/feature_config.py | Removed feature evaluator configs (moved to ml4t-diagnostic). |
| src/ml4t/engineer/config/init.py | Updated public config exports; removed feature-eval/D06 aliases; added note about move to ml4t-diagnostic. |
| src/ml4t/engineer/bars/vectorized.py | Removed deprecated initial_expectation parameter and references. |
| src/ml4t/engineer/bars/run.py | Removed deprecated initial_run_expectation parameter and warning logic. |
| src/ml4t/engineer/bars/imbalance.py | Removed deprecated initial_expectation parameter and warning logic. |
| src/ml4t/engineer/init.py | Removed exports/imports for deleted modules (pipeline/validation/visualization). |
| pyproject.toml | Added perf marker exclusion by default; removed mypy config; adjusted deps; added calendars dev dep. |
| docs/user-guide/preprocessing.md | Added preprocessing user guide. |
| docs/user-guide/ml-readiness.md | Added book reference note. |
| docs/user-guide/labeling.md | Expanded labeling guide significantly (all methods, config, weighting, etc.). |
| docs/user-guide/fractional-differencing.md | Added fractional differencing guide. |
| docs/user-guide/features.md | Expanded feature guide (API usage, categories, discovery, requirements). |
| docs/user-guide/discovery.md | Added feature discovery guide (registry/catalog). |
| docs/user-guide/dataset-builder.md | Added dataset builder guide. |
| docs/user-guide/bars.md | Added book reference note. |
| docs/index.md | Linked newly added user guides from docs index. |
| docs/getting-started/quickstart.md | Updated “Next steps” links to new guides. |
| docs/audit/book-integration-audit.md | Added book integration audit doc. |
| README.md | Added documentation links section. |
| CHANGELOG.md | Added changelog and documented removals/additions/changes. |
| .github/workflows/ci.yml | Updated CI test job to install TA-Lib C library and use --extra ta; removed prior calendar test deselections. |
Comments suppressed due to low confidence (1)
pyproject.toml:144
[project.optional-dependencies].devstill includesta-lib, which conflicts with the PR goal of separating TA-Lib from dev dependencies (so lint/typecheck/dev installs don't require the TA-Lib C library). Consider removingta-libfrom thedevextra and relying on the existingtaextra (keeping it inallif desired).
"pre-commit>=3.3.0",
"duckdb>=0.9.0",
"pandas-market-calendars>=4.0.0",
"ta-lib>=0.4.0",
"yfinance>=0.2.0",
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
selection/,validation/,visualization/,pipeline/, deprecated shims,__getattr__error traps, D06 aliases--extra ta), mark 57 perf tests with@pytest.mark.perfEquityCalendar._next_basic_open/_previous_basic_closereturning wrong times during market hoursTest plan
--extra taon Python 3.11/3.12/3.13ImportError(no custom error traps)