DEV · /dev only · ML framework comparison: rendered from ML_FRAMEWORK_COMPARISON.md · main site untouched
All-in-One ZZ Strategy Roman Edgeful 9 NQ Stats
Instruments All Futures Stocks Crypto Forex Ninja
← Back to ML catalog Source: ML_FRAMEWORK_COMPARISON.md (box: /workspace/ml4t_review/) · table scrolls sideways

Open-source ML-for-trading frameworks compared with ML4T 3rd ed., for the NQ 1-minute farm

Compiled 2 Oct 2026 (AEST). This builds on REVIEW.md (same folder), which covers Stefan Jansen's ML4T 3rd edition and already names Qlib/RD-Agent and NautilusTrader.

Method and sources

Fit scores run 0-5 for each need:

"Overall" is my judgement for this use case (rule-based NQ signals already exist; the job is to gate and filter them). It is not a sum.

1. Ranked table

Rank Name URL Stars Last commit / push (AEST) Licence a b c d e Pros Cons
1 purgedcv (purged-cross-validation) https://github.com/eslazarev/purged-cross-validation 35 commit 2026-10-01 19:22 MIT 5 2 5 4 4 One small package (numpy/pandas/scipy/scikit-learn only; Py>=3.10; 0.1.10 released 2026-10-01). It covers PurgedKFold, WalkForwardSplit, CombinatorialPurgedCV, CombinatoriallySymmetricCV (CSCV), probability_of_backtest_overfitting, PSR, DSR, min_track_record_length, minimum_backtest_length, effective_n_trials, and an Optuna TrialSharpeRecorder for honest trial counts (README API table). Purge and embargo are specified with timestamps such as "30min", so trade entry/exit times work directly (smoke-tested). Has a JOSS paper draft Very young, with one maintainer and 35★. No labelling, so it does not cover need b. Pin the version and keep a fallback (skfolio or RiskLabAI)
2 ml4t-diagnostic + ml4t-engineer (ML4T's own libraries) https://github.com/ml4t/diagnostic · https://github.com/ml4t/engineer 33 · 23 commit 2026-09-25 05:56 · 2026-09-25 03:59 MIT 5 5 3 3 3 Diagnostic: WalkForwardCV, CombinatorialCV, DSR, PBO, RAS, FDR, White's Reality Check, trade diagnostics (README). Engineer: triple-barrier, ATR-barrier, trend-scanning and meta-labels, 120 features, tick/volume/dollar bars (README). The ML4T book acts as the manual Polars-first, so the pandas pipeline needs pl.from_pandas. Versions are 0.1.x (0.1.8 / 0.1.6). Py 3.12-3.14 only. 56 / 66 declared dependencies (PyPI, extras included). Effectively one author. Examples are daily and cross-sectional
3 RiskLabAI.py https://github.com/RiskLabAI/RiskLabAI.py 5 commit 2026-08-27 04:08 BSD-3-Clause 5 5 4 3 3 The broadest open López de Prado implementation that is still maintained (3.1.0 on PyPI, 2026-08-26). It includes triple_barrier, meta_labeling, CUSUM event filters, sample-uniqueness weights, purged k-fold, walk-forward, CPCV (plus bagged and adaptive variants), MDI/MDA/SFI, PSR/DSR, PBO and bet sizing (README; module list verified locally). pandas-based, and installs cleanly on Py 3.13 Only 5★ on this repo, so community review is thin. Py 3.12-3.14. The package is broad (causal, PDE and so on), so import only what you need. Its API is AFML-style rather than scikit-learn style
4 Optuna https://github.com/optuna/optuna 14,867 commit 2026-09-30 17:26 MIT 2 3 5 3 5 The standard hyperparameter tuner (5.0.0, 2026-09-07, Py>=3.9). It can drive the farm optimizer and the meta-model tuning, and it logs every trial. That trial log is the input DSR needs, and purgedcv ships optuna_integration.TrialSharpeRecorder for it Not a finance library. With no gate in place it makes overfitting easier. Must be paired with #1
5 skfolio https://github.com/skfolio/skfolio 2,457 commit 2026-10-01 21:47 BSD-3-Clause 3 0 4 2 5 Mature, actively maintained scikit-learn-style library (1.4.10, 2026-09-30, Py>=3.10). CombinatorialPurgedCV with purged_size / embargo_size and path recombination (docs) makes a well-tested second implementation to cross-check #1 Built for portfolio optimisation. Purge and embargo are counted in rows, not time. I found no DSR or PBO in the docs I checked. Pulls in cvxpy/clarabel
6 vectorbt (open source) https://github.com/polakowo/vectorbt 9,258 commit 2026-09-26 21:24 Apache-2.0 + Commons Clause (LICENSE) 1 1 4 4 3 Very fast parameter sweeps on 1-min bars. Useful for building the strategies-by-variants return matrix that PBO/CSCV needs (1.1.1, Py 3.11-3.14) Not an ML or validation library. The Commons Clause forbids selling a product derived from it. VectorBT PRO is paid and closed, so it is not evaluated here
7 FinMLKit https://github.com/quantscious/finmlkit 113 commit 2025-09-23 23:26 MIT 1 4 3 4 2 Numba-fast: tick to bars, footprints/volume profile, triple barrier plus meta-labelling, label-concurrency weights (README) No commits for about a year. Tight pins (numpy~=2.2, pandas~=2.2, numba~=0.61) will clash with other packages. Crypto (Binance) data examples
8 Microsoft Qlib + RD-Agent https://github.com/microsoft/qlib · https://github.com/microsoft/RD-Agent 49,103 · 14,818 commit 2026-09-16 16:56 (push 09-22) · push 2026-09-30 11:47 MIT 2 2 1 2 3 Full AI-quant research platform; RD-Agent adds LLM-driven factor and model search (the "autonomous loop" idea) Equity cross-sectional design with its own data format. PyPI pyqlib was last released 0.9.7 on 2025-08-15. Heavy (rdagent declares 82 dependencies). Wrong shape for single-instrument 1-min rule filtering
9 NautilusTrader https://github.com/nautechsystems/nautilus_trader 29,559 commit 2026-10-01 23:21 LGPL-3.0 0 0 1 5 3 Production event-driven engine (Rust core), futures, native Databento, backtest/live parity (see REVIEW.md §5) Not ML. Replacing the Twin would be a re-platform. Py 3.12-3.14
10 PyBroker https://github.com/edtechre/pybroker 3,556 commit 2026-08-28 13:34 Apache-2.0 + Commons Clause 2 3 2 2 3 Walk-forward model training and bootstrapped metrics built in (README) Its own engine, so it would replace our backtester. The README advertises daily/weekly/monthly intervals. Commons Clause
11 tsfresh / featuretools https://github.com/blue-yonder/tsfresh · https://github.com/alteryx/featuretools 9,434 · 7,688 push 2026-07-06 · push 2026-09-11 MIT · BSD-3 0 3 4 3 3 Automatic feature extraction for the meta-model (pandas) Generates hundreds of features, which raises overfitting risk. featuretools' last PyPI release was 1.31.0 on 2024-05-14
12 Kronos / TimesFM / Chronos (time-series foundation models) https://github.com/shiyu-coder/Kronos · https://github.com/google-research/timesfm · https://github.com/amazon-science/chronos-forecasting 39,772 · 34,043 · 5,949 push 2026-04-13 · 2026-09-30 · 2026-09-17 MIT · Apache-2.0 · Apache-2.0 0 2 2 3 2 Kronos is K-line-native with fine-tuning scripts (README). TimesFM 3.0.2 and Chronos 2.3.2 are on PyPI. A forecast could serve as one meta-model feature Torch/GPU weight. Kronos max_context is 512 bars, about one RTH session of 1-min bars. No commits pushed to Kronos since April. No evidence of edge on NQ intraday; treat as an experiment
13 QuantConnect Lean https://github.com/QuantConnect/Lean 21,842 push 2026-10-02 02:43 Apache-2.0 1 1 1 4 3 Mature C#/Python engine with futures Built around its own data and cloud ecosystem. Not a library for our pandas loop
14 hftbacktest https://github.com/nkaz001/hftbacktest 4,801 push 2025-12-24 01:57 MIT 0 0 1 3 3 Tick/L2 queue-aware HFT backtesting Wrong granularity (order book, market making). No pushes since Dec 2025
15 pypbo https://github.com/esvhd/pypbo 140 push 2026-07-07 AGPL-3.0 4 0 4 3 2 PBO, PSR, MinTRL, MinBTL, DSR (README) AGPL, not on PyPI (404), README's own TODO says "Add test cases". purgedcv covers the same ground under MIT
16 mlfinpy https://github.com/baobach/mlfinpy 84 commit 2025-01-24 02:07 MIT 2 3 3 3 1 Open rewrite of mlfinlab ideas (labelling, sampling, bars) No commits since Jan 2025. Pins numpy<1.27 and numba<0.61 (PyPI 0.1.2), which conflicts with modern pandas stacks
17 mlfinlab (Hudson & Thames) https://github.com/hudson-and-thames/mlfinlab 4,935 push 2023-10-02 13:05 NOASSERTION – – – – 0 Historical reference The public repo is frozen. PyPI mlfinlab returns 404. The purgedcv README notes it "went closed-source and paid"
18 timeseriescv https://github.com/sam31415/timeseriescv 290 push 2022-02-16 MIT 2 0 3 3 0 Early purged k-fold / CPCV Abandoned (PyPI 0.2, 2018)
19 backtesting.py https://github.com/kernc/backtesting.py 9,010 push 2026-08-05 AGPL-3.0 0 0 3 3 2 Simple single-asset backtests No ML or validation. AGPL
20 freqtrade/FreqAI · Jesse https://github.com/freqtrade/freqtrade · https://github.com/jesse-ai/jesse 54,978 · 8,611 push 2026-10-02 04:51 · 2026-10-02 01:03 GPL-3.0 · MIT 1 2 0 0 2 FreqAI's self-retraining loop is a good design pattern Crypto-exchange bots only. No CME
21 FinRL / FinRL-Meta https://github.com/AI4Finance-Foundation/FinRL · https://github.com/AI4Finance-Foundation/FinRL-Meta 16,527 · 1,947 push 2026-09-29 09:32 (both) MIT 0 1 1 2 2 Deep RL environments RL is a poor match for filtering rule-based ORB/SMT entries. PyPI finrl last released 0.3.7 on 2024-04-12
22 LLM-agent newcomers: TradingAgents, Vibe-Trading https://github.com/TauricResearch/TradingAgents · https://github.com/HKUDS/Vibe-Trading 109,461 · 34,424 push 2026-09-29 · 2026-10-02 05:12 Apache-2.0 · MIT 0 0 1 0 2 Huge 2025-26 popularity (Vibe-Trading was created 2026-04-01) LLM discretionary or multi-agent analysis. Not statistical validation or intraday futures ML

Smaller 2026 newcomers also checked, not ranked:

NinjaTrader 8 bridge: none of the ranked libraries ships one. The only GitHub hit for "ninjatrader python" with more than 20★ was TheSnowGuru/CSharpNinja-Python-NinjaTrader8-trading-api-connector-drag-n-drop: 22★, no licence, last push 2024-07-15. Not recommended. The practical bridge is our own: export the trained meta-filter as a threshold or lookup (or a small ONNX/JSON tree) and read it from the existing smt-python/ninja/*.cs strategy. Alternatively, keep filtering in Python and push decisions through the farm's existing NT8 deploy path.

2. Recommendation

Is ML4T the best? It is still the best manual: the most complete, current explanation of how to do this correctly (REVIEW.md). As code to import into this farm, though, a combination of small libraries fits better than the ML4T stack on its own. The reasons: our pipeline is pandas on Python 3.13, the problem is filtering an existing rule signal (not cross-sectional alpha), and ml4t-* are polars-first 0.1.x packages tied to Python 3.12-3.14 with 50-60+ declared dependencies.

Recommended combination

  1. Overfitting gate: purgedcv (rank 1). It brings in only numpy, pandas, scipy and scikit-learn, it is MIT, and it fits the existing .venv. Use it as follows:
    • Run CSCV/PBO on the matrix of DNA variants tried during IS (2026-01-01 to 04-30, net of commissions).
    • Compute DSR using effective_n_trials across all optimizer trials, and PSR/MinTRL on the chosen variant.
    • Use purged walk-forward inside IS, with purge equal to the max trade hold and an embargo of at least one session.
    • Keep OOS (05-01 to 07-31) and holdout (08-01 to 10-09) as locked one-shot checks, alongside the existing ≥100-trade minimum.
    • Cross-check: run the same IS splits through skfolio CombinatorialPurgedCV (rank 5) or RiskLabAI once. This guards against bugs in a 35★ library.
  2. Meta-labelling: RiskLabAI (rank 3) or ml4t-engineer (rank 2) for labels and sample weights, plus a plain scikit-learn / LightGBM classifier.
    • Labels. Our trades already carry stop, target and timeout, so the meta-label is simply "did this trade hit target before stop, net of commission". You barely need a library for that. What is worth borrowing is the sample-uniqueness / concurrency weights and the event-filter utilities.
    • Which library. RiskLabAI stays in pandas. ml4t-engineer has the better docs and the book behind it.
    • Training and gate. Train the filter only inside purgedcv folds, and gate "Twin + filter" against "Twin alone" with the DSR/PBO from step 1.
  3. Optuna (rank 4), only if the meta-model needs tuning, with TrialSharpeRecorder so that every trial is counted in the DSR.
  4. Keep ml4t-diagnostic as a second opinion (RAS, White's Reality Check, trade diagnostics) in a separate Python 3.12+ env if wanted. Use the ML4T book chapters 7 and 16 as the reference.

Not recommended for this use case: Qlib/RD-Agent (wrong data shape; revisit only for automated feature search), NautilusTrader/Lean (engines, not ML), FinRL, freqtrade/Jesse, the LLM-agent frameworks, abandoned AFML ports (mlfinlab public, mlfinpy, timeseriescv), and AGPL or Commons-Clause code you might ever sell (pypbo, backtesting.py, vectorbt, PyBroker).

Caveats

← Back to ML catalog