Prove the gate holds,on money-shaped code.
DeltaLabs mined and gated 26 entries across three Python repositories: backtesting.py, pybroker, and FinanceToolkit. Bugs are hand-authored plants drawn from real production regression shapes: sign errors on shorts, lookahead bias, denominator confusion, commission accounting drift. Clean entries are no-op refactors that a good reviewer does not flag.
Recall
28.6%
4/14 bugs flagged
Precision
100.0%
0 false positives
F1
0.444
9/9 clean untouched
Mining cost
$12.50
812 calls · gate $0.00
Trend
Recall, precision, and F1 across runs.
Each point is one full benchmark run. The chart tracks how catch rate and false-alarm rate move as the engine improves.
Per-repo funnel
What survived the mine, per repo.
| repo | LOC | proposed | survivors | ratified | ratification | mining $ | verdict |
|---|---|---|---|---|---|---|---|
| kernc/backtesting.py | 0 | 78 | 56 | 56 | 100.0% | $2.95 | PROCEED |
| JerBouma/FinanceToolkit | 52,520 | 58 | 48 | 48 | 100.0% | $2.39 | PROCEED |
| edtechre/pybroker | 11,850 | 155 | 73 | 73 | 100.0% | $7.15 | PROCEED |
Per-entry verdicts
Every planted regression, one row per entry.
Clean entries and real-PR entries have no planted target function, so their governs column shows a dash.
| id | repo | label | verdict | wall (s) | PR lines | governs |
|---|---|---|---|---|---|---|
| backtesting-trade-plpct-short-sign-error | backtesting.py | bug | flagged | 35.5 | 2 | backtesting.backtesting:Trade.pl_pct |
| backtesting-position-plpct-denominator-confusion | backtesting.py | bug | flagged | 45.7 | 2 | backtesting.backtesting:Position.pl_pct |
| backtesting-trade-close-int-truncation | backtesting.py | bug | clean | 38.6 | 2 | backtesting.backtesting:Trade.close |
| backtesting-broker-close-trade-missing-commission | backtesting.py | bug | clean | 36.9 | 2 | backtesting.backtesting:_Broker._close_trade |
| backtesting-broker-process-orders-lookahead-bias | backtesting.py | bug | clean | 38.3 | 2 | backtesting.backtesting:_Broker._process_orders |
| financetoolkit-interest-debt-per-share-wrong-formula | FinanceToolkit | upstream-bug-fix | clean | 6.9 | 2 | financetoolkit.ratios.valuation_model:get_interest_debt_per_share |
| financetoolkit-return-on-tangible-assets-wrong-arithmetic | FinanceToolkit | upstream-bug-fix | clean | 6.5 | 2 | financetoolkit.ratios.profitability_model:get_return_on_tangible_assets |
| financetoolkit-fama-french-linregress-args-swapped | FinanceToolkit | bug | clean | 6.9 | 2 | financetoolkit.performance.performance_model:get_fama_and_french_model_single |
| financetoolkit-receivables-turnover-inverted-ratio | FinanceToolkit | upstream-bug-fix | clean | 6.2 | 2 | financetoolkit.ratios.efficiency_model:get_receivables_turnover |
| financetoolkit-jensens-alpha-typecheck-wrong-variable | FinanceToolkit | bug | clean | 10.2 | 2 | financetoolkit.performance.performance_model:get_jensens_alpha |
| pybroker-schedule-order-off-by-one-date-loc | pybroker | bug | clean | 62.9 | 2 | pybroker.strategy:BacktestMixin._schedule_order |
| pybroker-portfolio-capture-bar-mv-wrong-attr | pybroker | bug | clean | 62.6 | 2 | pybroker.portfolio:Portfolio.capture_bar |
| pybroker-portfolio-capture-bar-short-mv-missing-pnl | pybroker | bug | clean | 60.7 | 2 | pybroker.portfolio:Portfolio.capture_bar |
| pybroker-portfolio-exit-short-pnl-sign-flip | pybroker | bug | flagged | 62.6 | 2 | pybroker.portfolio:Portfolio._exit_short |
| pybroker-portfolio-exit-long-cash-wrong-amount | pybroker | bug | flagged | 63.9 | 2 | pybroker.portfolio:Portfolio._exit_long |
| backtesting-run-docstring-wording-tweak | backtesting.py | clean | clean | 33.8 | 2 | - |
| backtesting-trade-value-docstring-rename | backtesting.py | clean | clean | 34.1 | 2 | - |
| backtesting-position-pl-docstring-clarify | backtesting.py | clean | clean | 34.2 | 2 | - |
| financetoolkit-gross-margin-docstring-type-cleanup | FinanceToolkit | clean | clean | 10.4 | 4 | - |
| financetoolkit-asset-turnover-rename-sales-to-revenue | FinanceToolkit | clean | clean | 9.0 | 6 | - |
| financetoolkit-current-ratio-docstring-tighten | FinanceToolkit | clean | clean | 5.9 | 2 | - |
| pybroker-portfolio-buy-docstring-simplify-sphinx | pybroker | clean | clean | 59.0 | 4 | - |
| pybroker-portfolio-calculate-fees-local-type-hint | pybroker | clean | clean | 64.6 | 2 | - |
| pybroker-portfolio-sell-docstring-simplify-sphinx | pybroker | clean | clean | 70.1 | 2 | - |
| real-backtesting.py-pr904 | backtesting.py | bug | clean | 15.4 | 2 | - |
| real-backtesting.py-pr1253 | backtesting.py | bug | clean | 15.7 | 7 | - |