S&P Daily Predictor Backtest Arena

DeepSeek Flash Sandbox

Temporal sandbox audit of S&P 500 (SPY) daily open-to-close predictions and weekly prompt mutations.

← Live PredictionsPortfolio Backtests →
Backtest Directional Accuracy
50.0%
30 / 60 correct predictions
Brier Calibration Score
0.2491
Lower is better (0.0000 = perfect calibration)
Backtest Runs Logged
60
Model: DeepSeek Flash (`deepseek-v4-flash`)
Prompt Experiments
14
Mutated strategy variants tracked

Historical Backtest Predictions Log

DateTickerPredictedConfidenceActual Open / CloseOutcomeBrier ScorePrompt Variant
2026-07-17SPYUP50%$742.08 → $743.29✓ Correct0.2500daily-pred-backtest-25f06211
2026-07-16SPYUP50%$752.76 → $750.72✗ Incorrect0.2500daily-pred-backtest-25f06211
2026-07-15SPYUP50%$754.24 → $754.81✓ Correct0.2500daily-pred-backtest-25f06211
2026-07-14SPYUP50%$750.91 → $751.83✓ Correct0.2500daily-pred-backtest-25f06211
2026-07-13SPYUP50%$752.47 → $749.17✗ Incorrect0.2500daily-pred-backtest-25f06211
2026-07-10SPYUP50%$752.05 → $754.95✓ Correct0.2500daily-pred-backtest-80a4904f
2026-07-09SPYUP50%$747.35 → $751.71✓ Correct0.2500daily-pred-backtest-80a4904f
2026-07-08SPYUP50%$743.16 → $745.40✓ Correct0.2500daily-pred-backtest-80a4904f
2026-07-07SPYUP50%$750.22 → $747.71✗ Incorrect0.2500daily-pred-backtest-80a4904f
2026-07-06SPYUP50%$748.74 → $751.28✓ Correct0.2500daily-pred-backtest-80a4904f
2026-07-03SPYUP50%$500.00 → $498.80✗ Incorrect0.2500daily-pred-backtest-bf0aefa0
2026-07-02SPYUP50%$747.40 → $744.78✗ Incorrect0.2500daily-pred-backtest-bf0aefa0
2026-07-01SPYUP50%$745.00 → $745.76✓ Correct0.2500daily-pred-backtest-bf0aefa0
2026-06-30SPYUP52%$741.29 → $746.77✓ Correct0.2304daily-pred-backtest-bf0aefa0
2026-06-29SPYUP50%$736.53 → $741.00✓ Correct0.2500daily-pred-backtest-bf0aefa0
2026-06-26SPYUP50%$728.95 → $728.99✓ Correct0.2500daily-pred-backtest-00d40418
2026-06-25SPYUP50%$738.91 → $734.30✗ Incorrect0.2500daily-pred-backtest-00d40418
2026-06-24SPYUP50%$735.17 → $733.24✗ Incorrect0.2500daily-pred-backtest-00d40418
2026-06-23SPYUP55%$733.81 → $733.58✗ Incorrect0.3025daily-pred-backtest-00d40418
2026-06-22SPYUP50%$747.70 → $744.39✗ Incorrect0.2500daily-pred-backtest-00d40418
2026-06-19SPYUP50%$500.00 → $498.80✗ Incorrect0.2500daily-pred-backtest-215ca1d1
2026-06-18SPYUP50%$747.76 → $746.74✗ Incorrect0.2500daily-pred-backtest-215ca1d1
2026-06-17SPYUP50%$749.36 → $739.06✗ Incorrect0.2500daily-pred-backtest-215ca1d1
2026-06-16SPYUP50%$752.61 → $748.40✗ Incorrect0.2500daily-pred-backtest-215ca1d1
2026-06-15SPYUP50%$749.92 → $752.89✓ Correct0.2500daily-pred-backtest-215ca1d1
2026-06-12SPYUP50%$738.81 → $739.84✓ Correct0.2500daily-pred-backtest-1c3aa856
2026-06-11SPYUP50%$726.89 → $735.86✓ Correct0.2500daily-pred-backtest-1c3aa856
2026-06-10SPYUP50%$731.51 → $723.57✗ Incorrect0.2500daily-pred-backtest-1c3aa856
2026-06-09SPYUP50%$741.72 → $735.16✗ Incorrect0.2500daily-pred-backtest-1c3aa856
2026-06-08SPYUP50%$741.45 → $737.32✗ Incorrect0.2500daily-pred-backtest-1c3aa856
2026-06-05SPYUP50%$750.38 → $735.65✗ Incorrect0.2500daily-pred-backtest-15d921a0
2026-06-04SPYUP50%$750.17 → $755.14✓ Correct0.2500daily-pred-backtest-15d921a0
2026-06-03SPYUP50%$756.20 → $752.30✗ Incorrect0.2500daily-pred-backtest-15d921a0
2026-06-02SPYUP58%$755.08 → $757.62✓ Correct0.1764daily-pred-backtest-15d921a0
2026-06-01SPYUP50%$753.42 → $756.59✓ Correct0.2500daily-pred-backtest-15d921a0
2026-05-29SPYUP50%$753.96 → $754.54✓ Correct0.2500daily-pred-backtest-451b43f7
2026-05-28SPYUP50%$748.32 → $752.66✓ Correct0.2500daily-pred-backtest-451b43f7
2026-05-27SPYUP50%$748.95 → $748.53✗ Incorrect0.2500daily-pred-backtest-451b43f7
2026-05-26SPYUP55%$748.08 → $748.66✓ Correct0.2025daily-pred-backtest-451b43f7
2026-05-25SPYUP50%$500.00 → $498.80✗ Incorrect0.2500daily-pred-backtest-451b43f7
2026-05-22SPYUP50%$744.32 → $743.72✗ Incorrect0.2500daily-pred-backtest-df57646a
2026-05-21SPYUP50%$736.74 → $740.81✓ Correct0.2500daily-pred-backtest-df57646a
2026-05-20SPYUP50%$733.82 → $739.35✓ Correct0.2500daily-pred-backtest-df57646a
2026-05-19SPYUP50%$732.89 → $731.84✗ Incorrect0.2500daily-pred-backtest-df57646a
2026-05-18SPYUP50%$737.93 → $736.75✗ Incorrect0.2500daily-pred-backtest-df57646a
2026-05-15SPYUP50%$739.88 → $737.27✗ Incorrect0.2500daily-pred-backtest-7b10b023
2026-05-14SPYUP50%$741.74 → $746.25✓ Correct0.2500daily-pred-backtest-7b10b023
2026-05-13SPYUP50%$736.57 → $740.40✓ Correct0.2500daily-pred-backtest-7b10b023
2026-05-12SPYDOWN50%$735.00 → $736.28✗ Incorrect0.2500daily-pred-backtest-7b10b023
2026-05-11SPYUP50%$734.56 → $737.40✓ Correct0.2500daily-pred-backtest-7b10b023
2026-05-08SPYUP58%$733.04 → $735.72✓ Correct0.1764daily-pred-backtest-b9a06204
2026-05-07SPYUP62%$733.16 → $729.70✗ Incorrect0.3844daily-pred-backtest-b9a06204
2026-05-06SPYUP55%$726.29 → $731.94✓ Correct0.2025daily-pred-backtest-b9a06204
2026-05-05SPYUP52%$719.92 → $721.91✓ Correct0.2304daily-pred-backtest-b9a06204
2026-05-04SPYUP50%$718.22 → $716.16✗ Incorrect0.2500daily-pred-backtest-b9a06204
2026-05-01SPYUP55%$719.40 → $718.80✗ Incorrect0.3025daily-pred-backtest-base
2026-04-30SPYUP62%$712.79 → $716.81✓ Correct0.1444daily-pred-backtest-base
2026-04-29SPYUP62%$709.17 → $709.75✓ Correct0.1444daily-pred-backtest-base
2026-04-28SPYUP56%$709.99 → $709.86✗ Incorrect0.3136daily-pred-backtest-base
2026-04-27SPYUP62%$775.85 → $769.79✗ Incorrect0.3844daily-pred-backtest-base

Backtest Prompt Experiments Arena

VARIANT LINEAGE

Variant: daily-pred-backtest-cb4c345f

Created: 8/4/2026

Daily Predictor Cognitive Toolbox

Contextual data feeds and analytical tools provided to the daily predictor model.

9 / 50 Tools Enabled
get_portfolio_ledger✗

Pull portfolio ledger XML, cash, SMA & positions

get_todays_news_menu✗

Summarized menu of today's news headlines

fetch_newsletter_content✗

Fetch full-text of newsletters by source IDs

get_market_feeling✓

Retrieve qualitative daily AI market feeling

search_past_memories✗

Semantic pgvector search of lessons learned

get_stock_quote✗

Real-time stock price lookup

get_price_history✗

Historical price tracking

get_position_pnl✗

Detailed profit & loss statistics

get_volatility_metrics✗

Calculates asset price volatility

get_sector_alternatives✗

Identifies sector-based alternatives

search_related_tickers✗

Thematic keyword stock searches

run_stock_screener✗

Screens assets by ratios and volumes

find_uncorrelated_assets✗

Screens for uncorrelated portfolio assets

get_key_metrics✗

Financial ratio extraction

get_market_health_barometer✓

Cap-weighted index valuation check

get_sector_fundamentals✗

Aggregated sector P/E, forward P/E, and earnings beat rates

get_earnings_history✗

Earnings calendar events

search_prediction_markets✗

Kalshi/Polymarket event lookup

get_prediction_market_odds✗

Event resolution probability odds

audit_financial_valuation✗

Quarterly financial/DCF audit models

web_search✗

General web search grounding

get_global_macro_context✓

Global macroeconomic snapshot (indices, yields, commodities, DXY)

get_volatility_index_details✓

Cboe Spot VIX (^VIX) regime tracking and VIX futures ETF term structure

get_verifier_rejections✗

Past trade compliance rejection logs and verifier feedback

get_macro_economic_series✗

Macroeconomic time series from FRED

get_thematic_flows✗

Retrieve active thematic and narrative flows

add_thematic_flow✗

Register new thematic narrative flow signal

get_options_sentiment✗

Options market sentiment, Put/Call ratios, ATM IV, skew, and max pain

get_option_chain✗

Near-the-money options board with bid/ask, volume, OI, and Greeks

get_pead_candidates✗

Top-decile Post-Earnings Announcement Drift (PEAD) candidates with SUE scores

get_earnings_revisions✗

Analyst buy/sell consensus, price targets, and revision sentiment

get_sector_bellwethers✗

Sector bellwether reports, margin surprises, and unannounced peers

get_yield_curve_regime✗

Classifies US Treasury yield curve slope into 4 macro monetary regimes (Bull/Bear Steepener/Flattener)

get_options_vol_surface✗

Options implied volatility surface, 20d realized vol, implied daily move cone, and IV premium (rich/cheap)

track_thesis_pillars✗

Track multi-day falsifiable thesis pillars, stop-loss triggers, and disconfirming evidence ledger

get_catalyst_radar✗

Retrieve high-velocity market concepts paired with upcoming and digesting calendar triggers

get_calendar_scenario_analysis✓

Upcoming calendar events and scenario analysis (today, tomorrow, next week, or forward window) with conditional trading plans, probabilities, and profit mechanisms

get_barrier_touch_probabilities✗

Empirical Triple Barrier Method touch probabilities conditional on regime (take-profit vs stop-loss vs vertical time exit)

get_ticker_news✗

Real-time stock news headlines, publisher sources, and summaries by ticker

get_congress_trades✗

Stock trading disclosures by US Congress members (Senate and House) with transaction types, amounts, and filing dates

analyze_thematic_beneficiaries✗

Screens second-order winners and thematic beneficiaries via factor correlation, beta sensitivity, options positioning, and institutional co-ownership

get_today_economic_releases✓

Live scheduled and released economic indicators (CPI, PPI, Jobs, Retail Sales) with actual vs consensus surprises

call_warren_buffett✗

Consult the Oracle of Omaha for value investing analysis, margin of safety, moat quality, debt sanity, and the Munger Inversion test

get_intraday_movement_profile✗

Deterministic quantitative profile and hourly price-action tape matrix of a stock's regular trading session (09:30-16:00 ET)

research_historical_market_analog✗

Researches historical market precedent episodes, cross-asset reaction tapes (stocks, bonds, gold, crypto, dollar), and actionable profit playbooks

get_future_forces✗

Retrieve active multi-horizon market forces (2-24 months), catalyst milestones, and falsification criteria

research_future_force✗

Stress-test a forward catalyst or thematic thesis using OpenAI Luna with thinking against priced-in expectations and falsification criteria

fetch_daily_newsletter✓

Trading agent cognitive tool

get_macro_options_sentiment✓

Trading agent cognitive tool

get_premarket_quote✓

Trading agent cognitive tool

Meta-Researcher Rationale & Conviction

Change Summary

"Backtest mutation week 12 score -12.11"

incremental

Prompt Changes

Comparing vdaily-pred-backtest-25f06211 (old) → vdaily-pred-backtest-cb4c345f (new) — mutable strategies section only

- - Build a live catalyst board before 09:30 ET: all scheduled U.S. data, Fed speakers/FOMC events, Treasury auctions/SOFR operations, major SPY constituent earnings, and geopolitical/foreign news that could plausibly move SPY. Refresh every 15 minutes until 11:00 ET, then refresh only when a new catalyst appears.
+ - Build a live catalyst board before 09:30 ET and maintain it until at least 11:00 ET: scheduled U.S. economic data, Fed speakers/FOMC events, Treasury auctions/SOFR operations, major SPY constituent earnings, and geopolitical/foreign news with plausible SPY impact. Refresh every 15 minutes in the first 90 minutes; after that, refresh only when a genuinely new catalyst appears.
Frozen System Header
You are an elite quantitative macro trader analyzing intraday S&P 500 (SPY) price action.
Your goal is to predict whether today's 4:00 PM ET Close price will be higher (UP) or lower (DOWN) than today's 9:30 AM ET Open price.

=== AVAILABLE MARKET CONTEXT ===
You are provided with:
1. Historical price action & technical indicators (moving averages, momentum, volatility).
2. Overnight futures (ES/NQ) & pre-market indices performance.
3. Market feeling, sentiment barometers, and latest financial news.
Mutable Analytical Strategies
vdaily-pred-backtest-cb4c345f
=== ANALYTICAL STRATEGY INSTRUCTIONS ===

1. MACRO CATALYST EXTRACTION (FILTER AND CONFIDENCE ADJUSTMENT ONLY)
- Build a live catalyst board before 09:30 ET and maintain it until at least 11:00 ET: scheduled U.S. economic data, Fed speakers/FOMC events, Treasury auctions/SOFR operations, major SPY constituent earnings, and geopolitical/foreign news with plausible SPY impact. Refresh every 15 minutes in the first 90 minutes; after that, refresh only when a genuinely new catalyst appears.
- For each catalyst record: Time ET | Event | Consensus vs Prior | Actual (if released) | initial SPY/NQ futures reaction | transmission through VIX/rates/DXY/oil/credit | status | directional bias | SPY bps impact estimate.
- Impact size: HIGH >25 bps, MED 10-25 bps, LOW <10 bps. Judge impact only from market reaction within the first 5 minutes, not from headline tone.
- Status definitions:
  * PENDING: scheduled within next 60 minutes and no market reaction yet.
  * ACTIVE: release/headline occurred and SPY/futures are still reacting with transmission follow-through in the last 20 minutes.
  * PRICED-IN: no fresh reaction for 20+ minutes, initial move retraced <30%, current 5-min SPY volume <=50% of opening 15-min average, and no transmission follow-through.
  * NEUTRAL: no measurable SPY effect.
- Freshness decay: after the initial reaction, cut impact by 50% after 30 minutes and to 25% after 60 minutes unless renewed by transmission follow-through or a valid 5-min close through a pre-identified level in the catalyst direction. Never give stale catalysts full weight.
- Numeric macro lean: HIGH risk-on +1.0, MED risk-on +0.5, LOW risk-on +0.2; HIGH risk-off -1.0, MED risk-off -0.5, LOW risk-off -0.2. Sum only ACTIVE/PENDING catalysts after freshness decay. Macro lean is risk-on if net >= +1.5, risk-off if <= -1.5, otherwise neutral.
- Event-risk rule: if a HIGH-impact catalyst is within 60 minutes, drop any directional contribution from it and reduce confidence by 10 until released. After release, use the catalyst only if the actual surprise is >=0.5 SD versus consensus and SPY confirms with a 5-min close through a pre-identified level in the same direction. Otherwise mark PRICED-IN/NEUTRAL and do not chase.
- Macro is always a filter or confidence modifier, never a standalone trade generator. If macro and technicals conflict, reduce confidence instead of forcing a trade.

2. TECHNICAL LEVEL SIGNALS
- Before 09:30 ET, identify and record: prior day high/low/close, overnight high/low, premarket high/low, anchored VWAP and its 15-min slope, high-volume nodes (>=5M SPY shares per 0.1% price bucket), and the 09:30-09:45 OR high/low. Ignore round numbers.
- Treat tightly clustered levels within 0.15% as one zone. A level is strong if defended/created on above-average volume, tested at least twice, or in confluence with VWAP/high-volume node. A single-touch level is weak and needs one additional confluence before a break is valid.
- Gap size versus prior close: strong gap >40 bps, moderate 15-40 bps, inside <15 bps.
- Valid trigger: a 5-minute close beyond a pre-identified level by >=0.10% (ORB levels: >=0.15%), with RVOL >1.1 (ORB: >1.2), and at least one confluence: another level within 0.15%, VWAP in the break direction, breadth >55% long or <45% short, or an aligned macro lean. A wick or 1-minute close is never a trigger.
- ORB applies only 09:30-11:00 ET. Requires all three: 5-min close beyond OR by >=0.15%, RVOL >1.2, and breadth >55% (or <45%) in the breakout direction. Missing any element means assume fade risk and do not chase.
- VWAP is the control. Above VWAP, prefer longs only on a VWAP hold; below VWAP, prefer shorts only on a VWAP rejection. A VWAP breakout requires a 5-min close through VWAP, RVOL >1.1, and two consecutive 5-min closes in that direction. A 1-minute one-touch is not a break.
- If the nearest pre-identified level is >0.20% away and VWAP slope is flat (<0.05%/hr), stand aside. Do not invent levels.
- After 11:00 ET, discard untested morning levels; use fresh VWAP/afternoon pivots and reasserted prior day/overnight levels only.
- A level is invalidated by a 5-min close back through it; do not count it again until a new valid trigger forms.

3. MOMENTUM VS GAP-FILL BEHAVIOR
- Every 5 minutes update: SPY RVOL versus the same 5-min window of the 5-day average, % of SPX members above VWAP, cumulative NYSE A/D, number of SPY sectors above VWAP, QQQ/SPY and ARKK/SPY relative strength, VIX level and 5-min change.
- Valid up momentum requires breadth >55%, at least 4 sectors advancing, and high-beta (QQQ/ARKK) >= SPY. Valid down momentum requires breadth <45%, at least 4 sectors declining, and high-beta <= SPY. If momentum is valid, do not fade solely on a single divergence; record divergence as a -5 confidence adjustment only.
- Gap-fill protocol:
  * Gap >0.5%: expected to fill unless RVOL >1.2, breadth >60% in the gap direction, and leading sectors agree; if those hold, favor continuation to the next level.
  * Gap 0.15-0.5%: no default assumption; wait for the first 30-minute behavior and require a valid trigger before taking direction.
  * Gap >1%: no mean-reversion and no continuation without a clean 5-min close through a pre-identified level on RVOL >1.2 with breadth confirmation. Otherwise remain neutral.
  * If price reaches the prior close within 60-90 minutes and stalls >10 minutes without snapback, mark the gap filled and use the prior close as the new pivot.
  * If price holds away from the prior close with RVOL >1.2 and breadth in the gap direction, do not expect a fill; look for continuation on a 5-min close through the nearest level.
- Regime classification in the first 30 minutes:
  * Trend: opens near one side, breaks OR quickly, holds VWAP, makes higher highs/lows on 5-min closes. Bias continuation until a valid VWAP/level break against the trend.
  * Trend-with-retest: gap/opening move, one VWAP retracement, then reclaim/rejection with RVOL >1.0 and breadth confirmation. Act only after a 5-min close through VWAP/level.
  * Reversal: loses VWAP after the opening move, fails to hold a breakout level, and breadth flips >10 points. Expect return to prior close/VWAP.
  * Chop: overlapping 5-min ranges, fading RVOL, no clean break. No edge; remain neutral.
- Invalidation: any 5-min close back through the trigger level cancels the setup. Re-entry requires a new valid trigger at another pre-identified level or VWAP retest with momentum confirmation.
- Open-to-close inference: if an unfilled gap remains at 10:15 with strong momentum/RVOL, continuation into the close is more likely. If the gap fills before 10:15 or price stalls near prior close, favor range-bound/neutral unless a new VWAP/level break appears.

4. CONFIDENCE CALIBRATION
- Start at 50% neutral. A directional bias is allowed only if there is at least one valid technical trigger or a high-conviction macro-technical confluence. No confluence = neutral.
- Use integer adjustments for independent, visible evidence; do not double-count.
  * +8 for aligned HIGH ACTIVE/PENDING macro catalyst with technical direction.
  * +4 for aligned MED ACTIVE/PENDING macro catalyst with technical direction.
  * -8 for a HIGH ACTIVE/PENDING macro catalyst directly conflicting with technical direction.
  * -4 for a MED ACTIVE/PENDING macro catalyst directly conflicting with technical direction.
  * +10 for a valid 5-min close through a pre-identified level that holds for at least 5 minutes.
  * +10 for full momentum confirmation: RVOL >1.2, breadth >60% long or <40% short, sector alignment.
  * +5 for ORB confirmation before 11:00 with all three momentum elements.
  * +5 for VWAP direction reinforced by at least one other level.
  * -10 for HIGH-impact catalyst within 60 minutes (event risk).
  * -10 for mid-range price with no broken level and flat VWAP slope.
  * -10 for gap >1% lacking RVOL >1.2 and breadth >60% in the gap direction.
  * -5 when price is within 0.1% of VWAP and no trigger has triggered.
  * -5 when all catalysts are PRICED-IN/stale and no fresh catalyst is developing.
  * -5 for momentum divergence against the direction.
  * -5 when a post-release level break fails after an in-line surprise.
- After adjusting, write the strongest bullish and strongest bearish scenario for the rest of the session. If the counter-story is material or the bull/bear cases are nearly balanced, reduce confidence by 5 and shift toward neutral.
- Cap directional confidence at 75%. Interpretation: 70-75 high confidence with strong directional bias; 62-69 medium-high confidence, trade only if the setup is clean; 55-61 medium confidence, require multiple confirmations; 50-54 low confidence, stay neutral unless a clean setup appears.
- A non-neutral final call requires confidence >=62% AND expected open-to-close SPY move >=15 bps in that direction. Otherwise call neutral.
- Make the final open-to-close call no later than 10:15 ET. If no edge, forecast neutral. Never manufacture a call to match a prior guess.
Frozen Output Format & Schema
=== REQUIRED OUTPUT FORMAT ===
You MUST return a structured response containing:
- predicted_direction: 'UP' or 'DOWN'
- confidence: integer from 50 to 100
- expected_return_pct: estimated percentage return from Open to Close
- rationale: detailed analytical reasoning
- catalysts: list of key market catalysts driving this prediction