Can AI Predict Financial Markets and Future Prices? Testing Five Models Against Simple Baselines

What “predict the market” actually means

Three different claims often get compressed into one phrase:

Success on the first task does not establish either of the others. A 70% chance of a rise says nothing by itself about how large that rise might be. It can coexist with small frequent gains and occasional large losses.

There is credible evidence that language models can extract useful financial signals in particular settings. Lopez-Lira and Tang report that GPT-4 assessments of post-cutoff company headlines predict market reactions and subsequent drift. Their approximately 90% portfolio-day hit rate refers to an initial reaction they explicitly describe as non-tradable. That is not a universal stock-picking accuracy claim, and it does not establish an edge in hourly Bitcoin forecasts. [1].

Other work finds that language models can put excessive weight on recent stock returns and produce optimistic forecasts. These findings argue for measurement rather than confidence in persuasive explanations. [2].

One precise question for the Council

The Council has five model seats: Qwen, Llama, GPT-OSS, Nemotron and Mistral. These labels identify families, not reproducible experiments. Each forecast record needs the exact model identifier, provider and version available at the time, alongside the prompt and input-packet version. A seat that changes checkpoints begins a new versioned series.

The test concerns Bitcoin's next hourly candle, rather than all financial markets. For a target hour from 14:00 to 15:00 UTC, calls are sealed at 13:55 UTC and evaluated after 15:00. The outcome compares the 15:00 close with the 14:00 open. It does not compare the close with the price when the forecast was sealed.

The Council's outcome rules use a narrow neutral band:

Pushes remain in the ledger but are excluded from binary up/down scoring. This makes the probability interpretation important: for this test, model instructions must explicitly ask for the chance of up conditional on the hour resolving outside the push band. Existing forecasts cannot simply be assumed to use that definition. A study of unconditional up probability would need to account for the third outcome explicitly.

The evaluation must name its exact instrument and price source. “Bitcoin” alone is insufficient: a futures candle from one exchange should not silently be resolved using another venue's spot price. Missing or corrected market data also needs a rule fixed before results are examined.

The published protocol specification records these Council thresholds. The live-run prompt must explicitly define the conditional probability before collection begins; this article has not verified that any deployed model prompt already does so. [7]

• Up: close/open minus one is at least +0.01%, or one basis point.
• Down: the change is at most −0.01%.
• Push: the change falls between those thresholds.

The simple forecasts the models must beat

The Council's reference forecasts make the challenge concrete. They use only information available by the seal deadline.

The historical reference excludes pushes and must disclose its actual observation count. Momentum uses the completed five-minute bars available by the seal, not a candle that finishes afterward. Its 52/48 probabilities are fixed benchmark settings, not claims that those probabilities are empirically correct.

The Council average also needs a precise rule. The current interface averages the on-time, available seats; it displays 0.50 if none answer. That means a three-seat hour is a different ensemble from a five-seat hour. The ledger must preserve membership and response count, not merely the final average.

Why probability scores matter more than a win streak

Our primary probability metric is the binary Brier score:

mean((forecast_probability − outcome)²)

An up outcome is 1 and a down outcome is 0. Lower scores are better. The score evaluates the probability that was actually submitted rather than replacing every forecast above 50% with an identical “up” prediction. [3].

A 70% call scores 0.09 if the hour goes up and 0.49 if it goes down. A constant 50% call scores 0.25 either way. That 0.25 reference is an arithmetic identity, not a measured Bitcoin result.

An individual failed 70% forecast is not proof that the model is broken. A trustworthy 70% forecast should still fail sometimes. The useful question is whether its probabilities earn better average scores over the predeclared sample.

Worked example — synthetic, not Council results

Imagine ten hours containing six up outcomes and four down outcomes. Two fictional methods call “up” every hour. One assigns 60%; the other assigns 90%.

Both directional methods have the same headline accuracy. The overconfident method performs worse than the reference because its four wrong calls are expensive in probability-score terms. These invented numbers explain the calculation; they say nothing about any Council model's ability.

The chart that answers the 70% question

A calibration chart groups forecasts into probability bands. Each point places the band's average forecast on the horizontal axis and its observed up-frequency on the vertical axis. Reliable forecasts lie near the diagonal: forecasts averaging 70% should resolve up roughly 70% of the time. [4].

For the synthetic example, the two points are (60%, 60%) and (90%, 60%), each based on ten observations. The second point exposes overconfidence. Ten observations are far too few to establish stable calibration.

The interactive chart below is a scoring demonstration. It starts with a fictional 70% forecast and seven up outcomes out of ten, then lets you change both values. It is not a measured Council calibration curve.

The eventual empirical chart must show the count behind every point, uncertainty and the probability distribution. Otherwise, a sparse 90% bin can look as trustworthy as a well-populated 55% bin. Empty bins should stay empty.

Brier score alone does not establish calibration: it also reflects the ability to distinguish events. Equally, a model that says 50% forever can be calibrated in a balanced sample without providing useful separation. We need both the score and the chart. [4].

Try the probability scoring example

Change the forecast probability and the hypothetical outcomes. The scores are arithmetic demonstrations; the diagonal shows where a group’s average forecast equals its observed up-frequency. A point on that line with ten invented observations does not validate any model.

What makes this a genuine forward test

Historical market data can be useful for development. Asking a modern model about an old, recognizable event is a weaker test of future forecasting: its training or retrieved context may contain the answer. ForecastBench addresses this problem by soliciting probabilities for events unresolved at submission time. The same principle belongs in the Council test. [5].

A defensible run needs its dates, input cutoff, model versions, prompts, baselines, missing-call rules and evaluation code published before outcomes arrive. All five models should receive the same frozen information packet. If one gets live search and another gets only prices, the experiment compares different systems as well as different models.

Every scheduled seat must remain visible. The Council interface describes missing or late seats as scored at 0.50. An operational comparison should retain that policy, disclose each failure and score the agreed schedule. A separate on-time-only view can examine the model's submitted forecasts. A common-hours comparison helps ensure different models are not being judged on different market conditions.

The period must be fixed in advance. Stopping after an attractive winning streak or changing a prompt after inspecting losses turns the sample into development data. New rules need a new version and a fresh evaluation period.

Hourly observations may share market conditions, so apparent sample size can overstate independent evidence. Report uncertainty that respects time dependence and avoid declaring universal superiority from a tiny score difference. A public integrity hash is useful, but proof that a call existed before the outcome also needs a retained public or independently timestamped commitment.

Protocol version 1 specifies the scoring rules, but a prospective run has not been registered in this publication. The exact start and end, exchange instrument, model identifiers, prompts, input packet and data-outage policy must be filled in and committed before the first eligible hour. Those fields remain explicitly unset in the download; no study start or sample size is invented here.

What a reader should be able to download

A complete study release should include all scheduled forecast rows, exact model identifiers, seal and response times, raw and scored probabilities, missing/late flags, ensemble members, baseline inputs, market open/close values, up/down/push outcomes and scoring-code version.

Readers should be able to reproduce the Brier table and calibration points from that ledger. The published comparison must show sample counts and differences against the same-hour references, including unfavorable results. A CSV template or illustrative dataset is useful documentation; it cannot substitute for the real observations.

This edition provides three downloads: the published protocol specification, an empty forecast-ledger template and the complete ten-row synthetic scoring example used above. The illustrative CSV contains fictional methods, not Qwen, Llama, GPT-OSS, Nemotron or Mistral calls. No empirical Council ledger is released here. Future results require that complete record, including failures and pushes.

Future prices and trading profits need their own tests

To investigate exact future prices, we would add a separate experiment with a fixed horizon, submitted numerical forecasts and price-error metrics. A natural starting reference is the last observed price. This naïve forecast is a standard benchmark for financial time series; it should be evaluated rather than dismissed for its simplicity. [6].

A price interval would need another check: how often the realized price actually falls inside it, alongside interval width. A chart cone derived from a directional probability is not automatically a validated price interval.

Trading claims require still more evidence: an entry and exit rule, executable prices, fees, spread, slippage, position sizing and drawdowns. The Council's paper display uses (p − 0.5) × 2 as its hourly position and omits fees. That display is not evidence of net returns a trader could achieve.

The honest answer to “Can AI predict financial markets?” is therefore conditional. Some studies find useful signals in specific settings. For these five Council seats, useful hourly Bitcoin forecasting remains a claim to demonstrate with forecasts made before outcomes, transparent scores and a reproducible ledger. The simplest baseline winning would be a valuable result too.