Why accurate data doesn't guarantee better decisions

Give a leadership team real-time sales figures, live customer behaviour, and clean operational data, and the assumption almost writes itself: they'll make sharper calls than a team working without that visibility. That's the promise of data analytics, after all, to give firms the insight they need to decide well.

But the promise doesn't always hold. Access to accurate data doesn't guarantee better decisions. Two separate problems get in the way. The first is psychological, rooted in how human bias distorts the way people read and act on information, often without anyone realising it's happening. The second is structural, tied to what data fundamentally is: a record of what has already happened, in markets that rarely stay still long enough for the past to remain a reliable guide.

The psychological problem: how data quietly shifts accountability

Data can quietly reduce a person's sense of ownership over a decision, and make it easier to accept a conclusion that already matches what they expected to find.

Consider how blame works. When someone acts purely on intuition and the decision fails, there's no question about where the judgment came from. A decision backed by a model's output works differently. If it fails, the failure doesn't feel entirely personal, because the data suggested it would work, and part of the blame quietly shifts onto the data itself.

This mechanism has a name: automation bias. Deloitte UK describes it as what happens once people build enough trust in a system and stop scrutinising its output as critically as they otherwise would. Under pressure, this tendency gets stronger, not weaker. A leader who has come to trust a forecasting model is less likely to question its output the way they would question their own instinct.

The problem compounds when the data agrees with us. A 2025 study from researchers at Emory University and Georgia Tech found that confirmation bias becomes measurably stronger when a chart's summary confirms what someone already expected, and weaker when it contradicts that expectation. So a leader whose existing belief lines up with what the data shows is less likely to examine the decision with rigour. That can be the difference between catching a flawed assumption early and committing significant resources to a strategy that turns out to be wrong.

The structural problem: data only describes the past

The second problem has nothing to do with psychology. It arises simply from what data is: a record of what has already happened. Every analytics tool is limited to tracking metrics that have already occurred, things like past transaction volumes, past click rates, and past sales trends. By the time a pattern becomes visible in the numbers, it has already played out.

Markets don't move only on what gets recorded. They also move on things that are much harder to capture in real time, such as a shift in customer sentiment, a sudden economic shock, or a competitor's unexpected move. None of this shows up in historical data until well after it has already reshaped the market. A firm that leans entirely on this kind of tracking risks optimising itself for conditions that no longer exist by the time the optimisation is finished. The dashboard can be completely accurate and still be describing yesterday.

The counterargument: models still beat people over time

So far the argument has treated data models as something to be wary of. But there's a strong case on the other side: despite their flaws, models make better decisions than people do over time, and that alone might be reason enough to let them lead.

A model being wrong about the future sometimes isn't a valid reason to dismiss it. Forecasting always carries error, and no model escapes that completely. What matters is not whether a model is occasionally wrong, but whether it holds up better than the alternative across many decisions, not just one.

This creates a genuine difficulty for the view that human judgment should have the final say. It's hard to distinguish, in the moment, between genuine expertise, the kind of insight an experienced person brings that a model can't replicate, and something that only resembles insight but is actually fear, overconfidence, or fatigue. If that distinction can't be reliably made, a business might be better off deferring to the model, even with its occasional errors, than trusting judgment that varies unpredictably.

Where the counterargument breaks down

That case holds up best in a specific kind of environment: one with fixed rules and variables that don't change. In a closed system, letting an algorithm decide is mathematically superior. There's no emotional noise to correct for, and the model only has to handle conditions it has already seen.

Real markets aren't closed systems. They're open, and they get reshaped by forces a model has never been trained on: a new competitor, a regulatory change, a pandemic. When a shift like this happens, historical data stops being a reliable guide, because the future no longer resembles the past.

Zillow, one of the largest online real estate platforms in the United States, learned this directly. Its 'Zillow Offers' program used an algorithm to predict home prices accurately enough to buy houses directly and resell them at a profit. The algorithm failed when pandemic-era price swings moved faster than it could account for. Zillow shut the program down in November 2021 and admitted in its own SEC filing that price unpredictability had exceeded what it had planned for.

This kind of failure is dangerous for the reason established earlier. Trust in a model tends to strengthen, not weaken, under pressure, and an unpredictable market is exactly that kind of pressure. The moment conditions are least suited to historical data is also the moment people are least likely to question it.

This is where human judgment still matters, and it has nothing to do with overriding a model on a gut feeling.It's the ability to notice when the environment has shifted enough that a model's historical assumptions no longer apply, and to act on that recognition before the data reflects the change.

The real advantage lies in interpretation, not information

Automation bias and Zillow's collapse are bound by the same irony: confidence in a model tends to be strongest exactly when conditions make it least trustworthy. A leader stops questioning a forecast that has been right before. A company keeps faith in a pricing model through a market it was never built for. In both cases, the moment that called for more scrutiny was the moment scrutiny disappeared.

A model's consistency only goes unchallenged in a closed system, and most real business decisions aren't made in one. So the real advantage lies less in how much data a firm holds or how advanced its model is, and more in how honestly that information gets interpreted, and in the willingness to question a conclusion even when the data appears to confirm it.

Analytics can sharpen a decision. It can't replace the judgment needed to know when the data itself has stopped being a reliable guide. As you build out your own analytics capability, the question worth asking isn't only 'what is the data telling us?' It's 'would we still challenge this if it told us something we didn't want to hear?' Make room for that question in the room where decisions get made, and you'll get far more from your data than accuracy alone can offer.