Are We Becoming the Cows? Or Are We Asking the Wrong Question?
Notes by SDG ·
Yuval Noah Harari warns that AI could create a world humans no longer understand. But how much of today’s world do we actually understand?
Imagine you’re running a business.
Your bank suddenly reduces your $500,000 credit line. You call your relationship manager to find out why.
The answer? An AI risk model made the decision.
The banker can show you a risk score, perhaps even an explanation generated by the system. But when you ask what specifically changed, or what you need to do to reverse the decision, nobody has a particularly good answer.
Now imagine thousands of banks, investment firms, and insurers using AI systems that make decisions and react to one another.
Each might be doing exactly what it was designed to do. But the overall outcome could be something nobody anticipated.
This scenario brings me to something very interesting thatYuval Noah Harari said recently
“In 10 years, we could be the cows.”
The Cow Analogy
In a September 2026 interview on The Rest Is Politics, Harari offered a striking analogy.
A cow doesn’t understand stock markets, corporate debt, bonds, or currency fluctuations. Yet those abstract mechanisms determine its entire existence. A holding company owning a farm could go bankrupt because of financial trades executed thousands of miles away, dictating what happens to the animals on that farm.
The cow understands its immediate surroundings, completely unaware of the invisible financial machinery shaping its fate.
Harari worries that AI could build financial instruments and operational systems so mathematically complex that humans end up in the exact same position, living, investing, borrowing, and planning within systems we no longer comprehend.
I don’t know about his ten-year timeline. But something about the fundamental premise bothers me.
We Have Always Outsourced Complexity
Do we really understand the systems we depend on today?
Consider software development.
Decades ago, programmers worked directly with machine code and assembly language. Today, most developers writing Python, Java, or C# don’t spend their time worrying about the specific CPU instructions their compiler generates.
There are specialists who understand those lower layers, but most developers don’t need to. We don’t view that as a failure of technology. It’s progress. We built layers of abstraction so people could focus on solving higher-level problems.
Or look at tax infrastructure.
How many business owners genuinely understand every rule surrounding input tax credits in India’s GST system, or every transaction code and validation step in Brazil’s Nota Fiscal framework?
Millions of transactions rely on these frameworks every day, but business owners rely on accountants, software, and tax specialists to navigate them.
The same applies to daily life. I use GPS without calculating satellite relativity math, board flights without understanding avionics software, and send international wire transfers without knowing every intermediary settlement protocol.
We have been outsourcing complexity for centuries. Hiding complexity under intuitive abstractions is the very definition of technological progress.
So why should AI be any different?
The Missing Distinction: Comprehension vs. Agency
Harari’s cow analogy assumes that if we don't understand a system, we're helpless against it. But I think that misses an important distinction
The fundamental difference between a human and a cow isn’t just that the cow doesn’t understand the financial system. It’s that a cow lacks legal agency and recourse.
A cow cannot appeal a decision, retain counsel, audit a balance sheet, or challenge the rules of the farm.
Not understanding how something works is abstraction. Not being able to investigate, challenge, or correct a specific decision is a total loss of agency.
I don’t understand every machine instruction my compiler outputs, but I can test my software, debug errors, examine runtime logs, and modify the source code.
If my GST calculation is flagged as incorrect, there are documented rules, audit procedures, professionals, and tax courts to resolve the dispute. Those mechanisms can be painfully slow, but the system remains fundamentally bound to human influence and legal appeal.
The real danger begins when a system becomes entirely beyond human recourse, not because it is too complex to grasp, but because no one has the authority or ability to override it.
Two Very Different Risks
When evaluating AI in critical infrastructure, we need to separate two distinct challenges.
1. Macro Risk: Systemic Interdependence
Algorithmic trading and automated risk models aren’t new, but AI accelerates their speed, autonomy, and cross-system interaction.
Consider an investment firm using AI to manage portfolios, an insurer using it to price risk, and a bank using it for credit allocation.
In a sudden economic shift, these independent models, trained on similar datasets, might simultaneously attempt to de-risk.
One system’s sensible risk-reduction decision triggers automated sell-offs in another, potentially cascading into a systemic liquidity crisis before human operators even receive an alert.
This is a macro complexity issue.
2. Micro Risk: The Erosion of Individual Recourse
The more immediate, everyday threat happens at the individual level: losing the ability to question decisions that impact livelihoods.
If a bank introduces an AI credit model that cuts loan losses by 20% while speeding up approvals, the business case is irresistible.
Lower defaults, higher margins, faster throughput.
Why slow it down?
This is how control is gradually surrendered, not through a hostile AI takeover, but through standard economic incentives. One rational business decision at a time.
When systems achieve high operational efficiency, “human-in-the-loop” governance can degrade into automation bias.
When a human manager is assigned to oversee 1,000 automated recommendations a day from a model with a 99% accuracy rate, oversight becomes a rubber stamp.
The human is no longer exercising judgment. They are acting as a legal liability shield.
If you cannot realistically investigate or override a recommendation, “human oversight” is an illusion.
The Regulatory Realignment
Regulators are recognizing that the problem isn’t the math. It’s accountability.
In November 2024, the Financial Stability Board (FSB) highlighted that widespread AI adoption could amplify systemic vulnerabilities, such as correlated market behavior and third-party dependency.
More recently, in October 2026, the FSB released 12 sound practices for responsible AI adoption in financial institutions, emphasizing explainability, governance, clear lines of accountability, and active third-party oversight.
In the US, consumer protection rules, such as CFPB adverse action requirements, already mandate that lenders state specific reasons when denying credit. Using a black-box machine learning model does not exempt an institution from that legal obligation.
This sets a critical principle:
We don’t need every user to understand every matrix multiplication, but whenever a system affects someone’s financial standing or livelihood, there must be a mechanism to audit the decision, correct errors, and assign human responsibility.
Are We Becoming the Cows?
Harari’s warning is valuable, but it points to the wrong root cause.
We don’t need every software engineer to understand assembly code, every business owner to be a tax attorney, or every airline passenger to be an aeronautical engineer.
Abstraction is a necessity.
AI has immense potential to simplify complex processes, catch systemic fraud earlier, democratize financial advice, and eliminate human bias. That is a future worth building.
Simplifying a process for the user, however, is fundamentally different from making the underlying system impossible to challenge.
What matters isn’t whether every human can comprehend every calculation an AI makes. What matters is whether we preserve the structural ability to investigate what happened, assign legal responsibility, and intervene when the system fails.
If an AI system consistently makes better lending decisions than humans, but cannot explain why it cut your company’s credit line, should we allow it to make that decision?
The danger isn’t that machines will build a world too complex for us to comprehend. It’s that we will build a world where “the model said so” becomes a final, unchallengeable answer.
Sources & Further Reading
- Yuval Noah Harari on The Rest Is Politics, September 17, 2026.
- Financial Stability Board: The Financial Stability Implications of Artificial Intelligence, November 2024.
- Financial Stability Board: Sound Practices for the Responsible Adoption of AI in Financial Institutions, October 2026.
- Consumer Financial Protection Bureau: Credit Decisions and Complex Algorithms.
- India Goods and Services Tax Portal.
- Brazil’s Official Nota Fiscal Eletrônica Portal.