Your blog post

The Greatest Threat in AI Isn’t Malice—It’s Cognitive Surrender

Why Dietrich Bonhoeffer’s “Theory of Stupidity,” my own experience with human nature, and the rise of algorithmic convenience point to the same uncomfortable conclusion.

There is a particular kind of human behaviour I have encountered across virtually every plane of my life.

I have seen it in business.

I have seen it in institutions.

I have seen it dealing with regulators.

I have seen it in subordinates and superiors.

I have seen it in people whose behaviour was profoundly narcissistic and self-absorbed.

I have seen it in otherwise intelligent, educated and accomplished people.

I have been given orders that were plainly unlawful.

I refused to follow them and, where necessary, reported them further up the chain.

In two cases, doing what I believed was unquestionably right was incredibly destructive to my career and to aspirations I had worked years to achieve.

I have also taken the bullet for other people's foolish mistakes because, at the time, I thought loyalty, mercy and protecting another human being were the right things to do.

More than once I discovered later that the person I had protected was quite prepared to protect themselves at my expense.

Those experiences teach you something.

Sometimes the problem is not that people do not know the truth.

Sometimes they know enough of the truth to understand exactly why they do not want to look at it.

That distinction matters enormously in the age of artificial intelligence.

Because I increasingly believe that the greatest danger presented by AI is not some science-fiction machine becoming malicious.

It is something much older.

It is us.

More specifically, it is the human capacity to surrender independent judgment while finding increasingly sophisticated excuses for having done so.

AI simply gives that behaviour a new theatre in which to perform.

Bonhoeffer Saw the Human Problem Before AI Existed

In late 1942, while Germany was living through the moral catastrophe of National Socialism, theologian Dietrich Bonhoeffer wrote an essay called After Ten Years.

One of its sections was titled “On Stupidity.”

Bonhoeffer's observation was uncomfortable:

Stupidity is a more dangerous enemy of the good than malice.

He was not talking about IQ.

He was not saying that some people are naturally unintelligent and therefore dangerous.

His point was far more sophisticated.

In Bonhoeffer's framework, stupidity was essentially a human and sociological condition in which otherwise capable people surrendered their independent judgment to an external power.

People could remain educated.

Articulate.

Professionally accomplished.

Technically competent.

They could still quote facts, policies, authorities and procedures.

Yet something important had disappeared.

Their capacity—or willingness—to stand outside the prevailing narrative and ask:

Is this actually true?

Is this right?

Does the evidence support what I am being told?

What happens to another human being if I am wrong?

That last question matters to me.

Because truth and mercy are related.

Truth without mercy can become cruelty.

Mercy without truth can become cowardice.

And self-interest can destroy both.

Bonhoeffer observed that once people surrendered themselves to a sufficiently powerful external narrative, conventional reasoning often stopped working.

Contradictory facts could simply be rejected.

Irrefutable facts could be dismissed as irrelevant.

Slogans replaced independent thought.

The person could become strangely self-satisfied in the certainty that they were right precisely when the evidence suggested they should be asking harder questions.

I have seen versions of that phenomenon throughout my life.

And now we are giving it a machine.

I Have Learned That Intelligence Is No Defence Against Stupidity

One of the harder lessons of getting older is discovering that intelligence and wisdom are not remotely the same thing.

I have dealt with extremely intelligent people who could rationalize almost anything once their ego, authority, reputation, career or financial interest depended on maintaining a particular version of events.

I have watched people become more inventive in defending an obviously failing position than they had ever been in establishing whether that position was true in the first place.

I have dealt with regulators where the process appeared to become more important than the purpose the process was supposedly created to serve.

I have dealt with superiors who issued directions that I knew I could not lawfully or ethically follow.

I have dealt with subordinates who made mistakes and hoped rank, loyalty or circumstances would somehow transfer responsibility somewhere else.

I have also been the person who stepped forward and absorbed consequences that properly belonged to somebody else.

Sometimes that was leadership.

Sometimes it was mercy.

Sometimes, looking back, it was misplaced mercy.

There is a difference.

One of the most painful forms of education is discovering that the person whose mistake you covered, whose reputation you protected or whose humanity you tried to preserve may not extend the same consideration when circumstances reverse.

That does not mean mercy was wrong.

It means mercy cannot require blindness.

There are people who make honest mistakes and deserve compassion.

There are people who become frightened and make poor decisions.

There are people who need someone senior to them to shoulder a little more weight while they recover their footing.

I still believe strongly in doing that.

But there are also people who construct a reality around themselves in which every fact must ultimately serve them.

They are not asking what happened.

They are asking what version of what happened best protects them.

Evidence that supports the story is important.

Evidence that contradicts it becomes unfair, irrelevant, technical, misunderstood, procedurally inadmissible, taken out of context—or somebody else's fault.

That is where stupidity in Bonhoeffer's sense becomes dangerous.

Not because the person cannot understand.

Because understanding would impose an obligation they do not wish to accept.

Sometimes Blindness Is Chosen

This is the part we are often too polite to discuss.

Human beings can be genuinely mistaken.

They can be misinformed.

They can misunderstand complex circumstances.

They can be fooled by authority.

They can become exhausted, frightened or overwhelmed.

All of that deserves understanding.

But there is another condition.

Intentional blindness.

The point at which continuing not to know becomes more comfortable than knowing.

The point where another person's suffering becomes an inconvenience to the narrative.

The point where admitting a mistake would require an apology, restitution, loss of authority or simple humility—and suddenly the evidence becomes extraordinarily difficult to understand.

I have encountered that too many times to believe it is rare.

And this is precisely why AI worries me.

Because artificial intelligence is rapidly becoming the greatest excuse generator humanity has ever invented.

AI Is the New Theatre. The Human Behaviour Is Ancient.

Generative AI is remarkably good at producing something humans find irresistible:

a coherent explanation.

Not necessarily the truth.

An explanation.

It can explain why a decision makes sense.

It can explain why a risk is low.

It can explain why a policy applies.

It can explain why a person is wrong.

It can explain why an organization acted reasonably.

It can summarize the evidence.

It can produce the executive briefing.

It can draft the investigation report.

It can write the legal-sounding clause.

It can create the compliance rationale.

It can make almost anything sound organized, informed and authoritative.

That is useful.

It is also dangerous.

Because human beings have always been capable of fooling themselves.

We have simply given ourselves a machine capable of helping us do it in excellent prose.

The Seduction of Algorithmic Fluency

The danger facing high-performing organizations today is rarely a rogue algorithm deciding independently to destroy the company.

The more immediate danger is cognitive offload becoming systemic complacency.

Large language models and automated synthesis systems do not hesitate in the way humans hesitate.

They do not necessarily signal uncertainty proportionately to their actual uncertainty.

They can deliver answers in polished, authoritative and beautifully structured language even when they are hallucinating facts, misunderstanding a legal standard, overlooking an important exception or simply reasoning from incomplete information.

The presentation creates psychological weight.

Fluency starts being mistaken for accuracy.

Speed starts being mistaken for competence.

Complexity starts being mistaken for intelligence.

And confidence starts being mistaken for truth.

When professionals are handed tools that eliminate much of the friction associated with research, drafting and analysis, there is an entirely understandable temptation to accept the result.

At first, it saves time.

Then the checking becomes lighter.

Then the source documents are consulted less frequently.

Then the summary becomes the source.

Eventually somebody asks where a particular statement came from and nobody quite knows.

But everybody remembers seeing it somewhere.

That is not efficiency.

That is functional surrender.

The moment a team stops verifying underlying evidence and begins defending synthetic output simply because the machine produced it, independent critical judgment has been bypassed.

The deficit is not primarily inside the model.

It is inside the human decision architecture surrounding the model.

What Frightens Me Is Not That AI Can Be Wrong

Humans have always been wrong.

I have certainly been wrong.

Every experienced leader has.

The ability to say I was wrong is one of the most important pieces of intellectual equipment a person can possess.

What concerns me is something different.

AI makes it increasingly possible to remain wrong with extraordinary confidence and supporting documentation.

That is new.

A person who wants to avoid an uncomfortable fact no longer has to invent the entire justification themselves.

The machine can help.

A manager who has already decided what happened can ask AI to summarize the evidence around that conclusion.

A regulator can process information through an automated framework and gradually begin treating the framework's result as reality.

An investigator can receive a machine-generated chronology and unconsciously allow the chronology to determine what evidence appears relevant.

An executive can receive a polished AI-generated briefing and never see the ambiguity, conflicting testimony or missing information underneath it.

A junior employee can assume the system knows better than they do.

A senior employee can use the system's apparent authority to overcome the junior employee's objection.

And everyone can tell themselves that the process was objective.

That is the theatre.he underlying human tendency is considerably older.

Mapping Bonhoeffer's Warning to the AI Enterprise

Here is the parallel as I see it:

That final row may be the most important.

Because this is where I believe a great deal of current AI governance thinking remains too shallow.

You Cannot Policy-Memo Your Way Out of Human Nature

Most corporate AI governance currently relies heavily on instruction.

Do not upload confidential information.

Verify the answer.

Use approved tools.

Check your sources.

Keep a human in the loop.

Be responsible.

All perfectly sensible.

And completely inadequate on their own.

We already tell people not to text while driving.

We already tell people to follow safety procedures.

We already tell managers not to retaliate.

We already tell investigators to remain objective.

We already tell executives to challenge assumptions.

We already tell organizations to follow their own policies.

Instruction matters.

But human behaviour does not magically reorganize itself because someone created a PDF and added an acknowledgement checkbox.

The more convenient the technology becomes, the less reliable exhortation becomes.

If an AI tool provides a convincing answer in six seconds and verification requires twenty minutes, organizational pressure will gradually migrate toward the six-second answer.

Not necessarily because anyone is malicious.

Because people are busy.

Because deadlines exist.

Because cognitive effort is expensive.

Because seniority creates pressure.

Because nobody wants to be the person holding up the workflow.

Because the machine sounds certain.

And because once an organization has invested heavily in a system, questioning its output can feel uncomfortably similar to questioning the judgment of the people who bought it.

This is how cognitive surrender becomes institutional.

I Have Seen the Same Thing Without AI

That is why none of this feels theoretical to me.

I have stood in situations where authority said one thing and my own judgment said another.

I have been told to do things I would not do.

I refused.

I reported matters further up the chain when I believed that was required.

There were consequences.

In two cases, serious consequences.

Doing the right thing can be extremely destructive to a career when the wrong person has invested their authority in the wrong answer.

That experience stays with you.

So does another kind.

Taking responsibility for somebody else's mistake because you believe that is what leadership requires.

Protecting a subordinate.

Giving somebody room to recover.

Not humiliating a person merely because you have the authority to do it.

Choosing mercy.

Then discovering later that the person involved had been operating from a very different moral framework.

These are not abstract lessons about organizational psychology.

They are scars.

And scars have instructional value if you are prepared to look at them without becoming bitter.

What I learned is that authority does not guarantee wisdom.

Education does not guarantee wisdom.

Professional status does not guarantee wisdom.

Good intentions do not guarantee wisdom.

Technology certainly does not guarantee wisdom.

And being absolutely certain does not guarantee that you are right.

The Most Dangerous Sentence May Become: “The AI Says…”

We have heard versions of this sentence before.

“The policy says…”

“The computer says…”

“The regulations say…”

“Head office says…”

“The system won't allow it…”

“The numbers say…”

Sometimes those statements are entirely legitimate.

Sometimes they are ways of avoiding personal responsibility.

AI gives us a spectacularly powerful new version:

“The AI says…”

The machine becomes the unseen participant in the room.

The synthetic authority.

The invisible witness.

The consultant nobody hired but everybody cites.

And because its language is fluent, its apparent authority can exceed the quality of its evidence.

That should worry leaders.

Especially because nobody has to consciously decide to surrender judgment.

It can happen one tiny convenience at a time.

Human Beings Do Not Always Want the Truth

This may be the hardest part of the argument.

We talk endlessly about giving people better information.

Sometimes better information solves the problem.

Sometimes it does not.

Because information is useful only when the person receiving it is still willing to change their conclusion.

I have lived through situations where facts did not resolve disagreement because the disagreement was no longer about facts.

It was about identity.

Authority.

Ego.

Self-preservation.

Status.

Money.

Fear.

Pride.

Or the intolerable possibility of having to say:

I was wrong.

Once that happens, reason becomes theatrical.

Evidence is no longer being evaluated.

It is being recruited.

That is the precise human weakness AI can amplify.

A person seeking truth can use AI to challenge themselves.

A person seeking validation can use exactly the same machine to protect themselves from challenge.

The difference is not the technology.

The difference is the human purpose.

And This Is Where Mercy Matters

I do not want an argument about critical thinking to become an argument for mercilessness.

Quite the opposite.

If anything, life has taught me that truth and mercy need each other.

People make mistakes.

People become frightened.

People fail.

People misunderstand things.

Good people occasionally do stupid things.

So do intelligent people.

So do experienced people.

So do I.

An organization that treats every error as moral failure eventually teaches its people to hide errors.

That is terrible governance.

Mercy creates room for truth.

If employees believe an honest admission will automatically destroy them, they will rationalize, conceal and defend.

But mercy has a boundary.

Mercy cannot mean agreeing that something false is true.

Mercy cannot require the innocent person to carry the consequences forever.

Mercy cannot become an institutional excuse for refusing accountability.

And mercy certainly should not be manipulated by someone who repeatedly creates harm and then relies on somebody else's conscience to rescue them from the consequences.

I have learned that distinction the expensive way.

The AI Version of Self-Absorbed Blindness

Imagine a workplace conflict.

An employee raises a concern.

Management feeds documents into an AI system.

The system produces a concise summary supporting management's existing interpretation.

The employee produces contradictory evidence.

Management asks the system another question.

A revised explanation appears.

More polished.

More detailed.

More authoritative.

The underlying assumption remains untouched.

Everybody believes they are investigating.

But what may actually be happening is automated rationalization.

The machine has become a mirror.

This can happen in legal work.

Compliance.

Human resources.

Insurance.

Healthcare administration.

Risk management.

Financial decisions.

Public-sector decision-making.

Military planning.

Investigations.

Hiring.

Discipline.

Performance management.

Procurement.

Anywhere human consequences follow from information.

That is why “human in the loop” is not sufficient if the human has already surrendered independent judgment.

A human rubber stamp is still a rubber stamp.

Why Persuasion Fails—and Architecture Must Intervene

Bonhoeffer's insight was that reasoning alone cannot always free a person who has surrendered independent judgment.

That observation has enormous implications for AI governance.

We already see versions of it:

• A team uses an uncalibrated tool to generate contract language or compliance material.

• A subject-matter expert identifies a critical disconnect.

• Instead of returning to the statute, source document or primary evidence, the team defends the result by referring to the sophistication of the platform.

Or:

• An automated risk system scores something green.

• Frontline employees report that conditions on the ground are deteriorating.

• Management trusts the dashboard because the dashboard feels objective.

Or:

• An AI-generated briefing contains an incorrect premise.

• Several downstream reports consume the briefing.

• The repetition of the error begins to look like corroboration.

This is how synthetic consensus emerges.

One machine-generated statement becomes an input to another system.

The second system summarizes it.

A human copies the summary.

Someone else cites the human document.

The same unsupported proposition has now acquired three layers of apparent authority.

Nothing became more true.

It merely became more institutional.

Telling knowledge workers to “double-check everything” will not solve this.

Cognitive fatigue is real.

Convenience is powerful.

Organizational pressure is powerful.

Authority is powerful.

And beautifully written synthetic language is extraordinarily persuasive.

Resilience therefore cannot rely on individual vigilance alone. It has to be built into operational doctrine.

An enterprise does not fail because its people lack intelligence. It fails when its workflow makes independent judgment unnecessary, inconvenient—or dangerous.

We Need to Engineer Some Friction Back Into High-Stakes Decisions

For years technology has treated friction as something to eliminate.

Usually that is sensible.

Nobody wants friction when resetting a password or ordering office supplies.

But some friction exists for a reason.

A cockpit checklist is friction.

Two signatures on a high-risk transaction are friction.

Independent legal review is friction.

A second medical opinion is friction.

A weapons safety procedure is friction.

An audit is friction.

A challenge function is friction.

Separation of duties is friction.

The question is not whether a process is frictionless.

The question is where friction protects us from irreversible mistakes.

High-stakes AI requires deliberate cognitive friction.

Not bureaucracy for its own sake.

A structural moment where somebody other than the generator is required to ask:

Where did this come from?

What evidence supports it?

What contradicts it?

What assumptions were made?

Who could be harmed if this is wrong?

Who is accountable for the final decision?

And perhaps most importantly:

What would make us change our minds?

If the answer to that final question is “nothing,” you are no longer conducting analysis.

You are defending a belief.

Three Practical Principles to Prevent Automated Cognitive Surrender

1. Enforce Distributed Human-in-the-Loop Oversight

Never assume that the person who prompts, configures or operates an automated system should automatically become the sole human authority approving its output.

That is not meaningful human oversight.

It is self-review.

High-consequence decisions need independent points of human judgment.

That is why I favour Distributed Human-in-the-Loop (DHITL) oversight.

Different people see different things.

The operator sees the workflow.

The subject-matter expert sees technical errors.

The frontline employee sees reality on the ground.

The manager sees operational consequences.

The compliance or legal professional sees obligations.

The affected human being may see facts that none of them can see from the dashboard.

Distributed oversight does not eliminate error.

It makes cognitive surrender harder.

2. Demand Grounded Provenance, Not Plausible Prose

An AI-generated summary with no demonstrable connection to primary evidence is not evidence.

It is a proposition.

Possibly a useful proposition.

Possibly an accurate proposition.

But still a proposition.

Organizations need evidence trails connecting important outputs back to the material from which they were derived.

Primary documents.

Raw data.

Version histories.

Decision records.

Human approvals.

Source references.

Provenance.

Where appropriate, immutable or tamper-evident records.

The more consequential the decision, the less acceptable “the model generated it” should become as an explanation.

3. Separate Generation From Verification

Treat machine output as draft intelligence.

Never as completed judgment.

Generation and verification are different cognitive functions.

The person excited by what an AI tool has produced is not always the best person to discover why it is wrong.

Domain experts must remain critical evaluators rather than becoming downstream consumers of synthetic conclusions.

This is particularly important when AI output aligns perfectly with what the decision-maker already wanted to believe.

That is when verification matters most.

But I Would Add a Fourth Principle

Protect the dissenter.

I believe this more strongly because of things I have lived through.

If your governance structure technically permits challenge but punishes the person who actually challenges authority, you do not have meaningful oversight.

You have theatre.

Organizations love the language of “speak up.”

The real test comes when somebody speaks up about something senior people do not want to hear.

If the person pointing out the error risks their career while the person protecting the error risks nothing, the organization has already told everyone which behaviour it values.

I know what that looks like from the inside.

AI will make this issue more important, not less.

Because increasingly the dissenter may be challenging not just a senior manager.

They may be challenging a system that management spent hundreds of thousands or millions of dollars selecting, implementing and publicly praising.

That takes courage.

Governance must protect that courage.

The Dashboard Cannot Be Allowed to Defeat Reality

One of the great dangers of modern management is believing the abstraction instead of the thing itself.

The dashboard says green.

The employee says there is a problem.

The model says low risk.

The experienced operator says something feels wrong.

The synthetic report says compliant.

The underlying records say otherwise.

Which one wins?

There should be no automatic answer.

Technology should inform human judgment.

It should not anesthetize it.

Experienced people often possess forms of knowledge that are extremely difficult to encode.

Pattern recognition.

Context.

Institutional memory.

Moral intuition.

Understanding of human behaviour.

The faint sense that the pieces simply do not fit.

We should not romanticize intuition.

Intuition can be wrong.

But dismissing human experience merely because the machine's output is quantifiable is every bit as irrational as rejecting data because we dislike it.

The objective is not machine supremacy or human supremacy.

The objective is truthful decision-making.

AI Governance Is Really Human Governance

We use terms like AI governance because they are useful.

But ultimately we are not governing artificial intelligence.

The machine does not have a mortgage.

It does not want a promotion.

It is not embarrassed.

It does not fear a regulator.

It does not want to preserve its reputation.

It does not resent the employee who contradicted it.

It does not need the board to believe the project was successful.

Humans do.

So when an AI system produces a bad output, the governance question is not merely:

Why did the model do that?

It is:

Who accepted it?

Who checked it?

Who benefited from believing it?

Who was permitted to challenge it?

What evidence was available?

What evidence was ignored?

What would have happened to the person who said no?

And who ultimately made the decision?

Those are human governance questions.

AI merely makes them harder to avoid.

The Greatest Risk Is Not Artificial Intelligence Becoming Human

We spend a remarkable amount of time worrying about machines becoming more like us.

I am increasingly concerned about the opposite.

Humans becoming more like machines.

Receiving an input.

Following a procedure.

Repeating the authorized answer.

Accepting the generated conclusion.

Ignoring contradictory context.

Passing the output downstream.

And calling that judgment.

That is cognitive surrender.

It can happen to intelligent people.

It can happen to good people.

It can happen inside excellent organizations.

It can happen to me.

The defence is not believing that we are too smart to fall for it.

That belief is probably the beginning of the problem.

The defence is building systems that assume human beings are fallible, authority can be wrong, technology can be wrong, institutions can protect themselves, and every one of us is capable of seeing what we want to see.

The Bottom Line

Artificial intelligence should elevate human capability.

It should not replace human judgment.

When we outsource labour, we create leverage.

When we outsource repetitive calculation, we create efficiency.

When we outsource retrieval, organization and drafting, we can dramatically increase productivity.

But when we outsource critical discernment, we create vulnerability.

And when we use technology to protect ourselves from truths we do not wish to confront, we have created something considerably worse than a productivity problem.

We have mechanized an ancient human weakness.

Bonhoeffer's warning was never really about unintelligent people.

It was about the conditions under which people surrender themselves.

I have seen those conditions in different forms throughout my life.

I have seen authority preferred over truth.

I have seen careers damaged because somebody refused to accept an improper direction.

I have seen people protect themselves through narratives.

I have seen mercy extended and exploited.

I have seen intelligent people become astonishingly resistant to facts when accepting those facts would require humility.

And I have also seen the opposite.

People admitting mistakes.

Leaders taking responsibility.

Subordinates showing courage.

Superiors listening when it would have been easier not to.

People changing their minds because the evidence required it.

People choosing mercy when they had every opportunity to punish.

Those moments are why I am not pessimistic about AI.

But I am not complacent either.

The challenge before us is not simply to build more intelligent machines.

It is to make certain that, surrounded by those machines, we continue doing the difficult work of being thinking, responsible, merciful human beings.

That means maintaining the courage to ask inconvenient questions.

The discipline to verify.

The humility to change our minds.

The willingness to hear the person with less authority.

The wisdom to distinguish mercy from enabling.

And the integrity to accept responsibility when the truth points toward us.

Build faster models if they help.

Automate what should be automated.

Use AI aggressively where it improves human capability.

But engineer the structural discipline necessary to keep independent human judgment awake at the controls.

Because the greatest danger may never be that artificial intelligence develops a mind of its own.

It may be that we stop using ours.

Supporting

Getting Veterans and First Responders back on mission.!

Veteran-inspired AI Governance & Trust Infrastructure
Trusted by Heroes and Mounted Rifles Management

Leadership and peer support are taught through RedFridayTalks.Help

The same governance protections are available to everyone.

info@trustedbyheroes.com

© 2026. All rights reserved.