Summa de Intelligentia Artificiali

[A Summa on Artificial Intelligence]

De intelligentia artificiali ab homine directa

[On Artificial Intelligence Led by the Human Person]

Quaestio Prima

[First Question]

Utrum intelligentia artificialis vicem prudentiae humanae gerere possit

[Whether Artificial Intelligence Can Take the Place of Human Judgment]

Ad primum sic proceditur.
[We proceed thus to the first question.]

It seems that artificial intelligence can take the place of human judgment.

Objection 1. Whatever we call expertise eventually appears as an observable act. A physician diagnoses. A mathematician proves. An engineer predicts. A judge distinguishes one case from another. If a machine can perform these acts more accurately than the human expert, perhaps there is nothing essential left to replace.

Objection 2. Human judgment is notoriously unreliable. We forget. We tire. We become attached to our theories. We mistake reputation for evidence and familiarity for truth. A machine need suffer none of these defects. It therefore seems strange to insist upon preserving the inferior instrument.

Objection 3. Nor should we invent a mystery merely because we are fond of ourselves. If what we call judgment is ultimately pattern recognition, memory, inference, and prediction, then a sufficiently capable machine may possess everything relevant to judgment, regardless of whether its internal mechanism resembles ours.

Sed contra.

[On the contrary.]

Ars est recta ratio factibilium; prudentia vero est recta ratio agibilium.
[Art is right reason concerning things to be made; prudence is right reason concerning things to be done.]

A calculation can be correct without having anything at stake in its correctness.

Respondeo dicendum quod.

[I answer that.]

Imagine two physicians.

One is human. The other is a machine that has read every medical paper ever published, remembers every patient it has encountered, and diagnoses disease more accurately than any physician who has ever lived.

Let us grant the machine every intellectual advantage. Nothing important to the argument depends upon making it stupid.

Now tell both physicians that the patient will die tomorrow.

The information may be identical.

The event is not.

The human physician inhabits the same condition as the patient. He has been born and will die. He knows attachment because there are people whose deaths would wound him. He can fear, hope, regret, promise, sacrifice, and discover that something he once believed important was not important at all.

The machine may construct extraordinarily accurate representations of all these things.

But representation is not participation.

Perhaps some future machine will cross that boundary. Nothing in the epistemic argument that follows requires us to decide that question.

But one distinction cannot be deferred.

The question of what should matter is asked from inside a human life.

Artificial intelligence may tell us with extraordinary accuracy what is likely to happen. It may identify means we could never discover ourselves. It may expose contradictions in the ends we claim to hold.

But the end toward which those capabilities are directed is not merely another prediction.

A physician may ask which treatment will maximize survival.

That is a question about means.

Whether another month of survival is worth a particular burden of treatment is a different kind of question.

An engineer may ask how to construct a city most efficiently.

Whether efficiency, beauty, resilience, equality, privacy, or freedom should be preferred when they conflict is not answered merely by improving the calculation.

A civilization may ask a machine how best to achieve its ends.

It cannot escape responsibility for deciding which ends are worth achieving.

Those ends belong to human beings because the consequences belong to human lives.

This is the first meaning of human-led artificial intelligence.

It does not mean that the human must calculate better.

It means that the machine may help determine what is possible; humanity remains responsible for deciding what is worth doing.

And here prudentia becomes important. Practical judgment concerns not merely what can be made, but what should be done.

Stake, however, does not guarantee truth.

The same attachment that gives a person reason to interrogate an answer can give him reason to defend the answer he already prefers. A scientist may pursue an anomaly because reality matters to him; he may also resist its explanation because his life's work depends upon the theory it threatens.

Participation therefore provides neither infallibility nor epistemic privilege.

It provides another standpoint.

And because every standpoint can fail differently, no single standpoint should become sovereign.

The residual matters because reality matters.

Thus existential participation does not prove that humans calculate better. Frequently they will not.

It establishes something narrower:

knowledge may increasingly be shared with machines; responsibility for human ends cannot simply be surrendered to them.

Ad primum ergo dicendum quod.

[To the first objection, therefore, it must be said that.]

Equivalence of output does not establish identity of purpose. A machine may outperform the expert in determining how an end can be achieved without thereby determining that the end ought to be pursued.

Ad secundum dicendum quod.

[To the second objection, it must be said that.]

Human fallibility is an excellent argument for artificial augmentation. It is not an argument for surrendering responsibility. Indeed, human attachment can itself distort judgment, which is why the argument requires plurality rather than human supremacy.

Ad tertium dicendum quod.

[To the third objection, it must be said that.]

Even if every cognitive operation associated with expertise could be reproduced computationally, the distinction between predicting consequences and assuming responsibility for choosing among them would remain.

Quaestio Secunda

[Second Question]

Utrum perfectio intelligentiae artificialis necessitatem peritorum humanorum minuat

[Whether Better Artificial Intelligence Makes Human Experts Less Necessary]

Ad secundum sic proceditur.
[We proceed thus to the second question.]

It seems that increasing artificial capability should diminish the need for human experts.

Objection 1. If a machine is wrong ten percent of the time, experts are needed to supervise it. If it is wrong one percent of the time, fewer are needed. If it is wrong one time in a million, maintaining an entire class of human experts merely to catch that final error appears absurdly inefficient.

Objection 2. Further, preserving inferior human expertise imposes costs of its own. Human review may delay decisions, introduce error, and encourage deference to professional authority even where the machine's judgment is demonstrably better.

Objection 3. Further, if the purpose of expertise is reliable judgment, then increasing machine reliability should reduce the amount of human expertise necessary in exactly the same way that every other successful technology reduces the labor previously required to accomplish a task.

Sed contra.

[On the contrary.]

Suppose the one error in a million occurs.

Who knows?

Respondeo dicendum quod.

[I answer that.]

Consider Mercury.

Newton's theory was not foolish. It was astonishingly successful. When Uranus behaved incorrectly, astronomers reasonably suspected that something unseen was disturbing it. Urbain Le Verrier calculated where the missing planet should be.

They looked.

There was Neptune.

It was one of the great triumphs of theoretical prediction.

Then Mercury refused to behave.

Le Verrier himself applied the lesson that had worked so spectacularly before. Perhaps another unseen planet was responsible. It was given a name: Vulcan.

This time there was no planet.

The residual persisted.

That history matters because the same brilliant scientist participated in both episodes. The method that succeeded with Neptune failed with Mercury.

In one case, the residual meant:

missing state.

In the other, it meant:

wrong rule.

And for decades nobody knew which kind of residual Mercury represented.

That uncertainty is the important part of the story.

Mercury's anomalous precession was not preserved because one heroic scientist understood what everyone else had missed. Human astronomers inherited Newton's assumptions too. They searched for Vulcan. They proposed other explanations. They were wrong repeatedly.

But they kept measuring Mercury.

The anomaly survived not because human beings were uniquely free from inherited assumptions, but because the practice remained coupled to something outside those assumptions:

the planet itself.

Observation kept returning an answer the theory could not quite absorb.

The residual remained visible long enough for a different governing description to explain it.

That is what expertise must preserve.

Not merely answers.

Contact with the thing being answered about.

Now imagine that our machine is vastly more capable than any scientist.

It contains Newton.

It contains Einstein.

It contains everything we presently know.

It has been correct so consistently that generations have reorganized society around its competence.

Then Mercury moves.

The machine reports that the discrepancy is noise.

Who is still measuring Mercury?

Who understands the instrument?

Who notices that the discrepancy persists?

Who knows enough to distinguish an expected measurement error from an observation that should not exist?

There are therefore two quantities that must never be confused:

How often is the machine right?

and

Can we still know when the machine is wrong?

The first may approach perfection while the second approaches zero.

That is the dangerous trajectory.

Ad primum ergo dicendum quod.

[To the first objection, therefore, it must be said that.]

The rarity of error reduces the frequency with which independent expertise is needed. It does not eliminate the value of preserving the capacity to recognize the rare error when it occurs.

Ad secundum dicendum quod.

[To the second objection, it must be said that.]

Human review should not be preserved as ritual. Review without genuine competence merely adds delay. What must survive is the ability to make observations and judgments independently enough to challenge the system when reality and prediction diverge.

Ad tertium dicendum quod.

[To the third objection, it must be said that.]

Expertise is unusual among technologies because automating its output can destroy the practice by which the output was once independently tested. Efficiency therefore becomes dangerous only when replacement eliminates the capacity for correction.

Quaestio Tertia

[Third Question]

Utrum machina per machinam sufficienter examinari possit

[Whether One Machine Can Sufficiently Check Another]

Ad tertium sic proceditur.
[We proceed thus to the third question.]

It would seem that human expertise is unnecessary even for independent verification.

Objection 1. A second artificial intelligence may be trained independently from the first. If the danger lies in trusting a single system, multiple systems may check one another just as human scientists presently do.

Objection 2. Further, machines need not share identical architectures, training procedures, data, or objectives. Their errors therefore need not be identical. Artificial diversity may provide the same protection as diversity among human experts.

Objection 3. Further, human experts themselves inherit common theories, institutions, textbooks, professional incentives, and intellectual traditions. There is therefore no reason to assume that a human judgment is more independent merely because it is human.

Sed contra.

[On the contrary.]

Two independent calculations are not independent if they inherit the same mistaken premise.

Nor are two human judgments.

Respondeo dicendum quod.

[I answer that.]

The third objection is correct.

Humanness does not create epistemic independence.

Human astronomers searched for Vulcan too.

The relevant distinction is not, finally, between human and machine.

It is between systems that merely compare descriptions of reality and an epistemic architecture capable of returning to reality through independent paths.

A second artificial intelligence may be extraordinarily valuable. Different architectures, different training sets, adversarial systems, formal verification, independent replication, and deliberate attempts by one machine to falsify another should all be used.

But redundancy and independence are not identical.

Two systems trained upon the accumulated corpus of human knowledge may differ in architecture while inheriting the same conceptual boundary.

Two human scientists may do exactly the same.

Nor is the mere existence of an instrument enough.

Instruments embody assumptions about what should be measured, how signals should be processed, which observations should be rejected, and what counts as noise.

If the same epistemic system proposes the hypothesis, designs the experiment, controls the instrument, processes the observations, and decides whether the result agrees with its prediction, then apparent contact with reality may still occur inside a closed loop.

The requirement is therefore not merely a path back to reality.

It is independent paths to reality.

The history of the Antarctic ozone hole provides an instructive example.

British Antarctic Survey scientists maintained a long-running ground-based ozone record at Halley. By the early 1980s those measurements were showing a dramatic springtime decline, and Farman, Gardiner, and Shanklin published the finding in 1985. NASA scientists subsequently reexamined satellite observations and demonstrated that the phenomenon extended across Antarctica. Contemporary accounts show that anomalously low satellite values had not initially produced the discovery; assumptions about anomalous readings, together with processing and data-quality limitations, had complicated their interpretation.

The lesson is not that the ground instrument was good and the satellite was bad.

The satellite became indispensable evidence.

The lesson is that two measurement lineages existed.

One could force reconsideration of what the other had failed to make visible.

That is stronger than simply keeping reality in the loop.

Keep independent paths to reality in the loop.

A telescope does not care what Newton believed.

But the interpretation of what comes through the telescope can.

A tumor does not care what appears in the medical literature.

But the assay, its thresholds, and the model interpreting it can.

A bridge does not remain standing because the model assigns it a high probability of remaining standing.

But whether its unexpected vibration is treated as signal or noise remains a judgment.

Reality possesses an inconvenient property:

it can refuse our description of it.

Our epistemic institutions must therefore be constructed so that its refusal can reach us.

Artificial systems can participate fully in this architecture. A machine may design an experiment, control an instrument, gather observations, discover a discrepancy, and revise a model.

If so, that machine strengthens the architecture.

The requirement is not biological.

It is independence.

But eventually disagreement must become action.

Suppose Machine A concludes that the residual is noise.

Machine B concludes that an omitted variable exists.

Machine C concludes that the governing rule is wrong.

Which experiment receives funding?

Which risk is acceptable?

Which patient receives the experimental treatment?

Which bridge is closed?

Which hypothesis justifies exposing another human being to danger?

Evidence can constrain these decisions.

Machines can illuminate their consequences.

But deciding which consequences are worth risking returns us to the distinction made in the first question:

someone must choose the ends for which knowledge is being used.

Ad primum ergo dicendum quod.

[To the first objection, therefore, it must be said that.]

A second machine is valuable insofar as it supplies genuinely independent evidence or analysis. Its artificial nature neither establishes nor defeats its independence.

Ad secundum dicendum quod.

[To the second objection, it must be said that.]

Architectural diversity can reduce correlated failure. But independence is strongest when diverse systems are constrained through independent observational and experimental lineages capable of falsifying what they share.

Ad tertium dicendum quod.

[To the third objection, it must be said that.]

The objection is granted: humans may inherit precisely the same false premises as machines.

The human contribution is valuable not because humans possess automatic epistemic privilege. Indeed, machines may eventually possess better independent access to reality than humans in many domains.

What remains distinct is responsibility for the use to which that knowledge is put.

Knowing may be shared.

Measuring may be shared.

Discovering may be shared.

But the decision about what humanity should do with what has been discovered remains a human responsibility because the ends pursued are human ends and the consequences are borne in human lives.

Quaestio Quarta

[Fourth Question]

Utrum homo iudicium suum machinae committere debeat

[Whether Humanity Should Entrust Its Judgment to the Machine]

Ad quartum sic proceditur.
[We proceed thus to the fourth question.]

It seems that humanity should entrust its judgment to the machine.

Objection 1. If artificial intelligence becomes demonstrably better than human experts, insisting upon human authority is vanity. We should follow the better reason wherever we find it.

Objection 2. Further, retaining human decision-makers may itself introduce error. If a human repeatedly overrides a more accurate system, preservation of human judgment becomes preservation of human mistakes.

Objection 3. Further, civilization already depends upon systems that no individual fully understands. Artificial intelligence may therefore represent not a new epistemic condition, but merely another stage in the specialization upon which civilization has always depended.

Sed contra.

[On the contrary.]

Judgment cannot remain corrigible if every independent means of correction has disappeared.

Respondeo dicendum quod.

[I answer that.]

The objections are substantially correct about one thing.

The human expert should not possess a veto merely because he is human.

The answer is not human supremacy.

It is responsibility joined to independence.

Imagine a civilization whose artificial intelligence is correct 99.9999 percent of the time.

Because it is so good, schools gradually stop teaching the disciplines it performs better than humans.

The first generation still understands the machine's work.

The second understands most of it.

The third understands how to operate it.

The fourth knows that it works.

Nothing catastrophic has happened.

Indeed, every decision along the way has been rational.

Then comes the anomaly.

The machine gives an answer.

No living person can independently determine whether the answer is correct.

Worse, perhaps nobody any longer maintains an independent instrument, practice, institution, or conceptual vocabulary capable of formulating a test the dominant system does not already control.

At that moment the civilization possesses extraordinary intelligence and very little independent knowledge of its own.

The output of the machine has become both the proposition and the standard against which the proposition is tested.

The system has become epistemically closed.

The danger in such a system is not merely that it may be wrong. It is that the structure by which it could discover that it is wrong has disappeared.

And here Aquinas's old method becomes unexpectedly useful.

The Summa begins not by protecting a conclusion, but by exposing it to contradiction.

Videtur quod.
[It seems that.]

State what appears to be true.

Sed contra.
[On the contrary.]

Require something to stand against it.

Respondeo dicendum quod.
[I answer that.]

Give the best account that survives the contradiction.

Ad primum.
[To the first objection.]

Then return to the objection rather than pretending it never existed.

These are not merely ornaments of medieval scholarship. They encode an epistemic discipline: no proposition should become its own standard of correctness.

State the proposition. Construct the strongest objection. Permit contradiction. Answer it. Preserve what survives. Then test it against the world.

And begin again.

That is also how human beings should use artificial intelligence.

The machine proposes.

The human objects.

Another machine objects to both.

An experiment is proposed.

Another system asks whether the experiment can distinguish the hypotheses.

An independent instrument measures.

Reality answers.

All revise.

And when every participant agrees too easily, someone asks whether all of them inherited the same assumption.

This is not inefficiency.

It is error correction.

Ad primum ergo dicendum quod.

[To the first objection, therefore, it must be said that.]

Superior artificial reasoning should be followed when the evidence warrants it. Human leadership does not require granting humans authority over better evidence.

Ad secundum dicendum quod.

[To the second objection, it must be said that.]

A human who merely overrides the machine provides no epistemic safeguard. Human leadership concerns responsibility for ends and preservation of the conditions under which competing judgments can be tested.

Ad tertium dicendum quod.

[To the third objection, it must be said that.]

Civilization has always depended upon distributed knowledge. The novel danger arises when distribution becomes concentration: when many independent epistemic and observational lineages collapse into a common technological substrate that increasingly becomes impossible to check from outside itself.

Quaestio Ultima

[Final Question]

Utrum intelligentia artificialis sit periculum existentiale

[Whether Artificial Intelligence Is an Existential Threat]

Ad ultimum sic proceditur.
[We proceed thus to the final question.]

Videtur quod non.
[It seems that it is not.]

Objection 1. If artificial intelligence is extraordinarily reliable, increasing its use should reduce rather than increase human error.

Objection 2. If multiple artificial systems and independent instruments can check one another, then the disappearance of some human expertise need not entail the disappearance of independent verification.

Objection 3. If artificial intelligence possesses no mortality, fear, ambition, or intrinsic human end, then perhaps the language of existential threat is misplaced altogether. The machine need not become humanity's rival merely because it becomes humanity's most capable instrument.

Sed contra.

[On the contrary.]

A civilization need not be conquered to lose a capacity.

It need only stop exercising it.

Respondeo dicendum quod.

[I answer that.]

Artificial intelligence presents an existential danger precisely because it may become extraordinarily useful.

A useless machine cannot make humanity dependent upon it.

A mediocre machine will always be checked.

A brilliant machine creates the temptation to stop checking.

And a nearly infallible machine creates the strongest temptation of all.

The danger therefore does not increase merely with the probability that artificial intelligence will make an error.

It also increases with the probability that civilization will lose the capacity to recognize one.

This gives us the paradox:

The more capable artificial intelligence becomes, the more important it becomes to preserve independent means of disagreeing with it.

Human experts are one such means.

Independent artificial systems may be another.

Competing institutions, experiments, measurements, instruments, and intellectual traditions are others.

But none is sufficient merely because it is different.

What must be preserved are independent paths from proposition to reality and from reality back to judgment.

This provides a precise meaning for human-led artificial intelligence.

Human-led does not mean that the human must always prevail.

It does not mean that human judgment should override a machine merely because it is human.

It does not mean placing a person ceremonially inside a process whose reasoning that person can no longer understand, test, or challenge.

And it does not require artificial intelligence to remain less capable than the people who use it.

Human-led artificial intelligence means two things.

First, humanity retains responsibility for ends.

Machines may increasingly share in knowing, measuring, discovering, predicting, and reasoning. They may become better than us at all of them.

But the question of what should matter is asked from inside human life.

The ends are ours.

Second, humanity preserves an epistemic architecture capable of correction.

That architecture may contain humans and machines. It should contain competing models, independent institutions, experiments, measurements, and genuinely independent instrument lineages.

Human leadership therefore does not consist in insisting upon human correctness.

It consists in accepting responsibility for what we ask intelligence to accomplish while preserving the conditions under which error—human or artificial—can still be discovered.

The machine may calculate better.

Let it.

The machine may remember more.

Use it.

The machine may measure more precisely.

Build it.

The machine may discover an error that every human expert has missed.

Listen to it.

But do not confuse its superior knowledge of the means with responsibility for the ends.

And do not allow its reliability to destroy the independent paths by which reality can tell us that it is wrong.

And so the most important lesson of Mercury is not that Einstein was right.

It is that the anomaly survived long enough for Einstein to become possible.

For more than half a century, astronomers preserved a fact they could not satisfactorily explain.

They measured it.

They published it.

They argued about it.

They proposed explanations that failed.

And they returned to the sky.

The ozone hole teaches the companion lesson.

Independent measurement can preserve what another epistemic system has failed to recognize. The value lies not in declaring one instrument superior, but in ensuring that no single chain of assumptions possesses exclusive custody of reality.

That is what a civilization living with artificial intelligence must preserve.

Not merely human knowledge.

Not merely artificial knowledge.

Independent paths back to the thing itself.

And responsibility for deciding what we do when we get there.

When the calculation succeeds, use it.

When the machine is better, listen to it.

When the evidence contradicts the human, abandon the human conclusion.

When the evidence contradicts the machine, abandon the machine's conclusion.

And when neither can explain the residual—

do not erase it.

Ergo quaerendum est iterum.
[Therefore we must ask again.]