Heavy industry does not lack data, and it no longer lacks models. A modern concentrator, sugar mill or treatment plant records millions of readings a day, and a new generation of industrial AI tools can turn them into fluent, confident advice in seconds. What operators still lack is a way to know, before they act, whether that advice is right.
Plausible is not the same as possible
Machine-learning models are very good at finding patterns and poor at recognising when they are wrong. They learn from history, and a plant rarely stays inside its history for long: the ore changes, a new effluent stream is connected, a sensor drifts, the rainy season arrives. Large language models add a new failure mode. They are built to produce answers that read well, and a well-written wrong answer is harder to catch than a badly written one.
In a plant, the consequences of a wrong answer are physical. A setpoint change that lifts recovery in one circuit can quietly break the water balance in another, push a discharge parameter past its permit limit, or consume more reagent than it saves. On a dashboard, each of those failures looks like a good idea until the shift after it has been applied.
What engineers mean by an oracle
In software engineering, an oracle is the mechanism that tells you whether an output is correct. A test without an oracle only tells you that the program ran. The same distinction applies to industrial AI: a recommendation that has not been checked against something independent of the model that produced it has been generated, not verified.
Here heavy industry has an advantage. Unlike many fields where AI is deployed, a process plant comes with oracles built into nature and law. Mass is conserved. Water that enters a circuit has to leave it somewhere. Energy balances close or they do not. The permit sets limits in writing, and the economics can be expressed in money and probability. Any recommendation worth acting on should survive all of them.
An answer that cannot be checked is an opinion, however confident it sounds.
Three tests before anyone acts
The first test is physical. Does the recommended action respect the conservation of mass, water and energy across the whole plant, not only in the circuit being optimised? Is it thermodynamically feasible, and does it keep the equipment inside its safe operating envelope? A recommendation that improves one number by breaking a balance elsewhere is not an optimisation. It is a transfer of loss.
The second test is regulatory. Does the action respect the permit in force today, including water-use grants, discharge limits and conditions on tailings or residues, and does it still respect them at the edges of normal variability, not only on an average day? Permits change, and the check has to change with them.
The third test is economic, and it is the one most often skipped. A recommendation should carry a cost as well as a benefit, and the benefit should come with an honest range rather than a single number. Plant decisions are usually justified by an expected gain, but the expectation is only half of the story. The other half is the spread: how bad the bad case is, and how likely.
Questions to ask any industrial AI vendor
Operators do not need to understand a vendor’s architecture to judge whether its recommendations can be verified. They need straight answers to a few questions:
- What happens when a recommendation violates a mass, water or energy balance? Is it stopped before it reaches the operator, or flagged afterwards?
- How are permit limits and the safe operating envelope represented, and who updates them when the permit changes?
- Can the system say that it does not know? When the plant moves outside the conditions the model has seen, does it abstain, and how would you find out?
- Does each recommendation come with a range of outcomes and a stated downside, or with a single number?
- Can an engineer, an auditor or a regulator reconstruct, months later, why a given recommendation was made?
- Has the system been tested on your own plant’s data, or only on a demonstration data set?
A vendor that answers these questions clearly is offering a decision tool. A vendor that answers them with accuracy statistics alone is offering a forecast.
Accountability stays with people
None of this takes the operator out of the loop. It changes what the operator is asked to trust. Instead of trusting a model, the shift supervisor trusts a recommendation that has already passed the tests the plant would apply anyway, and can defend the decision to the plant manager, the environmental regulator and the financier.
Regulation is moving in the same direction. The EU AI Act classifies AI systems used as safety components in critical infrastructure, including water supply, as high-risk, with requirements for human oversight and record-keeping. The NIST AI Risk Management Framework treats validity and reliability as the foundation of trustworthy AI, and Brazil’s AI bill, approved by the Senate in December 2024, follows a similar risk-based logic. Verification is becoming part of the licence to operate, not a feature.
From principle to practice
The chemistry changes from site to site; the discipline does not. Banker Group treats the oracle principle as a design requirement across its industrial-AI ventures, including PreAur in plant-wide operations and HydroQ in water engineering, and encourages operators to put the same questions to every system they evaluate, the group’s own included.
Sources
- European Union, Regulation (EU) 2024/1689, the Artificial Intelligence Act, 2024
- NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), 2023
- Federal Senate of Brazil, Bill PL 2338/2023 on artificial intelligence, approved December 2024
