Copper rivets, rogue agents and why AI failures happen at the joins
In the first article in this series I wrote about how the California Gold Rush created an entire economy around people hoping to find gold. Picks, shovels, pans, boots, food, transport and accommodation all became businesses in their own right. Every gold rush seems to acquire its fair share of people selling the shovels. [1]
The modern AI gold rush has done much the same thing. Prompt packs, courses, frameworks, AI literacy programmes, critical-thinking courses, verification systems, custom GPTs, agents, and courses have sprung up, many professing to teach people how to turn themselves into millionaires overnight.
The thing is Marketing sells, but reality and actual mainstream usability are two entirely different things. There is however a really useful lesson to be taken from the Gold Rush that isn’t about shovels at all.
It’s about trousers, or rather jeans to be precise.
By the early 1870s, a tailor in Reno called Jacob W. Davis was making work trousers for people doing the sort of work that destroys work trousers. The problem wasn’t the cloth, it was the place where the cloth kept failing.
The pockets.
Davis started reinforcing those points with metal rivets. He approached Levi Strauss, whose San Francisco dry-goods business had the commercial infrastructure Davis didn’t have, and on 20 May 1873 US Patent 139,121 was granted for an “Improvement in Fastening Pocket-Openings”. [2]
Davis wasn’t trying to make trousers generally stronger. He had identified exactly where repeated pressure and strain caused the seams to rip and reinforced those particular points.
He didn’t invent another tool. He found the joint that kept failing under strain and put copper through it. That’s where I think the current AI conversation has a really interesting parallel.
What is Shadow AI?
At its simplest, Shadow AI is the use of AI tools inside an organisation without its approval or oversight. IBM describes it as the unsanctioned use of AI tools or applications by employees or end users without the approval or oversight of IT. [3]
The Intriguing bit for me is shadow, purely because curious employees haven’t waited for organisations to develop AI operating models or processes before using publicly available tools, and as we say, curiosity sometimes kills the cat.
Microsoft and LinkedIn surveyed 31,000 people across 31 countries in 2024. Among people already using AI at work, 78% said they were bringing their own AI tools into the workplace, rising to 80% in small and medium-sized organisations. At the same time, 60% of leaders said they worried their organisation’s leadership lacked a plan and vision for implementing AI. [4]
Those are survey findings, not a census of every British business, but the pattern is difficult to miss. People have picked up the metaphorical shovels and already started digging.
So a business situation could be that one team uses Copilot, and another ChatGPT. Somebody has connected Claude to a workflow, someone else has installed a browser extension. A meeting bot is storing transcripts somewhere, an automation is passing information from one system to another and a spreadsheet on somebody’s personal account quietly becomes part of a business-critical process.
Then somebody leaves, or a model changes, an integration breaks, or an experiment becomes a process, an output becomes a decision, a workaround becomes infrastructure and before you know it…
Six months later somebody has to explain why the bloody thing did what it did, what information it had access to, where it put the result and whether anybody checked it.
So why is Shadow AI a problem?
Because you cannot control what you cannot see, and you cannot properly assess the consequences of a change when you don’t know what is connected to it.
That makes Shadow AI more than a security problem. It’s a visibility problem, then a control problem, then a dependency problem and eventually an accountability problem, which ultimately makes it a people problem.
Buying another AI platform doesn’t solve that, neither does banning every AI tool in the shed.
The same problem becomes more obvious once AI stops merely producing things and starts doing things. I know that one from experience.
I let an agent loose inside an operational folder. It went off and did exactly what it wanted, including filing things in folders I had not authorised among many other things. I still find the odd remnant of the consequences months later.
I’d already developed my own description for it before I found the formal guidance: an agent let off the lead is rather like an ADHD spaniel, fine until it sees a squirrel, then it’s off doing its own thing at a rate of knots.
The National Cyber Security Centre is slightly more restrained in its language. Its August 2026 interim guidance on agentic AI says organisations should decide how much autonomy an agent actually needs, think about what could go wrong, place agents inside appropriate sandboxes, log and monitor their activity, make actions attributable and retain the ability to pull the plug. [5]
It also warns not to rely on prompting alone, and that part is especially important.
How do you control an AI agent?
Not simply by giving it better instructions. Instructions matter, but controls have to survive when the instruction doesn’t. The greater the autonomy, the NCSC says, the greater the potential impact when an agent malfunctions, accesses information it shouldn’t or acts outside its intended scope. [5]
Which brings us straight back to Davis.
- Autonomy limits.
- Sandboxing.
- Logging.
- Attribution.
- Human approval at relevant points.
- Emergency shutdown.
None of those is another shovel, they are reinforcements placed where failure would matter. They are essentially the rivets that hold it all together.
So where are the seams in an AI-enabled business?
- They appear where AI meets human judgement.
- Where one tool hands work to another.
- Where data leaves one environment and enters another.
- Where an output becomes a decision.
- Where experimentation becomes production.
- Where an employee starts using an unapproved tool.
- Where a prompt becomes an automated workflow.
- Where an agent acquires permission to act.
- Where somebody assumes something has been checked.
- Where evidence needs to survive the output that used it.
- Where the person who built the process leaves.
Crucially, they appear where a platform, model, regulation or business process changes underneath everything built on top of it, as I discovered with the Cowork debacle.
These are not theoretical questions.
- Who decides?
- Where does human judgement have to remain human judgement?
- What has to be checked?
- What evidence has to be retained?
- What happens when the AI output is wrong?
- What happens when the tool changes?
- What happens when the route that worked yesterday doesn’t work tomorrow?
Those questions are much less glamorous than another demonstration of what the latest model can produce in thirty seconds, or how much money you can make every month by letting agents do the work or your thinking for you.
They are also the questions that determine whether the resulting system survives contact with reality. This is where the shovel, the map, the manual and the rivet align.
The shovel gives you capability.
The map shows you the topology: what connects to what across your own environment, so that when something upstream moves you know what else it touches.
The manual records what is supposed to happen, what isn’t, who decides and what happens when something goes wrong.
The rivets reinforce the points where repeated strain can tear the whole thing apart.
Capability was never the issue. A better shovel doesn’t tell you whether you’re digging in the right place. It doesn’t tell everybody else where you’ve already been, it doesn’t tell you whether the ground has moved underneath you, and it certainly doesn’t reinforce the seam when the load starts pulling in two directions.
The AI landscape is moving far too quickly to build an operating method around one product. Platforms will change, models already change regularly, agents have changed at least three times since I started writing this (not really but you get my point).
Most importantly, regulation will change, businesses will have to change to accommodate the changes, and the answer cannot be to keep rebuilding the business around whichever new shovel was launched last Tuesday.
Jacob Davis solved a much less fashionable problem. He worked out where the thing kept tearing, then he reinforced it.
Gold attracted the rush. Copper made the clothes last.
Maybe the next phase of AI adoption isn’t about acquiring more capability at all.
Maybe it’s about finding the seams before they rip.
If you are confident that your AI jeans will hold up under pressure, you are doing better than the majority of people using it. If you are a little bit unsure contact me for a quick health check.
References
[1] Samantha Maeer, “The Shovels Have Changed, The Digging Has Not: Prompt Engineering to AI Critical Thinking,” contain.digital, 5 September 2026. https://contain.digital/blog/the-shovels-have-changed-the-digging-has-not/
[2] Jacob W. Davis, “Improvement in Fastening Pocket-Openings,” US Patent 139,121, 20 May 1873. https://patents.google.com/patent/US139121A/en
[3] IBM, “What Is Shadow AI?,” IBM Think, n.d. https://www.ibm.com/think/topics/shadow-ai
[4] Microsoft and LinkedIn, “AI at Work Is Here. Now Comes the Hard Part,” 2024 Work Trend Index Annual Report, 8 May 2024. https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part
[5] National Cyber Security Centre, “Managing the Cyber Risk of Agentic AI,” 20 August 2026. https://www.ncsc.gov.uk/blogs/managing-the-cyber-risk-of-agentic-ai