During the California Gold Rush of 1849, tens of thousands of people poured into the goldfields hoping to strike it rich, most of them did not. The shovel sellers however, made a killing.
Sam Brannan, a San Francisco merchant and newspaper publisher, worked it out fast. Rather than panning, he bought up every mining supply he could find and filled his store with buckets, pans, heavy clothing and provisions. He then walked down the streets waving a bottle of gold flakes, shouting about his wares which he resold at a steep mark-up. Making him California’s first millionaire without swirling a single pan or stepping on a shovel. [1] He was not alone either, Levi Strauss arrived in 1853 to set up a wholesale dry goods business. Merchants, suppliers and service providers repeatedly did better out of the rush than the miners did. [2]
Nearly two centuries later, AI produced its own gold rush, and, right on schedule, the shovel sellers arrived.
In 2023 the hot new skill was “prompt engineering”. There were prompt libraries, marketplaces, courses, frameworks, certifications and consultants promising to teach people the language required to unlock AI.
Anthropic’s much-publicised “Prompt Engineer and Librarian” vacancy offered between $175,000 and $335,000 a year, asking for applicants with “a creative hacker spirit” who “love solving puzzles”. [3] One instructor’s two prompt-engineering courses had already taken around two thousand students, starting at $150 and running up to $3,970 for custom training and certification. [3]
There was a genuine technical issue underneath the excitement. Early generative-AI systems could be extraordinarily sensitive to how instructions were phrased. Prompting mattered, but what was being sold as an entirely new human capability looked suspiciously familiar.
Define what you need, explain the context, set the constraints, specify what good looks like, give examples where necessary. Then review the output, refine the instruction. Rinse and repeat.
To be honest that’s not particularly exotic. In project-management language, much of it is scope and requirements definition followed by iterative improvement.
In management, it is called a briefing. In journalism, commissioning. In writing, it is clarity, most industries have a version of the same thing.
OpenAI’s guidance sounds rather less mystical than much of the prompt-engineering industry that grew around its models. It advises people to identify the task clearly, provide the necessary context, be specific about the desired result and refine requests iteratively. More tellingly, it now says users can rely increasingly on “natural, goal-driven language” rather than perfect phrasing. [4] After several years of increasingly elaborate prompt systems, we appear to have travelled an impressively long road back to giving a decent brief, although some of the Facebook experts seem to have missed the memo, but I digress.
Ethan Mollick of the Wharton School warned in 2023 that it was “not clear that prompt engineering is going to matter long-term because AI programs are getting better at anticipating what users need and generating prompts”. [3] More strikingly, Rob Lennon, the man selling the $3,970 course, said in the same article: “in six months, 50,000 people will be able to do that job. The value of this knowledge is greater today than it will be tomorrow.” [3]
A shovel seller, on the record, telling you the shovel market has a shelf life.
The technology was new. The underlying human skills were not, that was the first shovel market.
The second one seems to have arrived, as suddenly there’s a lot of chat in the AI world about critical thinking.
That is not entirely surprising because the first few years of generative AI gave us a fairly brutal demonstration of what happens when humans stop checking what the machines produce.
NIST uses the term “confabulation” for what most of us call hallucination, defining it as the production of confidently stated but erroneous or false content by which users may be misled or deceived. It sits in NIST’s list of twelve risks that are unique to or made worse by generative AI, which is to say it is treated as an inherent property of the technology rather than a passing defect. [5] The numbers have at times been spectacular.
OpenAI’s own o3 and o4-mini system card reported hallucination rates on its SimpleQA benchmark of 51 per cent for o3 and 79 per cent for o4-mini, against 44 per cent for the older o1. [6] Newer models have improved substantially, but the underlying problem has not vanished.
So we spent several years being told: Check the AI, as it can confidently make things up. Which is to be honest, fair enough.
Then we started being sold systems and training designed to help AI users perform “critical thinking”.
Microsoft Research and Carnegie Mellon University surveyed 319 knowledge workers and collected 936 real examples of people using generative AI at work. Its 2025 study found that greater confidence in GenAI was associated with less critical thinking, while greater confidence in one’s own abilities was associated with more. AI also shifted the human role towards verification, integration and what the researchers called task stewardship. [7]
That should have produced a fairly obvious conclusion: Keep the human intellectually engaged. Instead, a new market has appeared around teaching people how to think critically with AI.
Udemy carries a course literally called Critical Thinking: The Human Skill AI Can’t Replace, promising to help you evaluate AI outputs and make decisions that hold up at work. [8] Coursera carries one from the University of Michigan called Leveraging GenAI to Develop Critical Thinking Skills, added in November 2025, six hours long, with 2,822 already enrolled. Its third module teaches you to build your own Critical Thinking Framework. [9]
There is nothing inherently wrong with any of that, as in the gold rush the miners really did need the shovels. Likewise, people using AI need to understand hallucination, bias, provenance, verification and uncertainty, so yes, training can be useful.
The problem begins when the shovel gets confused with the skill required to know where to dig. Critical thinking existed long before generative AI. As did research, scepticism, cross-referencing, the ability to distinguish a primary source from somebody’s interpretation of it, questioning incentives, funding, omissions, framing and authority, and changing your mind when the evidence changed.
None of those capabilities were invented by AI, and critically, none of them should be surrendered to it. That is where I fundamentally part company with a lot of the current conversation.
AI can help me investigate, find documents, compare versions, search large amounts of material quickly, challenge an argument, identify inconsistencies, test a hypothesis and even point out something I may have missed. All of those things are useful.
What I object to is AI silently deciding the intellectual architecture of the investigation for me. Nor do I want it to decide that one source belongs in a “trusted” category and another does not unless I have explicitly asked it to apply criteria that I understand and have chosen.
I do not want it to invent categories because categorisation happens to make the output look orderly, nor decide which evidence matters before I have decided what I am actually investigating. Put simply I do not want a score standing in for my personal judgement. Moreover, I certainly do not want another automated system determining what I should regard as credible because the first automated system proved unreliable. I have written about what that looks like when a government proposes it here. I have written about what that looks like when a government proposes it here https://www.linkedin.com/pulse/you-watch-space-while-we-samantha-maeer-th3ue/
That is not critical thinking, it is delegation of judgement and the difference is immense.
Microsoft Research has since started talking about designing AI as a tool for thought, rather than simply an assistant that performs cognitive work for us. Its researchers explicitly raise the risk of knowledge workers becoming little more than validators of machine-produced opinions. [10]
That, to me, is far closer to the stance we should be pursuing. AI should extend human capability, it should not quietly replace human direction.
Perhaps that is the real pattern running through the AI gold rush. First we were told that the new essential skill was prompt engineering, a market appeared selling prompts, courses, templates and frameworks.
The models improved and the magic wording became less important.
Now we are being told that the essential skill is critical thinking.
A new market is already emerging selling tailored courses, frameworks, verification systems and methods for doing that. The shovels have changed, as has the pitch, the human capability underneath them has not.
Clarity, research, knowledge, judgement, discernment, direction and often intuition, are not AI skills, they are human ones. AI can support them magnificently, but the minute we start outsourcing them to the machine and calling the result “critical thinking”, we may want to check whether we are still digging for gold or just buying another shovel.
Which is, I am aware, exactly the point at which someone in my position tells you they have built the answer. So let me be precise about what I have not built. Not a course on prompting, a framework for thinking, or a meaningless certificate proving you can use AI.
What I built, after months of expensive proof that the tools alone do not do it, is contAIn™, a methodology layer that directs AI to the outcome you define. Yours, decided by you, before the machine starts. The problem was never the AI. The problem was the missing methodology layer. I know because I spent a long time being the case study.
Government, business and technology have always been rather good at creating new names for old things. The AI industry may simply have industrialised the process.
The gold rush keeps moving, the shovel sellers keep following it, but underneath it all the human still has to decide where to dig.
References
[1] Sam Brannan’s supply business and his role in publicising the discovery are recorded in California Gold Rush histories including the Coloma and Marshall Gold Discovery State Historic Park account, and in PBS’s History in a Nutshell: The California Gold Rush. Brannan bought up mining supplies, stocked his store at Sutter’s Fort, publicised the find in San Francisco and became California’s first millionaire selling to miners. Verified 29 August 2026.
[2] Levi Strauss established his San Francisco wholesale dry goods business in 1853; merchants, suppliers and service providers are widely documented as having profited more reliably than prospectors. Verified 29 August 2026.
[3] TIME, How to Get a Six-Figure Job as an AI Prompt Engineer, Nik Popli, 14 April 2023. Carries the Anthropic salary range and listing wording, Rob Lennon’s course pricing of $150 to $3,970 and approximately 2,000 students, Lennon’s remark on the value of the knowledge declining, and Ethan Mollick’s caution. Verified live 29 August 2026. https://time.com/6272103/ai-prompt-engineer-job/
[4] OpenAI, How do I create a good prompt for an AI model?, OpenAI Help Center, updated August 2026. States that “the models often perform best when interacted with as if you are sending another human a request”, advises identifying the task clearly, providing necessary context, being specific and working iteratively, and says that as ChatGPT grows more intuitive “you can rely more on natural, goal-driven language and less on perfect phrasing”. Verified live 29 August 2026. https://help.openai.com/en/articles/4936848
[5] National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024, section 2. Defines confabulation as “the production of confidently stated but erroneous or false content (known colloquially as ‘hallucinations’ or ‘fabrications’) by which users may be misled or deceived”, one of twelve named risks unique to or exacerbated by generative AI. Verified 29 August 2026. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
[6] OpenAI, OpenAI o3 and o4-mini System Card, 16 April 2025. SimpleQA hallucination rate 0.51 for o3, 0.79 for o4-mini, 0.44 for o1. Verified live 29 August 2026. https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf
[7] Lee, Sarkar, Tankelevitch, Drosos, Rintel, Banks and Wilson, The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers, Microsoft Research and Carnegie Mellon University, Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 319 knowledge workers, 936 first-hand examples. Verified live 29 August 2026. DOI: 10.1145/3706598.3713778
[8] Udemy, Critical Thinking: The Human Skill AI Can’t Replace. Verified live 29 August 2026. https://www.udemy.com/course/critical-thinking-the-human-skill-ai-cant-replace/
[9] University of Michigan via Coursera, Leveraging GenAI to Develop Critical Thinking Skills, instructor John K. Thompson, added November 2025, six hours, 2,822 enrolled at time of checking. Verified live 29 August 2026. https://www.coursera.org/learn/leveraging-genai-to-develop-critical-thinking-skills
[10] Microsoft Research, The Future of AI in Knowledge Work: Tools for Thought at CHI 2025. Verified live 29 August 2026. https://www.microsoft.com/en-us/research/blog/the-future-of-ai-in-knowledge-work-tools-for-thought-at-chi-2025/