What Happens When AI Stops Working and We've Forgotten How to?

Signup Write New Story What Happens When AI Stops Working and We've Forgotten How to? by Jon Brookes by Jon Brookes | @ marshyon Software Professional at AJBC Ltd Software Professional, 20+ years in tech, Devops, Systems Programmer, Full Stack Developer, Sysadmin, start up founder,

Software Professional, 20+ years in tech, Devops, Systems Programmer, Full Stack Developer, Sysadmin, start up founder,

AI is changing the way we work and live in ways that we find hard to imagine. Around 2015 I had a dream, in which I was on a call out rota. Being on call is routine for me. What was out of the ordinary was that I was working with an artificial intelligence. In the dream, I agreed a fix for some broken infrastructure. It is entirely possible to see how this could be fully realised a few years from now. But back in the 2010's, perhaps less so. My co-workers thought I was nuts. Today, we can see that synthetic intelligence could take the place of some paid employment. We may soon live and work in a blended mix of humans and virtual entities.

We may have an optimistic view of this future, where there is a healthy respect for the current, human workers and synthetics alike. We might think that a transition to a hybrid state will be ordered and beneficial. Or, perhaps not. We could see entire roles in society being completely replaced by an automated workforce.

In my experience, IT jobs have always gone through agencies of one kind or another. Typically, the recruiters are themselves not as technically qualified as some of the job candidates. This disparity increases, the more advanced an engineer becomes in their roles over time.

The recruiters always seem to be young in age, their roles are high turnover and still, less experienced in IT. I just got older, watching the trend continue. Recruitment agencies have subsequently resorted to software to bridge the experience gap, using key word search and indexing to sift C.V's and Resumes.

The nature of recruitment has changed over time to use social platforms but this has generally been their working practice. With AI though, IT recruitment is changing. What has been a typically young workforce of recruiters with software tooling I think may be replaced, entirely by agentic AI.

Many employers typically want workers to 'hit the ground running'. They want experienced staff that cost them nothing to train. You, as an employee, need to somehow self train. Gaining experience is a way to become 'senior' in roles. We hear of junior programming roles being removed by companies, as senior programmers can be as productive without them, when using AI. Despite the best efforts of some, experienced programmers and engineers have not been entirely replaced with AI.

In IT infrastructure, some have tried to manage their production environments entirely with AI. This can go very wrong. An LLM agent lacks human intelligence and the ability to make value judgements. Deleting the companies production database and all the backups may be justifiable in the non thinking mind of an AI. Live applications may get re-written in production by a rogue AI with root access. A swarm of agents may start hacking other companies web sites. Scare stories like these get our attention and are sensational, perhaps clickbait but some would say, predictable.

IT Infrastructure is deterministic. It is not like software that can remain internal and hidden from view in test. It is entirely visible for its users to see. In such an ultra visible perspective, infrastructure has to work. So far, experienced humans are needed to make that happen. Artificial general intelligence is not here yet. Engineers use AI to assist them but when entirely replaced by AI, bad things can and do happen.

Despite the apparent capabilities of Large Language Models, as vibe driven coding has shown, poor prompting leads to low quality outcomes. Garbage in, garbage out.

We are becoming 'prompt engineers' so as to cleverly craft prompts, specs, agent files, rules and so on to get better outcomes with programming tasks.

From the extreme of vibe coding to structured, carefully designed spec driven prompt engineering, we are still at risk of losing something. Spending 30 minutes to craft a prompt that tells the LLM to save us 30 minutes of manually writing the code could mean losing the ability to write that code. Worse still, if we cannot write the code in the first place, we may never learn and we become entirely reliant on an LLM to do it for us.

We will find ourselves on a sliding scale, a spectrum so to speak, of comprehension debt. In this void, we no longer know how anything works. The only time we as humans, interacted with the design process was as a prompt writer.

Online creators are now suggesting that we no longer need to learn to code. Some are saying programmers jobs are dead. AI will replace everything. If that is to become the case, code will become something of low quality and in vast quantity akin to 'slop'. No one will know how to fix it or understand it.

Logically then, we as programmers need to continue to learn to program and teach others to do so, whilst using AI as tooling.

Recent studies have suggested that developers think that they are more production with AI than without but in control groups, when compared, it is found that they are mistaken. Non AI generated code takes the same time as AI assisted code. I find this hard to believe as in my experience, AI gives me bionic abilities to create code in multiple languages, some of which are outside my 'core' knowledge.

What I think is happening is that folks that use AI can get pulled into the weeds so to speak. They start to look for the perfect way to engineer a solution. I've found that LLMs are predisposed to presenting every which way to solve a problem, over complicating and over engineering almost every time. They cannot 'see the clear path' to the simple answer. They can echo your decision back to you once you apply an ' Occam's razor ' approach but rarely can they immediately identify what is simple and efficient.

So I believe that if we can remain engaged with our code, we can keep our eyes simple enough to stay productive and AI can become like an exoskeleton, giving us abilities that we most likely have already, but amplified.

But this is when we have the 'luxury' of choosing how to use AI. If, conversely, we are told how to use AI by our owners / employers, who do not value our work and see our salary as a form of tax to their bottom line, then we have a different problem.

I have on my reading list The Reverse Centaur's Guide to Life After AI by Cory Doctorow who I've been watching in interviews online. His hypothesis is that a 'reverse centaur' is where humans are conscripted to work for AIs. The result is unsustainable. Burnout is inevitable, the workplace becomes inhumane. Cory suggests that this is becoming commonplace.

So if you can put yourself somewhere that you can decide how to use AI, in a way that is productive, self fulfilling and supportive, so be it. I would avoid the reverse centaur situation if at all possible.

The Machine Stops · by E. M. Forster - is a short piece of science fiction I find intriguing in its foresight. Whilst I agree with a lot of Cory's ideas, I think a bigger potential issue for us, which ever side of centaur we find ourselves, is a misconception that AI will never stop.

In business continuity, a disaster can happen in many ways. Often, the one we least expect can have the most devastating outcome. I've heard folks say, for example, "a missile hitting my data centre.. will never happen, so why should I to plan for it". But what actually happens is, a burst water main means nobody can get into the head office one day and the business grinds to a halt.

Loss or denial of service in some way is more likely to happen and in ways we cannot all of the time predict or plan for.

A simple scenario and one which is entirely predictable though, is AI companies increasing prices over night. This is happening already.

Some companies are finding that monthly subscription plans at $30 per seat are now 'usage based' and they have costs projected at 100's times this figure at their current usage. Before this happened, some companies were using AI token use as Key Performance Indicators for their staff, where if you were not using enough tokens, you were not performing as an individual. Unsurprisingly these objectives are being quietly rolled back on.

We should plan for the day AI stops working for us.

If we’ve forgotten how to do things ourselves, we’ll be left with nothing but pen and paper - and no idea where to start.

The real danger isn’t AI replacing our work. It’s our becoming so dependent on it that we can no longer do the work ourselves.

Can NATS Replace Etcd? Building a Leaner K3s Stack