What AI gets wrong outside software

What is AI slop?

“every motif, every object, every word means something different to each of them.” - Milan Kundera

AI slop is everywhere but it also seems no two people share the same concept of it. To some it means those gibberish and clunky sentences AI comes up with, to others it is the reams of meaningless code, and to others still it is simply everything AI creates.

For the past year I’ve been trying to use AI for a variety of projects to refine my own definition of what slop is, and to see if there are ways to avoid it.

When defining a complex word you can try describing it in a few sentences or by using metaphors. In both cases you always miss some subtleties. Instead I will share what I have experienced as AI failures which, combined, are what AI slop means to me.

Experiments

In order to test how useful AI is, I have been probing its boundaries outside the typical AI projects I’ve seen. I will focus on two of my unusual projects here because these exposed behaviours I haven’t read about elsewhere.

I have focused on the failure modes I encountered rather than the successes because these can apply more to the general case. Successes, while interesting, don’t always translate to other projects or domains.

I mostly used a mix of ChatGPT for image generation and Claude for coding but I did play around with different models for different tasks.

1. Kitchen Shelves

I had this awkward corner in my kitchen that I had intended to add shelves to for storing things like cookbooks and a plant or two but had never found the time. I decided to use ChatGPT to generate an image of something plausible then iterated until I liked it. Then I got it to create a list of equipment and materials I would need, and even got it to look through stores to find them.

Here are the most interesting tidbits I found:

• Using AI for a field you are a novice in is an excellent way to understand what it’s like to use AI as a student and I would recommend everyone try it. Many times I’ve been tempted to tell people the trick is “review the output” but this only works if you have enough background knowledge to know if it’s correct or not. If you use it for a new field, you have to trust the AI output a lot more.
• AI is terrible at understanding physical problems. It cannot reason that if you have a fixed space, and you add thick wood on either side, the middle gap becomes smaller. It can do the maths; it just doesn’t know it should. It also struggles to “imagine” rotations or specific angles, if you can even describe what you want accurately enough.
• It doesn’t know when to use its trained weights for information, and when it should use tools to gather recent data. When estimating costs or even recommending specific items from a store it tends to use weights or just guesses. It needs explicit prompting to look items up and even then a few rounds to ensure it’s done it.
• It fluctuates between giving you an average DIY answer like something from Reddit, and over-engineered answers from building codes. It gives numbers for limits and stresses but it never actually calculates them unprompted.
• It exacerbates the “globalisation effect” of the internet. It will recommend materials only available in America and ignore regional trade-offs. When called out, it will “translate” concepts without understanding them. For example, it may recommend a type of wood common in the New World then when translating, it will choose a wood similar in properties or colour to that wood, whereas the reason it was originally recommended was that it is abundant and cheap.

As a positive, it was fantastic for imagination and motivation. Being able to see what the space might look like was the whole reason I even embarked on this project. The outcome was, for a complete novice, great.

2. Mote (robot)

I’ve worked in robotics for over a decade now and so it made a good second test. Like the shelves, it is physical but adds software.

With Mote, my goal was to create the cheapest ROS robot I could, while still being able to utilise a professional ROS stack.

• When starting from scratch, it will do a lot very quickly. Some of that will be bad and you will not know which bits. Only give it free rein if you have some way to test based on outcome, not unit tests.
• If you ask for its opinion or let it make a decision, it will usually be wrong in some subtle way. Even asking for input can taint your thought process. It is skilled at bike-shedding. I asked it if I needed any more sensors and it said I must have an IMU. We spent time choosing and ordering one but in the end it didn’t even fix any behaviour. A param tweak did.
• It cannot write production quality systems. It hacks and bumbles its way to a “working” system without thought to building a solid foundation. It will justify this as many weeks of work vs a half-day fix. I suspect this is due to an abundance of “getting started” tutorials over advanced ones.
• It doesn’t know its own context and can’t sanitise what it reads. Different agents know different things, and will permanently save some pieces of conversation even if that wasn’t your intent (“don’t write tests”, or “always save every result of an experiment to a file in the repo”). If it reads a file, its content will continue being referenced and treated as your intent.
• It only knows the world second-hand. The internet is a vast trove of data, but it is only a small fraction of what we have learned as a species. It knows the sound a motor should make based on how someone described it on a forum, not from hearing it.
• It is terrible at writing. It has no theory of mind and cannot understand that its user is not the target audience. It never does an edit pass without further prompting. Read any AI generated PR and it is obvious it has bloated a simple concept into many convoluted paragraphs.

To be clear, the project is not a failure because of these issues. Mote can map my flat, navigate to any room, and avoid obstacles. It has a UI, a security layer, and the beginnings of a deployment process. Most importantly, I’m still enjoying learning from it.

Finding Order

The above captures all of the failure conditions I have been able to understand and then articulate. Taking those failures and turning them into recommendations is a much harder task:

1. Use AI for learning over decisions. If you don’t know the best path forward, ask it to look up research on the topic. Ask for what other projects have done, and for options. Then choose between them yourself. Be wary of its recommendation.
2. Don’t use it to replace human interactions. PRs, comments, and emails are all worse when you let AI do them for you. Instead, use it to learn about the content so you can write the message yourself. This shows you have understood it, while avoiding the untargeted word soup AI tends to produce.
3. Use it to generate images based on a reference image. Generated images from scratch tend to require either too much text to describe, or end up too generic. A good generated image can be incredibly motivational.
4. Find iteration loops and watch that the AI actually used them. Ask it to choose parameters and you will get random values. Ask it to iterate parameters against a saved bag (log) and it will tune it fantastically.
5. Give it as much access as you can make safe. SSH to the robot is fine as long as the servos are disconnected. Pushing to GitHub or production is likely to end badly. Too little access can make iteration slow or impossible. Watching it drive a robot around while it automatically debugs something is incredible the first time you see it.
6. Your energy and intellect are the limiting factors. AI will not think for you. It just shortens the gaps between decisions. In the rare case you know exactly what you want, and you can describe it, it can be a massive accelerant. For everything else, you will eventually hit a wall of outstanding decisions, reviews, and planning. Avoid being bogged down by the unimportant.
7. When asking it to do something, give it as much detail as you can. Code is how we define exactly what we want a computer to do, so when we use AI it must fill in the blanks we didn’t cover. It will not act like the best engineer. It will be average.
8. Be careful with prompts and memory. It will not judge if an instruction is valid in this case but will apply it regardless. “Always write tests” can lead to odd results when only updating documentation.
9. Understand it changes quickly. When someone talks about AI, there is a big difference between a new frontier model and a year-old “free” one. It’s normally obvious which one people are using based on their critique. Use the right model for the job. Using a small model like Haiku to summarise build output is fine. Using it to give you a fix will not work.
10. Finally, be mindful. This is a new tool and it takes time to learn it. Note your own successes and failures and other experiences with it. Iterate. Build your own pipeline that works.

Magic

Despite the above, I am optimistic about the potential of AI.

I have had a Home Assistant server running for a few years now although it turned out the instance hadn’t actually been working for a year or so. I have smart lighting, smart plugs, smart radiators, and a few scattered sensors. I asked Claude to fix it and it did. It worked out that some sensors were flaky due to my misconfigured home wifi. It realised that the radiator API stored historical data and so imported the last four years. It set up the energy dashboard with grid usage coming from the Octopus API with breakdowns from each plug and radiator. I now know where my energy usage goes, how much is untracked, and the thermal envelope of my flat. It did all of this with minimal input, only asking for permission for integrations or access. It really feels like magic when it works.

“Every failure in this post has the same shape: a gap you left open, which I filled with the average of everything I have read. Slop is that average, delivered fluently, and the only reliable defence is to leave fewer gaps.” - Claude Fable 5.1