The Fallacy of "Anyone Can Build an App with AI": A Short Story
Updated: Aug 19
Bob works in accounting. Bob loves his job, but Bob is a little frustrated. See, Bob has been asking IT for an app to "streamline his workflow." It's a simple app, he thinks, but unfortunately IT has been very busy with higher-priority projects, so Bob has to wait.
Recently, Bob's company started allowing users to access Claude. As Bob plays around with Claude, he has an idea: Why not use Claude to make my small app?
Bob and Claude
Bob starts to chat with Claude. He provides his requirements, uploads some data, and tells it what the app should look like. As Claude starts to create the app, Bob starts to smile. The smile gets bigger and bigger as Bob sees Claude create a working app in minutes.
"Wow," says Bob. "This is great."
Bob tinkers with Claude a bit more, asking it to rework this and move that.
After an hour, Bob is very happy. Bob has a working app that, on the surface, does exactly what he wants. Bob is excited and wants to tell his boss about it, so he calls him over.
"Look! It can do this, and it can do that. Now watch, when I toggle this, it automatically calculates this and estimates that," says Bob excitedly.
"Oh, this looks great, Bob. And you did this on your own?" asks his boss.
"Yes! I used Claude to create it and it's working so well that we can cancel that IT ticket and push this out to others to use," replied Bob.
"Sounds like a plan. Let's do it," said his boss.
Bob is very happy. Bob is very excited. It is a great feeling to see your first app work the way you want it to, but unbeknownst to him, Bob's app relies on polyfill.
The Hidden Danger Surfaces
Over the course of the week, Bob's app is pushed out to other people in accounting. Everyone loves it.
Weeks go by and everything is working great. Then suddenly, Sally desperately calls Bob to come over.
"Bob, I just tried opening your app and I got a message saying an adult site was blocked!"
"WHAT?!" yelled Bob.
"Bob, get over here! Your app just loaded a gambling site!" screamed Tim from the other side of the office.
Bob is in a panic. Why is his app doing this? It has been working perfectly for weeks! Bob immediately calls IT.
IT looks at their logs and starts to see that a bunch of accounting users are being blocked from malicious or bad websites. IT tells all of accounting to stop using the app until they figure out what's going on.
IT asks Bob what components his app uses. Are there any CDNs it points to? What framework does it use? What libraries does it load?
This is all foreign to Bob.
Bob doesn't know because Bob didn't create the app, Claude did. Bob is not a developer. He has no experience in software development, so even if he were to look at the code Claude wrote, he wouldn't understand it.
Bob is anxious. His beautiful app is sending people to bad sites. He starts to worry: Could it also be doing other things it's not intended to do? How could this have happened?
Bob's app got hit by the Polyfill.io supply chain attack.
What is Polyfill.io and How Did Bob's App Get Infected?
Polyfill is like a translator that allows older browsers to understand newer code. It was a very popular tool. Because Bob is in a corporate environment, Claude included a link to the website polyfill.io to download this "translator" so Bob's app could work on outdated versions of Microsoft Edge.
In February 2024, a Chinese company named Funnull acquired the polyfill.io domain. Shortly thereafter, they added malicious code to redirect users to unwanted websites.
Since Bob's app linked to polyfill.io to pull the code it needed, it was impacted by this malicious change.
Bob didn't know about Polyfill. Bob had never heard of Polyfill. And because Bob is in accounting and not IT, he probably wasn't even aware of this supply chain attack.
Fortunately for Bob, his app only redirected users to bad websites, but it could have easily injected malicious code into their browsers, stolen their credentials, or worse. The scariest part? Had the app not redirected users to those obvious bad websites, no one would have ever known.
Even if Bob happened to read about the Polyfill.io attack, he would not have known his app was using it because Bob didn't create the app, Claude did. Bob didn't review the code, Claude did. Bob just sat back and let Claude do all the work.
The Fallacy of "Anyone Can Build an App with AI"
When used as a tool by experienced, professional developers, AI is an incredible asset that allows them to be more efficient, from writing code to debugging complex issues. A professional developer will not blindly use AI-generated code; they will review and understand exactly what it does, what it references, and what it requires.
When used by someone with no development experience to create an app, an AI-built app can be a ticking time bomb waiting to explode.
When the Polyfill.io attack happened, it was such a massive risk that Google started flagging sites referencing polyfill.io as compromised. Even years after the attack made headlines, unmonitored sites across the web are still unknowingly running this dangerous code.
So the question isn't "Can anyone build an app with AI?"
The real question is: "Should anyone be allowed to build an app with AI?"
For any business that cares about security and data protection, the answer should be a resounding no.
The fallacy that "anyone can build an app" only exposes your business to hidden vulnerabilities, and you might not even know you've been breached until it's too late.
More In-Depth Analysis of the Polyfil.io Attack
This video by the SecureTechIn YouTube channel provides a detailed outline of how this attack worked, what it did, and what can be done to minimize the risk.