This is such a big topic I am not sure where to start with, so let’s begin with some initial thoughts
1. AI is smart, but what about the operator?
AI as we all know can do many many things, but it is well known at this point it can also be incredibly dumb, whether that’s a big or small model, and depending on the task you’re doing letting it blindly run will most definitely not at this point in time achieve the right result. That being said, it’s still an incredible tool to learn and build.
Anthropic CEO Dario Amodei says AI will produce a country of geniuses, that certainly is true, but those geniuses are likely locked behind data centers, and these geniuses are controlled by smart and dumb people. So the question becomes, how do you use a country of geniuses? After all, the public’s always had unfiltered knowledge for years on all sorts of things, but those who don’t strive to learn can’t apply the knowledge open to them to produce something tangible/scale-able in the long term.
2. Accelerates good and bad work equally
I know I said AI accelerates good and bad work equally, but maybe it’s not true. I constantly see low-effort code, documents and while it may be good for mvp testing, or in edge case uses that doesn’t need scaling, everywhere else it is incredibly destructive. This accelerated production “garbage” puts pressure on the system.
Imagine a power plant that requires safety contingencies for a variety of situations, previously people took the time to test, ensure requirements were met before turning on the power plant, but now the new designers disregard risk because speed has trumped safety/accuracy, and as long as the power plant can output energy that to them is a job well done. This analogy can be applied to any system, not just software, because after all nothing is spared the touch of AI in this day and age.
3. The review bottle neck
With so many things produced how will we know what is right or wrong? There’s a constant tug of war between getting things done fast with AI to capitalize on its advantage, versus also ensuring your task at hand is properly solved.
On top of this, there’s a lack of review process on the performance of an agent itself. So we have
- Lack of capacity to accurately judge the output of the AI
- Lack of monitoring into how the AI produced the output
There are LLM judges for all sorts of workflow, and that’s likely going to be the future, a constant tug of war between the AI and itself.
Anyhow, a very late blog post on the topic of AI, but nevertheless I think a good generalization of the industry problems. The world is in a different place now whether it be better or for worse, time will only tell.
Thank you for reading. I hope to continue writing by hand without the use of AI to hopefully make these posts a bit more meaningful and soulful to read.
