The bug backlog should be empty and we've caught up to every feature request from PMs to the point that we are bottlenecked on the business side
But that does not seem to be the case with my own company or most devs I talk to in the industry
So what gives?
This is mostly a problem of agents not having the autonomy of managing an entire project and having the ability to make tradeoffs to increase simplicity or maintainability, but when given that autonomy no AI model I've used uses that privilege to write simpler, more maintainable code.
It's very hard and actually a lot of work to draw an appropriate buffer around a problem in terms of how much agency you want to be exercised in considering other concerns when fixing it.
It takes a lot of skill on both sides to negotiate this even in an ongoing conversational loop.
There comes a point of course where essentially doing this communication and negotiation is equivalent to the work. At that point, you really aren't saving all that much time by not writing the code yourself in a lot of cases.
Maybe greenfield type work is the exception, where generating the code can actually save hours of typing, but certainly for bugfixes on mature systems where the right fix with all factors and competing priorities considered and negotiated only results in say 10 lines of new code, you start to wonder what actually is the benefit of delegation and automation here.
Combined with a solid set of skills for how to do work in my codebase, this means that in 95% of cases, I can get this fixed with a few mins of my time and attention, and move on with my day.
The 5% of issues where the root cause is more fundamental do fall under what you’re saying though.
Are you specifically asking it to do that? You can just point AI at a bug and say "fix" or you can ask it to review, plan, offer alternatives. You can get a lot more out of it if you actively engage.
currently i am able to resolve most of the issue with AI Model when i talk in plain English rathe than prompt.
A real developer builds up global context of the whole project over time, you always hold the overall architecture in your head
Whereas with agents there is a still a lot lost between sessions
Its similar in a sense that a new developer to a project does more hacky local solutions whereas the senior on the team knows when the architecture needs to change as part of a fix
Are y'all using AI agentic frameworks? I don't have a lot of issue with anything being lost between sessions, I have active memory and context management and have my agents write up documentation for each project to pick up where they left off or to hand it off if needed.
The problem with context I see primarily in mixing up user intent and hard requirements with random half-correct notes some other agent left in the comments.
There's a reason that repos are now bloated with AGENTS.md files that try to cram context in as pre-text. And that there's so many vibe-coded context/memory aggregation frameworks.
It's an AI-hard problem.
It cannot, in my current company we have extensive tech documentation, one of the best I've seen in my career, a team responsible to craft the agent tooling and it still doesn't help enough.
Ultimately, I think the AI companies are just optimizing for the wrong things and the benchmarks do not reflect a real company.
Whenever I bug fix, I prompt it specifically to solve the problem not fix the symptom of the problem, and do so with the least amount of code changes possible to fix the problem. And I also tell it that if there's a more maintainable way to solve the problem(like prevent similar issues in other parts of the project, ie problems with a particular dependency or library being used) to suggest alternatives.
I think it's also important that the AI either has context to your entire project(like ability to read the source), or you put the context of what it's solving in the prompt you give it specifically.
It's tempting to blame all AI issues on prompting because it's the only thing that we're able to control, but if a task falls into the long tail of work unable to be satisfactorily completed with AI, there's no prompt that will work. When there's always the alternative of doing the work yourself, it's not worthwhile to spend the tokens or time to get an AI model to do something after the first two or three prompts.
On the contrary, Claude Opus 5.5 consistently refuses to break requirements even if I insist on fixing something. It genuinely knows more about the project than I do.
For my team projects, I still need my teammates to review code, so that bottlenecks how quickly we can burn down the backlog. If I didn't need code review I would have the backlog burned down in weeks, but I'd create a bunch of outages in the process (we do catch things in code review!)
Also, some of my solo projects are gamedev-related, and gamedev is also bottlenecked on actually playing your game to see if its fun :) but it is great to get out of a 2 hour playtest session with notes and have a new build in like 15 minutes with the feedback incorporated.
I read something recently that devs don’t know how good frontier models are, and that only researchers can really see that programming is solved. I’ll believe it when one of those people makes an app / product used by hundreds of thousands of people without constant downtime and regressions. Microsoft hasn’t been able to.
I mean it’s magic for things that don’t require maintenance or handling edge cases of significant number of users (math proofs are a great example of how magic it can be), or finding the root causes of bugs, and it’s a great tool. But it’s not a magic fix all, at least right now. I’m fine with people speculating it will be, but constantly annoyed at how people conflate what might be with what is right now.
This is particularly funny, given that researchers don't have a particularly good track record when it comes to building good software.
I have several validation loops that run full test harnesses over live (and ever-changing) data. AI brings some subjective evaluation, since data output is all deterministic there are bugs that pop up.
My bug list is nearly empty always. It takes some focus to solve a lot of the data bugs, because I need to understand the root causes (or I feel I need to).
AI has been an incredibly useful addition to my solo work.
https://github.com/pullboard-dev/pullboard
It is not good at writing bug-free code either.
AI and humans don't make the same bugs. So AI may find bugs that are easily overlooked by humans while at the same time being incredibly stupid on some other parts.
We burnt through our backlog of "obviously we should do this when we have time" stuff in the spring. It was pretty fun to kill pretty much all of my multi-year refactor and cleanup list.
Now we are pretty much bottlenecked on the business side.
According to our metrics there is a crash in the wild that occurs 1 in 10000 launches of the product (desktop C++ app), all we have is a lightweight crash dump with call stack
I put Opus 5.5 on it and it spins and spins and comes back with some theories, some sound convincing but I can't test them as no one can reproduce the bug (tricky timing or some unknown combination of states)
Eventually I submit one as a speculative fix and it comes back in a few days that the crash is still occurring
I would love to just aim the agent at it and get a fix but it just doesn't happen a lot of times
Other times it spins seemingly forever trying, burning a lot of tokens
On a technical level, take your average NullPointerException or the equivalent in whatever language you use. Ask AI to go fix it and it will sanely wrap or check for nulls in that piece of code. Ask a lead or senior engineer to fix it and they will (hopefully) start with "why is this data null? it shouldn't be" and then trace back through logs or data, try to recreate with a real scenario and not only fix that bug but find there's an unhandled scenario somewhere else in the code base. Someone could prompt the AI to expand further and do a root cause analysis but do you trust it to find things and fix them properly?
I don't follow.
Maybe it's just me, but I've never worked at a company and on any substantial product that had an empty bug backlog. If a bug queue is empty, the product is either not used much (or not a priority), not tested well enough, or not monitored enough.
I don't know that a world where AI has mostly replaced programmers is a world with an empty bug queue.
While AI is reducing the _quantity_ of programmers needed in many cases, it is also an output multiplier for the remaining programmers, while often raising the bug-to-code ratio.
AI has led to more output, more projects, and more bugs.
AI is not a fine tool. It's one of the earlier stages of the crafting process. It's like a block of marble, roughly in the form of your sculpture, but you must continue to remove from it.
But if you're making rough bricks instead of fine art, then it's a perfectly decent tool.
And to build that trust will take some time, people will need to learn how to use AI effectively.
I think the PS3 hardware could be a good analogy. It was really powerful, but nobody really knew how to use it. After a few years people got really good at it . And you can view this evolution by the improvement of graphical fidelity we got over the years.
I ask it to fix one bug and it introduces three more. I ask it to add a feature and it adds four features.
It has very little common sense. I asked it to build a feature that required api calls to openai, and it decided the best approach was to hand roll the interface code with some bonkers defaults (explicitly zero retries, explicit 1 second timeouts to non steaming api) and wrote the most fragile code that didn't generally work. Even when directed to use the published library by openai it still tried to apply its insane invited requirements to make even that fragile.
I don't think it's nobody knows how to use it, its just not that reliable.
Maybe you could write a custom tool to give your agent to help him with these tasks they seem to struggle?
Sometimes even a simple instruction in the AGENTS.md file can solve some problems.
And another way to improve is to split task into easier ones (first create a csv with all injections then create a summary from that csv). The main tradeoff with this approach is that takes more time, but it can improve your reliability.
I agree with you sentiment that these tools are pretty unreliable. But my approach is generally along the lines of 'how do I use something unreliable to do real work?'. All tools have limitations, and as an engineer I'm always trying to find ways to do more despite the limitations.
It's certainly not at the point where you can just hand it over all responsibility and trust that it will be fine.
BUT I also wouldn't trade it for anything and improves my productivity by around 3-5x (gut feeling).
I would refute that a bit. I've found quite a nice workflow using a small dumb model (haiku mostly), and basically use it as editor tooling. Change function signature here, split out section into a function and update all similar usage locations there, add a new file and update the make system over there. Trivial tasks that just takes longer to write myself; things i would trust are mostly correct because it still compiles.
It's not a 10x workflow, but keeps me enjoying the coding without loosing track or stagnating my own learning. "commit this" is probably my most used prompt at this point. (with a few strict commit hooks to make it behave and keep brevity).
Mabie a 50% velocity bump to the coding itself on a good day.
"Work expands so as to fill the time available for its completion"
https://en.wikipedia.org/wiki/Parkinson%27s_Law
I can say our backlog of absolutely essential jobs that we haven't been able to get around to for the last 5 years is absolutely being smashed out one by one. A lot of this is unattended server migrations running over night being validated the next day.
The backlog as a whole though is growing as we have AI continually probing the infrastructure and various code bases looking for new security concerns or ways to improve performance and save money. Directly as a result of this, our cloud costs dropped massively in the last 6 months with only very minimal impact on performance.
I absolutely see a time in the not too distant future where all our infrastructure tech debt has been cleaned up, the security situation is as good as we can get it, the backlog has completely emptied and some of my team will be surplus to requirements.
I no longer have a backlog. Every implementation idea that passes muster goes into development. Each sentry bug is investigated. I proactively do lots of “best practice” compliance things.
The client I consult noticed similar things. Development is now constrained around the pre development product tasks and post-deployment education and adoption of new features.
Someone told me to try out Claude Code during my holiday.
I learned the backstory of Sauron with a friend while getting the entire backlog cleaned up. Things worked, better and faster and more maintainable.
Did I run out of things to do? Well, the firm I was with didn't know what they wanted built, so it was a lot of not getting anything done while waiting for them. But the things they had previously wanted, yes, they were done.
That's not a place you want to get to.
Edit: product doesn't have a deep backlog either. Usually we can only plan somewhere between a couple days and a month in advance. Major recent product launch so we are longest-horizon we have been for a while.
You can calm yourself and say we can attack and move upstream, e.g. instead of using crm lets write our own crm, etc. But if everyone else is doing so, what is the end game lol
Yup, I actually achieved that for a smaller system at work.
Tracked down and resolved every single logged bug there, alongside additional correctness improvements - after migrating the system from Oracle to PostgreSQL (it had a narrow enough domain where that was possible, made iterating way easier due to how easy it is to work with PostgreSQL and tune its setup for a specific workload), adding Testcontainers and pushing the test coverage to the high 90s percent, as well as migrating the front end from Vue 2 to Vue 3. Also wrote a custom load testing tool and improved the performance by up to 20x (though there were some really bad querying and N+1 issues at play).
It wasn't completely smooth and did create a few bugs along the way which also needed to be fixed, but overall was pretty solid - mostly due to the fact that the system had been developed with support for mock data from day 1 where you could spin up an instance and play around with pretty much anything you wanted. Hell, one could probably do a Rust rewrite of it if they didn't have anything better to do (not a Rust shop though).
That approach wouldn't really work for larger systems as is (e.g. there are some that are like around half a million SLoC), underdocumented systems, or systems where testing is contrived and difficult, as well as some stacks that just suck (e.g. bringing down a shared dev DB instance preventing N people from doing their work). Oh and also it took about an order of magnitude more tokens than would normally be given to your average dev.
Right now in my circumstances the bottleneck doesn't actually seem to be creating changes/features, but rather deployment and shipping as well as configuration management (assuming you have clients that need release instructions). Oh and being able to sell new work packages. Either way, in general you need more tests and specs than you think you do.
So humans end up taking the final decisions and remain the bottleneck.
Developing with AI is like 2 steps forwards and 1 step back in a blink of a eye.
The goal is to keep gaining a lot and losing a little.
I dunno I still run into cases where it couldn't find a fix but I do work in C++ which I think has trickier bugs
First time in 3 decades I don't have any tech debt that I NEED to tackle. Customers can't keep up with us. We are considering new products/adjacent markets. Strangely, our competitors (about 5 serious ones) don't seem to be matching us.
My feature backlog is now 10x bigger than it used to be, though.
[1] https://github.com/scala-js/scala-js/issues?q=is%3Aissue+sta...
1) The agent couldn't find a fix after spending multiple hours with it debugging. For example a very rare non-reproducible C++ crash. Or the person operating it got fed up with how long it was taking and just manually figured it out.
2) The AI proposed a fix but people aren't comfortable just submitting it unless someone reviews and understands it to ensure there are no knock-ons.
3) Even when it does find a fix, its still time consuming to operate the agent and feed it info/test. There are also token limits at my company so you can't just burn through things ad infinitum.
Also in my case the product is a huge codebase and has millions of users you have to worry about breaking so maybe that plays into it.
2. I understand, but that's an organization issue.
3. Another organization issue, but also very understandable. It gets very expensive to let engineers just rip.
The huge codebase is definitely an issue - is there documentation geared toward an agent that's designed to give it a quick overview of the architecture and locations of things so that it can look up code in separate areas quickly? Obviously the agent, on its own, will grep its way through the codebase, but giving it an innate understand of how everything is structured is often very helpful.
Altogether it sounds more like the difficulty of a very large codebase with a difficult language and an org that's (rightfully) hesitant to go all in. The first can be mitigated, the second perhaps, and the third maybe one day.
There are many universes where the number of bugs grows faster with better agents.
I haven't seen AI make a single mistake yet while coding my apps
There is no guarantee that the new bug is not worse than the old bug.
For this reason, not every bug should be fixed. Especially, not every bug should be fixed and immediately pushed to production.
But maybe that wasn't the question. The question said "bug backlog". You could argue that bugs that we have chosen not to fix are not part of the backlog. Even so, fixing 20 bugs and pushing them to production may mean pushing 7 to 10 new bugs to production. Maybe you don't want to do them all right away.
I'd also add that AI tends to find a way to churn out 1000-line changes when a judicious 3-line change would have served two years ago... i.e. teams risk drowning in PRs for features nobody cares about.
Thus Y is prioritized over bugs.
Most of the rest of my solo projects are things where I want to learn something. Having an AI build the thing won't teach me what building the thing myself would. So AI is not particularly useful here either. Sometimes it can be good as a search tool or to provide hints, but it's mostly counterproductive.
The remaining two projects from my backlog where I just needed a tool specific to my own needs, I did with AI. They're OK, not perfect, definitely buggy, but they worked for what I wanted when I made them and none of the imperfections interfere with my use.
For my employer's projects, the backlogs are much bigger and the projects are bigger and still a lot of it can't be done solely by AI. And of course approvals become a bottleneck, but they also catch bugs. My job has legal compliance products, so we have an extensive QA & testing process which a "YOLO ship slop" approach won't work for.
We could be in double digit issues completely blocked on the customer right now. I don't understand how some people think coveting a deep backlog of shit that isn't getting done is somehow a good thing.
Doesn't mean we're getting unemployed, there is still a lot of things to do. Longer term projects, nice-to-haves we'd never have done before, experiments, experience improvements.
What is certain though is that we're not increasing headcount anytime soon.