Is any of you using LLMs to create full features in big enterprise apps?

Let me be clear first. I don't dislike LLMs, I query them, trigger agents to do stuff where I kind of know what the end goal is and to make analisys of small parts of an application.

That said, everytime I give it something a little more complex that do something in a single file script it fails me horribly. Either the code is really bad, or the approach is as bad a someone who doesn't really know what to do or it plains start doing things that I explicitly said not to do in the initial prompt.

I have sometimes asked my LLM fan's coworkers to come and help when that happens and they also are not able to "fix it", but somehow I am the one doing it wrong due "wrong prompt" or "lack of correct context".

I have created a lot of "Agents.md" files, drop files into the context window... Nothing.

When I need to do green field stuff, or PoCs it delivers fast, but then applying it to work inside an existent big application fails.

The only place where I feel as "productive" as I heard from other people is when I do stuff in languages or technologies I don't know at all, but then again, I also don't know if that functional code I get at the end is broken in things I am not aware of.

Are any of you guys really using LLMs to create full features in big enterprise apps?

6 points | by not_that_d 1 day ago

3 comments

  • kasey_junk 9 hours ago
    With agentic coding people underestimate the agent and over estimate the models value. So it’s important to be specific. What agent are you using? You will see radical performance differences between Claude code and codex compare to copilot for instance. You will also see pretty big differences if you have well groomed, agent specific agents files. Especially if the code base is very large, the agents files need to be able to guide the agent to make connections in the code.

    But other than that what I’ve found to be the most important is static tooling. Do you have rules that require tests to be run, do you have linters and code formatters that enforce your standards? Are you using well known tools (build tools, dependency management tools etc) or is that bespoke.

    But the less sexy answer is that no, you can’t drop an agent cold into a big codebase and expect it to perform miracles. You need to build out agentic flows as a process that you iterate and improve on. If you prompt an agent and it gets it wrong, evaluate why and build out the tools so next time it won’t get it wrong. You slowly level up the capabilities of the tool by improving it over time.

    I can’t emphasize enough the difference in agents though. I’ve been doing a lot of ab tests with copilot against other agents and it’s wild how bad it is, even backed with the same models.

  • linesofcode 1 day ago
    The quality of an LLM outputs is greatly dependent on how many guard rails you have setup to keep it on track and heuristics to point it on right direction (type checking + running tests after every change for example).

    What is health of your enterprise code base? If it’s anything like ones I’ve experienced it’s a legacy mess then it’s absolutely understandable that an LLMs output is subpar when taking on larger tasks.

    Also depends on the models and plan you’re on. There is a significant increase in quality when comparing Cursors default model on a free plan vs Opus 4.5 on a maximum Claude plan.

    I think a good exercise is to prohibit yourself from writing any code manually and force yourself to do LLM only, might sound silly but it will develop that skill-set.

    Try Claude code in thinking mode with the some super powers - https://github.com/obra/superpowers

    I routinely make an implementation plan with Claude and then step away for 15 mins while it spins - the results aren’t perfect but fixing that remaining 10% is better than writing 100% of it myself.

    [-]
    • not_that_d 1 day ago
      The code is quite easy to follow to be honest, we have documented a lot of stuff and segmented functionality into libraries that follow an app/feature/models pattern. Almost every service we have, has unit tests explicitly describing what the public api is doing or supposed to do on several scenarios, we never test implementation details.

      Given it to new people of course carry questions, but most of them (juniors) could just follow the code given an entry point for that task, this from BE to FE.

      I use the github copilot premium models available.

      > I routinely make an implementation plan with Claude and then step away for 15 mins while it spins - the results aren’t perfect but fixing that remaining 10% is better than writing 100% of it myself.

      I have to be honest, I just did this two times and the amount of code that needed to be fixed, and the mental overload to find open bugs was much more than just guide the LLM on every step. But this was a couple of months ago.

    • not_that_d 1 day ago
      Besides my other response, it can also be I am not smart enough for it.
  • raw_anon_1111 17 hours ago
    Rule #1 I don’t do agentic coding. I keep my hands on the steering wheel and have it build everything up step by step, validate the code, commit. Repeat.