Understanding the Cost-Benefit Analysis of AI in App Creation

How CodoraTech is funded: CodoraTech is supported by advertising and, in some articles, by affiliate links. Where an article contains affiliate links, we say so at the top of that article.

The honest starting point for a cost-benefit analysis of AI in application development is that the public evidence is thin. Nothing in the research assembled here measures what AI assistance does to the cost, schedule or defect rate of building an app. What exists is broader and adjacent: one large field study of AI at work across a national labour market, one survey of what low-quality AI output costs its recipients, one vendor survey of how its own users feel, and a small academic literature measuring model output on a different task. A buyer weighing a licence must reason from those, and should know it.

What the evidence covers

• No study cited here measures AI’s effect on application development specifically
• The largest field measurement, across about 25,000 Danish workers, found 2.8 percent of work hours saved
• The same study found no significant effect on earnings or recorded hours, ruling out effects above 1 percent
• BetterUp Labs and Stanford put low-quality AI output at nearly two hours of rework per instance
• Microsoft’s 2026 figures are self-reported perceptions, collected by the vendor from AI users only

The strongest field number comes from outside software

Anders Humlum of the University of Chicago Booth School of Business and Emilie Vestergaard of the University of Copenhagen published “Large Language Models, Small Labor Market Effects” as NBER Working Paper 33777 in May 2025. They surveyed roughly 25,000 workers at 7,000 Danish workplaces across 11 occupations in two rounds, in late 2023 and 2024, and linked the responses to administrative labour-market records running through June 2024. That design is unusually strong for this subject: it observes real payroll and hours data rather than asking people to estimate their own gains in isolation.

Users did report savings. Between 64 and 90 percent of them did so, in every exposed occupation. The average saving was 2.8 percent of total work hours. But the administrative records showed no significant impact on earnings or recorded hours in any occupation, with confidence intervals ruling out effects larger than 1 percent, and only 3 to 7 percent of the reported savings passed through into higher pay. The tools also generated new tasks for 8.4 percent of workers, including some who never used them.

Two cautions attach to importing that figure into a software budget. The paper does not state that software development was among the 11 occupations studied, so treating 2.8 percent as a developer-specific result is an assumption, not a finding. And the authors explicitly contrast their field result with the 15 to 50 percent gains reported in controlled experiments. If a vendor quotes a number in that upper range, the relevant question is whether it came from a laboratory task or from a workplace.

Perception data from a vendor is not an outcome measure

The most widely circulated positive figures come from Microsoft’s 2026 Work Trend Index. Microsoft states the method plainly: an online survey of 20,000 full-time employed or self-employed knowledge workers who already use AI at work, fielded by Edelman Data x Intelligence between 18 February and 7 April 2026, twenty minutes per respondent, across ten markets including Australia, Brazil, France, Germany, India, Italy, Japan, the Netherlands, the United Kingdom and the United States.

The results are striking and should still be read for what they are. Sixty-six percent of AI users report spending more time on high-value work. Fifty-eight percent say they produce work they could not have produced a year earlier. Microsoft also reports that 49 percent of Microsoft 365 Copilot conversations support cognitive work. Every one of those is a self-reported perception, gathered only from people who use the tool, published by the company that sells it. None is a measured output, cost or defect count, and the 2026 report contains no development-specific metric at all. Set against the Danish payroll records, the gap between what users believe they saved and what showed up in the accounts is the central problem in this whole category.

The costs that land on someone else’s desk

A cost-benefit model that counts only licence fees and saved hours misses the largest documented cost, which is downstream cleanup. Research by BetterUp Labs with the Stanford Social Media Lab, published in Harvard Business Review in September 2025, found that 41 percent of workers had received what the authors call workslop: output that looks polished but lacks substance and transfers the real work to the recipient. Each instance cost nearly two hours of rework, which the researchers put at $186 per employee per month in lost productivity. Forty-two percent of recipients trusted the sender less afterwards, over half were annoyed and more than a third were confused. The sample size and field dates were not available in the published material, so those percentages should be used without an implied denominator.

The verification cost has a measured analogue, though it comes from a different task. Kirstein, Kumar, Ruas and Gipp at the University of Gottingen published “Re-FRAME the Meeting Summarization SCOPE” on 14 November 2025. Scoring against the MESA error metric on a five-point scale where lower is better, baseline GPT-4o summaries of real meeting transcripts registered 3 hallucination errors on the QMSum dataset and 4 on FAME, with omission errors of 3 and 4 respectively. Their own pipeline brought both down to 1.

That is meeting summarisation, not code generation, and it must not be read as a defect rate for a coding assistant. It is cited here for a narrower claim that does transfer: a frontier model applied to a defined workplace task produced measurably unfaithful output before mitigation, and the mitigation was itself engineering work someone had to build and maintain. Any model of AI cost in app creation that assumes generated output arrives correct is assuming away the part the research actually measured.

Working the arithmetic with the only numbers available

What follows is a worked example, not a documented deployment. No organisation is being described. It applies published figures drawn from outside software development to a hypothetical ten-person team, purely to show which quantities decide the answer and where the evidence runs out.

Take the Danish field result at face value and apply 2.8 percent of a 40-hour week. That is about 67 minutes per person per week, or roughly 11 hours a week across ten people. Against that, the Harvard Business Review figure of $186 per employee per month in workslop-related losses would be $1,860 a month for the same team, although that estimate is an all-employee average and was not measured on engineers. The third quantity is the licence cost, and no source used here establishes one, so the reader has to supply it. The example is only useful as a shape: two of the three inputs come from populations that are not software teams, and the third is not in the public evidence at all.

The more important caveat is what the same study says about converting hours into money. The Danish records found that the reported savings did not appear as higher earnings or fewer recorded hours, and that only 3 to 7 percent passed through. Valuing saved minutes at a loaded hourly rate is therefore precisely the step the strongest field data does not support. A defensible model treats hours saved and money saved as separate claims requiring separate evidence.

1. Baseline the specific task you expect to change, in hours, before the tool arrives rather than after.
2. Record rework as its own line, since the Harvard Business Review finding puts the largest measured cost on the person receiving the output rather than the person producing it.
3. Track review and verification time explicitly, because the Gottingen results show that unmitigated model output on a defined task needed correction.
4. Test whether saved hours changed anything downstream in schedule or headcount, which is the check the Danish administrative records applied and where the reported gains disappeared.
5. Ask of every vendor figure whether it is an outcome or a perception, and who published it.

Sources: National Bureau of Economic Research · Harvard Business Review · Microsoft WorkLab · arXiv