Simon Willison recently revisited a timeless lesson: counting lines of code is a terrible way to measure progress. Conceptual integrity—the coherence of the whole system—matters far more for speed, safety, and long-term impact (source).
Fred Brooks called conceptual integrity the “most important consideration in system design.” In plain English: fewer, clearer concepts beat more code every time (The Mythical Man-Month).
The problem with counting lines of code
- Rewards bloat and punishes refactoring and reuse.
- Incentivizes copy-paste over solid abstractions and libraries.
- Discourages AI-assisted improvements that often shrink code.
- Tracks motion, not outcomes—quality, reliability, and user value stay invisible.
What to measure instead (for AI + software)
- Flow and reliability: lead time, deployment frequency, change failure rate, and MTTR—the DORA metrics (dora.dev).
- Model/feature outcomes: task success rate, evaluation harness pass rate, data quality thresholds, hallucination/error rates.
- User and business impact: activation, retention, task completion, NPS/CSAT for AI features, revenue or cost-to-serve effects.
- Engineering health: time-in-review, test coverage on critical paths, incident rate, escaped defect rate.
Practices that protect conceptual integrity
- Write a short set of product and architecture principles—review PRs against them.
- Draw a one-page system context and data flow diagram; keep it current.
- Define module and team boundaries; own contracts and interfaces, not files.
- Use ADRs for irreversible decisions; delete them when superseded.
- Standardize prompt patterns and eval harnesses for AI features; version prompts like code.
- Establish data contracts (schemas, SLAs); break the build on drift.
- Refactor aggressively—celebrate PRs that remove code without losing capability.
- Adopt a “concept budget”: every new concept must justify itself or replace two older ones.
A quick example
Replacing 4,000 lines of one-off LLM glue with a 400-line library and shared evals can look like “negative productivity” if you count LOC. In practice, throughput rises, bugs fall, and new features ship faster—because the system has fewer, clearer ideas.
Takeaway
Stop grading output by volume. Optimize for conceptual integrity and measure flow, reliability, and outcomes. Your roadmap—and your users—will thank you.
Get weekly, no-fluff insights like this. Subscribe to The AI Nuggets newsletter.
