16KB
Spec Driven Development — Chapter-04: 1 - Fundamentals: where SDD comes from
- Source: /library/Spec Driven Development/source-file.pdf
- PDF pages: 23–32
- Pages without text: none
1 - Fundamentals: where SDD comes from In Chapter 0 you walked away with one idea: specifying before you ask for the code is what separates the people who build solid things from the people who just paste back answers they don't understand. It makes sense. But you may also have walked away with an uncomfortable suspicion, and it is better to face it head- on: isn't describing everything carefully before building precisely the old way of making software, the heavy, bureaucratic one the world spent thirty years trying to abandon? If you have ever worked on a team, you know the fatigue. A document nobody reads, a meeting to approve a meeting, a giant plan that reality runs over in the first week. Anyone who lived through that learned, rightly, to distrust whoever shows up preaching “let's plan everything up front.” The suspicion is fair, and this chapter meets it head-on. What it will do is separate two things that usually come glued together: the instinct to think before building, which has always had value, and the cost of changing late, which is what actually sank the old model. For that we need to go back in time a little and look, without rushing, at how software was made before you arrived. Not out of nostalgia: you will see that each method was born fixing the mistake of the one before it, and that SDD is the next step in that line, not a return to its beginning.
Waterfall: the right instinct, the wrong cost Imagine building a house. Nobody hands bricks to the bricklayer and says start. First comes the blueprint: where the walls go, how many rooms, where the water runs. Only then does the structure go up, and only once it is done does the paint come. Each stage begins when the previous one ends, and going back is expensive: knocking down a wall that is already up costs far more than moving a line on the blueprint. That is the intuition behind waterfall: the model that specifies everything at the start and then builds in phases that flow downward in sequence, like a waterfall, never going back up. The classic phases are these: gather requirements, design, implement, integrate, test, and maintain, one after another. This model is usually attributed to a 1970 paper by Winston Royce, “Managing the Development of Large Software Systems.“1 And here lies an irony worth knowing: Royce drew the waterfall diagram to say that it did not work well. He presented the pure sequence as a risky example and argued for adding back-and- forth between the phases. The industry copied the diagram he criticized and ignored the remedies he suggested. A curiosity for history buffs: the term “waterfall” itself does not appear in Royce's paper; it caught on later, in 1970s texts that cited his work. The name that became a synonym for “the old way” was born from an incomplete reading of the author who described it. Look at what actually happened. The waterfall instinct was right: thinking about the problem before you start building keeps you from raising the wall in the wrong place. That instinct was never the flaw. The flaw was the bet that you could get the entire specification right in one shot, at the start, and that nothing
would change afterward. Reality always changes. The client understands what they want better only once they see something finished; the market moves; a forgotten detail shows up at the end. And changing it there at the end, with the build standing on the wrong blueprint, was expensive. Estimates of the era spoke of a late fix costing dozens of times more than an early one.2 The problem with waterfall, then, was never planning: it was having no way to plan again without paying a fortune. The agile turn: learning to iterate cheaply If changing late is expensive, the way out is not to leave the change for late. Instead of raising the whole house at once on a closed blueprint, why not put up one room first, live in it, see what bothers you, and adjust before moving on? That is the idea of iterative and incremental development: building in short cycles, delivering and adjusting bit by bit, rather than all at once at the end. Each cycle produces something usable, gets feedback, and corrects the route of the next cycle, while correcting is still cheap. It may sound like a recent invention, but it isn't. There are records of iterative development back in the 1950s, and NASA's Mercury space program in the 1960s is a documented example of building and testing in small steps.3 Iterating was not born with the trend; what was missing was a name, shared values, and people willing to defend the practice against the weight of waterfall. That name arrived in 2001. Seventeen software professionals gathered in Snowbird, Utah, and wrote the Agile Manifesto: a short document that set four values for developing software.4 Translated from the original source, they say you should value:5 individuals and interactions over processes and tools;
working software over comprehensive documentation; customer collaboration over contract negotiation; responding to change over following a plan. Read that last line carefully, because it is the direct answer to waterfall's pain. Waterfall treated the plan as sacred and change as failure. The Manifesto flips it: change is expected, and the value is in responding to it quickly. The plan still exists; what ends is the fiction that it would be right from start to finish. Agile is exactly that: delivering in short cycles, with frequent feedback, adjusting the route at every step instead of betting everything on a fixed plan. Scrum, the pillar; XP and Kanban, the support Values need practice to become routine, and agile took shape in concrete methods. The main one, the one that most shaped how teams work to this day, is Scrum. Scrum is an agile approach based on short cycles with defined roles and events and frequent feedback. The short cycle has its own name: sprint (a burst, a short and intense run), a fixed- length period, usually one to four weeks (two is the most common), at the end of which there is something ready to show and evaluate. Each sprint the team plans what fits in the period, works, delivers, and reviews what it did, deciding the next step based on what it learned. Instead of a single giant bet at the start, there are many small bets, each correcting the one before it. Scrum was presented publicly by Ken Schwaber and Jeff Sutherland at the 1995 OOPSLA conference.6 The name comes from earlier: a 1986 paper by Hirotaka Takeuchi and Ikujiro Nonaka, “The New New Product Development Game,” which
compared high-performing product teams to a rugby scrum, where the team advances together, pushing in the same direction.7 For those already working with Scrum: it enters here only for the lesson that matters to SDD, the short cycle that makes feedback cheap. Roles like Product Owner and Scrum Master and events like the daily standup exist and are useful, but they are not the point of this chapter. Alongside Scrum, two supporting methods added pieces that will reappear later. Extreme programming (XP), by Kent Beck, takes engineering good practices to the extreme. Beck developed it on Chrysler's C3 project, around 1996, and consolidated it in his 1999 work “Extreme Programming Explained.“8 Two of its practices matter here. Continuous integration: merging and testing everyone's work frequently, instead of waiting for the end, so that errors show up early. And TDD (Test-Driven Development): writing the test before the code, so that the code is born already proving it does what it should. Both push the error close to its origin, where it is cheap to fix. The other support is Kanban, formulated for software by David Anderson out of work at Corbis in the mid-2000s and described in his 2010 book, with its root in Toyota's production system.9 Kanban visualizes the work on a board and limits work in progress, or WIP: what has been started and not yet finished. Limiting WIP avoids the habit of starting a lot and finishing little; the team focuses on completing before pulling the next item. The work runs in a continuous flow, without batches. What each method taught us
Look at the whole sequence at once and a pattern appears: a chain, each link answering the weakness of the one before it. From it come three lessons that go straight into what follows. The first came from waterfall: specifying gives direction. Thinking about the problem before building keeps you from building the wrong thing competently. That instinct was right and still holds. The second came from agile: iterating gives adaptation. Since reality changes, building in short cycles lets you adjust the route before the deviation gets expensive. It was the direct answer to waterfall's blind spot, which treated change as an accident instead of a rule. The third came from Scrum, XP, and Kanban together: early feedback reduces risk. Short sprints, testing before coding, integrating constantly, limiting work in progress. Everything points in the same direction: finding out what is wrong as soon as possible, while the fix is still cheap. Each method refined the previous one on this point, shortening the distance between making a mistake and noticing it. Direction, adaptation, and controlled risk. Hold on to the three. SDD doesn't pick one and discard the others; it tries to keep all three at the same time, and the rest of the chapter is about how that stopped being a dream. SDD: the synthesis that only now became viable Why didn't anyone simply combine the two strengths before? Why not specify with waterfall's clarity and still iterate cheaply like agile? The answer is that a piece was missing, and the piece was the cost of rewriting.
Think about the house again. If changing the blueprint meant knocking down finished walls, you would hold on to the blueprint tooth and nail and avoid touching it, exactly waterfall's reflex. If raising and knocking down walls were instant and nearly free, you would experiment freely, adjust at every visit, and the blueprint would become a living document instead of a sentence. What separated those two worlds was always the price of change. Agile lowered that price with short cycles and team discipline, but rewriting real software was still slow and expensive, done by hand, line by line. This is where the age of AI changes the equation. When producing and redoing code stops being the bottleneck, the cost of rewriting plummets. A specification that changed can be run again in minutes, not weeks. And that unlocks the combination that didn't add up before: specifying first, with the direction waterfall taught, and still iterating cheaply, with the adaptation agile taught. Specifying before building is an idea proven over decades; AI gives it new meaning rather than resurrecting it, handing the old discipline of thinking first a cost of change it never had. Let me be blunt so there is no doubt: this is not going back to waterfall. Waterfall froze the specification and punished anyone who changed their mind. SDD does the opposite: it treats the specification as a living artifact, made precisely to change and be run again as many times as needed. The blueprint is still valuable, but it stopped being a prison. Why AI needs specification It is worth closing the case for the why, now with AI at the center. Chapter 0 showed the mechanics: without a specification, the AI's context is the entire conversation, which only grows and
gets expensive; with one, the reference is a short, stable document, which the AI runs against directly. There is also a second economy, less obvious and more lasting: the specification becomes the project's memory. The decisions, the rules, and the reasoning behind each choice are recorded in one place, which survives the end of the conversation, the change of whoever is at the keyboard, and the forgetting of six months from now. A chat with the AI evaporates; a specification stays, and it is from there that the next cycle starts. How much of that material fits into each call, and at what price, belongs to Context Engineering (2026, https://books.kodel.com.br/en/books/context-engineering/), the third volume in this trilogy, which answers what the agent sees right now, in the window of this one call, and at what cost. You do not need it to follow along here: the rule Chapter 0 already gave is enough, that the model keeps no memory between one request and the next. That is where the edge of writing before talking comes from. The cycle, now with a name This synthesis has a working rhythm, and it is organized into four steps that will reappear from the beginning to the end of the material. It is worth fixing the vocabulary now, still with no tool in front of us: specify: describe clearly what you want, the problem, and what counts as correct, before asking for the code. plan: decide how it will be built, the approach and the technical decisions that hold up the specification. tasks: break the plan into concrete, executable steps, small enough to keep track of.
implement: build, now with direction, fulfilling the specification and the plan. It is the same logic as the three lessons, now in a working sequence: specifying gives direction, the plan and the tasks keep the adaptation organized, and implementing in short steps brings feedback early. There are tools that give shape to this cycle and handle the mechanical part of each step, and the material gets to them later. For now, what matters is recognizing the vocabulary: when you read specify, plan, tasks, and implement in the coming chapters, these are the four steps. The next step You now know where SDD comes from and why it makes sense now. What is left is to see up close the central piece of all this: the specification itself. In the next chapter we open the blueprint and examine the parts of a good specification, what it needs to contain to guide the build and what makes it clear enough for the machine and for you. From intent to the document that truly guides what will be built.
Footnotes “Waterfall model”, Wikipedia, includes the attribution to Winston W. Royce, “Managing the Development of Large Software Systems” (1970), the origin of the term, and the growing cost of late fixes: https://en.wikipedia.org/wiki/Waterfall_model “Waterfall model”, Wikipedia, includes the attribution to Winston W. Royce, “Managing the Development of Large Software Systems” (1970), the origin of the term, and the growing cost of late fixes: https://en.wikipedia.org/wiki/Waterfall_model “Iterative and incremental development”, Wikipedia, on the roots of iterative development predating 2001 (records since the 1950s and NASA's Mercury program): https://en.wikipedia.org/wiki/Iterative_and_incremental_development “Agile software development”, Wikipedia, on the writing of the Agile Manifesto in 2001 in Snowbird, Utah, by seventeen signatories: https://en.wikipedia.org/wiki/Agile_software_development Values cited from the primary source, “Manifesto for Agile Software Development” (2001): https://agilemanifesto.org “Scrum (software development)", Wikipedia, on Ken Schwaber and Jeff Sutherland, the 1995 OOPSLA presentation, and the concept of the sprint: https://en.wikipedia.org/wiki/Scrum_(software_development) Hirotaka Takeuchi and Ikujiro Nonaka, “The New New Product Development Game”, Harvard Business Review (1986), origin of the “scrum” metaphor: https://hbr.org/1986/01/the-new-new-product-development-game “Extreme programming”, Wikipedia, on Kent Beck, Chrysler's C3 project (around 1996), the 1999 work “Extreme Programming Explained”, and practices like TDD and continuous integration: https://en.wikipedia.org/wiki/Extreme_programming “Kanban (development)", Wikipedia, on David J. Anderson, the work at Corbis (mid- 2000s), the 2010 book, the root in the Toyota Production System, and the work-in- progress (WIP) limit: https://en.wikipedia.org/wiki/Kanban_(development)