In the 1990s, the world of software development experienced a crisis similar to what we are experiencing in education today. With increased competition came increased expectations. Projects became more complex; standards were raised for results.
In reaction, companies implemented more detailed project planning and more stringent process control in an effort to maximize efficiency and minimize risk. But development teams were increasingly unable to execute these plans, and companies incurred record expenditures while somehow getting less productivity and poorer end results.
Engineers could not contain costs, meet schedules, or reliably deliver quality products. Software development companies, and many internal systems development groups, became paralyzed by uncertainty. In an engineering-based industry that prided itself on reliability, failure was rampant and some practitioners felt that the reputation of the entire profession was at stake.
A Method for the Madness
Frustrated with his own company’s failures on big projects with clients like Coopers & Lybrand and IBM, software developer, Ken Schwaber, founder of Advanced Development Methods, sought help in 1995 from Japanese process theory expert Babatunde Ogannaike and others at the DuPont Experimental Station.
As Schwaber recounts: “Suddenly, something in me clicked and I realized why everyone in my industry had such problems building systems. I realized why the industry was in such trouble and had such a poor reputation. We were wasting our time trying to control our work by thinking we had an assembly line when the only proper control was frequent and first-hand inspection, followed by immediate adjustments.”
Schwaber learned that there are two major approaches to managing large projects: a “defined process control model” and an “empirical process control model”.
The “defined” model requires a well-defined set of inputs and a clear understanding of what will happen at every stage of the process. Like an assembly line in a manufacturing plant, every aspect of the manufacturing process is defined in advance.
As long as the same input is fed in at the beginning, and the machine runs properly at each stage, the output will be the same every time. This is how we wish education worked and what our current Industrial Age implementation is being reformed to produce.
The “empirical” model, however, is a more appropriate choice when inputs are not well-defined, when actions to be taken at various stages cannot easily be predicted, and when the end result is not the same for all inputs.
This is the way education really works and what an Information Age education system must be designed to facilitate. Kids entering the system differ dramatically. Each requires different interactions at each stage in the process, and the outcome for a single student cannot be standardized or easily predicted.
Rugby, Anyone?
A decade before Ken Schwaber had his Road to Damascus moment with Babatunde Ogannaike, a pair of Japanese business experts, attempting to describe important changes in the work processes of successful companies, minted a metaphor for a new kind of teamwork, a metaphor Ken Schwaber used to develop a new and better way of making software.
In the January 1986 issue of the Harvard Business Review, Hirotaka Takeuchi and Ikujiro Nonaka wrote, in an article called The New New Product Development Game, that “In today’s fast-paced, fiercely competitive world of commercial new product development, speed and flexibility are essential. Companies are increasingly realizing that the old, sequential approach to developing new products simply won’t get the job done. Instead, companies in Japan and the United States are using a holistic method; as in rugby, the ball gets passed within the team as it moves as a unit up the field.”
From this metaphor, and his experience with Japanese process theorists at the DuPont Experimental Station, Ken Schwaber, along with software developer Jeff Sutherland, created a new way of making software designed specifically to deal with the uncertainty of new product development in a rapidly changing world.
If You Build It, They Will Scrum
Schwaber and Sutherland created an approach called Scrum. It was new. It was different. And it worked. As Schwaber and co-author Mike Beedle write in the Preface of their book, Agile Software Development with Scrum, “…Scrum doesn’t provide marginal productivity gains like process improvements that yield 5%-25% efficiencies. When we say Scrum provides higher productivity, we often mean several orders of magnitude higher, i.e. several hundred percents higher.”
One key to Scrum’s success is that it is based on an “empirical” model of control. Real work is not delayed during long, drawn-out planning phases. Rather, work begins immediately, and proceeds in short “sprints” (often 2-4 weeks long) where incremental degrees of functionality are planned, designed, tested, and delivered.
Team members meet every day for a brief “stand-up meeting” where each person announces what they accomplished the day before, what they play to accomplish today, and what, if any, impediments stand in their way. A team leader, or “ScrumMaster”, is responsible for removing impediments so work can proceed as efficiently as possibly.
By getting rid of long planning periods, teams save time. By conducting daily “stand-up meetings”, teams assess project status and make rapid adjustments to optimize productivity. By breaking large projects into small “sprints”, teams deliver working software for frequent review by customers and receive feedback to improve.
Sprints also help teams learn more about customer requirements and how to use this information to create a “feature backlog” that is prioritized and addressed in the next sprint. Frequent releases increase customer feedback. More customer feedback means changing requirements. But, working in short sprints with prioritized backlogs, teams adapt easily.
Increased adaptability has a dramatic effect on productivity, often simply by avoiding time spent on the wrong things. By doing away with the long pre-development planning sessions, teams don’t waste time planning things they may not need to build.
Like all “empirical” process models, Scrum assumes and anticipates change and increases a team’s ability to respond quickly and appropriately. Scrum makes teams more adaptable to unpredictable circumstances.
As Schwaber and Beedle note, “When we say higher adaptability we mean coping with radical change. In some case studies, we present cases where software projects morphed from simple applications across multiple domains: Scrum still managed while providing greater human comfort to everyone involved.”
The key differences between Scrum and traditional “defined” software development methodologies can be found in improved communication, improved workflow, improved team interaction, and improved customer satisfaction. As Schwaber and Beedle explain: “Scrum reduces risk and uncertainty by making everything visible early and often to all people involved to all the people involved and by allowing adjustments to be made as early as possible.”
Validating Schwaber’s hunch about the importance of switching from a “defined” process to an “empirical” process model, Scrum produced immediate gains: “The success of Scrum is overwhelming. Scrum has produced by now [in 2001] billions of dollars in operating software in domains as varied as finance, trading, banking, telecommunications, benefits management, healthcare, insurance, e-commerce, manufacturing, and even scientific environments.”
From Scrum to Agile
Between 1995 and 2001, the software world erupted with new product development methods. These “lightweight” approaches were popular because they didn’t weigh people down with ridiculous degrees of what was often referred to as “Dilbertesque” micromanagement. In addition to Scrum, there was Crystal Clear, Extreme Programming, Adaptive Software Development, Feature Driven Development, and Dynamic Systems Development Method.
Eventually, the popularity and increased use of lightweight empirical models coalesced into a single unified movement. In February of 2001, at The Lodge at Snowbird ski resort in the Wasatch mountains of Utah, 17 leaders in the lightweight methodology movement met to find common ground. Their discussions resulted in the beginning of Agile software development and the creation of the Agile Alliance. The Alliance was formalized with the writing of The Agile Manifesto and the The Twelve Principles of Agile Software Development.
Discussion
Add your feedback below. To keep track of who is saying what, use an H1 tag (a plus sign "+" in the edit window) to identify yourself. For example: