A software development methodology is a process of how to build software. It is usually more than just a process, and will have some underlying philosophy behind it and an era that it evolved from. There are many many different methodologies out there, and it is quite a task to work out which is the most appropriate methodology for you. There are some pretty cool ideas worth experimenting with.
For example, a couple of lesser known methodologies are Mob Programming and Anarchy Programming. Mob Programming was started by Woody Zuill and his team with the idea of everyone on the team working together on same thing, at the same time, in the same space, and on the same computer. This can sound a little crazy, but if you stop to ask why this is successful, some lessons can be learned. Anarchy Programming is another example and was started by Fred George with the idea of removing the typical structures of management to allow for stronger self-organised teams with greater responsibility.
Both of these methodologies are very interesting and it is great to see people thinking about improving the way they build software. Personally, I believe it is important to keep an open mind to new ideas and pay them the respect they deserve. Like all methodologies, they are pinned by an underlying philosophy that deals with the situations people are facing at that point in time.
In this article, we are going to look at some of the more well-known methodologies and present a way to categorise them to help with your understanding. In the summary, we put in some recommendations about improving your way of working.
What are the different categories of software development methodologies?
In the following table, three categories are presented based on their delivery cadence. I find this a good way to think about the different methodologies available. Waterfall is a single linear process. Iterative approaches use cycles or sprints, and continuous is about being in a constant flow or a continuous mode of delivery with no iterations or sprints.
| Category | Roots | Summary |
|---|---|---|
| Waterfall | Construction (Possibly) | As early as 1970, waterfall approaches have been criticised for not working within the context of building software. Possibly, the approach was adopted based on historical human achievements in construction where something hard (physical) was built. |
| Iterative | Agile Manifesto | In 2001, the Agile Manifesto was agreed upon by representatives from Extreme Programming, SCRUM, DSDM, Adaptive Software Development, Crystal, Feature-Driven Development, Pragmatic Programming, and others sympathetic to the need for an alternative to documentation driven, heavyweight software development processes. |
| Continuous | Lean Manufacturing | In 1938, Kiichiro Toyoda (the founder of Toyota Motor Corporation) introduced the philosophy that the ideal conditions for making things are created when machines, facilities, and people work together to add value without generating any waste. Lean Software Development (LSD) and its offspring are heavily influenced by these ideals. |
Waterfall software development
Waterfall is a linear approach to software development that requires a significant amount of effort scoping the project. An example of a waterfall methodology is the COCOMO model that was popular in the early years of software engineering. How early? Like in the previous century. During that period there was a significant failure rate of software projects. Things have gotten better in recent times.
One thing people were drawn to about waterfall approaches was their apparent certainty. People could get an estimate with a fixed time of how long the project would take and an estimate with a fixed scope of what was going to be delivered. Having a fixed time and a fixed scope gave certainty to the people approving the projects. Paradoxically, there was little certainly as the number of projects failing told otherwise.
A movement emerged from this period and the Agile manifesto was born in the early 2000’s (you can read some history here). Agile is a set of values and principles on better ways of developing software. It does not prescribe any specifics on implementation. For example, no where does it say that iterations (or sprints) must be used. So, if you come across someone saying Agile is an iterative approach unlike waterfall, then you can call them out and pass on some valuable knowledge.
It is my recommendation not to use waterfall but to look for a methodology that is iterative or continuous. If you find yourself challenged by people that are stuck in a waterfall frame of mind (fixed time with a fixed scope), have a read on how do you convince people not to use waterfall.
Iterative software development
Iterative software development uses cycles or sprints to deliver value to users early, by releasing often. The iteration can be fixed or variable in length. Using iterations is usually synonymous with Agile even though the Agile manifesto does not mandate their use. Two of the most well known iterative methodologies are covered next; Scrum and Extreme Programming (XP).
Scrum
Scrum is awesome. It falls into the iterative category for software development methodologies. Scrum uses fixed length (time-boxed) sprints and each sprint has a goal the team works towards. The Scrum Guide states Scrum is: “a framework within which people can address complex adaptive problems, while productively and creatively delivering products of the highest possible value.”
Scrum has three pillars of transparency, inspection, and adaptation. It also has five values of commitment, courage, focus, openness and respect. The mantra is that the successful use of Scrum comes when the team becomes proficient in living the five values and the three pillars uphold every implementation.
The following image shows how the sprints work. At the beginning of each sprint a planning session is held where the team put stories from the product backlog into the spring backlog. The scrum team then works on the backlog for the fixed length having daily scrums (meetings). At the end of the sprint, the software should be released and the team does a sprint review and a sprint retrospective.
Scrum actually predates Agile. Jeff Sutherland, one of the Scrum co-founders, explains more about that here. All in all, Scrum is an excellent choice and we praise Jeff and Ken for all their hard work over the years.
Extreme Programming (XP)
XP is awesome too. The XP genesis comes from Kent Beck’s work on the Chrysler Comprehensive Compensation System (C3) payroll project in the 90’s. Later he collaborated with Ron Jeffries and Ward Cunningham and all three are considered the founders of XP. XP uses fixed length iterations like Scrum.
The five values of XP are communication, simplicity, feedback, courage, and respect. There are thirteen core practices (bullet pointed below) and Jeffries explains them in what is extreme programming.
- Whole Team
- Planning Game
- Small Releases
- Customer Tests
- Simple Design
- Pair Programming
- Test-Driven Development
- Design Improvement
- Continuous Integration
- Collective Code Ownership*
- Coding Standard
- Metaphor
- Sustainable Pace
The following image shows the practices and it is a great way to visualise their importance and to get into the XP mind set. For example, every contributor to the project is an integral part of the Whole Team (found at the top of the circle). Personally, I really like the emphasis on the team, customer, planning, and small releases.
XP (like Scrum) is considered to be an Agile software development methodology. However, Jeffries is now telling us developers should abandon Agile! This can seem quite alarming but I believe Jeffries is frustrated with the tentacles of big business and how it has manipulated Agile away from its original intention of making the lives of software teams better. I agree Ron, we should be thinking for ourselves! Thank you for your honesty.
Continuous software development
Continuous software development does not use iterations or sprints but is about the flow of work. This category of software development was inspired by lean manufacturing and the Toyota Production System with the philosophy of achieving the complete elimination of all waste in pursuit of the most efficient methods.
The Toyota Production System (TPS) was established based on two concepts: “jidoka” (which can be loosely translated as “automation with a human touch”), as when a problem occurs, the equipment stops immediately, preventing defective products from being produced; and the “Just-in-Time” concept, in which each process produces only what is needed for the next process in a continuous flow.
Two of the most well known continuous methodologies are covered next; Kanban and DevOps.
The Kanban Method
The Kanban Method has been developed by David Anderson and he published a book in 2010. People sometimes get a little confused because the name of the method includes the word Kanban, which is part of the principles used in the Toyota Production System (TPS). I assume David choose that name to align with that philosophy.
The Kanban Method places an emphasis on visualising Work In Progress (WIP) and maximising efficiencies in the system. A Kanban board (see below) is used to help the process. There are 4 core principles:
- Start With What You Do Now
- Agree to Pursue Incremental, Evolutionary Change
- Respect the Current Process, Roles & Responsibilities
- Encourage Acts of Leadership at All Levels
To eliminate waste (maximise efficiencies) the DOWNTIME acronym is used to highlight where leadership can be taken.
- Defects
- Overproduction
- Waiting
- Not using talent
- Transportation
- Inventory excess
- Motion waste
- Excess processing
One final note on the Kanban method is to discuss Scrumban (I have found these can get mixed up). Scrumban is a hybrid of Scrum and Kanban and was first published in 2008 in Scrumban: Essays on Kanban Systems of Lean Software Development. Scrumban stems from some frustrations teams were having in implementing a pure Scrum approach. They found that incorporating a Kanban board into their scrum processes helped them significantly.
DevOps
DevOps is the new kid on the block. And when I say new kid, I mean it seems to have begun around 2007-2008 but has more recently gained traction.
DevOps is a set of practices that brings together software development, testing, and operations. Traditionally, these functions have operated in silos and they still do in some organisations. By bringing these together into DevOps, it is possible to have faster software releases among other things. I am not sure why poor testing got left out of the acronym.
As teams become more mature they move up from continuous integration, to continuous delivery and finally to continuous deployment. There is an emphasis on the various tools, or toolchains, that can be used to help with a DevOps teams maturity. The following categories are used to describe how any tool can help with DevOps:
- Plan
- Code
- Build
- Test
- Release
- Deploy
- Operate
- Monitor
Summary
There are even more software development methodologies out there beyond just these. It can be quite overwhelming and it can feel like you need several lifetimes to get your head around all of them. With this overload, what is the best software development methodology?
Firstly, never use waterfall. Period.
If you have a blank sheet (like a startup or a new business unit), it is my recommendation to use either an iterative or continuous software development methodology. If your team is relatively new, I would use an iterative methodology and take some ideas from Scrum and XP. If your team is relatively experienced, I would use a continuous methodology and work from a Kanban board with an emphasis on DevOps.
If you are in an existing organisation, no doubt you will have some existing practices and there are many risks in introducing any methodology. It is very difficult to introduce an entire methodology like Scrum, XP, or any of them. It can take many months to train a team up on different proficiencies and this endeavour has failed inside many organisations. There ends up being a clash between the existing and the new.
The truth is that most organisations end up with a hybrid methodology of some description.
For this reason, my recommendation is to implement a What is a Way of Working?. A way of working is something that we have been using at Codebots for some time and have introduced to many organisations. In summary, a way of working allows you to draw a line in the sand of how your people operate today, and then use experiments to empower the team to implement the best ideas from whichever methodology presents a solution to some challenges you face today.
In this article, I have attempted to help you use the delivery cadence of a methodology to distinguish between different methodologies. Personally, I find it a good way to differentiate between waterfall, iterative, and continuous methodologies like this. One final note on this, as it has been niggling at me for the weeks it has taken me to write this article, all of these different methodologies have grown up together and they have influenced each other. Putting the methodologies into strict categories is quite difficult. So, you can think of them as having overlap sometimes.
Written by Eban Escott
Founder of Codebots
Dr Eban Escott received his Doctorate from UQ (2013) in Model-Driven Engineering and his Masters from QUT (2004) in Artificial Intelligence. He is an advocate of using models as first class artefacts in software engineering and creating not just technologies, but methodologies that enhance the quality of life for software engineers.
