Scrum and the Theory of Constraints – Part 2

January 20, 2015

So what if you have user stories that are dependent on external resources? The answer act a little differently during sprint planning by subordinating specific user stories to resource availability:

  1. Check and see if there are user stories that have been marked with constraints. If so they take precedence over regular user stories. For each of them check if their constraints still hold and can be fulfilled – if so choose the story. If the constraints no longer hold, put them into the normal backlog and continue as usual.
  2. For every selected unmarked user story choose and analyse them as usual – if there are no external dependencies, continue. Otherwise mark the user story as one that has constraints. These stories require a bit deeper analysis to see if they can be split into 2 stories – one that requires the dependency, and another that doesn’t. Add the user story without the dependency to the current sprint, and mark the story with constraints to be handled later.
  3. For all the stories that have dependencies, the scrum master must work with management to schedule and assign the story to a specific future sprint (that of course is dependent on the availability of the resource).

Scrum and the Theory of Constraints – part 1

January 13, 2015

One of the tenants of scrum is that the group has all the people and resources they need to complete their stories within the designated timebox. You won’t start a scrum unless you have available all the people and resources needed to complete it. In the Theory of Constraints this is called the “complete kit“.

In the real world (especially in large organizations) many times the resources needed are constrained – people get snatched to other tasks, skills are not always available when needed or planned, machines aren’t available or are broken etc. Which means that the “complete kit”assumption of scrum doesn’t hold.

Standard scrum methodology doesn’t address this issue at all, which is one the reasons it is difficult to apply agile to organisations that have teams that support multiple productsservices with conflicting needs.

The way to handle this at scrum zero (and keep it updated as part of the backlog refinement for every sprint)

1. The team must define the “complete kit” for each user story selected from the backlog (e.g. availability of environments, business experts).

2. It is the responsibility of the scrum master is to subordinate user stories assignment to available resources, and plan ahead to assign user stories to future sprints to account for anticipated resource availability.

This is a critical responsibility since if not handled correctly the project will be “starved“. In part 2 we will describe how the scrum master can do this.

Test Driven Development (TDD) vs. Business Value

December 10, 2014

Test Driven Development (TDD) is broken (or at least is implemented very badly by most organizations) see our previous post for a great explanation about why it doesn’t work. What TDD needs to become is business oriented testing that can be used as the “definition of done” as described in scrum.

Our approach is similar to the approach taken in the Sprint Planning Meeting where the Product Owner and the team negotiate which stories a team will tackle during that sprint. The difference is that we believe that rather than just doing an estimation of effort – the first task is to use business value as the main vector in estimating what should be achieved during a sprint. Together the product owners and developers create a chart for each proposed user story.  In that chart the product owner uses the y-axis to indicate the business value and the developers use the x-axis to indicate the associated effort. Then stories are classified into one of four categories as shown in the figure below – Oyster, Pearl, Quick Win and White Elephant.

ease value matrix

The idea is find the mix of stories that maximize business value subject the timeresource constraints. The methodology is to first focus on Pearls and trash the White Elephants  – don’t do them just because they are there. Oysters are usually too difficult to complete in a single sprint and should be subdivided into user stories that can be implemented in a single sprint – i.e. Pearls, Quick Wins or White Elephants. Finally use whatever timeresources you have left for Quick Wins – do as many as possible.

An example is the failed Apple Navigator project. Apple tried to incorporate a cool new story – navigation enhanced by real-time 3D imagery. This story turned out to be the oyster that ruined the project. What Apple should have done is decompose the oyster into its components: navigation and imagery. From a business perspective navigation is a pearl, while imagery is a white elephant.

Once the stories are selected – developers should work on implementation and testers should start working on the business acceptance tests – the tests that define the required business results from the sprint. These should be generated from the sprint’s user stories.

These business acceptance tests are the ones that matter and should live as long as the user story (as regression tests). They are independent of implementation – and therefor also independent of refactoring or other changes to the system.

7 Principles of BuDD – Design by Delivery vs. Delivery by Design

November 29, 2014

Software delivery usually takes a backseat to the development process. Gather requirements, architect the system, implement, test, deliver is the standard order of the steps taken by IT.  Some agile methodologies like TDD (Test Driven Delivery) try to change the order to requirements, architecture, test  and finally deliver. This is what we call delivery by design – delivery is always the last step.

Design by delivery is different. Design by delivery can use either of those cycles – but the idea is to create software in increments that are delivered and verified by actual users. Delivery considerations need to be part of architecture,  implementation and testing – since only delivery can ensure that the system is on track from both a functional and non-functional perspective.This in many ways encapsulates the most important part of scrum methodology. Instead of each sprint creating only an user deliverable, it must deliver the software to actual users and provide value to the business.

This is possible if first two principles of  business driven delivery are applied:

  • Measure, manage and focus on the value to the business.
  • Integrate early, integrate often – and always have something to show the business.

7 Principles of BuDD – Integrate early, integrate often

November 25, 2014

The second tenant of business driven delivery (BuDD) is to integrate early and often. You also need frequently show something to the different stakeholders – e.g. management and customers. The reason for the focus on integration is to make sure that folks aren’t off developing in parallel components that just won’t work together like in this example: http://www.theguardian.com/world/2014/may/21/french-railway-operator-sncf-orders-trains-too-big. This helps make sure you are building things right.

To ensure that you end up delivering the right thing – you need to continually demo the evolving system to the stakeholders. This helps in two ways – you get feedback about what isn’t right, and buy-in to the final product.

Software delivery solvency and subprime technical debt

November 15, 2014

Software delivery solvency is the ability of the IT department to deliver working services that fulfill a business need in a timely fashion. An example of delivery insolvency is the original Obamacare program.

A key factor in delivery insolvency is that software naturally deteriorates over time because of initial kludgy shortcuts, changing and misunderstood requirements, misaligned and misunderstood technical architecture as well as a changing business and technical environment. The deteriorated code becomes so complex and brittle that any attempt to change is so expensive as to render the code difficult to maintain and almost unmodifiable. Legacy is one example of insolvent code, but there are many examples of code becoming insolvent long before it is legacy.

Technical debt describes the cost accrued by code deterioration, and the standard way to repay that debt is refactoring. Refactoring is the act of redesigning code so that it has the exact same functionality and interface, but an improved more robust internal structure that both meets the current requirements and lays the groundwork going forward. Just like in nature this cycle of deterioration and renewal is what keeps software viable. For example gcc (the GNU c compiler still widely used today) dates to 1987 – almost 30 years old.

In finance, some debt is more risky to the lender and more costly to the lendee – usually loans given to people who may have difficulty maintaining the repayment schedule. We believe that the equivalent type of debt is “delivery debt” – essentially problems in the code or architecture (i.e. technical debt) that affect delivery rather than functionality. Because delivery is an after thought in most organizations (which DevOps is trying to fix) – most development organization are happy to take shortcuts with respect to the “deliverability'” of their code. They leave it to Ops to sort things out – and it is not unusual for Ops to have to work around the same bugs for multiple releases. This type of subprime debt causes code to deteriorate and become insolvent much faster than normal debt.

A real life example of technical debt causing insolvency is of a telecommunication company that wanted to introduce prepaid SMS packages but the billing system architecture assumed that SMS is a part of standard billing. It required months to change the architecture and code such that it allowed of a prepaid model.

Agile and test driven development (TDD) can be useful software development paradigms, but there is no doubt that agile and TDD can cause weaker software solvency and can encourage development of subprime software products.

Slide1

No Owner = No Delivery

November 11, 2014

Here is an extreme example of what happens if you break the first rule of BuDD (No Measure, manage or focus on the value to the business). This is taken from Mikey Dickerson Velocity NY 2014 Keynote: “One Year After healthcare.gov”.

Really Interesting Talk on the Problems with Test Driven Development

November 8, 2014

Excellent talk by Ian Cooper at InfoQ – “TDD: Where Did It All Go Wrong?

Interesting New Research on the Value of Combining Agile and Waterfall

November 8, 2014

Like many management philosophies  – agile development is a valuable tool in any organization’s toolbox but not a magic bullet fix-all. Here is a interesting report: 2014 CAST Research on Application Software Health (CRASH) that shows the benefit from mixing agile and waterfall methodologies.

7 Principles of Business Driven Delivery

November 3, 2014

Software development (Dev) has undergone disruptive changes in the last years – broadly defined as Agile. There have also been disruptive changes in delivery (e.g. Virtualization, Cloud, SaaS) also known as Ops. These changes were expected to empower IT to be closer to customers and deliver functionality faster and set business expectations that IT will no longer be a bottleneck: changes initiated by business leaders will be delivered to customers within weeks rather than years. This was at the peak of the agile and cloud hype curves.

That is the promise of agile, but in many cases the reality is very different. IT needs a holistic view that is focused on the true value of the end result to business and customers. This is a journey, much more than a destination. To guide us on our journey we have come up with these 7 principles:

  1. Measure, manage and focus on the value to the business.
  2. Integrate early, integrate often – and always have something to show the business.
  3. Design by doing vs. doing by design.
  4. Rigorous automated testing.
  5. Repeatable, scalable, automated release processes.
  6. Continuous delivery, not continuous deployment
  7. Plan for exception handling, rollback/rollforward and corrective-and-preventative-operations (COPO).