Is there a White elephant in the Agile Camp?

So what is this White elephant in the Agile camp?

Well, I am thinking that all team based work-flows have components of process and practices. So, when it comes to Agile approaches why are some not more explicit about the process components? Let me explain.

In the webinar by Alan Shalloway, where he presented The Different Agile Approaches: First (XP, Scrum) and Second (Lean/Kanban) Generation Methods. On seeing the tabulated difference/similarities with Agile approaches (Scrum, XP, Kanban, Scrumban) a light bulb went on, especially when he was talking about TOC (Eli Goldratt).

So, for sometime I have been thinking based on my experience, when using Scrum approach in developing projects, and this doesn’t have to be limited to software projects only, that one looses sight of the critical path that is otherwise formally expressed if the project was to be planned out in detail based on the assumption that all requirements have been expressed, collected and analyzed. Now, I am not suggesting we go back to doing waterfall when it comes to software development just to gain what the critical path in the project is. However, what I am suggesting is that bottlenecks arise quite naturally in any team based work-flow, making team resources and/or the work area critical path and in the way of the flow. In fact this is common in any thing that exhibits network like behavior. So for instance you see bottlenecks on a daily basis on freeways or even at grocery check outs as capacity limits are reached at some points on the path. Based on this I realize the following:

In reality all team based work-flows have a process component and a practice component. So when it comes to getting things done (GTD, David Allen), the best approach is to deal with requests in small batches, tune the practice of work according to the flow experienced in the process with the purpose of maintaining a harmony between these two components. For software development the process steps are often just thought of as A.D.D.T, Analysis, Design, Develop, Test. Well actually the D.T process step, well the practice is intertwined especially if you adopt TDD, but lets just say there is a final acceptance test.

Now secretly but surely, lets all say it all together: This doesn’t make it waterfall development. As pointed out in Alan’s presentation that at the granular level you have to follow these steps and really its a matter of scale. Small batch, iterative development approach not waiting for all the project requirements have been nailed down first before coding is clearly the root for any Agile approach. Seriously though, its unfortunate that for quite some time the term waterfall is used as mud, often to harass scrum teams with. This is often by some layer of management and at times scrum masters, who are pushing for team velocity over say helping the team overcome systemic issues that an explicit process of steps would have helped uncover.

When it comes to the Kanban approach, and specifically WIP Limits. Well at the team level this expresses what is needed to get and maintain the practice in harmony with the process. In software development TDD, ATDD, CI & some of the XP practices are patterns that best help in creating a resonance between the practice and the process.

So what of orchestrating this at the enterprise level, which really means work of multiple teams where some have cross team dependencies, potentially dealing with multiple products and at times vying for constrained resources. I see each team has its own harmonic and the following questions came to mind:

[Q] How does one conduct such an orchestra?

- For instance with Scrum, its stated that there ought to be an SOS but as far as I know nothing else is spelled out about SOS. It’s assumed that this follows the pattern of Scrum, Fractal view of Scrum. However, in absence of any specific guide and best practice I’ve witnessed SOS being used to air out an issue log, with out the same level of accountability, transparency and expectation being prescribed as it is at the individual scrum team level. In fact the way I see it SOS team should have a prioritized backlog (in my thinking issue log expresses one level of acceptance criteria) representing the product|sprint backlog. This in turn helps with the business decision-making, including the business of product releases at the enterprise level as well as aligning elements of the business operations (marketing, sales, operations, finance, support etc.) to enable customers can attain the best value. I would say the paper on Enterprise Scrum by Dan Greening better expresses how to orchestrate this with a top-down approach at the enterprise level.

[Q] What is the process at this level?

- I can only think of the overall release process for the domain, in software environments this is some combination of (release engineering, support, marketing, sales, education …)

[Q] What are the patterns that can be called on to help reduce the impedance and create smooth flow at this level?

- Can’t think what this would be right now, but recognize that there must be patterns to deal with bottlenecks at this level too. I think what ever it is, must include “effective collaboration”, meaning things that need to get everyone on the same page and hopefully on a single glass pane.

Finally, as for TOC this is well recognized in critical chain planning, a technique used in traditionally planned projects to balance critical resource on a critical path. This leads to the other observation, that I think Alan was expressing, and I have seen frustration in both software and technical operations teams when it comes to Scrum. Whether its Scrum or XP, the frameworks don’t prescribe or guide the team to pick a process path that is best suited for their domain. Even worse is that most practitioners (scrum masters, coaches etc.) don’t see this hole and thus fail to guide teams in addressing this during the adoption phase, often resulting in the cargo cult behaviour, or as I call it monkey see monkey do. The way I see it, process expresses work-flow, transition states and interfaces for hand-offs in the chain of adding value to the product. For instance in technical operations I’ve seen poor handoff’s, and in a way I am sure this implies poorly defined or agreed upon acceptance criteria. There is no doubt in my mind that this was but one symptom of poorly expressed and agreed upon work-flow process that the Agile practice, in this case trying to bolt on the same Scrum framework that the development teams were using, helped expose. Alas in the same environment the core principle of the Agile practice, one of continuous improvement or Kaizen really meant just tinkering rather then seeing the whole work chain and recognizing the constraints for what they were.

So what does this all mean? and why does it matter? Well for one, which ever Agile framework you pick make sure you explicitly specify the process component that you want teams to subscribe to. This will help mature teams to become that much more self organizing and less dependent on manager types to make decisions for them when they hit bottlenecks (constraints) or when the team needs to set right level of expectations with other teams within the organization.

Scrum and XP from the Trenches – a great Scrum Master & Scrum Team notebook, pragmatic with common sense practice for using Scrum

Well, I just finished reading Scrum and XP from the Trenches, by Henrik Kniberg. I found my experience resonated with all the practices that is documented in this scrum masters notebook - a common sense approach to the tactical work of Scrum. I highly recommend this those new to Agile and specifically Scrum. You don’t have to be a Scrum Master to make use of this. Its a fast read with a journey into how Henrik and his teams practiced Scrum. He expresses situations they encountered and what they experimented with. he is good enough to share what was thought of but hadn’t been tried out at the time. The good part of a lightweight framework and practice of Scrum is that you can adapt it to your environment, and as the situation changes tailor the practice to best suit your needs. You don’t have to worry about the Scrum Police, mind you like everything else in life there are purist who insist on practices that they consider a must, of course this contradicts Agile. You can use Lean to assert the Scrum practices that your team is successfully using and living up to the need for transparency, continuous learning and improvement as a result, rapid delivery of what the customer values.

I have no doubt in my mind that you will pick some good practices, as for the best practices well that is for you to work out with your team taking into account your environment (company and team culture, business needs etc.) – context is everything!

As for the must haves (listed below), they are indeed a best practice. The thing about a best practice is that there is no time like now! to start doing

• The Product Owner must have a Product Backlog with estimates created by the team
• The team must have a Burndown chart and know their velocity

Is this a requirement for Scrum ? – A must have ….. Part II

So here is the Part II, a continuation to Part I, where I raised the couple of must haves that have been expressed in Scrum as measured by the iterative Nokia standards – ref: Scrum and XP from the Trenches, by Henrik Kniberg. Just as a reminder the two must haves were stated as:

• The Product Owner must have a Product Backlog with estimates created by the team
• The team must have a Burndown chart and know their velocity

The first of these was dealt with in Part I, so:

• What of the team must have a Burndown chart and know their velocity?

Burndown charts are unique to a team, given the mix of talent and experience, and in reality they are not forward looking – which is to say allowing one to project what is expected in the next sprint based on current burndown chart. They really are a record of past performance and really a barometer in highlighting the anomalies that may have occurred during the sprint that blocked the team achieving what they had planned. During the sprint it is used to signal to the team to make decisions of reallocating resources within the team, doubling down on the most difficult area as well as having the reality check in terms of scope that was targeted for the sprint. This is after all agile in practice, making good sound decisions to support what needs to be done in order to get the team back on its course to meet its commitments. Really I’d like to say such occasions are rare, but the truth is far from this. Lets remind ourselves that the endeavor of software and system development, and really the realm that knowledge and creative workers play in is not a production line, often there are managers who loose sight of this. So at times like this, when decisions are being made, it is helpful for the team to think lean. Ideally one wants the team to think lean through out all iterations, but in particular get into the practice of putting their hat of lean thinking when ever weighing up decisions to address how can the team best to reach the goals that they have committed to. As I said sometimes this may mean doubling down with resources from within the team on a particular task at other times the hard reality may be that the team has overloaded the plate and can’t possibly consume all that in time boxed for the current iteration. The hard reality of decision making may mean delaying the release, this may be especially true if the rhythm of release in your environment is of high frequency, say every week or two weeks versus every three or four months.

So what about knowing your velocity. Let me start by some basic physics, velocity is a vector, it has a direction and magnitude. The magnitude is measured as speed – you know in m/s. Fundamentally given my Engineering and Physics background I have a problem with the loose use of this term to connote what a team is capable of accomplishing as number of story points delivered per iteration or in the macro view story points delivered per release. Aside this objection, one wants to be smart about using this metric in predicting and really looking forward as to what the team is capable of accomplishing. I say smart, as far to often variables that have a strong bearing on this metric may have changed. For instance my experience has been that the team complement changes after one of two releases, or for that matter the release cycle changes, and or most critically one hasn’t sized the backlog to the same standard. After all this is not the repeat cycle of building the same widget over and over. For the greater part there is variability in developing something different but for only using similar patterns that one may have encountered when developing earlier items of the backlog. Even when this is the case there may be new set of complexities that were just not present previously. Far too often stakeholders do a linear intra/extrapolation and then confront the team with the 64,000 dollar question – why not as much as previous releases? Some see it as holding the teams feet to the fire, thats one way to make stew!

My good sense says be pragmatic with use of this metric; first monitor it over a reasonable period of time. Then use statistical methods of measuring standard deviation about the mean that the team should try to achieve. One can set the lower and upper limits that are 6 sigma from the mean in this metric as the boundary points for the pessimistic and optimistic with the mean value representing the most likely in size for delivery during a release. Over time, with the team maturing as a work unit the tolerance may be narrowed down to 3 sigma thus giving higher fidelity in the estimates of what the deliverables are going to look like. In essence work with the team to establish this metric pointing out the use it is going to be put to as there are supporting cast besides the Product Owner, such as Sales, Marketing and Capacity planning, who are in the business of drumming up more customers for the product as well as setting expectations to existing customers.

Fundamentally, are these truly must haves, sure for high performing teams, but such teams are not build in a day or by virtue of few days spent in training on Scrum or one other Agile practices. Truth be told more often then not it is both the organizational and/or team culture that one has to foster in order to achieve teams to become high performing teams in nature. Let me know which team doesn’t want to become high performing and is this even a willful act on the part of the team? – Possibly in an environment that has fostered the wrong culture, but for most part these must haves are a work in progress. By the way these must haves are not the only metrics of measure to assert a team with the title of high performing in nature. I will save this topic for another day.

Is this a requirement for Scrum ? – A must have ….. Part I

I am reading Scrum and XP from the Trenches, by Henrik Kniberg. This is the online edition, written couple of years ago, so I am catching up on some reading what can I say?. Anyways, in a forward for the book, Jeff Sutherland conveys a story from a London conference where there was a discussion about Google’s implementation of Scrum. By the sounds of this discussion this is not Google specific, as there were others who must have participated in this discussion. Jeff had asked questions relating to how many were doing Scrum and really iterative development by Nokia standards. This standard apparently asserts, amongst other things:

• The Product Owner must have a Product Backlog with estimates created by the team
• The team must have a Burndown chart and know their velocity

These two must haves in Scrum as measured by the iterative Nokia standard are interesting, and I will address these in 2 parts, as they open up the following questions:

• When is the Product Backlog considered to be true backlog given that estimates from the team are required to make it so?

Surely the role of the Product Owner is responsible in developing a meaningful product backlog that has been prioritized. When I say meaningful, it means this has been written in a form of User stories (including epics). This backlog is prioritized based on the following needs:

• expressed by customer(s)
• identified as gaps in the market place that not only differentiates the product amongst its competitors, but critically appear on the product roadmap as having strategic value

In a sense the backlog is providing the scope horizon for the product. Clearly in the A, B, C of steps to follow, the items of highest priority on the product backlog will have been reviewed by the team, broken down further if need be and sized with estimates on an arbitrary number scale that the team has agreed on to represent as a common measure. This is a relative scale uniquely representative for this team, it expresses the mix of skills, experience and appetite for risk that this team has – which by the way is unlikely to be represented exactly in the same way by another team. So, during the sizing exercise, which usually takes place during formal planning meetings but there is no reason why a Scrum team can’t decide to size a story one or or more (based on some agreed rule by the team) per week during the current sprint. When ever needed I encourage this practice with the mindset that “a story or two sized keeps long planning sessions at bay” – okay it isn’t as good as an apple a day keeps a dentist away.

Based on my experience to date I have found this practice has been a sustainable approach to having a backlog that is primed for the next sprint. Don’t get me wrong, I am not advocating a long head with detailed planned backlog, clearly this would not be lean not to mention as a process it doesn’t bind the team with commitment in mind either. Lets say, just do enough planning to help prime part of the next sprint. In the mean time the Product Owner, who after all has been interfacing with the customers and has identified what is being valued and needed in the market place can seed the remaining backlog with estimates for size. This helps express the target of what can be expected that is of value for the next release, and in turn allowing other stakeholders from such quarters as engineering, operations, support, sales and marketing to influence and help refine this target. One can expect to have discussions, where you have multiple product teams, around cross dependencies that are going to impact teams and as a result having to reset priorities or refine the product backlog in order to mitigate the impact.

When it comes to estimates for size, it is well known from traditional Project Management techniques that using single point estimates for sizing just doesn’t work. It is far better to make use of statistical estimation, using 3 point estimates and if need be Z scores for % of confidence level to support the basis of the estimate. In the 3 point estimate one can cover the bases of pessimistic, optimistic and most likely estimates accounting for the expert viewpoint, the individual who is committing to do the work and the voice of caution addressing the risks be it in terms of resources available to accomplish all the testing that may be needed to verify and validate the functionality.

So really the conclusion I, and really many others have long ago come to is that the Product backlog is to be imagined as a pebble being dropped in a pool of water. When this occurs, it creates a set of waves some of these are of relative high amplitude, marking close proximity to the center, and then others of relative low amplitude further away you are from the center. The larger ripples closer to the point of drop means greater information being made available, this information has higher fidelity of knowledge supporting it and in contrast to ripples at the edge of the echo where information is imprecise and of low fidelity. Yes the Product backlog is a mix, with higher priority items floating to the top and as these float up they are adorned with more information, including decomposing epic level backlog into component backlog as well as having attributes such as acceptance criterion’s that customers will measure their need being satisfied to.

As for the second must have I will address this in the follow on article.