back
|04 Sep 2017|Klaus Leopold

Prioritization, Cost of Delay and Risk Assessment

0

Klaus, Markus and Katrin in Workshop Mode

In my new book Practical Kanban I conducted an interview with Markus Andrezak about the topics prioritization, cost of delay and risk assessment. Markus has been developing high traffic, high revenue internet products for more than 15 years. Before starting his product consulting company “überproduct“, he worked for a broad spectrum of companies such as eBay Classifieds Group, AOL, the Scout-Gruppe, Gruner & Jahr. He has a vast experience with prioritization and risk assessment.
Klaus: You have quite a bit of experience with prioritization, Cost of Delay and risk estimation. In your opinion, where can these concepts be utilized?
Markus: From my point of view, Cost of Delay and Risk Estimation, as you also describe them, are generally applicable tools that can be used for the estimation of projects, proposals, initiatives, features, etc., in order to get away from decision making based on opinion, pure intuition or even worse: HiPPO (Highest Paid Person’s Opinion). The topic is a bit like an enlightenment in this area. It is most obvious perhaps in a company’s portfolio planning. A portfolio does nothing more than describe the things that you do. However, the things that you do should have a few positive characteristics, which are understood in advance. But a portfolio should be balanced with regard to risk. Risk can be determined by examining the possible opportunities and drawbacks of a proposal. The trick is to limited downside and give yourself as many opportunities as possible. Then things will thrive.
Klaus: The estimation of upsides and downsides can eventually be expressed really well with Cost of Delay, don’t you think?
Markus: Yes, but now it’s going further: Who needs to determine the Cost of Delay? In the case of a portfolio, it should be discussed by a group that is responsible for leading the company’s path ahead. In this discussion, the exercise is to specify the Cost of Delay, simply as a learning tool. Trying to define CoD helps to ask the right questions about the proposal and to quantify the learnings. Say, I have three projects to choose from, but can only implement one based on the resources available. Then Cost of Delay and the underlying discussions can help the group to make a coherent and informed decision which option might have the best payoff. This is completely different than just saying “I believe we have to do this because the competition has it” or similar prevailing justifications. Cost of Delay has many functions. First, it is used to understand, and then decide, the value of a proposal. At the same time, you can be certain that different opinions will lead to varying estimations. Interestingly, it is exactly these diverse points of view that are valuable, because they help to understand the proposal from various perspectives. Continuous and frequent—highly frequent—discussions about this topic lead to a great amount of trust within the group, as well as a homogenous, reasonable and informed point of view. Now, not only can each individual from the group support the company’s decisions, but each project or initiative can be well-justified. For that reason, it is necessary to continually have these discussions. I also take extreme care to keep these discussions away from controlling and monitoring aspects, in order to make them purely decision-making meetings.
Klaus: What constitutes “highly frequent” for you in this context?
Markus: My preferred frequency is, actually, to have these meetings weekly and to keep them short. In a company with up to 200 employees, a half-hour is adequate if the discussion is limited to only the project decisions and new information. The conciseness of the meeting is certainly a way to reduce the “one more meeting effect”. Eventually, it will be clear to everyone involved that there is no other meeting as important is this one.
Klaus: It can also happen, of course, that not everyone is immediately in agreement. That’s why I also recommend working with a range of values when quantifying, so these disagreements and uncertainties are made explicit. This is all well and good, but what do we do when the range is large?
Markus: When the opinions in the meetings continue to drift further apart after a month, it is a sign that orientation has been lost. There isn’t enough clarity about the objectives, the vision, the strategy and the strategic goals of the company. Then, opinions must naturally drift apart. In this case, it’s important to quickly clarify these points, because without clear measurements guidelines, it is unavoidable that the proposal will be estimated differently by different people.
Klaus: Can you think of an example, when the views are further apart and when they are closer together?
Markus: In product maintenance, you should arrive at a very clear estimation for Cost of Delay, otherwise you haven’t done your homework. With completely new products, services and proposals, on the other hand, it is completely normal that the Cost of Delay estimations will show large ranges, uncertainty and ambiguity. There is nothing wrong with this: Something new is always uncertain, but doing new stuff is necessary. Cost of Delay simply shows that something new and uncertain is being undertaken.
Klaus: Well, the boundaries between product maintenance and new products are often not clearly drawn. In which case should Cost of Delay be determined?
Markus: Yes, the borders get blurry. I believe, though, that Cost of Delay can be used in both cases, because Cost of Delay at least two functions. On one hand, it simply helps represent the value and risk of a proposal as a number. On the other hand, Cost of Delay is an instrument to lead a discussion and to ask the right questions. By standard procedures—as you discussed in the billing example—you need to investigate something based on quantifiable facts. You can receive qualitative data from marketing, but the data is completely anecdotal for new products, because there isn’t any data available.
It’s necessary to do the routine work when determining Cost of Delay, and there is enough material available to assist in this. The more work that is put into determining Cost of Delay, the more exact the values become and the range of values will become smaller—the forecast becomes more accurate. The work required is different with new products. The huge starting range requires asking questions and investigating why a new product makes sense and produces value. The whole arsenal of user, market and product research exploration (Fuzzy Front End), etc., can be used. Ultimately, you learn more about the customer, their needs and their interactions. The purpose is to get information as quickly as possible through preferable frequent and cheap interactions (experiments) with the customers. The funny thing is, these little tidbits of information have great value when dealing with large uncertainty. They help us to better approximate Cost of Delay.
Klaus: Isn’t there a danger, that precise measurements will always win when compared to abstract estimations with larger range?
Markus: Yes, absolutely! Cost of Delay from Fuzzy Front End projects will be measured completely different than those from production projects. The discussion is more abstract, but this is exactly the reason why they should be had and a consensus about the abstract estimations should be achieved. It’s also important that the differences in the estimations are understood and forced. Otherwise, precise and safe proposals will always win over the uncertain, abstract ones. This is the death of renewal and innovation. The value of a new product is also derived from the strategy: When we want to achieve something specific for our customers, we must solve specific problems. We know there are needs, but we don’t know how to satisfy those needs. When we satisfy those needs, there also should be measurable effects. Ultimately, the uncertainty, or the complete lack of knowledge at the start, drives us to search for the right questions when determining Cost of Delay. Asking questions is the common denominator. In every case, you must allow for different perceptions and, above all, see the value in the various opinions.
Klaus: We’re talking the whole time about assumptions and ranges. How can you actually verify whether or not the correct decision was made?
Markus: That’s an interesting question. Eric Ries would tell us, that we simply need to build our model, based on the definition of product success, and then measure it (Ries, 2011). In reality, it’s quite often not that simple. In your example of process optimization in billing, it is relatively easy since the previously accrued costs could be compared with the actual outcome. With anything new, it’s difficult, because, for example, new features must be explained and learned and must be established in the market. In addition, they have side-effects on other product parts and these effects are not linear. Even a Killer-Feature does not necessarily have a linear increase in the number of new users, or result in customers willing to pay more for it. It’s possible the customers simply stay with what they know out of convenience, instead of trying something new. In some cases, this can only be measured after a long time, like a year. It’s similar in marketing. It’s obvious that marketing is important, but the relationship between the expenditures for each marketing channel and their impact can sometimes be difficult to measure. Even though there are better tracking tools for this now, much remains unclear.
That is why I make the case for a more probabilistic approach. We carry out measurements at wide intervals and compare our success over this larger timeframe based on the important statistics. This way, instead of evaluating our portfolio from a detailed local perspective, we evaluate our portfolio from an across-the-board perspective over a period of time. Product managers and developers can certainly contribute a more detailed point of view. At the portfolio level, however, we should simply force ourselves to measure at a higher level. When we make well-grounded decisions, usually we are better off than always making decisions instinctively. This also helps us to see proposals as building blocks and experiments, and to avoid placing the blame for failures on the executor of the proposal. Especially since the decision to do something was made at the portfolio level.
Klaus: Cost of Delay and risk estimation are also not the one-size-fits-all approach with which everything will be okay?
Markus: That would be even nicer—a tool that would solve all problems. Even better, of course, would be an all-encompassing theory that could be used to simply calculate correct decisions. However, deliberate assessment of proposals using Cost of Delay and risk estimation is very powerful and a common tool for continuous learning within a company. Companies who are open to continuous learning, should look more closely at these concepts. Cost of Delay and risk estimation begin at the portfolio level, i.e. at the place where vision and strategy are put into operation, and continue all the way through to the feature level. The portfolio level is where the substance, or character, of the company is determined. This should be a very deliberate process, and has very little to do with guessing games. Cost of Delay and risk estimation help when discussing the substance of the company because they uncover assumptions, and they help us ask the right questions at the places where it’s not yet working. All in all, companies who consistently utilize the concepts of Cost of Delay and risk estimation profit immensely on consistency, trust and efficiency.


This interview is part of my book Practical Kanban. Get it on Leanpub now!

No comments
This conversation lacks your voice:
Your E-Mail Address will not be published.