How to build a Roadmap – Prioritize (Part II) | Applied Enterprise Architecture

An earlier post in this series (How to Build a Roadmap) summarized the specific steps required to develop a well thought out road map. This roadmap identified specific actions using an overall pattern ALL roadmaps should follow. The steps required to complete this work:

Develop a clear and unambiguous understanding of the current state
Define the desired end state
Conduct a Gap Analysis exercise
Prioritize the findings from the Gap Analysis exercise into a series of gap closure strategies
Discover the optimum sequence of actions (recognizing predecessor – successor relationships)
Develop and Publish the Road Map

This post will continue with the prioritization steps we started with in How to build a Roadmap – Prioritize (Part I). The activity will use the results from steps 1 – 3 to prioritize the actions we have identified to close the gap or difference (delta) from where we are to what we aspire to become. In short, we want to IDENTIFY what is feasible and what has the highest business value balancing business need with the capability to execute.

Prioritize Gap Analysis Findings
After the gaps analysis is completed further investigation should be performed by a professional with subject matter expertise and knowledge of generally accepted best practices. It is best to use prepared schedules in the early field work (if possible) to begin gathering and compiling the facts needed during the interview processes to mitigate “churn” and unnecessary rework. This schedule will be populated with our findings, suggested initiatives or projects, expected deliverables, level of effort, and a preliminary priority ranking of the gap identified. At this point we have a “to-do” list of actionable activities or projects we can draft the working papers with for the prioritization review process using a variety of tools and techniques.

The prioritization review is an absolutely essential step prior to defining the proposed program plans in the road map. Scheduled review sessions with key management, line managers, and other subject matter experts (SME) serves to focus all the major activities for which the function or subject area is responsible. We are asking this group of stakeholders to verify and confirm the business goals and objectives as value criteria and to score the relative importance of each gap identified. The following section describes the available techniques but is by no means an exhaustive survey of what is available.

Prioritization Methods
There are a number of approaches to prioritization that are useful. There are also some special cases where you’ll need other tools if you’re going to be truly effective. Many of these techniques are often used together to understand the subtle aspects and nuances of organizational dynamics. The tools and methods include:

Paired Comparison Analysis
Paired Comparison Analysis (sometimes referred to as Pairwise Comparison) is most useful when the decision criteria are vague, subjective or inconsistent (like we find most of the time). This is not a precision exercise. The analysis is useful for weighing up the relative importance of different options. It’s particularly helpful where:

priorities aren’t clear,
the options are completely different,
evaluation criteria are subjective, or they’re competing in importance.

We prioritize options by comparing each item on a list with all other items on the list individually. By deciding in each case which of the two is most important, we can consolidate results to get a prioritized list. This can be especially challenging when the choices are quite different from one another, decision criteria are subjective, or there is no objective data to use. This makes it easy to choose the most important problem to solve, or to pick the solution that will be most effective. It also helps set priorities where there are conflicting demands or resource constraints that need to be considered.

Grid Analysis
Grid Analysis helps prioritize a list of tasks where you need to take many different factors into consideration. Grid Analysis is the simplest form of Multiple Criteria Decision Analysis (MCDA), also known as Multiple Criteria Decision Aid or Multiple Criteria Decision Management (MCDM). Sophisticated MCDA can involve highly complex modeling of different potential scenarios, using advanced mathematics. Grid Analysis is particularly powerful where you have a number of good alternatives to choose from, and many different factors to take into account. This is a good technique to use in almost any important decision where there isn’t a clear and obvious preferred option. Grid Analysis works by listing options as rows on a table, and the factors you need consider as columns. Conversely you can populate the rows with business drivers and the rows with technical factors. The intent is to score each factor combination, weight this score by the relative importance of the factor, and add these scores up to give an overall score for each option.

Action Priority Matrix
Closely related to Grid analysis is the Action Priority matrix. This quick and simple diagramming technique plots the value of the candidate tasks against the level of effort or technical feasibility to deliver. By doing this we can quickly spot the “quick wins” giving you the greatest rewards in the shortest possible time, and avoid the “hard rocks” which soak up time for little eventual reward. This is an ingenious approach for making highly efficient prioritization decisions.

I illustrated this approach in detail in the first part of this step in How to build a Roadmap – Prioritize (Part I). This earlier post described how we take relatively “subjective” set of inputs and prepare a simple quadrant graph where technical feasibility and business value represent the X and Y axis respectively. This quantifies the relative priority of each candidate set of processes impacted subjected to technical feasibility constraints. The result is an easy to see visual representation of what is important (relative business value) and technically feasible. In short the method (as summarized here) has helped us prioritize where to focus our efforts and aligned business need with the technical capability to deliver.

Urgent/Important Matrix
Similar to the Action Priority Matrix this technique considers whether candidate tasks or projects are urgent or important. Sometimes, seemingly urgent gaps really aren’t that important. The Urgent/Important Matrix helps by using the gap analysis task list and quickly identifying the activities we should focus on. By prioritizing using the Matrix, we can deal with truly urgent issues while working towards important goals. The distinction is subtle but important. Urgent activities are often the ones we concentrate on; they demand attention because the consequences of not dealing with them are immediate. “What is important is seldom urgent and what is urgent is seldom important,” sums up the concept of the matrix perfectly. This so-called “Eisenhower Principle” is said to be how Eisenhower organized his tasks. As a result, the matrix is sometimes called the Eisenhower Matrix..

Somewhat related to this technique is the MoSCoW method. This is named because the acronym is related to grouping the alternative actions into four discrete types.

M – MUST: Describes an action (or requirement) that must be satisfied in the final solution for the solution to be considered a success. In other words not a high, non-negotiable priority.
S – SHOULD: Represents a high-priority item that should be included in the solution if it is possible. This is often a critical action (requirement) but one which can be satisfied in other ways if necessary.
C – COULD: Describes an action (requirement) which is considered desirable but not necessary. This will be included if time, dependencies, and resources permit.
W – WOULD: Represents a requirement that stakeholders have agreed will not be implemented in a given release, but may be considered for the future (sometimes the word “Won’t” is substituted for “Would” to give a clearer understanding of this choice). This is usually associated with low priority actions.

The o’s in MoSCoW are added simply to make the word pronounceable, and are often left lower case to indicate that they don’t stand for anything. Most often used with time boxing where a deadline is fixed so that the focus can be on the most important actions or requirements. This is often used as a core aspect of rapid application development (RAD) software development processes, such as Dynamic Systems Development Method (DSDM) and agile software development techniques. In this context we are “borrowing” from application development to quickly arrive at a priority scheme or relative measure of how important each proposed action is to the business.

Ansoff Matrix and the Boston Matrices
These give you quick “rules of thumb” for prioritizing opportunities uncovered in the gap analysis. The Ansoff Matrix helps evaluate and prioritize opportunities by risk. The Boston Matrix does a similar job, helping to prioritize opportunities based on the attractiveness of a market and our ability to take advantage of it. There is much more information about the Ansoff Matrix and the Boston Matrix at these links.

Pareto Analysis
Pareto Analysis helps identify the most important of the gap analysis changes identified to make. It is a simple technique for prioritizing problem-solving work so that the first piece of work proposed resolves the greatest number of gaps or challenges uncovered. The Pareto Principle (known as the “80/20 Rule”) is the idea that 20% of the gap candidates may generate 80% of results. In other words we are seeking to find the 20% of closure actions that will generate 80% of the results expected that attacking all of the identified work would deliver. While this approach is great for identifying the most important root cause to deal with, it doesn’t take into account the cost of doing so. Where costs are significant, you’ll need to use techniques such as Cost/Benefit Analysis, or use IRR and NPV methods to determine which priority based changes you should move ahead with. To use Pareto Analysis, identify and list the gap candidates and their causes. Score each problem and group them together by their cause. Then add up the score for each group. Finally, work on finding a solution to the cause of the problems in the group with the highest score. Pareto Analysis not only shows you the most important gap to solve, it also gives you a score showing how severe the gap is relative to the others.

Analytic Hierarchy Process (AHP)
The Analytic Hierarchy Process is useful when there are difficult choices to make in a complex, subjective situation with more than a few options. The method (AHP) is included in most operations research and management science textbooks, and is taught in numerous universities. While the general consensus is that it is both technically valid and practically useful, the method does have its critics. Most of the criticisms involve a phenomenon called rank reversal which is described here.

To address this problem, Thomas Saaty created the Analytic Hierarchy Process (AHP) combining qualitative and quantitative analysis. This technique is useful because it combines two approaches –mathematics, and the subjectivity and of psychology – to evaluate information and make priority decisions that are easy to defend.

Simply put ranking is a relationship between a set of items such that, for any two items, the first is either ‘ranked higher than’, ‘ranked lower than’ or ‘ranked equal to’ the second. In mathematics, this is known as a weak order or total preorder of objects. It is not necessarily a total order of objects because two different objects can have the same ranking. The rankings themselves are totally ordered. By reducing detailed measures to a sequence of ordinal numbers, rankings make it possible to evaluate complex information according to certain criteria. This is how an Internet search engine may rank the pages it finds according to an estimation of their relevance making it possible for us to quickly select the pages we are most likely interested in.

Results
I think you can see there are a couple of techniques we can use to prioritize the findings from our gap analysis into a coherent, thoughtful set of ordered actions we can take. Armed with this consensus view we can now proceed to step five (5) to assemble and discover the optimum sequence of actions (recognizing predecessor – successor relationships) as we move to developing the road map. Many of the techniques discussed here can (and should be combined) to capture consensus on priority decisions. My professional favorite is the approach found in the Action Priority Matrix due to its structure and group involvement for making highly efficient prioritization decisions. Now that we have the priorities in hand we have the raw materials needed to move forward and produce the road map.