maturity

SCM Test Plans – Prepare

In our last post introducing a world-class SCM testing strategy, we considered the importance of a collaborative approach when testing enabling technologies in complex SCM environments. A hands-off mindset in such contexts is naïve at best, generally leading to frustration and sub-optimal solution value, if not outright failure. Experienced SCM professionals recognize the nature of this problem and adopt a comprehensive, collaborative strategy for ensuring success.

Four distinct types of tests are relevant when addressing this challenge, corresponding to four, distinct SCM project phases; each kind of test plays a necessary role in ensuring project success.

The first type of test recommended in complex SCM planning scenarios is the UFT – the Unit Functional Test – and enables the Prepare project phase, prior to Design and Construct. The UFT:

Tests a specific solution feature on a small, static data set with a known result;
Might use dummy data rather than realistic production data;
Is useful for explaining and illustrating functional solution behavior, both in isolation and in limited combinations, as well as solution design, data and configuration requirements.

UFTs can be stand-alone, independent of any given software implementation, so they are easily developed and executed early in the Prepare project phase. These tests are typically short and simple, based on small data sets which are relatively easily understood, have a very focused scope, and a known result. They each comprise a simple, clearly-documented workflow and Pass/Fail criterion, such that they can be executed and understood by those who do not yet know the new software.

Due to their simplicity and limited scope, the UFT can serve to familiarize users with the application UI, navigation and workflows, train users in solution-specific terminology and behavior, and clarify data requirements for solution architects and data integrators. Since each UFT scenario tends to focus on a single aspect of the solution using a simple, static data model, developing Pass/Fail criteria also tends to be straightforward.

Further, the inherent simplicity of the UFT implies that there is no reason to reinvent the wheel in creating them; the software solution provider should be able to deliver a relevant suite of UFTs out of the box, ready to go and used as-is, or adapted to current business needs with very minimal effort, requiring an insignificant time commitment in the project timeline for positioning them.

Executing and passing a comprehensive set of UFTs relevant to the current software deployment, demonstrating key solution features and behavior in a small, informal test environment, can be conveniently leveraged to enable super-user training, again with no excessive strain on project timelines or resources, and positioned to define a key milestone for the completion of the Prepare phase.

Since UFTs should seldom fail, executing a UFT test suite tends to build momentum and confidence in the new cutting-edge technology through hands-on experience, and provide valuable, practical business-critical insights into the capability and value inherent in an SCM enhancement project early in the implementation, avoiding misinformation and confusion, alleviating fears, and thereby facilitating change management and encouraging solution acceptance and adoption.

If a UFT does happen to fail, which might be due to a solution defect, but more often than not is rooted in incorrect or inaccurate data or solution configuration, the earlier this problem is detected and resolved the better, which is all the more reason to execute this type of test early in the project timeline.

Knowing that all required solution features are represented in the UFT test suite, and that all of them have passed as a milestone to complete the Prepare project phase, simplifies resolution of the problems inevitably encountered in more complex scenarios involving combinations of features and complex interactions with less clearly understood data, and may, in fact, avoid them altogether through aligned user expectations, correct solution design, and thorough data configuration, cleansing and validation.

Finally, and significantly, well-designed UFTs also enable business users to understand the importance and protocol of the overall testing and issue resolution process, and familiarize them with the required test plan templates and issue reporting and resolution workflows and protocols.

The value of Unit Functional Testing during the Prepare phase is significant, but not readily apparent to those unfamiliar with the difficulty and complexity often inherent in SCM implementations. Engaging users in this activity early in the project with simple tests equips them to engage more fully and efficiently in creating and executing the remaining types of tests leveraged in SCM testing methodology in subsequent project phases, as they take on the next test type, the IFT – the Integrated Functional Test – during the Design Phase, which we describe in detail in our next post.

Value-Driven Business Reporting

Business Intelligence (BI), has been corporate center stage for a while now, but experience tells us that creating a new report for a business and getting it right can be immensely challenging, time consuming, and frustrating.

More often than not, those asking for a new report don’t fully understand what they want, or why they want it, or exactly what to ask for, and those providing the report don’t understand much more. The result can be a number of painful iterations, and significant wasted time and effort. In most IT departments, this type of waste is expensive, in both opportunity cost and monetary cost. What are some keys that can help us collaborate efficiently to get the outcome we need without all the hassle?

The first question to ask when designing a new report, or sprucing up an existing one, is: What action should this report trigger? When a user views the report, what decision(s) are they looking to make? This is another way of stating the business requirement enabled by the report. What does a user want to do as a result of looking at it? This is, by definition, the purpose of the report. Until this is clear, the need is suspect; answer this question before proceeding with any design.

And if we understand what a business requirement is, then we also realize that every well-defined business requirement drives a valuable business behavior, measured by a business metric. This gives the What of our report the needed Why, providing a sense of the business value potential inherent in the report.

Once the purpose and value of the report are clearly understood, the next question to ask is: What data is both necessary and sufficient to inform the user to take this action? The user needs information to act on, not too much and not too little, so determine what this specific information is, what the user needs to know to accomplish their purpose, and how the user is going to interpret this data.

Next, we must source the data, and synchronize it with the user’s purpose. Locate or envision the data store, determine who should be responsible to maintain this data and technical infrastructure, and specify when the data is needed and how often it must be refreshed, so that we understand supporting system requirements to keep the input data up-to-date and accurate. We may determine from this research that the data is not available in any of our systems, that it will be difficult to acquire and maintain, and/or that the user will need to interact with the report to supply additional tribal or business knowledge to get the desired outcome. We may also find that the data is sensitive or crucial to our business, such that special permissions are needed to see or edit it, which implies more difficulty and cost to develop and manage the report.

The next step is to determine how best to present the required information so that the report enables the user to understand what needs to be done with the least possible effort in the shortest amount of time. This typically involves minimizing the number of screens the user needs to view, the amount of scrolling needed, and the number of mouse clicks required to obtain and correctly interpret the relevant information. This usually means maximizing the use of screen real estate by compacting column headers, smart filtering, and eliminating unnecessary detail. Using colors, graphs and pictures to draw the eye of the user to essential details is often a must, and it’s smart to assume the user understands the report and how to use it, rather than designing for a first-time experience, since this will most often be the case and allows for much more efficiency and elegance in the report layout.

With a concept for the report layout and content in mind, it’s time to create a quick and dirty mock up for the users so they can make tweaks and corrections before investing in the final product. There’s nothing like seeing to put things in perspective, especially for those of us with a more visually-oriented learning style.

Finally, now that we understand the data sourcing and reporting requirements, we begin to understand the cost of the creating and maintaining the report, which ought to be justified by the report’s value. So here is where we get ask and answer the key fiscal question: Is the value we expect from using this report worth the cost we expect in developing and maintaining it?

If the expected cost of the report is warranted by the expected value, if the payoff appears to be worth the investment in time and company resources, then it’s time to get to work and build it, working closely with the user community to ensure that the report provides optimal value for our business.

And, last but not least, be sure to document how to use the report and the justification for its design, why the report is built the way it is. This helps train new users, keeps the report’s purpose and value in view, and prevents recycling through bad designs as new people inherit and begin using the report. Provide this documentation on line through a Help button built into the report, so that this information is always readily accessible to everyone with access to the report.

Good reporting design and infrastructure makes life easier for everyone, and can bring immense, ongoing value to our business. So let’s be smart about it, and do it right.

Value-Driven Metric Maturity

In this series on value-driven metrics we have looked at the purpose and structure of business metrics, and have identified a number of characteristics of a world-class business metric design. How do we leverage this information to set our sights on actually making this a reality? How do we get started, and make it happen?

Knowing what we need in each metric through a world-class metric definition, and how each component of the metric fits with the others and completes the whole, allows us to identify inter-dependencies and precedence relationships between the metric elements. This insight allows us to sort the requirements into a series of milestones, forming roadmap for developing a complete metric.

Once we understand the roadmap, we have an intuitive way to classify the maturity or completeness of any given metric, and by extension, the overall maturity of a business metric infrastructure.

Rating	Status	Description
1	Conceptual	identified as useful
2	Manual	calculated and reported manually
3	Automated	calculated and reported with no manual intervention
4	Systematic	consistently monitored in a business process
5	Bounded	trigger points define acceptable behavior
6	Actionable	corrective actions are defined and carried out
7	Detailed	business case details are fully documented and accessible
8	Integrated	placed in the global metric hierarchy
9	Tuned	trend/tension analysis maintains globally optimized trigger point(s)
10	Mature	intrinsic to system governance, enables root cause analysis, and full business value is established

By classifying each metric as above and taking a weighted average across all of the metrics required for a business, based on a relative prioritization of their perceived importance, one may derive an overall maturity score for a metrics system and set goals for enhancing overall metrics capability.

A well-defined metrics system is essential for the value-driven enterprise. Understanding world-class metrics methodology is the first step toward realizing it.