Tuesday’s Tip: How To Evaluate Tech Projects For Business Value

Published on May 25, 2010 by R "Ray" Wang

Today’s Evaluation Frameworks Miss The Point

Business leaders seek clarity in understanding which technology solutions drive the most business value.  Today’s frameworks often focus on a combination of:

  • Features – ability to execute, current offering, product depth, etc
  • Direction – vision, strategy, corporate leadership, etc.

Unfortunately, the intersection of features and direction only addresses potential.  Clients continue to express how potential alone no longer provides enough justification in vendor selection or understanding the overall adoption strategy.

Next Gen Evaluation Frameworks Must Start With Business Value

Business leaders seek returns on investment.  IT leaders strive to reduce cost of delivery.  ROI alone may not answer the question as business value needs to be expressed and compared.  Business impact remains the goal in building out new frameworks.  In addition, the cost of technologies must be factored.   As a result, key questions in vendor selection often include:

  • Will this product provide an ROI?
  • Where will the organization realize a business impact?
  • How do various solutions compare in delivering business value?

Organizations Must Balance Business Impact And Cost Of Technology Delivery

Based on conversations with 113 end user clients and vendors, there appears to be a market demand in building out a new framework that compares how (see Figure 1):

  • Business impact extends across the value chain. Business impact extends into 3 levels of maturity from internal to external (i.e. department, cross enterprise, and business value chain).  The greater the penetration of the solution, the greater the business value.
  • Cost of technology delivery measures against percentage of revenue. Both scaling and return on investment reduce the cost of delivery.  Reference data provided via customer references and case studies will provide key data points by industry, size, and geo.

Figure 1. Extent Of Business Impact And Cost Of Technology Delivery Drive Business Value

The Bottom Line – It’s Time To Apply The New Model

Evaluation frameworks of the future must begin the process of identifying the key criteria required for end users to be successful with both their business and IT stakeholders.  A confluence of disruptive business models, rapid adoption and obsolescence of technologies, and demand for business value data drive the need for this new model.   Expect next gen business and technology leaders, research firms, and technology consultants to start today.

Pilot’s Log (Behind The Scenes)

On the buy side, I’ve spoken with over 100 end user customers who remain frustrated with the existing evaluation models.  They are ready to embrace models that measure business value.  On the sell side, I was meeting with SAP a few weeks back to discuss their new industry strategy, which they launched at SAPPHIRE.  When talking about what customers really need, we rekindled this concept of business value and IT value.  In fact SAP is aiming their new industry strategy at providing a healthy balance between the two.  Conversations with other vendors, hint that this concept will soon catch on.   Look for some new evaluation frameworks to emerge from Altimeter this year as we build against these core principles.

Your POV.

Ready to put this model to work in evaluations?  Do you think analyst firms should conduct research using this framework?  You can post or send on to rwang0 at gmail dot com or r at softwaresinsider dot org and we’ll keep your anonymity.

Please let us know if you need help with your apps strategy efforts.  Here’s how we can help:

  • Evaluating tech projects for business value
  • Assessing apps strategies (e.g. single instance, two-tier ERP, upgrade, custom dev, packaged deployments”
  • Designing end to end processes and systems
  • Comparing SaaS/Cloud integration strategies
  • Assisting with legacy ERP migration
  • Planning upgrades and migration
  • Performing vendor selection
  • Providing contract negotiations and software licensing support

Copyright © 2010 R Wang and Insider Associates, LLC. All rights reserved

  • […] Refresh cycles, demanding business requirements, and shift to hybrid deployment options fuel growth for this quarter.  Organizations continue to free up budget to support legacy apps optimization to fund innovation projects.  SaaS will be the predominant entry point for new innovation while organizations seek two-tier approaches to consolidate legacy apps.  The theme for enterprise software customers remains “Show us the business value!” […]

  • Bruce – thanks for your unabashed endorsement of Focal Point (j/k). Actually, I’ve seen the product in use and it does a good job with decision model support. – Ray

  • IBM has a great decision science tool called focal point in their rational software line, it allows for multivariate analysis that is easily configured to the businesses decision model, performs pairwise comparison trade off analysis, uses visualizations to focus teams on impact and value (whatever financial, organizational, strategic etc) and is web based. I would ask anyone to take a look.

  • Ray,

    I sometimes wonder whether ROI is mathematics or an art form. There has been a lot of creative license used by vendors when describing (dubious) ROI benefits!

    There are measurable up-front benefits that can be described in ROI terms exactly as you have described. There are hard-to-measure, possibly intangible benefits that are often impossible to measure up front. So, we shouldn’t use the McNamara Fallacy and assume that these are not important.

    These intangible benefits come into play when technology can transform the business. ROI measurements from the previous generation are often irrelevant. How could businesses in the 50’s do an ROI study on photocopiers or telephones? Or the 70’s on PCs? – that is, with any degree of accuracy.

    So, incremental technology improvement projects such as moving to VoIP from POTS, or to SaaS can have reasonably accurate ROI measurements up-front. These can be tracked and the final ROI can be calculated. As you have shown here.

    Your model gives the perspective of looking beyond narrow ROI by looking at the impact across the organizations and with the entire value chain. This is often missing with most ROI analysis.

    Disruptive technology projects like moving to Web 2-based project coordination internally and with customers, or supporting an internal presence system like Yammer are much more difficult to quantify. Yet, these types of projects can result in more positive returns.

    The dark side of ROI is that it can limit organizations from transforming. These types of technology projects often requires culture and organizational structure changes, hence there can be resistance. ROI becomes the rallying cry of status quo.

  • Anthony – thank you. Decision engines and decision sciences are a great tool. We have to find a way to simplify the user experience but once we do, we’ll be able to embed these tools to support complex decision processes. Expert Choice is one of many good tools out there. Any others people recommend? – Ray

  • Gabriel – very good points. the key will be building the criteria that they can use to come up with the right solutions. Broad short list tools that lead to a “chinese menu” consultative approach seem to work best. You guys at TEC have some great tools for companies to start with – Ray

  • @Scott – I think when management decides to make investments using a putt-putt-putt strategy it takes far too long to play a round of golf, and the score is never in their favor.

  • Hey Ray,

    Have you taken the opportunity to look at the decision sciences / decision making tools out there? I think you’re directly addressing the proper design of decisions.

    Take a look at expertchoice.com – they have a nifty solution for companies trying to evaluate and prioritize in a manner that isn’t as 2 dimensional as cost and benefit.

    Keep up the good work.

  • The value of technology can be different from one customer to another, which makes it hard to measure.

    Also, the disconnection between IT and other departments (sales, manufacturing, HR, etc.) makes things even more complicated.

    You’re not the first to think about this and many project managers with common sense tried it – some succeeded, but most of them didn’t, mostly due to internal problems.

    There are at least two parties involved in an evaluation, selection and implementation process and they are both responsible for success or failure. People, especially decision makers, should stop thinking that the tools or frameworks will do the job for them.

  • Roger – Thanks for providing some additional food for thought. I’m hoping one of our readers will jump in with additional POV and insights! Meanwhile, I’ll be mulling this over the weekend – Ray

  • Ray

    Great to see you pick this up. Had always hoped more would happen around the TEI.

    Just was discussing this general topic with a colleague during the commute (we train in downtown Chicago) and three areas hope you will get to:
    – how to better get to a common methodology around value of technology flexibility (adapting to business change, scale change, extending to new areas, etc). You started to address this in the TEI model, but could mature more. This strikes me as the equivalent of of static vs dynamic analysis by the OMB …. so a bit challenging
    – how business people (and we vendors who hope to sell them something) quantify business risk mitigation in a way that doesnt require one to be an ERM guru
    – how to evaluate the impact of complexity of a technology solution, eg if have to get 27 people from 7 organizations involved vs 3 from 2 organizations, what is the impact on project risk

    Maybe way beyond what you’re thinking, but looking to capture more of the value of cloud



  • Barbara – actually all these cost models are important and may be weighted differently by each client. Type of solution and deployment options will dictate the cost metrics. Look forward to additional feedback! – Ray

  • Scott – happy to help. maximizing value will require a solution to be extended as far as possible into the value chain. However, some solutions just don’t reach or apply into these areas. however, the point is to be able to apply and reuse where possible. – Ray

  • Ray, Interesting idea. Clarification question: “Cost of purchase”, “cost of implementation” and “cost of ownership” vary by product/service and organization. What kind of standardized model are you proposing to calculate those costs?

  • Ray,
    Your point that the breadth of impact throughout the organization is proportional to the value is dead-on, but in my area, manufacturing, many executive teams are reluctant with projects that are seen as broad or wide-sweeping, preferring, instead to limit the scope as a risk-mitigation placebo. This deeply ingrained belief may be part of what’s helping to decay the manufacturing sector.

    It’s part of my job to help spread the message and reassure the decision-makers. Thanks for your insight and efforts – they will help in the future!

    -Scott Priestley

  • Justin – thanks. the TEI framework at Forrester should be combined into part of the Wave Criteria if Forrester is serious about BT =). BTW, some revisionist history is at work with BT, the real founder of BT is Craig Symons from the Giga days. – Ray

  • Debbie – thanks for the comments. We are hearing ROI cycles have moved from 12 months to 3 months. It’s definitely getting compressed! – Ray

  • Ray- great post. There is no question- that the value equation is key to today’s investment. ROI is expected in shorter cycles for sure, the reality is- what you are saying goes beyond and is right on target.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts