In Part One of this series, “Assess, then Search,” I began with a description of the needs assessment phase. For Step Two, I described the actual product search phase. Here, in Step Three, I turn to the evaluation and decision phase.
Sifting through the rubble
Depending on the type of procurement, you may have anywhere from zero to 30 responses to an RFI. It may seem daunting, but there are tricks to sorting through the responses to help you narrow the list to a few key options.
Start with your most firm requirements. Do you need something that allows a certain number of concurrent connections without performance loss? Does each product work with the other applications you plan to keep? Will it adequately accommodate your remote access needs? Does it work with your server OS? Whatever they are, the core “must have” features should drive your first pass through the product materials and RFI answers. Create your short list and set the others aside in case you need to go back to them.
Next, rank the “nice to have” features and start with a weighted rating system. Some product selection committees use the “greatest number of hits” method (where every positive answer gets one point). Others use a weighting and voting formula that assigns a relative value to the features based on importance, then rates how well the solution addresses each item. For example, Mac compatibility may get a relative value of 6, compared to Windows compatibility’s 10. If the product is very strong on its Mac OS compatibility, the most that vendor will get is a 6, whereas if it also is not ideally used on a Windows PC, it might get a 5 or 6. Because the Windows feature is more important, even a 50% rating counts about the same as a 100% hit on the Mac feature.
Do your homework
Armed with a short list, the committee members need to become sleuths, using all available tools to find out as much as possible about the company, the product and the customers who use it. Try web searches (and be sure to browse at least 3-5 pages deep into the results), call references (and ask them for the names of other customers who use the product in your industry who may have not been put on the reference list), contact companies listed as technology partners (find out if the partnership is robust or just passive) and go visit the company’s office (make sure you are dealing with more than a garage project). "Trust, but verify."
If you have the time and budget, go sit with users who already employ the solution in business environments like yours. You can learn more from just one of these trips than you may learn in a product demo by a skilled sales rep. If you divide and conquer the list of customers, make sure each person uses the same checklist and reporting form to get data for apples-to-apples review back in the office.
Structure the demo
With a lot of good background information, it is time to prepare for the product demo. Even if you have seen a generic product demo already, go to the effort to host a quasi-proof of concept. Based on your own expected daily usage, write a script of challenges that you will expect the software to handle once the product has been installed. Explain your goals for each scenario (“to see how fast a typical user can accomplish the task,” for example, or “to see how many efficient ways a user may complete the task”) and the other systems that would normally be involved.
Allow adequate time for the vendors to perform each scenario, followed by specific question & answer time to review the results. Your research will help you target questions as you address any concerns uncovered when talking to other customers, for example, or that have arisen since the RFI responses came back.
Make sure you carefully manage the demo to avoid wasting time on lesser topics to the detriment of discussions on important matters. If necessary, have a facilitator who can make sure the discussion moves around the room and one or two eager participants who may not accurately represent all interests do not dominate the Q&A session.
Consider the consequences
Once the demos are over, reconvene the committee for a frank discussion. The risks you must worry about at this step are typically bias, hidden agendas and ignorance. Some people got on the committee in order to make sure their department’s needs receive priority over others’ needs, while other people may simply have had their minds made up before the entire process began. One of the worst and typically unexpected obstacles for product selection committees, however, is ignorance.
Sifting through the rubble
Depending on the type of procurement, you may have anywhere from zero to 30 responses to an RFI. It may seem daunting, but there are tricks to sorting through the responses to help you narrow the list to a few key options.
Start with your most firm requirements. Do you need something that allows a certain number of concurrent connections without performance loss? Does each product work with the other applications you plan to keep? Will it adequately accommodate your remote access needs? Does it work with your server OS? Whatever they are, the core “must have” features should drive your first pass through the product materials and RFI answers. Create your short list and set the others aside in case you need to go back to them.
Next, rank the “nice to have” features and start with a weighted rating system. Some product selection committees use the “greatest number of hits” method (where every positive answer gets one point). Others use a weighting and voting formula that assigns a relative value to the features based on importance, then rates how well the solution addresses each item. For example, Mac compatibility may get a relative value of 6, compared to Windows compatibility’s 10. If the product is very strong on its Mac OS compatibility, the most that vendor will get is a 6, whereas if it also is not ideally used on a Windows PC, it might get a 5 or 6. Because the Windows feature is more important, even a 50% rating counts about the same as a 100% hit on the Mac feature.
Do your homework
Armed with a short list, the committee members need to become sleuths, using all available tools to find out as much as possible about the company, the product and the customers who use it. Try web searches (and be sure to browse at least 3-5 pages deep into the results), call references (and ask them for the names of other customers who use the product in your industry who may have not been put on the reference list), contact companies listed as technology partners (find out if the partnership is robust or just passive) and go visit the company’s office (make sure you are dealing with more than a garage project). "Trust, but verify."
If you have the time and budget, go sit with users who already employ the solution in business environments like yours. You can learn more from just one of these trips than you may learn in a product demo by a skilled sales rep. If you divide and conquer the list of customers, make sure each person uses the same checklist and reporting form to get data for apples-to-apples review back in the office.
Structure the demo
With a lot of good background information, it is time to prepare for the product demo. Even if you have seen a generic product demo already, go to the effort to host a quasi-proof of concept. Based on your own expected daily usage, write a script of challenges that you will expect the software to handle once the product has been installed. Explain your goals for each scenario (“to see how fast a typical user can accomplish the task,” for example, or “to see how many efficient ways a user may complete the task”) and the other systems that would normally be involved.
Allow adequate time for the vendors to perform each scenario, followed by specific question & answer time to review the results. Your research will help you target questions as you address any concerns uncovered when talking to other customers, for example, or that have arisen since the RFI responses came back.
Make sure you carefully manage the demo to avoid wasting time on lesser topics to the detriment of discussions on important matters. If necessary, have a facilitator who can make sure the discussion moves around the room and one or two eager participants who may not accurately represent all interests do not dominate the Q&A session.
Consider the consequences
Once the demos are over, reconvene the committee for a frank discussion. The risks you must worry about at this step are typically bias, hidden agendas and ignorance. Some people got on the committee in order to make sure their department’s needs receive priority over others’ needs, while other people may simply have had their minds made up before the entire process began. One of the worst and typically unexpected obstacles for product selection committees, however, is ignorance.
You cannot safely assume that everyone in the group came equipped with the same knowledge or comprehends the needs or solutions. Ask. Does everyone know about the platform compatibility issues you have or expect? Does everyone understand the new technologies proposed and how to compare them? Does everyone have a clear understanding of how each vendor proposes to address need X? You cannot rely on the vendors’ ability to sell their solution unless you are willing to take the risk that you may not end up with the best result.
No comments:
Post a Comment