7 Steps Needed for Successful Benchmarking, using the COMPARE Method

Successful benchmarking tends to have at least one thing in common: a structured approach. We help associations to conduct benchmarking among members successfully, efficiently, effectively and with massive value. We developed the COMPARE method to help you in using a structured approach.

Benchmarking among members is very common for professional and trade associations. Unfortunately, I have seen many benchmarking efforts, for even many reasons. When I analysed them, I could bring it back to three main reasons: lack of interest of members, non-relevant content, too much focus on mean. Finally, I could bring it back to only one reason: lack of a structured approach.

That is why I created the COMPARE method. To help associations conducting successful benchmarking by using a structured approach. Advantage of such an approach is that associations follow a step by step method, in which each following step only can start when the former step has been finished. And in finishing each step, you get a clear view on the quality of the (potential) benchmark. Other advantage is, that if you follow the steps as described, that you create a successful benchmark, which is relevant and valuable for your members, and for your association.

Difference between professional and trade association benchmarking

Professional and trade associations both conduct benchmarking among members, but do it in a different way. Trade associations benchmarking mostly results in individual reports to each participant, including their own data and for each participant relevant reference group data.

Participants can compare themselves easily, see their performance gap and start creating actions to decrease the performance gap. In addition, the association creates an overall market analysis and report with overall and several reference group performance analyses. The goal of this kind of benchmark should be helping the members improve their performances.

Professional associations benchmarking mostly only results in a market analysis and report. For professionals, it is not directly interesting and relevant to see what they have answered themselves. But the overall picture could be very important for the professional.

The goal of this kind of benchmarking is informing professionals with best practices, so they can make better decisions and improve the quality of their profession.

For both kinds of benchmarking, associations use the same method.

COMPARE method

The COMPARE method consists of the following 7 steps:

1. consciousise
2. organise
3. measure
4. picture
5. analyse
6. realise
7. evaluate

See the 7 steps as a wheel. When you have finished the evaluation step, you start again with getting conscious. The steps repeat itself, since you want to do benchmarking as a service, offering many different benchmarks over time, and not as a one-time project.

Let’s focus on the steps in more detail.

Consciousise

I know, consciousise is not a proper English word. But it sounds much better than ‘getting aware’. Getting aware is very important when you start with benchmarking. You should ask yourself WHY you want to do benchmarking? What is your reason to do benchmarking? Have your members asked for it? How many members have asked for it? Do you want to create additional revenues? What is your goal for benchmarking?

Be aware that benchmarking can only be successful if your members have asked for it or if you see that they struggle and think that offering benchmarking could help them. Compare it to the introduction of a new product. Most products are created to solve a problem or to satisfy a demand.

Sometimes products have been introduced for problems we weren’t aware of. If Henry Ford had asked his clients what they wanted, then they would have responded a faster horse. Unfortunately, benchmarking does not fit in this category. So, it has to create a gain or solve a pain. That is why you need a member demand. Not from all members, but you must have heard at least a couple of times a demand for market information or a problem for which you think benchmarking could be the solution. From there, you can ask directly some more members if they would be interested. This could be done by phone, but also by a small member survey. Be sure to ask them at least the following questions:

1. Would you like to see your organisations/individual (professional) results compared to your peers? Comparison could include salaries, financial and operational performance, capabilities, etc.
2. Would you like us to organise and conduct benchmarking surveys among you and your peers?
3. What is keeping you up at night concerning your profession/organisation?
4. What are the three most important questions you would like to see in a benchmark?
5. What are the three most important KPI’s you use in your management information (not for professionals)
6. What would you like us to benchmark?
7. What other services relating to benchmarking would you like us to arrange?

Make sure that you only ask question 3-7 if they have responded that they would like a benchmarking project.

Only if in your opinion enough members have indicated that they would like you to do benchmarking for them, and only if you think that you are the best party to conduct this benchmarking, then you can proceed to step 2, organise.

Organise

Preparation is 90% of success. That is a common statement for most work done. It also applies to benchmarking and what I call ‘organise’. Organisation includes creating the right questionnaire, finding partners, preparing communication.

But is starts with creating a team within your association. Who will be responsible for the benchmark? Who else will be in the team? What are their capabilities? Do they have time to conduct the benchmark? It is important that you have at least the following capabilities within your team: marketing (you have to sell the benchmark), data analysis (you have to analyse the data provided to you), report writing (you have to write a market report with valuable and relevant information. And you need to have the capacity to ask the right questions and to ask the questions right.

You do the capacity analysis with your team. If you see that you lack some capabilities, you can see if you can hire this capability outside of your association. There are many benchmarking consultants, data analysts and marketing professionals outside, for you to hire. And if you feel more comfortable by outsourcing the project or part of the work, then you can start finding an outsourcing partner.

Together with the team you prepare the benchmarking project. A thorough preparation starts with filling a benchmark model canvas.

This canvas is retrieved from the business model canvas, created by Alexander Ostenwalder & Yves Pigneur¹ . It contains the 9 building blocks of a thorough preparation of a benchmark. First fill in the why and the goal of benchmarking.

1. Member segments: What are the member segments you want to serve with your benchmark? To figure out what your members truly need from you, you need to find out who they are. Are all members subject of a benchmark? Or do you have more member groups, for which several benchmarks might be interesting? What do you know about these member groups? What are their pains? What are their gains they are looking for? How are they organised? Who has asked for market information, comparisons, benchmarks?
2. Value proposition: What products and services do we create for the member segments? How do these products and services relieve their pains, create gains and help them organise better? What will be the benefits and the benefits of the benefits for the member segments?
3. Channels: How do we deliver the benchmark? Do you want a paper report? Do you want an online survey? How do you add value in delivering the benchmark?
4. Communication/relationships: How do we communicate with the member segments about the benchmark and their demands? How do we include them in the process of successful benchmarking?
5. Revenue streams: How do we use the benchmark to get more revenues? Do we want a free benchmark for all or for some? What kind of pricing could we use?
6. Key resources: Which key resources do we have/need for the benchmark? Think of key resources needed for communication, revenue generation, delivery. This could include specialist staff members, tools, financing, etc.
7. Key activities: What are the most important activities to be done? First on a high over level, later more in details. Think of creating a relevant questionnaire, motivating communication, data validation, reporting. If you are going to use tools, then think of implementing these tools.
8. Key partners: What are the most important partners in this model? This could be suppliers of tools, outsourcing or staff. This could also be partners who collaborate in this project, like other associations.
9. Costs: What are the most important cost factors and what are their expected costs? Include expected hours of staff, licences for tools, costs for outsourcing, cost for communication, cost for revenue generation, etc.

Start with filling the benchmark model canvas on a higher level, to create a complete picture of the project. Then work out each block in more detail. This gives a valuable insight in all that is needed for successful benchmarking.

The benchmark model gives you a clear picture of your benchmark project. Now you are ready to select your partners, and get the recourses you need.

• If you want to outsource the benchmarking of part of it to a third party, you will have to find an outsourcing partner. Decide about what to outsource and use your purchase procedures to find a good one. They will do most of the upcoming work, albeit together with your team.
• If you want to do the benchmark yourself and use a sophisticated benchmark tool or just a survey tool, then it is time to find the right tool. Use your purchase procedures to find the right tool and corresponding services. Keep mobile in mind, to be able to conduct the survey also on handhelds. Perhaps some of the upcoming work can be done by the tool provider.
• If you need additional team members, like specialists, to help you, then find them now. Use your purchase procedures to find the right consultant.

Together with your team and benchmark partners, you create a relevant questionnaire, or questionnaires. This includes qualitative and quantitative questions that give the participating members insight, when they compare themselves during reporting. For quantitative questions think of data points and corresponding key performance indicators. A questionnaire for professionals is a total different questionnaire then one for organisations. So, keep in mind for whom you create the questionnaire. Also keep in mind that the questions must be relevant to them.

Set data quality improving validation rules. These rules are necessary to check each submitted questionnaire on quality of data. The amount of validation rules highly depends on the types of questions and the length of the questionnaire. Quality questions don’t need to be validated. Quantity questions could be subject to validation.

Last but not least, it is important to prepare all communication. Create a special website for the benchmark, if applicable. Create all the emails, newsletter articles, promotion material. Make sure it included the ‘what’s in it for me’, since you have to convince your members to participate and submit data.

Implement the tools you decided to use and test these thoroughly.

Measure

When you finished the implementation and are sure everything works as expected, then you can start measuring. Measuring is all about getting data and ensure the quality is top.

Send invitations to the targeted member group(s). Make sure all other channels are also set in place. If others than the invited members are also able to participate, update the website with the link to the survey. Announce that the benchmark is open for data entry.

Check regularly how much participants you have and act upon it. If there are less participants than expected, send reminders to those who have not responded yet.

Depending on the necessity of validation, validate each submitted questionnaire. I’ve seen several quantitative benchmark projects fail, due to a lack of validation. You cannot rely on the data submitted directly. You need a kind of validation check, to ensure the data is of good quality. If you find a questionnaire of a participant with potential errors, then ask the participant for explanation and act correspondingly.

Make sure you have enough participants with validated questionnaires. As long as you don’t have enough qualitative participants, then you are not ready to move on to the next step.

Picture

Now that you have all the data and you have checked and improved the quality of the data, you are ready to create a picture of the results.

Download all data to Excel or an analysis tool and look for relevant reference group possibilities. A reference group is a group of participants with more or less the same results on a specific question in the questionnaire. This could be region, size, age, function, department, % of costs, etc. For example, age 15-24, 25-34, 35-44, 45-54, 55-64, 65 and up, or region north, mid, south or by state or country.

Make sure each group has enough participants to be able to say something about the group. A group with only two participants is too small, then you disclose almost individual data. A group should at least contain 10 participants, but sometimes less participants are defendable. If groups are to small, then combine them or do not include them at all.

Create a book of tables of the statistical values of each group. These values include number of responses, mean, structure, quartiles, deciles, highest, lowest, sum. The values needed differ per question. For a qualitative question, you need the number of responses and structure. For key performance indicators, you need mean, quartiles, deciles, highest, lowest and number of responses.

Now you have a complete picture of the results of the benchmark. And you can proceed with analysis.

Analyse

Group results need to be analysed to be able to discover and explain differences between the different groups. It could be quite interesting to see which group has the best results. This goes for individual performance benchmarks as for general benchmarking studies. In individual performance benchmarks, it could be valuable to discover which participant has the best overall performance, creating a kind of best practice.

If you see strange group values for certain questions, then go back to the individual data to see if mistakes have been made or go back to syntaxes of calculations to see if the calculation is correct. For example, if you see that the lowest value on hourly tariff is 0, or 10, then this is too low. Find the participant(s) with this erroneous data and change (delete, amend) the input. Consult the participant if necessary. Off course, you should have done this in the validation step, but my experience is that it is always possible that you miss one or two.

Determine for yourself what would be interesting to include in the report. This can include the list of questions, the relevant reference groups and explanation of differences.

Realise

In the realise step you write and create the reports. Write the general (market) report with the results of the analysis step. Determine which questions you discuss on which page and which tables and graphs to include. Try to combine questions, data points and key performance items that relate to each other on the same set of pages.

In performance benchmarks, also create the individual reports for each participant. The individual report for a specific participant contains its own data and the group values of a relevant reference group. Send the report(s) to the participants.

If you use specialised benchmarking software, then you decide on the report templates, and let the participants create their own reports with relevant data points, and group values of reference groups of their choice.

If you decided to offer additional benchmarking services, like training, individual analysis, roundtable discussions, webinars about results, then implement these and offer them to the participants.

And finalise your communication. Communicate about the results with press releases, summaries and possible downloads on the website (public or private membership area).

Evaluate

The last step is evaluation of the total benchmark project. Sit together with you team to discuss what went well and what went wrong. Be honest, since evaluation is to learn and improve.

Ask participants about their experience and feedback. You have the obligation to ask the participants, since they took the time and effort to participate. Have they experienced the value they expected? Is the benchmark relevant for them? What made them participate? What was difficult for them? Which questions would they advise to include the next time? And which could better be left out of the benchmark?

Ask non-participants why they did not participate. You have to know this feedback from your members to adjust your approach next time. Sometimes it is just words in your communication that did not work well. Sometimes they just don’t want to spend time on it. Or they don’t see the value for them.

Store all feedback securely, to use it in the next version of the benchmark.

Wrap up

I have discussed 7 steps needed for successful benchmarking. These steps are applicable in each benchmarking project, but the thoroughness per step will differ per benchmark and per version of the benchmark you conduct. The more you repeat the benchmark, the easier it becomes and the smaller each step will be.

There is much more to these 7 steps then I can explain in this article. But if you use these steps, in the provided order, then you will certainly create a successful benchmarking project, in time, in budget and in number of participants, over and over again.

Just give it a try. Good luck.

¹ Osterwalder & Pigneur (2010), Business Model Generation: A Handbook for Visionaries, Game Changers, and Challengers, John Wiley & Sons Inc. Hoboken New Jersey