The Major Don’ts of Benchmarking by Associations

Knowing how to use benchmarking for performance improvement does not guarantee that businesses will use it. It also does not ensure that people will take an association’s benchmarking activities seriously. The biggest mistake we’ve seen associations make is not understanding their industry. Some of the associations we’ve helped over the years seem disconnected from their members’ concerns. However, what if an association is connected to each member, and familiar with how they work in addition to what keeps them up at night? What other mistakes could an association be making?

Interestingly, there are many mistakes, too many to discuss in this article. That said we’ll highlight some of the most common ‘Don’ts’ for associations in respect to conducting benchmarking surveys. Anyone who is running an association needs to sit up and take notice if their association is doing any one of the things we’ve outlined below.

Don’t Cater to All Members

Many associations want to attract and retain a large membership list. After all large associations become larger and can become an all-encompassing body in the industry. However, in their bid to grow, associations make a mistake to try and serve every member at the same time with their benchmarking service. Trying to do that especially in the context of benchmarking leads to many questions in the questionnaires. It also leads to frustration from members who are not receiving the attention they think is deserved.

If members know how to use benchmarking for performance improvement, they also need to know that focused benchmarking data is the only data that matters. So, the more focused a benchmarking campaign is, the more useful it is going to be.

Example: If an association in the finance industry has members who are tax advisors and financial advisors, the mistake that could be made here is running benchmarks for every data point for both type of businesses. When in fact the one common data point across both is growth in addition to customer satisfaction and perhaps revenue.

The ideal way is for an association to cater to both types of businesses individually by instituting two sperate benchmarking campaigns. The campaigns will have two different types of questionnaires. Though it will stretch the resources of the association too thin for it to be manageable for years to come. Plus, it will be hard to motivate all members to stay on the same page.

Don’t Ask for the Same Data Twice

We see many associations ask members for data that they should have on record or which is easily available online in other data sources. The purpose of collecting data from members is so that the most relevant and recent data is used to compare them to the rest of the industry. However, when you ask for the same data multiple times that causes frustration.

For instance, some associations running financial benchmarks will ask for the company’s annual profits, even though that information is also available in the member invoicing process. The same goes for some data that the participants may have already sent via another survey. So, by asking for it again in another survey, they may not want to provide it again. The chances of members dropping out from the survey increase tremendously as a result.

The results and data from all the previous years should be on record in the association’s database. The only reason why a member should be asked for that data is if they are new or the files were somehow destroyed.

When drafting and reporting on the market, associations can use readily available data, generally available online like statistical data, public records, market data, etc. The purpose of the report should always be to add value to what members can use. So, information that’s readily available online can be used to draw parallels or comparisons.

Don’t Take Too Much Time Collecting Facts

Depending on how data was gathered, the process can end up taking many weeks or even months depending on how many members an association has. Also, on how prompt members are with returning the questionnaire.

The rule of thumb is that more frequently occurring benchmarks mean shorter questionnaires and hence an overall quicker process. However, many benchmarks that matter can’t be run multiple times a year. Though some benchmarks can be running multiple times depending on the industry. So, it is up to the association and its members to decide a way around formulating a win-win situation.

In our experience, an annual benchmark is only useful until it takes place until September of the following year. However, that can make acting on what the benchmarks reveal challenging. There is very little time to implement changes since the year ends in just after three months.

To avoid being stuck with results that can’t be useful the best time to schedule an annual benchmark is at the start of a new year or when it draws to a close. February to March happens to be ideal in our experience. Avoid spending too much time gathering data and validating it. A month should be enough even to analyse and report the data. That way the benchmark should be ready to use by April or May at the latest making it far better than September.

Monthly and quarterly benchmarks require that associations work faster to report within the month. Data collection has to be tightly regulated, and members should be given a strict deadline for submission. Factors like reporting, validation, and analysis should be automated.

Tip: Fact collection, surveys, and other data gathering tactics can be optimized by using the internet, and especially a Benchmark Tool. Use the Tool to streamline the data gathering efforts. It will reduce costs dramatically.

Don’t Sweat Over the Mean

Here is the thing! In our experience, the mean does not mean a lot. As far as benchmarks go, associations make the mistake of assuming and talking about the mean as though it reveals the ultimate truth. The mean without supporting information like a number of participants, quartiles, median, etc. does not reveal the full picture. So, the mean values should never be touted as revealing anything other than averages.

Businesses often mistake the mean for being the benchmark, and so they will aim to do better based on that figure instead of aiming for the best performance. That lowers the benchmark results as a whole.

Associations reporting should include medians, mean, deviation, number of participants and other factors. The benchmarking figures should explain how the data should be interpreted and how to use benchmarking for performance improvement. Mean needs to compare the different reference groups with each other as well as the total market analysis.

Going from Here on Out

Members of most trade associations have no idea of what they want. Benchmarking gives members a tool by which they can improve various facets of their business. However, the abovementioned ‘don’ts’ coupled with others we’ve not discussed can negate the benefits of running benchmarks. So, if you’re going to put in the effort, it should be done correctly so that everyone benefits from it.