Data – We Collect It…Then What?
Rebecca Harris, M.A., CAE
Executive Director, Ontario Association of Residences Treating Youth
I’ve been thinking about data.
Associations are good at collecting data. We send out member value surveys, event and conference evaluations, and many other evaluations and surveys. But are we doing anything with the data other than a cursory analysis and a ‘one time’ report?
Take for example an event evaluation, we all do them, but are we overlooking what the data has to tell us? Oftentimes, the data is collected via survey post-event and a wrap up report is created that gives us a snapshot of how the event performed. What we often miss is doing a deeper dive into the data to determine the relationships, patterns, and trends that can be found within. The data we collect can be segmented and examined in various ways to give us a deeper glimpse into our delegate’s minds and our event’s performance. For example, do we segregate the final data by various stakeholder types to determine if the effectiveness of the program varies by user group? If we do this, are we looking at more than member versus non-member results? Do we extrapolate and examine data based on volunteer ratings versus non-volunteers? Long time members versus new members? Membership type/segment? Age group?
Sometimes connections and correlations exist within our data that we are not taking into consideration. That’s not to say that one variable causes the other or that the two variables have the same cause but rather that certain correlations can give us an idea of what to expect (a trend) or can give us greater insight into how our programs are performing. You may find that when you segregate volunteer responses from the overall evaluation results for example that volunteers rate the program higher. This may be due to several factors, including but not limited to a greater sense of connection to the association and higher engagement levels. If you then look at the remaining data you will have a better idea of how non-engaged/less engaged delegates rate your program, and if it is indeed lower you can then analyze your data further to find the cause or at least form a hypothesis of why this may be true. For example, if non-volunteers rate the program on average 2 points lower than volunteers and the mention of “not enough networking” shows up numerous times in your quantitative data, then you may surmise that the delegates who do not already have formalized networks within the membership due to their volunteer activities need assistance in networking at your event or may benefit from a more formalized networking component.
Almost all of us collect data on a regular basis but we don’t always combine data sources and look for trends over time. If we do look for trends, in say our conference evaluations, we often simply look at the overall ‘event satisfaction score’ to determine if our programs are improving as opposed to analyzing it on a more granular level. If we see our event satisfaction scores improving do we know why? Are we looking to our data to see if there are correlations between topics, speakers, locations, etcetera?
The need for greater overall analysis of our data exists. There are important metrics and trends lurking in our forgotten survey results we just need to find it. By taking a more in-depth look into all of our data on a regular (or semi-regular) basis we may find association rules between data points; i.e. if a member buys X then they also buy Y, or if a delegate likes speaker X then they also like information on Y.
Our data is one of our most valuable assets and dedicating the time and resources to data analysis will allow us to transform disparate data points from an unwieldy pile of information into a valuable tool and resource. Starting small and deciding what you want to measure is the first step. Are young professionals a key stakeholder in your association? If so, do you know what they want? If you have event or member value survey data that includes a breakdown by age you may be able to determine what you are offering of value to this particular segment, which in turn will assist you in improving your programming for this segment and in marketing your products and services to them. If you are not including basic demographic data collection in your surveys you may want to consider adding some questions to the end of your surveys for the purposes of this type of data analysis in the future.
At a bare minimum we should be determining our key stakeholder segments and ensuring our data collection, analysis, and reporting is taking these segments into consideration. It’s time to dive deeper into our data to find key correlations and trends.
I’ve been thinking about data and realizing that we can do more.
I’ve been thinking about data and realizing that I need to revamp my data collection processes and analysis to ensure I am getting the information I need to monitor the outcomes that are important to my association.
Rebecca Harris, CAE is the Executive Director of the Ontario Association of Residences Treating Youth (OARTY) and has been working in the association sector for over 10 years. She is a Certified Association Executive (CAE) and possesses an MA degree in Art History and BFA degree in Fine Arts. Follow Rebecca on twitter @beckymharris
