š Hello everyone. I'm Michael Hall and I will be your guide in Rosieland today.
This is a companion discussion topic for the original entry at https://rosie.land/posts/stop-measuring-community-engagement/
This is a companion discussion topic for the original entry at https://rosie.land/posts/stop-measuring-community-engagement/
2 Likes
Wow such a great post @mhall119, really resonates.
It reminds me of the obsession folks have with email engagement metrics ā opens and clicks primarily. Iāve been telling folks that these metrics are useless and that one needs to measure the impact of the campaigns to see if their audience performed the desired action after opening an email.
I think itās the same story with community metrics ā people feel good about the numbers instead of doing the hard work of measuring the impact on members.
3 Likes
The conclusion to this article undermines the authorās entire point.
In the section āMeasure what you value,ā @mhall119 argues that we can and should measure āmeaningful relationshipsā instead.
Theyāre not as easy to measure as engagement, sure, but they can be measured.
But thatāsā¦ not true at all? There are no real measurements for āmeaningful relationships.ā
In fact, the only proxy metric we have for āmeaningful relationshipsā is engagement!
And, yes, engagement is not a very good proxy metric for meaningful relationships, but since thereās no alternative, itās all weāve got.
1 Like
Something tells me we should be measuring outcomes.
e.g.
- a job board: are people actually getting hired?
- content/courses/podcasts: are people applying what they learn? are they writing or sharing about? are they expanding on the ideas shared?
- conferences: are we helping elevate peopleās speaking careers? do people who attend become better off than those who donāt attend?
etc.
And engagement data can still be useful, but I think the balance is currently off.
5 Likes
Props to Rosie for continuing to elevate the discussion around community and the industry itself - and for continuing to bring more voices into the conversation and help spread awareness about community building. And props to Michael for generating a super lively discussion and thread - both here and on Hacker News!
Our cofounder Tom shared these comments on Hacker News, so wanted to add them here as well as part of the discussion 
Early on in a community, there often arenāt enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all ā measuring engagement is a useful way to begin understanding whatās resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health.
Measuring engagement then becomes one piece of data thatās valuable, but shouldnāt be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), whatās topically important to members and why, and overall sentiment and change in sentiment over time - thatās when community builders can begin to better understand the health of their communities and their impact on their business.
Engagement is an important piece, but just one of the pieces, that helps paint the full picture.
Disclaimer: Iām a co-founder of Common Room and a weāve invested a lot of energy in solving for this exact problem. Youāre welcome to check out the product (itās free to sign up) and would love to hear of ideas or feedback.
1 Like
Amen.
I had this conversation with one of our VPs. We were talking about the KPIs that matter to a community program.
Gist = If you had to choose a single metric to share with the board, what would it be? What would the success story build on?
Engagement and activity metrics are good metrics for CMs to follow. Theyāre indicative. If activity goes down, you should investigate.
But does the activity level matter to the board? Probably not. There isnāt a story there.
Is the community program bringing more people in? Is it converting them to signups? Is it helping them onboard & activate? What about retention? Resolved support issues? Do members have a higher AOV or LTV? etcā¦
Those metrics have a story to tell.
1 Like
Copied from Michaleās Savannah communityā¦
How on earth did you come with such and amazing comparison??? Monsters INCā¦ brilliant!!! Absolutelly trueā¦ iāve been struggling for years now on the feeling that my community needs some more engagement but feeling that the love our community and find a lot of value in it!!! You just gave me peace of mind! Priceless! Iāve been using Savannah for more than a month now and even knowing that i have still a lot to explore and take advantage of, iām pretty confident on what youāre builing here! Iāve havenāt seen anything similar and as sure as i am that communities are the future for most companies, the future is huge!
I really hope you find a way (AI has a lot of potential here) to measure real value in Savannah and hopefully on more languages soon (Spanish in my case )
Kudos for such a great article!! EVERYONE building a community should read it! 
@rosiesherry : Iām now curious on how much traffic is the article getting??? 
And engagement? 
2 Likes
Hah, well you can see engagement in the conversation weāre having right now.
Thereās some on Twitter too:
But the biggest engagement points go to Hackernews:
Traffic below for reference (I have analytics open):
But all the above is āengagementā, right? None of it really matters if it doesnāt help with outcomes, progress, etc.
If this conversation leads to new ideas, new ways of thinking and people making (attempts at) better solutions, then Iāll be one happy bunny!
Itās almost like, yes relationships are important, and @mhall119 has a thing that shows relationships visually.
But maybe we can look past relationships, whilst still valuing them. Maybe community is as much about the ideas and how they evolve through people and conversations.
We are inspired by everyone and everything around us. And I get belonging and relationships are important, we can feel connected to people, their contributions and ideas but not actually have any real visible connection.
Infact, much of my work is built upon not very new ideas. From work from people who will never show up here. I will likely reference people at times, however, other times the lines are blurred when I just donāt know where my ideas come from or remember the source. We are an evolution of our surroundings.
Measuring community should also consider for factors outside of the community. The current measurements are so capitalistic in nature and always seem to take the side of the company, not the complete picture of all the people they are supposed to be serving.
Hah, didnāt mean for this to turn into something this long.
2 Likes
I wrote a guest article for Invision Community precisely about the Engagement Trap (as well as better metrics to potentially measure)
You do NOT need to measure solely engagement. You can measure (and Iām going to plagiarize myself!):
- The number of questions or feedback requests that were answered in high-value boards of functional content
- The number of educational resources that were added to a certain category
- The number of new topics that were posted in a growing section
- The amount of special keywords or tags
- The number of informative reactions that were given out in a certain period
- The participation of high-value experts in your community
If your community platform also has these metrics, you can also measure things like:
- Speed to first response
- Speed to resolution
- Speed to a specific member milestone
These are all metrics that communities are starting to communities are starting to build into their communities.
Then more broadly, you should be conducting external measurements for the community like NPS scores, satisfaction surveys, etc.
1 Like
I think that there is space for engagement metrics as a secondary vector. As a primary metric set, measuring general engagement isnāt very useful and puts undue pressure on CMs, but you still need something to gauge your other metrics against in many circumstances.
e.g. If your community focus is to support students to learn (with a primary objective of selling books or courses) then it is helpful to know how many people are being exposed to those resources and broken down further, how many people that are actively engaging in the community go on to covert to a sale.
The engagement trap is only a trap if youāre not measuring the right things for the right reasons.
2 Likes