Stop Measuring Community Engagement

👋 Hello everyone. I'm Michael Hall and I will be your guide in Rosieland today.


This is a companion discussion topic for the original entry at https://rosie.land/posts/stop-measuring-community-engagement/
2 Likes

Wow such a great post @mhall119, really resonates.

It reminds me of the obsession folks have with email engagement metrics – opens and clicks primarily. I’ve been telling folks that these metrics are useless and that one needs to measure the impact of the campaigns to see if their audience performed the desired action after opening an email.

I think it’s the same story with community metrics – people feel good about the numbers instead of doing the hard work of measuring the impact on members.

3 Likes

The conclusion to this article undermines the author’s entire point.

In the section “Measure what you value,” @mhall119 argues that we can and should measure “meaningful relationships” instead.

They’re not as easy to measure as engagement, sure, but they can be measured.

But that’s… not true at all? There are no real measurements for “meaningful relationships.”

In fact, the only proxy metric we have for “meaningful relationships” is engagement!

And, yes, engagement is not a very good proxy metric for meaningful relationships, but since there’s no alternative, it’s all we’ve got.

1 Like

Something tells me we should be measuring outcomes.

e.g.

  • a job board: are people actually getting hired?
  • content/courses/podcasts: are people applying what they learn? are they writing or sharing about? are they expanding on the ideas shared?
  • conferences: are we helping elevate people’s speaking careers? do people who attend become better off than those who don’t attend?

etc.

And engagement data can still be useful, but I think the balance is currently off.

5 Likes

Props to Rosie for continuing to elevate the discussion around community and the industry itself - and for continuing to bring more voices into the conversation and help spread awareness about community building. And props to Michael for generating a super lively discussion and thread - both here and on Hacker News!

Our cofounder Tom shared these comments on Hacker News, so wanted to add them here as well as part of the discussion :slight_smile:

Early on in a community, there often aren’t enough data points to cover other possible metrics. With that in mind, starting somewhere is better than not starting at all — measuring engagement is a useful way to begin understanding what’s resonating in your community. As you continue building your community, there are many different angles that you should use to evaluate its health.

Measuring engagement then becomes one piece of data that’s valuable, but shouldn’t be the only piece. When engagement data is combined with qualitative community surveys, enthusiasm from members to contribute to a community, clear and timely responsiveness to community needs, membership growth over time and geographies, depth of member interactions (whether across community channels or within specific channels), what’s topically important to members and why, and overall sentiment and change in sentiment over time - that’s when community builders can begin to better understand the health of their communities and their impact on their business.

Engagement is an important piece, but just one of the pieces, that helps paint the full picture.

Disclaimer: I’m a co-founder of Common Room and a we’ve invested a lot of energy in solving for this exact problem. You’re welcome to check out the product (it’s free to sign up) and would love to hear of ideas or feedback.

1 Like

Amen.

I had this conversation with one of our VPs. We were talking about the KPIs that matter to a community program.

Gist = If you had to choose a single metric to share with the board, what would it be? What would the success story build on?

Engagement and activity metrics are good metrics for CMs to follow. They’re indicative. If activity goes down, you should investigate.

But does the activity level matter to the board? Probably not. There isn’t a story there.

Is the community program bringing more people in? Is it converting them to signups? Is it helping them onboard & activate? What about retention? Resolved support issues? Do members have a higher AOV or LTV? etc…

Those metrics have a story to tell.

1 Like

Copied from Michale’s Savannah community…

How on earth did you come with such and amazing comparison??? Monsters INC… brilliant!!! Absolutelly true… i’ve been struggling for years now on the feeling that my community needs some more engagement but feeling that the love our community and find a lot of value in it!!! You just gave me peace of mind! Priceless! I’ve been using Savannah for more than a month now and even knowing that i have still a lot to explore and take advantage of, i’m pretty confident on what you’re builing here! I’ve haven’t seen anything similar and as sure as i am that communities are the future for most companies, the future is huge!
I really hope you find a way (AI has a lot of potential here) to measure real value in Savannah and hopefully on more languages soon (Spanish in my case )
Kudos for such a great article!! EVERYONE building a community should read it! :raised_hands:

@rosiesherry : I’m now curious on how much traffic is the article getting??? :sweat_smile: And engagement? :rofl:

2 Likes

Hah, well you can see engagement in the conversation we’re having right now.

There’s some on Twitter too:

But the biggest engagement points go to Hackernews:

Traffic below for reference (I have analytics open):

But all the above is ‘engagement’, right? None of it really matters if it doesn’t help with outcomes, progress, etc.

If this conversation leads to new ideas, new ways of thinking and people making (attempts at) better solutions, then I’ll be one happy bunny!

It’s almost like, yes relationships are important, and @mhall119 has a thing that shows relationships visually.

image

But maybe we can look past relationships, whilst still valuing them. Maybe community is as much about the ideas and how they evolve through people and conversations.

We are inspired by everyone and everything around us. And I get belonging and relationships are important, we can feel connected to people, their contributions and ideas but not actually have any real visible connection.

Infact, much of my work is built upon not very new ideas. From work from people who will never show up here. I will likely reference people at times, however, other times the lines are blurred when I just don’t know where my ideas come from or remember the source. We are an evolution of our surroundings.

Measuring community should also consider for factors outside of the community. The current measurements are so capitalistic in nature and always seem to take the side of the company, not the complete picture of all the people they are supposed to be serving.

Hah, didn’t mean for this to turn into something this long. :sweat_smile:

2 Likes

I wrote a guest article for Invision Community precisely about the Engagement Trap (as well as better metrics to potentially measure)

You do NOT need to measure solely engagement. You can measure (and I’m going to plagiarize myself!):

  • The number of questions or feedback requests that were answered in high-value boards of functional content
  • The number of educational resources that were added to a certain category
  • The number of new topics that were posted in a growing section
  • The amount of special keywords or tags
  • The number of informative reactions that were given out in a certain period
  • The participation of high-value experts in your community

If your community platform also has these metrics, you can also measure things like:

  • Speed to first response
  • Speed to resolution
  • Speed to a specific member milestone
    These are all metrics that communities are starting to communities are starting to build into their communities.

Then more broadly, you should be conducting external measurements for the community like NPS scores, satisfaction surveys, etc.

1 Like

I think that there is space for engagement metrics as a secondary vector. As a primary metric set, measuring general engagement isn’t very useful and puts undue pressure on CMs, but you still need something to gauge your other metrics against in many circumstances.

e.g. If your community focus is to support students to learn (with a primary objective of selling books or courses) then it is helpful to know how many people are being exposed to those resources and broken down further, how many people that are actively engaging in the community go on to covert to a sale.

The engagement trap is only a trap if you’re not measuring the right things for the right reasons.

2 Likes