Measure what matters: collect data with intention 

October 23, 2023 / Comments Off on Measure what matters: collect data with intention 

Blog Featured

Data is intentionally collected information. It can be stories, photos, and maps. It can be numbers entered in a database, or demographic information pulled from an intake form. 

Data can reveal patterns, gaps, and progress. It serves as the colors and shapes that move us from confusion to clarity,  bringing stories to life, and providing the details we remember much later.  It can inform program decisions, rally support and widen opportunities for funding & impact. 

Data can also be daunting when we don’t have the time, resources and training. It can be a headache for staff, burdensome to program participants, and a source of ongoing frustration to organization leadership who just want to communicate the powerful work they know is happening(!!).   

In the remainder of this post, we’ll explore some common challenges around data and ideas to address them.

Do any of these scenarios sound familiar to you: 

  • Your organization collects basic program use data, and has for years. The data is the bare minimum and not particularly inspiring or moving. 
  • Your organization collects so much data it’s practically coming out of your ears. It’s time-consuming for staff, burdensome for those you serve and there’s little time or expertise to actually turn it into meaningful insights that add value. 
  • Your organization regularly collects data, such as surveys, however it never seems to be exactly what’s needed. The team plays around with changing questions and methods, hoping they’ll get it ‘right’ one of these days. 
  • You know your organization can use data more effectively, however you’re not sure where to start. You’re small and lack staff with time and data experience. Data never feels like a priority, even though you have an inkling that doing it well could revolutionize your organization. 

If any of these scenarios resonate, you’re not alone. Many small to medium-sized nonprofits (and you’d be surprised–large ones too) –struggle to collect useful data and use what they collect effectively. 

Build an intentional data strategy

I love the signature image from the book Essentialism by Greg Mckeown. A complex, chaotic mass becomes intentional and orderly. Many of the essentialist principles are a great place to start when thinking about data. Our data should not try to ‘do it all’ –rather it should be directed, specific, and aligned with our goals. Here are some ideas to combat the typical scenarios above: 

1. Start with alignment.

Take a deep breath. Now reflect on the purpose of your organization. What is the difference your work is making? How are people (or animals, the earth or whomever/whatever else) impacted by what you do. 

Take a moment to write it down. 

Now take a moment to think about the programmatic data you collect as an organization. Programmatic data is about the programs you and the results (e.g. different from financial or donor data).

Take a moment to consider–is the data you collect helpful to tell the story of your work? Is it helpful in showing if your work is effective and having the results you intend? If you collect a lot of data, what is the most critical metric you currently collect? If you don’t collect much, what is the one metric that would be most helpful in either telling the story of your work, or assessing if your work is accomplishing what you hope? 

Some good news. Oftentimes we don’t need to start at zero when demonstrating our work makes a difference. As an example, when I worked at a child welfare nonprofit, we built our data strategy on the vast research that already proved having a caring adult in a child’s life is a protective factor, which increases the likelihood of educational achievement, health and other outcomes. By aligning our data strategy with existing research, our mission, and everyday work we honed in on what was most important for us to measure and used our resources efficiently. 

2. Only collect data you’ll use.

If you’re collecting more data than you’re using, it’s time to reassess. More data is not better, in fact, more data can be distracting and burdensome for all involved. It takes staff time to collect and analyze data and is disrespectful to collect client data you won’t use, especially if you’re collecting sensitive information or asking for open-ended responses that take time or could bring up discomfort. If you’re collecting data via a survey, long length may detract from completion and accuracy as participants’ time and attention dwindle. While it can be hard to narrow down what data we collect, a good data strategy does just that. 

Go back through surveys or other routine forms of data collection and consider eliminating data you’ve never used and/or is not central to your work. Here are some questions to help decide what to keep and eliminate: 

  • How will knowing this/not knowing this influence decision-making? 
  • Does knowing this increase our ability to be equitable or just? 
  • Why did we start collecting this data? Does that reason hold meaning now? 
  • What is the cost of collecting this data? Is it worth it? (effort, time $$, risks, privacy, discomfort, etc) 
  • How is this data benefiting or not benefiting those providing the data? 

One note: I’ve worked with nonprofits who’ve shied away from collecting data on race and ethnicity for various reasons, or allowed this data to be incomplete and inaccurate.  This is a case where collecting data IS extremely important. Collecting data on race and ethnicity allows us to identify inequities and ensure more just programs and services. 

3. Take the time to get it right.  

At nonprofits we’re used to wearing many hats. It’s one of our strengths! We can do it all. We can figure it out. We can alchemize. 

This positive mindset, combined with free and easy survey tools has unfortunately resulted in some blunders. Many surveys are thrown together quickly, with little thought to the impact of wording, the type of question, or how either will affect results. There’s also the issue of whether the survey is the right tool, among many possible options–maybe a focused conversation, key-stakeholder interviews or facilitated prioritization process would have served better. The costs are paid in poor quality data, the need to change questions, which leads to inconsistent data, and user-frustration –often resulting in incomplete/inaccurate data, low completion rates and drop-offs with ongoing data collection. 

So, the advice is simple. If your surveys are going well, and this isn’t an issue for you, carry-on. If you recognize this is happening within your organization, you might consider doing some training or working with an external consultant, who can either provide training to your team and/or help design your next survey.

Data is often most useful when it is collected consistently so you can see trends and changes over time. If the data methods or questions change this becomes challenging or impossible. Doing things once, well will save time and money in the long-run, and ensure better results.

Final thoughts

You took the time to read all those words. Now also take a final moment to reflect: where is your organization’s at with data right now? What’s one small step that could improve your organization’s data strategy? 

Guest Author: Lily Sussman

Lily is the owner of LAS Visions LLC, which helps nonprofits with community-based research, evaluation and facilitation. LAS Visions is currently offering MNA members a discount on a Data Strategy Audit. During a DSA we take a deep dive into your data landscape, examining what you collect, how you collect it, and how you use it. From there we provide our top findings and recommendations, so you have clear next steps to build a data strategy that aligns with your organization’s goals.