Planning for data collection

It’s important to remember there is no perfect method for collecting data. Trade-offs are a part of the process, but by carefully reflecting on the following three questions, you can work towards getting as high a response rate as possible.

Who will distribute your survey — staff, volunteers, partners?

If you can, we recommend having a neutral party administer the survey. Why? Because some clients may not feel comfortable providing negative feedback if the staff they work with is in the room or sending them the survey.

Ideally, this neutral party would be someone with the language skills and cultural competency to make your clients feel safe and welcome.

Regardless of who administers the survey, it’s critical to assure clients that:

  • Feedback is consequence-free and has no bearing on their access to services
  • Responses are anonymous and/or confidential
  • They will hear back from your organization on how you plan to address their feedback

To bring everything together, it may be helpful to hold a short training for survey administrators. This will establish a consistent protocol and align everyone on how to communicate with clients.

[Survey Admin]

[Use the worksheet to help make team decisions about the who, when, and how of administering your survey. Once you’ve done this, you’ll be ready to set up your survey links and begin collecting data.]

When is a good time to survey?

Pick a time when your clients have the space to be reflective, objective, and honest. A time when they’re feeling rushed or when something particularly good or bad just happened is usually not the best time.

Consider the following:

  • Go with a time when clients are able to respond thoughtfully
  • The best timing may differ across programs or sites, so don’t focus on a one-size-fits-all solution
  • Make sure clients have had sufficient experience with your services

To increase response rates, we sent an initial text about the survey after they first called our helpline, and also added gift cards as an incentive.

Exactly how are you going to survey your clients?

Determining your survey administration method(s) involves weighing a variety of trade-offs.

For instance, paper surveys may be more accessible for clients, but involve more administrative work when it comes to entering responses one-by-one into SurveyMonkey.

The method you choose will also have an impact on your response rate. Onsite surveys — delivered in-person or during services or programming — tend to result in much higher response rates than offsite.

But offsite surveys — delivered remotely outside of services or programming — can provide greater confidentiality and flexibility for your clients.

Using incentives can help boost participation in some cases. Review the table below to learn about some of the benefits and drawbacks of different survey administration methods, as well as their median response rates (RR) based on the experience of previous L4G organizations.

Also be aware that your response rate depends on a variety of interrelated factors in addition to your chosen administration method, such as your service model and the size/scale of your client population and data collection effort.

We started with computer set up, but students were afraid they'd be photographed, so we switched to cell phones, but still there was a fear of being identified, so now we use paper surveys.

Method Benefits Drawbacks Median RR*
COMPUTER or TABLET
(onsite using shared device)
No staff time needed for data entry

Some clients are intimidated by technology or uncomfortable with using shared devices

Requires Wifi
ONSITE OFFSITE
Computer: 77%

Tablet: 47%
_
PAPER
(onsite or offsite by mail/drop-off)
May be less intimidating for some clients, especially those with lower literacy levels

No technology required for clients
Responses must be manually entered into SurveyMonkey for analysis 69% 54%
INTERVIEW
(onsite or offsite by phone or videoconference)
Appropriate for clients not comfortable reading, with visual impairments, or who need more support with surveys

Can be an intervention itself for clients who feel isolated

By phone/ videoconference: Appropriate for clients receiving remote services

By phone: almost universal access
Requires significant staff or volunteer time to read the survey and answer options to clients during interview

May raise anonymity or courtesy bias concerns; this can be alleviated by making sure that the client does not have a direct relationship with the interviewer

May require staff conversationally fluent in other languages to conduct interviews
** Phone: 43%
TEXT/ SMS Good for large or geographically-dispersed populations, subgroups that are especially tech-savvy, or those used to communicating through text

Allows people to answer when and where they prefer
May raise anonymity concerns (linked phone numbers); this can be alleviated by sending a general link that all respondents use (text only) _ 47%
EMAIL Can be easily integrated into other databases and platforms (e.g., CRM systems)

Allows people to answer when and where they prefer
Clients may lack access

May raise anonymity concerns (linked email); this can be alleviated by sending a general link that all respondents use
_ 32%

* Average response rates (RR) based on previous L4G organizations. **Currently there is insufficient data to provide average response rates for this method.

 

Bringing an Equity Lens

ACCESSIBILITY

Do your survey administration methods take into account your clients’ comfort level and access to technology? For example, if your clients do not have consistent internet access, phone interviews or mailing out paper surveys might be a better method.

REPRESENTATION

Do you have a plan to make sure demographic groups that are underrepresented in your client base get extra prioritization in your outreach efforts?

PSYCHOLOGICAL SAFETY

Have you trained your staff to provide the survey in a way that will promote candor and assure clients that they will not lose service access as a result of their feedback?

Data Collection Planning Worksheet

Use the worksheet to help make team decisions about the who, when, and how of administering your survey. Once you’ve done this, you’ll be ready to set up your survey links and begin collecting data.