Planning for Data Collection

It’s important to remember there is no perfect method for collecting data. Trade-offs are a normal part of the process, but by carefully reflecting on the who, when, and how, you can work towards getting as high a response rate as possible. Getting feedback from as many people as possible is key to having an impactful feedback process.

Who will distribute your survey — staff, volunteers, partners?

If you can, we recommend having a neutral party administer the survey. Why? Because some clients may not feel comfortable providing negative feedback if the staff they work with is in the room or sending them the survey.

Ideally, this neutral party would be someone with the language skills and cultural competency to make your clients feel safe and welcome.

Regardless of who administers the survey, it’s critical to assure clients that:

Feedback is consequence-free and has no bearing on their access to services
Responses are anonymous or confidential
They will hear back from your organization on how you plan to address their feedback

To bring everything together, it may be helpful to hold a short training for survey administrators. This will establish a consistent protocol and align everyone on how to communicate with clients.

Data Collection Training Guide

Use this guide to prepare your colleagues or volunteers who will be administering your survey. It will help them explain the what, when, how, and why behind your organization’s feedback work.

When is a good time to survey?

Pick a time when your clients have the space to be reflective, objective, and honest. A time when they’re feeling rushed or when something particularly good or bad just happened is usually not the best time.

Consider the following:

Go with a time when clients are able to respond thoughtfully
The best timing may differ across programs or sites, so don’t focus on a one-size-fits-all solution
Make sure clients have had sufficient experience with your services

To increase response rates, we sent an initial text about the survey after they first called our helpline, and also added gift cards as an incentive.

SOUTHWEST HUMAN DEVELOPMENT

How are you going to survey your clients?

Determining your survey administration method(s) involves weighing a variety of trade-offs.

For instance, paper surveys may be more accessible for clients, but involve more administrative work when it comes to entering responses one by one into SurveyMonkey.

The method you choose will also have an impact on your respon se rate. Onsite surveys — delivered in person or during services or programming — tend to result in much higher response rates than offsite.

But offsite surveys — delivered remotely outside of services or programming — can provide greater confidentiality and flexibility for your clients.

Using incentives can also help boost participation in some cases. Review the table below to learn about the benefits, drawbacks, and median response rates (RR) of different survey administration methods based on previous L4G organizations.

Also, keep in mind that a variety of interrelated factors, like your service model and the size/scale of your client population and data collection effort, will impact your response rate in addition to your data collection method(s).

We started with computer set up, but students were afraid they'd be photographed, so we switched to cell phones, but still there was a fear of being identified, so now we use paper surveys.

CATIE'S CLOSET

Method	Benefits	Drawbacks	Median RR*
COMPUTER or TABLET (onsite using shared device)	No staff time needed for data entry	Some clients are intimidated by technology or uncomfortable with using shared devices Requires Wi-Fi	ONSITE	OFFSITE
COMPUTER or TABLET (onsite using shared device)	No staff time needed for data entry		Computer: 77% Tablet: 47%	_
PAPER (onsite or offsite by mail/drop-off)	May be less intimidating for some clients No technology required for clients	Responses must be manually entered into SurveyMonkey for analysis	69%	54%
INTERVIEW (onsite or offsite by phone or videoconference)	Appropriate for clients not comfortable reading, with visual impairments, or who need more support with surveys Can be an intervention itself for clients who feel isolated By phone/ videoconference: Appropriate for clients receiving remote services By phone: almost universal access	Requires significant staff or volunteer time to read the survey and answer options to clients during interview May raise anonymity or courtesy bias concerns; this can be alleviated by making sure that the client does not have a direct relationship with the interviewer May require staff conversationally fluent in other languages to conduct interviews	**	Phone: 43%
TEXT/ SMS	Good for large or geographically-dispersed populations, subgroups that are especially tech-savvy, or those used to communicating through text Allows people to answer when and where they prefer	May raise anonymity concerns (linked phone numbers); this can be alleviated by sending a general link that all respondents use (text only)	_	47%
EMAIL	Allows people to answer when and where they prefer	Clients may lack access May raise anonymity concerns (linked email); this can be alleviated by sending a general link that all respondents use	_	32%

* Median response rates (RR) based on previous L4G organizations. **Currently there is insufficient data to provide average response rates for this method.

Bringing an Equity Lens

ACCESSIBILITY

Do your survey administration methods take into account your clients’ comfort level and access to technology? For example, if your clients do not have consistent internet access, phone interviews or mailing out paper surveys might be a better method.

REPRESENTATION

Do you have a plan to make sure demographic groups that are underrepresented in your client base are prioritized in your outreach efforts?

PSYCHOLOGICAL SAFETY

Have you trained your staff to provide the survey in a way that will promote candor and assure clients that they will not lose service access as a result of their feedback?