How to run a remote diary study
About a year ago I ran my first remote diary study. The need for this study arose because I wanted to explore a fairly uncommon experience among a fairly specific user group — something that was not possible to meaningfully explore in interviews alone. I was looking to understand how people in this user group thought about a particular concept and the experiences associated with it, in what circumstances they encountered these experiences, how they navigated through them and what their key pain points were.
A diary study was an ideal format for this type of discovery research as it enabled me to gather near-real-time data about the experience I was interested in. This meant that the data was as close to real-life as I could get in a remote setting, untinted by visual prompts, research session limitations or recollection biases.
The methodology
I designed the study to run in three stages:
- Initial interviews to establish rapport and explore research themes at a high level.
- Two-week diary study with daily entries related to the topic of the study.
- Follow-up interviews with select participants to dive deeper into the study topic and explore their diary entries in more detail.
The tool stack
- Recruitment: User Interviews
- Interviews: Zoom
- Diary Study entries: Whatsapp
- Participation tracking: good ol’ Google Sheets
- Analysis: Airtable + a research repository tool
- Reporting: Google Slides
Key learnings
Use initial interviews to build rapport and set the tone
Initial interviews are the perfect opportunity to get to know your participants, understand how they speak and think about the topic of your study, and establish common ground around what is in and out of scope for your research.
Note, however, that while it’s useful to nudge the participants in the direction that will yield the most valuable data with the least noise through these early conversations, it’s also important to not bias their diary entries — so tread carefully.
Meet your participants where they are
When designing my study, I reviewed a number of professional diary study tools on the market. But ultimately, I decided to run the study over Whatsapp.
This was because I wanted to remove as many barriers to participation as possible. Participating in a diary study is already a big ask, and I didn’t want to complicate things for people who signed up to it by asking them to install and learn a whole new way of communicating.
While my personal preference would have been Signal, I was also aware that most people in my user group wouldn’t be familiar with Signal. To mitigate privacy concerns with using Whatsapp, I created explicit instructions on what kind of data the participants should not be sharing during the study (eg, anything that contained their PII) and discussed this with them in detail ahead of time, too.
Other apps you could consider are MarcoPolo, Slack, Snapchat… There are too many to list, really. But in general, your choice should depend on who your target audience is and which channel you think would be most natural and accessible to them.
Pro-tip: if you use a messaging app like Whatsapp or Signal to collect diary submissions, it can be a good idea to get a separate phone and SIM specifically for research purposes. This prevents research-related communications and data from leaking onto your personal device and makes data management a whole lot easier after the study is over. Factory reset FTW!
Overcommunicate your study instructions
The quality of the data you get depends on this. At a minimum, you need to be very clear when articulating the what and the how of the diary submissions.
The what is the overall ask for each diary entry. Depending on the subject of your study, the main ask will look something like this: “Over the next two weeks, every time you do X, submit an entry”.
The how is the description of what you expect each entry to look like. Do you want your participants to take photos or videos? Record a voice note? Send a text message? Tell your participants exactly what you expect them to send you in order to make sure you get the right kind of data for your research.
Pro-tip: if you’re dealing with emotions or reactions in your research, try leveraging the power of gifs and emojis as a communication tool. This is a great way to help your participants tell you how they’re feeling about the subject of their diary entries without making the submission process too lengthy or difficult for them. The caveat here is that gifs and emojis don’t always translate across generations or cultures, so again, tread carefully. As a bonus, the gifs and emojis chosen by the study participants can be a powerful tool for communicating user sentiment to research stakeholders.
Track study submissions on a regular basis
For my study, this meant daily. Tracking submissions on a regular basis enabled me to:
- Make sure I was getting the data I needed throughout the study period,
- Follow up on interesting or confusing entries in real-time, while they’re still fresh in the participant's mind,
- Nudge people when participation wanes.
If you make incentives payout contingent on a minimum number of submissions — which is one approach to keep participants active throughout the study — routinely tracking submissions becomes even more important.
Start early on the data analysis
Even with a small-scale diary study, you are likely to end up with a huge amount of data.
15 people that participated in my diary study ended up sending through nearly 200 entries over the two-week period. That’s 200 voice notes to listen to and transcribe, 200 text messages to read through, 200 gifs and emojis to interpret. Plus about two hours worth of transcripts for each participant to code. In other words — a bucketload of data.
Starting on the analysis early can help you make it less of a mammoth task once the data collection is over. And, perhaps more importantly, it will allow you to track emerging themes and trends. This means that you can probe on these while the study is still running, tweak anything that needs tweaking in the study design, get more out of the follow-up interviews if you’re running them, and in general, get more valuable and clear data.
What have you found to work well when running remote diary studies? Share your tips in the comments below!
For more inspiration and ideas on designing and running remote diary studies, see these articles that helped me:
- Applying Participant Experience Principles to Diary Study Design by Anya Zeitlin
- Diary studies using WhatsApp by Clicked Research
- Peaking into your participants lives with a diary study by Valsplat
- Diary Studies: Understanding Long-Term User Behavior and Experiences by NN Group
- How to Conduct a Diary Study: A Start-to-Finish Guide by Dscout
And many thanks to Anya Zeitlin and Guy Simpson for chatting with me about their approaches and generously sharing their experience.
