Method Guidelines
Diary Study
Intensive longitudinal research for fine-grained empirical evidence. Capture temporal dynamics without intense field studies. Economical way to collect in-depth, contextual, longitudinal data.
Phases
Time: 2-5 days
Planning and Preparation
- Define behavior to understand with the study
- Define timeline, tools for reporting, recruit participants and prepare instructions
Time: 2-3 days
Pre-Study Briefing
- Face to Face meeting or phone call to walk participants through the schedule and discuss expectations
- Answer questions of participants
Time: few days - several months
Logging Period
- Participants log following a simple framework
- Regular check-ins to keep up engagement and answer questions
Time: 1 week
Post-Study Interview
- Follow up to discuss logs in detail (individual interviews)
- Ask for feedback on experience (groups)
Time: 2-3 days
Data Analysis
- Evaluate targeted behaviors' evolvement, identify drivers for change
- If you were evaluating an existing product, design a customer journey map
Planning and Preparation
2-5 days
- Define behavior to understand with the study
- Define timeline, tools for reporting, recruit participants and prepare instructions
Briefing
2-3 days
- Face to Face meeting or phone call to walk participants through the schedule and discuss expectations
- Answer questions of participants
Logging Period
few days - several months
- Participants log following a simple framework
- Regular check-ins to keep up engagement and answer questions
Post-Study Interview
1 Week
- Follow up to discuss logs in detail (individual interviews)
- Ask for feedback on experience (groups)
Data Analysis
2-3 days
- Evaluate targeted behaviors' evolvement, identify drivers for change
- If you were evaluating an existing product, design a customer journey map
Image by Indeemo
Collecting Data for Diary Studies
Open vs. Closed Diary
You can also combine both Diary forms.
Free form / Open Diary
- After initial guidance, the participants decide how and when to record feedback
- To be used for open-ended exploration of a topic
Why it’s useful
Requires less training
Potential challenges
Encourage the participants to speculate and offer unnecessary information/ leave out important data
Offline vs. Online Logging Formats
You can also combine both Diary forms (e.g. in a daily/regular questionnaire via mobile combined with paper notes )
Online Logging
- Platforms like Google Docs or Whtasapp
- Electronic diaries (writing, some offer speech recognition or voice recording)
Why it’s useful
Data already comes transcribed (or even analyzed for electronic diaries), quick to process
Potential challenges
Require net coverage & digital skills on participants' side
Logging Intervals
Fix intervals
To observe behaviours evolving over time
You want to explore a topic and don’t know when significant events will occur
- Diaries are filled out in regular, predetermined intervals (e.g. every x hours, once per day/week)
- Might require reminders for consistent logs
On a signal
To compare behaviours
You are worried participants will forget to make entries, or you require a specific amount of entries
- Notification alerts (calls, test messages or else) the participants are prompted to make an entry
- Effective but intrusive
Event based (“in situ logging”)
To test services & products
Choose event based logging:
- For small volume of entries
- To record users in the moment
- Useful for participants with busy schedules and for random events
- Notification alerts (calls, test messages or else) the participants are prompted to make an entry
- Effective but intrusive
Sample questions for a structured or hybrid diary study
“Please describe your morning. ”
“What is the most important thing you have to do today? “
“What were you doing immediately before [task]?”
“What was the hardest part about doing [task]?”
“Which parts of [task] were you unsure about/lacked confidence in?”
“How did you feel while doing [task]?”
“Why did you feel that way?”
“How could that experience have been better?”
These questions from userinterviews.com give a direction to the participants, but leave room to express their routines, thoughts and feelings. Follow the link for more detail.
Diary Methods for Participants in Rural Areas
For target groups in rural areas in the Global South, some specific considerations apply. See below for some options to facilitate communication and logging when working with these groups.
Pictorial Diaries
Pictorial diaries are common
If using written diaries, nominate a person in the household (often the schoolkids) that can take the role of scribe. This can only be done for information that is not private/ sensitive.
IVR technology
IVR technology has been described as a convenient technology to allow participants with limited digital skills or access to technology to record real-time events in structured diaries via their phone
Base your diary format on prior research around the users’ perception of the topic (language, local symbols etc.) and test designs
Prompts
Use prompts (e.g. via calls or SMS) and printout guides to support and remind the participants of how to fill in your diaries. Regular visits by field staff can be used for follow ups and answering questions
Pictorial diary used in The Gambia
This pictorial diagram was used by Wiseman et al. (2015)* to collect data on household expenditures in The Gambia.
Careful, details can matter!
The team reports that “during piloting in The Gambia, it was revealed that few people were recording any expenditure on livestock because the cow was missing a hump!
IVR flow used for diary research in medical contexts
This IVR flow diagram illustrates the IVR-based call-in process to track symptoms of myotonia of patients in the US, Canada and UK.
At the baseline visit, participants were given a brief orientation to an automated interactive voice response (IVR) system and were expected to call in once weekly for 8 consecutive weeks (reminders were sent).
When the participants first use the system by dialing the toll-free phone number, they hear a brief welcome message, a reminder that their participation is voluntary, and are instructed to enter their participant ID. Then they are asked to enter 1 for yes or 2 for no whether they had a given symptom (stiffness, pain, weakness, tiredness) in the preceding week. Second tier questions ask them to estimate the number of days in each week the symptom is experienced and then to estimate the symptom severity on a 1-9 ranked scale.
IVR technology is currently used mostly in clinical diary research but could be very useful for diary research with low-literacy populations.
To take into account
Virgina Wiseman assembled this checklist for using diary studies in applied research:
- What is the most appropriate design (i.e. structured or unstructured? written information or pictorial?)
- Can prompts such as an aide-mémoire be used?
- Can an existing diary or prompt be adapted for use?
- How long should diaries be maintained?
- When should diary entries be made?
- Who within a household should complete the diary?
- Should multiple diaries be kept by a household?
- What level of field support is needed?
- Can diary entries be safely stored by participants?
- How often should entries be collected by researchers?
Source: Wiseman, V. 2015
Best Practices
Find here best practice examples with helpful tips and tricks.
Do’s and Don’ts
Do’s
- Recruiting dedicated participants is key. Be clear about expectations and ask screening questions to be sure about their level of commitment & possible biases their bring
- Regular check ins with participants will remind them and create engagement, as well as provide the chance to ask upcoming questions
- Provide incentives: compensation is very common along time-based milestones. Verbal recognition of efforts also matters!
- Diary studies can involve sensitive or personal information. You need a data management plan (including a plan for data destruction) and the informed consent of your participants.
- A quick pilot study can help you test your study design and materials, practice the briefing and get feedback on the templates and experience. You can also test drive your study with your team.
Don’ts
- Too long study durations will make your participants drop out or become unresponsive
- Don't let incoming data pile up! It is best to evaluate data on a constant basis to be able to ask follow-up questions
- Don't rely on participants' memory regarding logging schedules and instructions. Provide clear written instructions to follow and example log to help them understand the level of detail you need from them
- Don't ask participants to write hypotheses about their behavior, keep their focus on past or current behavior
- Don't rely only on self-reported data from diaries. Combine it with quantitative data or moderated tests (for product or prototype tests)
Potential Bias To Be Aware Of
Find a detailed overview of potential biases with counter actions here.
Below a list of potential bias to be aware of when conducting Diary Study.
Image by Storyset from Freepik
People tend to judge an experience more on how they felt at its most intense point (snapshot one) and its end (snapshot two) rather than based on the total sum or average of every moment of the experience. The remembered value of snapshots dominates the actual value of an experience (positive or negative). For researchers, they might tend to overemphasize results from their first and last interview (e.g. in a row of 5-10).
Image by Storyset from Freepik
Image by Storyset from Freepik
People tend to make more “socially acceptable” decisions when they are around other people. Same holds true for interviews, people want to make you feel good and will answer what they think you find pleasant and acceptable.
Image by Storyset from Freepik
The hindsight bias refers to people attempting to subconsciously filter memories of past events through the lens of present knowledge.
Image by Storyset from Freepik
Image by Storyset from Freepik
The Hawthorne Effect
The very act of being observed can cause participants to change their behavior. The quality of observational data is heavily impacted by this.
Image by Storyset from Freepik
We associate our attitudes and stereotypes to people without our conscious knowledge. Our observations and interpretations of data can be steered by that and produce biased results.
Image by Storyset from Freepik