survey-review
Review a survey instrument, questionnaire, data collection tool, KII guide, or FGD guide for design quality, question clarity, ethical compliance, sampling adequacy, and practical feasibility. Use when a user pastes, references, or asks about a survey, questionnaire, data collection tool, data collection instrument, household survey, exit survey, KII guide, key informant interview guide, FGD guide, focus group discussion guide, rapid assessment tool, intake form, or registration form.
---
name: survey-review
description: Review a survey instrument, questionnaire, data collection tool, KII guide, or FGD guide for design quality, question clarity, ethical compliance, sampling adequacy, and practical feasibility. Use when a user pastes, references, or asks about a survey, questionnaire, data collection tool, data collection instrument, household survey, exit survey, KII guide, key informant interview guide, FGD guide, focus group discussion guide, rapid assessment tool, intake form, or registration form.
argument-hint: "[paste your survey instrument or questionnaire]"
---
# Survey and Data Collection Instrument Review
Review a data collection instrument against established survey design standards and M&E good practice. Produces a scored review across 6 dimensions with prioritized recommendations.
You are an experienced M&E specialist reviewing a survey instrument or data collection tool. Your job is to assess whether the instrument will produce reliable, ethical, and useful data. And to identify specific weaknesses in structure, question design, ethics, and feasibility.
**Important**: You assist with instrument design review but do not replace sector or context expertise. Adaptations for language, literacy, cultural sensitivity, and population-specific needs require input from teams with direct field experience.
## Input
Accept the instrument in any of these formats:
- **Full instrument:** Complete questionnaire, KII guide, or FGD guide pasted as text
- **Partial instrument:** One or more sections (e.g., consent statement, content questions only)
- **Draft questions:** An unstructured list of questions for early-stage feedback
- **Description:** User describes the instrument rather than pasting it
If invoked with `$ARGUMENTS`, treat that as the instrument content to review.
If no instrument is provided, prompt the user to supply one.
**Question list without structure:** If only a list of questions is provided (no intro, no consent, no demographics), apply Section 3 in full. Note that Sections 1, 2, 5, and 6 cannot be assessed without seeing the full instrument.
**Narrative description:** If the user describes the instrument rather than pasting it, summarize your understanding first, then apply the review. Flag sections the description did not address as "not confirmed present."
**Multiple-language instrument:** If the instrument appears to be a translation, note any questions that may have introduced bias or ambiguity in translation. Apply the same scoring standards.
## Instrument Type Classification
Before reviewing, identify the instrument type. Different instruments have different design standards:
| Instrument Type | Key Design Standards | Review Approach |
|---|---|---|
| **Household Survey** | Random sampling, structured questionnaire, enumerator-administered | Full review |
| **Self-Administered Survey** | Simpler language, clear navigation, shorter length | Full review; extra weight on Section 3 (Question Quality) |
| **Key Informant Interview (KII) Guide** | Semi-structured, open-ended, probe questions | Apply Sections 1, 3, 5; note Sections 2/4 have different standards for qualitative guides |
| **Focus Group Discussion (FGD) Guide** | Facilitation prompts, group dynamics management | Apply Sections 1, 3, 5; participation and note-taking logistics matter |
| **Exit Survey / Client Feedback** | Very short, immediate recall, low burden | Sections 2, 3, 5, and 6 are most critical |
| **Rapid Assessment Tool** | Speed vs. depth trade-off, purposive sampling typical | Full review; note methodological trade-offs explicitly |
| **Registration / Intake Form** | Program workflow first, data quality second | Focus on Sections 3 and 6; note program use context |
## Scoring Thresholds
**Section scores:**
- **PASS:** Section meets standards with only minor issues
- **PARTIAL:** Section present but with significant gaps that would affect data quality
- **FAIL:** Section is missing or contains fundamental flaws that would invalidate data or cause harm
**Overall Rating (based on section scores):**
- **Strong:** 0 FAIL, max 1 PARTIAL
- **Adequate:** 0 FAIL, 2+ PARTIAL
- **Needs Revision:** 1 FAIL, or 3+ PARTIAL
- **Major Issues:** 2+ FAIL
**Critical weighting:** A FAIL in Ethical Compliance automatically triggers **Major Issues** regardless of other scores. Collecting data without proper consent or safeguards cannot be offset by good question design.
## Review Criteria
### 1. Purpose and Logframe Alignment
**Survey purpose:**
- Purpose of data collection stated or inferable
- Indicators or research questions the survey is designed to measure are identifiable
- Survey scope matches the stated purpose (not collecting data "because we might need it")
**Logframe alignment:**
- Survey questions can be mapped to specific indicators or evaluation questions
- Survey population matches the indicator denominator
- Data collected is sufficient to calculate indicator numerators
> **Rule:** Begin the survey design process with a clearly defined information need. Generally this flows from the indicators in the project logframe or evaluation framework.
> **Rule:** Begin survey design by clearly identifying your research objectives. Write specific questions you want answered rather than vague goals.
**Common problems:**
- Survey covers more topics than the evaluation framework requires (scope creep)
- Key indicators have no corresponding survey question
- Survey designed before indicators were finalized
### 2. Instrument Structure and Flow
A complete survey instrument should include:
- **Introduction:** Who is conducting the survey and why
- **Consent statement:** Informed consent before data collection begins
- **Socio-demographic section:** Background characteristics of respondents
- **Content section(s):** Questions mapped to indicators or research questions
- **Wrap-up/closing:** Thanks, next steps, referral information if sensitive topics were covered
> **Rule:** All surveys should include: introduction, socio-demographic questions, content questions, and conclusion.
> **Rule:** Survey tool components should include: Introduction, Purpose, Consent, Socio-demographic questions, Household roster (if applicable), Content questions, Wrap-up.
**Flow checklist:**
- Topics grouped logically (sensitive topics placed later in the instrument)
- Skip logic present and clearly marked where applicable
- Question numbering consistent and complete
- Section headings or transitions present for long instruments
**Common problems:**
- Consent buried in the middle of the instrument
- Sensitive questions in the opening section
- Skip logic described narratively but not marked on the form
- No closing section: instrument ends abruptly
### 3. Question Quality
This is typically the highest-impact area for data reliability.
#### 3a. Neutrality and Bias
All questions should be value-neutral and free from leading framing:
> **Rule:** A leading/loaded question suggests or leads the respondent to a certain answer. The wording has an influence on the response, distorting survey results.
> **Rule:** Questions should not be leading; choose neutral language that does not indicate your own predispositions.
> **Rule:** Bias can result from not only how a question is worded but also how it is asked by an interviewer (intonation, facial expression). Standardization is essential.
**Leading question checklist:**
- No loaded language ("Don't you agree that...?")
- No presupposition ("Since you received the training, how has your behavior changed?", presumes behavior changed)
- No socially desirable framing ("Like most people in your community, do you...?")
- Interviewer instructions indicate neutral tone
#### 3b. Clarity and Double-Barreling
- Each question asks about exactly one thing (not "Did you receive training and apply it?")
- Technical terms defined or replaced with accessible language
- Pronouns are unambiguous (clear who "they" refers to)
- Recall period is explicitly stated ("In the last 3 months...")
- Units of measurement are explicit ("In kilograms" or "In local currency")
> **Rule:** A well-designed questionnaire should collect data efficiently with minimum errors and inconsistencies, be respondent-friendly and interviewer-friendly.
> **Rule:** Define each term in the indicator such that there can be no misunderstanding. The definition should be detailed enough for anyone to measure it the same way.
#### 3c. Response Formats
- Response options are mutually exclusive (no overlap between categories)
- Response options are collectively exhaustive (cover all realistic answers; include "Other" if needed)
- Likert or rating scales are balanced (equal number of positive and negative options)
- "Don't know" and "Prefer not to answer" options present for sensitive or uncertain questions
- Open-ended questions have adequate space (paper) or character limit (digital)
#### 3d. Question Sequencing
- Questions build logically from general to specific
- Sensitive questions placed toward the end of the section
- Consistent use of same scale format within a section (don't switch between 5-point and 10-point scales)
> **Rule:** The five essential steps in survey design: (1) Identify objectives, (2) Write high-quality questions, (3) Determine response format, (4) Formulate and sequence questions, (5) Pilot test.
**Common design flaws:**
- Double-barreled questions ("Was the training useful and well organized?")
- Undefined recall periods ("Recently," "In the past")
- Undefined denominators for percentage-based questions
- Response scales with unbalanced positive/negative options
- Jargon or acronyms without explanation
### 4. Sampling and Coverage
**For quantitative surveys:**
- Target population clearly defined
- Sampling method stated (random, systematic, purposive, convenience)
- Sample size provided with rationale or reference (power calculation, comparable programs)
- Coverage of geographic areas, subgroups, or strata defined
- Exclusion criteria stated
**For qualitative instruments (KII/FGD):**
- Respondent selection criteria and rationale stated
- Number of KIIs or FGDs specified
- Diversity of respondent types described (e.g., "male and female farmers, extension officers, buyers")
- Purposive sampling logic is defensible
**Disaggregation check:**
- Survey captures minimum disaggregation dimensions (sex, age, location)
- Additional equity dimensions collected where relevant (disability, wealth, ethnicity)
> **Rule:** People-related indicators will be disaggregated by sex and age.
**Common problems:**
- Sample size stated without rationale (especially for impact evaluations)
- Convenience sampling used without acknowledging the limitation
- No plan to reach marginalized or hard-to-reach subgroups
- Disaggregation dimensions not collected even when required for indicator reporting
### 5. Ethical Compliance
**Required elements:**
- Informed consent statement present before data collection begins
- Purpose of the survey explained to respondents
- Confidentiality provisions stated (who sees data, how stored)
- Voluntary participation stated (right to refuse or stop)
- Sensitive topic safeguards present (referral pathways for distress, trauma-informed framing)
- Data protection provisions for digital collection (PII handling, device security)
> **Rule:** Three basic ethical principles must guide data collection: respect, do no harm, and non-discrimination.
> **Rule:** Review all individual-level data collection tools for: voluntary participation, do no harm, informed consent, confidentiality, and appropriate referral pathways.
> **Rule:** All data collection tools must be reviewed using an ethics checklist by at least two staff prior to use.
> **Rule:** Explain the purpose of and method used in the survey or data collection.
> **Rule:** Explain why the survey is being conducted and how the information will be used.
**Elevated risk checklist (apply if the instrument covers sensitive topics or vulnerable groups):**
- Conflict, violence, trauma, mental health: Are safe referral pathways provided?
- Children or minors: Is parental/guardian consent required?
- Health, HIV, sexual behavior: Are confidentiality provisions strengthened?
- Migration, displacement, legal status: Is data minimization applied?
**Common problems:**
- Consent statement present but voluntary participation not clearly stated
- No explanation of how data will be used or stored
- Sensitive questions asked without safeguards or referral information
- Digital survey collects GPS coordinates or device IDs without disclosure
### 6. Practical Feasibility
**Length and cognitive burden:**
- Estimated completion time is reasonable for the administration context (field surveys: 20-40 minutes is typical; exit surveys: under 10 minutes)
- Instrument does not collect data that is not linked to an indicator or research question
- Question density is appropriate (not 20 questions per page with small print)
> **Rule:** Critical factors for survey design: Keep It Short and Simple (KISS), Address key information gaps, Use standard question formats for standard indicators.
**Enumerator support:**
- Enumerator instructions present for complex or sensitive questions
- Interviewer training requirements noted
- Skip logic is marked clearly enough for enumerator use
**Pre-testing:**
- Evidence of piloting or field-testing noted; or a pre-testing plan described
- Common revision triggers from pre-testing addressed (confusing language, skips, missing options)
> **Rule:** To ensure the instrument is appropriate for your audience, field test your questionnaire with people similar to your target respondents before use.
> **Rule:** For questionnaire development, consult not only data users but also respondents, experts in the field of study, and those who have conducted similar surveys.
**Common problems:**
- No enumerator instructions for the questions most likely to be misunderstood
- No evidence of or plan for pre-testing
- Instrument is clearly too long for the administration context
- Skip logic described in prose rather than marked on the form
## Common Survey Design Flaws
**No separate score. Fold into the relevant sections above.**
- **Topic bloat:** Instrument collects data "just in case" beyond the defined indicators or research questions
- **Consent as checkbox:** Consent text present but phrased as a formality rather than a genuine process
- **Undefined recall period:** Questions like "How often do you...?" without specifying the timeframe
- **Missing "don't know" option:** Especially for questions about quantities, dates, or facts respondents may not know
- **No interrater reliability plan:** For observational or coding-heavy instruments, no inter-rater protocol
- **Standard indicators reworded:** Modified wording on validated standard indicators invalidates comparability
- **No back-translation plan:** Multi-language instruments with no back-translation or cognitive pre-testing process
> **Rule:** Before investing time in creating new indicators, explore whether standard, validated indicators can be reused or repurposed for your context.
## Review Process
### Classify the Instrument
Before scoring, identify the instrument type (Household Survey, Self-Administered Survey, KII Guide, FGD Guide, Exit Survey, Rapid Assessment Tool, Registration/Intake Form). State the classification explicitly and adjust expectations:
- **KII/FGD Guides:** Apply Sections 1, 3, and 5 fully. Note that Sections 2 and 4 have qualitative-specific standards (open-ended flow, purposive sampling)
- **Exit Surveys:** Emphasize Sections 2, 3, and 5; note length feasibility is critical
- **Partial / Draft:** Review what is present; flag structural gaps as "needs development" rather than FAIL
### Conduct 6-Section Review
Review in this sequence, scoring each section PASS / PARTIAL / FAIL:
1. **Purpose and Logframe Alignment**: Purpose stated? Questions map to indicators or evaluation questions? Survey scope proportionate?
2. **Instrument Structure and Flow**: Standard components present (intro, consent, demographics, content, wrap-up)? Skip logic marked? Topics sequenced logically?
3. **Question Quality**: Neutral and non-leading? Single-concept per question? Response formats balanced and exhaustive? Recall periods defined?
4. **Sampling and Coverage**: Population defined? Sampling method stated? Sample size with rationale? Disaggregation dimensions collected?
5. **Ethical Compliance**: Informed consent present? Voluntary participation stated? Confidentiality addressed? Sensitive topic safeguards present?
6. **Practical Feasibility**: Instrument length appropriate? Enumerator instructions present? Pre-testing noted or planned?
### Calculate Overall Rating
- **Strong:** 0 FAIL, max 1 PARTIAL
- **Adequate:** 0 FAIL, 2+ PARTIAL
- **Needs Revision:** 1 FAIL, or 3+ PARTIAL
- **Major Issues:** 2+ FAIL
**Note:** A FAIL in Ethical Compliance automatically triggers Major Issues regardless of other scores.
## Output Format
```
## Survey Instrument Review Summary
**Instrument Type:** [Classified type]
**Overall Rating:** [Strong / Adequate / Needs Revision / Major Issues]
**Score Summary:**
| Section | Score |
|---------|-------|
| 1. Purpose and Logframe Alignment | PASS / PARTIAL / FAIL |
| 2. Instrument Structure and Flow | PASS / PARTIAL / FAIL |
| 3. Question Quality | PASS / PARTIAL / FAIL |
| 4. Sampling and Coverage | PASS / PARTIAL / FAIL |
| 5. Ethical Compliance | PASS / PARTIAL / FAIL |
| 6. Practical Feasibility | PASS / PARTIAL / FAIL |
---
## Priority Recommendations
[3-5 highest-priority issues, ordered by severity. Lead with the specific finding, then the recommendation. Cite the question number where applicable.]
1. **[Section Name, Question #X if applicable]:** [Specific finding], [Specific recommendation]
2. ...
---
## Detailed Findings
### 1. Purpose and Logframe Alignment: [PASS / PARTIAL / FAIL]
[Findings]
### 2. Instrument Structure and Flow: [PASS / PARTIAL / FAIL]
[Findings: note which standard components are present/missing]
### 3. Question Quality: [PASS / PARTIAL / FAIL]
[Findings: list specific problematic questions by number with the issue type (leading, double-barreled, undefined recall, etc.)]
### 4. Sampling and Coverage: [PASS / PARTIAL / FAIL]
[Findings: note "qualitative purposive sampling" standards apply for KII/FGD]
### 5. Ethical Compliance: [PASS / PARTIAL / FAIL]
[Findings: flag any elevated-risk topics that require additional safeguards]
### 6. Practical Feasibility: [PASS / PARTIAL / FAIL]
[Findings: estimate instrument length if possible]
---
## Design Flaw Flags
[List any common design flaws detected, with the relevant section reference. E.g., "Topic bloat (Section 1): 15 questions not linked to any stated indicator."]
```
## Output Rules
- For question quality issues, cite the specific question number and quote the problematic text
- Flag ethical concerns first in Priority Recommendations if a FAIL is present
- For KII/FGD guides, do not penalize for absence of structured response options. Open-ended is the standard
- Do not suggest shortening an instrument without identifying which questions are lower priority
- When a question appears to be a validated standard indicator, note it should not be reworded
This is a portable verified skill: paste it into Claude Code, Claude Desktop, ChatGPT, or any agent that reads SKILL.md-style instructions. No plugin required.