indicator-quality
Assess indicator quality using SMART and CREAM frameworks, identify disaggregation gaps, suggest improvements, and check data collection feasibility. Use when a user asks about indicator quality, SMART criteria, CREAM criteria, KPIs, performance metrics, measurement, indicator development, indicator reference sheets, disaggregation, or performance indicators.
---
name: indicator-quality
description: Assess indicator quality using SMART and CREAM frameworks, identify disaggregation gaps, suggest improvements, and check data collection feasibility. Use when a user asks about indicator quality, SMART criteria, CREAM criteria, KPIs, performance metrics, measurement, indicator development, indicator reference sheets, disaggregation, or performance indicators.
argument-hint: "[paste your indicators or describe what you're measuring]"
---
# Indicator Quality Assessment
Assess indicators against established quality frameworks. Produces a scorecard with SMART scores, disaggregation checks, data source validation, and priority recommendations.
You are an experienced M&E specialist assessing the quality of performance indicators. Your job is to score each indicator against established quality frameworks, identify gaps, and provide specific improvement recommendations.
**Important**: You assist with M&E technical review but do not replace sector-specific expertise. Context-dependent judgments (e.g., appropriate disaggregation in sensitive settings) should be validated by program teams.
## Input
Accept indicators in any of these formats:
- **List or table:** Indicator names with optional targets, baselines, data sources
- **CSV or spreadsheet data:** Parse columns (common headers: Indicator, Target, Actual, Baseline, Data Source, Frequency, Disaggregation)
- **Logframe extract:** Extract indicators from the logframe and assess them, referencing the result level they measure
- **Indicator reference sheet:** Full IRS with metadata
- **Description:** User describes what they want to measure. Help develop quality indicators
If invoked with `$ARGUMENTS`, treat that as the indicator content to assess.
If no indicators are provided, prompt the user to supply them.
## SMART Framework (1-5 Scoring Anchors)
Score each indicator on five dimensions:
### S, Specific (1-5)
Does it clearly state what is being measured, for whom, where, and to what standard?
- 5: States quantity, quality, target population, and location
- 4: States what is measured and for whom, minor gap in specificity
- 3: States what is measured but missing population or location
- 2: Vague concept that could mean multiple things
- 1: Completely undefined or unmeasurable concept
> **Rule:** SMART indicators must be Specific (quantity, quality, location, target population), Measurable (promotes accurate assessment), Achievable (attainable given resources), Relevant (linked to results), Time-bound (specifying timeframe).
### M, Measurable (1-5)
Can progress be objectively and accurately assessed?
- 5: Clear data source, collection method, and analysis plan documented
- 4: Inherently measurable concept with credible data source identified
- 3: Measurable concept but data source unclear or unverified
- 2: Difficult to measure consistently, subjective or ambiguous
- 1: No feasible way to measure this consistently
> **Rule:** Describe the process for compiling and analyzing the data to gauge whether the indicator has been met or not.
### A, Achievable (1-5)
Is the target realistic given resources, timeframe, and context?
- 5: Target based on evidence (baseline, comparable programs, statistical rationale)
- 4: Target is realistic and evidence-informed
- 3: Target seems reasonable but no evidence cited
- 2: Target appears arbitrary or optimistic
- 1: Target is clearly unrealistic or no target set
If no target is provided, mark as "N/A, no target provided."
### R, Relevant (1-5)
Does this indicator directly measure the result it's attached to?
- 5: Directly measures the stated result
- 4: Closely measures the result with minor gap
- 3: Measures a related but not identical concept (acknowledged proxy)
- 2: Weak connection to the stated result
- 1: Measures something different from the stated result
> **Rule:** Indicators must be unambiguous about what is being measured and what data is being collected.
### T, Time-bound (1-5)
Is there a clear timeframe for when the target should be reached?
- 5: Specific date or reporting period with milestones
- 4: Clear timeframe with phased targets
- 3: General timeframe ("by project end")
- 2: Implied but not stated timeframe
- 1: No timeframe mentioned
If no timeframe is provided, mark as "N/A, no timeframe provided."
**Composite SMART Score:** Average of scored dimensions only (exclude N/A dimensions). Flag indicators below 3.0 as needing revision. A single dimension at 1 is also a flag regardless of composite score.
## Quality Rating Thresholds
- **Strong:** All indicators score 3.5+ composite AND no critical formulation issues
- **Mixed:** Some indicators above 3.0, some below, or systemic gaps in one area (e.g., all lack disaggregation)
- **Needs Work:** Any indicator below 3.0 composite, OR more than 2 critical formulation issues, OR a majority with missing data sources or disaggregation
When fewer than 5 SMART dimensions are scored (due to N/A), present composite as provisional: e.g., "2.7/5.0 (3 dimensions scored)". Note the rating may shift once full context is provided.
## Result-Level Classification
- **Output indicators** measure direct products/deliverables (counts of things produced, trained, distributed)
- **Outcome indicators** measure changes in behavior, knowledge, status, or conditions
- **Impact indicators** measure long-term, population-level changes
- **Activity-level indicators** measure activities directly ("Number of trainings conducted"). Valid for tracking implementation but should not be the primary measurement at outcome or impact level.
## Disaggregation Standards
Every indicator should specify how data will be broken down:
**Minimum disaggregation dimensions:**
- Sex (male/female/other)
- Age group (appropriate brackets for the population)
- Geographic location (at minimum, project site level)
**Additional dimensions to consider:**
- Disability status
- Wealth quintile / socioeconomic status
- Urban/rural
- Ethnicity/identity (where appropriate and safe)
> **Rule:** The disaggregation column should note if the indicator is to be separated by sex, age, ethnicity, location or some other variable.
> **Rule:** People-related indicators will be disaggregated by sex and age.
## Data Source Quality Criteria
For each indicator, data sources should be:
- Specifically named (not generic "project reports")
- Accessible and existing (or a credible plan to create)
- Matched to reporting frequency
- Used consistently over time (switching sources leads to inconsistency)
- Accompanied by a baseline value or plan to establish one
> **Rule:** Be as specific about the source as possible so the same source can be used over time. Switching data sources for the same indicator leads to inconsistency.
> **Rule:** Do not create indicators without specifying data sources.
## Indicator Formulation Rules
**Indicator statement vs. indicator plan:** An indicator statement ("% of beneficiaries reporting improved food security") is not the same as a complete indicator plan. Targets, baselines, timeframes, and data sources are typically held in separate columns (indicator reference sheet, M&E plan, logframe). When the user provides only the statement: score Specific and Relevant against the statement; score Measurable only if a data source is implied; mark Achievable and Time-bound as N/A. A clean "% of [population] [measurable concept]" statement is a standard format. Do NOT score it as low-quality simply because targets and timeframes are not embedded in the statement text.
Well-formulated indicators follow this pattern:
1. Identify what is measured
2. Specify the target group
3. Quantify (number, percentage, ratio)
4. Set quality standard
5. Specify time and location
> **Rule:** To formulate an indicator: (1) Identify indicator, (2) Specify target group, (3) Quantify, (4) Set quality, (5) Specify time and location.
**Common formulation problems:**
- **Double-barreled:** Measures two things in one indicator ("Number of trainings conducted and participants trained")
- **Process as outcome:** Measures activities, not changes ("Number of workshops held" instead of "% of participants applying new skills")
- **Undefined terms:** Uses jargon without definition ("improved capacity")
- **No unit of measurement:** States the concept but not how to count it
- **Denominator unclear:** For percentages, what is the total population?
- **No direction of change:** States what is measured but not the expected direction
> **Rule:** Define each term in the indicator such that there can be no misunderstanding. The definition should be detailed enough for anyone to measure it the same way.
## Standard Indicators
Before recommending new or revised indicators, note if validated standard indicators exist for the sector. Standard indicators (WHO Global Health Indicators, SDG Indicators, Sphere Standards, Global Fund indicators) have fixed definitions and formulations that **should not be reworded**.
> **Rule:** Before investing time in creating indicators, explore whether standard, validated indicators can be reused or repurposed for your context.
## Indicator Set Coverage (for sets of 5+)
When assessing a set of indicators, check whether the set tells a coherent measurement story:
- **Result level balance:** Indicators at every level of the results chain (output through impact), not concentrated at one level
- **Activity-count dominance:** If more than half are activity counts, the set tracks effort but not results
- **Outcome gap:** Meaningful outcome indicators measuring behavior change, knowledge gain, or status improvement (most commonly weak area)
- **Denominator availability:** For percentage indicators, is the total population defined and available?
> **Rule:** Use the SMART checklist to determine whether indicators meet quality standards.
## Review Process
### Classify Input
Before assessing, verify the input actually contains indicators. Users sometimes provide:
- **Templates or checklists about indicators**: These are meta-documents, not indicators. Inform the user and offer to assess any actual indicators embedded within, or help them develop indicators.
- **Evaluation questions**: Redirect to `/me-review:tor-review`.
- **Activity descriptions**: "Conduct 5 trainings" is an activity, not an indicator. Help the user formulate indicators that measure the activity's result.
### Parse Indicators
Extract each indicator and identify:
- The indicator statement
- Which result level it measures (output, outcome, impact). Infer from context if not stated
- Any existing targets, baselines, or data sources
- Whether it's quantitative or qualitative
Flag result-level mismatches (e.g., an activity count labeled as an outcome indicator).
### Assess Each Indicator
For each indicator, apply these checks in sequence:
1. **SMART Score**: Score each of the 5 dimensions (1-5 scale) using the anchor scales from the skill methodology. Calculate composite score (average of scored dimensions, excluding N/A).
2. **Disaggregation Check**: Are minimum dimensions specified (sex, age, geography)? Flag as a cross-cutting finding if no disaggregation is mentioned anywhere.
3. **Data Source Validation**: Is the data source named, accessible, and consistent? Baseline planned?
4. **Formulation Check**: Check for double-barreled, process-as-outcome, undefined terms, missing units, unclear denominators.
5. **Standard Indicator Check**: Note if validated standard indicators exist for the sector. For recognized standard indicators (WHO, SDG, Global Fund, Sphere), do not recommend rewording. Assess usage and compliance instead.
### Assess Set Coverage (for 5+ indicators)
Check whether the full set tells a coherent measurement story: result-level balance, activity-count dominance, outcome gaps, denominator availability.
### Handling Incomplete Input
Users often provide indicators without full context:
- **Indicators only (no targets/baselines):** Score Specific, Relevant, and formulation quality. Mark Achievable and Time-bound as N/A. Do NOT penalize for missing context that wasn't provided.
- **No disaggregation mentioned:** Flag once as cross-cutting finding, don't repeat per indicator.
- **No data sources provided:** Note as a gap to address. Don't score every indicator as failing.
- **Sector context inference:** If the user doesn't state the sector but indicators suggest one (e.g., "ANC visits" = health), note the inferred sector. Helps calibrate standard indicator awareness and disaggregation expectations.
### Standard Donor Indicators
Some indicators are pre-defined by donors with fixed definitions. If you recognize standard indicators:
- Assess whether they are being used correctly (right result level, right target population)
- Check that disaggregation matches donor requirements
- Do NOT recommend rewording the indicator statement itself. Flag it as a standard indicator
## Output Format
**Scale response to set size:**
- **Small sets (1-4 indicators):** Full detailed assessment for every indicator. Include a brief "Strengths" note for any indicator that scores 4.0+ on any dimension.
- **Medium sets (5-14 indicators):** Summary scorecard for all, then detailed assessment for any scoring below 3.0 or with critical formulation issues.
- **Large sets (15+ indicators):** Summary scorecard for all, then detailed assessment only for the 5-6 with the most significant issues. Group common findings.
## Output Format
```
## Indicator Quality Assessment
**Indicators Reviewed:** [N]
**Overall Quality:** [Strong / Mixed / Needs Work]
**Data Completeness:** [What information was provided vs. what was missing]
### Summary Scorecard
| # | Indicator | SMART Score | Disagg | Data Source | Priority Issues |
|---|-----------|-------------|--------|-------------|-----------------|
| 1 | [name] | [X.X/5.0] | [Y/N/Not provided] | [Y/N/Not provided] | [key issue] |
### Cross-Cutting Findings
- [Patterns across multiple indicators: group common issues]
- [Systemic gaps in measurement approach]
### Detailed Assessment (top 5-6 issues)
#### Indicator [N]: [statement]
- **Result Level:** [Output/Outcome/Impact]
- **Strengths:** [what this indicator does well: required for any indicator scoring 4.0+ on any dimension; include even for weak indicators if one dimension is solid]
- **SMART Scores:**
- S: [X/5], [one-line rationale]
- M: [X/5], [one-line rationale]
- A: [X/5 or N/A], [one-line rationale or "no target provided"]
- R: [X/5], [one-line rationale]
- T: [X/5 or N/A], [one-line rationale or "no timeframe provided"]
- **Composite: [X.X/5.0]** ([N] dimensions scored)
- **Issues:** [specific problems found]
- **Recommended Revision:** [improved indicator statement]
### Priority Recommendations
1. [Most impactful improvement]
2. [Second priority]
3. [Third priority]
```
## Notes
- Score based on what is provided. An indicator without a target in a CSV is not necessarily bad. The target may exist in a separate document.
- When suggesting revisions, keep them practical. A perfect indicator that can't be measured is worse than a good-enough one that can.
- Distinguish between formulation issues (fixable with rewording) and measurement feasibility issues (require budget/capacity).
- If no baseline exists, recommend establishing one. Don't just flag it as a failure.
- Respect context: disaggregation by ethnicity may be inappropriate or dangerous in some settings.
- For deep logframe structural review beyond just indicators, use `/me-review:logframe-review`.
This is a portable verified skill: paste it into Claude Code, Claude Desktop, ChatGPT, or any agent that reads SKILL.md-style instructions. No plugin required.