Operations
Part of Offline campaign measurement: a practical process from decision to documented result
Offline campaign measurement checklist: incrementality, cost and evidence checks
The offline campaign measurement checklist adds response capture, holdouts, cost, privacy and quality checks that a financial or operational audit does not establish.
What to take away
- The final result must reproduce from governed sources and rules.
- A launch gate requires a measurable decision and tested instruments.
- Data rights, minimization, correction, and deletion belong in the plan.
An offline campaign measurement checklist should stop launch when the decision, outcome, event definitions, baseline, comparison, instruments, permissions, identity rules, quality checks, maturity window, cost model, guardrails, or action threshold cannot be explained. Scale depth to the campaign's risk and spend.
What audits miss, and what to check
A financial or operational audit can verify approvals, invoices, quantities distributed, and recordkeeping. It does not, by itself, establish whether a campaign caused incremental responses or margin. Use these measurement checks alongside, not instead of, formal audits.
- Define the decision, owner, primary outcome, baseline, comparison, maturity window, threshold, and stop rule.
- Assign unique promo codes to each response route. Where appropriate, use dedicated call-tracking numbers, QR codes or short URLs, and receipt or mailer response capture.
- Define how redemptions are captured and deduplicated. For matchback, specify the identifiers, eligible records, exceptions, and correction process.
- Choose a geographic holdout or other documented comparison before launch. Check for exposure through other routes, and reconcile distributed quantities to delivery records.
- Define the denominator and calculation for response rate and redemption rate. Compare the test and comparison groups to assess incremental margin, and calculate cost per incremental response as incremental cost divided by incremental responses, using the planned cost scope.
- Record print, postage, and distribution costs using dated invoices or quotes, and state which costs are included in the full campaign cost.
- List collected data fields, their purpose, access, retention and deletion rules, and the applicable privacy review.
- Test every physical response route end to end, reconcile counts to source-system control totals, and have a reviewer reproduce the primary result.
Use a staged evidence gate
Staged evidence gate
| Gate | Required evidence |
|---|---|
| Decision | Owner, deadline, options, threshold, and stop rules |
| Mechanism | Audience, delivery, response, service, and outcome path |
| Measures | Primary outcome and event dictionary |
| Design | Baseline, assignment or comparison, and power rationale |
| Collection | Tested codes, routes, scripts, forms, systems, and clocks |
| Identity | Units, matching, deduplication, prior status, and consent |
| Privacy | Purpose, access, security, retention, correction, and deletion |
| Quality | Source reconciliation, missingness, drift, and anomalies |
| Analysis | Frozen windows, exclusions, sensitivity, and uncertainty |
| Economics | Mature incremental value and complete cost |
| Reporting | Limitations, guardrails, decision, owner, and archive |
One row, filled in, shows the standard. The row below is a hypothetical example.
| Gate | Evidence location | Reviewer | Date | Result |
|---|---|---|---|---|
| Decision | Decision brief v3, section 2 | Marketing analytics lead (role) | 2026-11-02 (example) | Pass |
A stop rule needs a number. Example: stop when validated incremental margin reaches zero after the maturity window, or when cost per incremental response passes the cap in the plan. Start sizing from the smallest effect the business would act on, then work back to sample size and duration using a pre-specified power target. An 80 percent power target is a commonly used planning convention, drawn from Jacob Cohen's Statistical Power Analysis for the Behavioral Sciences (1988). It is not a guarantee. Record the assumptions and method.
Estimate attrition from comparable campaign records. Document the source, date, and assumptions, and include the allowance in the sample-size and duration plan.
Section 5 of the Federal Trade Commission Act prohibits unfair or deceptive acts or practices, and campaign privacy representations and data practices can raise Section 5 issues. For each campaign, list every personal-data field, its purpose, collection point, access, retention and deletion rules, and correction route.
Check whether the California Consumer Privacy Act, as amended by the California Privacy Rights Act, applies to the campaign's data. Every state has its own breach-notification law, so check the law where each affected customer lives. The FTC's guide on protecting personal information is a practical aid, not a complete legal assessment for a particular program.
Preserve an auditable collection trail
Name the source, access, timestamps, retention, and incident handling for every campaign record. Campaign response logs serve a different purpose from security logs, but the same discipline of a named source and a dated record applies.
Auditable collection trail
- Run an end-to-end test through each physical response route
- Reconcile campaign counts to source-system control totals
- Verify the comparison was not exposed through another path
- Wait through return, cancellation, and adoption windows
- Have a reviewer reproduce the primary result
- Record the approved conclusion and resulting action
At release, each owner should state what evidence was inspected and what would trigger a stop. A checked box without a current definition, tested instrument, approved purpose, working source, documented comparison, or reproducible analysis is not sufficient.
Set a dated evidence freeze for the main report. Late corrections should remain possible, but every added record, rule change, reopened window, and revised result must be visible to reviewers and connected to the reason for change.
Record the gate result
For every checklist item, record pass, fail, not applicable, or accepted exception. Add the evidence location, reviewer, decision date, and next review. Stop the work when a mandatory privacy, security, data-quality, or correction condition fails, even if the remaining score looks favorable.
Keep the evidence record beside the decision so a reviewer can verify the result from the files rather than from memory. Set the next review date and name the change that would trigger an earlier check.
Record rejected options and the chosen path because the original constraint may later change. Use the same definitions in the interface, export, meeting note, and correction log.
Common questions
Who approves the checklist?
One accountable decision owner should coordinate marketing, operations, data, privacy, finance, service, and analytical approvals.
For example, a direct-mail decision gate can name expansion as the decision, incremental margin as the primary outcome, a geographic holdout as the comparison, and a unique promo code plus dedicated QR code or URL as response capture. Before launch, document the response-rate denominator, cost per incremental response, maturity window, delivery reconciliation, evidence locations, and accountable decision-owner role. This is an example of completed gate fields, not a result or a universal design.
Can a small campaign use a shorter checklist?
Yes, but it still needs every gate in the table above and a proportionate answer to each material risk.
When should the plan reopen?
Reopen it when creative, routes, prices, places, systems, rules, data quality, outcomes, or operating conditions change.
