Multi-touch Attribution QA Runbook

This runbook covers release QA, on-call checks and maintenance for multi-touch attribution in Prosper202 1.9.76 and later. Each check comes with the p202 command that runs it. The few jobs only SQL or a PHP script can do are marked as such. Commands that read need an API key with the attribution:read scope; model changes also need attribution:write.

How the engine works

1. Every conversion write, from a pixel, postback, upload or the API, records a ledger row and queues the conversion in the outbox, 202_attribution_pending.
2. The attribution worker drains the outbox. The minutely cron (202-cronjobs/index.php) runs it for 20 seconds each minute. 202-cronjobs/attribution-worker.php runs the same worker on its own, 50 seconds by default. A MySQL named lock keeps two workers from overlapping.
3. For each conversion, the worker builds one journey: up to 25 clicks by the same visitor, inside the widest lookback of your active models (never less than 30 days). Clicks are linked through visitor keys, never by IP address or user agent. The journey is written to 202_attribution_journeys, and how it was built goes to 202_attribution_journey_meta.
4. The worker then computes credits for every active model into 202_attribution_credits. Under each model, a conversion's credits add up to exactly 1, and its credited revenue adds up to exactly its counted amount.
5. Each write marks the report hours it touched. The worker re-sums those hours into 202_attribution_rollup, which reports and exports read.

Changing a model's type, weighting or lookback re-queues every conversion in the account. When two visitors are found to be one person, that person's conversions are re-queued too.

Upgrading from 1.9.55 or earlier: the hourly engine is gone. The removed pieces are:

• The tables 202_conversion_touchpoints, 202_attribution_settings, 202_attribution_snapshots and 202_attribution_touchpoints.
• The /api/v2/attribution endpoints.
• The scripts attribution-rebuild.php and backfill-conversion-journeys.php. Delete any crontab lines that call them.
• The Dashboard System Checks page.

The worker brings existing conversions into multi-touch attribution on its own, at roughly 20 minutes of worker time per million conversions. backfill in p202 attribution queue shows its progress. The upgrade is one-way, so back up the database first; see Upgrading.

1. Health check

Run this after every deploy or upgrade, and at the start of an on-call investigation.

1. Cron is ticking. p202 system cron shows the last run for each job type. The every-minute row should be under two minutes old. 202-cronjobs/health.php treats over 2 minutes as a warning and over 10 as critical.
2. The backlog is draining. Run p202 attribution queue. A healthy install looks like this:

   backfill:
   failing:                    0
   merges_awaiting_requeue:    0
   models_awaiting_recompute:  0
   oldest_enqueued_at:
   pending:                    0
   rows:                       []

   pending rises with traffic and should fall back within a minute or two. If it keeps growing, the worker isn't running. models_awaiting_recompute is above 0 only right after a model change. backfill stays empty once the post-upgrade backfill has finished.
3. Nothing is failing. When failing is above 0, rows lists each conversion with its attempts and last_error. Failed rows retry on their own, waiting longer each time, from one minute up to a day. To retry them now, fix what last_error names and run php 202-cronjobs/attribution-worker.php --retry-now. A database error stops the run without spending any row's retries, so nothing is lost.
4. Every model is usable. In p202 attribution model list, each model should be active or inactive. invalid means its stored definition failed validation. p202 attribution model get <id> shows why in status_reason. The other models keep computing in the meantime.

2. Models and configuration

1. One default per account, always active. A unique key in the database enforces this (one_default on 202_attribution_models). The default can't be unset, deactivated or deleted. To replace it, make another model the default first.
2. Types and weighting. The weighting config is a JSON object:

   Type	Weighting config
   last_touch, first_touch, linear	none
   time_decay	{"half_life_hours":48}, up to 8760
   position_based	{"first_weight":0.4,"last_weight":0.4}, each 0–1, together at most 1

   Lookback isn't part of the weighting config. It's a model setting, --lookback-days, from 1 to 365 days, default 30. Changing a model's type resets its weighting to that type's defaults.
3. Create a QA model: p202 attribution model create --model-name "QA linear" --model-type linear. The next worker run computes its credits for existing conversions. Watch models_awaiting_recompute go to 1, then back to 0.
4. Change a model: p202 attribution model update <id> --weighting-config '{"half_life_hours":24}'. The response sets recompute_pending, and the queue shows the recompute until it finishes.
5. Per-campaign model: p202 campaign update <id> --attribution-model-id <model_id> credits that campaign's conversions with a model other than the default. 0 returns the campaign to the default.
6. Audit trail: every model create, update and delete, and every export, writes a row to 202_attribution_audit. The actions are model_created, model_updated (with the fields changed and whether it recomputed), model_deleted and export_created. Neither the CLI nor the API reads these rows; this is SQL only:

   SELECT FROM_UNIXTIME(created_at) AS at, action, model_id, metadata
   FROM 202_attribution_audit
   WHERE user_id = 1
   ORDER BY audit_id DESC
   LIMIT 20;

3. Journey validation

1. Check one conversion end to end. p202 attribution journey <conv_id> shows three things. touches lists each click in order, with the signals that linked it. journey records how the journey was built. credits gives every model's split. From a two-touch conversion worth 208.00:

   "journey": {"built_lookback_days": 30, "identified": true, "touches": 2, "truncated": false},
   "credits": [
     {"model_name": "Last touch",  "touches": [{"position": 1, "credit": "1.00000000", "revenue": "208.00000"}]},
     {"model_name": "First touch", "touches": [{"position": 0, "credit": "1.00000000", "revenue": "208.00000"}]},
     {"model_name": "Position based (40/20/40)", "touches": [
       {"position": 0, "credit": "0.50000000", "revenue": "104.00000"},
       {"position": 1, "credit": "0.50000000", "revenue": "104.00000"}]}
   ]

   Expect positions to start at 0 and run without gaps, with the converting click last and present exactly once. Under each model, credits add up to 1 and revenue to amount. A conversion with counted: false has no credits; it was superseded, deleted or fully reversed. p202 click conversions <click-id> explains why a conversion counts or doesn't.
2. Check the population. p202 attribution journeys --period last7 gives the journey-length distribution, time to convert, and the one-touch share by browser. If nearly every journey is one touch, clicks aren't being joined. Check:
   • The campaign's --identity-signals setting (1 links its clicks into journeys).
   • That landing pages load the tracking script.
   • That clicks go through your tracking domain.
   Some browsers clear redirect-domain cookies, and the by-browser share shows how much that costs you.
3. Long journeys. A journey over 25 touches keeps the newest 25, including the converting click, and records truncated: 1 in its journey meta.
4. Database invariants (read-only SQL). Both queries should return no rows:

   -- every stored journey has positions 0..touches-1
   SELECT m.conv_id
   FROM 202_attribution_journey_meta m
   JOIN 202_attribution_journeys j ON j.conv_id = m.conv_id
   GROUP BY m.conv_id, m.touches
   HAVING COUNT(*) <> m.touches OR MAX(j.position) <> m.touches - 1;
   
   -- under every model, a conversion's credits add up to exactly 1
   SELECT conv_id, model_id, SUM(credit) AS total
   FROM 202_attribution_credits
   GROUP BY conv_id, model_id
   HAVING total <> 1;

4. Reports and exports

1. Totals reconcile. p202 attribution breakdown --group-by campaign --period last30 credits each campaign with its own model, or the default if it has none. --model <id> picks one model, and --compare-model <id> adds a second model's columns side by side. Every model credits the same total revenue; they differ only in which clicks receive it. --cohort click counts what the range's clicks earned, as the classic reports do.
2. 409 from a report means the model it asked for is inactive or invalid. Report on an active model, or fix that one.
3. Exports. p202 attribution export create queues a breakdown to CSV, optionally sent to a signed https webhook. The minutely cron runs due exports. For details and recovery:
   • p202 attribution export get <id> shows an export's status, row count, webhook answer and last_error.
   • p202 attribution export retry <id> queues a failed export again.
   • p202 attribution export download <id> fetches the file, whatever the webhook did.
   • To watch an export run, run php 202-cronjobs/attribution-exports.php by hand.
   A webhook must resolve to a public address. A receiver on your own network needs its range listed in P202_WEBHOOK_ALLOW_NETWORKS in 202-config.php, written in CIDR form.

5. Recompute and repair

1. Don't delete journey or credit rows by hand. Nothing re-queues a conversion whose rows are gone. A raw DELETE also skips the report-hour marks every engine write leaves, so reports for those hours go stale.
2. Recompute a model by changing it, with p202 attribution model update. Any change to its type, weighting or lookback re-queues every conversion in the account.
3. Watch the worker by running it in a terminal: php 202-cronjobs/attribution-worker.php --budget=300. Its only options are --budget=N (1–3600 seconds) and --retry-now. It prints one line per run, which also lands in the cron log:

   attribution-worker: processed 42 (cleared=2, credited=40); merges re-queued 1; model changes fanned out 0; still due 0

6. Incident response and rollback

1. A model is producing bad credit.
   • If it's the default, move the default to a safe model first: p202 attribution model update <safe_id> --default.
   • Then switch the bad model off: p202 attribution model update <bad_id> --status inactive. The other models keep computing.
   • To remove it entirely, run p202 attribution model delete <id> --dry-run first. Delete removes the model's credits and exports, and is refused while one of its exports is running.
   Over the REST API, the same switch is PUT /api/v3/attribution/models/{id} with {"status": "inactive"}. The API refuses fields it doesn't know with a 422, including the old is_active.
2. Reports have stopped updating. Work through the health check. A growing pending with a stale every-minute cron row means the cron isn't running. Run the worker by hand to see its error.
3. An upgrade went wrong. The upgrade changes the database in place, so restoring the backup taken before it is the only way back. Putting the old files back doesn't roll it back.
4. Escalate with these attached:
   • The output of p202 attribution queue --json and p202 attribution model list --json.
   • The worker's cron log lines.
   • The latest 202_attribution_audit rows.
   • p202 system info.
   Open an issue on GitHub.

Written for Prosper202 1.9.77. Check this runbook against the schema, cron and commands whenever the attribution engine changes.