Audience Segmentation Pipeline

Segment names, thresholds and business criteria are omitted throughout, and any example below is invented. What is described is the workflow and its failure modes — which is the part that generalises anyway.

Objective

Advertising and marketing needed behavioural audiences — built from reading habits, recency, intensity and subscription status — pushed into a customer data platform so they could be activated on site.

The requirement that shaped everything: audiences must be mutually exclusive and ordered by priority. A reader who qualifies for a high-value audience must not also appear in the generic one, or every downstream count is wrong and the same person gets targeted twice.

Approach

Exclusivity is enforced as a cascade, not as a set of independent filters. Segments are evaluated in priority order; each one removes its members from the pool available to the next. That makes the order of the configuration semantically meaningful — which is powerful and dangerous in equal measure, because inserting a segment in the middle silently changes every segment below it.

So the config is the contract. Names, group membership, priority and thresholds live in one place, and the job that builds the audiences derives everything from it. The failure mode I designed against is the nastiest kind: a name added in one file and not the other produces no error, just an audience that is quietly empty or quietly wrong.

The rule I settled on: any change that touches segment definitions must touch exactly the config and the builder, together, and a mismatch between them must fail the run rather than produce output. A pipeline that breaks loudly beats a pipeline that lies.

Architecture

Scheduled batch job. Behavioural events and subscription state are aggregated in the warehouse into a per-reader feature row; the builder applies the priority cascade in SQL and Python, producing one audience membership per reader; the result is pushed to the platform’s ingestion API in batches. Idempotent by construction — a rerun of the same day reproduces the same memberships.

What I owned

The segmentation layer: the criteria worked out with marketing and advertising, the configuration model, the cascade logic, the audience build job, and the runbook for adding or retiring an audience. Scheduling and infrastructure were data engineering’s.

Outcome

Adding an audience went from a careful multi-file edit with a real chance of silent breakage to a bounded change with an invariant that is checked rather than remembered.