Private by arithmetic

Differential privacy is usually introduced with a definition, which is the wrong way round for anyone deciding whether to use it. Here is the definition anyway, in one line, because the rest of the note depends on reading it correctly.

A randomised procedure is ε-differentially private if, for every possible output, the probability of producing it changes by at most a factor of depending on whether any one person’s data is included:

Three things follow immediately, and they are the things worth carrying.

It is a promise to a person, not a property of a dataset The guarantee is about what an observer can conclude about you, whatever else they already know. That is why it survives an adversary with outside information, which anonymisation famously does not.
ε is an odds ratio At ε = 1, anybody’s belief about you can move by a factor of 2.7. At ε = 10, by a factor of 22,000 — which is no protection at all, and ε = 10 deployments exist.
The noise is set by sensitivity, not by ε alone Adding Laplace(Δ/ε) gives ε-DP, where Δ is the most one person can change the answer. Halving what each user is allowed to contribute halves the noise at the same ε — which makes clipping a privacy lever, not a data-cleaning step.

Two places to put the noise

Central DP. Users send their real data; the server holds it and adds noise to whatever it publishes. You are trusting the server. The noise is added once, to an aggregate over everybody.

Local DP. Each user’s device adds noise before anything leaves it. You are trusting nobody. The noise is added once per user, and then all of it has to be averaged away.

That difference sounds like a matter of degree. It is a factor of , and at scale that is the difference between “free” and “impossible”.

Pay both bills

Below, 600 users with a real taste over ten topics. Each contributes a clipped histogram of what they interacted with, and Laplace noise calibrated to the budget is applied. Then two entirely different questions are asked of the same noised data.

The same budget, two different bills. Purple is how much of one user's own recommendation quality survives their noised profile. Teal is how accurately the population's topic mix can be recovered from all those noised profiles; grey is the same estimate under central DP, where the noise is added once. The horizontal axis is logarithmic.

The asymmetry

Set ε to 1 — a value most privacy engineers would call meaningful protection.

Population topic mix, central DP: 0.9% off. Essentially exact. You can build dashboards, train popularity models, do capacity planning, and report to the business on this.
That user’s own recommendations: 28% of their unnoised quality. Not degraded — destroyed. The profile the system receives is not a noisy version of this person; it is mostly noise with a person somewhere inside it.

Look at one profile at the bottom of the widget to see why. The user’s actual history is a handful of counts, most of them small. The noise has scale . Individual counts arrive as numbers like and . There is nothing to recover.

For the population count, the noise added to each user is independent, so summing across 600 users grows the signal by 600 and the noise by only — the relative error falls as . With central DP it is better still: one noise draw against a total of 600 users’ data. For an individual profile there is no to divide by. The noise that protects one person lands entirely on that person.

Drag ε up to 10 and personalization recovers to 87% — at a privacy guarantee that is no longer a guarantee. That is the trade in its bluntest form, and it is why almost nobody runs local DP on a personalization profile.

The lever people forget

Drag events counted per user instead.

The noise scale is , and for a histogram Δ is how many events one person may contribute. Counting 60 events per user instead of 10 makes the noise six times larger at the same ε. Clipping each user’s contribution is therefore doing exactly the same job as raising ε, and it is usually the cheaper place to spend: the hundredth event a heavy user contributes adds very little to a population estimate and a great deal to the sensitivity.

In practice this is where most of the tuning happens on real deployments. Pick ε by policy, then clip aggressively and check what the aggregate can still support.

Composition, which is the part that bites later

Every query against the same data spends budget, and the budgets add up. Ten queries at ε = 0.1 is an ε = 1 deployment. A dashboard that refreshes hourly against a DP mechanism is spending 24ε a day unless somebody is accounting for it.

The failure mode is organisational rather than mathematical: one team implements a DP mechanism correctly, another adds a second query against the same data, and the guarantee everyone believes they have is no longer the guarantee they have. If you deploy DP, the budget accountant is part of the system, not a spreadsheet.

What actually ships

Where DP earns its place

Analytics and aggregates — trending topics, popularity counts, telemetry, the histograms that feed dashboards. Also DP-SGD for training a model: the gradient is clipped per example and noised, so the model cannot memorise any individual. Both are cases where the answer is a summary over many people.

Where it does not

The individual profile that drives someone’s own recommendations. There is no population to average over, so the noise is the whole story. Every real system that claims DP personalization is either using a very large ε, protecting something other than the profile, or keeping the profile on the device.

The last of those is the interesting one. If the profile never leaves the phone, there is nothing to protect it from — on-device ranking over a server-provided candidate set gives you an unnoised profile and no collection at all. The same reasoning makes session-only personalization worth more than its accuracy alone suggests: a system that reconstructs intent from the current visit is holding much less about a person to begin with.

The honest summary

Differential privacy is the strongest formal privacy guarantee available and it is genuinely deployable — for the right questions. The mistake is treating it as a switch that makes an existing system private. It is a budget, and the bill depends entirely on whether the answer you want is about a population or about a person.

Count people privately. Describe them locally, or not at all.