Chapter 24 — Ozone-shaped moderation, bundled

By the end of chapter 19 we had a working operator surface: /admin for handle-gated dashboard work, com.atproto.admin.* XRPC for scripted operator actions, and a takedown column on records, blobs, and accounts. That's everything this PDS needs to moderate its own content.

It's not what the rest of the AT Protocol world expects, though. In Bluesky's deployment, moderation lives in a separate service — Ozone, deployed independently from the PDS. Ozone holds the event log (every takedown, every label, every operator comment), the moderator roster, the report queue, and a sophisticated team workflow on top. It exposes a different lexicon namespace — tools.ozone.* — that moderation clients (the Bluesky web app's "Ozone" interface, custom moderation tools) drive directly. Ozone is also what advertises labels to the network: it signs each label with its operator account's key and serves them via com.atproto.label.queryLabels so AppViews can decide whether to hide / blur / annotate a piece of content.

Our learning port bundles both into one Node process. The /mod web UI, the tools.ozone.moderation.* XRPC handlers, the labels table, and the labeler DID-document service entry all ship in this repo and run alongside the PDS itself. That's the central structural divergence from upstream — and the rest of this chapter is about why that's a reasonable choice for a self-hosted PDS and how the moving parts fit together.

Why bundle?

Upstream's split makes sense at Bluesky's scale: the moderation team operates independently of the PDS operator team, and Ozone needs to scale on a different axis (lots of moderators, lots of reports) than the PDS. Two services let those scale separately.

For a self-hosted PDS, the split is mostly cost. You'd run a second service that's idle most of the time, with a second database, second deploy pipeline, second TLS cert. The moderation surface doesn't need its own scaling envelope — a one-operator, ten-moderator deployment fits comfortably in the same process as the PDS itself.

So we bundle. Same Node process, same Postgres database, same deployment story. The two surfaces stay logically distinct (separate schema, separate UI, separate auth gate) but share the runtime.

What the team lead is

The moderation surface is "owned" by one atproto account on this PDS, configurable via PDS_MOD_TEAM_HANDLE (default mod.<hostname>). The operator creates that account through the normal signup flow. Detection is a single string comparison inside createAccount: input.handle === cfg.modTeamHandle. Every other handle takes the plain account path; only this single configured handle gets the labeler treatment.

Two phases of bootstrap exist, optimised for the common case:

Eager (signup-time) — when createAccount itself sees the lead handle, the genesis PLC op is built with #atproto_labeler already included and the app.bsky.labeler.service record is written inside the signup transaction, before the firehose #identity / #account events fire. By the time createAccount returns, the network's first view of the DID is "labeler + PDS" — no follow-up operations, no second plc.directory call.

Lazy (post-signup) — for the rarer cases where an account becomes the lead after signup (operator renames an existing account into the configured handle, or changes PDS_MOD_TEAM_HANDLE to point at an existing account), the same checks run lazily inside getModTeamLead(). Each routine (ensureLeadRow, ensureLeadLabelerService, ensureLeadLabelerRecord) is idempotent, so they self-heal on first mod-surface read. Once it exists, five things happen on the next getModTeamLead() call (lazy, cached for the process lifetime):

1. A row in mod_team with role='lead' is auto-seeded.
2. The account's PLC op is rotated to add an #atproto_labeler service entry pointing at the PDS's public URL. The genesis op only includes #atproto_pds; the rotation appends to plc_operations locally, publishes to plc.directory via publishPlcOp, and emits #identity on the firehose so AppViews re-resolve the DID document. Idempotent — ensureLabelerService in src/pds/did/plc.ts short-circuits when the entry is already present.
3. An app.bsky.labeler.service self-record is created in the lead's repo, declaring the labeler exists. The DID-doc service entry alone is necessary but not sufficient — bsky.app's AppView surfaces an account as a labeler in its UI by indexing this record. We ship the minimum valid declaration (policies: { labelValues: [] }); the operator can later edit it via putRecord to declare custom label values, definitions, and a self-applied profile label.
4. The account's local DID document grows the same entry. buildDidDocument's isLabeler flag is set when the DID matches the team lead, so our own resolveLocalDid / describeRepo responses include #atproto_labeler alongside #atproto_pds.
5. The labels table is signed with that account's key. Every label emitted via tools.ozone.moderation.emitEvent#modEventLabel is signed with the team-lead's repo signing key — the same key that signs the account's own MST commits. Downstream consumers fetch the DID document, find the #atproto verificationMethod, and verify labels against that public key without further coordination.

The bootstrap happens lazily and self-heals: if you change PDS_MOD_TEAM_HANDLE to point at a different account later, the next read clears the cache, the new lead's mod_team row gets seeded, and the new lead's PLC op gets rotated. The old lead's labeler entry stays in plc.directory's history — operators wanting to retract it must rotate the old account's op manually (none of the canonical Ozone clients require this, so we don't ship a knob).

The lead account's other facts — its handle, its records, its DID document — work like any other atproto account. The moderation surface piggybacks on the account; it doesn't replace it. Conceptually the team lead is the labeler.

Additional moderators are atproto accounts added to mod_team (role='moderator'). The v1 UI lists them read-only; add and remove via direct SQL until a follow-up wires a roster page. There's no "team admin" distinct from the team lead — the lead can act unilaterally, and admin Basic always unlocks everything regardless of team membership.

The data model

Four tables (migration 0016_moderation_service.sql):

mod_team             roster — DIDs and their roles
mod_events           append-only event log (every action ever taken)
mod_subject_status   denormalised current-state cache per subject
labels               signed labels (the public labeler payload)

mod_events is the source of truth. Every other read can be derived from it. We keep mod_subject_status as a cache because queryStatuses is on the hot path of any moderation dashboard and replaying the event log per request would scale poorly.

A subject is identified by a discriminator:

• com.atproto.admin.defs#repoRef — { did } — an account-level subject.
• com.atproto.repo.strongRef — { uri, cid } — a record-level subject.

The shape comes straight from the upstream tools.ozone.moderation.defs lexicon. mod_events and mod_subject_status store the discriminator type in subject_type and the typed columns (subject_did, subject_uri, subject_cid) reflect whichever subject shape was involved.

emitEvent — applying an action

The hot path. The lexicon defines 25+ event types (tools.ozone.moderation.defs#mod*Event); we implement sixteen of them:

Unsupported event types return EventTypeNotSupported with a clear message — a future Bluesky-defined type doesn't silently no-op, so when an upstream event type starts mattering you get explicit feedback to wire it.

All emit-time logic lives in src/pds/mod/events.ts — one applyEmitEvent() function the XRPC handler and the /mod web UI call. Both write the same row, run the same side effects, update the same cache. There's exactly one path from "operator picks an action" to "state changes."

Auth

requireModerator() in src/pds/mod/auth.ts accepts two modes:

1. Admin Basic — the operator with the admin password is always allowed. Matches the "admin can do anything" invariant from chapter 19. The audit-trail attribution falls back to the team-lead DID since the action wasn't taken under a moderator identity.
2. Moderator bearer — a normal atproto access JWT whose subject DID is in mod_team. createdBy on the event input must equal that DID (otherwise a moderator could impersonate the lead in the audit log).

Scheduled actions

tools.ozone.moderation.scheduleAction accepts a takedown wrapped in a schedulingConfig (executeAt ISO timestamp, or executeAfter ISO-8601 duration). The handler writes one mod_scheduled_actions row per subject with state='pending'.

A background sweep (src/pds/mod/scheduled_actions.ts) polls every 30 seconds for rows whose fires_at has passed. For each due row it reconstructs the emitEvent input from the stored DAG-CBOR payload and calls applyEmitEvent() — same code path the live emitEvent handler uses, so side effects + cache update + auto-resolution all flow identically. State flips to 'completed' on success or 'failed' (with a reason) on apply error.

tools.ozone.moderation.cancelScheduledActions flips state from 'pending' to 'cancelled'. Already-fired rows are untouched. tools.ozone.moderation.listScheduledActions paginates by id with optional filters on states[] and subjects[].

Sweep startup + shutdown is wired through the prod server.ts entry; in dev the sweep is dormant (we'd otherwise compete with the operator's manual triggers).

queryEvents, queryStatuses, getEvent — the read surface

Three XRPC handlers cover the read side:

• tools.ozone.moderation.queryEvents — paginated history with filters (subject, types, createdBy, time range).
• tools.ozone.moderation.queryStatuses — paginated current-state view, reading from mod_subject_status.
• tools.ozone.moderation.getEvent — single event by id.
• tools.ozone.moderation.getRepo — moderation-context view of an account: profile + current mod_subject_status + recent events + applied labels, in one round-trip. Honours requireModerator and does serve takendown accounts because moderators need to see what they're moderating.
• tools.ozone.moderation.getRecord — same shape for a single record, keyed by AT-URI. Also serves takendown records.
• tools.ozone.moderation.getRepos / getRecords / getSubjects — batched variants of the above, up to 50–100 entries per call. Mirror input order in the output; missing entries surface as repoViewNotFound / recordViewNotFound.
• tools.ozone.moderation.getAccountTimeline — merged stream of reports + events + labels for one account, sorted by createdAt desc. Each entry is tagged kind: 'report' | 'event' | 'label'.
• tools.ozone.moderation.searchRepos — substring search over accounts.handle and accounts.email, plus exact match by DID.
• tools.ozone.moderation.getReporterStats — per-DID summary (reportedCount, resolvedCount, isMuted) for up to 100 DIDs in one call.

The event view is reconstructed from the DAG-CBOR snapshot emitEvent persisted, so the response shape matches exactly what the caller submitted — full fidelity round-trip.

Queues and report management

The reference Ozone has two surfaces sitting on top of moderation_reports:

• tools.ozone.queue.* (8 endpoints) — operator-defined buckets of (subject-type, report-type, collection) routing rules. Created via createQueue, listed/edited/deleted via the obvious CRUD verbs, and populated by routeReports({startReportId, endReportId}) which matches each unrouted report's (subjectType, reasonType, collection) against every enabled queue and writes the queue id.
• tools.ozone.report.* (12 endpoints) — operator-side view of reports as first-class objects. queryReports / getReport / getLatestReport read, assignModerator / unassignModerator / reassignQueue mutate the row, listActivities / createActivity drive the append-only mod_report_activities log, and the stats trio (getLiveStats / getHistoricalStats / refreshStats) computes counters from SQL on every read (we don't cache; the upstream Redis-backed counter rebuild has no analogue here).

Backing schema:

• mod_queues(id, name UNIQUE, subject_types[], report_types[], collection?, enabled, created_by, created_at, updated_at, deleted_at) — operator-defined queues. deleteQueue is a soft-delete (sets deleted_at + enabled=false); the lexicon's optional migrateToQueueId rewires moderation_reports.queue_id in bulk before the queue is hidden.
• mod_queue_assignments(id, queue_id, did, start_at, end_at) — per-(queue, moderator) attachments. Open assignments have end_at IS NULL; the queue UI shows the current roster of moderators handling each queue.
• moderation_reports gains queue_id, assigned_to_did, assigned_at — populated by routeReports and the report-level assignModerator handlers.
• mod_report_activities(id, report_id FK, activity_type ∈ {queue, assignment, escalation, close, reopen, note}, previous_status, internal_note, public_note, meta, is_automated, created_by, created_at) — append-only audit log per report; previous_status captures the report's state at activity time so the UI can render transitions.

Status derivation for a report (open | queued | assigned | closed | escalated) comes from joining the row against mod_report_resolution and mod_subject_status:

• assigned_to_did set → assigned
• resolution row exists → closed
• subject's mod_subject_status.review_state is reviewEscalated → escalated
• queue_id set → queued
• otherwise → open

No state column on moderation_reports itself — the derived value is canonical, the activities log is the audit trail.

tools.ozone.server.getConfig

A single endpoint Ozone clients (the moderation UI) call on load to discover which features are available. We populate:

• pds.url — cfg.publicUrl
• appview.url — the canonical bsky.app AppView (https://api.bsky.app)
• viewer.role — roleAdmin for admin Basic, roleModerator otherwise
• verifierDid — the team-lead DID (the labeler this PDS hosts)

blobDivert and chat are omitted: we don't run a blob-divert quarantine bucket and chat moderation isn't self-hostable.

The labeler surface

com.atproto.label.queryLabels is the public read endpoint. Anyone can call it without auth and ask "what labels has the labeler applied to this URI?" That's how AppViews discover content moderation decisions: they fetch labels from every labeler their users have subscribed to and apply them to feeds.

Each label is signed with the team-lead's repo signing key. The canonical signed form is DAG-CBOR of { src, uri, val, cts, neg, cid? } — same fields atproto's @atproto/api signs. The sig blob travels on the wire alongside the label; consumers verify against the labeler DID's #atproto verificationMethod.

Subscribe — deferred. The full Ozone surface also exposes com.atproto.label.subscribeLabels over WebSocket so consumers can tail new labels in real time. We don't ship that yet; v1 polls via queryLabels. The implementation shape would mirror our firehose: hand the request off to a WebSocket attached to the same Node http server (the pattern from chapter 16), tail by labels.seq desc, re-emit on insert.

The /mod web UI

src/routes/mod/ — server-rendered HTML mirroring /admin's aesthetic.

The session is a cookie-backed JWT scoped to /mod, separate from the /admin cookie scope so the two surfaces don't bleed. Admin Basic in the Authorization header always works — the "signed in as" pill flips to read admin (Basic) and no cookie is required.

How this maps onto a real Ozone

If you wanted to run this PDS as a federation member of a real Ozone-driven moderation network, two things are true:

1. A real Ozone client can drive our XRPC surface. The tools.ozone.moderation.* endpoints match the canonical lexicon shapes — emitEvent, queryEvents, queryStatuses, getEvent. A client that speaks "talk to Ozone" sees this PDS as an Ozone instance.
2. AppViews can subscribe to our labels. The labeler DID-document service entry tells the network where to fetch our labels; the queryLabels endpoint serves them; the per-label signatures verify against our team-lead account's public key. No extra handshake.

The structural difference — one process vs. two — is invisible to network consumers.

Try it

# 1. Create the team-lead account.
curl -i -X POST http://localhost:3000/xrpc/com.atproto.server.createAccount \
  -H 'content-type: application/json' \
  -d '{
    "handle": "mod.localhost",
    "email": "mod@example.com",
    "password": "correcthorsebatterystaple",
    "inviteCode": "..."
  }'

# 2. Capture the access JWT.
TOKEN=$(curl -s -X POST http://localhost:3000/xrpc/com.atproto.server.createSession \
  -H 'content-type: application/json' \
  -d '{"identifier":"mod.localhost","password":"correcthorsebatterystaple"}' \
  | jq -r .accessJwt)

# 3. Emit a takedown on some account.
curl -i -X POST http://localhost:3000/xrpc/tools.ozone.moderation.emitEvent \
  -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{
    "event": { "$type": "tools.ozone.moderation.defs#modEventTakedown", "comment": "spam" },
    "subject": { "$type": "com.atproto.admin.defs#repoRef", "did": "did:plc:<target>" },
    "createdBy": "did:plc:<the mod.localhost DID>"
  }'

# 4. Apply a label.
curl -i -X POST http://localhost:3000/xrpc/tools.ozone.moderation.emitEvent \
  -H "authorization: Bearer $TOKEN" \
  -H 'content-type: application/json' \
  -d '{
    "event": {
      "$type": "tools.ozone.moderation.defs#modEventLabel",
      "createLabelVals": ["spam"]
    },
    "subject": { "$type": "com.atproto.admin.defs#repoRef", "did": "did:plc:<target>" },
    "createdBy": "did:plc:<mod did>"
  }'

# 5. Read the public labels.
curl 'http://localhost:3000/xrpc/com.atproto.label.queryLabels?uriPatterns=did:plc:<target>'

# 6. See the team-lead DID document advertise the labeler.
curl 'http://localhost:3000/xrpc/com.atproto.repo.describeRepo?repo=mod.localhost' | jq .didDoc.service

# Then visit http://localhost:3000/mod (log in as mod.localhost).

Takedown enforcement on reads

A moderation decision that doesn't bite isn't a decision. After modEventTakedown flips records.takedown_ref (or blobs.takedown_ref, or accounts.status), every relevant read endpoint short-circuits before serving:

The bytes stay in repo_blocks / on disk so a modEventReverseTakedown can restore them; we only stop serving them. From the caller's perspective the moderation decision is opaque — a takendown record looks indistinguishable from a deleted one. That's the property chapter 17 leans on when it says "the network honors signals or it doesn't; we just emit."

Two read endpoints intentionally do serve takendown content because they exist for moderation work:

• tools.ozone.moderation.getRecord — moderators need to see what they're moderating
• tools.ozone.moderation.getRepo — same logic, account scope

Both are gated by requireModerator.

Known gaps

• Age-assurance event types. The ageAssuranceEvent / ageAssuranceOverrideEvent / ageAssurancePurgeEvent family needs an age-attestation store we don't ship — the surface lives in the upstream Ozone primarily for KOSA-style compliance flows that the reference operator (Bluesky) implements via a third-party vendor.
• Passive accountEvent / identityEvent / recordEvent types. These mirror what the firehose already emits as #account / #identity / #commit — we don't double-record into mod_events because the firehose log is already the canonical history.
• Chat moderation (tools.ozone.chat.*). Four endpoints — getActorMetadata, getConvo, getConvoMembers, getMessageContext. We don't self-host chat, so the surface is empty by design. The lexicons reach the AppView via proxy when a caller targets chat.bsky.team.
• (Closed — every operator-facing tools.ozone.* surface in the upstream reference now has a paired /mod page or XRPC handler: labels, safelink, templates, verifications, sets, settings, signatures, team, events, queues, reports, server.getConfig.)

Exercises

1. Add modEventMute / modEventUnmute to the supported set. The side effect would be an entry in a new mod_muted_actors table; the /mod UI gains a "muted" pill.
2. Wire subscribeLabels as a second WebSocket route alongside subscribeRepos. Reuse the srvx-attached-http server pattern from chapter 16; tail by labels.seq ascending.
3. Add a mod_report_resolution table linking each moderation_reports row to the mod_events.id that closed it, then expose a resolved / open filter on the dashboard. This is the missing piece on the per-report resolution-state gap.