Notifications
Queue and scale
How VitNode delivers notifications to large audiences with queue tasks and cron jobs, batched delivery with saved progress, crash recovery, retention and worker settings.
Notifications run on VitNode's database-backed queue and cron. No extra service is needed, but the queue worker must run: without a working cron adapter, nothing is delivered in the background.
Queue tasks and cron jobs
All of these belong to @vitnode/core. Tasks appear in Debug Panel ā Queue (/admin/core/debug/queue) and cron jobs in Debug Panel ā Cron & queue (/admin/core/debug/cron).
| Name | Kind | Does |
|---|---|---|
notifications-fanout | Task, 10 attempts | Delivers one notification to its recipients, batch by batch |
notifications-email | Task, 5 attempts | Sends due emails and digests |
notifications-cleanup | Task, 3 attempts | Removes notifications older than the retention period |
notifications-schedule | Cron, every 5 minutes | Plans digests, recovers stalled work and queues the email task when due |
notifications-cleanup | Cron, daily at 02:00 | Queues the cleanup task |
A request that publishes starts delivery right after its response is sent. Anything that misses that, such as a publish from a queue task, waits for the queue worker, which runs every minute.
Batched delivery
The fan-out task walks the recipients in batches of fanoutBatchSize, 500 by default. Each batch is one short transaction that filters out deleted users, opted-out members and anyone access rejects, then writes inbox items, unread counts and immediate email deliveries, and saves its progress on the event. Realtime updates go out after the batch commits.
- A crash loses at most one batch. The retry continues after the last saved batch.
- A batch that runs twice changes nothing. Core keeps one receipt per notification and member, and only members without a receipt get an item.
- Large audiences do not block the queue. One task stops after 20 seconds and queues a new task for the rest.
- No deadlocks between writers. Every inbox write locks the affected members in user id order, so concurrent writers wait for each other instead.
The integration test fanout.load.integration.test.ts delivers one notification to 1,000 members with batches of 200, with two workers racing on the same event. It checks that every member gets exactly one item, that access runs about once per batch and that the number of queries stays below the number of recipients.
Recovery
| Stuck work | Recovered by |
|---|---|
| Queue task left processing by a stopped worker | The queue worker, after 15 minutes |
| Notification unfinished with no task left | notifications-schedule, after 10 minutes, by queuing a new task |
Email left in sending | notifications-schedule, after 15 minutes, by setting it back to pending |
| Two workers on one notification | A row lock. The second waits, then continues from the saved progress |
Retention
The nightly cleanup removes, in batches:
- inbox items with no activity for
retentionDays, lowering unread counts for unread ones; - older notifications that no inbox item points to;
- finished email delivery records older than
retentionDays; - pending digest email for notifications no digest picked up within 14 days. The inbox items stay.
Configure the workers
How much work each run does and how long notifications are kept are deployment settings, so they live in your API config, not in the AdminCP. Add a notifications block to buildApiConfig:
import { buildApiConfig } from '@vitnode/core/vitnode.config'
export const vitNodeApiConfig = buildApiConfig({
// other options unchanged
notifications: {
retentionDays: 30,
fanoutBatchSize: 1000,
emailBatchSize: 100,
emailConcurrency: 8,
},
})| Name | Type | Required | Default | Description |
|---|---|---|---|---|
retentionDays | number | No | 90 | Days to keep notifications, from 1 to 3,650 |
fanoutBatchSize | number | No | 500 | Recipients per delivery transaction, from 10 to 5,000 |
emailBatchSize | number | No | 50 | Emails one worker run picks up, from 1 to 500 |
emailConcurrency | number | No | 4 | Emails sent at the same time during a run, from 1 to 20 |
A value outside its range is clamped to the nearest limit, and a value that is not a whole number falls back to the default. Restart the API to apply a change.
The defaults suit most sites. Raise fanoutBatchSize when one announcement to a huge audience needs too many runs, and emailConcurrency when your email provider allows more parallel sends. Lower emailBatchSize when a slow provider makes runs time out.