Collaboration app scaling
Collaboration app scaling and launch checklist
A collaboration app stays fast by paginating every list, indexing the main questions, and deferring heavy work until a measured trigger fires. At launch, apply 11 rules, watch 10 signals and hold the performance targets below.
Last checked
Rules to apply at launch
- Fanout: Writes are small batched appends to doc_updates, not rewrites of a document row; clients coalesce keystrokes for 50-200 ms and send one batch, and the server allocates the whole seq range with a single UPDATE documents SET head_seq = head_seq + n RETURNING, so a busy document costs one row-lock per batch, not per keystroke.
- Realtime: Fan-out uses LISTEN/NOTIFY as a wake-up carrying only (doc_id, seq) (never the payload, which can exceed the 8 KB NOTIFY limit); each web process holds one LISTEN connection per node, not one per client, and reads the tail by (doc_id, seq) from the unique index.
- Realtime: Hard caps enforced at stream open and failing closed: MAX_EDITORS_PER_DOC (default 50 writers), MAX_VIEWERS_PER_DOC (default 500 read-only streams), MAX_STREAMS_PER_NODE (default 2000), MAX_STREAMS_PER_USER (default 10). Over the cap the server returns 429 with Retry-After; it never degrades silently.
- Realtime: Each stream has a bounded send buffer (default 1 MB or 200 events). A consumer that falls behind gets one `resync` event (snapshot key + tail seq) and the stream is closed; the client rebuilds from snapshot. Producers are never blocked by slow consumers, and memory per stream is constant.
- Idempotency: Every update batch carries (client_id, client_op_id); INSERT ... ON CONFLICT (doc_id, client_id, client_op_id) DO NOTHING returns the already-assigned seq range. CRDT updates are also idempotent by construction, so reconnect storms and double-sends never corrupt or duplicate state.
- Data growth: Compaction bounds both cold-load time and table size: a snapshot is cut every COMPACT_AFTER_UPDATES (default 500) updates, 5 MB of updates or 10 idle minutes; cold load is always one snapshot fetch + at most that many updates. doc_updates covered by a snapshot are deleted after the retention window.
- Rate limits: Per user per document: max 120 update batches/min and 1 MB per batch; per user: max 20 new documents/hour; per IP on public share links: 60 req/min; presence upserts max 10/s per session. Limits are token buckets keyed in Postgres (batched) and checked before any work.
- Pagination: Document lists, version history, comments and activity use keyset cursors on (sort_key, id) with page size <= 50; the tail read for sync is keyed by seq and capped at 500 rows per round trip.
- Caching: Snapshots are immutable objects in the bucket (key includes up_to_seq) served with Cache-Control: private, max-age=31536000, immutable; the app caches document metadata and the caller's resolved permission for 30 s in a per-process cache keyed by (user, doc), invalidated by NOTIFY on share changes. The editor shell is static and edge cached.
Performance targets
| What to measure | Target |
|---|---|
| Local edit to remote render (same region) | p95 < 500 ms on SSE; p95 < 150 ms on the websocket tier |
| Update append API | p95 < 80 ms, 2 statements per batch (seq allocation + multi-row insert) |
| Cold load of a typical document (<= 1 MB state) | p95 < 1.5 s to interactive: 1 snapshot fetch + <= 500 tail updates |
| Presence update propagation | p95 < 400 ms; cursor updates coalesced to <= 10 per second per client |
| Editor LCP | < 2.5 s; editor shell served from edge cache |
| Reconnect catch-up after 5 min offline | p95 < 2 s for <= 2000 missed updates (paged tail reads) |
| Compaction | p95 < 10 s for a 10 MB log; never blocks writers |
What to watch
- Metric update to delivery ms: histogram from append commit to SSE write on the receiving stream; the core real-time health number.
- Metric stream connections: gauge per node and per document, with cap utilisation; drives the room-cap and capacity alerts.
- Metric doc log depth: updates since last snapshot per document; p99 above 2x COMPACT_AFTER_UPDATES means compaction is behind.
- Metric resync events: count of backpressure resyncs; a spike means consumers or the notify path are slow.
- Metric update reject rate: oversize, rate-limited, forbidden and invalid updates by reason.
- Log structured sync logs: request id, doc id, user id hash, seq range, batch size; never payload bytes or tokens.
- Trace append -> notify -> deliver trace: one trace id from the POST through NOTIFY to the SSE write.
- Slo sync availability: 99.9% of update appends succeed and 99% are delivered within 2 s; error budget reviewed monthly.
Recurring work
- compact_document: runs queue (jobs) enqueued when updates since snapshot >= COMPACT_AFTER_UPDATES or bytes >= 5 MB, plus idle for 10 min. Guard: one job per doc at a time (unique key), merge in a resource-limited batch task, verify state hash before deleting covered updates, attempt cap 3.
- sweep_presence: runs cron 1m. Guard: deletes rows older than 60 s in bounded batches of 1000.
- prune_snapshots: runs cron daily. Guard: applies the plan retention policy, never deletes named snapshots, bounded batch and dry-run counter logged.
- export_document: runs queue (jobs). Guard: per-user concurrent cap, output size cap, exports charged to the workspace quota.
- rollup_collab_usage: runs cron 5m. Guard: reads counters in batch, upserts collab_usage by (workspace_id, day); idempotent.
- reindex_document_text: runs queue after compaction. Guard: extract plain text capped at 1 MB, skip when text hash is unchanged.
Add when the numbers say so
- Read replicas: Serve version history, comments lists, search and cold-load tail reads from a read replica once primary CPU stays > 60% or read p95 > 200 ms; the live tail read stays on the primary to avoid replica lag causing missed updates.
- Data growth: Partition doc_updates by hash of doc_id and shard large workspaces across tenant databases once a single database exceeds the storage or write ceiling; seq allocation stays per document so ordering is preserved.
Deferred until it is needed
| Deferred | Adopt it when |
|---|---|
| Dedicated websocket sync tier (Yjs protocol, one owner node per document) | > 5k concurrent streams, update-to-delivery p95 > 800 ms, or rooms regularly > 50 editors. |
| Redis or NATS pub/sub for cross-node fan-out | NOTIFY rate > 5k/s or primary CPU > 60% attributable to notify/tail reads. |
| Read replicas for history, comments and search | primary CPU > 60% sustained or read p95 > 200 ms. |
| Dedicated search engine (OpenSearch/Typesense) | > 5M documents or search p95 > 300 ms. |
| Cold-storage archive of old snapshots | snapshot storage > 50% of total storage cost. |
On Forge64
Forge64 matches your description to one of 12 reference architectures, writes the plan and data model into the project, generates the app, and runs a quality gate over the code before you see it. Each app gets its own Postgres database and sign-in with no per-user fee. Hosting on Forge Cloud and Forge Payments are rolling out.
Every metered resource on Forge64 has a hard cap, so a traffic spike stops at your limit instead of becoming a surprise bill. This is a reference design for the category, not a promise that every item is generated today. Built-ins such as email, background jobs, file uploads and search are coming soon, so anything here that needs them is something you add yourself for now.
FAQ
- When should I worry about scale for a collaboration app?
- Not before a measured trigger fires. For example, for "Dedicated websocket sync tier (Yjs protocol, one owner node per document)": > 5k concurrent streams, update-to-delivery p95 > 800 ms, or rooms regularly > 50 editors.
- What performance targets should a collaboration app aim for?
- Local edit to remote render (same region): p95 < 500 ms on SSE; p95 < 150 ms on the websocket tier. Update append API: p95 < 80 ms, 2 statements per batch (seq allocation + multi-row insert). Cold load of a typical document (<= 1 MB state): p95 < 1.5 s to interactive: 1 snapshot fetch + <= 500 tail updates.
- Does Forge64 cap my usage costs?
- Yes. Every metered resource has a hard cap, and paid plans add an overage spend cap that you set.
Build your app with Forge64
One prompt to a running app with auth, database, hosting and payments. Start free, no credit card required.
Start building for free