Skip to main content
Overlay is a Next.js app backed by Convex, server-mediated provider adapters, and runtime-configurable auth, billing, storage, model, tool, and telemetry providers.

Source Of Truth

  • src/app/api/v1/** is the public backend surface for web, mobile, desktop, and extension clients.
  • src/server/app-api/v1/** holds the route orchestration behind those Next route wrappers.
  • src/server/** holds server-only auth, billing, storage, AI, tools, route services, and provider adapters.
  • src/shared/** holds isomorphic contracts and client-safe helpers.
  • src/contexts/** holds app-wide client providers (AuthContext, WorkspaceContext); src/hooks/** holds app-wide client hooks (useWorkspaceChanged). Both may import @/shared only.
  • src/features/** holds web feature containers and feature-local helpers. Cross-feature imports are banned; shared primitives belong in @/shared, @/contexts, or @/hooks (e.g. the workspace provider, routing helpers, and types that used to sit inside features/workspaces).
  • src/server/workflows/** holds Vercel Workflow SDK durable orchestration (automation-run, personal-chat-work, workspace-agent-turn). Workflow bodies bundle for the workflow runtime — keep their imports to "use step" functions and @/ alias modules only.
  • convex/** owns durable app state, domain mutations, scheduled work, and Stripe webhook handling.
  • apps/admin/** holds the standalone operator console (see internal/develop/admin-console.md): its own Next.js app, WorkOS sign-in, session cookie, and a single convex-admin module that reaches the admin/console:* Convex functions with ADMIN_API_SECRET. apps/admin may not import src/** and src/** may not import it back.
  • scripts/ is split by purpose: scripts/ci/ (checks, boundary rules, baselines consumers), scripts/release/ (hosted build wrappers and trusted PR preview lifecycle control), scripts/qa/ (smoke tests, rehearsal/readiness harnesses), scripts/db/ (migrations, backfills, ops), scripts/lib/ (shared script helpers). scripts/dev-setup.sh stays at root. The retired staging ignore-build script is removed; overlay-landing uses native Git deployment and the hosted build wrapper.
  • packages/overlay-app-core/** holds shared cross-surface contracts and UI settings types.
  • packages/overlay-api-client/** holds typed transport wrappers for /api/v1/**.
  • packages/overlay-ui/** holds the shared presentation primitives (buttons, inputs, dialogs, and the list-page system: Tile/TileGrid/TileIcon/TileSkeleton/CreateTile for grid tiles, ListRow for list rows, and HeaderSearch for collapsible header search). List pages (Files, Agents, Extensions) must render their tiles, rows, and headers through these primitives instead of hand-rolled markup so spacing, radii, hover language, and dark-mode surfaces stay uniform.
  • packages/overlay-agent-bridge-protocol/** owns the vendor-neutral, versioned Zod wire protocol between Overlay and connected execution environments.
  • packages/overlay-agent-host/** is the portable outbound-only host executable. It owns local device keys, SQLite delivery state, filesystem grants, and fake/ACP adapters; it does not own workspace identity, transcripts, approvals, policy, or billing.
  • packages/overlay-sandbox-runtime/** is the provider-neutral managed execution boundary. It owns sandbox lifecycle, command/file/port/snapshot/network/usage contracts and provider translations (Box today, E2B planned). Provider SDK types and raw handles do not cross into public routes, billing, or connected-agent persistence.
App data lives exclusively in Convex. There is no database-provider switch or capability set to check: src/server/app-data/repositories.ts constructs the Convex repositories the server context hands to domain services, and module-level HTTP services construct their Convex repository directly. Browser code reads and writes Convex natively and never asks which backend is in use; npm run check:native-architecture keeps it that way (no provider switch, no browser token argument, and no new browser HTTP call for data Convex already serves). The bespoke Postgres repository layer, Drizzle schema, app-data migrations, and parity harness were removed; enterprise self-hosting runs the official convex-backend binary (see internal/develop/convex-only-self-hosting.md). Connected-agent persistence goes through the Convex-backed ConnectedAgentRepository. Environment, binding, command, remote-session, approval, sandbox-lease, enrollment, proof-challenge, short-lived credential, and request-replay records live in Convex tables. Remote output advances a per-session contiguous event cursor transactionally; it projects into the existing conversation and AgentRun lifecycle rather than creating a second transcript. Negotiated connected-agent text streams persist in bounded remoteMessageTextChunks owned by the existing assistant message. The message holds stable text references and a short preview cache; authorized transcript, room, context, share and knowledge reads hydrate canonical text before returning it. Storage boundaries do not become Markdown boundaries. Chunk updates emit conversation events while avoiding full message rewrites, and hard deletion cascades to the message’s chunks. Legacy rows remain readable without migration. Liveness heartbeats live on the run, independently of transcript text. A connected-agent environment is a machine or runtime boundary, not a working-directory or single-agent boundary. Bindings choose an agent and adapter; each command chooses an explicitly granted working directory. The host can grant multiple roots or explicit OS-user-wide access. Protocol versioning, validation, sequencing, enrollment proof canonicalization, and signed HTTP transport remain in the bridge/host packages so Convex does not become a public host API. Convex owns a separate TypeScript path map for its transitive shared imports. Align bridge-protocol source aliases with the web config so clean backend builds do not require generated package dist files. ConnectedAgentControlPlaneService is the server-only policy boundary for enrollment and host traffic. Browser management routes use the normal authenticated BFF and active-workspace resolution. Host routes are explicit BFF exceptions because they use outbound machine auth: method-scoped opaque credentials plus Ed25519 request proofs and atomic nonces. Route handlers never select a provider directly; both modes use ConnectedAgentRepository from app data. Overlay Cloud machines are provisioned for Computers and, later, for hosted other-agents; the private runtime is Box. The browser sees Overlay Cloud, an environment, and lease state, not a provider object. Clients should not call model providers, billing providers, identity admin APIs, object storage, or Convex directly unless a route explicitly exists for that surface. src/app/_components/AppShellLayout.tsx is the reusable web shell boundary. Canonical /app/** routes, ordinary auth flows, and public full-page product destinations compose it from their route-group layouts. The shell accepts an explicit showcase mode so public routes do not depend on hidden rewrite query parameters. Protocol callbacks, integration popups, public share links, and visual fixtures remain standalone by contract.

Domain Decomposition

Web file upload transport and error parsing live in src/features/knowledge/adapters/webKnowledgeFilePort.ts; KnowledgeView composes those ports into the shared surface. Keeping the upload implementation out of the host preserves the knowledge surface boundary check. Workspace behavior stays behind the existing service/repository contracts while large seams are split by responsibility. The WorkspaceSwitcher controller owns navigation and lifecycle state; WorkspaceSwitcherView owns the trigger, menu, and dialog presentation. Personal and organization workspaces use the same collaboration capabilities: people, agents, direct messages, channels, teams, invitations, and resource grants all follow the same service and repository contracts. Personal differs only in lifecycle: it is the account’s permanent default workspace, cannot be archived through workspace lifecycle APIs, and its creating principal remains the canonical active owner. Permanent account-deletion erasure is a separate audited compliance path.

Work (tasks, board, timeline)

work is a content domain: tasks that appear in a List, a Board and a Timeline, in Personal or Workspace scope. Where things live:
  • src/shared/work/ — the isomorphic rules: statuses and limits (status.ts), fractional string ranks (rank.ts), parent and “blocked by” validation (graph.ts) and who may work on a task (permissions.ts). Convex and the UI import these, so what the UI previews is what the backend enforces.
  • convex/work/workItems.ts and workItemEvents.ts — the workItems and workItemEvents tables. Visibility, move, archive and restore reuse convex/lib/resourceScope.ts and scopeMutations('workItems', 'content'); the activity log is written by afterApply and by each mutation. Helpers are in convex/lib/workItems.ts.
  • Editing is wider than for most content: any member can work on a workspace task when the workspace lets members edit workspace content, and the assignee always can. Move between scopes, archive and restore stay with the creator and admins, exactly as for other content.
  • Dependencies (blockedBy) are set through update, which reads the tasks they lead to and refuses a loop with checkBlockedBy (src/shared/work/graph.ts); blockers must be active tasks in the same scope. The Timeline draws them with dependencyArrow (src/shared/work/timeline.ts).
  • The assignee is a workspace principal (a person now, an agent later). Dates are YYYY-MM-DD strings so a due date never shifts with timezone.

Personal Chat execution lifecycle

Interactive Personal Chat runs use the provider-neutral AgentRun contract in src/shared/agents/agent-run.ts. AgentRun.status is the only persisted authority for queued, running, approval-waiting, completed, failed, or cancelled execution; conversationMessages.status is a rendering projection. Convex creates the assistant placeholder and AgentRun together, and completes, fails, or cancels both records in one storage transaction. Personal Chat uses one durable AgentRun lifecycle. A successful backend capability response selects the Overlay Harness; a successful readiness response from an older backend selects the existing workflows/personal-chat-work.ts workflow. Readiness errors remain fail-closed and do not switch to a direct ToolLoop. Both paths stream the UI message response over the Act SSE connection and persist the final assistant content, usage, routed model, and terminal AgentRun state. On reconnect, the client polls GET /api/v1/conversations/run and reloads the final message; a non-terminal run shows the disconnected generating state. Automation, service-auth, and room execution retain their separate paths. The two durable engines perform the same authorization, context and memory construction, model policy, tool filtering, billing reservation, and user-message persistence. Workflow calls and tool executions are durable steps; the workflow identity remains stored on its AgentRun. Approval-required tools transition AgentRun to waiting_for_approval; Stop cancels the run and its workflow. New personal AgentRuns use mode: "chat"; historical mode values and engine ownership remain readable and unchanged. Approval-required tools transition AgentRun to waiting_for_approval with a durable hook token and tool-call summary. POST /api/v1/conversations/run/approval resumes the hook; the workflow returns the same run to running. Stop cancels both AgentRun and the Workflow run. Internal Overlay tool mutations receive agent-run:<runId>:tool:<toolCallId> as their idempotency key. Connector providers outside Overlay must still honor their own idempotency semantics; Workflow step replay alone cannot make an external API idempotent. Both execution engines write comparable observations onto the terminal AgentRun. Provider token usage and estimated provider cost, tool outcomes and retries, cancellation timing, browser-disconnect outcomes, lease/process-loss observations, and stale-run detection feed GET /api/v1/conversations/run/metrics. The report keeps raw sample counts and denominators and never recommends a runner; product decisions must be made from the resulting evidence.

Auth

  • Browser auth uses the signed httpOnly overlay_session cookie.
  • Browser overlayAppClient requests, and direct app fetches that use fetchWithSessionRecovery (workspace bootstrap and policy reads), recover one same-origin /api/v1/* 401 through the shared browser-session broker and retry once after an authenticated refresh. AuthContext and Convex token recovery use that same in-flight refresh and Web Lock. Delayed responses from before a completed refresh do not start another refresh. The retry preserves JSON bodies, workspace headers and idempotency keys; it excludes explicit bearer credentials, omitted cookies, redirects, request/stream bodies, and server-side callers. A failed refresh or second 401 remains a failure; 403 permission denials are never retried. API handlers stay read-only with respect to session rotation.
  • Native/mobile/desktop auth uses supported bearer access tokens against the same /api/v1/* backend.
  • Internal server-to-server calls use short-lived service auth from src/server/auth/service-auth.ts.
  • Shared route authentication flows through the API boundary helpers in src/server/auth/app-api-auth.ts.
Every sensitive /api/v1/* route should reject requests that lack a valid browser session, mobile bearer token, or service auth token.

App Bootstrap

GET /api/v1/bootstrap is the canonical signed-in startup call. It returns user info, entitlements, UI settings, model catalogs, feature flags, navigation destinations, and defaults. New clients should start from this route and reuse existing API contracts before adding client-specific endpoints.

Billing

Stripe is the hosted billing provider for subscriptions, top-ups, auto top-up preferences, customer portal sessions, and webhook events. Private deployments can disable billing with runtime config. Convex is the durable entitlement/usage source of truth after webhook processing. Billing ownership is represented separately from user identity by an additive BillingAccount contract. An account has exactly one scope owner: a user for personal, or a workspace for workspace. markup_25_v1 explicitly preserves the current 25% markup policy. New personal accounts start with an empty budgeted balance so account creation never grants spend implicitly. The compatibility migration creates one personal account per user, attaches legacy billing and usage rows, and dual-writes user and account identifiers plus canonical subscription/balance shadows. User-keyed records remain the enforcement and rollback source until balance parity and exact Stripe-subscription verification pass; the migration never switches reads automatically. Migration balance parity sums only active reserved and reconcile_required rows through the compound billing-account/status index. Historical finalized and released reservations are deliberately excluded, so a high-volume account does not hit the active-reservation safety cap merely because it has more than 1,000 completed operations. BillingPayerResolver separates workspace authorization from payment selection. The workspace-wallet master flag and off -> internal -> selected -> general rollout select which organization workspaces have switched. Personal workspaces and non-selected organizations retain the compatibility path. Once selected, an organization call requires an active workspace account and has no personal fallback. Workspace reservations update the account-keyed balance and an optional member or programmatic fixed-period limit atomically; the original personal reservation method remains unchanged for rollback. The app bootstrap and subscription read use that same resolver. Active members of an eligible organization therefore receive the workspace account’s plan and budget for model gating, while mutating billing operations remain restricted to owners and admins. Workspace-scoped clients send the active workspace header explicitly; a member’s personal subscription is not treated as the organization’s entitlement or payment fallback. All enabled hosted-cost boundaries now pass the resource workspace plus a member or programmatic subject into the resolver. This includes shared chat and named agents, Convex knowledge embeddings, browser/Daytona, hosted media and transcription, gateway search, and automation/API execution. Scheduled automations use automation:{id} and API keys use api-key:{id}. Vercel Workflow overhead is a separate 20-event estimate charged at the provider event rate plus markup_25_v1. Integrations, functions, and MCP execution have no per-turn step limit; the workspace budget stops the work, and an MCP result is shortened only past 400,000 characters. File ingestion has an abuse rate limit (500 requests per user and 1,000 per IP per hour) plus the plan storage cap. src/server/billing/billable-feature-coverage.ts is the launch inventory and its test fails when a category lacks metering or a concrete quota. Box is the managed sandbox provider (Vercel Sandbox and Daytona were removed; E2B is planned for self-hosting). Machines hold nothing in advance: a machine starts only when its workspace has at least 2% of its budget left, and a per-lease meter charges observed usage periodically while the sandbox runs. Each tick reads provider instance.usage() counters, computes the delta against a cursor persisted on the lease (usage.meteredUsage + meterVersion), and commits debit + cursor atomically through meterSandboxLease on the connected-agent repository — retries replay as lease_conflict instead of double-charging. When a charge leaves the workspace under 2% of its budget (after one automatic top-up attempt), uses up an admin spend limit, or a tick is refused, the lease flips to stopping, pending commands and queued/running turns are cancelled with a usage_limited terminal error, and the provider sandbox is deleted by the reaper pass; a turn’s settle is just the final meter tick. All metering and cleanup reconnects use {resume:false} — passive reads never wake an idle-stopped sandbox. Some providers only publish cumulative usage counters once a session stops, so while a sandbox runs the meter bills the elapsed window against a virtual wall-time cursor instead; teardown paths (reap, revoke) stop the instance, take one final meter read that reconciles real totals against the virtual cursor, then delete — session usage is never lost. The meter also enforces the lease’s idle window: Box has no native idle timer, so a running sandbox whose lease shows no activity for usage.idleTimeoutMs (default 15 minutes) is provider-stopped while the lease stays running — every turn-slice acquire heartbeats usage.lastActiveAt, so an actively working sandbox never cools mid-turn, and the next turn’s resume:true acquire wakes it again. A Computer lease’s reservedUntil is a rolling session window, not a cap: while the Computer is in use (recent activity, an open action or a live run) the meter pushes it forward, and only the idle check stops the machine. A legacy environment lease (none are created any more) is still reaped once it runs past reservedUntil. The meter driver runs from the BFF agent-environments/operations/reconcile route (Convex cron calls it). The standalone sandbox route keeps the worst-case reservation model because it always denies outbound network access and can make a byte-exact provider-cost promise; it rejects oversized artifacts before provider transfer, and its reservation includes the maximum bounded artifact transfer. The default iad1 region and its configured unit rates must move together. Blank pricing or buffer variables fall back to conservative defaults rather than silently becoming zero. Workspace Stripe ownership is account-keyed, never admin-keyed. Owners and admins may initialize the account and open checkout, top-up, verification, and portal flows, but Stripe metadata and durable subscription state bind to billingAccountId plus workspaceId; userId is retained only as the initiating actor. The Convex Stripe webhook preserves the personal metadata path, deduplicates event IDs, and rejects stale subscription state with providerEventCreatedAt. Paid usage is never estimated or held before a call. Each paid call checks the workspace balance first (refusing only when less than 2% of the budget is left, after one automatic top-up attempt) and charges the actual cost afterwards through UsageRepository.recordBatch, keyed by a unique operation id. The budgetReservations table holds only historical rows; convex/billing/retireHolds.ts released the holds left from before this change. Important code:
  • src/shared/billing/billing-pricing.ts
  • src/shared/billing/billing-account.ts
  • src/shared/billing/billing-account-migration.ts
  • src/server/billing/billing-runtime.ts
  • src/server/billing/stripe-billing.ts
  • src/server/billing/BillingCustomerService.ts
  • src/server/billing/BillingPayerResolver.ts
  • src/server/billing/billable-feature-coverage.ts
  • src/server/billing/automation-workflow-billing.ts
  • src/server/billing/WorkspaceBillingService.ts
  • src/server/billing/providers/stripe-billing-provider.ts
  • convex/billing/subscriptions.ts
  • convex/billing/accounts.ts
  • convex/billing/accountSubscriptions.ts
  • convex/billing/spendLimits.ts
  • convex/platform/usage.ts
  • src/server/usage/UsageRepository.ts
Personal account plans and their verification script are retired. Billing QA targets workspace plans, balances, and payer resolution through src/server/app-data/contracts/convex-workspace-billing-provider-contract.test.ts.

Storage

Uploaded files and generated assets are stored through server-mediated flows. Cloudflare R2 remains the managed object store, but it is not used for DNS, web hosting, request proxying, or chat streaming. R2 and S3-compatible object access must remain private and owner-scoped; users should only receive short-lived, validated access.

Chat Streaming

Interactive chat streams directly from the Vercel-hosted /api/v1/conversations/act route to the browser. Persistent conversations tee a server-side reader so generation, usage accounting, and message persistence continue after a browser disconnect. Reconnects recover from the authoritative persisted conversation state; there is no external stream relay or stream-auth endpoint.

Observability and Metrics

Growth is counted from the database, not from analytics events. A nightly Convex job (convex/platform/growth.ts) writes daily totals: what was created, units of work, active users, W1/W4 retention cohorts, paid plans, top-ups and billed usage. The admin console’s Growth section reads only those rollups. PostHog keeps a small role:
  • src/components/providers/ObservabilityClient.tsx — $pageview with a redacted URL, and identify / reset as the session changes.
  • src/server/observability/posthog-server.ts — one server event, user.signed_up, from the user.created lifecycle event, so signups can be tied to the pages that brought them in. The same client serves feature flags (src/server/releases/feature-flags.ts).
Errors and traces go to Sentry. src/app/api/v1/_utils/bff.ts logs one [BFF_METRIC] line per request to the server logs.

Privacy

Analytics events contain only redacted page URLs, route names, opaque user IDs and the sign-in provider. No conversation content, message bodies, authorization tokens, private workspace JSON, or raw user data is sent to analytics.

Convex

Development commonly uses a separate dev deployment from production. Follow Worktree Staging QA and Convex Workflow for the deployment lane that matches the current revision. Deploy Convex only from the matching release worktree:
Never run a Convex deploy from a feature worktree, never use convex:push:all, and do not pass .env.local to production Convex deploy commands; that can point deploys at the dev slug.

Desktop web shell

The macOS desktop loads the same web app and routes in a sandboxed Electron window. @overlay/desktop-bridge defines the typed, versioned contract for native capabilities such as dictation, panels and screenshots. Browser code uses those capabilities only after a successful handshake; ordinary browsers omit the native controls. The desktop validates origin, top-frame identity and arguments again in main. Web owns chat, agent execution, files, billing and the cookie session.