---
title: "Concurrency and scheduler deadlines"
slug: architecture/concurrency
---


Saccade has an interaction path and a scene path. Their deadlines and overload behavior
are different, so they cannot share one serial loop.

## Interaction path

The interaction path owns keyboard reduction, hint filtering, pointer feedback, action
validation, and overlay publication. Its scheduler deadline is independent from the scene
path, and the path should normally be event-driven.

The interaction path reads an immutable target scene. It does not wait for capture,
accessibility, neural inference, model compilation, or a GPU fence.

Mutable hot state belongs to one thread. Capture staging, scene construction, input
reduction, and presentation scratch storage use thread-local or thread-affine fixed
buffers. Cross-thread state is reduced to bounded ownership transfers and immutable
publication. The design does not use CAS retry loops on a steady-state path.

## Scene path

The scene path has a separate neural-refresh deadline. Capture, preprocessing, inference,
postprocessing, and scene publication consume that budget when a neural refresh is
scheduled.

Inference is asynchronous: submission returns a ticket before the result is ready. A
worker or native runtime completes the ticket while input and presentation continue.
Polling, waiting, cancellation, and collection are explicit provider operations.

Region-bounded work means the scheduler can mark changed or high-value rectangles as
priority input and cap temporary work to known storage. It does not change the product
full-scope rule. A full-scope pass must still let every visible point affect the
result. Under overload, old work is canceled or replaced before a queue can build behind
the current scene.

## Bounded communication

The runtime design uses:

- A newest-frame mailbox between capture and scene scheduling.
- Bounded single-producer paths where ownership permits.
- Fixed ticket tables for asynchronous provider work.
- Immutable scenes published by handle or index.
- Explicit counters for replacement, queue pressure, cancellation, and lateness.

Queue depth is a policy, not a throughput fix. Deep queues increase latency and memory.
The normal neural path keeps at most one frame executing and one newer frame pending.

## Current implementation

Runtime creation, destruction, provider registration, and the rare global registry-domain
refill pass through one CAS-and-wait gate. This gate is never entered by frame publication,
frame release, capture callbacks, provider tickets, inference kernels, input, or overlay
presentation. Their scheduler thread owns each provider context, so the context needs no
internal serialization.

Host imports now publish generation-safe frame handles through one atomic latest-only
mailbox. Replacement and consumption are linearizable, fixed-capacity, and allocation
free. Each transfer uses one bounded atomic exchange with no retry loop, and it runs on
the scene boundary, outside the interaction path. Control-path
cancellation is serialized outside the mailbox. Producer, consumer, and control
counters are single-writer cache-line-separated fields sampled only while quiescent,
not shared atomic accumulators. A stress test accounts for every handle as either
replaced or consumed.

Registry construction obtains domain IDs from a thread-local block. Refilling a block
uses the same statically stored cold gate. Normal construction does not touch shared
allocator or atomic state. Repository checks permit CAS only in that gate and reject
mutex types everywhere.

The Windows desktop owner and DirectML worker each join the MMCSS `Games` task at high
relative priority. The owner also ensures that the process priority is at least
`ABOVE_NORMAL_PRIORITY_CLASS`. Each thread registers and reverts its own MMCSS handle
during cold lifetime transitions. The qualifier reads both registrations and rejects a
run when either is absent. The 120 Hz loop and DirectML command loop perform no scheduling
call, allocation, lock, compare-and-swap, or reference-count update.

Destructive-interference spacing is an architecture constant. Apple arm64 uses 128 bytes,
while the x86_64 path uses 64. Ordinary 64-byte buffer alignment serves a different
purpose and provides no destructive-interference spacing.

The portable dual-rate scheduler is implemented as thread-owned state. It emits at most
one interaction tick after a delayed wake, skips stale catch-up ticks, and tracks the two
deadlines independently. The scene side admits exactly one running item and one
newest pending timestamp. Another deadline replaces pending work without a queue. Scene
completion immediately promotes pending work.

The macOS scene capture set is owned by its scheduler thread. It reconciles fixed-capacity
display streams against topology epochs, acquires frames without waiting, and refuses to
tear down a stream while its frame lease is outstanding. Capture callbacks continue to
replace pending framework frames independently.

Immutable scene publication, neural ticket coordination, target fusion, hint reduction,
action-plan construction, physical input reduction, and the shared session owner are
implemented with fixed storage. The session owner keeps one immutable scene active while
labels are frozen, reduces prefix and selection events, validates current epochs, and
invokes a direct native-executor callback. The interaction-thread scene coordinator polls
one native accessibility ticket without waiting, consumes the newest neural scene,
publishes newer semantic-only snapshots between neural frames, fuses exact epoch matches,
and rejects late semantic results. It writes the fusion result directly into a scene-store
slot and allocates nothing after initialization. Native application event loops poll the
interaction path, platform capture, input leases, overlay presentation, and local agent
channels without waiting for neural completion.

Scene fusion consumes up to four already-validated desktop-Q8 packets in explicit
priority order. Accepted targets are indexed into five size-adaptive spatial levels.
Each new candidate probes only its own level and neighboring cells. A duplicate keeps
the higher-priority geometry and identity while merging provenance and capabilities.
Disabled or secure evidence dominates the merge and clears all action capabilities.
The kernel writes directly into the final immutable packet and owns no allocator path.

Native accessibility traversal remains outside the interaction owner. Windows uses one
MTA thread and Win32 events. macOS uses one pthread and Mach semaphores. Both expose one
in-flight query and one retained snapshot, publish completion with release/acquire state,
and perform framework traversal only on their worker. Coordinator polling never enters UI
Automation or Accessibility and never waits.

macOS action admission also owns a public-API secure-surface preflight. It combines the
console/login session state, frontmost process, focused Accessibility role and optional
subrole, protected-content state, ancestor window classification, and secure event input.
An activation and the final native input boundary take a fresh sample. While an action,
session, or synthetic lease is active, the owner refreshes the sample at the 250 ms
permission cadence. The ordinary idle path only reads the cached disposition.
Any missing, malformed, mismatched, untrusted, bounded-traversal, secure-text, system
dialog, or system-floating-window evidence blocks the action and follows the existing
release-all path. Diagnostics expose the bounded reason bits and sample epoch.

## Kernel work

The first owned image kernel converts packed color to exact integer luma. Its scalar
implementation is the oracle. Arm64 builds compile a NEON path, while x64 builds compile
an AVX2 path in a separate translation unit and select it only on a compatible CPU.
Tests force every available path, compare exact output, exercise vector tails, and check
that row padding is untouched.

Later CPU and GPU kernels use the same rule: optimize behind a stable tensor and output
contract, and compare against an exact fixture before measuring speed. The implemented
Metal preprocessor has one materialized output lane and no per-frame Saccade-owned
allocation. Metal and Direct3D variants must match integer geometry and ordering before
their speed matters.

Kernel tuning is measured in isolation and in the full scene path. A faster kernel that
adds copies, expands memory, blocks another queue, or changes target geometry is not an
improvement. Provider memory counters and per-stage timing make those tradeoffs visible.

## Overlay publication

The scene-rate overlay composer maps desktop-Q8 targets through one direct per-display
transform, clips them to the backing surface, resolves glyph indices, and places labels
with a fixed spatial hash. It emits a scene-index to surface-index table so active hint
updates do not search after per-display filtering. The maintained 10,000-target benchmark
measures this publication separately from GPU expansion and presentation.

## Physical input override

Native input emitters mark every synthetic event with one shared 64-bit value. Native
monitors ignore that value, so Saccade cannot trigger its own hotkeys or continuously
abort its own drag. macOS installs a listen-only `CGEventTap` after Accessibility
permission is granted. Windows owns low-level keyboard and mouse hooks on its shell
thread. Both notify a direct, allocation-free callback for physical user activity.

The application callback checks whether a synthetic action is active and invokes the
existing physical-input override or release path on the thread that owns the executor.
The monitor itself does not acquire locks, mutate an action plan, wait for inference, or
call an operating-system input injection API.

The fixed-storage interaction controller is that callback's portable ownership point.
It cancels the frozen-label session first, then asks the native executor adapter whether
its physical ledger has a lease and releases it only when needed. Hotkey actions, mode
changes, repeat action, and shell-only commands remain direct calls on the same owner
thread. `start_interaction_command` and `observe_interaction_input` fit the existing
router and monitor callback shapes without an event queue or shared action state.

The implemented scalar overlay expander validates immutable packets and produces the
same 8-byte rectangle and 4-byte metadata streams required from native compute kernels.
It is allocation-free and is the parity oracle.

The renderer contract owns three slots and requires static expansion only when a scene or
transform epoch changes. End-to-end profiling includes physical display timing in addition
to component timing. [Overlay packets and presentation](/docs/architecture/overlay) specifies the target
handoff.

macOS surface lifecycle and diagnostics are main-thread owned. The display-link callback
runs on the registered main run loop and records callback duration, missed presentation
deadlines, missing scenes, busy slots, failures, and the last rendered epochs. The frame
source is a direct function pointer plus context, not an allocating callable wrapper.
Initialized surfaces and surface sets are also destroyed on their owning main thread.
Debug builds assert that lifecycle contract.

The display catalog is thread owned. Its macOS collector rejects calls away from the
main thread before changing observations or topology state. The Windows collector
requires per-monitor DPI awareness and publishes physical desktop pixels. Rare display,
work-area, safe-area, scale, refresh, and rotation changes publish a new immutable epoch
without a shared atomic in the interaction loop.

Source: [View the pinned source](https://github.com/Cosmin-B/saccade/blob/d126ab2b05465839bf70702026081b2eaef62257/docs/architecture/concurrency.md) · [Edit this source](https://github.com/Cosmin-B/saccade/edit/main/docs/architecture/concurrency.md)