# savka777/jev-use

> Native macOS voice/typed computer-use app that reads the Accessibility tree and has the Jev model pick each action from a closed list.

- Category: [Browser & computer control](https://llms-technical-reviews.com/browser-control/)
- Repository: https://github.com/savka777/jev-use (reviewed at commit `b22e568adf110eeb64f89bf4d731a69d23c461dc`, 2026-09-21)
- Stars: 119 · Language: Swift · License: MIT
- Canonical page: https://llms-technical-reviews.com/p/jev-use/

## Overview

jev-use (the app is called "Desktop Voice") is a menu-bar app for macOS 14.2+ that turns a spoken or typed sentence into actions on your own desktop. It controls any app, not only browsers: it opens apps, folders and sites, clicks controls, types into fields, scrolls, uses menus and tiles windows. Browsers are just apps with a large Accessibility tree.

Its design choice is narrow and deliberate: **the model never writes anything**. Each step, the app walks the front app's Accessibility tree, builds a numbered table of controls and a closed list of operations, and sends them to TypeSafe's Jev model (`jev-latest`), which only answers multiple-choice and yes/no questions. Even the text to type is selected, not generated: Jev picks the first and last word of the span inside your own sentence. Swift code performs the action with Accessibility calls or synthetic `CGEvent` input, then reads the screen again.

The project is small (about 4,500 lines of Swift in two modules) with no third-party dependencies. It reads as a harness tuned by trial: many thresholds in the code carry comments explaining which logged failure they fixed.

## Architecture

```mermaid
flowchart LR
  V["Voice: SFSpeechRecognizer / hotkey / wake phrase"] --> M["AppModel.run"]
  T["Typed command / say.sh"] --> M
  M -->|"default"| C["runCycles (up to 14)"]
  M -->|"PlannerEnabled + key"| P["Planner.plan (OpenRouter)"]
  P --> E["executePlanned per step"]
  C --> D["Desktop.capture: AX walk"]
  E --> D
  C --> J1["JevClient.cycle"]
  E --> J2["JevClient.ground"]
  J1 --> TS["TypeSafe /v1/systemone"]
  J2 --> TS
  C --> X["Desktop.perform: AX actions + CGEvent"]
  E --> X
```

| Component | Path | Role |
|---|---|---|
| App model and loops | `Sources/JevDesktop/JevDesktopApp.swift` | `AppModel`: speech/typed entry, `run`, `runCycles`, `executePlanned`, window-chain `replay`, SwiftUI widget and settings |
| Desktop layer | `Sources/JevDesktop/Desktop.swift` | AX tree walk into `DesktopSnapshot`, `perform` for every `DesktopAction`, settling and change detection |
| Jev client | `Sources/JevCore/Decision.swift` | Request builders (`cycleBody`, `groundingBody`), HTTP calls, `Decision` parsing |
| Planner | `Sources/JevCore/Planner.swift` | Optional OpenRouter call that splits a sentence into `PlanStep`s |
| Command parsing | `Sources/JevCore/CommandInput.swift` | Extracts URLs, counts and durations from the sentence |
| Voice input | `Sources/JevDesktop/SpeechInput.swift`, `HotKey.swift`, `WakePhrase.swift` | Hold-to-talk, hands-free, "Hey Jev" |
| Secrets | `Sources/JevDesktop/KeyStore.swift` | TypeSafe and OpenRouter keys in the Keychain |

## How a request flows

Take "Go to youtube.com, search Rick Astley and play the first video" with default settings:

1. **Entry.** Speech or the text field calls `run`, which cancels any running task and chooses a path. The planner path runs only if a planner key exists **and** the `PlannerEnabled` user default is true. Otherwise it calls `runCycles` ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L397-L478)).
2. **Read.** Each cycle calls `Desktop.capture`, which walks the focused window off the main thread ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L186-L200)). After a navigation it re-reads for up to about 3 s until a browser page shows new controls ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L656-L697)).
3. **Offer.** Code builds an element table (index, role, label, value, page or toolbar) and an operation menu: `CLICK`, `TYPE_TEXT`, `OPEN_APP`, `OPEN_URL`, `MENU`, `PRESS_RETURN`, `SCROLL_*`, `SKIP_*`, `ARRANGE_WINDOWS`, `WAIT`, `DONE`, `BLOCKED` and others. Toolbar controls are hidden unless the sentence mentions tabs or the address bar. Lists are trimmed to stay under TypeSafe's 255-option limit ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L699-L777)).
4. **Choose.** `JevClient.cycle` sends one request: an `operation` choice, one speculative target choice per operation (`click_target`, `type_target`, `type_from`/`type_to` over the sentence's words, ...), and yes/no heads such as `finishes`, `create_first`, `counted` and `every_window` ([Decision.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L240-L303)).
5. **Gate.** Code may override the pick (for example `TYPE_TEXT` becomes `CLICK` or `MENU` when Jev says the wanted input is missing). An unsure Return (confidence < 0.5) is not pressed. A target below confidence 0.2, or 0.6 for destructive labels such as quit, delete or close, stops and asks "Which one?" ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L786-L865)).
6. **Act.** `Desktop.perform` brings the target app back to the front, checks the window and the control's label are unchanged, then acts ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1089-L1160)).
7. **Check.** It waits for a fingerprint change, records the effect ("window is now ...", "no visible effect") in `recentActions`, and loops. It returns early when Jev's `finishes` head was ≥ 0.8 and the action visibly worked. It stops on `DONE`, `BLOCKED`, three no-change actions, the same pick three times, or 14 cycles ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L866-L1010)).

## Key components

### Accessibility capture

`read` fetches about 20 attributes per element in one `AXUIElementCopyMultipleAttributeValues` round trip and names unlabeled inputs from their title element or static-text value ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L112-L152)). The walk skips secure text fields and hidden nodes, clips to scroll areas and web areas, and drops controls more than 200 px outside the visible region. Lists with more than 50 children fall back to the app's own visible-rows subset, which keeps very large tables fast ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L489-L523)). Chromium and Electron apps get extra waits because their web content builds the tree late.

### Executing actions

`DesktopAction` is a closed enum: open app/folder/website, quit, AX press, select row, click at centre, focus, key, scroll, type, position and arrange windows. Opening goes through `NSWorkspace`. Clicks on controls that accept no AX press are posted as `CGEvent`s, and the cursor is put back afterwards ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1051-L1064)). Typing first tries setting `kAXSelectedText`, falls back to per-character Unicode key events, and reads the value back to report whether the text actually appears ([Desktop.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1231-L1269)). Window tiling is pure arithmetic in code, with no model call per window.

### Optional planner

`Planner.plan` sends the sentence, the front app and the running apps to OpenRouter (default `inception/mercury-2.5`, temperature 0, reasoning off, strict JSON schema). It gets back steps from an 11-kind vocabulary ([Planner.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Planner.swift#L101-L153)). `executePlanned` runs keys, scrolls, skips and URLs directly in code. On-screen steps get one narrow `JevClient.ground` question ("which listed target does this step mean?" plus `already_done`) ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L481-L609)).

### Every-window chains

When Jev's `every_window` head says the goal applies to all windows, the steps that worked in the first window are recorded as role/label/ordinal descriptions. `finish()` replays them in the other windows without asking Jev again, unless the finish itself was uncertain ([JevDesktopApp.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L636-L655)).

## Extending it

There is no plugin surface. Changes are code edits:

- **New operation.** Add a `DesktopAction` case and its branch in `Desktop.perform`, add the operation description in `runCycles`, and map it to a target head.
- **Prompt rules.** All Jev instructions are string constants in `Decision.swift` (`cycleRules` and the per-head instructions).
- **Planner model.** `defaults write local.jev-use PlannerModel <openrouter id>`. To turn the planner on at all: `defaults write local.jev-use PlannerEnabled -bool true`, plus a key saved in Settings.
- **Diagnostics.** `defaults write local.jev-use DumpTree -bool true` logs every walked AX node. All logs are under the `local.jev-use` subsystem.

## Running it

- **Build.** Xcode and `bash build.sh`, which builds a release binary, signs it locally and installs `~/Applications/Desktop Voice.app`.
- **Permissions.** Accessibility, microphone and speech recognition. Keys are stored in the Keychain under service `local.jev-use` ([KeyStore.swift](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/KeyStore.swift#L4-L24)).
- **Services.** A TypeSafe API key is required. An OpenRouter key is optional. Apple Speech may send audio to Apple's servers.
- **Use.** Hold Control-Option-Space and speak, use hands-free mode (about 1.5 s of silence submits), say "Hey Jev", or run `scripts/say.sh "Open Finder"`.

## Strengths and caveats

- **Strength: no free text from the model.** Operations, targets and typed spans are all choices over offered options, which makes the agent cheap, fast and hard to talk into inventing actions.
- **Strength: whole-desktop reach.** The same loop drives Finder, native apps, Electron apps and browsers without per-app code.
- **Strength: careful verification.** Label re-checks before acting, read-back after typing, and effect strings fed into the next decision.
- **Caveat: macOS only, and accessibility-limited.** Apps or web pages with poor AX labelling (canvas UIs, unlabeled custom controls) are hard or impossible to target. There is no vision fallback.
- **Caveat: hidden planner switch.** The Settings text says an OpenRouter key enables planning. In code, `run` also needs a `PlannerEnabled` default that no UI sets, so a saved key alone changes nothing.
- **Caveat: dead fallback path.** `JevClient.decide` and the older `execute` loop (batched 251-option questions) are never called. Readers of `Decision.swift` should start from `cycle` and `ground`.
- **Caveat: your real session.** It acts on your logged-in apps and browser profile with only heuristic confirmation for destructive labels, and the control table (labels and values, not secure fields) goes to TypeSafe.

*Sources: code at b22e568, deepwiki-open wiki (12 pages), verified Q&A.*

## How savka777/jev-use answers the Browser & computer control questions

### How is the page represented to the model? (answered)

The page is represented **exclusively through the macOS Accessibility tree** — no screenshots, no DOM serialization, no set-of-marks images are captured or sent. The `Desktop.capture()` function (`Desktop.swift:187`) walks the Accessibility tree starting from the frontmost application's window (`AXUIElement` root), recursing through `AXUIElementCopyMultipleAttributeValues` to read roles, names, values, positions, enabled/hidden state, children, and DOM identifiers for every control (method `read()`, `Desktop.swift:112-151`). The walk is bounded: children lists longer than 50 nodes are pruned to the app's reported visible subset (`Desktop.swift:139-144`). Controls outside a clipped frame within a −200 px margin are skipped as off-screen (`Desktop.swift:514-517`). Unnamed text-less controls that are not recognised interactive roles are dropped (`Desktop.swift:633`). Secure fields (`kAXSecureTextFieldSubrole`) are skipped entirely (`Desktop.swift:504`). The walk also performs a pointer-coordinate sweep (`Desktop.swift:557-573`) to catch elements the tree walk missed. The result is serialised as a flat list of `Element` structs (index, role, label, value, place, operations) plus available apps/folders/sites/menus (`Decision.swift:198-222`). This is sent alongside recent actions as a JSON payload to the TypeSafe API. The model sees numbered elements in screen order (top-to-bottom, left-to-right, `Desktop.swift:678-683`), each described with '[index] role 'label' value...' strings. Total candidates are trimmed to stay under TypeSafe's 255-option-per-question limit (`Decision.swift:82`); elements are batched into multiple parallel choice questions when needed (`Decision.swift:95-124`). The `fingerprint` function (`Desktop.swift:203-218`) provides a cheap structural hash (window title, focused control, first 400 nodes, window position/size) used to detect screen changes between cycles.

> **Editor's note.** Correction: in the live loop (`runCycles`), oversized control lists are trimmed to about 250 options, preferring labels that share words with the command; the multi-batch splitting in `Decision.swift` `requestBody`/`decide` is only used by the `execute` loop, which nothing calls.

Citations: [Sources/JevCore/Decision.swift:198-222](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L198-L222) · [Sources/JevDesktop/Desktop.swift:112-151](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L112-L151) · [Sources/JevDesktop/Desktop.swift:557-573](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L557-L573) · [Sources/JevDesktop/Desktop.swift:678-683](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L678-L683) · [Sources/JevCore/Decision.swift:82-84](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L82-L84) · [Sources/JevDesktop/Desktop.swift:203-218](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L203-L218)

### How are actions executed and how are elements targeted? (answered)

Actions are executed through a **closed vocabulary of `DesktopAction` types** (`Desktop.swift:13-30`) and dispatched in `Desktop.perform()` (`Desktop.swift:1089-1326`). The action types are: `application` — opens/switches to an app via `NSWorkspace.shared.openApplication()`; `quit` — terminates a running app via `NSRunningApplication.terminate()`; `folder` — opens a Finder folder; `website` — opens a URL in a specific browser; `press` — performs an Accessibility press action on an AXUIElement (`AXUIElementPerformAction`); `select` — sets `kAXSelectedAttribute` on a row/cell (`Desktop.swift:1036-1046`); `clickAt` — synthesises a mouse click at the element's centre point using `CGEvent` posted via system tap (`Desktop.swift:1051-1064`); `focus` — sets `kAXFocusedAttribute`, falling back to a centre click if Accessibility focus fails (`Desktop.swift:1067-1080`); `key` — posts keyboard events via `CGEvent` (`Desktop.swift:1223-1228`); `scroll` — posts scroll wheel events (`Desktop.swift:1229-1230`); `type` — enters text by setting `kAXSelectedTextAttribute` or by posting per-character keyboard Unicode events (`Desktop.swift:1231-1269`); `position` — sets window position/size; `arrange` — tiles/columns/cascades all windows with arithmetic layout (`Desktop.swift:1278-1323`). **Element targeting is by Accessibility element reference** (`AXUIElement`), not by selector or coordinate index. The `DesktopSnapshot` maps candidate IDs to `DesktopAction` values that contain the actual `AXUIElement` pointer (`Desktop.swift:361-362`). Before acting, the code verifies the target element still exists and its label matches (`Desktop.swift:1160-1162`). Opening a website uses `NSWorkspace.shared.open()` which delegates to the OS, not CDP or Playwright.


Citations: [Sources/JevDesktop/Desktop.swift:13-30](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L13-L30) · [Sources/JevDesktop/Desktop.swift:1089-1100](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1089-L1100) · [Sources/JevDesktop/Desktop.swift:1231-1269](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1231-L1269) · [Sources/JevDesktop/Desktop.swift:1051-1064](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1051-L1064) · [Sources/JevDesktop/Desktop.swift:1036-1046](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1036-L1046) · [Sources/JevDesktop/Desktop.swift:1158-1165](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1158-L1165)

### How is the agent loop / planning implemented? (answered)

The agent loop lives in `AppModel.run()` (`JevDesktopApp.swift:397`) and takes two forms. **Default loop (no planner key):** `runCycles()` (`JevDesktopApp.swift:613-1010`) runs up to 14 cycles. Each cycle: captures the Accessibility tree → builds a `CycleState` with elements, available apps/folders/sites/menus, recent actions, and dictation → sends one `JevClient.cycle()` request (`Decision.swift:289`) that asks for an operation (choice) plus speculative targets (choices for click_target, type_target, app_target, etc.) and several yes/no questions (`finishes`, `create_first`, `every_window`, `counted`) in a single model call → executes the chosen action → detects screen change → loops until DONE, BLOCKED, WAIT, or exhaustion. **Planner path (with OpenRouter key):** `Planner.plan()` (`Planner.swift:137`) sends the utterance to OpenRouter with a JSON schema (`response_format: json_schema`, `Planner.swift:110-111`) to decompose into ordered `PlanStep` kinds (`open_app`, `open_url`, `click`, `type_text`, `press_key`, etc.). Then each step is grounded via a narrow `JevClient.ground()` call (`JevDesktopApp.swift:560`) that asks Jev to select the correct on-screen target. Deterministic steps (press key, scroll, skip, open URL) execute without any model call (`JevDesktopApp.swift:489-512`). **Stop conditions:** DONE, BLOCKED, WAIT; three consecutive no-change actions stop; same pick three times stops; a 14-cycle hard cap. **Memory between steps:** `recentActions` holds the last 10 actions (`Decision.swift:206`). `priorCommand` and `priorAction` carry context across clarification questions (`JevDesktopApp.swift:89-90`). The `JevClient.session` is a keep-alive TLS connection (`Decision.swift:306-310`).

> **Editor's note.** Correction: the planner path needs both a saved OpenRouter key and the `PlannerEnabled` user default (set only via `defaults write`), so the default for every user is the per-cycle loop. WAIT is not a stop condition: it sleeps 0.4 s and the loop continues.

Citations: [Sources/JevDesktop/JevDesktopApp.swift:397-478](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L397-L478) · [Sources/JevDesktop/JevDesktopApp.swift:613-1010](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L613-L1010) · [Sources/JevCore/Decision.swift:289-303](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L289-L303) · [Sources/JevCore/Planner.swift:137-153](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Planner.swift#L137-L153) · [Sources/JevCore/Decision.swift:306-310](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L306-L310) · [Sources/JevDesktop/JevDesktopApp.swift:1160-1206](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L1160-L1206)

### How are failures, retries and self-healing handled? (answered)

**Error classes caught:** `DesktopError` with a `stale` flag (`Desktop.swift:6-11`), `DecisionError.invalidResponse` (`Decision.swift:388-393`), `ServiceError` for TypeSafe HTTP errors (`Decision.swift:373-385`), `PlannerError` for OpenRouter failures (`Planner.swift:156-173`), `CancellationError` for user Escape. **Retries:** When `stale=true` fires on a `DesktopError`, the cycle retries the same action after a fresh capture, up to 2 times (`JevDesktopApp.swift:931-933`). In `executePlanned()`, if grounding returns no candidate, it retries up to 2 times with a 700 ms sleep (`JevDesktopApp.swift:571-581`). If `stale` fires during `Desktop.perform()`, the step re-captures and retries up to 2 times (`JevDesktopApp.swift:601-607`). In the fallback loop, `unavailable` retries twice (`JevDesktopApp.swift:1146-1152`). **Screen-change detection:** `fingerprint()` (`Desktop.swift:203-218`) gives a structural hash checked before and after each action. `waitForChange()` (`Desktop.swift:242-250`) polls it for change. `waitForQuiet()` (`Desktop.swift:229-238`) waits for the fingerprint to stabilise. Before execution, `Desktop.perform()` hit-tests the target centre point to confirm nothing is covering it (`Desktop.swift:1176-1186`). **Chain replay:** When Jev judges the goal applies to every window, `replay()` (`JevDesktopApp.swift:1021-1086`) repeats the recorded chain across all other windows without further model calls, matching targets by name, kind, place, and ordinal; a missing target after 12 attempts is reported, not guessed. **Timeout:** The TypeSafe HTTP request has a 20-second timeout (`Decision.swift:309`).

> **Editor's note.** Correction: in `runCycles` a stale-target error is not retried as the same action; it is recorded as 'NOT performed' and the next cycle captures again and lets Jev re-decide, bounded only by the 14-cycle cap and the no-change/same-pick stops. The cited 'fallback loop' (`execute`) is never called.

Citations: [Sources/JevDesktop/Desktop.swift:6-11](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L6-L11) · [Sources/JevDesktop/Desktop.swift:229-250](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L229-L250) · [Sources/JevDesktop/JevDesktopApp.swift:931-935](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L931-L935) · [Sources/JevDesktop/JevDesktopApp.swift:571-582](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L571-L582) · [Sources/JevDesktop/JevDesktopApp.swift:1021-1086](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/JevDesktopApp.swift#L1021-L1086) · [Sources/JevCore/Decision.swift:309-309](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L309-L309)

### Which models are supported and how are they called? (answered)

**Two models serve different roles.** The **primary model** is Jev (`jev-latest`), accessed through the TypeSafe API at `https://api.typesafe.ai/v1/systemone` (`Decision.swift:90,177-178`). Jev is a vision-free decision model: it receives element tables (not screenshots) and chooses from closed choice vocabularies. Calls use JSON-over-HTTPS with no streaming, no tool-calling, and no structured output schema — the response is always a `Decision` JSON object with choice/noul answer types (`Decision.swift:15-22`). Vision is not required because the page is represented entirely through the Accessibility tree. The **optional planner model** goes through OpenRouter (`https://openrouter.ai/api/v1/chat/completions`, `Planner.swift:138`). It defaults to `inception/mercury-2.5` (`Planner.swift:43`), overridable via `defaults write local.jev-use PlannerModel` (`Planner.swift:45`). The planner uses JSON schema `response_format: json_schema` (`Planner.swift:110-111`) for structured step output, low temperature (0), no reasoning tokens (`Planner.swift:106-108`), and 900 max tokens. When the planner is absent, the system falls back to the per-cycle loop where Jev handles both sequencing and targeting in each request. **How they are called:** Both are plain HTTPS POST requests. TypeSafe receives a state+questions JSON (`Decision.swift:88-140`) and returns JSON answers. OpenRouter receives a standard chat completions request (`Planner.swift:101-112`). API keys are stored in the macOS Keychain (`KeyStore.swift:6-41`). The provider distinction is sharp: TypeSafe for the decision/grounding model, OpenRouter only for the initial planning step.


Citations: [Sources/JevCore/Decision.swift:88-92](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L88-L92) · [Sources/JevCore/Decision.swift:177-194](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Decision.swift#L177-L194) · [Sources/JevCore/Planner.swift:43-48](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Planner.swift#L43-L48) · [Sources/JevCore/Planner.swift:101-113](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Planner.swift#L101-L113) · [Sources/JevCore/Planner.swift:137-153](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevCore/Planner.swift#L137-L153) · [Sources/JevDesktop/KeyStore.swift:6-41](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/KeyStore.swift#L6-L41)

### How are browser sessions, profiles, auth and anti-bot handled? (answered)

**Local macOS-only.** The app controls the user's own desktop — there is no remote/cloud browser, no Docker, no VNC. Browser control is through the macOS `NSWorkspace.shared.open()` API, which launches or activates the native browser binary (`Desktop.swift:1133`). The app reads a list of known browsers by bundle identifier (`Desktop.swift:60`): Brave, Chrome, Safari, Edge, Firefox, plus any Chromium/Electron app detected by the presence of a `(Renderer).app` helper (`Desktop.swift:67-77`). **Persistent profiles and cookies** are not managed by the app — it opens URLs through the OS, so the user's existing browser profile and cookies are used as-is. **Stealth/anti-bot** is not implemented: there is no user-agent manipulation, no browser fingerprint spoofing. **Proxies** are not configured by the app — the system proxy configured in macOS is used by default. **CAPTCHA handling** is not implemented and not claimed; if a site presents a CAPTCHA, the model cannot solve visual challenges. The code includes a `waitForPageToSettle()` function (`Desktop.swift:970-993`) that polls the Accessibility tree until the control count stabilises after page navigation. **Auth/Keychain:** API keys for TypeSafe and OpenRouter are stored in the macOS system Keychain using the `SecItem` API (`KeyStore.swift:14-36`) under the service name `local.jev-use`. **Wake phrase** detection is purely client-side text matching (`WakePhrase.swift:5-33`): the app matches "Hey Jev" (or the variant "Hey Jeff") at the start of an Apple Speech transcript fragment — no dedicated wake-word engine.


Citations: [Sources/JevDesktop/Desktop.swift:60-77](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L60-L77) · [Sources/JevDesktop/Desktop.swift:1128-1138](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L1128-L1138) · [Sources/JevDesktop/Desktop.swift:970-993](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L970-L993) · [Sources/JevDesktop/KeyStore.swift:14-36](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/KeyStore.swift#L14-L36) · [Sources/JevDesktop/WakePhrase.swift:5-33](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/WakePhrase.swift#L5-L33) · [Sources/JevDesktop/Desktop.swift:203-218](https://github.com/savka777/jev-use/blob/b22e568adf110eeb64f89bf4d731a69d23c461dc/Sources/JevDesktop/Desktop.swift#L203-L218)