Compare commits

...

47 Commits

Author SHA1 Message Date
Arvin Xu 6f723a2a6c test(server-config): preserve bootstrap flag fallback 2026-04-24 01:02:54 +08:00
Arvin Xu 231d1bdcf7 ♻️ refactor(runtime-config): drop feature flag bootstrap injection 2026-04-24 00:50:31 +08:00
Arvin Xu 92f34bcc0d feat(runtime-config): add redis-backed feature flag provider (#14098)
*  feat(runtime-config): add redis-backed feature flag provider with env fallback

* 🐛 fix(runtime-config): cache null snapshots in redis provider

* 🐛 fix(ci): sort runtime-config imports

* 🐛 fix(runtime-config): evict expired selector cache entries
2026-04-24 00:18:27 +08:00
Innei 7955a43a9e feat(desktop): gate screen capture on macOS permission and add overlay hint (#14097)
*  feat(desktop): gate screen capture on macOS recording permission

Prompt a native dialog before opening the capture overlay when macOS
Screen Recording permission is missing, with an Open Settings button
that deep-links to System Settings.

* 💄 style(desktop): add hint pill to screen capture overlay

Bottom-left pill with three grouped hints (hover to pick a window, drag
to crop a region, Esc to exit), sharing the WindowTag pill language.
Hidden during drag and after a selection so it doesn't clutter.

* 🚨 fix(test): mock MarketService in execGroupAgent integration test

The first test case was timing out (~9.5s) because execAgent makes a
real HTTP request to market.lobehub.com via MarketService.getLobehubSkillManifests().
Mock MarketService to return empty skill manifests, eliminating the
network dependency that caused the cold-start timeout in CI.
2026-04-24 00:06:27 +08:00
Innei fa0ec62d71 🐛 fix(conversation): stop repinning after manual scroll (#14099)
* 🐛 fix(conversation): stop repinning after manual scroll

* 🐛 fix(conversation): avoid stale pin cancellation
2026-04-23 23:45:06 +08:00
Arvin Xu 3b94f86303 🐛 fix(desktop): surface human approval notifications (#14092)
🐛 fix(desktop): notify when human approval is required
2026-04-23 23:29:51 +08:00
Rdmclin2 05b2aca92b 🐛 fix: remote device disabled in bot converation (#14096)
fix: remote device disabled in bot converation
2026-04-23 22:57:42 +08:00
Rdmclin2 e4b15caf74 feat: support bot emoji (#14091)
* feat: support bot emoji

* chore: add local bot error message

* feat: add emoji  replace action

* feat: add emoji reaction

* fix: test case
2026-04-23 19:25:45 +08:00
Arvin Xu 82096dcd89 feat(heterogeneous-agent): add Codex desktop integration (#14067)
*  feat(heterogeneous-agent): integrate Codex desktop MVP

*  feat(heterogeneous-agent): polish Codex profile and install guidance

* 🐛 fix(heterogeneous-agent): stabilize Codex desktop error handling

*  improve codex desktop integration

*  feat(desktop): support custom heterogeneous CLI commands

* 💄 style(profile): refine heterogeneous CLI status card

* 🐛 fix(chat): persist heterogeneous CLI auth errors

* 💄 style(profile): align CLI card radius with container

*  feat(chat): add heterogeneous CLI rate-limit guide

* 🐛 fix(heterogeneous-agent): split Codex multi-turn steps

* 📝 docs(skill): add heterogeneous-agent debugging guide

* ♻️ refactor: split heterogeneous agent status guide and fix i18n fallback

* 🐛 fix(heterogeneous-agent): align Codex step and tool-call boundaries

* 💄 style(skills): use capsule chip in activate inspector

* 🐛 fix(chat): resolve status guide type errors
2026-04-23 19:18:51 +08:00
LiJian 66d096e963 🐛 fix(creds): integrate Klavis authorization status into lobe-creds system (#14090)
*  feat(creds): integrate Klavis authorization status into lobe-creds system

Inject Klavis connected/available services into the creds systemPrompt so
agents are aware of Klavis-managed OAuth authorizations and stop asking
users for manual tokens. Add connectKlavisService API to allow agents to
initiate Klavis OAuth connections from within chat conversations.

Fixes LOBE-7243

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix(creds): cleanup dangling intervals and add server runtime for connectKlavisService

- Clear windowCheckInterval in cleanup to prevent dangling interval
- Add connectKlavisService to CredsExecutionRuntime for server-side support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-23 17:47:10 +08:00
Innei 50ffa5b100 🐛 fix: prevent Markdown stream replay when vlist remounts streaming items (#14086)
* 🐛 fix: prevent Markdown stream replay when vlist remounts streaming items

Long streaming replies replayed the token-by-token animation when users
scrolled them out of view and back. virtua VList was recycling streaming
items, so the Markdown component lost its animation state on remount.

- Pin currently-streaming messages via `keepMounted` on the VList so
  their DOM stays mounted regardless of scroll position.
- Scope the `animated` flag to the last answer segment inside an
  AssistantGroup. Finalized blocks now render as static markdown, so any
  future remount cannot replay completed content.

* ♻️ refactor: drop redundant `animated` prop drilling in AssistantGroup

The store already exposes per-block streaming state via
`isMessageGenerating(blockId)`: the streaming write target's
DB message id (== block.id) is associated to the running operation,
so finalized blocks naturally resolve to `generating=false` and the
active block to `true`. The prop drilling added in the prior commit
only duplicated this and did not actually prevent replay on the
streaming block itself.

Keep the real fix (`keepMounted` on the VList) which pins the
streaming item so vlist recycling never resets the Markdown
animation state in the first place.

*  feat: pin text-selection hosts in vlist keepMounted

Recycling a virtualized item whose node hosts a Selection anchor or
focus silently drops the user's highlight. Track message ids that
currently contain an active selection via a `selectionchange` listener
and merge their indices into `keepMountedIndices` alongside the
streaming pins.

- New hook `useSelectionMessageIds` walks Selection range endpoints up
  to the nearest `[data-message-id]` host and returns a stable Set of
  ids, returning the previous reference when the set is unchanged.
- VirtualizedList merges selection indices with streaming indices and
  hands the union to VList's `keepMounted`.
2026-04-23 17:24:40 +08:00
renovate[bot] 8e20bd182f Update dependency uuid to v14 [SECURITY] (#14083)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-04-23 15:51:42 +08:00
AmAzing- 53b4b4d4d3 feat(chat): inline resend when editing last user message (#14080) 2026-04-23 15:47:56 +08:00
Innei decbc4ce7f ♻️ refactor: alias buffer package as buffer.js for cleaner imports (#14081)
Replace the awkward `from 'buffer/'` trailing-slash workaround with a
pnpm alias `"buffer.js": "npm:buffer@^6.0.3"`, so import sites read
`from 'buffer.js'`.
2026-04-23 15:10:29 +08:00
Innei 4e31a33599 🐛 fix: strip manifest link in Vite dev to silence 404 warning (#14079)
Dev server does not serve /manifest.webmanifest, which causes a console
404 in the browser. Add a shared dev-only Vite plugin that removes the
<link rel="manifest"> tag via transformIndexHtml for web/mobile/desktop.
2026-04-23 15:10:00 +08:00
YuTengjing cad10007ef 📝 docs(skills): add sub-issue tree guide to linear skill (#14076) 2026-04-23 11:33:30 +08:00
René Wang 73860a9ffd 📝 docs: add April 20 weekly changelog (#14072) 2026-04-23 10:38:46 +08:00
Hardy 4696968edb 🐛 fix: add env var support for Coding Plan and OpenCode providers (#14064)
* 🐛 fix: add env var support for missing Coding Plan providers

Add zod schema and runtimeEnv mappings for BailianCodingPlan,
GLMCodingPlan, MinimaxCodingPlan, and VolcengineCodingPlan in llm.ts.
These were missing when the providers were added in #13203, causing
them to fall back to OPENAI_API_KEY instead of their own env vars.

* 🐛 fix: add env var support for OpenCode Zen and OpenCode CodingPlan providers

Add zod schema and runtimeEnv mappings in llm.ts for OpenCodeZen and
OpenCodeCodingPlan providers introduced in #13943. Without these,
getParamsFromPayload falls back to OPENAI_API_KEY.
2026-04-23 10:31:14 +08:00
Hardy 48760e353a feat: add OpenCode Zen and OpenCode Go providers (#13943)
*  feat: add OpenCode Zen and OpenCode Go providers

Add support for OpenCode Zen (dynamic model gateway) and OpenCode Go
(subscription-based coding plan) with full model definitions, runtime
implementations, and provider configurations.

- OpenCode Zen: curated models via single API key, dynamic model fetching
- OpenCode Go: coding models (GLM, Kimi, MiMo, Qwen, MiniMax)
- Both use @ai-sdk/openai-compatible runtime
- Go models include abilities, pricing, and extendParams settings

*  feat: add 35 preset models to OpenCode Zen provider

Populate OpenCode Zen with all non-deprecated models from models.dev API
including Anthropic (9), OpenAI (13), Google (2), Zhipu GLM (2), Alibaba
Qwen (2), Kimi (1), MiniMax (2), Nvidia (1), and OpenCode (1). Switch
from dynamic model fetching to static model list.

* ♻️ refactor: migrate OpenCode Zen/Go to RouterRuntime and align extendParams

Migrate both providers from openaiCompatibleFactory to createRouterRuntime
to match OpenCode's native multi-SDK architecture:

Zen (4 routers):
- anthropic for Claude, google for Gemini, openai+Responses for GPT-5.x,
  openai fallback for all others (GLM/Kimi/MiniMax/Qwen)

Go (2 routers):
- anthropic for MiniMax M2.5/M2.7, openai fallback for all others

Fix model-bank extendParams to match OpenCode variants() behavior:
- Remove extendParams from GLM/Kimi/MiniMax/BigPickle/Nemotron (variants return {})
- Change Qwen from enableReasoning+reasoningBudgetToken to reasoningEffort
- Change Go MiMo to reasoningEffort

* 🐛 fix: fix OpenCode Zen/Go Anthropic baseURL and remove Google router

- Add stripV1() to strip trailing /v1 from baseURL for Anthropic SDK
  since it auto-appends /v1/messages to the base URL
- Remove Google router from Zen - Gemini models fall to openai-compatible
  fallback as Zen Gateway does not support Google SDK format
- Keep user-configurable baseURL support while preventing /v1 duplication

* 🐛 fix: add missing package.json exports for opencode and stepfunCodingPlan

*  feat: limit default enabled models to latest versions for OpenCode Zen/Go

Zen: claude-opus-4-7, gemini-3.1-pro, gpt-5.4, glm-5.1,
     minimax-m2.5-free, nemotron-3-super-free, big-pickle
Go: glm-5.1, qwen3.6-plus, minimax-m2.7

* 🐛 fix: include opencodego in Coding Plan provider tag check

* ♻️ refactor: align model display names with official provider naming

Update Qwen3.6 Plus, Qwen3.5 Plus, and MiMo-V2 Omni display names
to use spaces instead of hyphens, matching the official provider naming
convention used in lobehub.

* ♻️ refactor: rename opencodego to opencodecodingplan for suffix consistency

Rename internal ID from opencodego → opencodecodingplan to align with
other Coding Plan providers. Display name remains "OpenCode Go".
This allows isCodingPlanProvider() suffix check to work without exceptions.

* 🐛 fix: remove broken stepfunCodingPlan export — file not on this branch

* ♻️ refactor: align MiMo-V2 Pro display name with official provider naming

* 🌐 i18n: add Chinese translations for OpenCode Coding Plan and Zen providers
2026-04-23 02:13:09 +08:00
Tsuki 70e7e441b2 🔨 chore: premerge Task detail page UI (#13653)
*  feat: add AgentTaskList component on agent welcome page (LOBE-6597)

- AgentTaskList with TaskListHeader, TaskItem, and styles
- Embedded in AgentWelcome below ToolAuthAlert
- Each task rendered as independent rounded card with status badge
- Status: green filled circle (Done), blue circle (In progress)
- Card width matches chat input (960px)
- i18n keys for taskList.title and taskList.viewAll
- Fix updateReview type to use TRPC-inferred type

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add Tasks page at /agent/:aid/tasks with route, breadcrumb, and view toggle (LOBE-6597)

- Register tasks route in both desktopRouter.config.tsx and .desktop.tsx
- Thin route page at src/routes/(main)/agent/tasks/index.tsx
- Feature components in src/features/AgentTasks/: page, breadcrumb, header with list/kanban toggle, full task list
- Wire up "View All Tasks" navigation from AgentTaskList welcome card
- Add i18n keys (taskList.activeTasks, taskList.breadcrumb.task) and generate translations via pnpm i18n

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add Task detail page at /agent/:aid/tasks/:taskId (LOBE-6597)

- Register :taskId child route in both desktopRouter configs
- TaskDetailPage with auto-save hint, breadcrumb, and scrollable content
- TaskDetailHeader: editable title (borderless Input), Run/Pause button, status/priority tags, delete
- TaskInstruction: click-to-edit Markdown with debounced auto-save
- TaskSubtasks: sub-issues list with status badges
- TaskActivities: timeline with topic/brief/comment icons
- TaskItem now navigates to detail page instead of just setting activeTaskId
- Add taskDetail.* i18n keys with generated translations

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add TaskModelConfig, TaskScheduleConfig, and refine Task detail UI (LOBE-6597)

Add model/provider selector and periodic execution config to Task detail page.
Refine TaskDetailHeader, TaskInstruction with auto-save and i18n support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: refine Task detail UI with Linear-style design (LOBE-6597)

- Redesign SubTasks with collapsible header, progress circle, hover + click navigation
- Redesign Activities with agent avatar, comment input box, and Linear-style layout
- Add TaskParentBar showing parent task relationship with sibling navigation popover
- Add delete confirmation modal using App.useApp().modal.confirm
- Move ModelSelect to separate row below action bar
- Fix zustand selector recreation in ActivityItem
- Replace hardcoded colors with cssVar tokens

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add Properties panel, parent link hover, activity icon, and lifecycle save status (LOBE-6597)

- Add TaskProperties sidebar with collapsible status/priority dropdowns
- Parent bar: clickable parent link with hover, sibling navigation popover on progress
- Activity title: add BotMessageSquare icon
- Fix lifecycle actions not updating taskSaveStatus (saving/saved indicator)
- Filter status dropdown to only user-selectable states (backlog/completed/canceled)
- Add test task creation script for dev

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add recursive tree view for subtasks with Linear-style connecting lines (LOBE-6597)

- Add buildTaskTree utility to convert flat getTaskTree API response into nested tree
- Implement SubtaskTreeItem recursive component with CSS connecting lines (├─ and └─)
- Fetch full task tree via taskService.getTaskTree for nested subtask display
- Show loading spinner during tree fetch, fallback to flat list on error
- Remove padding-inline from AgentTaskList container

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: address PR review — delete redirect, debounce cleanup, schedule resync (LOBE-6597)

- Redirect to task list after successful delete (P1)
- Clean up instruction debounce timer on unmount/task switch to prevent stale writes (P1)
- Resync TaskScheduleConfig local state when active task changes (P2)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: use backend nested subtasks directly, remove buildTaskTree (LOBE-6597)

Backend now returns nested subtasks in task.detail (LOBE-6814).
Remove buildTaskTree utility, getTaskTree API call, and loading state.
Use TaskDetailSubtask from @lobechat/types instead of local interface.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  perf: add optimistic update and save status for model config change (LOBE-6597)

updateTaskModelConfig now immediately reflects new model/provider in UI
via optimistic store dispatch, and tracks taskSaveStatus (saving/saved).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  perf: skip redundant refreshTaskDetail on successful model config update (LOBE-6597)

Optimistic update is trusted on success — no need for full detail re-fetch.
Aligns with updateTask pattern. Refresh kept only in error path for revert.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: use backend author info for activities, fix AgentTaskList after AgentHome refactor (LOBE-6597)

- Activity: use act.author (TaskDetailActivityAuthor) from backend instead of agentMap lookup (LOBE-7013)
- AgentTaskList: fix agentId from useParams instead of useAgentStore.activeAgentId (was undefined)
- AgentHome: integrate AgentTaskList into new AgentHome layout (replaces old AgentWelcome)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: show participant avatars on task cards, use backend author for activities (LOBE-6597)

- TaskItem: display up to 3 participant avatars next to task title (LOBE-6805)
- Activity: use act.author from backend instead of agentMap lookup (LOBE-7013)
- AgentHome: integrate AgentTaskList into new AgentHome layout
- Revert AgentTaskList/TaskItem agentId back to useAgentStore (works correctly when mounted)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: fix type safety, memoize participants filter, extract avatar styles (LOBE-6597)

- Use TaskParticipant type instead of `any` in filter/map
- Compute displayParticipants once with useMemo (was filtering twice per render)
- Move avatar overlap styles to CSS classes (was inline objects per render)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🔇 chore: hide kanban view toggle until implemented (LOBE-6597)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: export TaskStatus/TaskPriority/TaskActivityType from @lobechat/types (LOBE-6597)

Replace hardcoded string/number types with shared type aliases:
- TaskStatus: 'backlog' | 'canceled' | 'completed' | 'failed' | 'paused' | 'running'
- TaskPriority: 0 | 1 | 2 | 3 | 4
- TaskActivityType: 'brief' | 'comment' | 'topic'

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: update

* style: update

* style: update

* style: update

* style: update

* style: update

* style: update

* style: update

* style: update

* style: update

*  feat: add Daily Brief module to homepage (#13851)

*  feat: add Daily Brief module to homepage

Add a Daily Brief section below the chat input on the homepage that
displays unresolved briefs from the Agent Tasks system. Users can
resolve, comment, and provide feedback directly from the brief cards.

- Service: BriefService with listUnresolved, resolve, markRead, addComment
- Store: Independent Zustand store (src/store/brief/) with SWR data fetching
- Components: BriefCard, BriefCardActions (dynamic action buttons),
  BriefCardSummary (Markdown with expand/collapse), CommentInput (@lobehub/editor)
- Three action types: resolve (closes brief), comment (resolve with text),
  link (safe URL navigation with protocol validation)
- Fixed feedback button: adds task comment without resolving the brief
- Inline success state ("Feedback sent") with 1.5s auto-restore
- i18n: zh-CN + en-US translations
- Tests: 21 tests across service, store selectors, and components
- CLI: Register task and brief commands for local development

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add agent avatars to Daily Brief cards

Display stacked agent avatars next to brief card titles using the
new `agents` data from Arvin's enriched listUnresolved API (#13489).

- Add AgentAvatarInfo type and agents field to BriefItem
- Render overlapping circular avatars (20px, -6px overlap)
- Use cssVar.colorBgContainer for border (dark mode compatible)
- Extract avatar style to function to avoid inline object creation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: clean up Daily Brief components

- Extract duplicate success state JSX into reusable SuccessTag component
- Remove redundant comments that describe what code does
- Use DEFAULT_AVATAR from @lobechat/const instead of hardcoded emoji

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: address PR review feedback for Daily Brief

- Use cssVar.colorBgBase instead of hardcoded #fff for primary button
  text color (dark mode contrast fix)
- Add submitting state to CommentInput to prevent duplicate submissions
  (disable buttons + show loading during async submit)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🌐 chore: generate i18n translations for Daily Brief

Run pnpm i18n to generate translations for all 18 locales.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: use shared BriefType from @lobechat/types

Export BriefType union from packages/types and use it in
BRIEF_TYPE_COLOR and BRIEF_TYPE_ICON records for compile-time
key validation. Adding a new brief type now requires updating
the shared type, and TypeScript will flag missing mappings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: update

* style: update

* style: update

---------

Co-authored-by: Tsuki <976499226@qq.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: update

* style: update

* style: update

* style: update

* fix: stopPropagation

* fix: i18n

* 🐛 fix: wire comment inputs to editor instance so Send actually submits

CommentInput in AgentTasks and DailyBrief used antd TextArea inside
@lobehub/editor's ChatInput while reading content via
editor.getDocument('markdown'). The TextArea was never connected to the
editor instance, so getDocument always returned empty and handleSubmit
short-circuited silently — Send appeared to do nothing (no network
request fired).

Replace the TextArea with <Editor editor={editor} type="text"
variant="chat" /> so useEditor() actually drives the editable surface.
Keep plain-text behavior via markdownOption={false} +
enablePasteMarkdown={false}, and bind Cmd/Ctrl+Enter submit via
onPressEnter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: use participant.title after TaskParticipant schema rename (#13877)

PR #13877 renamed TaskParticipant.name → .title and added
.backgroundColor. Our branch's UI code (AgentAvatars, listViewOptions,
TaskList group header, Breadcrumb) was already written against the new
schema, but TaskProperties still read firstParticipant?.name — update
the last remaining call site so the type matches post-rebase.

backgroundColor is already plumbed through everywhere it applies within
#13877's scope; TaskActivities' TaskDetailActivityAuthor is a separate
type untouched by the PR and kept as-is.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: resolve type-check errors exposed after canary rebase

canary upgraded react-i18next to a version with typed i18n keys and
tightened @lobehub/editor's SendButton + IEditor APIs. Rebase pulled
these in, surfacing latent type errors in LOBE-6597 code.

- CommentInput: use editor.cleanDocument() (IEditor's actual API;
  clearContent never existed).
- TaskActivities / TaskLatestActivity / TaskTriggerTag: type t as
  TFunction<'chat'> so typed i18n accepts the known-literal keys used
  inside module-level helpers.
- TaskPriorityTag / TaskStatusTag / listViewOptions: add
  defaultValue: '' to dynamic-key t() calls (template literals and
  Record lookups) to match the broad-key i18n overload.
- BriefCardActions: swap unusable <SendButton> (no children, no
  iconPlacement) for <Button>; add defaultValue to the dynamic
  brief-action key lookup; drop stale @ts-ignore.
- DailyBrief/CommentInput: drop unsupported children on SendButton;
  keep label via title attribute.
- Recents/Item: type TYPE_ICON_MAP as Partial<Record<...>> so 'task'
  (rendered via TaskStatusIcon elsewhere) is a safe absent key.
- brief/slices/list/action: cast briefService.listUnresolved() result
  back to BriefItem[] (TRPC serialization widens BriefType to string).
- AgentTasks/TasksHeader: delete dead file — no importers and its
  ./style module was removed by an earlier refactor.

Also ran pnpm install to materialize the newly-extracted
@lobechat/agent-gateway-client workspace package (canary #13866),
clearing ~7 "cannot find module" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor(builtin-tool-task): polish task tool paths (#13869)

*  feat: navigate to task detail when clicking brief card header

Clicking the header row of a Daily Brief card (icon + title + time +
agent avatars) now jumps straight to the associated task, using the
brief's task-tree agent (with activeAgent / inbox as fallback).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: show parent task ids as clickable breadcrumb trail

Walk the cached parent chain from taskDetailMap and insert each ancestor's
identifier as a link between the "任务" entry and the current task name in
the task detail breadcrumb.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add cross-agent /tasks page with View All Tasks on Daily Brief

- Register `/tasks` route in desktop (web + Electron) and mobile router configs
- `useFetchTaskList` supports `allAgents` mode via options object API to fetch
  tasks without agent filter; backend already supports optional assigneeAgentId
- `Breadcrumb` accepts optional `agentId`, renders "All tasks" crumb when absent
- `AgentTaskItem` navigation uses `task.assigneeAgentId` so clicks work from
  the cross-agent page (falls back to `activeAgentId` for unassigned tasks)
- Extract `useScenarioEnabledTools` hook to share layout effect between
  `/tasks/_layout` and `/agent/:aid/tasks/_layout`

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ♻️ refactor: use assigneeAgentId for task avatar instead of participants array

Replace AgentAvatars (took participants[]) with AssigneeAvatar (takes agentId,
resolves meta from agent store). This correctly represents that a task is
assigned to a single agent via assigneeAgentId/detail.agentId.

- New AssigneeAvatar component reads agent meta from agent store by ID
- TaskProperties reads activeTaskAgentId from task detail store
- listViewOptions uses task.assigneeAgentId directly for groupBy/sort
- Extract shared isInboxAgentId helper to eliminate 4x inline duplication
- Group headers resolve agent title at render time via AssigneeLabel component

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: enable vertical scrolling on cross-agent tasks page

Add overflowY and flex to WideScreenContainer wrapper so the task list
can scroll when content exceeds viewport height.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add re-assign task agent with popover selector

- Add AssigneeAgentSelector component with Popover agent list
- Extract useAgentDisplayMeta hook for consistent agent name/avatar resolution
- Fix optimistic update mapping assigneeAgentId → agentId in task store
- Disable reassignment for running tasks with tooltip hint
- Integrate selector into task list and task detail property panel

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: reuse BriefCard in task detail activities & fix raw-id navigation

Render brief-type activities as full BriefCard (same as homepage) instead of
plain tree rows. Decouple BriefCardActions from useBriefStore for actions
lookup so it can be reused across pages. Fix infinite loading when navigating
to task detail via raw DB id (task_xxx) by storing detail under both the
identifier and the raw id key in taskDetailMap.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add TopicCard component for task detail activities

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: allow re-running completed tasks with dedicated button

Completed tasks now show a "Re-run" button (with rotate icon) instead of
hiding the action. The backend already supported this — only the frontend
selector gate needed updating.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: add create task modal with markdown editor

Add a "+" button on the tasks list page that opens a Linear-style modal
for manually creating tasks. The modal features a title input, a markdown
editor (EditorCanvas), and a bottom toolbar with priority and assignee
selectors. Existing tag components (TaskStatusTag, TaskPriorityTag,
AssigneeAgentSelector) are extended with an `onChange` controlled mode
so they can be used in creation context where no task exists yet.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 🐛 fix: suppress spurious updateTask on Task Detail page load

EditorDataMode was missing the contentChangeLockRef pattern that
DocumentIdMode already uses, causing Lexical's registerUpdateListener
to treat programmatic content hydration as a user edit and fire
onContentChange → updateTask on every page visit.

- Add contentChangeLockRef + lockIdRef staleness guard
- Extract loadContentWithLock to deduplicate lock/load/unlock logic
- Pass contentChangeLockRef to InternalEditor
- Remove unreachable dead code in loadEditorContent

Closes LOBE-7362

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

*  feat: task detail comment CRUD and various UX improvements

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix: move canceled status group to the end of task list

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style: polish task detail layout, title, and run button

- Title switched to auto-sizing TextArea so long names wrap (like Linear)
- Reduce title font-size from 32px to 24px and tighten paddings
- Make "运行任务" button small-sized to match the denser header
- Add 120px bottom padding for end-of-content scroll breathing room
- Default EditorCanvas paddingBottom trimmed from 64 to 32

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style: refine task assignee, priority, and comment input

- Assignee block uses filled variant in dark mode for better contrast
- Urgent priority (level 1) renders in orange for quick scanning
- Comment input keeps SendButton slot reserved to prevent layout shift

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  feat: task detail — inline subtasks, automation mode, chronological activity

- Inline subtask creation under a task via CreateTaskInlineEntry
  (parentTaskId/autoFocus/onCollapse/placeholder), refreshes parent on create
- Track agent-created tasks via createdByAgentId through service, router,
  types, and the builtin task executor
- Replace scheduler Segmented-only UI with an Enable switch + heartbeat/
  schedule mode; persist via automationMode on the task
- Sort detail activities oldest → newest for a natural timeline reading
- Reducer patches nested subtask entries on updateTaskDetail so in-place
  edits reflect in the parent's subtask tree

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style: render activate-tool chips as rounded pills

Switch inspector tool chips from monospace code tags to filled rounded
pills with ellipsis overflow, making multi-tool rows scan better in tight
headers.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix: keep finished tool call out of loading state while siblings run

The message-level isAssistantMessageBusy flag stays true while sibling
tool calls are still running. Without guarding on this tool's own
result, a finished tool would flip back to "loading". Now a tool that
has a real result or error is never shown as calling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style: use small Segmented in schedule config popover

Keeps the automation mode switcher visually aligned with the denser
popover controls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  feat: agent profile hover card on task activity author

- Extract shared AgentProfileCard + unified AgentProfilePopup (click / hover)
  with lazy agent fetch; move out of group sidebar path.
- Wire activity author avatar + name to a hover card; brighten title on hover;
  keep a small "agent" tag on the author row.
- Show inline skeletons (description + footer stats) while loading.
- Enrich subtask payload with assignee agent info for cleaner UI.

*  feat: open task topic chat in side drawer

Click a topic row in the task detail activities to open a right-side drawer
showing the topic's full chat history. Messages stream in live via the existing
agent gateway pipeline (gateway events land in chatStore.dbMessagesMap keyed by
the topic context), so a running topic refreshes its drawer in real time without
a dedicated subscription.

Reuses the Conversation feature (ConversationProvider + ChatList) with an
isolated context (agentId + topicId + isolatedTopic), so the drawer never
touches the global active topic and multiple panels coexist cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style: outline activate-tool chip with subtle border

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  feat: show topic handoff summary on activity card

Pull `handoff.summary` through the task service into TaskDetailActivity and
render it under the title in TopicCard so completed topics surface what was
accomplished without opening the drawer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🎸 chore: gate agent task feature behind agent_task flag

Hide every client-side entry point to the Agent Task feature when the
`agent_task` flag (default `isDev`, off in prod) is disabled:

- Sidebar: task tab in the agent sidebar nav
- Routes: `/agent/:aid/tasks/*` and `/tasks/*` layouts redirect to `/` when
  the flag is off (mobile router reuses the same layout)
- Home Recents: filter out `type='task'` items in both the list and the
  "all recents" drawer
- Daily Brief: skip fetch + hide the entire panel (all briefs link to tasks)

Backend TRPC / lifecycle stays on — the feature is already live for CLI
usage. Flag name mirrors `agent_onboarding` for consistency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix: prioritize includeTriggers in topic queries

* 🐛 fix: normalize task detail activity payloads

*  feat: add Kanban board view for task list with drag-and-drop

LOBE-7493

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* 💄 style: shorten schedule tag labels & fix time width in task cards

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update i18n

* 💄 style: hide task tool from user selectors

* 💄 style: hide task skill from user selectors

---------

Co-authored-by: canisminor1990 <i@canisminor.cc>
Co-authored-by: YuTengjing <ytj2713151713@gmail.com>
Co-authored-by: Arvin Xu <arvinx@foxmail.com>
2026-04-23 02:10:45 +08:00
Innei 5196203414 ♻️ refactor: replace antd Modal with base-ui Modal in FileEditor (#14054)
♻️ refactor: replace antd Modal with imperative base-ui createModal

Replace the declarative antd Modal in AttachKnowledgeModal with imperative
createModal from @lobehub/ui/base-ui. The antd Modal's event handling
conflicted with the three-dot DropdownMenu in the file list, causing the
menu to be unclickable in Group Chat context.

Closes #12389
2026-04-23 01:36:11 +08:00
Innei 5c2fe6c579 🐛 fix(onboarding): unify footer visibility behind AGENT_ONBOARDING_ENABLED (#14065)
🐛 fix(onboarding): show mode switch and skip footer based solely on AGENT_ONBOARDING_ENABLED

Remove route-based conditional so the footer visibility is controlled
entirely by the AGENT_ONBOARDING_ENABLED flag.
2026-04-23 01:17:43 +08:00
Arvin Xu 042987fe34 🐛 fix(agent-runtime): unwrap underlying PG error in formatErrorEventData (#14038)
* 🐛 fix(agent-runtime): unwrap underlying PG error in formatErrorEventData

Drizzle wraps driver errors as "Failed query: insert into ..." and buries
the real PostgreSQL diagnostic fields (code, severity, detail, constraint,
column, table) in `.cause`. `formatErrorEventData` in RuntimeExecutors only
read the outer `.message`, so the agent-gateway dashboard saw nothing but
the SQL text — no way to bucket errors by SQLSTATE or tell apart a UTF-8
validation failure from a unique-constraint hit from a row-too-big.

Add a `pgError` util that walks `.cause` up to 5 layers, duck-types real
PG errors via `code` + a known `severity`, and exposes
`{ formatPgError, pgErrorType, unwrapPgError }`. `formatErrorEventData`
now invokes the unwrap as a last-step enrichment — only when no typed
errorType was identified — so typed errors like `ConversationParentMissing`
keep their clean business messages.

After this, the dashboard gets:
  error:     PG 22021 · ERROR · invalid byte sequence ... · table=message_plugins · column=state
  errorType: pg_22021
instead of:
  error:     Failed query: insert into "message_plugins" ...
  errorType: Error

Related: LOBE-7158, LOBE-7334

* 🐛 fix(agent-runtime): unwrap PG diagnostics for raw driver errors regardless of error.name

Review feedback on the prior commit: the enrichment branch only ran when
errorType was missing or exactly 'Error', so raw top-level driver errors —
`PostgresError` (postgres-js), `DatabaseError` (node-postgres), any
provider-specific subclass — kept their driver class name as errorType
and never reached the pg_<sqlstate> bucket. This defeated the new
classification for the exact case it was meant to catch: a PG error
surfacing directly from the driver without a Drizzle wrapper.

Fix: track whether `errorType` came from a business-typed field on the
error payload (step 1 — e.g. `ConversationParentMissing`) vs. from
`error.name` (step 3 — a driver class name). Only skip PG unwrap for
business-typed errors. Driver-named errors now fall through to unwrap
and emit `pg_<sqlstate>` when PG info is identifiable.

Also extract `formatErrorEventData` out of RuntimeExecutors.ts into its
own file so it can be unit-tested directly. The surrounding
RuntimeExecutors module pulls in workspace packages (`@lobechat/markdown-patch`,
`@lobechat/agent-gateway-client`, etc.) that don't resolve in the test
environment, blocking any test that imports from it.

Test coverage added (10 cases): top-level PostgresError class, plain
DatabaseError-shaped object, Drizzle .cause unwrap, ConversationParentMissing
preservation, custom errorType preservation, Node ENOTFOUND rejection,
null/non-object fallbacks, plain-string inputs, payload-with-only-message.
2026-04-23 00:46:01 +08:00
Innei f00d95f4a6 🐛 fix(desktop): add Linux icon configuration to electron-builder (#14042)
The Linux target was missing the icon field, causing the .deb package
to show no application icon on Ubuntu and other Linux distributions.

Closes #9785
2026-04-23 00:34:20 +08:00
Innei ed6330362c 🐛 fix(conversation): pin user message to viewport top & fold long user messages (#14056)
* 🐛 fix(conversation): pin user message to viewport top after spacer settles

Observing the spacer DOM via ResizeObserver lets us re-fire scrollToIndex
once virtua finishes measuring it and scrollSize actually expands, so the
sent user message lands flush against the viewport top instead of
trailing below by the spacer growth delta. Also drop the height
transition on mount/grow so scrollSize jumps in a single frame; only the
collapse-to-zero (unmount) still animates.

* 🐛 fix(vite): detach spawn for debug proxy so dev server isn't blocked

Swap execFile for a detached spawn with stdio ignored and unref, so the
opened browser process no longer keeps the Vite dev process alive. Falls
back to treating a 200ms "no error" window as success, and routes
diagnostics through the Vite logger instead of swallowing them.

*  feat(conversation): fold long user messages so AI response stays visible

When a very long user message is pinned to the viewport top after send,
it can eat the entire viewport and leave no room for the AI reply.
Wrap the user text body in a CollapsibleContent that clamps content
past min(280px, 35vh) with a gradient mask and a Show more / Show less
toggle. Attachments, images and page selections stay fully visible.

* ♻️ refactor(conversation): scope spacer observer to this list via ref callback

ConversationProvider supports multiple conversation lists mounted at the
same time, so a document-wide querySelector would attach to whichever
spacer the DOM hands out first — possibly another panel's — and drive
spacerLayoutVersion from unrelated layout ticks. Switch to a ref
callback returned from useConversationSpacer and bound to the spacer div
rendered by the same VirtualizedList, guaranteeing the observer tracks
this instance's own spacer.

* 🐛 fix(conversation): cancel queued pin retries when user scrolls up

Clearing pendingScrollIndexRef alone wasn't enough — the retry wave fires
at 0/32/96ms, so if the user scrolled up between send and 96ms the
already-queued timers would still call scrollToIndex and yank the
viewport back down, contradicting the "don't fight user intent" rule.
Also invoke clearPendingPins in the same effect so the in-flight retry
window is cancelled along with the pending index.
2026-04-22 23:59:43 +08:00
YuTengjing 17834d41c3 🐛 fix(route-log): record image/video generation triggers (#14048) 2026-04-22 23:48:59 +08:00
Innei 5e9546c537 🐛 fix(page-editor): use remoteServerUrl for copy link on desktop (#14057)
Fix LOBE-7356 — PageEditor handleCopyLink used window.location.origin which resolves to app://renderer on desktop. Now uses electronSyncSelectors.remoteServerUrl on desktop, consistent with existing pattern in global.ts and Topic dropdown.
2026-04-22 23:40:25 +08:00
Innei 25e4b3e33b 🐛 fix(build): enable Rolldown strictExecutionOrder for production builds (#14058)
Made-with: Cursor
2026-04-22 23:14:11 +08:00
Innei 82ba3706a7 feat(desktop): screen capture overlay, Quick Chat tray, and upload pipeline improvements (#13818)
* feat: add screen capture functionality with overlay support

- Implemented ScreenCaptureManager to handle screen capture sessions.
- Added ScreenCaptureCtr for IPC methods related to screen capture.
- Created overlay.html and ScreenCaptureOverlay component for user interaction.
- Integrated window enumeration and capture logic using node-screenshots and get-windows.
- Updated menu options to include screen capture actions.
- Enhanced RendererUrlManager to support overlay routing.
- Introduced drag selection for capturing specific screen areas.
- Added necessary types and events for screen capture in electron-client-ipc.

Signed-off-by: Innei <tukon479@gmail.com>

*  feat(desktop): refine screen capture overlay flow

*  feat(desktop): refine screen capture overlay flow

*  feat(desktop): optimize screen capture overlay flow

* Delete apps/desktop/mockup/screen-capture-overlay.html

*  feat(desktop): open mini toolbar via double Option

* 🐛 fix(desktop): separate quick composer hotkey

* 💄 fix(desktop): remove stale quick composer accelerator

* 🐛 fix(desktop): stabilize double option monitor

* 🐛 fix(desktop): read hardware option key state

* 🐛 fix(desktop): standardize path imports and improve error handling

- Replaced `join` imports with `path` imports for consistency across files.
- Enhanced error handling in various modules to include error causes for better debugging.
- Updated test files to reflect changes in variable naming and mock implementations.

Signed-off-by: Innei <tukon479@gmail.com>

* 🔥 chore(hotkey): drop orphan renderer quickComposer i18n entries

The `quickComposer` hotkey is registered only on the Electron side
(DESKTOP_GLOBAL_SHORTCUT_DEFAULTS + BrowserWindowsCtr.openQuickComposer);
the renderer never referenced these i18n keys, so the entries were dead.
`desktop.quickComposer` covers the app-level trigger.

* ️ perf(screen-capture): parallelize overlay upload with route navigation

Overlay submit used to await screenshot upload before router.push,
blocking the main window for several seconds when the user was on an
unrelated page (e.g. /settings). Now we navigate immediately and run
upload in a background IIFE; MessageFromUrl waits on a new
`uploadStatus` field before calling sendMessage, so the chat page
mount and the upload proceed in parallel.

- Add `uploadStatus: 'uploading' | 'ready' | 'failed'` to
  PendingOverlayDispatch; canConsumePendingOverlayDispatch blocks
  while `'uploading'`.
- Store gains `markDispatchUploadComplete`; on failure it clears
  screenshotFileNames so the prompt still delivers.
- Dispatcher drops stale prev search params on push to prevent
  MessageFromUrl's message-param effect from double-firing.

* ️ perf(screen-capture): pre-upload captures in overlay preview + per-thumbnail status

Move uploads from post-submit to preview time, bypassing dataUrl round-trips:

- Main process assigns captureId at preview time and ships the PNG bytes
  as ArrayBuffer to the main renderer via `overlayUploadRequest`.
- Main renderer uploads through a dedicated pool (uploadWithProgress,
  no chatUploadFileList pollution); reports status back to the overlay
  through `overlayCaptureUploadStatus`.
- Overlay thumbnails render a spinner / error badge based on status;
  the send button stays grey until every capture resolves to `ready`.
- Submit now carries only captureIds; MessageFromUrl awaits the pool
  promises before sendMessage, removing the second upload pass.
- Carry overlay-selected modelId/provider into the agent config so the
  first message actually uses the user-chosen model (fixes the bug where
  switching the model on the overlay had no effect).

* update

*  feat(popup): add Quick Chat tray entry backed by Inbox agent

Tray menu now exposes a "Quick Chat" action that opens (or focuses)
a single-instance popup window at `/popup/agent/inbox`. Each fresh
open starts with no active topic; the first message creates one
through the normal agent flow.

- New `PopupAgentQuickPage` resolves the inbox slug via
  `builtinAgentSelectors.inboxAgentId` so `activeAgentId` points at
  the real entity in `agentMap` (fixes the stuck-loading / skeleton
  state from using the literal `'inbox'` slug).
- `BrowserManager.openQuickChatPopup` wraps
  `createMultiInstanceWindow` with a fixed `topicPopup_quick_inbox`
  uniqueId so repeat clicks focus rather than spawn.
- Wire the action into macOS / Windows / Linux tray menus and add
  the `tray.quickChat` i18n key.

* Add quick chat shortcut and desktop hotkey support

*  feat(screen-capture): enhance window enumeration with scale factor support

- Updated `enumerateWindows` to accept an optional `displayScaleFactor` parameter for improved window geometry normalization on high-DPI displays.
- Refactored `normalizeWindowBounds` to handle scaling based on the provided scale factor, ensuring accurate window dimensions across different platforms.
- Adjusted tests in `WindowSourceService.test.ts` to validate the new scaling behavior for both Windows and macOS environments.
- Minor adjustments in `ScreenCaptureManager` to accommodate the updated window enumeration logic.

---------

Signed-off-by: Innei <tukon479@gmail.com>
2026-04-22 22:00:30 +08:00
Rdmclin2 993f3f29ea 🐛 fix: slack webhook error (#14052)
* chore: add slack error docs

* chore: universal merge config and default schema settings

* fix: setting save lost

* chore: remove legacy webhook
2026-04-22 21:19:14 +08:00
Arvin Xu 2a3667493f feat(git-status): one-click pull/push from branch chip (#14041)
*  feat(git-status): one-click pull/push from branch chip

Split the ahead/behind indicator out of the BranchSwitcher trigger so
↓N / ↑N become standalone action chips: clicking ↓ runs `git pull
--ff-only`, clicking ↑ runs `git push`. Each chip swaps to a spinning
LoaderIcon while the operation is in flight and refreshes branch /
working-tree / ahead-behind state on success.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ♻️ refactor(electron-ipc): extract Git IPC types into dedicated git.ts

Move GitBranchInfo / GitLinkedPullRequest(Result) / GitBranchListItem /
GitWorkingTree(Status|Files) / GitCheckoutResult / GitPullResult /
GitPushResult / GitAheadBehind out of system.ts into a sibling git.ts
so the system surface stays focused on system/window/theme types.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(git-status): push chip failing under push.default=simple

Use `git push -u origin HEAD` instead of bare `git push` so the one-click
push action works on branches whose upstream name differs from the local
name (the common `git checkout -b feat/x origin/canary` workflow). Bare
`git push` refuses in that case under the default simple policy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(git-status): push tooltip lying about target ref

Push chip was reusing the pull upstream in its tooltip, which is wrong
when local branch name differs from upstream (e.g. feat/x tracking
origin/canary) — the push actually goes to origin/<local-name> per
our `git push -u origin HEAD`, not to the upstream.

Compute a separate `pushTarget` (`origin/<current-branch>`) and
`pushTargetExists` flag in getGitAheadBehind, and switch the push
tooltip to use that. When the target doesn't exist yet (one-click
creates a new remote branch) show a "(new branch)" variant so the
user knows what the click will do.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style(git-status): ring spinner + clearer create-branch tooltip

- Swap the lucide LoaderIcon (with hand-rolled CSS spin) for the shared
  RingLoadingIcon used in Topic items, so the in-flight pull/push chip
  matches the rest of the app's spinner style.
- Reword the new-branch push tooltip from "push N commits to X (new
  branch)" to "Click to create branch X" — the count is misleading when
  the remote doesn't exist yet (the action is creating, not catching
  up), and the shorter copy reads cleaner.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Simplify comments in pushGitBranch method

Removed detailed comments about git push behavior.

* 🐛 fix(git-status): serialize pull/push on diverged branches

Block the opposite sync action while a git sync is running — both chips
go disabled whenever pulling or pushing is true. Previously on a
diverged branch (ahead > 0 and behind > 0) a user could start pull and
still click push before the first finished, launching concurrent git
operations against the same worktree and producing lock / non-FF errors
plus confusing double toasts for a single intent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  feat(git-status): piggyback best-effort fetch on ahead/behind lookup

Problem: ahead/behind was computed purely against locally-cached refs, so
commits pushed to origin elsewhere (GitHub web UI, another machine) never
surfaced as ↓N until the user ran `git fetch` in a terminal.

Fix: run `git fetch --no-tags --quiet origin` at the start of
getGitAheadBehind with a 10s timeout; ignore failures and fall through
to compute against whatever refs we have. SWR's revalidateOnFocus
already re-invokes this IPC, so the fetch happens on window re-focus for
free — no new UI and no interval polling.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 20:53:35 +08:00
Innei 9c5831ac54 🐛 fix(vite): exclude .html from code-inspector to fix Vite 8 bundledDev (#14053)
* 🐛 fix(vite): exclude .html from code-inspector to fix bundledDev

* 🔥 chore: remove @code-inspector/vite patch, fixed via exclude config
2026-04-22 20:43:24 +08:00
Innei 31d76ccb90 ⬆️ chore: upgrade Vite to 8.0.0 (#12720)
* ⬆️ chore(vite): migrate SPA build pipeline to Vite 8

* 🔧 chore(vite): patch inspector tooling and stabilize rolldown output

* 🐛 fix(vite): apply Vite 8 follow-up fixes and dev proxy polish

* 🩹 chore(vite): drop oversized code-inspector core patch

* 🐛 fix(desktop): support vite 8 electron build

* 🐛 fix(desktop): declare mac permissions types ambiently

* 🐛 fix(desktop): externalize mac permissions in main build
2026-04-22 19:59:38 +08:00
Innei 9a03c182da ♻️ refactor(desktop): increase recent working directories from 5 to 20 with scroll container (#14036)
* ♻️ refactor(desktop): increase recent working directories from 5 to 20 with scroll container

* 🎨 style(branch-switcher): compact dropdown, immersive search, aligned icons

- Stop keydown propagation on inputs to bypass Base UI typeahead navigation
- Switch search input to borderless variant with bottom divider
- Align search prefix icon with list item icons at 12px
- Tighten item padding, line-height and meta spacing
- Match create-branch item radius to popup via calc(borderRadius - 4px)
2026-04-22 17:14:06 +08:00
YuTengjing 9d41c8b71c 🐛 fix(mobile): correct session list skeleton row layout (#14040) 2026-04-22 17:04:51 +08:00
YuTengjing 16f2b97de2 feat: add gpt-image-2 to LobeHub-hosted card (#14039) 2026-04-22 16:57:31 +08:00
Arvin Xu 6d339d6a64 🐛 fix(agent-runtime): sanitize invalid tool_call arguments to unbreak strict providers (#14033)
* 🐛 fix(agent-runtime): sanitize invalid tool_call arguments to prevent history poisoning

When a model emits malformed JSON as tool_calls[].arguments (e.g. Qwen
producing `{, "description": ...}`), the raw string was persisted to
`messages.tools[].arguments` and replayed verbatim on every subsequent
turn. Strict providers (NVIDIA NIM) validate the full history and 400
the whole request, terminating the op and wasting all accumulated tokens.

Add a shared `sanitizeToolCallArguments` helper in @lobechat/utils and
wire it in at three layers so both new captures and already-poisoned DB
history are safe:

- Server entry (RuntimeExecutors onToolsCalling) — mirrors the frontend's
  `internal_transformToolCalls` pattern; prevents new poisoning.
- Outbound context build (ToolCallProcessor) — last line of defense for
  historical messages that were persisted before this fix.
- Agent-runtime core (call_tools_batch normalization) — covers the
  old-format ToolsCalling[] path.

Behavior: valid JSON passes through unchanged (prompt cache stable);
partial-json recovers truncated streams; unrecoverable payloads fall
back to "{}" so the tool_call structure survives and the model can
replan on the next turn.

Fixes LOBE-7761
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(agent-runtime): preserve INVALID_JSON_ARGUMENTS feedback when sanitizing

Sanitizing `tool_calls[].arguments` at capture (onToolsCalling) was too
early — the normalized "{}" reached `BuiltinToolsExecutor.execute` and
bypassed the `INVALID_JSON_ARGUMENTS` branch, so the model got a generic
"missing required field" error instead of the precise "your JSON syntax
was broken, fix it" feedback. That regressed the self-reflection signal.

Move sanitization to the persist boundaries only:
- DB write via `messageModel.update({tools: ...})`
- `state.messages` push for the assistant message's `tool_calls`

The execution path keeps the raw `arguments` string so the executor can
still emit its `INVALID_JSON_ARGUMENTS` tool-result with the original
malformed payload echoed back — exactly the frontend-symmetric self-
reflection flow.

Add a regression test pinning the LOBE-7761 Qwen shape so future changes
can't silently drop the feedback again.

Fixes LOBE-7761
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(agent-runtime): drop sanitize from runtime normalization to avoid undeclared @lobechat/utils dep

Review flagged that `runtime.ts` imported `sanitizeToolCallArguments` from
`@lobechat/utils` while `agent-runtime/package.json` doesn't list utils as
a runtime dependency — in strict/hermetic installs this resolves to
MODULE_NOT_FOUND before the runtime can start.

Rather than add a new dep just for a belt-and-suspenders path, drop the
sanitize on the old-format `call_tools_batch` normalization. The actual
LOBE-7761 bug is server-side history poisoning; that's fully covered by:

- RuntimeExecutors persist-boundary sanitize (DB write + state.messages)
- context-engine ToolCallProcessor outbound sanitize (handles any DB
  history that was persisted before this fix)

Old-format agents in agent-runtime don't persist or replay to providers
on their own — sanitization is the consuming application's
responsibility and can live closer to its persistence layer.

Drops the dep-cycle-free path.
Related LOBE-7761
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(model-runtime): log tool_call parse errors in Anthropic adapter

The assistant→Anthropic conversion was swallowing `JSON.parse` errors
silently and falling back to empty `input: {}`. Combined with the
LOBE-7761 fix, bad arguments should always be sanitized upstream in
context-engine, so hitting this catch means something bypassed the
defense and we're about to send a tool_use with empty input to Claude.
That's worth knowing about.

Match the `console.error('parse tool call arguments error:', ...)`
pattern already used in openaiCompatibleFactory so logs are greppable.

Related LOBE-7761
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 16:09:26 +08:00
LiJian 9e4bcf88c6 🐛 fix: add the inbox agentid Sync to resume the /agnet/inbox/message=xxx link (#14035)
* fix: add the inbox agentid Sync

* fix: should have the fallback

* fix: update the test
2026-04-22 15:20:08 +08:00
AmAzing- b8cd21a257 chore: add Twitter to recommended skills (#14037) 2026-04-22 15:08:38 +08:00
YuTengjing b4de72b032 feat(mobile): full settings menu and responsive profile layout (#14019) 2026-04-22 15:08:34 +08:00
Arvin Xu e963c640b9 🎨 style(claude-code): tool inspector polish + unstick Read-on-image spinner (#14034)
* 💄 style(claude-code): prefix Agent inspector with "Agent:" and drop chip 60% cap

Row visibly reads as a subagent dispatch, not a generic tool; chip no longer
ellipsizes when there is room to the right.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(heterogeneous-agents): unstick Read tool spinner on image results (LOBE-7338)

CC's `Read` on images returns a `tool_result` whose `content` is an `image`
block (base64). The generic array mapper had no branch for it so resultContent
collapsed to '' and the UI's StatusIndicator stuck on the spinner. Emit a
minimal `[Image: <media_type>]` placeholder so the tool ends in completed
state. Richer image echo (thumbnails) is tracked separately and needs
structured ToolResultData.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style(claude-code): place "Agent:" prefix before the icon

Order is now `Agent: <icon> <subagent_type>` instead of `<icon> Agent: <subagent_type>` so the contextual label leads, the bot icon sits between as a visual separator, and the subagent name closes the row.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 14:33:54 +08:00
Arvin Xu 1f61e965a6 🎨 style(claude-code): tool inspectors + heterogeneous-agent follow-ups (#14030)
*  feat(claude-code): render ScheduleWakeup / TaskOutput / TaskStop in inspector

CC emits three tool calls we were previously rendering as raw JSON:
`ScheduleWakeup` (self-paced /loop), `TaskOutput` (read from background
task), `TaskStop` (terminate background task). Add dedicated inspectors
and register them alongside the existing CC tool set.

`TaskStop` accepts both `task_id` and the legacy `shell_id` field name
since older CC builds still emit the latter.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(chat-topic): stop completed topics from leaking past the sidebar filter

Two sibling components in each chat-topic sidebar were both calling
`useFetchTopics`, but with different args: the outer `Topic` passed the
preference-driven `excludeStatuses: ['completed']` filter while the
inner `List` / `TopicListContent` called it bare. Since `excludeStatuses`
is part of the SWR key, both calls fired independent requests whose
`onData` handlers wrote back to the same `topicDataMap[containerKey]`
slot — whichever response landed last won, and when the un-filtered
sibling won, completed topics reappeared in the sidebar despite the
"Include completed" preference being off.

Introduce `useFetchChatTopics` as the single call site for chat-topic
fetching. It reads `topicIncludeCompleted` from preferences and pins
`excludeTriggers` to the always-excluded cron/eval set, so every
sibling mounts with identical args, collapses onto one SWR key, and
SWR dedupes them to a single request. Group sidebars now also exclude
cron/eval triggers for parity with the agent sidebar (groups don't
produce either trigger today, so this is a no-op in practice but
prevents divergence if the rules change).

Popup and mobile-modal call sites keep using the raw `useFetchTopics`
because they deliberately need the unfiltered set — the popup has to
resolve a specific (possibly completed) topic's title from the map.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  feat(chat-input): heterogeneous-agent placeholder for Claude Code sessions

When the active agent is backed by a heterogeneous provider (currently
only `claude-code`), swap the generic "Ask, create, or start a task"
placeholder for a task-specific variant that names the provider
(e.g. "Ask Claude Code to do a task"). @-mention assignment hint is
suppressed in that mode since heterogeneous agents don't yet route to
sibling agents.

* 🌐 chore(i18n): translate sendPlaceholderHeterogeneous (en-US, zh-CN)

Local preview translations for the new heterogeneous-agent chat input
placeholder; en-US mirrors the default, zh-CN carries the Chinese
copy. CI regenerates locale JSON on release so this commit only seeds
dev preview.

* ♻️ refactor(workflow-summary): unify suffix to show total tool kinds and calls

Both branches of getWorkflowSummaryText now share the same suffix structure:
list · 共 N 种工具 · 共 X 次调用 · N 次失败. summaryMoreTools changes from
remaining count ("+N more" / "等 N 种工具") to total count, and the inline
(failed) per-tool marker is dropped in favor of the global error suffix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ♻️ refactor(workflow-summary): hide redundant kinds/calls suffixes

Show "N tool kinds" only when the displayed list is truncated, and "X calls
total" only when at least one tool was called more than once. Otherwise the
aggregates duplicate information already visible in the per-tool list.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🎨 style(chat-input): drop hotkey suffix from heterogeneous placeholder

Heterogeneous-agent placeholder (e.g. "让 Claude Code 帮你完成任务…") no
longer trails the "press ⌘↵ to insert a line break" hotkey hint, which read
awkwardly attached to a short single-clause prompt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🎨 style(claude-code): align ScheduleWakeup/Task* inspectors with ToolSearch

Drop leading lucide icons, add `:` suffix so the label row reads like
ToolSearch, and promote ScheduleWakeup's `reason` into the chip with
`delaySeconds` trailing as secondary context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(heterogeneous-agents): retain subagent tool-call lookup across turn boundaries

`findRunByInnerToolCallId` consulted `run.state.persistedIds`, but that
set is wiped every time `ensureSubagentRun` advances `subagentMessageId`.
A `tool_result` delayed past the owning turn therefore failed the lookup
and skipped the thread-bucket `run.stream.update`, leaving the in-thread
tool bubble stuck on its loading spinner until the user re-opened the
Thread (main-topic `fetchAndReplaceMessages` doesn't rehydrate thread
buckets). Add a run-lifetime `lifetimeToolCallIds` set that only grows
and route the lookup through it; leave `state.persistedIds` as-is so
`persistToolBatch`'s turn-scoped dedupe is untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:23:37 +08:00
Arvin Xu 3b306a8aed 🐛 fix(agent-runtime): preserve Gemini 3 thoughtSignature in call_tools_batch normalization (#14032)
The `ToolsCalling` -> `ChatToolPayload` mapping in `runtime.ts` explicitly
enumerated 5 fields and dropped `thoughtSignature`, while the type itself
never declared the field. As a result, any Gemini 3.x tool call beyond
the first one in a conversation would 400 with a misleading
"function call turn must come after user/function response turn" error —
Google's validator maps a missing signature to that generic ordering message.

Fix LOBE-7759.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:14:28 +08:00
Arvin Xu 4af6fddd7a 🐛 fix(context-engine): downgrade image_url parts when target model lacks vision (#14029)
* 🐛 fix(context-engine): downgrade image_url parts when target model lacks vision

Historical messages persisted as multimodal parts (content is an array
with `image_url` entries, or assistant messages with `metadata.isMultimodal`)
bypassed the legacy `imageList` vision check and got forwarded verbatim to
the provider. DeepSeek rejects the `image_url` variant outright, so any
topic containing an image broke the moment the user switched to a
non-vision model.

Replace image parts with a textual placeholder so the conversation still
carries the signal that an image was sent, without including content
non-vision providers reject. Applies uniformly across user array content,
assistant multimodal content, and legacy `imageList` paths.

Fixes LOBE-7214.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

*  test: update vision-disabled expectations after downgrade placeholder

Two tests in the app suite asserted the silent-drop behavior the
MessageContentProcessor used to exhibit for `imageList` + vision-off:

- src/services/chat/chat.test.ts
- src/services/chat/mecha/contextEngineering.test.ts

After this PR the processor appends the downgrade placeholder instead of
silently dropping the image, so the expected content grows by one line.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 💄 style(context-engine): place vision downgrade placeholder before SYSTEM CONTEXT

The placeholder stands in for an image the user actually sent, so it
should sit adjacent to the user text rather than trailing after the
SYSTEM CONTEXT metadata block. Reorder so the payload reads:

  <user text>

  [image omitted: not supported by this model]

  <!-- SYSTEM CONTEXT ... -->

Keeps the conversational flow intact and matches the semantic position
the image occupied in the original message.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 13:07:42 +08:00
YuTengjing e9600407ff 🐛 fix: reduce subagent task status error noise (#14026) 2026-04-22 12:58:30 +08:00
Arvin Xu f3fca500e4 🐛 fix(heterogeneous-agents): stream subagent Thread + fix parallel-tool orphan (#14024)
*  feat(heterogeneous-agents): stream subagent Thread + fix parallel-tool orphan

When a main-agent step emits a parallel tool_use (e.g. `[Grep, Agent]`),
the gateway handler's stream_chunk branch was forwarding the subagent's
inner `tools_calling` chunks onto `currentAssistantMessageId` (main),
overwriting main.tools[] with subagent tools — main's own Task/Agent
tool_use then had no matching entry and every tool message under it
rendered with the "orphan tool call" banner.

Two coordinated changes:

1. Main-bucket isolation: the executor now drops subagent-tagged
   `stream_chunk` events before forwarding to the gateway handler. DB
   persistence continues via `persistSubagent*Chunk` so the subagent
   content is never lost; only the main-handler in-memory dispatch is
   suppressed for subagent chunks.

2. Thread-bucket streaming: `internal_dispatchMessage` now accepts a
   `threadId` override that snaps scope to `thread`, routing
   create/update payloads to the thread's `messagesMap` bucket. Each
   `SubagentRunState` carries a thread-scoped dispatcher; ensureSubagentRun
   seeds user + assistant on lazy Thread creation and at turn boundaries,
   persistToolBatch gets an `onToolCreated` hook that the subagent path
   uses to seed role:'tool' rows, persistSubagent*Chunk dispatches
   tools[] / content / reasoning updates on every chunk, and the
   tool_result branch mirrors subagent tool_result content (+ pluginState)
   into the thread bucket. Thread view now streams token-by-token with
   the same cadence as the main bubble.

Tests:
- `does NOT forward subagent-tagged stream_chunks to the gateway handler`
  — asserts main bucket isolation under parallel main+subagent tool use.
- `streams subagent create/update dispatches into the thread messagesMap
  bucket` — asserts user/assistant/tool createMessage dispatches land in
  the thread scope, plus streaming updateMessage for tools[], content,
  and tool_result, with no bleed into the main bucket.

Local repro verified end-to-end: main assistant.tools=[Grep, Agent]
stays intact across two parallel runs, thread bucket populates 14 rows
(user + 2 subagent assistants with Bash/Glob then Read×8 + 10 tool
results) during the run, `mainOrphans`/`threadOrphans`/
`threadIntoMainBleed` all empty, orphan warning DOM count = 0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ♻️ refactor(heterogeneous-agents): route subagent stream through a per-spawn sub-operation

Replace the threadId-override on `internal_dispatchMessage` with a
proper per-spawn child operation, eliminating the second context
expression at the dispatch boundary.

The previous design accepted `{ operationId, threadId? }` and snapped
scope to `'thread'` when the override was present. That was a leaky
parallel path to the operation registry — the same "which messagesMap
bucket should this dispatch hit?" question got answered two different
ways. `startOperation` already supports `parentOperationId` + context
inheritance + recursive cancel cascade, so the right move is to model
the subagent run as a first-class child op and let
`internal_getConversationContext` do its normal job.

Changes:
- Add `'subagentThread'` to `OperationType` (NOT in
  `AI_RUNTIME_OPERATION_TYPES` — it's a context container, not an
  independent loading state, so it shouldn't double-count for spinners).
- `executeHeterogeneousAgent` opens the sub-op in `beginSubagentRun`
  via `startOperation({ type: 'subagentThread', parentOperationId,
  context: { ...context, threadId, scope: 'thread' } })` and binds a
  thread-scoped dispatcher to that sub-op's id.
- `SubagentRunState.subOperationId` carries the id so `finalizeSubagentRun`
  can mark it completed when the spawn's tool_result arrives (or on the
  `onComplete` fallback for crash/abort paths). Cancel cascade + cleanup
  flow through the existing parent/child op linkage.
- Revert the `threadId` override in `internal_dispatchMessage` — the
  store boundary is back to a single context expression
  (`{ operationId? }`).

Test:
- Add `startOperation` mock to `createMockStore` (returns monotonic
  `sub-op-N` ids).
- Update the streaming regression to identify the sub-op via the
  `startOperation` call with `type: 'subagentThread'`, assert the
  sub-op's parent + context shape, filter Thread bucket dispatches by
  `ctx.operationId === subOperationId`, and verify
  `completeOperation(subOperationId)` fires when the run finalizes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* 🐛 fix(heterogeneous-agents): drain subagent buffers only after DB flush confirms

`finalizeSubagentRun`'s buffer reset used to run unconditionally after
the flush try/catch, so a transient `messageService.updateMessage`
failure silently wiped the accumulated streamed text/reasoning — the
later `onComplete` fallback then had nothing left to retry, leaving the
subagent's streamed content absent from persisted thread history.

Move the clear into the success branch. A second concern surfaces once
the clear moves: after the flush block, the `resultContent` branch
advances `currentAssistantMsgId` to the newly created terminal
assistant, so a naive retry that reads `currentAssistantMsgId` would
overwrite the authoritative terminal content with the leftover streamed
buffer — corrupting the subagent summary with stale partial text.

Pin the flush target via a new `SubagentRunState.pendingFlushTarget`:
captured before the DB attempt, carried on the run when the flush
fails, cleared alongside the buffers on success. The retry uses the
pinned target instead of the live `currentAssistantMsgId`, so leftover
streamed buffers always land on the streaming turn's assistant — never
on the terminal row.

Test: `retains subagent buffers + pinned target when the finalize flush
fails` stubs `updateMessage` to throw once for the subagent streaming
write, runs streamed text → spawn `tool_result` → `onComplete`, and
asserts (1) the leftover content eventually reaches DB across ≥2
write attempts and (2) every attempt targets the streaming turn's
assistant — not the terminal row created by `resultContent`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 22:18:30 +08:00
AmAzing- 6ddef95249 chore: fix follow-up chat input state during message queueing (#14020)
* 💄 style(chat-input): improve agent assignment placeholder

*  improve follow-up queue input ux

* 💄 sync runtime placeholder locale keys

* Update SKILL.md

* 💄 style(chat-input): hide send menu while generating

Co-Authored-By: Oz <oz-agent@warp.dev>

* fix: ensure sendMenu is correctly cleared in store when prop becomes undefined and add test coverage

---------

Co-authored-by: Oz <oz-agent@warp.dev>
2026-04-21 18:56:52 +08:00
691 changed files with 34153 additions and 3695 deletions
+1 -1
View File
@@ -23,7 +23,7 @@ LobeChat agents can answer inside external chat platforms. Inbound messages flow
`supportsMarkdown=false` ⇒ outbound markdown is stripped to plain text via `stripMarkdown` and the AI is told not to use markdown. `supportsMessageEdit=false` ⇒ no progress edits — only the final reply is sent.
**Multi-mode connection** — Slack/Feishu/Lark/QQ shipped as websocket but support `webhook` per-provider via `settings.connectionMode`. Legacy rows without that field stay on `webhook` (see `LEGACY_WEBHOOK_PLATFORMS` in `platforms/utils.ts`) — **never add new platforms to that list**.
**Multi-mode connection** — Slack/Feishu/Lark/QQ ship as websocket but support `webhook` per-provider via `settings.connectionMode`. The runtime always merges schema defaults into stored settings before resolving the mode (`resolveBotProviderConfig` / `resolveConnectionMode` in `platforms/utils.ts`), so the schema's `field.default` is the source of truth — set it correctly when adding a new multi-mode platform.
## Inbound Flow (one webhook → reply)
@@ -0,0 +1,83 @@
---
name: heterogeneous-agent
description: Guide for implementing and debugging LobeHub heterogeneous agent integrations such as Claude Code, Codex, and future external CLI agents. Use when working on adapter event mapping, Electron IPC transport, renderer persistence, tool-call chaining, subagent threads, resume/session handling, or regressions like mixed multi-tool messages, broken step boundaries, stuck tool loading, and orphan tool messages. Triggers on 'heterogeneous agent', 'hetero agent', '异构 agent', 'claude code adapter', 'codex adapter', 'external agent CLI', '孤立 tool 消息', 'raw Codex trace', or adapter/executor bugs.
---
# Heterogeneous Agent Development
Use this skill when the bug or feature lives in the external CLI agent pipeline, not the normal server-side agent runtime.
## Use This Skill For
- Adding or changing a driver under `apps/desktop/src/main/modules/heterogeneousAgent/drivers/`
- Editing an adapter under `packages/heterogeneous-agents/src/adapters/`
- Debugging `heteroAgentRawLine` transport, `window.__HETERO_AGENT_TRACE`, or `executeHeterogeneousAgent`
- Fixing Claude Code stream-json bugs such as duplicate partial/full chunks, broken `message.id` boundaries, missing `tool_result`, TodoWrite state drift, or subagent thread routing
- Fixing Codex JSONL bugs such as mixed multi-tool messages, broken turn boundaries, or missing tool-result mapping
- Fixing step-boundary, tool persistence, subagent thread, or resume bugs in Claude Code / Codex flows
- Reproducing multi-tool mixing, orphan tool messages, or stuck tool-result loading
## Pipeline Map
1. CLI raw stdout / JSONL
2. Electron main spawns the CLI and broadcasts `heteroAgentRawLine`
3. Adapter maps raw provider events into `HeterogeneousAgentEvent`
4. `executeHeterogeneousAgent` persists assistant/tool messages and forwards stream events
5. `createGatewayEventHandler` hydrates the UI
6. Only after this path looks correct should you move on to `agent-tracing` or context-engine debugging
## Read These Files First
- `apps/desktop/src/main/controllers/HeterogeneousAgentCtr.ts`
- `apps/desktop/src/main/modules/heterogeneousAgent/drivers/claudeCode.ts`
- `apps/desktop/src/main/modules/heterogeneousAgent/drivers/codex.ts`
- `packages/heterogeneous-agents/src/adapters/claudeCode.ts`
- `packages/heterogeneous-agents/src/adapters/codex.ts`
- `src/store/chat/slices/aiChat/actions/heterogeneousAgentExecutor.ts`
- `src/store/chat/slices/aiChat/actions/__tests__/heterogeneousAgentExecutor.test.ts`
## Default Debug Order
1. Prove whether the raw CLI output is correct before touching UI code.
2. If raw output is correct, compare it with adapter output. In dev, `executeHeterogeneousAgent` exposes `window.__HETERO_AGENT_TRACE`.
3. If adapted events look correct, inspect `persistToolBatch`, `persistToolResult`, step transitions, and subagent routing.
4. Turn the repro into a focused test before fixing.
5. Only after the transport/adapter/executor path looks sound should you debug later-stage message processing.
## Critical Invariants
- One raw tool item must map to one stable `ToolCallPayload.id`.
- A new main-agent step must emit a boundary signal before events are forwarded to the new assistant.
- In Claude Code, multiple assistant events with the same `message.id` are one turn, not multiple turns.
- In Claude Code, `tool_result` lives in `type: 'user'` events, not assistant events.
- In Claude Code partial mode, `message_delta.usage` is authoritative; do not trust echoed usage on every assistant block.
- `persistToolBatch` must pre-register assistant `tools[]` before creating tool messages.
- Every tool message must keep `parentId` equal to the owning assistant and `tool_call_id` equal to the tool id.
- `tool_result` must resolve an existing `toolMsgIdByCallId`.
- Subagent chunks must stay in thread scope and must not be forwarded into the main assistant stream.
- Never clear the global `toolMsgIdByCallId` map at main step boundaries.
## Common Bug Patterns
- Claude Code duplicates text or thinking:
check whether partial deltas and the later full assistant block are both being emitted.
- Claude Code opens too many assistant messages:
check whether the adapter is cutting steps on every assistant event instead of only on `message.id` changes.
- Claude Code tool results never land:
check whether `type: 'user'` `tool_result` blocks are being ignored because the code only inspects assistant events.
- Claude Code TodoWrite cards look stale:
check whether synthesized `pluginState.todos` is being attached at tool-result time.
- Claude Code subagent transcript leaks into the main bubble:
check `parent_tool_use_id` handling and whether subagent chunks are being forwarded to the main gateway handler.
- Multiple Codex tools collapse into one assistant message:
first check whether the adapter emits a usable step boundary such as `newStep` or an equivalent turn-change signal.
- Orphan tool messages:
first check step-transition ordering and whether `persistToolBatch` Phase 1 ran before tool message creation.
- Tool bubble stays loading:
look for `tool_result for unknown toolCallId` and missing `result_msg_id` backfill.
- Subagent tools show up in the main bubble:
check for subagent chunks reaching the main gateway handler.
## References
- For commands, trace capture, invariants, and focused test commands, read [references/debug-workflow.md](./references/debug-workflow.md).
@@ -0,0 +1,246 @@
# Heterogeneous Agent Debug Workflow
## Contents
1. Pipeline map
2. Capture raw CLI traces first
3. Compare raw and adapted events
4. Check step boundaries before persistence
5. Check tool persistence invariants
6. Focused tests
7. Repro-to-fix workflow
## 1. Pipeline Map
```
CLI raw stdout
-> HeterogeneousAgentCtr (Electron main)
-> heteroAgentRawLine broadcast
-> createAdapter(...)
-> executeHeterogeneousAgent(...)
-> persistToolBatch / persistToolResult
-> createGatewayEventHandler(...)
-> UI hydration
```
Start at the leftmost broken layer. Do not jump straight to UI rendering unless raw and adapted events already look correct.
## 2. Capture Raw CLI Traces First
### Codex raw JSONL
Use a read-only prompt and save traces under the repo-local scratch directory `.heerogeneous-tracing/`.
```bash
ts=$(date +%Y%m%d-%H%M%S)
out=".heerogeneous-tracing/codex-${ts}.jsonl"
last=".heerogeneous-tracing/codex-${ts}.last.txt"
cat << 'EOF' | codex exec --json --skip-git-repo-check --sandbox read-only -C "$PWD" -o "$last" - > "$out"
You are being run only to collect a raw Codex JSON event trace.
Do not modify any files.
Use at least 4 separate shell tool invocations, one invocation per command.
Run a short sequence of read-only repo checks and then reply with a one-sentence summary.
EOF
```
What to look for in the JSONL:
- `thread.started`
- `turn.started`
- `item.started` / `item.completed`
- `item.type === 'command_execution'`
- `item.type === 'agent_message'`
- `turn.completed`
If raw Codex already merges tools into one item, the adapter is innocent. If raw Codex emits independent items but UI collapses them, the bug is downstream.
If the repo already contains useful traces under `.heerogeneous-tracing/`, inspect them before reproducing.
### Claude Code raw NDJSON
Mirror the arguments from `apps/desktop/src/main/modules/heterogeneousAgent/drivers/claudeCode.ts`.
- `-p`
- `--input-format stream-json`
- `--output-format stream-json`
- `--verbose`
- `--include-partial-messages`
- `--permission-mode bypassPermissions`
You can capture a local raw trace like this:
```bash
ts=$(date +%Y%m%d-%H%M%S)
out=".heerogeneous-tracing/claude-${ts}.ndjson"
cat << 'EOF' | claude -p \
--input-format stream-json \
--output-format stream-json \
--verbose \
--include-partial-messages \
--permission-mode bypassPermissions \
> "$out"
{"type":"user","message":{"role":"user","content":[{"type":"text","text":"Do a few read-only repo checks, use several tool calls, and then summarize briefly."}]}}
EOF
```
What to look for in Claude Code raw traces:
- `type: 'system', subtype: 'init'`
- `type: 'assistant'` blocks for `thinking`, `tool_use`, and `text`
- `type: 'user'` blocks containing `tool_result`
- `type: 'stream_event'` with `message_start`, `content_block_delta`, and `message_delta`
- `type: 'result'`
- `type: 'rate_limit_event'`
Important Claude Code semantics:
- Each content block often arrives as its own assistant event.
- Multiple assistant events can share the same `message.id`; that is still one turn.
- `message.id` change is the main-step boundary.
- Partial deltas arrive before the later full assistant block.
- `message_delta.usage` is the authoritative per-turn usage.
- Subagent events are tagged with `parent_tool_use_id`.
If the repo already contains useful references, inspect these first:
- `.heerogeneous-tracing/cc-monitor-real-trace.jsonl`
- `.heerogeneous-tracing/cc-stream-chain-reference.md`
If you only need boundary semantics or tool persistence behavior, prefer existing adapter tests under:
- `packages/heterogeneous-agents/src/adapters/claudeCode.test.ts`
- `packages/heterogeneous-agents/src/adapters/claudeCode.e2e.test.ts`
## 3. Compare Raw And Adapted Events
In dev builds, `executeHeterogeneousAgent` stores raw lines plus adapted events on:
- `window.__HETERO_AGENT_TRACE`
Use that trace to compare:
- raw `item.started` / `item.completed`
- adapted `stream_chunk { chunkType: 'tools_calling' }`
- adapted `tool_result`
- adapted `tool_end`
For Codex, the usual mapping is:
- raw `item.started(command_execution)` -> `tools_calling` + `tool_start`
- raw `item.completed(command_execution)` -> `tool_result` + `tool_end`
- raw `item.completed(agent_message)` -> `stream_chunk(text)`
If the raw trace is right but adapted events are wrong, fix the adapter before touching persistence.
## 4. Check Step Boundaries Before Persistence
This is the first thing to verify for "mixed tools in one assistant" bugs.
### Claude Code
Claude Code step boundaries are keyed off assistant `message.id` changes. The adapter should emit:
- `stream_end`
- `stream_start { newStep: true }`
Also verify these Claude-specific invariants:
- the first assistant after init does not open a new step
- repeated assistant events with the same `message.id` do not open a new step
- partial `content_block_delta` text/thinking does not get duplicated by the later full assistant event
- `tool_result` from `type: 'user'` updates the matching tool row
- `parent_tool_use_id` creates thread-scoped subagent chunks instead of main-stream chunks
- TodoWrite `tool_use.input` is converted into synthesized `pluginState.todos` on `tool_result`
Good references:
- `packages/heterogeneous-agents/src/adapters/claudeCode.ts`
- `packages/heterogeneous-agents/src/adapters/claudeCode.test.ts`
### Codex
Codex raw traces usually provide turn-level boundaries through:
- `turn.started`
- `turn.completed`
The executor only cuts a new assistant message when it receives a step-boundary signal it understands. If the adapter emits `stream_start` without `newStep`, multiple Codex tools and text chunks can accumulate under the same assistant longer than intended.
Relevant files:
- `packages/heterogeneous-agents/src/adapters/codex.ts`
- `src/store/chat/slices/aiChat/actions/heterogeneousAgentExecutor.ts`
## 5. Check Tool Persistence Invariants
Read `persistToolBatch` and `persistToolResult` before changing UI code.
### `persistToolBatch`
The expected order is:
1. Pre-register assistant `tools[]`
2. Create `role: 'tool'` messages
3. Backfill `result_msg_id` onto assistant `tools[]`
If tool rows are created before assistant `tools[]` are registered, orphan tool messages are likely.
### `persistToolResult`
`tool_result` must resolve the tool row through `toolMsgIdByCallId`.
Warning signs:
- `tool_result for unknown toolCallId`
- tool rows with empty content forever
- missing `result_msg_id`
For Claude Code, remember that tool results originate from raw `type: 'user'` events.
### Main vs subagent scope
- Main-agent tool state is per-step.
- `toolMsgIdByCallId` is global across main and subagent scopes.
- Subagent chunks must not be forwarded into the main gateway handler.
If subagent events leak to the main handler, the main bubble can inherit the wrong `tools[]` and content.
## 6. Focused Tests
Run the smallest useful test set first.
```bash
bunx vitest run --silent='passed-only' 'packages/heterogeneous-agents/src/adapters/codex.test.ts'
bunx vitest run --silent='passed-only' 'packages/heterogeneous-agents/src/adapters/claudeCode.test.ts'
bunx vitest run --silent='passed-only' 'src/store/chat/slices/aiChat/actions/__tests__/heterogeneousAgentExecutor.test.ts'
```
Especially useful places:
- `packages/heterogeneous-agents/src/adapters/codex.test.ts`
- `packages/heterogeneous-agents/src/adapters/claudeCode.test.ts`
- `src/store/chat/slices/aiChat/actions/__tests__/heterogeneousAgentExecutor.test.ts`
Claude Code-specific assertions worth adding when fixing bugs:
- same `message.id` does not emit `newStep`
- changed `message.id` does emit `stream_end` plus `stream_start { newStep: true }`
- partial text/thinking is emitted once
- `tool_result` from `user` events reaches the right tool row
- subagent chunks carry `subagent.parentToolCallId`
- TodoWrite result synthesizes `pluginState.todos`
When the bug comes from a real trace, distill it into the closest existing test file instead of relying on manual UI-only repros.
## 7. Repro-To-Fix Workflow
1. Capture a raw trace and save it under `.heerogeneous-tracing/`.
2. Confirm whether the bug appears in raw events, adapted events, or persistence.
3. Add or update the narrowest failing test near the broken layer.
4. Fix the smallest layer that can explain the symptom.
5. Re-run focused tests.
6. Only then do an Electron smoke test with the `local-testing` skill if UI confirmation is still needed.
Do not start with a broad Electron repro if a raw trace or adapter test can prove the fault zone faster.
+1 -1
View File
@@ -5,7 +5,7 @@ description: Internationalization guide using react-i18next. Use when adding tra
# LobeHub Internationalization Guide
- Default language: Chinese (zh-CN)
- Default language: English (en-US)
- Framework: react-i18next
- **Only edit files in `src/locales/default/`** - Never edit JSON files in `locales/`
- Run `pnpm i18n` to generate translations (or manually translate zh-CN/en-US for dev preview)
+57
View File
@@ -30,6 +30,63 @@ This is NON-NEGOTIABLE. Skipping Linear comments is a workflow violation.
When creating issues with `mcp__linear-server__create_issue`, **MUST add the `claude code` label**.
## Creating Sub-issue Trees
When breaking a parent issue into a tree of sub-issues (e.g., task decomposition for LOBE-xxx), follow these rules — they work around real limitations of the Linear MCP tools.
### 1. ALWAYS prefix titles with an ordering index
The Linear Sub-issues panel displays children by `sortOrder`, which **defaults to newest-first** (most recently created appears on top). Neither parallel nor serial creation will produce the intended top-to-bottom reading order, and the MCP `save_issue` tool does **not expose a `sortOrder` parameter** — you cannot set order at create time.
**Workaround**: encode execution order in the title itself:
```plaintext
[1] [db] add schema fields
[2] [db] new table + repository
[3] [service] business logic layer
[4] [api] REST endpoints
[4.1] [sdk] client SDK wrapper
[4.1.1] [app] consumer integration
[4.1.2] [app] UI surface
[4.2] [ui] dashboard page
```
Even when the panel shuffles, the reader can mentally reconstruct the dependency graph at a glance. Dotted numbering `[n.m.k]` should mirror the parent-child nesting so the index and the tree agree.
### 2. Nest sub-issues by logical parent-child, not flat under the root
Linear supports **unlimited sub-issue depth**. A flat list of 8+ siblings under one root is hard to scan. Group by main-subordinate logic:
- Core service → its SDK → SDK consumers
- Don't create a sibling when a child is more accurate
Use `parentId: "LOBE-xxxx"` at creation (or `save_issue` to move). Moving an issue's parent does not disturb its `blockedBy` relations.
### 3. Sub-issue creation order is dictated by `blockedBy`
`blockedBy` requires the blocker to exist first (you need its LOBE-id). So:
1. **Topologically sort** the DAG — leaves (no deps) first, roots last
2. Create issues with zero deps in the first wave
3. Create dependent issues only after collecting the blocker IDs from prior responses
4. `blockedBy` is **append-only**; passing it again does not overwrite — safe to re-run
### 4. Don't waste rounds trying to parallelize
MCP tool calls in a single message look parallel but execute sequentially on the server, and you still need blocker IDs from earlier responses. Just issue calls in dependency order; optimizing for parallelism gains nothing here.
### 5. Keep each sub-issue description self-contained
Each sub-issue should state:
- Goal (12 lines)
- Key files to touch
- Concrete changes / acceptance criteria
- Dependencies (link to blocker issues by `LOBE-xxxx`)
- Validation steps
The implementer may open only the sub-issue, not the parent — don't rely on context that lives only in the parent description.
## Completion Comment Format
Every completed issue MUST have a comment summarizing work done:
+1 -1
View File
@@ -56,7 +56,7 @@ export function registerTaskCommand(program: Command) {
const client = await getTrpcClient();
const input: Record<string, any> = {};
if (options.status) input.status = options.status;
if (options.status) input.statuses = [options.status];
if (options.root) input.parentTaskId = null;
if (options.parent) input.parentTaskId = options.parent;
if (options.agent) input.assigneeAgentId = options.agent;
+4
View File
@@ -1,3 +1,7 @@
## 专题文档
- [桌面端全屏 Overlay 截图方案设计与集成说明](./WindowOverlayCapture.md)
## 核心框架组件目录架构
### 主进程核心组件
+502
View File
@@ -0,0 +1,502 @@
# 桌面端全屏 Overlay 截图方案设计与集成说明
| 字段 | 内容 |
| ------------ | ----------------------------------------------------- |
| 状态 | 已完成技术预研与 demo 验证 |
| 最后更新 | 2026-04-14 |
| 适用范围 | Electron 桌面端全屏遮罩、窗口高亮、点击截窗、区域截图 |
| 当前验证载体 | `tmp/electron-window-overlay-demo` |
| 目标读者 | 后续将该能力接入 LobeHub Desktop 主业务的开发者 |
## 1. 文档目标
本文档用于沉淀以下内容:
| 目标 | 说明 |
| -------------------- | ------------------------------------------------------------- |
| 记录方案演进 | 保存从纯 Electron、native、自研、开源库到最终 demo 的决策过程 |
| 固化关键技术结论 | 明确哪些能力 Electron 可做,哪些能力必须借助额外库 |
| 提供业务接入蓝图 | 指出应修改的真实仓库文件、模块边界、IPC 设计与 UI 接入点 |
| 降低后续重复调研成本 | 使后续实现可以直接沿用本文档,不必重新验证底层假设 |
## 2. 需求回顾
| 需求项 | 结论 |
| ----------------------------------- | --------------------------------------------------- |
| 新增一个 “全屏” 入口 | 需要,但本质上是一个覆盖整块屏幕的透明 overlay 窗口 |
| 覆盖用户整个 screen | 需要,且在 macOS 上要覆盖菜单栏与 Dock 所在区域 |
| 获取系统窗口几何信息 | 需要,至少需要 `appName + bounds + windowId` |
| 在 overlay 上高亮窗口边框并显示 Tag | 需要 |
| 点击高亮窗口即截图该窗口 | 需要 |
| 拖拽任意区域截图 | 需要 |
| 输出先写入剪贴板 | 需要,作为 MVP |
| 避免自研 native addon | 明确要求避免 |
| 跨平台预留 | 需要,至少不能被 macOS-only 自研方案锁死 |
## 3. 关键术语澄清
### 3.1 “压住 macOS 菜单栏与 Dock” 的准确含义
这里的含义不是 “调用系统 fullscreen API”,而是:
| 项目 | 含义 |
| -------- | ------------------------------------------------------------ |
| 覆盖范围 | 窗口尺寸必须基于 `display.bounds`,而不是 `display.workArea` |
| Z 轴层级 | 窗口需要位于普通应用窗口之上,并且进入菜单栏所在区域 |
| 视觉效果 | 用户看到的是整块屏幕都被半透明遮罩覆盖 |
必须区分以下两件事:
| 易混概念 | 实际含义 |
| ----------------------------------- | ---------------------------------------------------- |
| `app.dock.hide()` | 仅隐藏应用在 Dock 中的图标,不会隐藏系统 Dock 栏本身 |
| `BrowserWindow.setFullScreen(true)` | 更接近原生全屏行为,未必适合作为截图 overlay |
## 4. 预研结论总览
### 4.1 方案对比
| 方案 | 能否覆盖菜单栏 / Dock | 能否拿到系统窗口 bounds | 能否按窗口截图 | 跨平台性 | 结论 |
| ------------------------------------- | --------------------: | ----------------------: | -----------------: | -------: | -------------------------- |
| 纯 Electron `desktopCapturer` | 是 | 否 | 部分可做,但不精确 | 高 | 不足以满足需求 |
| 自研 native addon | 是 | 是 | 是 | 中 | 能做,但被明确拒绝 |
| 参考 Claude.app 的 native quick entry | 是 | 是 | 是 | 低到中 | 可借鉴思路,不适合直接照搬 |
| `node-screenshots` 单库 | 是 | 是 | 是 | 中到高 | 核心方案成立 |
| `node-screenshots + get-windows` | 是 | 是 | 是 | 中到高 | 当前最终方案 |
### 4.2 最终选型
| 能力 | 最终实现 |
| --------------------- | -------------------------- |
| 全屏 overlay 窗口 | Electron `BrowserWindow` |
| 系统窗口枚举 | `node-screenshots` |
| 指定窗口截图 | `node-screenshots` |
| 隐藏 / 伪关闭窗口过滤 | `get-windows` 作为白名单 |
| 区域截图 | Electron `desktopCapturer` |
| 输出介质 | `clipboard.writeImage()` |
## 5. 对 Claude.app 的观察结论
本轮曾直接检查过本机解包后的 Claude.app 产物,结论如下:
| 观察对象 | 结论 |
| ------------------ | ------------------------------------------------------------------------------------------------ |
| `quick_window` | 不是全屏 overlay;它是小尺寸 `panel` 弹窗 |
| `nativeQuickEntry` | Claude.app 存在原生 quick entry 能力,说明其真实覆盖式入口并不完全依赖纯 Electron |
| `cu-glow` | 这是最接近本需求的 Electron overlay 实现:使用 `display.bounds`、透明窗、`screen-saver` 置顶层级 |
据此可以得出两个重要判断:
| 判断 | 含义 |
| -------------------------------------------- | ---- |
| Electron 可以做 “整屏遮罩” | 成立 |
| Claude 的 “整屏入口” 并不等于 `quick_window` | 成立 |
## 6. 当前 demo 的最终方案
### 6.1 架构图
```text
┌──────────────────────────────┐
│ Tray / Menu / Future Action │
└──────────────┬───────────────┘
│ startOverlaySession
┌────────────────────────────────────────────┐
│ Main Process │
│ │
│ 1. 选定当前光标所在 display │
│ 2. 枚举窗口:node-screenshots │
│ 3. 过滤隐藏窗口:get-windows 白名单 │
│ 4. 创建整屏 overlay BrowserWindow │
└──────────────┬─────────────────────────────┘
│ preload / IPC
┌────────────────────────────────────────────┐
│ Overlay Renderer │
│ │
│ 1. 渲染窗口高亮框与左上角 tag │
│ 2. 点击窗口 => captureWindow(windowId) │
│ 3. 拖拽区域 => captureRect(rect) │
└──────────────┬─────────────────────────────┘
│ IPC
┌────────────────────────────────────────────┐
│ Main Process Capture Path │
│ │
│ Window: node-screenshots.captureImage() │
│ Region: desktopCapturer + crop │
│ Output: clipboard.writeImage() │
└────────────────────────────────────────────┘
```
### 6.2 demo 文件职责
| 文件 | 作用 |
| -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- |
| [`tmp/electron-window-overlay-demo/main.mjs`](../../tmp/electron-window-overlay-demo/main.mjs) | 主进程入口;创建 overlay,枚举窗口,执行截图 |
| [`tmp/electron-window-overlay-demo/preload.cjs`](../../tmp/electron-window-overlay-demo/preload.cjs) | 为 overlay renderer 暴露 IPC bridge |
| [`tmp/electron-window-overlay-demo/renderer/index.html`](../../tmp/electron-window-overlay-demo/renderer/index.html) | overlay 渲染宿主页 |
| [`tmp/electron-window-overlay-demo/renderer/app.js`](../../tmp/electron-window-overlay-demo/renderer/app.js) | 窗口高亮、点击截窗、拖拽截区交互 |
| [`tmp/electron-window-overlay-demo/renderer/styles.css`](../../tmp/electron-window-overlay-demo/renderer/styles.css) | 遮罩视觉样式 |
| [`tmp/electron-window-overlay-demo/README.md`](../../tmp/electron-window-overlay-demo/README.md) | demo 的运行说明 |
## 7. 全屏 overlay 的关键实现参数
### 7.1 必要窗口参数
| 参数 / 调用 | 用途 | 必要性 |
| ----------------------------------- | ---------------------------------- | ------ |
| `x/y/width/height = display.bounds` | 覆盖整块屏幕,包括菜单栏区域 | 必需 |
| `transparent: true` | 允许渲染半透明遮罩 | 必需 |
| `frame: false` | 去除系统边框 | 必需 |
| `skipTaskbar: true` | 避免出现在任务栏 / Dock 窗口列表中 | 建议 |
| `hasShadow: false` | 避免覆盖层产生自身投影 | 建议 |
| `focusable: true` | 允许接收鼠标交互 | 必需 |
| `fullscreenable: false` | 避免进入原生 fullscreen 流程 | 建议 |
| `enableLargerThanScreen: true` | 提升跨平台稳健性 | 建议 |
| `type: 'panel'`macOS) | 更接近工具层窗口行为 | 建议 |
### 7.2 必要层级调用
| 调用 | 作用 |
| ---------------------------------------------------------------- | --------------------------------- |
| `setAlwaysOnTop(true, 'screen-saver')` | 让窗口位于更高层级 |
| `setVisibleOnAllWorkspaces(true, { visibleOnFullScreen: true })` | 避免 Space / 全屏窗口场景下不可见 |
| `setHiddenInMissionControl(true)` | 降低该窗口对系统窗口管理的干扰 |
### 7.3 重要结论
| 结论 | 说明 |
| ------------------------- | ------------------------------------------- |
| `display.workArea` 不可用 | 它会排除菜单栏 / Dock 区域 |
| `display.bounds` 必须使用 | 只有它能覆盖整个 display |
| `screen-saver` 层级有效 | 这是当前 macOS 上最接近需求的 Electron 方案 |
## 8. 系统窗口枚举与过滤策略
### 8.1 为什么不能只用 Electron
| Electron 能力 | 缺口 |
| --------------------------------------------------- | --------------------------------------------------------- |
| `desktopCapturer.getSources({ types: ['window'] })` | 能列出可捕获源,但没有稳定的窗口 bounds 用于 overlay 画框 |
| `DesktopCapturerSource.thumbnail` | 可截图缩略图,但不适合 “按原窗口精确高亮 + 点击即截” |
因此,纯 Electron 不足以完成 “系统窗口高亮 + 点击截窗”。
### 8.2 `node-screenshots` 的职责
| API | 用途 |
| --------------------------------- | -------------- |
| `Window.all()` | 枚举系统窗口 |
| `window.id()` | 稳定识别窗口 |
| `window.appName()` | 获取应用名 |
| `window.title()` | 获取标题 |
| `window.x()/y()/width()/height()` | 获取几何信息 |
| `window.captureImage()` | 截取该窗口图像 |
### 8.3 `get-windows` 的职责
`get-windows` 在当前方案中不负责截图,而只负责 “第二层白名单过滤”。
| 问题 | 处理方式 |
| ------------------------------------------ | ------------------------------------------------------------- |
| 某些应用逻辑上已隐藏,但底层枚举仍可能残留 | 只保留同时出现在 `get-windows``node-screenshots` 中的窗口 |
| Electron 自身的假关闭 /hide 行为 | 该白名单对这类情况更稳 |
### 8.4 当前过滤规则
| 规则 | 目的 |
| ------------------------------------------------ | ---------------------------- |
| `isMinimized() === false` | 排除最小化窗口 |
| 最小尺寸阈值:`80x60` | 排除菜单栏控件、过小悬浮面板 |
| 排除 `Dock` / `Window Server` / `Control Centre` | 排除系统 UI |
| 排除 demo 自身窗口 | 避免 overlay 自我高亮 |
| 必须与目标 display 相交 | 只画当前屏幕可见窗口 |
| 必须出现在 `get-windows` 白名单中 | 排除隐藏 / 伪关闭残留窗口 |
## 9. 截图路径设计
### 9.1 点击窗口截图
```text
点击高亮框
└───> renderer 发送 windowId
└───> main 查找对应 node-screenshots Window
└───> overlay.hide()
└───> captureImage()
└───> PNG Buffer
└───> nativeImage
└───> clipboard.writeImage()
```
### 9.2 拖拽区域截图
```text
拖拽区域
└───> renderer 发送全局 rect
└───> main 隐藏 overlay
└───> desktopCapturer 获取目标 display 图像
└───> 按 scaleFactor 计算 cropRect
└───> clipboard.writeImage()
```
### 9.3 为什么两条路径采用不同技术
| 路径 | 技术 | 原因 |
| ---------- | ------------------ | --------------------------------- |
| 按窗口截图 | `node-screenshots` | 它天然理解 “窗口” 这一对象 |
| 按区域截图 | `desktopCapturer` | 区域本质上是 display 上的矩形裁剪 |
## 10. 权限与平台边界
### 10.1 macOS 权限
| 权限 | 是否需要 | 用途 |
| ---------------- | ---------------- | ----------------------------------------------------- |
| Screen Recording | 需要 | 窗口截图、区域截图 |
| Accessibility | 当前方案不强依赖 | `get-windows` 已使用 `accessibilityPermission: false` |
### 10.2 当前已知平台边界
| 平台 / 场景 | 状态 | 说明 |
| ------------- | -------- | --------------------------------------------------------------------- |
| macOS | 已验证 | 当前主要验证平台 |
| Windows | 理论可行 | `node-screenshots` / `get-windows` 均支持,但尚未在本仓库内做实机验证 |
| Linux X11 | 理论可行 | 需要单独验证打包与权限 |
| Linux Wayland | 风险较高 | 上游库虽宣称支持,但必须做专项验证 |
### 10.3 特殊窗口风险
| 风险类型 | 当前处理 |
| ---------------------- | -------------------------------------------------------------- |
| 菜单栏状态窗 / 面板 | 通过尺寸阈值与排除名单降低噪音 |
| 系统 UI | 通过应用名黑名单排除 |
| 某些应用截图结果为黑图 | 已观察到个别状态面板存在此现象,应在业务层继续限制候选窗口类别 |
## 11. 已完成验证
| 验证项 | 结果 | 产物 |
| ----------------------------------- | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------- |
| overlay 覆盖整屏 | 通过 | [`tmp/electron-window-overlay-demo/.cache/window-overlay-visual.png`](../../tmp/electron-window-overlay-demo/.cache/window-overlay-visual.png) |
| `node-screenshots` 直接截图普通窗口 | 通过 | [`tmp/electron-window-overlay-demo/.cache/cursor-direct.png`](../../tmp/electron-window-overlay-demo/.cache/cursor-direct.png) |
| 点击高亮窗口后写入剪贴板 | 通过 | [`tmp/electron-window-overlay-demo/.cache/window-capture-probe-final.png`](../../tmp/electron-window-overlay-demo/.cache/window-capture-probe-final.png) |
| 拖拽区域截图 | 通过 | [`tmp/electron-window-overlay-demo/.cache/self-test-clipboard-final.png`](../../tmp/electron-window-overlay-demo/.cache/self-test-clipboard-final.png) |
## 12. 推荐的业务接入方式
### 12.1 总体建议
| 维度 | 建议 |
| -------------------- | ---------------------------------------------------------------------------------- |
| overlay 窗口生命周期 | 不建议直接挂进现有 `BrowserManager` 的常规窗口体系 |
| 原因 | overlay 是瞬态、全屏、平台特化、不可持久化的工具窗口,与主业务窗口生命周期明显不同 |
| 推荐做法 | 新增独立主进程模块管理 overlay;渲染内容仍建议走现有 SPA 路由体系 |
### 12.2 为什么不直接复用 `BrowserManager`
| 观察 | 影响 |
| ----------------------------------------- | ------------------------------- |
| `Browser` 默认承担普通业务窗口职责 | overlay 并非普通业务窗口 |
| `WindowStateManager` 倾向保存窗口状态 | overlay 不应持久化位置与大小 |
| `BrowserManager` 以 “可复用业务窗口” 建模 | overlay 更接近 “一次性工具会话” |
因此,更合理的做法是:
```text
┌────────────────────────────┐
│ BrowserManager │ 负责常规业务窗口
└────────────────────────────┘
┌────────────────────────────┐
│ CaptureOverlayManager │ 负责全屏截图 overlay 会话
└────────────────────────────┘
```
## 13. 建议的生产代码落点
### 13.1 主进程
| 建议文件 | 作用 |
| ---------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `apps/desktop/src/main/modules/screenCapture/CaptureOverlayManager.ts` | 创建 / 销毁 overlay 窗口;管理一次截图会话 |
| `apps/desktop/src/main/modules/screenCapture/WindowSourceService.ts` | 封装 `node-screenshots + get-windows` 的窗口枚举与过滤 |
| `apps/desktop/src/main/modules/screenCapture/CaptureService.ts` | 封装窗口截图、区域截图、剪贴板输出 |
| `apps/desktop/src/main/modules/screenCapture/permission.ts` | 封装 macOS 屏幕录制权限检查 |
| `apps/desktop/src/main/controllers/ScreenCaptureCtr.ts` | 对 renderer 暴露 `start / captureRect / captureWindow / close` IPC |
| `apps/desktop/src/main/controllers/registry.ts` | 注册 `ScreenCaptureCtr` |
### 13.2 IPC 类型
| 建议文件 | 作用 |
| --------------------------------------------------------- | ------------------------------------------------- |
| `packages/electron-client-ipc/src/types/screenCapture.ts` | 定义 overlay 会话、窗口元数据、截图参数与返回结果 |
| `packages/electron-client-ipc/src/types/index.ts` | 导出新类型 |
建议定义的核心类型:
| 类型名 | 用途 |
| -------------------------- | --------------------------------------------------- |
| `ScreenCaptureDisplayInfo` | display id / bounds / scaleFactor |
| `ScreenCaptureWindowInfo` | `windowId/appName/title/bounds/overlayBounds/order` |
| `ScreenCaptureSession` | `display + windows` |
| `CaptureRectParams` | 全局屏幕坐标的矩形 |
| `ScreenCaptureStartResult` | 权限状态、会话状态、错误信息 |
| `ScreenCaptureOutput` | `clipboard`、后续可扩展 `file``attachment` |
### 13.3 Preload 与 renderer service
| 建议文件 | 作用 |
| ----------------------------------------- | -------------------------------------------------- |
| `apps/desktop/src/preload/electronApi.ts` | 通常无需特殊改造;沿用统一 `invoke` 即可 |
| `src/services/electron/screenCapture.ts` | 前端统一调用 `ensureElectronIpc().screenCapture.*` |
### 13.4 Renderer 路由
生产环境存在两种可选实现:
| 方案 | 优点 | 缺点 | 建议 |
| ------------------ | -------------------------------- | -------------------------------- | ---------------- |
| 独立静态 HTML 页面 | 轻量、与业务隔离、最接近 demo | 与现有 React/i18n / 业务状态脱节 | 仅适合 spike |
| 独立桌面 SPA 路由 | 可复用现有构建、i18n、业务事件链 | 需要维护 desktop router 双配置 | **推荐生产使用** |
若采用 SPA 路由,建议新增:
| 建议文件 | 作用 |
| ------------------------------------------------------- | ------------------------------------ |
| `src/routes/(desktop)/screen-capture-overlay/index.tsx` | overlay 页面入口;仅负责挂载 UI 组件 |
| `src/features/DesktopScreenCaptureOverlay/*` | 业务组件、hooks、样式 |
| `src/spa/router/desktopRouter.config.tsx` | 动态路由配置 |
| `src/spa/router/desktopRouter.config.desktop.tsx` | 同步路由配置 |
必须注意:
| 规则 | 说明 |
| -------------------------------- | ------------------------------------ |
| 两份 desktop router 必须同时更新 | 否则 Electron 本地构建可能出现空白页 |
| overlay route 应保持极薄 | 不在 route 文件中堆叠业务逻辑 |
## 14. 托盘入口的真实接入点
若要从托盘启动 overlay,会涉及以下文件:
| 文件 | 作用 |
| ----------------------------------------------- | -------------------- |
| `apps/desktop/src/main/menus/impls/macOS.ts` | macOS 托盘菜单模板 |
| `apps/desktop/src/main/menus/impls/windows.ts` | Windows 托盘菜单模板 |
| `apps/desktop/src/main/menus/impls/linux.ts` | Linux 托盘菜单模板 |
| `apps/desktop/src/main/locales/default/menu.ts` | 托盘菜单文案 |
推荐新增文案键:
| Key | 语义 |
| -------------------------- | ------------------------ |
| `tray.captureScreen` | 启动截图 overlay |
| `tray.captureScreenWindow` | 启动窗口截图模式(可选) |
## 15. 业务接入分阶段计划
### 阶段一:桌面主进程能力落地
| 步骤 | 目标 |
| ---- | ---------------------------------------------------------------------------------- |
| 1 | 将 `node-screenshots``get-windows` 加入 `apps/desktop/package.json#dependencies` |
| 2 | 新建 `screenCapture` 主进程模块与 controller |
| 3 | 跑通托盘菜单触发 overlay |
| 4 | 继续以剪贴板为唯一输出 |
### 阶段二:接回现有业务 UI
| 步骤 | 目标 |
| ---- | -------------------------------------------------- |
| 1 | 新增桌面专用 overlay route /feature |
| 2 | 将截图结果从 “仅写剪贴板” 升级为 “回传 attachment” |
| 3 | 支持从 chat 输入区触发 |
| 4 | 支持截图后自动插入当前会话 |
### 阶段三:体验完善
| 步骤 | 目标 |
| ---- | ------------------------------------ |
| 1 | 多 display 支持 |
| 2 | Hover 高亮 / 文案优化 |
| 3 | 保存文件、编辑器标注、OCR 等增强能力 |
| 4 | 平台差异补齐(尤其 Windows / Linux |
## 16. 依赖落点与版本建议
### 16.1 应加入的位置
| 文件 | 说明 |
| --------------------------- | --------------------------------- |
| `apps/desktop/package.json` | Electron 桌面运行时的真实依赖落点 |
### 16.2 建议依赖
| 包名 | 用途 | 当前 demo 使用版本 |
| ------------------ | --------------------------- | ------------------ |
| `node-screenshots` | 枚举窗口 + 窗口截图 | `^0.2.8` |
| `get-windows` | 白名单过滤隐藏 / 伪关闭窗口 | `^9.3.0` |
说明:
| 项目 | 结论 |
| ---------------------------- | ---- |
| 这不是 “纯 Electron” 方案 | 成立 |
| 这也不是 “自研 native addon” | 成立 |
| 当前依赖的是开源原生库 | 成立 |
## 17. 测试建议
建议避免写 “窗口列表快照” 这类低信号测试,优先做行为测试。
| 测试层级 | 建议内容 |
| -------------- | ---------------------------------------------------------- |
| 单元测试 | 过滤逻辑:尺寸阈值、系统应用排除、自身窗口排除、白名单交集 |
| 主进程集成测试 | 权限失败、overlay 会话生命周期、错误分支 |
| 手工验证 | 菜单栏覆盖、点击截窗、拖拽截区、隐藏窗口过滤 |
建议手工验证清单:
| 检查项 | 期望 |
| ------------------------ | ------------------------ |
| 当前活动屏幕启动 overlay | 只覆盖当前目标 display |
| 已隐藏的 Electron 子窗口 | 不再出现边框 |
| 点击普通应用窗口 | 剪贴板中得到该窗口图像 |
| 拖拽区域截图 | 剪贴板中得到对应裁剪区域 |
| 取消操作 | `Esc` 可关闭 overlay |
## 18. 当前已确认的非目标
| 非目标 | 说明 |
| ----------------------------------- | ----------------------------------------------------------------------- |
| 当前阶段支持全平台一致体验 | 尚未完成 |
| 当前阶段支持窗口标题绝对准确 | `get-windows` 在无额外权限时标题可为空;当前主要依赖 `node-screenshots` |
| 当前阶段支持多 display 同时 overlay | 尚未实现 |
| 当前阶段支持标注编辑器 | 未实现 |
## 19. 后续实现时的推荐决策
| 决策点 | 推荐 |
| ----------------------------------------------- | ------------------------ |
| overlay 窗口是否复用 `BrowserManager` | 不推荐 |
| renderer 是否走 SPA route | 推荐 |
| 主进程是否继续保留 “剪贴板优先” 输出 | 推荐,先保持最小可用闭环 |
| 是否继续保留 `desktopCapturer` 作为区域截图路径 | 推荐 |
| 是否用 `get-windows` 继续做白名单过滤 | 推荐 |
## 20. 实施摘要
```text
┌──────────────────────────────────────────────┐
│ 已验证的技术事实 │
├──────────────────────────────────────────────┤
│ 1. Electron 可以创建覆盖整块 display 的窗体 │
│ 2. 纯 Electron 无法独立完成系统窗口高亮 │
│ 3. node-screenshots 可完成窗口枚举与截窗 │
│ 4. get-windows 可帮助过滤隐藏 / 残留窗口 │
│ 5. 最终可形成“点击窗口即截图 + 拖拽截区”闭环 │
└──────────────────────────────────────────────┘
```
本文档可视为后续将该能力正式接入 `apps/desktop` 主业务的实施基线。
+1
View File
@@ -255,6 +255,7 @@ const config = {
generateUpdatesFilesForAllChannels: true,
linux: {
category: 'Utility',
icon: 'build/icon.png',
maintainer: 'electronjs.org',
target: ['AppImage', 'snap', 'deb', 'rpm', 'tar.gz'],
},
+22 -3
View File
@@ -1,6 +1,7 @@
import { readFileSync } from 'node:fs';
import path from 'node:path';
import { vanillaExtractPlugin } from '@vanilla-extract/vite-plugin';
import dotenv from 'dotenv';
import { defineConfig } from 'electron-vite';
import type { PluginOption, ViteDevServer } from 'vite';
@@ -52,6 +53,11 @@ function electronDesktopHtmlPlugin(): PluginOption {
next();
return;
}
if (pathname === '/overlay' || pathname === '/overlay.html') {
req.url = '/apps/desktop/overlay.html';
next();
return;
}
if (pathname === '/popup.html') {
req.url = '/apps/desktop/popup.html';
next();
@@ -92,6 +98,8 @@ const updateChannel = process.env.UPDATE_CHANNEL;
const desktopPackageJson = JSON.parse(
readFileSync(path.resolve(__dirname, 'package.json'), 'utf8'),
) as { version: string };
const electronRuntimeExternals = ['electron'];
const mainProcessRuntimeExternals = [...electronRuntimeExternals, 'node-mac-permissions'];
console.info(`[electron-vite.config.ts] Detected UPDATE_CHANNEL: ${updateChannel}`);
@@ -100,10 +108,15 @@ export default defineConfig({
build: {
minify: !isDev,
outDir: 'dist/main',
rollupOptions: {
rolldownOptions: {
// Native modules must be externalized to work correctly.
// bufferutil and utf-8-validate are optional peer deps of ws that may not be installed.
external: [...getExternalDependencies(), 'bufferutil', 'utf-8-validate'],
external: [
...mainProcessRuntimeExternals,
...getExternalDependencies(),
'bufferutil',
'utf-8-validate',
],
output: {
// Prevent debug package from being bundled into index.js to avoid side-effect pollution
manualChunks(id) {
@@ -137,6 +150,9 @@ export default defineConfig({
build: {
minify: !isDev,
outDir: 'dist/preload',
rolldownOptions: {
external: electronRuntimeExternals,
},
sourcemap: isDev ? 'inline' : false,
},
resolve: {
@@ -150,9 +166,10 @@ export default defineConfig({
root: ROOT_DIR,
build: {
outDir: path.resolve(__dirname, 'dist/renderer'),
rollupOptions: {
rolldownOptions: {
input: {
main: path.resolve(__dirname, 'index.html'),
overlay: path.resolve(__dirname, 'overlay.html'),
popup: path.resolve(__dirname, 'popup.html'),
},
output: sharedRollupOutput,
@@ -166,10 +183,12 @@ export default defineConfig({
plugins: [
forceAbsoluteBasePlugin(),
electronDesktopHtmlPlugin(),
vanillaExtractPlugin(),
...(sharedRendererPlugins({ platform: 'desktop' }) as PluginOption[]),
],
resolve: {
dedupe: ['react', 'react-dom'],
tsconfigPaths: true,
},
},
});
+2 -1
View File
@@ -36,7 +36,8 @@ export const nativeModules = [
// macOS-only native modules
...(isDarwin ? ['node-mac-permissions'] : []),
'@napi-rs/canvas',
// Add more native modules here as needed
'get-windows',
'node-screenshots',
];
/**
+15
View File
@@ -0,0 +1,15 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
html, body, #root { width: 100%; height: 100%; overflow: hidden; background: transparent; }
</style>
</head>
<body>
<div id="root"></div>
<script type="module" src="/apps/desktop/src/overlay/entry.tsx"></script>
</body>
</html>
+10 -4
View File
@@ -42,7 +42,10 @@
"update-server": "sh scripts/update-test/run-test.sh"
},
"dependencies": {
"@napi-rs/canvas": "^0.1.70"
"@lobehub/fluent-emoji": "^4.1.0",
"@napi-rs/canvas": "^0.1.70",
"get-windows": "^9.3.0",
"node-screenshots": "^0.2.8"
},
"devDependencies": {
"@electron-toolkit/eslint-config-prettier": "^3.0.0",
@@ -64,6 +67,8 @@
"@types/semver": "^7.7.1",
"@types/set-cookie-parser": "^2.4.10",
"@typescript/native-preview": "7.0.0-dev.20251210.1",
"@vanilla-extract/css": "^1.17.4",
"@vanilla-extract/vite-plugin": "^5.1.0",
"async-retry": "^1.3.3",
"consola": "^3.4.2",
"cookie": "^1.1.1",
@@ -76,7 +81,7 @@
"electron-log": "^5.4.3",
"electron-store": "^8.2.0",
"electron-updater": "^6.6.2",
"electron-vite": "^5.0.0",
"electron-vite": "6.0.0-beta.1",
"electron-window-state": "^5.0.3",
"es-toolkit": "^1.43.0",
"eslint": "10.0.0",
@@ -96,13 +101,14 @@
"resolve": "^1.22.11",
"semver": "^7.7.3",
"set-cookie-parser": "^2.7.2",
"strip-ansi": "6.0.1",
"stylelint": "^15.11.0",
"superjson": "^2.2.6",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"undici": "^7.16.0",
"uuid": "^13.0.0",
"vite": "^7.3.1",
"uuid": "^14.0.0",
"vite": "^8.0.9",
"vitest": "^3.2.4",
"zod": "^3.25.76"
},
+2
View File
@@ -1,4 +1,6 @@
packages:
- '../cli'
- '../../packages/agent-gateway-client'
- '../../packages/const'
- '../../packages/electron-server-ipc'
- '../../packages/electron-client-ipc'
+2 -1
View File
@@ -86,5 +86,6 @@
"window.minimize": "تصغير",
"window.title": "نافذة",
"window.toggleFullscreen": "تبديل وضع ملء الشاشة",
"window.zoom": "تكبير"
"window.zoom": "تكبير",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Минимизирай",
"window.title": "Прозорец",
"window.toggleFullscreen": "Превключи на цял екран",
"window.zoom": "Мащаб"
"window.zoom": "Мащаб",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Minimieren",
"window.title": "Fenster",
"window.toggleFullscreen": "Vollbild umschalten",
"window.zoom": "Zoom"
"window.zoom": "Zoom",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -15,6 +15,11 @@
"fullDiskAccess.openSettings": "Open Settings",
"fullDiskAccess.skip": "Later",
"fullDiskAccess.title": "Full Disk Access Required",
"screenCaptureAccess.cancel": "Later",
"screenCaptureAccess.detail": "Open System Settings, enable Screen Recording for LobeHub, then try Quick Composer again.",
"screenCaptureAccess.message": "Quick Composer needs Screen Recording permission before it can capture screenshots.",
"screenCaptureAccess.openSettings": "Open Settings",
"screenCaptureAccess.title": "Screen Recording Permission Required",
"update.downloadAndInstall": "Download and Install",
"update.downloadComplete": "Download Complete",
"update.downloadCompleteMessage": "Update downloaded. Install now?",
@@ -71,6 +71,8 @@
"macOS.services": "Services",
"macOS.unhide": "Show All",
"tray.open": "Open {{appName}}",
"tray.openMiniToolbar": "Quick Composer",
"tray.quickChat": "Quick Chat",
"tray.quit": "Quit",
"tray.show": "Show {{appName}}",
"view.forceReload": "Force Reload",
@@ -86,5 +86,6 @@
"window.minimize": "Minimizar",
"window.title": "Ventana",
"window.toggleFullscreen": "Alternar pantalla completa",
"window.zoom": "Zoom"
"window.zoom": "Zoom",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "کوچک کردن",
"window.title": "پنجره",
"window.toggleFullscreen": "تغییر به حالت تمام صفحه",
"window.zoom": "زوم"
"window.zoom": "زوم",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Réduire",
"window.title": "Fenêtre",
"window.toggleFullscreen": "Basculer en plein écran",
"window.zoom": "Zoom"
"window.zoom": "Zoom",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Minimizza",
"window.title": "Finestra",
"window.toggleFullscreen": "Attiva/disattiva schermo intero",
"window.zoom": "Zoom"
"window.zoom": "Zoom",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "最小化",
"window.title": "ウィンドウ",
"window.toggleFullscreen": "フルスクリーン切替",
"window.zoom": "ズーム"
"window.zoom": "ズーム",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "최소화",
"window.title": "창",
"window.toggleFullscreen": "전체 화면 전환",
"window.zoom": "줌"
"window.zoom": "줌",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Minimaliseren",
"window.title": "Venster",
"window.toggleFullscreen": "Schakel volledig scherm in/uit",
"window.zoom": "Inzoomen"
"window.zoom": "Inzoomen",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Zminimalizuj",
"window.title": "Okno",
"window.toggleFullscreen": "Przełącz tryb pełnoekranowy",
"window.zoom": "Powiększenie"
"window.zoom": "Powiększenie",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Minimizar",
"window.title": "Janela",
"window.toggleFullscreen": "Alternar Tela Cheia",
"window.zoom": "Zoom"
"window.zoom": "Zoom",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Свернуть",
"window.title": "Окно",
"window.toggleFullscreen": "Переключить полноэкранный режим",
"window.zoom": "Масштаб"
"window.zoom": "Масштаб",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Küçült",
"window.title": "Pencere",
"window.toggleFullscreen": "Tam Ekrana Geç",
"window.zoom": "Yakınlaştır"
"window.zoom": "Yakınlaştır",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -86,5 +86,6 @@
"window.minimize": "Thu nhỏ",
"window.title": "Cửa sổ",
"window.toggleFullscreen": "Chuyển đổi toàn màn hình",
"window.zoom": "Thu phóng"
"window.zoom": "Thu phóng",
"tray.openMiniToolbar": "Quick Composer"
}
@@ -15,6 +15,11 @@
"fullDiskAccess.openSettings": "打开设置",
"fullDiskAccess.skip": "稍后",
"fullDiskAccess.title": "需要完全磁盘访问权限",
"screenCaptureAccess.cancel": "稍后",
"screenCaptureAccess.detail": "请打开系统设置,为 LobeHub 开启“屏幕录制”权限,然后再次尝试 Quick Composer。",
"screenCaptureAccess.message": "Quick Composer 需要“屏幕录制”权限后才能进行截图。",
"screenCaptureAccess.openSettings": "打开设置",
"screenCaptureAccess.title": "需要屏幕录制权限",
"update.downloadAndInstall": "下载并安装",
"update.downloadComplete": "下载完成",
"update.downloadCompleteMessage": "已下载更新。现在安装吗?",
@@ -72,6 +72,8 @@
"macOS.services": "服务",
"macOS.unhide": "全部显示",
"tray.open": "打开 {{appName}}",
"tray.openMiniToolbar": "Quick Composer",
"tray.quickChat": "快捷聊天",
"tray.quit": "退出",
"tray.show": "显示 {{appName}}",
"view.forceReload": "强制重新加载",
@@ -86,5 +86,6 @@
"window.minimize": "最小化",
"window.title": "視窗",
"window.toggleFullscreen": "切換全螢幕",
"window.zoom": "縮放"
"window.zoom": "縮放",
"tray.openMiniToolbar": "Quick Composer"
}
Binary file not shown.

Before

Width:  |  Height:  |  Size: 807 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 738 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 393 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 704 B

@@ -0,0 +1,51 @@
#!/usr/bin/env node
/**
* Generate the macOS tray template icon set (black + alpha).
*
* Template images must contain only black pixels and an alpha channel;
* macOS then recolors them automatically based on the menu bar theme.
*
* Renders two files in apps/desktop/resources:
* - trayTemplate.png (@1x, 18x18)
* - trayTemplate@2x.png (@2x, 36x36)
*
* Run: bun run apps/desktop/scripts/generate-tray-template.mjs
*/
import { mkdir } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import sharp from 'sharp';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const outDir = path.resolve(__dirname, '..', 'resources');
// Silhouette derived from the LobeHub logo. Eyes and mouth are cut as
// transparent holes via fill-rule=evenodd so they remain visible when
// macOS tints the entire shape in a single color.
const svg = `<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 320">
<path fill="#000" d="M172.997 19.016c-14.027 0-19.5-11.5-41-11-23.394 0-34 13-45.5 23-1.958 1.702-11.5 7-16 9-19.683 8.748-34.5 21.5-34.5 40.5 0 20.711 17.461 37.5 39 37.5 3.536 0 6.963-.453 10.22-1.301 8.7 10.539 22.179 16.658 37.28 17.301 23.5 1 31-15.25 44.5-8.5 9.259 4.629 13.83 8.5 28.5 8.5 17.108 0 25.057-5.233 30-11 9-10.5 22.879-4 31.5-4 18.778 0 34-14.551 34-32.5 0-17.95-15.222-32.5-34-32.5-5.15 0-14.856 1.27-17-7-3.5-13.5-20.148-29-44-29-9.318 0-17.691 1-23 1z"/>
<path fill="#000" fill-rule="evenodd" d="M294 172.519c0 75.655-59.442 128.5-134 128.5-74.558 0-134-53.845-134-129.5 0-22.5 5-32.141 31.5-35.671 47.5-6.329 72.542-3.829 102.5-3.829 29.959 0 72.556-1.27 102.5 3.829 24.5 4.171 30 8.671 31.5 36.671zM101 221.012c15.464 0 28-12.536 28-28s-12.536-28-28-28-28 12.536-28 28 12.536 28 28 28zM219 221.012c15.464 0 28-12.536 28-28s-12.536-28-28-28-28 12.536-28 28 12.536 28 28 28zM159.75 242.51c-28.25 0-35.75 3.5-35.75 3.5s3.5 27 35.75 27 35.75-27 35.75-27-7.5-3.5-35.75-3.5z"/>
</svg>
`;
async function render(size, outFile) {
const buf = Buffer.from(svg);
await sharp(buf, { density: Math.max(72, size * 12) })
.resize(size, size, { fit: 'contain', background: { r: 0, g: 0, b: 0, alpha: 0 } })
.png()
.toFile(outFile);
console.log(`wrote ${path.relative(process.cwd(), outFile)} (${size}x${size})`);
}
async function main() {
await mkdir(outDir, { recursive: true });
await render(18, path.join(outDir, 'trayTemplate.png'));
await render(36, path.join(outDir, 'trayTemplate@2x.png'));
}
main().catch((err) => {
console.error(err);
process.exit(1);
});
+10 -10
View File
@@ -1,29 +1,29 @@
import { join } from 'node:path';
import path from 'node:path';
import { app } from 'electron';
export const mainDir = join(__dirname);
export const mainDir = path.join(__dirname);
export const preloadDir = join(mainDir, '../preload');
export const preloadDir = path.join(mainDir, '../preload');
export const resourcesDir = join(mainDir, '../../resources');
export const resourcesDir = path.join(mainDir, '../../resources');
export const buildDir = join(mainDir, '../../build');
export const buildDir = path.join(mainDir, '../../build');
export const binDir = app.isPackaged
? join(process.resourcesPath, 'bin')
: join(resourcesDir, 'bin');
? path.join(process.resourcesPath, 'bin')
: path.join(resourcesDir, 'bin');
const appPath = app.getAppPath();
export const rendererDir = join(appPath, 'dist', 'renderer');
export const rendererDir = path.join(appPath, 'dist', 'renderer');
export const userDataDir = app.getPath('userData');
export const appStorageDir = join(userDataDir, 'lobehub-storage');
export const appStorageDir = path.join(userDataDir, 'lobehub-storage');
// Legacy local database directory used in older desktop versions
export const legacyLocalDbDir = join(appStorageDir, 'lobehub-local-db');
export const legacyLocalDbDir = path.join(appStorageDir, 'lobehub-local-db');
// ------ Application storage directory ---- //
+5 -5
View File
@@ -1,16 +1,16 @@
import os from 'node:os';
import { dev, linux, macOS, windows } from 'electron-is';
import * as electronIs from 'electron-is';
import { getDesktopEnv } from '@/env';
export const isDev = dev();
export const isDev = electronIs.dev();
export const OFFICIAL_CLOUD_SERVER = getDesktopEnv().OFFICIAL_CLOUD_SERVER;
export const isMac = macOS();
export const isWindows = windows();
export const isLinux = linux();
export const isMac = electronIs.macOS();
export const isWindows = electronIs.windows();
export const isLinux = electronIs.linux();
function getIsMacTahoe(): boolean {
if (!isMac) return false;
@@ -16,11 +16,21 @@ export default class BrowserWindowsCtr extends ControllerModule {
static override readonly groupName = 'windows';
@shortcut('showApp')
async toggleMainWindow() {
toggleMainWindow() {
const mainWindow = this.app.browserManager.getMainWindow();
mainWindow.toggleVisible();
}
@shortcut('quickComposer')
async openQuickComposer() {
await this.app.screenCaptureManager.startSession();
}
@shortcut('quickChat')
openQuickChat() {
this.app.browserManager.openQuickChatPopup();
}
@IpcMethod()
async openSettingsWindow(options?: string | OpenSettingsWindowOptions) {
const normalizedOptions: OpenSettingsWindowOptions =
+96 -1
View File
@@ -9,6 +9,8 @@ import type {
GitBranchListItem,
GitCheckoutResult,
GitLinkedPullRequestResult,
GitPullResult,
GitPushResult,
GitWorkingTreeFiles,
GitWorkingTreeStatus,
} from '@lobechat/electron-client-ipc';
@@ -263,10 +265,24 @@ export default class GitController extends ControllerModule {
* Count commits HEAD is ahead/behind its upstream tracking ref.
* Returns `hasUpstream: false` when the branch has no upstream configured
* (e.g. local-only branches, or after the remote branch is deleted).
*
* Does a best-effort `git fetch` first so the result reflects what's
* actually on the remote — the renderer calls this via SWR with
* `revalidateOnFocus`, so the fetch piggybacks on window re-focus. Fetch
* failures (offline, no credentials, no `origin` remote) are swallowed so
* we still return whatever can be computed against the cached refs.
*/
@IpcMethod()
async getGitAheadBehind(dirPath: string): Promise<GitAheadBehind> {
const execFileAsync = promisify(execFile);
try {
await execFileAsync('git', ['fetch', '--no-tags', '--quiet', 'origin'], {
cwd: dirPath,
timeout: 10_000,
});
} catch {
// swallow — fall through to compute against cached refs
}
try {
const { stdout: upstreamOut } = await execFileAsync(
'git',
@@ -284,7 +300,36 @@ export default class GitController extends ControllerModule {
const [behindStr, aheadStr] = stdout.trim().split(/\s+/);
const behind = Number.parseInt(behindStr ?? '0', 10) || 0;
const ahead = Number.parseInt(aheadStr ?? '0', 10) || 0;
return { ahead, behind, hasUpstream: true, upstream };
// `git push -u origin HEAD` always targets origin/<current-branch-name>,
// which may differ from upstream (the branched-off-canary case).
let pushTarget: string | undefined;
let pushTargetExists = false;
try {
const { stdout: branchOut } = await execFileAsync(
'git',
['symbolic-ref', '--short', 'HEAD'],
{ cwd: dirPath, timeout: 5000 },
);
const branch = branchOut.trim();
if (branch) {
pushTarget = `origin/${branch}`;
try {
await execFileAsync(
'git',
['rev-parse', '--verify', '--quiet', `refs/remotes/${pushTarget}`],
{ cwd: dirPath, timeout: 5000 },
);
pushTargetExists = true;
} catch {
pushTargetExists = false;
}
}
} catch {
// detached HEAD — leave pushTarget undefined
}
return { ahead, behind, hasUpstream: true, pushTarget, pushTargetExists, upstream };
} catch {
// No upstream configured, detached HEAD, or git error — all treated as "no upstream"
return { ahead: 0, behind: 0, hasUpstream: false };
@@ -322,4 +367,54 @@ export default class GitController extends ControllerModule {
return { error: stderr || 'git checkout failed', success: false };
}
}
/**
* Pull the current branch's upstream via fast-forward only.
*
* `--ff-only` avoids creating accidental merge commits when the local branch
* has diverged — in that case the user should resolve merge/rebase in their
* own terminal. For the common "just behind" case this is a safe one-click.
*/
@IpcMethod()
async pullGitBranch(payload: { path: string }): Promise<GitPullResult> {
const { path: dirPath } = payload;
const execFileAsync = promisify(execFile);
try {
const { stdout } = await execFileAsync('git', ['pull', '--ff-only'], {
cwd: dirPath,
timeout: 60_000,
});
const noop = /Already up to date/i.test(stdout);
return { noop, success: true };
} catch (error: any) {
const stderr: string = (error?.stderr ?? error?.message ?? '').toString().trim();
logger.debug('[pullGitBranch] failed', { stderr });
return { error: stderr || 'git pull failed', success: false };
}
}
/**
* Push the current branch to its same-named remote on `origin`.
*
* Uses `git push -u origin HEAD` instead of plain `git push` so the action
* works even when local branch name differs from the configured upstream
*/
@IpcMethod()
async pushGitBranch(payload: { path: string }): Promise<GitPushResult> {
const { path: dirPath } = payload;
const execFileAsync = promisify(execFile);
try {
const { stderr } = await execFileAsync('git', ['push', '-u', 'origin', 'HEAD'], {
cwd: dirPath,
timeout: 60_000,
});
// git push writes progress/status to stderr even on success
const noop = /Everything up-to-date/i.test(stderr);
return { noop, success: true };
} catch (error: any) {
const stderr: string = (error?.stderr ?? error?.message ?? '').toString().trim();
logger.debug('[pushGitBranch] failed', { stderr });
return { error: stderr || 'git push failed', success: false };
}
}
}
@@ -1,53 +1,56 @@
import type { ChildProcess } from 'node:child_process';
import { spawn } from 'node:child_process';
import { createHash, randomUUID } from 'node:crypto';
import { mkdir, readFile, writeFile } from 'node:fs/promises';
import { access, mkdir, readFile, writeFile } from 'node:fs/promises';
import path from 'node:path';
import type { Readable, Writable } from 'node:stream';
import type { HeterogeneousAgentSessionError } from '@lobechat/electron-client-ipc';
import {
CLAUDE_CODE_CLI_INSTALL_COMMANDS,
CLAUDE_CODE_CLI_INSTALL_DOCS_URL,
CODEX_CLI_INSTALL_COMMANDS,
CODEX_CLI_INSTALL_DOCS_URL,
HeterogeneousAgentSessionErrorCode,
} from '@lobechat/electron-client-ipc';
import { app as electronApp, BrowserWindow } from 'electron';
import { getHeterogeneousAgentDriver } from '@/modules/heterogeneousAgent';
import type {
HeterogeneousAgentImageAttachment,
HeterogeneousAgentParsedOutput,
} from '@/modules/heterogeneousAgent/types';
import { buildProxyEnv } from '@/modules/networkProxy/envBuilder';
import { detectHeterogeneousCliCommand } from '@/modules/toolDetectors';
import { createLogger } from '@/utils/logger';
import { ControllerModule, IpcMethod } from './index';
const logger = createLogger('controllers:HeterogeneousAgentCtr');
const CODEX_RESUME_THREAD_NOT_FOUND_PATTERNS = [
/no conversation found/i,
/thread .*not found/i,
/conversation .*not found/i,
/resume.*not found/i,
] as const;
const CLI_AUTH_REQUIRED_PATTERNS = [
/failed to authenticate/i,
/invalid authentication credentials/i,
/authentication[_ ]error/i,
/not authenticated/i,
/\bunauthorized\b/i,
/\b401\b/,
] as const;
const CODEX_RESUME_CWD_MISMATCH_PATTERNS = [
/working directory/i,
/\bcwd\b/i,
/different directory/i,
/directory.*mismatch/i,
] as const;
/** Directory under appStoragePath for caching downloaded files */
const FILE_CACHE_DIR = 'heteroAgent/files';
// ─── CLI presets per agent type ───
// Mirrors @lobechat/heterogeneous-agents/registry but runs in main process
// (can't import from the workspace package in Electron main directly)
interface CLIPreset {
baseArgs: string[];
promptMode: 'positional' | 'stdin';
resumeArgs?: (sessionId: string) => string[];
}
const CLI_PRESETS: Record<string, CLIPreset> = {
'claude-code': {
baseArgs: [
'-p',
'--input-format',
'stream-json',
'--output-format',
'stream-json',
'--verbose',
'--include-partial-messages',
'--permission-mode',
'bypassPermissions',
],
promptMode: 'stdin',
resumeArgs: (sid) => ['--resume', sid],
},
// Future presets:
// 'codex': { baseArgs: [...], promptMode: 'positional' },
// 'kimi-cli': { baseArgs: [...], promptMode: 'positional' },
};
// ─── IPC types ───
interface StartSessionParams {
@@ -69,14 +72,9 @@ interface StartSessionResult {
sessionId: string;
}
interface ImageAttachment {
id: string;
url: string;
}
interface SendPromptParams {
/** Image attachments to include in the prompt (downloaded from url, cached by id) */
imageList?: ImageAttachment[];
imageList?: HeterogeneousAgentImageAttachment[];
prompt: string;
sessionId: string;
}
@@ -115,15 +113,19 @@ interface AgentSession {
cwd?: string;
env?: Record<string, string>;
process?: ChildProcess;
resumeSessionId?: string;
sessionId: string;
}
type SessionErrorPayload = HeterogeneousAgentSessionError | string;
/**
* External Agent Controller — manages external agent CLI processes via Electron IPC.
*
* Agent-agnostic: uses CLI presets from a registry to support Claude Code,
* Codex, Kimi CLI, etc. Only handles process lifecycle and raw stdout line
* broadcasting. All event parsing and DB persistence happens on the Renderer side.
* Agent-agnostic: delegates spawn-plan construction and stdout framing to a
* per-agent driver so Claude Code, Codex, and future CLIs can differ in
* prompt transport, resume semantics, and raw stream shape without turning
* this controller into a giant `switch`.
*
* Lifecycle: startSession → sendPrompt → (heteroAgentRawLine broadcasts) → stopSession
*/
@@ -132,6 +134,203 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
private sessions = new Map<string, AgentSession>();
private resolveSessionCommand(session: AgentSession): string {
const resolvedCommand = session.command.trim();
if (resolvedCommand) return resolvedCommand;
return session.agentType === 'codex' ? 'codex' : 'claude';
}
private buildCodexCliMissingError(session: AgentSession): HeterogeneousAgentSessionError {
const command = this.resolveSessionCommand(session);
return {
agentType: 'codex',
code: HeterogeneousAgentSessionErrorCode.CliNotFound,
command,
docsUrl: CODEX_CLI_INSTALL_DOCS_URL,
installCommands: CODEX_CLI_INSTALL_COMMANDS,
message: `Codex CLI was not found. Install it and make sure \`${command}\` can be executed.`,
};
}
private buildClaudeCodeCliMissingError(session: AgentSession): HeterogeneousAgentSessionError {
const command = this.resolveSessionCommand(session);
return {
agentType: 'claude-code',
code: HeterogeneousAgentSessionErrorCode.CliNotFound,
command,
docsUrl: CLAUDE_CODE_CLI_INSTALL_DOCS_URL,
installCommands: CLAUDE_CODE_CLI_INSTALL_COMMANDS,
message: `Claude Code CLI was not found. Install it and make sure \`${command}\` can be executed.`,
};
}
private buildCliMissingError(session: AgentSession): HeterogeneousAgentSessionError | undefined {
switch (session.agentType) {
case 'claude-code': {
return this.buildClaudeCodeCliMissingError(session);
}
case 'codex': {
return this.buildCodexCliMissingError(session);
}
default: {
return;
}
}
}
private buildCliAuthRequiredError(
session: AgentSession,
stderr: string,
): HeterogeneousAgentSessionError | undefined {
const command = this.resolveSessionCommand(session);
switch (session.agentType) {
case 'claude-code': {
return {
agentType: 'claude-code',
code: HeterogeneousAgentSessionErrorCode.AuthRequired,
command,
docsUrl: CLAUDE_CODE_CLI_INSTALL_DOCS_URL,
message:
'Claude Code could not authenticate. Sign in again or refresh its credentials, then retry.',
stderr,
};
}
case 'codex': {
return {
agentType: 'codex',
code: HeterogeneousAgentSessionErrorCode.AuthRequired,
command,
docsUrl: CODEX_CLI_INSTALL_DOCS_URL,
message:
'Codex could not authenticate. Sign in again or refresh its credentials, then retry.',
stderr,
};
}
default: {
return;
}
}
}
private getErrorMessage(error: unknown): string | undefined {
return typeof error === 'string'
? error
: error instanceof Error
? error.message
: typeof error === 'object' &&
error &&
'message' in error &&
typeof error.message === 'string'
? error.message
: undefined;
}
private buildCodexResumeError(
code:
| typeof HeterogeneousAgentSessionErrorCode.ResumeCwdMismatch
| typeof HeterogeneousAgentSessionErrorCode.ResumeThreadNotFound,
stderr: string,
session: AgentSession,
): HeterogeneousAgentSessionError {
const message =
code === HeterogeneousAgentSessionErrorCode.ResumeCwdMismatch
? 'The saved Codex thread can only be resumed from its original working directory.'
: 'The saved Codex thread could not be found, so it can no longer be resumed.';
return {
agentType: 'codex',
code,
command: session.command,
message,
resumeSessionId: session.resumeSessionId,
stderr,
workingDirectory: session.cwd,
};
}
private getCodexResumeError(
error: unknown,
session: AgentSession,
): HeterogeneousAgentSessionError | undefined {
if (session.agentType !== 'codex' || !session.resumeSessionId) return;
const message = this.getErrorMessage(error);
if (!message) return;
if (CODEX_RESUME_CWD_MISMATCH_PATTERNS.some((pattern) => pattern.test(message))) {
return this.buildCodexResumeError(
HeterogeneousAgentSessionErrorCode.ResumeCwdMismatch,
message,
session,
);
}
if (CODEX_RESUME_THREAD_NOT_FOUND_PATTERNS.some((pattern) => pattern.test(message))) {
return this.buildCodexResumeError(
HeterogeneousAgentSessionErrorCode.ResumeThreadNotFound,
message,
session,
);
}
}
private getCliAuthRequiredError(
error: unknown,
session: AgentSession,
): HeterogeneousAgentSessionError | undefined {
const message = this.getErrorMessage(error);
if (!message || !CLI_AUTH_REQUIRED_PATTERNS.some((pattern) => pattern.test(message))) return;
return this.buildCliAuthRequiredError(session, message);
}
private getSessionErrorPayload(error: unknown, session: AgentSession): SessionErrorPayload {
if (typeof error === 'object' && error && 'code' in error && error.code === 'ENOENT') {
const cliMissingError = this.buildCliMissingError(session);
if (cliMissingError) return cliMissingError;
}
const resumeError = this.getCodexResumeError(error, session);
if (resumeError) return resumeError;
const authRequiredError = this.getCliAuthRequiredError(error, session);
if (authRequiredError) return authRequiredError;
return error instanceof Error ? error.message : String(error);
}
private async getSpawnPreflightError(
session: AgentSession,
): Promise<HeterogeneousAgentSessionError | undefined> {
const defaultCommand =
session.agentType === 'claude-code'
? 'claude'
: session.agentType === 'codex'
? 'codex'
: undefined;
if (!defaultCommand) return;
const command = this.resolveSessionCommand(session);
const status =
command === defaultCommand
? await this.app.toolDetectorManager?.detect?.(defaultCommand, true)
: await detectHeterogeneousCliCommand(
session.agentType === 'claude-code' ? 'claude-code' : 'codex',
command,
);
const cliMissingError = this.buildCliMissingError(session);
if (!status || status.available || !cliMissingError) return;
return cliMissingError;
}
// ─── Broadcast ───
private broadcast<T>(channel: string, data: T) {
@@ -164,7 +363,7 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
* Download an image by URL, with local disk cache keyed by id.
*/
private async resolveImage(
image: ImageAttachment,
image: HeterogeneousAgentImageAttachment,
): Promise<{ buffer: Buffer; mimeType: string }> {
const cacheDir = this.fileCacheDir;
const cacheKey = this.getImageCacheKey(image.id);
@@ -201,12 +400,71 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
return { buffer, mimeType };
}
private guessImageExtension(
mimeType: string,
image: HeterogeneousAgentImageAttachment,
): string | undefined {
const knownByMime: Record<string, string> = {
'image/gif': '.gif',
'image/jpeg': '.jpg',
'image/png': '.png',
'image/webp': '.webp',
};
if (knownByMime[mimeType]) return knownByMime[mimeType];
try {
const pathname = new URL(image.url).pathname;
const ext = path.extname(pathname);
return ext || undefined;
} catch {
return undefined;
}
}
/**
* Materialize an image attachment into a stable local file path so CLIs like
* Codex can consume it through `--image <file>`.
*/
private async resolveCliImagePath(image: HeterogeneousAgentImageAttachment): Promise<string> {
const { buffer, mimeType } = await this.resolveImage(image);
const cacheKey = this.getImageCacheKey(image.id);
const ext = this.guessImageExtension(mimeType, image) || '';
const filePath = path.join(this.fileCacheDir, `${cacheKey}${ext}`);
try {
await access(filePath);
} catch {
await mkdir(this.fileCacheDir, { recursive: true });
await writeFile(filePath, buffer);
}
return filePath;
}
private async resolveCliImagePaths(
imageList: HeterogeneousAgentImageAttachment[] = [],
): Promise<string[]> {
const resolved = await Promise.all(
imageList.map(async (image) => {
try {
return await this.resolveCliImagePath(image);
} catch (err) {
logger.error(`Failed to materialize image ${image.id} for CLI:`, err);
return undefined;
}
}),
);
return resolved.filter(Boolean) as string[];
}
/**
* Build a stream-json user message with text + optional image content blocks.
*/
private async buildStreamJsonInput(
prompt: string,
imageList: ImageAttachment[] = [],
imageList: HeterogeneousAgentImageAttachment[] = [],
): Promise<string> {
const content: any[] = [{ text: prompt, type: 'text' }];
@@ -226,10 +484,10 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
}
}
return JSON.stringify({
return `${JSON.stringify({
message: { content, role: 'user' },
type: 'user',
});
})}\n`;
}
// ─── IPC methods ───
@@ -241,6 +499,7 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
async startSession(params: StartSessionParams): Promise<StartSessionResult> {
const sessionId = randomUUID();
const agentType = params.agentType || 'claude-code';
getHeterogeneousAgentDriver(agentType);
this.sessions.set(sessionId, {
// If resuming, pre-set the agent session ID so sendPrompt adds --resume
@@ -251,6 +510,7 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
cwd: params.cwd,
env: params.env,
sessionId,
resumeSessionId: params.resumeSessionId,
});
logger.info('Session created:', { agentType, sessionId });
@@ -268,32 +528,31 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
const session = this.sessions.get(params.sessionId);
if (!session) throw new Error(`Session not found: ${params.sessionId}`);
const preset = CLI_PRESETS[session.agentType];
if (!preset) throw new Error(`Unknown agent type: ${session.agentType}`);
const useStdin = preset.promptMode === 'stdin';
// Build stream-json payload up-front so any image download errors
// surface before the process is spawned.
let stdinPayload: string | undefined;
if (useStdin) {
stdinPayload = await this.buildStreamJsonInput(params.prompt, params.imageList ?? []);
const preflightError = await this.getSpawnPreflightError(session);
if (preflightError) {
this.broadcast('heteroAgentSessionError', {
error: preflightError,
sessionId: session.sessionId,
});
throw new Error(preflightError.message);
}
return new Promise<void>((resolve, reject) => {
// Build CLI args: base preset + resume + user args
const cliArgs = [
...preset.baseArgs,
...(session.agentSessionId && preset.resumeArgs
? preset.resumeArgs(session.agentSessionId)
: []),
...session.args,
];
const driver = getHeterogeneousAgentDriver(session.agentType);
const spawnPlan = await driver.buildSpawnPlan({
args: session.args,
helpers: {
buildClaudeStreamJsonInput: (prompt, imageList) =>
this.buildStreamJsonInput(prompt, imageList),
resolveCliImagePaths: (imageList) => this.resolveCliImagePaths(imageList),
},
imageList: params.imageList ?? [],
prompt: params.prompt,
resumeSessionId: session.agentSessionId,
});
const useStdin = spawnPlan.stdinPayload !== undefined;
if (!useStdin && preset.promptMode === 'positional') {
// Positional mode: append prompt as a CLI arg (legacy / non-CC presets).
cliArgs.push(params.prompt);
}
return new Promise<void>((resolve, reject) => {
const cliArgs = spawnPlan.args;
// Fall back to the user's Desktop so the process never inherits
// the Electron parent's cwd (which is `/` when launched from Finder).
@@ -318,45 +577,37 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
stdio: [useStdin ? 'pipe' : 'ignore', 'pipe', 'pipe'],
});
// In stdin mode, write the stream-json message and close stdin.
if (useStdin && stdinPayload && proc.stdin) {
// In stdin mode, write the prepared payload and close stdin.
if (useStdin && spawnPlan.stdinPayload !== undefined && proc.stdin) {
const stdin = proc.stdin as Writable;
stdin.write(stdinPayload + '\n', () => {
stdin.write(spawnPlan.stdinPayload, () => {
stdin.end();
});
}
session.process = proc;
let buffer = '';
const streamProcessor = driver.createStreamProcessor();
// Stream stdout lines as raw events to Renderer
const broadcastParsedOutputs = (parsedOutputs: HeterogeneousAgentParsedOutput[]) => {
for (const parsedOutput of parsedOutputs) {
if (parsedOutput.agentSessionId) {
session.agentSessionId = parsedOutput.agentSessionId;
}
this.broadcast('heteroAgentRawLine', {
line: parsedOutput.payload,
sessionId: session.sessionId,
});
}
};
// Stream stdout events as raw provider payloads to Renderer.
const stdout = proc.stdout as Readable;
stdout.on('data', (chunk: Buffer) => {
buffer += chunk.toString('utf8');
const lines = buffer.split('\n');
buffer = lines.pop() || '';
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
const parsed = JSON.parse(trimmed);
// Extract agent session ID from init event (for multi-turn)
if (parsed.type === 'system' && parsed.subtype === 'init' && parsed.session_id) {
session.agentSessionId = parsed.session_id;
}
// Broadcast raw parsed JSON — Renderer handles all adaptation
this.broadcast('heteroAgentRawLine', {
line: parsed,
sessionId: session.sessionId,
});
} catch {
// Not valid JSON, skip
}
}
broadcastParsedOutputs(streamProcessor.push(chunk));
});
stdout.on('end', () => {
broadcastParsedOutputs(streamProcessor.flush());
});
// Capture stderr
@@ -368,11 +619,12 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
proc.on('error', (err) => {
logger.error('Agent process error:', err);
const sessionError = this.getSessionErrorPayload(err, session);
this.broadcast('heteroAgentSessionError', {
error: err.message,
error: sessionError,
sessionId: session.sessionId,
});
reject(err);
reject(new Error(typeof sessionError === 'string' ? sessionError : sessionError.message));
});
proc.on('exit', (code, signal) => {
@@ -396,11 +648,12 @@ export default class HeterogeneousAgentCtr extends ControllerModule {
} else {
const stderrOutput = stderrChunks.join('').trim();
const errorMsg = stderrOutput || `Agent exited with code ${code}`;
const sessionError = this.getSessionErrorPayload(errorMsg, session);
this.broadcast('heteroAgentSessionError', {
error: errorMsg,
error: sessionError,
sessionId: session.sessionId,
});
reject(new Error(errorMsg));
reject(new Error(typeof sessionError === 'string' ? sessionError : sessionError.message));
}
});
});
@@ -4,12 +4,11 @@ import { isEqual, merge } from 'es-toolkit/compat';
import { defaultProxySettings } from '@/const/store';
import { createLogger } from '@/utils/logger';
import type {
ProxyTestResult} from '../modules/networkProxy';
import type { ProxyTestResult } from '../modules/networkProxy';
import {
ProxyConfigValidator,
ProxyConnectionTester,
ProxyDispatcherManager
ProxyDispatcherManager,
} from '../modules/networkProxy';
import { ControllerModule, IpcMethod } from './index';
@@ -104,7 +103,7 @@ export default class NetworkProxyCtr extends ControllerModule {
} catch (error) {
const errorMessage = error instanceof Error ? error.message : 'Unknown error';
logger.error('Proxy connection test failed:', errorMessage);
throw new Error(`Connection failed: ${errorMessage}`);
throw new Error(`Connection failed: ${errorMessage}`, { cause: error });
}
}
@@ -3,7 +3,7 @@ import type {
ShowDesktopNotificationParams,
} from '@lobechat/electron-client-ipc';
import { app, Notification } from 'electron';
import { linux, macOS, windows } from 'electron-is';
import * as electronIs from 'electron-is';
import { getIpcContext } from '@/utils/ipc';
import { createLogger } from '@/utils/logger';
@@ -20,7 +20,7 @@ export default class NotificationCtr extends ControllerModule {
if (!Notification.isSupported()) return 'denied';
// Keep a stable status string for renderer-side UI mapping.
// Screen3 expects macOS to return 'authorized' when granted.
if (!macOS()) return 'authorized';
if (!electronIs.macOS()) return 'authorized';
// Electron 38 no longer exposes `systemPreferences.getNotificationSettings()` in types,
// and some runtimes don't provide it at all. Use the renderer's Notification.permission
@@ -43,7 +43,7 @@ export default class NotificationCtr extends ControllerModule {
// On macOS, ask permission via Web Notification API first when possible.
// This helps keep `Notification.permission` in sync for subsequent status checks.
if (macOS()) {
if (electronIs.macOS()) {
try {
const mainWindow = this.app.browserManager.getMainWindow().browserWindow;
await mainWindow.webContents.executeJavaScript('Notification.requestPermission()', true);
@@ -83,12 +83,12 @@ export default class NotificationCtr extends ControllerModule {
}
// On macOS, we may need to explicitly request notification permissions
if (macOS()) {
if (electronIs.macOS()) {
logger.debug('macOS detected, notification permissions should be handled by system');
}
// Set app user model ID on Windows
if (windows()) {
if (electronIs.windows()) {
app.setAppUserModelId('com.lobehub.chat');
logger.debug('Set Windows App User Model ID for notifications');
}
@@ -99,7 +99,9 @@ export default class NotificationCtr extends ControllerModule {
}
}
/**
* Show system desktop notification (only when window is hidden)
* Show system desktop notification.
* By default notifications only appear when the main window is hidden or unfocused.
* High-priority callers can pass `force` to surface a banner even while focused.
*/
@IpcMethod()
async showDesktopNotification(
@@ -117,12 +119,16 @@ export default class NotificationCtr extends ControllerModule {
// Check if window is hidden
const isWindowHidden = this.isMainWindowHidden();
if (!isWindowHidden) {
if (!params.force && !isWindowHidden) {
logger.debug('Main window is visible, skipping desktop notification');
return { reason: 'Window is visible', skipped: true, success: true };
}
logger.info('Window is hidden, showing desktop notification:', params.title);
if (params.requestAttention && isWindowHidden) {
this.requestUserAttention();
}
logger.info('Showing desktop notification:', params.title);
const notification = new Notification({
body: params.body,
@@ -136,7 +142,7 @@ export default class NotificationCtr extends ControllerModule {
// due to heavy gnome-shell processing. Using 'low' urgency routes notifications to the
// message tray instead, preventing the banner's X button from being shown.
// The urgency option is ignored on macOS and Windows.
urgency: linux() ? 'low' : 'normal',
urgency: electronIs.linux() ? 'low' : 'normal',
});
// Add more event listeners for debugging
@@ -178,6 +184,23 @@ export default class NotificationCtr extends ControllerModule {
}
}
private requestUserAttention(): void {
try {
const mainWindow = this.app.browserManager.getMainWindow().browserWindow;
if (mainWindow.isDestroyed()) return;
if (electronIs.macOS()) {
app.dock?.bounce?.('informational');
return;
}
mainWindow.flashFrame(true);
} catch (error) {
logger.error('Failed to request user attention:', error);
}
}
/**
* Set the app-level badge count (dock red dot on macOS, Unity counter on Linux,
* overlay icon on Windows). Pass 0 to clear.
@@ -192,7 +215,7 @@ export default class NotificationCtr extends ControllerModule {
try {
const next = Math.max(0, Math.floor(count));
app.setBadgeCount(next);
if (macOS() && app.dock) {
if (electronIs.macOS() && app.dock) {
app.dock.setBadge(next > 0 ? String(next) : '');
}
} catch (error) {
@@ -0,0 +1,72 @@
import type {
CapturePreviewResult,
CaptureRectParams,
OverlayCaptureUploadStatusPayload,
ScreenCaptureSubmitParams,
} from '@lobechat/electron-client-ipc';
import type { OverlaySnapshotPayload } from '@/modules/screenCapture/ScreenCaptureManager';
import { createLogger } from '@/utils/logger';
import { ControllerModule, IpcMethod } from './index';
const logger = createLogger('controllers:ScreenCaptureCtr');
export default class ScreenCaptureCtr extends ControllerModule {
static override readonly groupName = 'screenCapture';
@IpcMethod()
async traceOverlayEvent(payload: { data?: unknown; event: string }): Promise<void> {
console.info('[screenCapture:overlay]', payload.event, payload.data ?? '');
}
@IpcMethod()
async previewWindow(windowId: number): Promise<CapturePreviewResult> {
logger.debug(`previewWindow request: ${windowId}`);
return this.app.screenCaptureManager.handlePreviewWindow(windowId);
}
@IpcMethod()
async previewRect(params: CaptureRectParams): Promise<CapturePreviewResult> {
logger.debug(`previewRect request: ${JSON.stringify(params)}`);
return this.app.screenCaptureManager.handlePreviewRect(params);
}
@IpcMethod()
async submit(params: ScreenCaptureSubmitParams): Promise<void> {
logger.debug(`submit request: prompt-len=${params.prompt.length}`);
await this.app.screenCaptureManager.handleSubmit(params);
}
/**
* Status update reported by the main renderer after it finishes (or fails)
* uploading a capture's bytes. Forwarded to the overlay to drive the send
* button's enabled state.
*/
@IpcMethod()
async reportUploadStatus(payload: OverlayCaptureUploadStatusPayload): Promise<void> {
logger.debug(
`reportUploadStatus captureId=${payload.captureId} status=${payload.status} fileId=${payload.fileId ?? '-'}`,
);
this.app.screenCaptureManager.reportUploadStatus(payload);
}
@IpcMethod()
async close(): Promise<void> {
logger.debug('close overlay request');
this.app.screenCaptureManager.close();
}
/**
* Renderer-driven snapshot of agents/models for the overlay selector. The
* main renderer pushes this whenever its data layer (TRPC stores) reports
* a change; main process only caches and forwards — it does not fetch.
*/
@IpcMethod()
async publishOverlaySnapshot(payload: OverlaySnapshotPayload): Promise<void> {
logger.debug(
`publishOverlaySnapshot — agents=${payload.agents?.length ?? 0} models=${payload.models?.length ?? 0}`,
);
this.app.screenCaptureManager.publishOverlaySnapshot(payload);
}
}
@@ -2,7 +2,7 @@ import process from 'node:process';
import type { ElectronAppState, ThemeMode } from '@lobechat/electron-client-ipc';
import { app, dialog, nativeTheme, shell } from 'electron';
import { macOS } from 'electron-is';
import * as electronIs from 'electron-is';
import { pathExists, readdir } from 'fs-extra';
import { legacyLocalDbDir } from '@/const/dir';
@@ -103,7 +103,7 @@ export default class SystemController extends ControllerModule {
return 'granted';
}
if (!macOS()) {
if (!electronIs.macOS()) {
logger.info('[FullDiskAccess] Not macOS, returning granted');
return 'granted';
}
@@ -1,14 +1,18 @@
import { exec } from 'node:child_process';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import type { ClaudeAuthStatus } from '@lobechat/electron-client-ipc';
import type {
ClaudeAuthStatus,
DetectHeterogeneousAgentCommandParams,
} from '@lobechat/electron-client-ipc';
import type { ToolCategory, ToolStatus } from '@/core/infrastructure/ToolDetectorManager';
import { detectHeterogeneousCliCommand } from '@/modules/toolDetectors';
import { createLogger } from '@/utils/logger';
import { ControllerModule, IpcMethod } from './index';
const execPromise = promisify(exec);
const execFilePromise = promisify(execFile);
const logger = createLogger('controllers:ToolDetectorCtr');
@@ -34,6 +38,14 @@ export default class ToolDetectorCtr extends ControllerModule {
return this.manager.detect(name, force);
}
@IpcMethod()
async detectHeterogeneousAgentCommand(
params: DetectHeterogeneousAgentCommandParams,
): Promise<ToolStatus> {
logger.debug('Detecting heterogeneous agent command:', params);
return detectHeterogeneousCliCommand(params.agentType, params.command);
}
/**
* Detect all registered tools
*/
@@ -125,9 +137,14 @@ export default class ToolDetectorCtr extends ControllerModule {
* Returns null if the CLI is unavailable or the command fails.
*/
@IpcMethod()
async getClaudeAuthStatus(): Promise<ClaudeAuthStatus | null> {
async getClaudeAuthStatus(command = 'claude'): Promise<ClaudeAuthStatus | null> {
const resolvedCommand = command.trim() || 'claude';
try {
const { stdout } = await execPromise('claude auth status --json', { timeout: 5000 });
const { stdout } = await execFilePromise(resolvedCommand, ['auth', 'status', '--json'], {
timeout: 5000,
windowsHide: true,
});
return JSON.parse(stdout.trim()) as ClaudeAuthStatus;
} catch (error) {
logger.debug('Failed to get claude auth status:', error);
@@ -64,7 +64,7 @@ vi.mock('@/const/env', () => ({
let randomBytesCounter = 0;
vi.mock('node:crypto', () => ({
default: {
randomBytes: vi.fn((size: number) => {
randomBytes: vi.fn((_size: number) => {
randomBytesCounter++;
return {
toString: vi.fn(() => `mock-random-${randomBytesCounter}`),
@@ -30,6 +30,7 @@ const mockMinimizeWindow = vi.fn();
const mockMaximizeWindow = vi.fn();
const mockIsWindowMaximized = vi.fn();
const mockRetrieveByIdentifier = vi.fn();
const mockStartSession = vi.fn();
const testSenderIdentifierString: string = 'test-window-event-id';
const mockGetIdentifierByWebContents = vi.fn(() => testSenderIdentifierString);
@@ -66,6 +67,9 @@ const mockApp = {
},
),
},
screenCaptureManager: {
startSession: mockStartSession,
},
} as unknown as App;
describe('BrowserWindowsCtr', () => {
@@ -78,10 +82,21 @@ describe('BrowserWindowsCtr', () => {
});
describe('toggleMainWindow', () => {
it('should get the main window and toggle its visibility', async () => {
await browserWindowsCtr.toggleMainWindow();
it('should toggle the main window visibility', () => {
browserWindowsCtr.toggleMainWindow();
expect(mockGetMainWindow).toHaveBeenCalled();
expect(mockToggleVisible).toHaveBeenCalled();
expect(mockStartSession).not.toHaveBeenCalled();
});
});
describe('openQuickComposer', () => {
it('should start the quick composer session', async () => {
await browserWindowsCtr.openQuickComposer();
expect(mockStartSession).toHaveBeenCalled();
expect(mockGetMainWindow).not.toHaveBeenCalled();
expect(mockToggleVisible).not.toHaveBeenCalled();
});
});
@@ -4,6 +4,7 @@ import { tmpdir } from 'node:os';
import path from 'node:path';
import { PassThrough } from 'node:stream';
import { HeterogeneousAgentSessionErrorCode } from '@lobechat/electron-client-ipc';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import HeterogeneousAgentCtr from '../HeterogeneousAgentCtr';
@@ -32,18 +33,34 @@ vi.mock('@/utils/logger', () => ({
// Captures the most recent spawn() call so sendPrompt tests can assert on argv.
const spawnCalls: Array<{ args: string[]; command: string; options: any }> = [];
let nextFakeProc: any = null;
vi.mock('node:child_process', () => ({
spawn: (command: string, args: string[], options: any) => {
spawnCalls.push({ args, command, options });
return nextFakeProc;
},
const { execFileMock } = vi.hoisted(() => ({
execFileMock: vi.fn(),
}));
vi.mock('node:child_process', async (importOriginal) => {
const actual = (await importOriginal()) as Record<string, unknown>;
return {
...actual,
execFile: execFileMock,
spawn: (command: string, args: string[], options: any) => {
spawnCalls.push({ args, command, options });
nextFakeProc?.__start?.();
return nextFakeProc;
},
};
});
/**
* Build a fake ChildProcess that immediately exits cleanly. Records every
* stdin write on the returned `writes` array so tests can inspect the payload.
*/
const createFakeProc = () => {
const createFakeProc = ({
exitCode = 0,
stdoutLines = [],
}: {
exitCode?: number;
stdoutLines?: string[];
} = {}) => {
const proc = new EventEmitter() as any;
const stdout = new PassThrough();
const stderr = new PassThrough();
@@ -60,15 +77,26 @@ const createFakeProc = () => {
};
proc.kill = vi.fn();
proc.killed = false;
// Exit asynchronously so the Promise returned by sendPrompt resolves cleanly.
setImmediate(() => {
stdout.end();
stderr.end();
proc.emit('exit', 0);
});
let started = false;
proc.__start = () => {
if (started) return;
started = true;
// Exit asynchronously so the Promise returned by sendPrompt resolves cleanly.
setImmediate(() => {
for (const line of stdoutLines) {
stdout.write(line);
}
stdout.end();
stderr.end();
proc.emit('exit', exitCode);
});
};
return { proc, writes };
};
const getFlagValues = (args: string[], flag: string) =>
args.flatMap((arg, index) => (arg === flag ? [args[index + 1]] : []));
describe('HeterogeneousAgentCtr', () => {
let appStoragePath: string;
@@ -144,10 +172,15 @@ describe('HeterogeneousAgentCtr', () => {
describe('sendPrompt (claude-code)', () => {
beforeEach(() => {
spawnCalls.length = 0;
execFileMock.mockReset();
});
const runSendPrompt = async (prompt: string, sessionOverrides: Record<string, any> = {}) => {
const { proc, writes } = createFakeProc();
const runSendPrompt = async (
prompt: string,
sessionOverrides: Record<string, any> = {},
stdoutLines: string[] = [],
) => {
const { proc, writes } = createFakeProc({ stdoutLines });
nextFakeProc = proc;
const ctr = new HeterogeneousAgentCtr({
@@ -162,7 +195,7 @@ describe('HeterogeneousAgentCtr', () => {
await ctr.sendPrompt({ prompt, sessionId });
const { args: cliArgs, command, options } = spawnCalls[0];
return { cliArgs, command, options, writes };
return { cliArgs, command, ctr, options, sessionId, writes };
};
it('passes prompt via stdin stream-json — never as a positional arg', async () => {
@@ -221,5 +254,258 @@ describe('HeterogeneousAgentCtr', () => {
expect(options.cwd).toBe(explicitCwd);
});
it('captures the Claude Code session id from stream-json init events', async () => {
const { ctr, sessionId } = await runSendPrompt('hello', {}, [
`${JSON.stringify({ session_id: 'sess_cc_123', subtype: 'init', type: 'system' })}\n`,
]);
await expect(ctr.getSessionInfo({ sessionId })).resolves.toEqual({
agentSessionId: 'sess_cc_123',
});
});
});
describe('sendPrompt (codex)', () => {
beforeEach(() => {
spawnCalls.length = 0;
execFileMock.mockReset();
});
const runSendPrompt = async (
prompt: string,
sessionOverrides: Record<string, any> = {},
stdoutLines: string[] = [],
sendPromptOverrides: Partial<{ imageList: Array<{ id: string; url: string }> }> = {},
) => {
const { proc, writes } = createFakeProc({ stdoutLines });
nextFakeProc = proc;
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
} as any);
const { sessionId } = await ctr.startSession({
agentType: 'codex',
command: 'codex',
...sessionOverrides,
});
await ctr.sendPrompt({ prompt, sessionId, ...sendPromptOverrides });
const { args: cliArgs, command, options } = spawnCalls[0];
return { cliArgs, command, ctr, options, sessionId, writes };
};
it('fails fast when Codex CLI is unavailable instead of attempting spawn', async () => {
const detect = vi.fn().mockResolvedValue({ available: false });
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
toolDetectorManager: { detect },
} as any);
const { sessionId } = await ctr.startSession({
agentType: 'codex',
command: 'codex',
});
await expect(ctr.sendPrompt({ prompt: 'hello', sessionId })).rejects.toThrow(
'Codex CLI was not found',
);
expect(detect).toHaveBeenCalledWith('codex', true);
expect(spawnCalls).toHaveLength(0);
});
it('fails fast when Claude Code CLI is unavailable instead of attempting spawn', async () => {
const detect = vi.fn().mockResolvedValue({ available: false });
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
toolDetectorManager: { detect },
} as any);
const { sessionId } = await ctr.startSession({
agentType: 'claude-code',
command: 'claude',
});
await expect(ctr.sendPrompt({ prompt: 'hello', sessionId })).rejects.toThrow(
'Claude Code CLI was not found',
);
expect(detect).toHaveBeenCalledWith('claude', true);
expect(spawnCalls).toHaveLength(0);
});
it('fails fast when a customized Claude command is unavailable instead of checking the default detector', async () => {
execFileMock.mockImplementation(
(
file: string,
_args: string[],
optionsOrCallback: unknown,
callback?: (error: Error | null, stdout: string, stderr: string) => void,
) => {
const resolvedCallback =
typeof optionsOrCallback === 'function' ? optionsOrCallback : callback;
resolvedCallback?.(
Object.assign(new Error(`${file} not found`), { code: 'ENOENT' }),
'',
'',
);
},
);
const detect = vi.fn().mockResolvedValue({ available: true });
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
toolDetectorManager: { detect },
} as any);
const { sessionId } = await ctr.startSession({
agentType: 'claude-code',
command: 'claude-alt',
});
await expect(ctr.sendPrompt({ prompt: 'hello', sessionId })).rejects.toThrow(
'Claude Code CLI was not found',
);
expect(detect).not.toHaveBeenCalled();
expect(spawnCalls).toHaveLength(0);
});
it('passes prompt via stdin to codex exec instead of argv', async () => {
const prompt = '--run a shell-like prompt safely';
const { cliArgs, command, writes } = await runSendPrompt(prompt);
expect(command).toBe('codex');
expect(cliArgs).not.toContain(prompt);
expect(cliArgs).toEqual(
expect.arrayContaining(['exec', '--json', '--skip-git-repo-check', '--full-auto', '-']),
);
expect(writes).toEqual([prompt]);
});
it('materializes image attachments into local files and forwards them via --image', async () => {
const imageList = [
{ id: 'image-1', url: 'data:image/png;base64,UE5HX1RFU1Q=' },
{ id: 'image-2', url: 'data:image/jpeg;base64,SlBFR19URVNU' },
];
const { cliArgs, writes } = await runSendPrompt('describe these screenshots', {}, [], {
imageList,
});
const imagePaths = getFlagValues(cliArgs, '--image');
expect(cliArgs).not.toContain('describe these screenshots');
expect(cliArgs.filter((arg) => arg === '--image')).toHaveLength(2);
expect(imagePaths).toHaveLength(2);
expect(imagePaths[0]).toMatch(/\.png$/);
expect(imagePaths[1]).toMatch(/\.jpg$/);
expect(
imagePaths.every((filePath) =>
filePath.startsWith(path.join(appStoragePath, 'heteroAgent/files')),
),
).toBe(true);
await expect(
Promise.all(imagePaths.map((filePath) => readFile(filePath, 'utf8'))),
).resolves.toEqual(['PNG_TEST', 'JPEG_TEST']);
expect(writes).toEqual(['describe these screenshots']);
});
it('skips images that fail to materialize and still forwards the remaining --image args', async () => {
const imageList = [
{ id: 'good-image', url: 'data:image/png;base64,VkFMSURfSU1BR0U=' },
{ id: 'bad-image', url: 'bad://broken-image' },
];
const { cliArgs, writes } = await runSendPrompt('inspect the valid screenshot only', {}, [], {
imageList,
});
const imagePaths = getFlagValues(cliArgs, '--image');
expect(cliArgs.filter((arg) => arg === '--image')).toHaveLength(1);
expect(imagePaths).toHaveLength(1);
expect(imagePaths[0]).toMatch(/\.png$/);
await expect(readFile(imagePaths[0], 'utf8')).resolves.toBe('VALID_IMAGE');
expect(writes).toEqual(['inspect the valid screenshot only']);
});
it('uses codex exec resume syntax when continuing an existing thread', async () => {
const { cliArgs } = await runSendPrompt('continue', { resumeSessionId: 'thread_abc' });
expect(cliArgs.slice(0, 2)).toEqual(['exec', 'resume']);
expect(cliArgs).toContain('thread_abc');
expect(cliArgs).not.toContain('--resume');
expect(cliArgs.at(-1)).toBe('-');
});
it('captures the Codex thread id from json output for later resume', async () => {
const { ctr, sessionId } = await runSendPrompt('hello', {}, [
`${JSON.stringify({ thread_id: 'thread_codex_123', type: 'thread.started' })}\n`,
]);
await expect(ctr.getSessionInfo({ sessionId })).resolves.toEqual({
agentSessionId: 'thread_codex_123',
});
});
it('classifies stale Codex resume stderr as a structured resume error', () => {
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
} as any);
const payload = (ctr as any).getSessionErrorPayload(
'No conversation found for thread thread_stale_123',
{
agentSessionId: 'thread_stale_123',
agentType: 'codex',
args: [],
command: 'codex',
cwd: '/Users/fake/projects/repo',
resumeSessionId: 'thread_stale_123',
sessionId: 'session-1',
},
);
expect(payload).toEqual({
agentType: 'codex',
code: HeterogeneousAgentSessionErrorCode.ResumeThreadNotFound,
command: 'codex',
message: 'The saved Codex thread could not be found, so it can no longer be resumed.',
resumeSessionId: 'thread_stale_123',
stderr: 'No conversation found for thread thread_stale_123',
workingDirectory: '/Users/fake/projects/repo',
});
});
it('classifies CLI authentication failures as auth-required errors', () => {
const ctr = new HeterogeneousAgentCtr({
appStoragePath,
storeManager: { get: vi.fn() },
} as any);
const payload = (ctr as any).getSessionErrorPayload(
'Failed to authenticate. API Error: 401 {"type":"error","error":{"type":"authentication_error","message":"Invalid authentication credentials"}}',
{
agentType: 'claude-code',
args: [],
command: 'claude',
sessionId: 'session-1',
},
);
expect(payload).toEqual({
agentType: 'claude-code',
code: HeterogeneousAgentSessionErrorCode.AuthRequired,
command: 'claude',
docsUrl: 'https://docs.anthropic.com/en/docs/claude-code/setup',
message:
'Claude Code could not authenticate. Sign in again or refresh its credentials, then retry.',
stderr:
'Failed to authenticate. API Error: 401 {"type":"error","error":{"type":"authentication_error","message":"Invalid authentication credentials"}}',
});
});
});
});
@@ -34,6 +34,9 @@ vi.mock('electron', () => {
},
Notification: MockNotification,
app: {
dock: {
bounce: vi.fn(),
},
setAppUserModelId: vi.fn(),
},
};
@@ -48,6 +51,7 @@ vi.mock('electron-is', () => ({
// Mock browserManager
const mockBrowserWindow = {
flashFrame: vi.fn(),
focus: vi.fn(),
isDestroyed: vi.fn(() => false),
isFocused: vi.fn(() => true),
@@ -181,6 +185,24 @@ describe('NotificationCtr', () => {
expect(result).toEqual({ success: true });
});
it('should show notification when force is true even if window is visible and focused', async () => {
const { Notification } = await import('electron');
vi.mocked(Notification.isSupported).mockReturnValue(true);
mockBrowserWindow.isVisible.mockReturnValue(true);
mockBrowserWindow.isFocused.mockReturnValue(true);
mockBrowserWindow.isMinimized.mockReturnValue(false);
const promise = controller.showDesktopNotification({
...params,
force: true,
});
vi.advanceTimersByTime(100);
const result = await promise;
expect(Notification).toHaveBeenCalled();
expect(result).toEqual({ success: true });
});
it('should use low urgency on Linux to prevent GNOME Shell freeze', async () => {
const { linux } = await import('electron-is');
const { Notification } = await import('electron');
@@ -252,6 +274,40 @@ describe('NotificationCtr', () => {
);
});
it('should request window attention when requested and window is hidden', async () => {
const { Notification } = await import('electron');
vi.mocked(Notification.isSupported).mockReturnValue(true);
mockBrowserWindow.isVisible.mockReturnValue(false);
const promise = controller.showDesktopNotification({
...params,
requestAttention: true,
});
vi.advanceTimersByTime(100);
await promise;
expect(mockBrowserWindow.flashFrame).toHaveBeenCalledWith(true);
});
it('should bounce dock on macOS when attention is requested', async () => {
const { app, Notification } = await import('electron');
const { macOS } = await import('electron-is');
vi.mocked(macOS).mockReturnValue(true);
vi.mocked(Notification.isSupported).mockReturnValue(true);
mockBrowserWindow.isVisible.mockReturnValue(false);
const promise = controller.showDesktopNotification({
...params,
requestAttention: true,
});
vi.advanceTimersByTime(100);
await promise;
expect(app.dock.bounce).toHaveBeenCalledWith('informational');
vi.mocked(macOS).mockReturnValue(false);
});
it('should register click handler to show main window', async () => {
const { Notification } = await import('electron');
vi.mocked(Notification.isSupported).mockReturnValue(true);
@@ -19,7 +19,7 @@ const mockGetShortcutsConfig = vi.fn().mockReturnValue({
toggleMainWindow: 'CommandOrControl+Shift+L',
openSettings: 'CommandOrControl+,',
});
const mockUpdateShortcutConfig = vi.fn().mockImplementation((id, accelerator) => {
const mockUpdateShortcutConfig = vi.fn().mockImplementation((_id, _accelerator) => {
// Simply mock a successful update
return true;
});
@@ -43,14 +43,14 @@ const mockApp = {
} as unknown as App;
describe('UploadFileCtr', () => {
let controller: UploadFileCtr;
let _controller: UploadFileCtr;
beforeEach(() => {
vi.clearAllMocks();
ipcHandlers.clear();
ipcMainHandleMock.mockClear();
(IpcHandler.getInstance() as any).registeredChannels?.clear();
controller = new UploadFileCtr(mockApp);
_controller = new UploadFileCtr(mockApp);
});
describe('uploadFile', () => {
+3 -1
View File
@@ -16,7 +16,9 @@ const shortcutDecorator = (name: string) => (target: any, methodName: string, de
/**
* shortcut inject decorator
*/
export const shortcut = (method: DesktopHotkeyId) => shortcutDecorator(method);
type DesktopHotkeyIdCompatible = DesktopHotkeyId | 'quickComposer';
export const shortcut = (method: DesktopHotkeyIdCompatible) => shortcutDecorator(method);
const protocolDecorator =
(urlType: string, action: string) => (target: any, methodName: string, descriptor?: any) => {
@@ -15,6 +15,7 @@ import NetworkProxyCtr from './NetworkProxyCtr';
import NotificationCtr from './NotificationCtr';
import RemoteServerConfigCtr from './RemoteServerConfigCtr';
import RemoteServerSyncCtr from './RemoteServerSyncCtr';
import ScreenCaptureCtr from './ScreenCaptureCtr';
import ShellCommandCtr from './ShellCommandCtr';
import ShortcutController from './ShortcutCtr';
import SystemController from './SystemCtr';
@@ -39,6 +40,7 @@ export const controllerIpcConstructors = [
NotificationCtr,
RemoteServerConfigCtr,
RemoteServerSyncCtr,
ScreenCaptureCtr,
ShellCommandCtr,
ShortcutController,
SystemController,
+10 -9
View File
@@ -1,11 +1,11 @@
import os from 'node:os';
import { join } from 'node:path';
import path from 'node:path';
import type { ElectronIPCEventHandler } from '@lobechat/electron-server-ipc';
import { ElectronIPCServer } from '@lobechat/electron-server-ipc';
import { app, nativeTheme, protocol } from 'electron';
import installExtension, { REACT_DEVELOPER_TOOLS } from 'electron-devtools-installer';
import { macOS, windows } from 'electron-is';
import * as electronIs from 'electron-is';
import { name } from '@/../../package.json';
import { binDir, buildDir } from '@/const/dir';
@@ -14,6 +14,7 @@ import { ELECTRON_BE_PROTOCOL_SCHEME } from '@/const/protocol';
import type { IControlModule } from '@/controllers';
import AuthCtr from '@/controllers/AuthCtr';
import { generateCliWrapper, getCliWrapperDir } from '@/modules/cliEmbedding';
import { ScreenCaptureManager } from '@/modules/screenCapture/ScreenCaptureManager';
import {
astSearchDetectors,
browserAutomationDetectors,
@@ -62,6 +63,7 @@ export class App {
protocolManager: ProtocolManager;
rendererUrlManager: RendererUrlManager;
toolDetectorManager: ToolDetectorManager;
screenCaptureManager: ScreenCaptureManager;
chromeFlags: string[] = ['OverlayScrollbar', 'FluentOverlayScrollbar', 'FluentScrollbar'];
/**
@@ -141,6 +143,7 @@ export class App {
this.staticFileServerManager = new StaticFileServerManager(this);
this.protocolManager = new ProtocolManager(this);
this.toolDetectorManager = new ToolDetectorManager(this);
this.screenCaptureManager = new ScreenCaptureManager(this);
// Register built-in tool detectors
this.registerBuiltinToolDetectors();
@@ -246,10 +249,8 @@ export class App {
await this.browserManager.initializeBrowsers();
// Initialize tray manager
if (process.platform === 'win32') {
this.trayManager.initializeTrays();
}
// Initialize tray manager on all platforms (macOS menu bar, Windows / Linux tray).
this.trayManager.initializeTrays();
// Initialize updater manager
await this.updaterManager.initialize();
@@ -258,7 +259,7 @@ export class App {
this.isQuiting = false;
app.on('window-all-closed', () => {
if (windows() || process.platform === 'linux') {
if (electronIs.windows() || process.platform === 'linux') {
logger.info(`All windows closed, quitting application (${process.platform})`);
app.quit();
}
@@ -420,8 +421,8 @@ export class App {
logger.debug('Setting up dev branding');
app.setName('lobehub-desktop-dev');
if (macOS()) {
app.dock!.setIcon(join(buildDir, 'icon-dev.png'));
if (electronIs.macOS()) {
app.dock!.setIcon(path.join(buildDir, 'icon-dev.png'));
}
};
+19 -7
View File
@@ -1,5 +1,5 @@
import console from 'node:console';
import { join } from 'node:path';
import path from 'node:path';
import { APP_WINDOW_MIN_SIZE } from '@lobechat/desktop-bridge';
import type { MainBroadcastEventKey, MainBroadcastParams } from '@lobechat/electron-client-ipc';
@@ -139,7 +139,7 @@ export default class Browser {
webPreferences: {
backgroundThrottling: false,
contextIsolation: true,
preload: join(preloadDir, 'index.js'),
preload: path.join(preloadDir, 'index.js'),
sandbox: false,
webviewTag: true,
},
@@ -238,7 +238,7 @@ export default class Browser {
logger.debug(`[${this.identifier}] Window 'ready-to-show' event fired.`);
if (this.options.showOnInit) {
logger.debug(`Showing window ${this.identifier} because showOnInit is true.`);
browserWindow.show();
this.show();
} else {
logger.debug(`Window ${this.identifier} not shown because showOnInit is false.`);
}
@@ -296,6 +296,7 @@ export default class Browser {
show(): void {
logger.debug(`Showing window: ${this.identifier}`);
this.ensureForegroundAppOnMac();
if (!this._browserWindow?.isDestroyed()) {
this.determineWindowPosition();
}
@@ -328,7 +329,7 @@ export default class Browser {
if (this._browserWindow?.isVisible() && this._browserWindow.isFocused()) {
this.hide();
} else {
this._browserWindow?.show();
this.show();
this._browserWindow?.focus();
}
}
@@ -387,11 +388,22 @@ export default class Browser {
this._browserWindow!.setPosition(newX, newY, false);
}
private ensureForegroundAppOnMac(): void {
if (!isMac || this.identifier !== 'app') return;
try {
app.setActivationPolicy('regular');
app.dock?.show();
} catch (error) {
logger.warn(`[${this.identifier}] Failed to restore regular activation policy:`, error);
}
}
// ==================== Content Loading ====================
loadPlaceholder = async (): Promise<void> => {
logger.debug(`[${this.identifier}] Loading splash screen placeholder`);
await this._browserWindow!.loadFile(join(resourcesDir, 'splash.html'));
await this._browserWindow!.loadFile(path.join(resourcesDir, 'splash.html'));
logger.debug(`[${this.identifier}] Splash screen placeholder loaded.`);
};
@@ -422,7 +434,7 @@ export default class Browser {
private async handleLoadError(urlWithLocale: string): Promise<void> {
try {
logger.info(`[${this.identifier}] Attempting to load error page...`);
await this._browserWindow!.loadFile(join(resourcesDir, 'error.html'));
await this._browserWindow!.loadFile(path.join(resourcesDir, 'error.html'));
logger.info(`[${this.identifier}] Error page loaded successfully.`);
this.setupRetryHandler(urlWithLocale);
@@ -445,7 +457,7 @@ export default class Browser {
} catch (err: any) {
logger.error(`[${this.identifier}] Retry connection failed:`, err);
try {
await this._browserWindow?.loadFile(join(resourcesDir, 'error.html'));
await this._browserWindow?.loadFile(path.join(resourcesDir, 'error.html'));
} catch (loadErr) {
logger.error(`[${this.identifier}] Failed to reload error page:`, loadErr);
}
@@ -39,8 +39,15 @@ export class BrowserManager {
showMainWindow() {
logger.debug('Showing main window');
const window = this.getMainWindow();
window.show();
const browser = this.getMainWindow();
const window = browser.browserWindow;
if (window.isMinimized()) {
window.restore();
}
browser.show();
window.focus();
}
broadcastToAllWindows = <T extends MainBroadcastEventKey>(
@@ -204,6 +211,23 @@ export class BrowserManager {
return true;
}
/**
* Open (or focus) the single-instance Quick Chat popup.
*
* The window is backed by the `topicPopup` template and the route
* `/popup/agent/inbox`, so it mounts a fresh Inbox conversation with no
* active topic. The first message creates a topic via the normal agent
* flow. The `uniqueId` is fixed repeated invocations focus the existing
* window rather than spawning additional ones.
*/
openQuickChatPopup() {
const uniqueId = 'topicPopup_quick_inbox';
const result = this.createMultiInstanceWindow('topicPopup', '/popup/agent/inbox', uniqueId);
result.browser.show();
result.browser.browserWindow.focus();
return result;
}
private emitTopicPopupsChanged(): void {
this.broadcastToAllWindows('topicPopupsChanged', { popups: this.listTopicPopups() });
}
@@ -1,4 +1,4 @@
import { join } from 'node:path';
import path from 'node:path';
import { TITLE_BAR_HEIGHT } from '@lobechat/desktop-bridge';
import { type BrowserWindow, type BrowserWindowConstructorOptions, nativeTheme } from 'electron';
@@ -118,7 +118,7 @@ export class WindowThemeManager {
private getWindowsConfig(isDarkMode: boolean): WindowsThemeConfig {
return {
backgroundColor: isDarkMode ? BACKGROUND_DARK : BACKGROUND_LIGHT,
icon: isDev ? join(buildDir, 'icon-dev.ico') : undefined,
icon: isDev ? path.join(buildDir, 'icon-dev.ico') : undefined,
titleBarOverlay: this.getWindowsTitleBarOverlay(isDarkMode),
titleBarStyle: 'hidden',
};
@@ -5,12 +5,13 @@ import Browser, { type BrowserWindowOpts } from '../Browser';
// Use vi.hoisted to define mocks before hoisting
const {
mockElectronApp,
mockAppModule,
mockBrowserWindow,
mockNativeTheme,
mockIpcMain,
mockScreen,
MockBrowserWindow,
mockEnv,
} = vi.hoisted(() => {
const mockBrowserWindow = {
center: vi.fn(),
@@ -51,15 +52,24 @@ const {
},
};
const mockElectronApp = {
dock: { setBadge: vi.fn() },
setBadgeCount: vi.fn(),
};
return {
mockAppModule: {
dock: {
setBadge: vi.fn(),
show: vi.fn(),
},
setActivationPolicy: vi.fn(),
setBadgeCount: vi.fn(),
},
MockBrowserWindow: vi.fn().mockImplementation(() => mockBrowserWindow),
mockElectronApp,
mockBrowserWindow,
mockEnv: {
isDev: false,
isLinux: false,
isMac: false,
isMacTahoe: false,
isWindows: true,
},
mockIpcMain: {
handle: vi.fn(),
removeHandler: vi.fn(),
@@ -86,7 +96,7 @@ const {
// Mock electron
vi.mock('electron', () => ({
app: mockElectronApp,
app: mockAppModule,
BrowserWindow: MockBrowserWindow,
ipcMain: mockIpcMain,
nativeTheme: mockNativeTheme,
@@ -111,11 +121,21 @@ vi.mock('@/const/dir', () => ({
}));
vi.mock('@/const/env', () => ({
isDev: false,
isLinux: false,
isMac: false,
isMacTahoe: false,
isWindows: true,
get isDev() {
return mockEnv.isDev;
},
get isLinux() {
return mockEnv.isLinux;
},
get isMac() {
return mockEnv.isMac;
},
get isMacTahoe() {
return mockEnv.isMacTahoe;
},
get isWindows() {
return mockEnv.isWindows;
},
}));
vi.mock('../../../const/theme', () => ({
@@ -158,6 +178,10 @@ describe('Browser', () => {
mockBrowserWindow.loadURL.mockResolvedValue(undefined);
mockBrowserWindow.loadFile.mockResolvedValue(undefined);
mockNativeTheme.shouldUseDarkColors = false;
mockEnv.isLinux = false;
mockEnv.isMac = false;
mockEnv.isMacTahoe = false;
mockEnv.isWindows = true;
// Create mock App
mockStoreManagerGet = vi.fn().mockReturnValue(undefined);
@@ -482,6 +506,19 @@ describe('Browser', () => {
expect(mockBrowserWindow.show).toHaveBeenCalled();
});
it('should restore regular activation policy when showing the main window on macOS', () => {
mockEnv.isMac = true;
mockEnv.isWindows = false;
const mainBrowser = new Browser({ ...defaultOptions, identifier: 'app' }, mockApp);
vi.spyOn(mainBrowser, 'loadUrl').mockResolvedValue(undefined as any);
mainBrowser.show();
expect(mockAppModule.setActivationPolicy).toHaveBeenCalledWith('regular');
expect(mockAppModule.dock.show).toHaveBeenCalled();
});
});
describe('hide', () => {
@@ -6,10 +6,13 @@ import { BrowserManager } from '../BrowserManager';
// Use vi.hoisted to define mocks before hoisting
const { MockBrowser, mockAppBrowsers, mockWindowTemplates } = vi.hoisted(() => {
const createMockBrowserWindow = () => ({
focus: vi.fn(),
isMaximized: vi.fn().mockReturnValue(false),
isMinimized: vi.fn().mockReturnValue(false),
maximize: vi.fn(),
minimize: vi.fn(),
on: vi.fn(),
restore: vi.fn(),
unmaximize: vi.fn(),
webContents: { id: Math.random() },
});
@@ -136,6 +139,16 @@ describe('BrowserManager', () => {
const appBrowser = manager.browsers.get('app');
expect(appBrowser?.show).toHaveBeenCalled();
expect(appBrowser?.browserWindow.focus).toHaveBeenCalled();
});
it('should restore a minimized main window before showing it', () => {
const appBrowser = manager.getMainWindow();
vi.mocked(appBrowser.browserWindow.isMinimized).mockReturnValue(true);
manager.showMainWindow();
expect(appBrowser.browserWindow.restore).toHaveBeenCalled();
});
});
@@ -1,5 +1,5 @@
import { readFile, stat } from 'node:fs/promises';
import { basename, extname } from 'node:path';
import path from 'node:path';
import { app, protocol } from 'electron';
import { pathExistsSync } from 'fs-extra';
@@ -234,7 +234,7 @@ export class RendererProtocolManager {
private isAssetRequest(pathname: string) {
const normalizedPathname = pathname.endsWith('/') ? pathname.slice(0, -1) : pathname;
const ext = extname(normalizedPathname);
const ext = path.extname(normalizedPathname);
return (
pathname.startsWith('/assets/') ||
@@ -246,6 +246,6 @@ export class RendererProtocolManager {
}
private is404Html(filePath: string) {
return basename(filePath) === '404.html';
return path.basename(filePath) === '404.html';
}
}
@@ -1,4 +1,4 @@
import { extname, join } from 'node:path';
import path from 'node:path';
import { pathExistsSync } from 'fs-extra';
@@ -12,9 +12,10 @@ import { RendererProtocolManager } from './RendererProtocolManager';
const logger = createLogger('core:RendererUrlManager');
// Vite build with root=monorepo preserves input path structure,
// so index.html / popup.html end up under apps/desktop/ in outDir.
const SPA_ENTRY_HTML = join(rendererDir, 'apps', 'desktop', 'index.html');
const POPUP_ENTRY_HTML = join(rendererDir, 'apps', 'desktop', 'popup.html');
// so index.html / overlay.html / popup.html end up under apps/desktop/ in outDir.
const SPA_ENTRY_HTML = path.join(rendererDir, 'apps', 'desktop', 'index.html');
const OVERLAY_ENTRY_HTML = path.join(rendererDir, 'apps', 'desktop', 'overlay.html');
const POPUP_ENTRY_HTML = path.join(rendererDir, 'apps', 'desktop', 'popup.html');
export class RendererUrlManager {
private readonly rendererProtocolManager: RendererProtocolManager;
@@ -62,23 +63,30 @@ export class RendererUrlManager {
*/
buildRendererUrl(path: string): string {
const cleanPath = path.startsWith('/') ? path : `/${path}`;
return `${this.rendererLoadedUrl}${cleanPath}`;
const normalizedBase = this.rendererLoadedUrl.replace(/\/+$/, '');
return `${normalizedBase}${cleanPath}`;
}
/**
* Resolve renderer file path in production.
* Static assets map directly; popup routes go to popup.html, all other
* routes fall back to index.html (SPA).
* Static assets map directly; /overlay routes fall back to overlay.html;
* popup routes go to popup.html; all other routes fall back to index.html (SPA).
*/
resolveRendererFilePath = async (url: URL): Promise<string | null> => {
const pathname = url.pathname;
// Static assets: direct file mapping
if (pathname.startsWith('/assets/') || extname(pathname)) {
const filePath = join(rendererDir, pathname);
if (pathname.startsWith('/assets/') || path.extname(pathname)) {
const filePath = path.join(rendererDir, pathname);
return pathExistsSync(filePath) ? filePath : null;
}
// Overlay entry (separate MPA page)
if (pathname === '/overlay' || pathname === '/overlay.html') {
return OVERLAY_ENTRY_HTML;
}
// Topic popup window has its own SPA bundle.
if (pathname === '/popup' || pathname.startsWith('/popup/')) {
return POPUP_ENTRY_HTML;
@@ -92,6 +92,18 @@ describe('RendererUrlManager', () => {
expect(manager.buildRendererUrl('/settings')).toBe('http://localhost:5173/settings');
});
it('should normalize trailing slashes from ELECTRON_RENDERER_URL', async () => {
mockIsDev = true;
process.env['ELECTRON_RENDERER_URL'] = 'http://localhost:5173/';
const { RendererUrlManager } = await import('../RendererUrlManager');
const manager = new RendererUrlManager();
manager.configureRendererLoader();
expect(manager.buildRendererUrl('/')).toBe('http://localhost:5173/');
expect(manager.buildRendererUrl('/overlay')).toBe('http://localhost:5173/overlay');
});
it('should fall back to protocol handler when ELECTRON_RENDERER_URL is not set', async () => {
mockIsDev = true;
+58 -26
View File
@@ -1,15 +1,12 @@
import { join } from 'node:path';
import path from 'node:path';
import type { MainBroadcastEventKey, MainBroadcastParams } from '@lobechat/electron-client-ipc';
import type {
DisplayBalloonOptions,
MenuItemConstructorOptions} from 'electron';
import {
app,
Menu,
nativeImage,
Tray as ElectronTray,
Menu as ElectronMenu,
MenuItemConstructorOptions,
} from 'electron';
import { app, Menu, nativeImage, Tray as ElectronTray } from 'electron';
import { resourcesDir } from '@/const/dir';
import { createLogger } from '@/utils/logger';
@@ -30,6 +27,12 @@ export interface TrayOptions {
*/
identifier: string;
/**
* Mark the icon as a macOS template image (black + alpha). macOS will
* then tint it to match the menu bar appearance automatically.
*/
isTemplateImage?: boolean;
/**
* Tray tooltip text
*/
@@ -44,6 +47,13 @@ export class Tray {
*/
private _tray?: ElectronTray;
/**
* Current context menu. We keep this in-house and pop it up manually on
* right-click so that macOS does not swallow the left-click (which would
* happen automatically if we called `_tray.setContextMenu(menu)`).
*/
private _contextMenu?: ElectronMenu;
/**
* Identifier
*/
@@ -87,15 +97,16 @@ export class Tray {
return this._tray;
}
const { iconPath, tooltip } = this.options;
const { iconPath, isTemplateImage, tooltip } = this.options;
// Load tray icon
logger.info(`Creating new tray instance: ${this.identifier}`);
const iconFile = join(resourcesDir, iconPath);
const iconFile = path.join(resourcesDir, iconPath);
logger.debug(`[${this.identifier}] Loading icon: ${iconFile}`);
try {
const icon = nativeImage.createFromPath(iconFile);
if (isTemplateImage) icon.setTemplateImage(true);
this._tray = new ElectronTray(icon);
// Set tooltip
@@ -107,12 +118,22 @@ export class Tray {
// Set default context menu
this.setContextMenu();
// Set click event
// Left-click: open Quick Composer.
this._tray.on('click', () => {
logger.debug(`[${this.identifier}] Tray clicked`);
this.onClick();
});
// Right-click: pop the stored context menu manually so left-click stays
// free (macOS would auto-open the menu on either button if we called
// `_tray.setContextMenu`).
this._tray.on('right-click', () => {
logger.debug(`[${this.identifier}] Tray right-clicked`);
if (this._contextMenu && this._tray) {
this._tray.popUpContextMenu(this._contextMenu);
}
});
logger.debug(`[${this.identifier}] Tray instance created successfully`);
return this._tray;
} catch (error) {
@@ -148,40 +169,51 @@ export class Tray {
];
const contextMenu = Menu.buildFromTemplate(defaultTemplate);
this._tray?.setContextMenu(contextMenu);
// Store the menu instead of calling `_tray.setContextMenu`. The latter
// makes macOS intercept left-clicks to show the menu, which conflicts
// with our Quick Composer trigger on click.
this._contextMenu = contextMenu;
logger.debug(`[${this.identifier}] Tray context menu has been set`);
}
/**
* Handle tray click event
* Handle tray click event opens the Quick Composer overlay.
* Right-click opens the context menu (handled by Electron automatically).
*/
onClick() {
logger.debug(`[${this.identifier}] Handling tray click event`);
const mainWindow = this.app.browserManager.getMainWindow();
if (mainWindow) {
if (mainWindow.browserWindow.isVisible() && mainWindow.browserWindow.isFocused()) {
logger.debug(`[${this.identifier}] Main window is visible and focused, hiding it now`);
mainWindow.hide();
} else {
logger.debug(`[${this.identifier}] Showing and focusing main window`);
mainWindow.show();
mainWindow.browserWindow.focus();
}
logger.debug(`[${this.identifier}] Tray click → startSession`);
try {
void this.app.screenCaptureManager.startSession();
} catch (error) {
logger.error(`[${this.identifier}] Failed to start capture session:`, error);
}
}
/**
* Replace the tray context menu with a pre-built Electron Menu instance.
* Stored in-house and popped up manually on right-click to preserve
* left-click for the Quick Composer trigger.
*/
setMenu(menu: ElectronMenu) {
logger.debug(`[${this.identifier}] Attaching prebuilt context menu`);
this._contextMenu = menu;
}
/**
* Update tray icon
* @param iconPath New icon path (relative to resource directory)
* @param isTemplateImage Whether to mark the new icon as a macOS template image
*/
updateIcon(iconPath: string) {
updateIcon(iconPath: string, isTemplateImage?: boolean) {
logger.debug(`[${this.identifier}] Updating icon: ${iconPath}`);
try {
const iconFile = join(resourcesDir, iconPath);
const iconFile = path.join(resourcesDir, iconPath);
const icon = nativeImage.createFromPath(iconFile);
const nextIsTemplate = isTemplateImage ?? this.options.isTemplateImage;
if (nextIsTemplate) icon.setTemplateImage(true);
this._tray?.setImage(icon);
this.options.iconPath = iconPath;
if (isTemplateImage !== undefined) this.options.isTemplateImage = isTemplateImage;
logger.debug(`[${this.identifier}] Icon updated successfully`);
} catch (error) {
logger.error(`[${this.identifier}] Failed to update icon:`, error);
+15 -10
View File
@@ -1,5 +1,4 @@
import type { MainBroadcastEventKey, MainBroadcastParams } from '@lobechat/electron-client-ipc';
import { nativeTheme } from 'electron';
import { name } from '@/../../package.json';
import { isMac } from '@/const/env';
@@ -41,7 +40,15 @@ export class TrayManager {
logger.debug('Initialize application tray');
// Initialize main tray
this.initializeMainTray();
const mainTray = this.initializeMainTray();
// Attach the platform-specific context menu built by MenuManager so the
// tray right-click entries stay in sync with the app menu i18n.
try {
mainTray.setMenu(this.app.menuManager.buildTrayMenu());
} catch (error) {
logger.error('Failed to attach tray context menu:', error);
}
}
/**
@@ -52,18 +59,16 @@ export class TrayManager {
}
/**
* Initialize main tray
* Initialize main tray. On macOS we ship a template image (black + alpha)
* so the system recolors it automatically for light / dark menu bars.
*/
initializeMainTray() {
logger.debug('Initialize main tray');
return this.retrieveOrInitialize({
iconPath: isMac
? nativeTheme.shouldUseDarkColorsForSystemIntegratedUI
? 'tray-dark.png'
: 'tray-light.png'
: 'tray.png',
identifier: 'main', // Use app icon, ensure this file exists in resources directory
tooltip: name, // Can use app.getName() or localized string
iconPath: isMac ? 'trayTemplate.png' : 'tray.png',
identifier: 'main',
isTemplateImage: isMac,
tooltip: name,
});
}
@@ -28,6 +28,7 @@ vi.mock('@/utils/logger', () => ({
// Mock desktop global shortcut defaults
vi.mock('@lobechat/const/desktopGlobalShortcuts', () => ({
DEFAULT_ELECTRON_DESKTOP_SHORTCUTS: {
quickComposer: 'Alt+Shift+Space',
showApp: '',
openSettings: 'CommandOrControl+,',
},
@@ -56,8 +57,10 @@ describe('ShortcutManager', () => {
// Mock shortcut method map
mockShortcutMethodMap = new Map();
const quickComposerMethod = vi.fn();
const showAppMethod = vi.fn();
const openSettingsMethod = vi.fn();
mockShortcutMethodMap.set('quickComposer', quickComposerMethod);
mockShortcutMethodMap.set('showApp', showAppMethod);
mockShortcutMethodMap.set('openSettings', openSettingsMethod);
@@ -77,7 +80,8 @@ describe('ShortcutManager', () => {
});
it('should populate shortcuts map from app shortcut method map', () => {
expect(shortcutManager['shortcuts'].size).toBe(2);
expect(shortcutManager['shortcuts'].size).toBe(3);
expect(shortcutManager['shortcuts'].has('quickComposer')).toBe(true);
expect(shortcutManager['shortcuts'].has('showApp')).toBe(true);
expect(shortcutManager['shortcuts'].has('openSettings')).toBe(true);
});
@@ -114,15 +118,17 @@ describe('ShortcutManager', () => {
expect(mockStoreManager.get).toHaveBeenCalledWith('shortcuts');
expect(globalShortcut.unregisterAll).toHaveBeenCalled();
expect(globalShortcut.register).not.toHaveBeenCalledWith('Control+E', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+Shift+Space', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith(
'CommandOrControl+,',
expect.any(Function),
);
expect(globalShortcut.register).not.toHaveBeenCalledWith('', expect.any(Function));
});
it('should handle stored config with filtering', () => {
const storedConfig = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+Shift+P',
invalidKey: 'Ctrl+I', // Should be filtered out
@@ -132,6 +138,7 @@ describe('ShortcutManager', () => {
shortcutManager.initialize();
const config = shortcutManager.getShortcutsConfig();
expect(config.quickComposer).toBe('Alt+Shift+Q');
expect(config.showApp).toBe('Alt+E');
expect(config.openSettings).toBe('Ctrl+Shift+P');
expect(config.invalidKey).toBeUndefined();
@@ -333,6 +340,13 @@ describe('ShortcutManager', () => {
describe('unregisterAll', () => {
it('should unregister all shortcuts', () => {
shortcutManager['shortcutsConfig'] = {
quickComposer: 'Alt+Shift+Space',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
};
shortcutManager['registerConfiguredShortcuts']();
shortcutManager.unregisterAll();
expect(globalShortcut.unregisterAll).toHaveBeenCalled();
@@ -362,6 +376,7 @@ describe('ShortcutManager', () => {
it('should filter invalid keys from stored config', () => {
const storedConfig = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
invalidKey1: 'Ctrl+I',
@@ -372,6 +387,7 @@ describe('ShortcutManager', () => {
shortcutManager['loadShortcutsConfig']();
const config = shortcutManager['shortcutsConfig'];
expect(config.quickComposer).toBe('Alt+Shift+Q');
expect(config.showApp).toBe('Alt+E');
expect(config.openSettings).toBe('Ctrl+P');
expect(config.invalidKey1).toBeUndefined();
@@ -384,19 +400,21 @@ describe('ShortcutManager', () => {
it('should add missing default shortcuts', () => {
const incompleteConfig = {
showApp: 'Alt+E',
// Missing openSettings
// Missing quickComposer and openSettings
};
mockStoreManager.get.mockReturnValue(incompleteConfig);
shortcutManager['loadShortcutsConfig']();
const config = shortcutManager['shortcutsConfig'];
expect(config.quickComposer).toBe('Alt+Shift+Space');
expect(config.showApp).toBe('Alt+E');
expect(config.openSettings).toBe('CommandOrControl+,'); // Default value
});
it('should not save config if no invalid keys were found', () => {
const validConfig = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
};
@@ -425,11 +443,16 @@ describe('ShortcutManager', () => {
describe('saveShortcutsConfig', () => {
it('should save shortcuts config to store', () => {
shortcutManager['shortcutsConfig'] = { showApp: 'Alt+E', openSettings: 'Ctrl+P' };
shortcutManager['shortcutsConfig'] = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
};
shortcutManager['saveShortcutsConfig']();
expect(mockStoreManager.set).toHaveBeenCalledWith('shortcuts', {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
});
@@ -448,6 +471,7 @@ describe('ShortcutManager', () => {
describe('registerConfiguredShortcuts', () => {
beforeEach(() => {
shortcutManager['shortcutsConfig'] = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
};
@@ -459,24 +483,28 @@ describe('ShortcutManager', () => {
shortcutManager['registerConfiguredShortcuts']();
expect(globalShortcut.unregisterAll).toHaveBeenCalled();
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+Shift+Q', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+E', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith('Ctrl+P', expect.any(Function));
});
it('should skip shortcuts not defined in default electron desktop shortcuts', () => {
shortcutManager['shortcutsConfig'] = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
invalidKey: 'Ctrl+I',
};
shortcutManager['registerConfiguredShortcuts']();
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+Shift+Q', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+E', expect.any(Function));
expect(globalShortcut.register).not.toHaveBeenCalledWith('Ctrl+I', expect.any(Function));
});
it('should skip shortcuts with empty accelerator', () => {
shortcutManager['shortcutsConfig'] = {
quickComposer: '',
showApp: '',
openSettings: 'Ctrl+P',
};
@@ -492,12 +520,14 @@ describe('ShortcutManager', () => {
mockShortcutMethodMap.delete('openSettings');
shortcutManager = new ShortcutManager(mockApp);
shortcutManager['shortcutsConfig'] = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+P',
};
shortcutManager['registerConfiguredShortcuts']();
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+Shift+Q', expect.any(Function));
expect(globalShortcut.register).toHaveBeenCalledWith('Alt+E', expect.any(Function));
expect(globalShortcut.register).not.toHaveBeenCalledWith('Ctrl+P', expect.any(Function));
});
@@ -506,6 +536,7 @@ describe('ShortcutManager', () => {
describe('integration tests', () => {
it('should complete full initialization flow', () => {
const storedConfig = {
quickComposer: 'Alt+Shift+Q',
showApp: 'Alt+E',
openSettings: 'Ctrl+Shift+P',
invalidKey: 'Ctrl+I',
@@ -517,11 +548,12 @@ describe('ShortcutManager', () => {
// Should filter config and register valid shortcuts
const config = shortcutManager.getShortcutsConfig();
expect(config.quickComposer).toBe('Alt+Shift+Q');
expect(config.showApp).toBe('Alt+E');
expect(config.openSettings).toBe('Ctrl+Shift+P');
expect(config.invalidKey).toBeUndefined();
expect(globalShortcut.register).toHaveBeenCalledTimes(2);
expect(globalShortcut.register).toHaveBeenCalledTimes(3);
expect(mockStoreManager.set).toHaveBeenCalledWith('shortcuts', config);
});
@@ -1,4 +1,4 @@
import { app, Menu, nativeImage,Tray as ElectronTray } from 'electron';
import { app, Menu, nativeImage, Tray as ElectronTray } from 'electron';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type { App } from '../../App';
@@ -47,6 +47,7 @@ describe('Tray', () => {
mockElectronTray = {
setToolTip: vi.fn(),
setContextMenu: vi.fn(),
popUpContextMenu: vi.fn(),
setImage: vi.fn(),
on: vi.fn(),
destroy: vi.fn(),
@@ -74,11 +75,16 @@ describe('Tray', () => {
showMainWindow: vi.fn(),
getMainWindow: vi.fn(() => mockMainWindow),
},
screenCaptureManager: {
startSession: vi.fn(),
},
} as unknown as App;
// Mock electron constructors
vi.mocked(ElectronTray).mockImplementation(() => mockElectronTray);
vi.mocked(nativeImage.createFromPath).mockReturnValue({} as any);
vi.mocked(nativeImage.createFromPath).mockReturnValue({
setTemplateImage: vi.fn(),
} as any);
vi.mocked(Menu.buildFromTemplate).mockReturnValue({} as any);
});
@@ -168,7 +174,7 @@ describe('Tray', () => {
expect(mockElectronTray.on).toHaveBeenCalledWith('click', expect.any(Function));
});
it('should set default context menu', () => {
it('should build the default context menu and store it in-house', () => {
tray = new Tray(
{
iconPath: 'tray.png',
@@ -178,7 +184,23 @@ describe('Tray', () => {
);
expect(Menu.buildFromTemplate).toHaveBeenCalled();
expect(mockElectronTray.setContextMenu).toHaveBeenCalled();
// We no longer hand the menu to Electron directly; macOS would hijack
// left-click if we did. The menu is popped up manually on right-click.
expect(mockElectronTray.setContextMenu).not.toHaveBeenCalled();
});
it('should register both click and right-click listeners', () => {
tray = new Tray(
{
iconPath: 'tray.png',
identifier: 'test-tray',
},
mockApp,
);
const events = mockElectronTray.on.mock.calls.map((c: any[]) => c[0]);
expect(events).toContain('click');
expect(events).toContain('right-click');
});
it('should handle errors when creating tray', () => {
@@ -221,7 +243,9 @@ describe('Tray', () => {
expect.objectContaining({ label: 'Quit' }),
]),
);
expect(mockElectronTray.setContextMenu).toHaveBeenCalled();
// Menu is stored for manual popup on right-click — never handed to
// `_tray.setContextMenu`, which would steal left-click on macOS.
expect(mockElectronTray.setContextMenu).not.toHaveBeenCalled();
});
it('should set custom context menu when template provided', () => {
@@ -233,7 +257,37 @@ describe('Tray', () => {
tray.setContextMenu(customTemplate);
expect(Menu.buildFromTemplate).toHaveBeenCalledWith(customTemplate);
expect(mockElectronTray.setContextMenu).toHaveBeenCalled();
expect(mockElectronTray.setContextMenu).not.toHaveBeenCalled();
});
it('should pop up the stored menu on right-click', () => {
// beforeEach cleared mocks after constructing the tray, so capture the
// right-click handler from a fresh instance.
const mockTrayForRightClick = {
setToolTip: vi.fn(),
setContextMenu: vi.fn(),
popUpContextMenu: vi.fn(),
setImage: vi.fn(),
on: vi.fn(),
destroy: vi.fn(),
displayBalloon: vi.fn(),
};
vi.mocked(ElectronTray).mockImplementationOnce(() => mockTrayForRightClick as any);
const builtMenu = { _mockMenu: true } as any;
vi.mocked(Menu.buildFromTemplate).mockReturnValue(builtMenu);
const freshTray = new Tray({ iconPath: 'tray.png', identifier: 'rc-tray' }, mockApp);
freshTray.setContextMenu();
const rightClickHandler = mockTrayForRightClick.on.mock.calls.find(
(c: any[]) => c[0] === 'right-click',
)?.[1];
expect(rightClickHandler).toBeDefined();
rightClickHandler?.();
expect(mockTrayForRightClick.popUpContextMenu).toHaveBeenCalledWith(builtMenu);
});
it('should call showMainWindow when Show Main Window is clicked', () => {
@@ -270,40 +324,23 @@ describe('Tray', () => {
);
});
it('should hide window when it is visible and focused', () => {
mockBrowserWindow.isVisible.mockReturnValue(true);
mockBrowserWindow.isFocused.mockReturnValue(true);
it('should start the Quick Composer capture session', () => {
tray.onClick();
expect(mockMainWindow.hide).toHaveBeenCalled();
expect(mockApp.screenCaptureManager.startSession).toHaveBeenCalled();
});
it('should not touch main window visibility', () => {
tray.onClick();
expect(mockMainWindow.hide).not.toHaveBeenCalled();
expect(mockMainWindow.show).not.toHaveBeenCalled();
});
it('should show and focus window when it is not visible', () => {
mockBrowserWindow.isVisible.mockReturnValue(false);
mockBrowserWindow.isFocused.mockReturnValue(false);
tray.onClick();
expect(mockMainWindow.show).toHaveBeenCalled();
expect(mockBrowserWindow.focus).toHaveBeenCalled();
expect(mockMainWindow.hide).not.toHaveBeenCalled();
});
it('should show and focus window when it is visible but not focused', () => {
mockBrowserWindow.isVisible.mockReturnValue(true);
mockBrowserWindow.isFocused.mockReturnValue(false);
tray.onClick();
expect(mockMainWindow.show).toHaveBeenCalled();
expect(mockBrowserWindow.focus).toHaveBeenCalled();
expect(mockMainWindow.hide).not.toHaveBeenCalled();
});
it('should handle case when main window is null', () => {
vi.mocked(mockApp.browserManager.getMainWindow).mockReturnValue(null);
it('should not throw when startSession rejects', () => {
vi.mocked(mockApp.screenCaptureManager.startSession).mockImplementationOnce(() => {
throw new Error('capture failed');
});
expect(() => tray.onClick()).not.toThrow();
});
@@ -504,11 +541,9 @@ describe('Tray', () => {
tray.updateTooltip('New Tooltip');
expect(mockElectronTray.setToolTip).toHaveBeenCalledWith('New Tooltip');
// Test click behavior
mockBrowserWindow.isVisible.mockReturnValue(true);
mockBrowserWindow.isFocused.mockReturnValue(true);
// Test click behavior — now opens the Quick Composer session
tray.onClick();
expect(mockMainWindow.hide).toHaveBeenCalled();
expect(mockApp.screenCaptureManager.startSession).toHaveBeenCalled();
// Destroy
tray.destroy();
@@ -1,16 +1,11 @@
import { nativeTheme } from 'electron';
import { beforeEach, describe, expect, it, vi } from 'vitest';
import type { App } from '../../App';
import { Tray } from '../Tray';
import { TrayManager } from '../TrayManager';
// Mock electron modules
vi.mock('electron', () => ({
nativeTheme: {
shouldUseDarkColorsForSystemIntegratedUI: false,
},
}));
// Mock electron modules (empty shim — TrayManager no longer reads nativeTheme)
vi.mock('electron', () => ({}));
// Mock logger
vi.mock('@/utils/logger', () => ({
@@ -50,12 +45,17 @@ describe('TrayManager', () => {
identifier: 'main',
broadcast: vi.fn(),
destroy: vi.fn(),
setMenu: vi.fn(),
updateIcon: vi.fn(),
updateTooltip: vi.fn(),
};
// Mock App
mockApp = {} as unknown as App;
// Mock App — initializeTrays now pulls a prebuilt menu from MenuManager.
mockApp = {
menuManager: {
buildTrayMenu: vi.fn(() => ({ _mockMenu: true }) as any),
},
} as unknown as App;
// Mock Tray constructor
vi.mocked(Tray).mockImplementation(() => mockTray);
@@ -86,22 +86,24 @@ describe('TrayManager', () => {
expect(spy).toHaveBeenCalled();
});
it('should attach the platform tray menu to the main tray', () => {
trayManager.initializeTrays();
expect(mockApp.menuManager.buildTrayMenu).toHaveBeenCalled();
expect(mockTray.setMenu).toHaveBeenCalledWith({ _mockMenu: true });
});
});
describe('initializeMainTray', () => {
it('should create main tray with dark icon on macOS when dark mode is enabled', () => {
Object.defineProperty(nativeTheme, 'shouldUseDarkColorsForSystemIntegratedUI', {
value: true,
writable: true,
configurable: true,
});
it('should create main tray with a template image on macOS', () => {
const result = trayManager.initializeMainTray();
expect(Tray).toHaveBeenCalledWith(
expect.objectContaining({
iconPath: 'tray-dark.png',
iconPath: 'trayTemplate.png',
identifier: 'main',
isTemplateImage: true,
tooltip: 'test-app',
}),
mockApp,
@@ -109,25 +111,6 @@ describe('TrayManager', () => {
expect(result).toBe(mockTray);
});
it('should create main tray with light icon on macOS when light mode is enabled', () => {
Object.defineProperty(nativeTheme, 'shouldUseDarkColorsForSystemIntegratedUI', {
value: false,
writable: true,
configurable: true,
});
trayManager.initializeMainTray();
expect(Tray).toHaveBeenCalledWith(
expect.objectContaining({
iconPath: 'tray-light.png',
identifier: 'main',
tooltip: 'test-app',
}),
mockApp,
);
});
it('should add created tray to trays map', () => {
trayManager.initializeMainTray();
+1 -3
View File
@@ -1,4 +1,4 @@
import 'vite/client';
/// <reference types="vite/client" />
/**
* `node-mac-permissions` is a macOS-only native module.
@@ -30,5 +30,3 @@ declare module 'node-mac-permissions' {
export function askForScreenCaptureAccess(openPreferences?: boolean): void;
export function askForFullDiskAccess(): void;
}
export {};
@@ -16,6 +16,13 @@ const dialog = {
'fullDiskAccess.openSettings': 'Open Settings',
'fullDiskAccess.skip': 'Later',
'fullDiskAccess.title': 'Full Disk Access Required',
'screenCaptureAccess.cancel': 'Later',
'screenCaptureAccess.detail':
'Open System Settings, enable Screen Recording for LobeHub, then try Quick Composer again.',
'screenCaptureAccess.message':
'Quick Composer needs Screen Recording permission before it can capture screenshots.',
'screenCaptureAccess.openSettings': 'Open Settings',
'screenCaptureAccess.title': 'Screen Recording Permission Required',
'update.downloadAndInstall': 'Download and Install',
'update.downloadComplete': 'Download Complete',
'update.downloadCompleteMessage': 'Update downloaded. Install now?',
@@ -71,7 +71,9 @@ const menu = {
'macOS.preferences': 'Preferences...',
'macOS.services': 'Services',
'macOS.unhide': 'Show All',
'tray.openMiniToolbar': 'Quick Composer',
'tray.open': 'Open {{appName}}',
'tray.quickChat': 'Quick Chat',
'tray.quit': 'Quit',
'tray.show': 'Show {{appName}}',
'view.forceReload': 'Force Reload',
@@ -61,6 +61,7 @@ const createMockApp = () => {
'dev.forceReload': 'Force Reload',
'dev.devTools': 'Developer Tools',
'dev.devPanel': 'Dev Panel',
'tray.openMiniToolbar': 'Quick Composer',
'tray.open': `Open ${params?.appName || 'App'}`,
'tray.quit': 'Quit',
};
@@ -455,6 +455,14 @@ export class LinuxMenu extends BaseMenuPlatform implements IMenuPlatform {
click: () => this.app.browserManager.showMainWindow(),
label: t('tray.open', { appName }),
},
{
click: () => this.app.screenCaptureManager.startSession(),
label: t('tray.openMiniToolbar'),
},
{
click: () => this.app.browserManager.openQuickChatPopup(),
label: t('tray.quickChat'),
},
{ type: 'separator' },
{
click: () => this.app.browserManager.retrieveByIdentifier('settings').show(),
@@ -13,6 +13,9 @@ vi.mock('electron', () => ({
setApplicationMenu: vi.fn(),
},
app: {
dock: {
setMenu: vi.fn(),
},
getAppPath: vi.fn(() => '/mock/app/path'),
getName: vi.fn(() => 'LobeChat'),
getPath: vi.fn((type: string) => {
@@ -63,6 +66,9 @@ const createMockApp = () => {
show: vi.fn(),
})),
},
screenCaptureManager: {
startSession: vi.fn(),
},
updaterManager: {
checkForUpdates: vi.fn(),
getUpdaterState: vi.fn(() => ({ stage: 'idle' })),
@@ -96,6 +102,7 @@ describe('MacOSMenu', () => {
expect(Menu.buildFromTemplate).toHaveBeenCalled();
expect(Menu.setApplicationMenu).toHaveBeenCalled();
expect(app.dock.setMenu).toHaveBeenCalled();
expect(menu).toBeDefined();
});
@@ -172,6 +179,13 @@ describe('MacOSMenu', () => {
expect(template.some((item: any) => item.label?.includes('Show'))).toBe(true);
expect(template.some((item: any) => item.label === 'Quit')).toBe(true);
});
it('should include the mini toolbar action in the dock menu', () => {
macOSMenu.buildAndSetAppMenu();
const dockMenu = (app.dock.setMenu as any).mock.calls[0][0];
expect(dockMenu.template.some((item: any) => item.label === 'Quick Composer')).toBe(true);
});
});
describe('refresh', () => {
@@ -276,6 +290,19 @@ describe('MacOSMenu', () => {
expect(preferencesItem.accelerator).toBe('Command+,');
});
it('should not show a fixed accelerator for Quick Composer', () => {
macOSMenu.buildAndSetAppMenu();
const template = (Menu.buildFromTemplate as any).mock.calls[0][0];
const fileMenu = template.find((item: any) => item.label === 'File');
const quickComposerItem = fileMenu.submenu.find(
(item: any) => item.label === 'Quick Composer',
);
expect(quickComposerItem).toBeDefined();
expect(quickComposerItem.accelerator).toBeUndefined();
});
it('should use role for quit (accelerator handled by Electron)', () => {
macOSMenu.buildAndSetAppMenu();
+39 -1
View File
@@ -1,4 +1,4 @@
import * as path from 'node:path';
import path from 'node:path';
import type { MenuItemConstructorOptions } from 'electron';
import { app, BrowserWindow, clipboard, Menu, shell } from 'electron';
@@ -12,6 +12,7 @@ import { BaseMenuPlatform } from './BaseMenuPlatform';
export class MacOSMenu extends BaseMenuPlatform implements IMenuPlatform {
private appMenu: Menu | null = null;
private dockMenu: Menu | null = null;
private trayMenu: Menu | null = null;
buildAndSetAppMenu(options?: MenuOptions): Menu {
@@ -20,6 +21,7 @@ export class MacOSMenu extends BaseMenuPlatform implements IMenuPlatform {
this.appMenu = Menu.buildFromTemplate(template);
Menu.setApplicationMenu(this.appMenu);
this.buildAndSetDockMenu();
return this.appMenu;
}
@@ -154,6 +156,11 @@ export class MacOSMenu extends BaseMenuPlatform implements IMenuPlatform {
label: t('file.newPage'),
},
{ type: 'separator' },
{
click: () => this.app.screenCaptureManager.startSession(),
label: t('tray.openMiniToolbar'),
},
{ type: 'separator' },
{
accelerator: 'CmdOrCtrl+W',
click: () => {
@@ -673,6 +680,14 @@ export class MacOSMenu extends BaseMenuPlatform implements IMenuPlatform {
click: () => this.app.browserManager.showMainWindow(),
label: t('tray.show', { appName }),
},
{
click: () => this.app.screenCaptureManager.startSession(),
label: t('tray.openMiniToolbar'),
},
{
click: () => this.app.browserManager.openQuickChatPopup(),
label: t('tray.quickChat'),
},
{
click: async () => {
const mainWindow = this.app.browserManager.getMainWindow();
@@ -685,4 +700,27 @@ export class MacOSMenu extends BaseMenuPlatform implements IMenuPlatform {
{ label: t('tray.quit'), role: 'quit' },
];
}
private buildAndSetDockMenu() {
if (!app.dock?.setMenu) return;
this.dockMenu = Menu.buildFromTemplate(this.getDockMenuTemplate());
app.dock.setMenu(this.dockMenu);
}
private getDockMenuTemplate(): MenuItemConstructorOptions[] {
const t = this.app.i18n.ns('menu');
const appName = app.getName();
return [
{
click: () => this.app.browserManager.showMainWindow(),
label: t('tray.show', { appName }),
},
{
click: () => this.app.screenCaptureManager.startSession(),
label: t('tray.openMiniToolbar'),
},
];
}
}
@@ -56,6 +56,7 @@ const createMockApp = () => {
'dev.forceReload': 'Force Reload',
'dev.devTools': 'Developer Tools',
'dev.devPanel': 'Dev Panel',
'tray.openMiniToolbar': 'Quick Composer',
'tray.open': `Open ${params?.appName || 'App'}`,
'tray.quit': 'Quit',
};
@@ -462,6 +462,14 @@ export class WindowsMenu extends BaseMenuPlatform implements IMenuPlatform {
click: () => this.app.browserManager.showMainWindow(),
label: t('tray.open', { appName }),
},
{
click: () => this.app.screenCaptureManager.startSession(),
label: t('tray.openMiniToolbar'),
},
{
click: () => this.app.browserManager.openQuickChatPopup(),
label: t('tray.quickChat'),
},
{ type: 'separator' },
{
click: () => this.app.browserManager.retrieveByIdentifier('settings').show(),
@@ -152,7 +152,7 @@ export abstract class BaseContentSearch {
const regex = new RegExp(pattern, flags);
// Determine files to search
let filesToSearch: string[] = [];
let filesToSearch: string[];
const stats = await stat(searchPath);
if (stats.isFile()) {
@@ -1,5 +1,5 @@
import { stat } from 'node:fs/promises';
import * as path from 'node:path';
import path from 'node:path';
import type { GlobFilesParams, GlobFilesResult } from '@lobechat/electron-client-ipc';
@@ -1,6 +1,6 @@
import { stat } from 'node:fs/promises';
import * as os from 'node:os';
import * as path from 'node:path';
import path from 'node:path';
import { execa } from 'execa';
@@ -369,10 +369,10 @@ export class MacOSSearchServiceImpl extends UnixFileSearch {
continue;
}
const match = line.match(/^(\w+)\s+=\s+(.*)$/);
if (match) {
currentKey = match[1];
const value = match[2].trim();
const keyValue = line.split(/\s+=\s+/, 2);
if (keyValue.length === 2 && /^\w+$/.test(keyValue[0])) {
currentKey = keyValue[0];
const value = keyValue[1].trim();
if (value.includes('(') && !value.includes(')')) {
isMultilineValue = true;
@@ -403,8 +403,7 @@ export class MacOSSearchServiceImpl extends UnixFileSearch {
if (value === 'Yes' || value === 'true') return true;
if (value === 'No' || value === 'false') return false;
const dateMatch = value.match(/^(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} [+-]\d{4})$/);
if (dateMatch) {
if (/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} [+-]\d{4}$/.test(value)) {
try {
return new Date(value);
} catch {
@@ -412,7 +411,7 @@ export class MacOSSearchServiceImpl extends UnixFileSearch {
}
}
if (/^-?\d+(\.\d+)?$/.test(value)) {
if (/^-?\d+(?:\.\d+)?$/.test(value)) {
return Number(value);
}
@@ -1,4 +1,3 @@
import { type Stats } from 'node:fs';
import { stat } from 'node:fs/promises';
import * as os from 'node:os';
@@ -191,7 +190,8 @@ export abstract class UnixFileSearch extends BaseFileSearch {
logger.debug('Performing find search', { keywords: options.keywords, searchDir });
try {
const args: string[] = [searchDir,
const args: string[] = [
searchDir,
'-maxdepth',
'10',
'-type',
@@ -207,7 +207,8 @@ export abstract class UnixFileSearch extends BaseFileSearch {
'*/*cache*/*',
')',
'-prune',
'-o'];
'-o',
];
// Limit depth and exclude common directories
@@ -280,7 +281,7 @@ export abstract class UnixFileSearch extends BaseFileSearch {
return this.processFilePaths(limitedFiles, options, 'fast-glob');
} catch (error) {
logger.error('fast-glob search failed:', error);
throw new Error(`File search failed: ${(error as Error).message}`);
throw new Error(`File search failed: ${(error as Error).message}`, { cause: error });
}
}
@@ -1,4 +1,3 @@
import { type Stats } from 'node:fs';
import { stat } from 'node:fs/promises';
import * as os from 'node:os';
@@ -290,7 +289,7 @@ export class WindowsSearchServiceImpl extends BaseFileSearch {
return this.processFilePaths(limitedFiles, options, 'fast-glob');
} catch (error) {
logger.error('fast-glob search failed:', error);
throw new Error(`File search failed: ${(error as Error).message}`);
throw new Error(`File search failed: ${(error as Error).message}`, { cause: error });
}
}
@@ -0,0 +1,41 @@
import { JsonlStreamProcessor } from '../jsonlProcessor';
import type { HeterogeneousAgentBuildPlanParams, HeterogeneousAgentDriver } from '../types';
const CLAUDE_CODE_BASE_ARGS = [
'-p',
'--input-format',
'stream-json',
'--output-format',
'stream-json',
'--verbose',
'--include-partial-messages',
'--permission-mode',
'bypassPermissions',
] as const;
export const claudeCodeDriver: HeterogeneousAgentDriver = {
async buildSpawnPlan({
args,
helpers,
imageList,
prompt,
resumeSessionId,
}: HeterogeneousAgentBuildPlanParams) {
const stdinPayload = await helpers.buildClaudeStreamJsonInput(prompt, imageList);
return {
args: [
...CLAUDE_CODE_BASE_ARGS,
...(resumeSessionId ? ['--resume', resumeSessionId] : []),
...args,
],
stdinPayload,
};
},
createStreamProcessor() {
return new JsonlStreamProcessor({
extractSessionId: (payload) =>
payload?.type === 'system' && payload?.subtype === 'init' ? payload?.session_id : undefined,
});
},
};
@@ -0,0 +1,50 @@
import { JsonlStreamProcessor } from '../jsonlProcessor';
import type { HeterogeneousAgentBuildPlanParams, HeterogeneousAgentDriver } from '../types';
const CODEX_REQUIRED_ARGS = ['--json', '--skip-git-repo-check'] as const;
const CODEX_AUTO_EXECUTION_FLAGS = [
'--full-auto',
'--dangerously-bypass-approvals-and-sandbox',
'--sandbox',
'-s',
] as const;
const hasAnyFlag = (args: string[], flags: readonly string[]) =>
args.some((arg) => flags.includes(arg as (typeof flags)[number]));
const buildCodexOptionArgs = async ({
args,
helpers,
imageList,
}: Pick<HeterogeneousAgentBuildPlanParams, 'args' | 'helpers' | 'imageList'>) => {
const imagePaths = await helpers.resolveCliImagePaths(imageList);
const imageArgs = imagePaths.flatMap((filePath) => ['--image', filePath]);
const autoExecutionArgs = hasAnyFlag(args, CODEX_AUTO_EXECUTION_FLAGS) ? [] : ['--full-auto'];
return [...CODEX_REQUIRED_ARGS, ...autoExecutionArgs, ...imageArgs, ...args];
};
export const codexDriver: HeterogeneousAgentDriver = {
async buildSpawnPlan({
args,
helpers,
imageList,
prompt,
resumeSessionId,
}: HeterogeneousAgentBuildPlanParams) {
const optionArgs = await buildCodexOptionArgs({ args, helpers, imageList });
return {
args: resumeSessionId
? ['exec', 'resume', ...optionArgs, resumeSessionId, '-']
: ['exec', ...optionArgs, '-'],
stdinPayload: prompt,
};
},
createStreamProcessor() {
return new JsonlStreamProcessor({
extractSessionId: (payload) =>
payload?.type === 'thread.started' ? payload?.thread_id : undefined,
});
},
};
@@ -0,0 +1,18 @@
import { claudeCodeDriver } from './drivers/claudeCode';
import { codexDriver } from './drivers/codex';
import type { HeterogeneousAgentDriver } from './types';
const heterogeneousAgentDrivers: Record<string, HeterogeneousAgentDriver> = {
'claude-code': claudeCodeDriver,
'codex': codexDriver,
};
export const getHeterogeneousAgentDriver = (agentType: string): HeterogeneousAgentDriver => {
const driver = heterogeneousAgentDrivers[agentType];
if (!driver) {
throw new Error(`Unknown heterogeneous agent type: ${agentType}`);
}
return driver;
};
@@ -0,0 +1,61 @@
import type { HeterogeneousAgentParsedOutput, HeterogeneousAgentStreamProcessor } from './types';
export interface JsonlProcessorOptions {
extractSessionId?: (payload: any) => string | undefined;
}
/**
* Parses stdout as JSONL / NDJSON while tolerating non-JSON noise lines.
* Different CLIs still end up sharing this framing logic even when the
* payload schema differs.
*/
export class JsonlStreamProcessor implements HeterogeneousAgentStreamProcessor {
private buffer = '';
constructor(private readonly options: JsonlProcessorOptions = {}) {}
push(chunk: Buffer | string): HeterogeneousAgentParsedOutput[] {
this.buffer += chunk instanceof Buffer ? chunk.toString('utf8') : chunk;
return this.drainCompleteLines();
}
flush(): HeterogeneousAgentParsedOutput[] {
const trailing = this.buffer.trim();
this.buffer = '';
if (!trailing) return [];
try {
return [this.toParsedOutput(JSON.parse(trailing))];
} catch {
return [];
}
}
private drainCompleteLines(): HeterogeneousAgentParsedOutput[] {
const lines = this.buffer.split('\n');
this.buffer = lines.pop() || '';
const parsed: HeterogeneousAgentParsedOutput[] = [];
for (const line of lines) {
const trimmed = line.trim();
if (!trimmed) continue;
try {
parsed.push(this.toParsedOutput(JSON.parse(trimmed)));
} catch {
// Ignore non-JSON stdout noise.
}
}
return parsed;
}
private toParsedOutput(payload: any): HeterogeneousAgentParsedOutput {
return {
agentSessionId: this.options.extractSessionId?.(payload),
payload,
};
}
}
@@ -0,0 +1,42 @@
export interface HeterogeneousAgentImageAttachment {
id: string;
url: string;
}
export interface HeterogeneousAgentBuildPlan {
args: string[];
stdinPayload?: string;
}
export interface HeterogeneousAgentBuildPlanHelpers {
buildClaudeStreamJsonInput: (
prompt: string,
imageList: HeterogeneousAgentImageAttachment[],
) => Promise<string>;
resolveCliImagePaths: (imageList: HeterogeneousAgentImageAttachment[]) => Promise<string[]>;
}
export interface HeterogeneousAgentBuildPlanParams {
args: string[];
helpers: HeterogeneousAgentBuildPlanHelpers;
imageList: HeterogeneousAgentImageAttachment[];
prompt: string;
resumeSessionId?: string;
}
export interface HeterogeneousAgentParsedOutput {
agentSessionId?: string;
payload: any;
}
export interface HeterogeneousAgentStreamProcessor {
flush: () => HeterogeneousAgentParsedOutput[];
push: (chunk: Buffer | string) => HeterogeneousAgentParsedOutput[];
}
export interface HeterogeneousAgentDriver {
buildSpawnPlan: (
params: HeterogeneousAgentBuildPlanParams,
) => Promise<HeterogeneousAgentBuildPlan>;
createStreamProcessor: () => HeterogeneousAgentStreamProcessor;
}
@@ -1,5 +1,5 @@
import type { NetworkProxySettings } from '@lobechat/electron-client-ipc';
import type {SocksProxies } from 'fetch-socks';
import type { SocksProxies } from 'fetch-socks';
import { socksDispatcher } from 'fetch-socks';
import { Agent, getGlobalDispatcher, ProxyAgent, setGlobalDispatcher } from 'undici';
@@ -120,6 +120,7 @@ export class ProxyDispatcherManager {
logger.error(`Failed to create proxy agent for ${proxyType}:`, error);
throw new Error(
`Failed to create proxy agent: ${error instanceof Error ? error.message : 'Unknown error'}`,
{ cause: error },
);
}
}
@@ -71,9 +71,8 @@ export class ProxyConfigValidator {
*/
private static isValidHost(host: string): boolean {
// Simple host validation (IP address or domain name)
const ipRegex = /^(\d{1,3}\.){3}\d{1,3}$/;
const domainRegex =
/^[\dA-Z]([\dA-Z-]*[\dA-Z])?(\.[\dA-Z]([\dA-Z-]*[\dA-Z])?)*$/i;
const ipRegex = /^(?:\d{1,3}\.){3}\d{1,3}$/;
const domainRegex = /^[\dA-Z](?:[\dA-Z-]*[\dA-Z])?(?:\.[\dA-Z](?:[\dA-Z-]*[\dA-Z])?)*$/i;
return ipRegex.test(host) || domainRegex.test(host);
}
@@ -0,0 +1,167 @@
import type { CaptureRectParams } from '@lobechat/electron-client-ipc';
import { Monitor } from 'node-screenshots';
import { createLogger } from '@/utils/logger';
import { findWindowById } from './WindowSourceService';
const logger = createLogger('screenCapture:CaptureService');
const CAPTURE_RETRY_DELAY_MS = 120;
const CAPTURE_RETRY_TIMES = 2;
interface DisplayBounds {
height: number;
width: number;
x: number;
y: number;
}
/**
* Capture a specific window by its native window id.
*/
export async function captureWindow(windowId: number): Promise<Buffer | null> {
try {
const win = findWindowById(windowId);
if (!win) {
logger.warn(`Window ${windowId} not found`);
return null;
}
const image = await win.captureImage();
const pngBuffer = Buffer.from(await image.toPng());
return pngBuffer;
} catch (error) {
logger.error('Failed to capture window:', error);
return null;
}
}
/**
* Capture a rect region from the monitor that contains the rect.
* `absoluteRect` is in absolute DIP coordinates.
*/
export async function captureRect(
absoluteRect: CaptureRectParams,
scaleFactor: number,
displayBounds?: DisplayBounds,
): Promise<Buffer | null> {
try {
const centerX = Math.round((absoluteRect.x + absoluteRect.width / 2) * scaleFactor);
const centerY = Math.round((absoluteRect.y + absoluteRect.height / 2) * scaleFactor);
const monitor = resolveMonitor({
centerX,
centerY,
displayBounds,
scaleFactor,
});
if (!monitor) {
logger.warn(`No monitor found at point (${centerX}, ${centerY})`);
return null;
}
const image = await captureMonitorImageWithRetry(monitor);
if (!image) {
return null;
}
const physX = Math.round(absoluteRect.x * scaleFactor) - monitor.x();
const physY = Math.round(absoluteRect.y * scaleFactor) - monitor.y();
const physW = Math.round(absoluteRect.width * scaleFactor);
const physH = Math.round(absoluteRect.height * scaleFactor);
const cropX = Math.max(0, physX);
const cropY = Math.max(0, physY);
const cropW = Math.min(physW, image.width - cropX);
const cropH = Math.min(physH, image.height - cropY);
if (cropW <= 0 || cropH <= 0) {
logger.warn(`Crop rect out of monitor bounds: ${cropX},${cropY} ${cropW}x${cropH}`);
return null;
}
const cropped = await image.crop(cropX, cropY, cropW, cropH);
const pngBuffer = Buffer.from(await cropped.toPng());
return pngBuffer;
} catch (error) {
logger.error('Failed to capture rect:', error);
return null;
}
}
async function captureMonitorImageWithRetry(
monitor: Monitor,
): Promise<Awaited<ReturnType<Monitor['captureImage']>> | null> {
for (let attempt = 1; attempt <= CAPTURE_RETRY_TIMES; attempt += 1) {
try {
const image = await monitor.captureImage();
return image;
} catch (error) {
logger.error(`captureImage failed on attempt ${attempt} for monitor ${monitor.id()}:`, error);
if (attempt < CAPTURE_RETRY_TIMES) {
await delay(CAPTURE_RETRY_DELAY_MS);
continue;
}
}
}
return null;
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function resolveMonitor({
centerX,
centerY,
displayBounds,
scaleFactor,
}: {
centerX: number;
centerY: number;
displayBounds?: DisplayBounds;
scaleFactor: number;
}): Monitor | null {
const monitors = Monitor.all();
const displayMonitor = displayBounds
? findMonitorByDisplayBounds(monitors, displayBounds, scaleFactor)
: null;
if (displayMonitor) {
return displayMonitor;
}
return Monitor.fromPoint(centerX, centerY);
}
function findMonitorByDisplayBounds(
monitors: Monitor[],
displayBounds: DisplayBounds,
scaleFactor: number,
): Monitor | null {
const expected = {
height: Math.round(displayBounds.height * scaleFactor),
width: Math.round(displayBounds.width * scaleFactor),
x: Math.round(displayBounds.x * scaleFactor),
y: Math.round(displayBounds.y * scaleFactor),
};
let bestMonitor: Monitor | null = null;
let bestScore = Number.POSITIVE_INFINITY;
for (const monitor of monitors) {
const score =
Math.abs(monitor.x() - expected.x) +
Math.abs(monitor.y() - expected.y) +
Math.abs(monitor.width() - expected.width) +
Math.abs(monitor.height() - expected.height);
if (score < bestScore) {
bestMonitor = monitor;
bestScore = score;
}
}
return bestMonitor;
}
@@ -0,0 +1,341 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { ScreenCaptureManager } from './ScreenCaptureManager';
const {
mockBrowserWindow,
MockBrowserWindow,
mockDialogShowMessageBox,
mockScreen,
mockEnumerateWindows,
mockIsMac,
mockCaptureWindow,
mockCaptureRect,
mockGetScreenCaptureStatus,
mockRequestScreenCaptureAccess,
} = vi.hoisted(() => {
const mockBrowserWindow = {
destroy: vi.fn(),
focus: vi.fn(),
hide: vi.fn(),
isDestroyed: vi.fn().mockReturnValue(false),
loadURL: vi.fn().mockResolvedValue(undefined),
moveTop: vi.fn(),
setAlwaysOnTop: vi.fn(),
setHiddenInMissionControl: vi.fn(),
setOpacity: vi.fn(),
setVisibleOnAllWorkspaces: vi.fn(),
show: vi.fn(),
webContents: {
on: vi.fn(),
once: vi.fn((_event, listener) => {
listener();
}),
send: vi.fn(),
},
};
return {
mockBrowserWindow,
MockBrowserWindow: vi.fn(() => mockBrowserWindow),
mockCaptureRect: vi.fn(),
mockCaptureWindow: vi.fn(),
mockDialogShowMessageBox: vi.fn(async () => ({ response: 0 })),
mockEnumerateWindows: vi.fn().mockResolvedValue([]),
mockGetScreenCaptureStatus: vi.fn(() => 'granted'),
mockIsMac: { value: true },
mockRequestScreenCaptureAccess: vi.fn(async () => false),
mockScreen: {
getCursorScreenPoint: vi.fn(() => ({ x: 10, y: 10 })),
getDisplayNearestPoint: vi.fn(() => ({
bounds: { height: 900, width: 1440, x: 0, y: 0 },
id: 1,
scaleFactor: 2,
})),
},
};
});
vi.mock('electron', () => ({
BrowserWindow: MockBrowserWindow,
dialog: {
showMessageBox: mockDialogShowMessageBox,
},
screen: mockScreen,
}));
vi.mock('@/const/dir', () => ({
preloadDir: '/mock/preload',
}));
vi.mock('@/const/env', () => ({
get isMac() {
return mockIsMac.value;
},
}));
vi.mock('@/utils/logger', () => ({
createLogger: () => ({
debug: vi.fn(),
error: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
}),
}));
vi.mock('@/utils/permissions', () => ({
getScreenCaptureStatus: mockGetScreenCaptureStatus,
requestScreenCaptureAccess: mockRequestScreenCaptureAccess,
}));
vi.mock('./WindowSourceService', () => ({
enumerateWindows: mockEnumerateWindows,
}));
vi.mock('./CaptureService', () => ({
captureRect: (...args: unknown[]) => mockCaptureRect(...args),
captureWindow: (...args: unknown[]) => mockCaptureWindow(...args),
}));
describe('ScreenCaptureManager', () => {
const createApp = ({ mainWindowVisible = true }: { mainWindowVisible?: boolean } = {}) => {
const mainWindow = {
browserWindow: {
id: 1,
isVisible: vi.fn(() => mainWindowVisible),
},
};
return {
browserManager: {
broadcastToAllWindows: vi.fn(),
broadcastToWindow: vi.fn(),
getMainWindow: vi.fn(() => mainWindow),
showMainWindow: vi.fn(),
},
buildRendererUrl: vi.fn().mockResolvedValue('http://localhost:5173/overlay'),
i18n: {
ns: vi.fn(() => (key: string) => key),
},
} as any;
};
beforeEach(() => {
vi.clearAllMocks();
mockBrowserWindow.isDestroyed.mockReturnValue(false);
mockDialogShowMessageBox.mockResolvedValue({ response: 0 });
mockEnumerateWindows.mockResolvedValue([]);
mockGetScreenCaptureStatus.mockReturnValue('granted');
mockIsMac.value = true;
mockRequestScreenCaptureAccess.mockResolvedValue(false);
});
it('keeps the app in regular mode when showing overlay on macOS', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
expect(mockBrowserWindow.setVisibleOnAllWorkspaces).toHaveBeenCalledWith(true, {
skipTransformProcessType: true,
visibleOnFullScreen: true,
});
});
it('focuses the overlay after showing it', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
expect(mockBrowserWindow.show).toHaveBeenCalled();
expect(mockBrowserWindow.focus).toHaveBeenCalled();
expect(mockBrowserWindow.moveTop).toHaveBeenCalled();
});
it('blocks quick composer and prompts for permission when screen recording is unavailable', async () => {
mockGetScreenCaptureStatus.mockReturnValue('denied');
mockDialogShowMessageBox.mockResolvedValue({ response: 0 });
const app = createApp();
const manager = new ScreenCaptureManager(app);
await manager.startSession();
expect(mockDialogShowMessageBox).toHaveBeenCalledWith(
app.browserManager.getMainWindow().browserWindow,
expect.objectContaining({
message: 'screenCaptureAccess.message',
title: 'screenCaptureAccess.title',
}),
);
expect(mockRequestScreenCaptureAccess).toHaveBeenCalled();
expect(mockEnumerateWindows).not.toHaveBeenCalled();
expect(MockBrowserWindow).not.toHaveBeenCalled();
});
it('does not open settings when permission prompt is dismissed', async () => {
mockGetScreenCaptureStatus.mockReturnValue('denied');
mockDialogShowMessageBox.mockResolvedValue({ response: 1 });
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
expect(mockRequestScreenCaptureAccess).not.toHaveBeenCalled();
expect(mockEnumerateWindows).not.toHaveBeenCalled();
expect(MockBrowserWindow).not.toHaveBeenCalled();
});
it('shows an app-modal prompt when the main window is hidden', async () => {
mockGetScreenCaptureStatus.mockReturnValue('denied');
const manager = new ScreenCaptureManager(createApp({ mainWindowVisible: false }));
await manager.startSession();
expect(mockDialogShowMessageBox).toHaveBeenCalledWith(
expect.objectContaining({
message: 'screenCaptureAccess.message',
title: 'screenCaptureAccess.title',
}),
);
expect(mockDialogShowMessageBox).not.toHaveBeenCalledWith(
expect.objectContaining({ id: 1 }),
expect.anything(),
);
});
describe('preview handlers', () => {
it('hides overlay via opacity while capturing rect and restores after', async () => {
const app = createApp();
const manager = new ScreenCaptureManager(app);
await manager.startSession();
const pngBuffer = Buffer.from([1, 2, 3, 4]);
mockCaptureRect.mockResolvedValue(pngBuffer);
const result = await manager.handlePreviewRect({ height: 50, width: 100, x: 10, y: 20 });
expect(result.success).toBe(true);
expect(result.captureId).toEqual(expect.any(String));
expect(result.dataUrl).toBe(`data:image/png;base64,${pngBuffer.toString('base64')}`);
expect(mockBrowserWindow.setOpacity).toHaveBeenCalledWith(0);
expect(mockBrowserWindow.setOpacity).toHaveBeenLastCalledWith(1);
expect(mockCaptureRect).toHaveBeenCalledWith({ height: 50, width: 100, x: 10, y: 20 }, 2, {
height: 900,
width: 1440,
x: 0,
y: 0,
});
expect(app.browserManager.broadcastToWindow).toHaveBeenCalledWith(
'app',
'overlayUploadRequest',
expect.objectContaining({
captureId: result.captureId,
filename: `screen-capture-${result.captureId}.png`,
mimeType: 'image/png',
}),
);
});
it('returns failure when previewRect has no session', async () => {
const manager = new ScreenCaptureManager(createApp());
const result = await manager.handlePreviewRect({ height: 50, width: 100, x: 10, y: 20 });
expect(result.success).toBe(false);
expect(mockCaptureRect).not.toHaveBeenCalled();
});
it('returns dataUrl after previewWindow and attaches window bounds', async () => {
mockEnumerateWindows.mockResolvedValue([
{
appName: 'Safari',
bounds: { height: 200, width: 300, x: 5, y: 6 },
order: 0,
overlayBounds: { height: 200, width: 300, x: 5, y: 6 },
title: 'Docs',
windowId: 42,
},
]);
const app = createApp();
const manager = new ScreenCaptureManager(app);
await manager.startSession();
const pngBuffer = Buffer.from([9, 9, 9]);
mockCaptureWindow.mockResolvedValue(pngBuffer);
const result = await manager.handlePreviewWindow(42);
expect(result.success).toBe(true);
expect(result.captureId).toEqual(expect.any(String));
expect(result.dataUrl).toBe(`data:image/png;base64,${pngBuffer.toString('base64')}`);
expect(result.rect).toEqual({ height: 200, width: 300, x: 5, y: 6 });
expect(mockCaptureWindow).toHaveBeenCalledWith(42);
expect(app.browserManager.broadcastToWindow).toHaveBeenCalledWith(
'app',
'overlayUploadRequest',
expect.objectContaining({ captureId: result.captureId }),
);
});
it('restores opacity even when capture fails', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
mockCaptureRect.mockResolvedValue(null);
const result = await manager.handlePreviewRect({ height: 50, width: 100, x: 10, y: 20 });
expect(result.success).toBe(false);
expect(mockBrowserWindow.setOpacity).toHaveBeenLastCalledWith(1);
});
});
describe('submit', () => {
it('closes overlay on submit', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
await manager.handleSubmit({
captureIds: ['capture-1'],
prompt: 'hello',
});
expect(mockBrowserWindow.destroy).toHaveBeenCalled();
});
});
describe('reportUploadStatus', () => {
it('forwards status updates to the overlay after a preview', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
const pngBuffer = Buffer.from([1, 2, 3]);
mockCaptureRect.mockResolvedValue(pngBuffer);
const result = await manager.handlePreviewRect({ height: 50, width: 100, x: 0, y: 0 });
expect(result.captureId).toBeTruthy();
mockBrowserWindow.webContents.send.mockClear();
manager.reportUploadStatus({
captureId: result.captureId!,
fileId: 'file-1',
status: 'ready',
});
expect(mockBrowserWindow.webContents.send).toHaveBeenCalledWith(
'overlayCaptureUploadStatus',
{ captureId: result.captureId, fileId: 'file-1', status: 'ready' },
);
});
it('ignores status updates for unknown captureIds', async () => {
const manager = new ScreenCaptureManager(createApp());
await manager.startSession();
mockBrowserWindow.webContents.send.mockClear();
manager.reportUploadStatus({ captureId: 'unknown', status: 'ready' });
expect(mockBrowserWindow.webContents.send).not.toHaveBeenCalledWith(
'overlayCaptureUploadStatus',
expect.anything(),
);
});
});
});
@@ -0,0 +1,376 @@
import { randomUUID } from 'node:crypto';
import type {
CapturePreviewResult,
CaptureRectParams,
OverlayCaptureUploadStatus,
OverlayCaptureUploadStatusPayload,
ScreenCaptureAgentOption,
ScreenCaptureModelOption,
ScreenCaptureOverlayTheme,
ScreenCaptureSession,
ScreenCaptureSubmitParams,
} from '@lobechat/electron-client-ipc';
import { BrowserWindow, dialog, screen } from 'electron';
import { BrowsersIdentifiers } from '@/appBrowsers';
import { preloadDir } from '@/const/dir';
import { isMac } from '@/const/env';
import type { App } from '@/core/App';
import { createLogger } from '@/utils/logger';
import { getScreenCaptureStatus, requestScreenCaptureAccess } from '@/utils/permissions';
import { captureRect, captureWindow } from './CaptureService';
import { enumerateWindows } from './WindowSourceService';
const logger = createLogger('screenCapture:ScreenCaptureManager');
const HIDE_SETTLE_MS = 40;
export interface OverlaySnapshotPayload {
agents?: ScreenCaptureAgentOption[];
defaultAgentId?: string;
defaultModelId?: string;
defaultProvider?: string;
models?: ScreenCaptureModelOption[];
theme?: ScreenCaptureOverlayTheme;
}
interface CaptureUploadEntry {
fileId?: string;
filename: string;
status: OverlayCaptureUploadStatus;
}
export class ScreenCaptureManager {
private overlayWindow: BrowserWindow | null = null;
private session: ScreenCaptureSession | null = null;
/**
* Most recent agent/model snapshot published by the main renderer via
* `screenCapture.publishOverlaySnapshot`. Populated asynchronously; the
* overlay still opens with an empty selector list if the renderer has not
* pushed yet.
*/
private snapshot: OverlaySnapshotPayload = {};
/**
* Per-capture upload state used to drive the overlay send button and to
* resolve captureIds back to uploaded fileIds on submit. Cleared when the
* session closes.
*/
private captureUploads = new Map<string, CaptureUploadEntry>();
constructor(private readonly app: App) {}
publishOverlaySnapshot(payload: OverlaySnapshotPayload): void {
this.snapshot = payload;
// If a session is already on screen, push the updated lists so the user
// sees the current agents without reopening the overlay.
if (this.session) {
this.session = { ...this.session, ...this.snapshot };
if (this.overlayWindow && !this.overlayWindow.isDestroyed()) {
this.overlayWindow.webContents.send('screenCaptureSession', this.session);
}
}
}
get isActive(): boolean {
return this.overlayWindow !== null && !this.overlayWindow.isDestroyed();
}
async startSession(): Promise<void> {
if (!(await this.ensureScreenCaptureAccess())) {
return;
}
if (this.isActive) {
logger.warn('Capture session already active');
this.close();
}
const cursor = screen.getCursorScreenPoint();
const display = screen.getDisplayNearestPoint(cursor);
const { bounds, scaleFactor } = display;
logger.info(
`Starting capture session on display ${display.id} (${bounds.width}x${bounds.height} @${scaleFactor}x)`,
);
const windows = await enumerateWindows(bounds, scaleFactor);
this.session = {
displayBounds: bounds,
scaleFactor,
windows,
...this.snapshot,
};
await this.createOverlayWindow(bounds);
}
async handlePreviewWindow(windowId: number): Promise<CapturePreviewResult> {
if (!this.session) {
return { error: 'no active session', success: false };
}
const winInfo = this.session.windows.find((w) => w.windowId === windowId);
if (!winInfo) {
return { error: `window ${windowId} not found`, success: false };
}
logger.info(`Previewing window ${windowId} (${winInfo.appName})`);
const pngBuffer = await this.withOverlayHidden(() => captureWindow(windowId));
if (!pngBuffer) {
return { error: 'capture failed', success: false };
}
const captureId = randomUUID();
const filename = `screen-capture-${captureId}.png`;
this.dispatchUpload(captureId, filename, pngBuffer);
return {
captureId,
dataUrl: `data:image/png;base64,${pngBuffer.toString('base64')}`,
rect: {
height: winInfo.overlayBounds.height,
width: winInfo.overlayBounds.width,
x: winInfo.overlayBounds.x,
y: winInfo.overlayBounds.y,
},
success: true,
};
}
/**
* Preview a rect from the overlay. `params` is in overlay-local DIP
* (relative to the current display); main translates to absolute before
* handing to the capture pipeline.
*/
async handlePreviewRect(params: CaptureRectParams): Promise<CapturePreviewResult> {
if (!this.session) {
return { error: 'no active session', success: false };
}
const { displayBounds, scaleFactor } = this.session;
const absolute = {
height: params.height,
width: params.width,
x: params.x + displayBounds.x,
y: params.y + displayBounds.y,
};
logger.info(`Previewing rect (${params.x},${params.y} ${params.width}x${params.height})`);
const pngBuffer = await this.withOverlayHidden(() =>
captureRect(absolute, scaleFactor, displayBounds),
);
if (!pngBuffer) {
return { error: 'capture failed', success: false };
}
const captureId = randomUUID();
const filename = `screen-capture-${captureId}.png`;
this.dispatchUpload(captureId, filename, pngBuffer);
return {
captureId,
dataUrl: `data:image/png;base64,${pngBuffer.toString('base64')}`,
rect: params,
success: true,
};
}
/**
* Record an upload status update from the main renderer and forward it to
* the overlay so the send button can reflect live progress.
*/
reportUploadStatus(payload: OverlayCaptureUploadStatusPayload): void {
const entry = this.captureUploads.get(payload.captureId);
if (!entry) {
logger.warn(`reportUploadStatus for unknown captureId=${payload.captureId}`);
return;
}
entry.status = payload.status;
if (payload.fileId) entry.fileId = payload.fileId;
logger.debug(
`upload status captureId=${payload.captureId} status=${payload.status} fileId=${payload.fileId ?? '-'}`,
);
if (this.overlayWindow && !this.overlayWindow.isDestroyed()) {
this.overlayWindow.webContents.send('overlayCaptureUploadStatus', payload);
}
}
async handleSubmit(params: ScreenCaptureSubmitParams): Promise<void> {
logger.info(
`Submit capture — promptLen=${params.prompt.length} captureIds=${params.captureIds.length} agentId=${params.agentId ?? '-'} modelId=${params.modelId ?? '-'}`,
);
// Close the overlay first so focus transfers cleanly to the main window.
this.close();
try {
this.app.browserManager.showMainWindow();
} catch (error) {
logger.error('Failed to show main window on submit:', error);
}
this.app.browserManager.broadcastToAllWindows('overlayDispatchMessage', params);
}
close(): void {
if (this.overlayWindow && !this.overlayWindow.isDestroyed()) {
this.overlayWindow.destroy();
}
this.overlayWindow = null;
this.session = null;
this.captureUploads.clear();
logger.info('Capture session closed');
}
/**
* Fade overlay out via opacity so the capture pipeline sees clean pixels
* underneath, then restore opacity. Keeping the window alive (as opposed to
* hide/show) avoids focus/z-order glitches.
*/
private async withOverlayHidden<T>(task: () => Promise<T>): Promise<T> {
const win = this.overlayWindow;
if (!win || win.isDestroyed()) {
return task();
}
win.setOpacity(0);
await delay(HIDE_SETTLE_MS);
try {
return await task();
} finally {
if (!win.isDestroyed()) {
win.setOpacity(1);
}
}
}
/**
* Hand the PNG buffer to the main renderer so the upload pipeline (TRPC +
* hash dedup + S3) runs there; keep a local entry so the overlay can
* observe status transitions via reportUploadStatus.
*
* The main renderer receives an `ArrayBuffer` via Electron's structured
* clone, avoiding the ~33% base64 overhead of a dataUrl round-trip.
*/
private dispatchUpload(captureId: string, filename: string, pngBuffer: Buffer): void {
this.captureUploads.set(captureId, { filename, status: 'uploading' });
// Copy into a fresh ArrayBuffer so the IPC structured-clone layer owns
// the memory outright (Node's Buffer pool can otherwise alias bytes).
const bytes = new ArrayBuffer(pngBuffer.byteLength);
new Uint8Array(bytes).set(pngBuffer);
this.app.browserManager.broadcastToWindow(BrowsersIdentifiers.app, 'overlayUploadRequest', {
bytes,
captureId,
filename,
mimeType: 'image/png',
});
}
private async ensureScreenCaptureAccess(): Promise<boolean> {
if (!isMac) {
return true;
}
const status = getScreenCaptureStatus();
if (status === 'granted') {
return true;
}
const t = this.app.i18n.ns('dialog');
const mainWindow = this.app.browserManager.getMainWindow();
const parentWindow = mainWindow?.browserWindow?.isVisible?.() ? mainWindow.browserWindow : null;
const options = {
buttons: [t('screenCaptureAccess.openSettings'), t('screenCaptureAccess.cancel')],
cancelId: 1,
defaultId: 0,
detail: t('screenCaptureAccess.detail'),
message: t('screenCaptureAccess.message'),
noLink: true,
title: t('screenCaptureAccess.title'),
type: 'warning' as const,
};
const result = parentWindow
? await dialog.showMessageBox(parentWindow, options)
: await dialog.showMessageBox(options);
if (result.response !== 0) {
logger.info(`Screen capture permission prompt dismissed; status=${status}`);
return false;
}
logger.info(`Opening screen capture permission settings; status=${status}`);
await requestScreenCaptureAccess();
return false;
}
private async createOverlayWindow(bounds: Electron.Rectangle): Promise<void> {
const win = new BrowserWindow({
...(isMac ? { type: 'panel' } : {}),
enableLargerThanScreen: true,
focusable: true,
frame: false,
fullscreenable: false,
hasShadow: false,
height: bounds.height,
resizable: false,
skipTaskbar: true,
transparent: true,
webPreferences: {
backgroundThrottling: false,
contextIsolation: true,
preload: `${preloadDir}/index.js`,
sandbox: false,
},
width: bounds.width,
x: bounds.x,
y: bounds.y,
});
win.setAlwaysOnTop(true, 'screen-saver');
win.setVisibleOnAllWorkspaces(true, {
...(isMac ? { skipTransformProcessType: true } : {}),
visibleOnFullScreen: true,
});
if (isMac) {
win.setHiddenInMissionControl(true);
}
this.overlayWindow = win;
win.webContents.on('did-fail-load', (_event, code, description) => {
logger.error(`Overlay did-fail-load code=${code} description=${description}`);
});
const url = await this.app.buildRendererUrl('/overlay');
logger.info(`Loading overlay URL: ${url}`);
win.webContents.once('did-finish-load', () => {
logger.info('Overlay did-finish-load');
if (this.session && !win.isDestroyed()) {
logger.info(`Sending overlay session with ${this.session.windows.length} windows`);
win.webContents.send('screenCaptureSession', this.session);
}
});
await win.loadURL(url);
win.show();
win.focus();
win.moveTop();
logger.info('Overlay window created and shown');
}
}
function delay(ms: number): Promise<void> {
return new Promise((resolve) => setTimeout(resolve, ms));
}
@@ -0,0 +1,130 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
const mockWindows = vi.fn();
const mockOpenWindowsSync = vi.fn();
const originalPlatform = process.platform;
vi.mock('electron', () => ({
app: {
getName: vi.fn(() => 'LobeHub'),
},
}));
vi.mock('node-screenshots', () => ({
Window: {
all: mockWindows,
},
}));
vi.mock('get-windows', () => ({
openWindowsSync: mockOpenWindowsSync,
}));
describe('WindowSourceService', () => {
beforeEach(() => {
vi.clearAllMocks();
Object.defineProperty(process, 'platform', { value: originalPlatform });
});
it('normalizes window geometry to display DIPs on Windows high-DPI displays', async () => {
Object.defineProperty(process, 'platform', { value: 'win32' });
mockOpenWindowsSync.mockReturnValue([{ owner: { processId: 42 } }]);
mockWindows.mockReturnValue([
{
appName: () => 'Finder',
height: () => 1200,
id: () => 1001,
isMinimized: () => false,
pid: () => 42,
title: () => 'Example',
width: () => 1600,
x: () => 400,
y: () => 200,
z: () => 10,
},
]);
const { enumerateWindows } = await import('./WindowSourceService');
const windows = await enumerateWindows(
{
height: 1080,
width: 1920,
x: 0,
y: 0,
},
1.5,
);
expect(windows).toEqual([
{
appName: 'Finder',
bounds: {
height: 800,
width: 1066.6666666666667,
x: 266.6666666666667,
y: 133.33333333333334,
},
order: 0,
overlayBounds: {
height: 800,
width: 1066.6666666666667,
x: 266.6666666666667,
y: 133.33333333333334,
},
title: 'Example',
windowId: 1001,
},
]);
});
it('preserves window geometry on retina displays without dividing by scale factor', async () => {
Object.defineProperty(process, 'platform', { value: 'darwin' });
mockOpenWindowsSync.mockReturnValue([{ owner: { processId: 42 } }]);
mockWindows.mockReturnValue([
{
appName: () => 'Finder',
height: () => 900,
id: () => 1001,
isMinimized: () => false,
pid: () => 42,
scaleFactor: () => 2,
title: () => 'Example',
width: () => 1440,
x: () => 200,
y: () => 100,
z: () => 10,
},
]);
const { enumerateWindows } = await import('./WindowSourceService');
const windows = await enumerateWindows({
height: 1620,
width: 2880,
x: 0,
y: 0,
});
expect(windows).toEqual([
{
appName: 'Finder',
bounds: {
height: 900,
width: 1440,
x: 200,
y: 100,
},
order: 0,
overlayBounds: {
height: 900,
width: 1440,
x: 200,
y: 100,
},
title: 'Example',
windowId: 1001,
},
]);
});
});
@@ -0,0 +1,140 @@
import type { ScreenCaptureWindowInfo } from '@lobechat/electron-client-ipc';
import { app } from 'electron';
import { openWindowsSync } from 'get-windows';
import { Window } from 'node-screenshots';
import { createLogger } from '@/utils/logger';
const logger = createLogger('screenCapture:WindowSourceService');
const MIN_WIDTH = 80;
const MIN_HEIGHT = 60;
const SYSTEM_APP_BLACKLIST = new Set([
'Dock',
'Window Server',
'WindowServer',
'Control Centre',
'Control Center',
'SystemUIServer',
'Notification Centre',
'Notification Center',
]);
interface DisplayBounds {
height: number;
width: number;
x: number;
y: number;
}
interface PreparedWindow {
appName: string;
bounds: DisplayBounds;
title: string;
windowId: number;
z: number;
}
interface WindowWithOptionalScaleFactor {
scaleFactor?: () => number;
}
function intersects(a: DisplayBounds, b: DisplayBounds): boolean {
return a.x < b.x + b.width && a.x + a.width > b.x && a.y < b.y + b.height && a.y + a.height > b.y;
}
function normalizeWindowBounds(
bounds: DisplayBounds,
scaleFactor: number | undefined,
): DisplayBounds {
if (process.platform !== 'win32') return bounds;
const normalizedScaleFactor =
typeof scaleFactor === 'number' && Number.isFinite(scaleFactor) && scaleFactor > 0
? scaleFactor
: 1;
if (normalizedScaleFactor === 1) return bounds;
return {
height: bounds.height / normalizedScaleFactor,
width: bounds.width / normalizedScaleFactor,
x: bounds.x / normalizedScaleFactor,
y: bounds.y / normalizedScaleFactor,
};
}
export async function enumerateWindows(
displayBounds: DisplayBounds,
displayScaleFactor?: number,
): Promise<ScreenCaptureWindowInfo[]> {
const selfName = app.getName();
let visiblePids: Set<number> | undefined;
try {
const visible = openWindowsSync({
accessibilityPermission: false,
screenRecordingPermission: false,
});
visiblePids = new Set(visible.map((w) => w.owner.processId));
} catch (error) {
logger.warn('get-windows unavailable, skipping whitelist filter:', error);
}
const preparedWindows = Window.all()
.map((win): PreparedWindow | null => {
if (visiblePids && !visiblePids.has(win.pid())) return null;
const appName = win.appName();
if (SYSTEM_APP_BLACKLIST.has(appName) || appName === selfName) return null;
if (win.isMinimized()) return null;
const width = win.width();
const height = win.height();
if (width < MIN_WIDTH || height < MIN_HEIGHT) return null;
const bounds = {
height,
width,
x: win.x(),
y: win.y(),
};
const normalizedBounds = normalizeWindowBounds(
bounds,
displayScaleFactor ?? (win as WindowWithOptionalScaleFactor).scaleFactor?.(),
);
if (!intersects(normalizedBounds, displayBounds)) return null;
return {
appName,
bounds: normalizedBounds,
title: win.title(),
windowId: win.id(),
z: win.z(),
};
})
.filter((win): win is PreparedWindow => win !== null)
.sort((left, right) => right.z - left.z);
const results = preparedWindows.map((win, index) => ({
appName: win.appName,
bounds: win.bounds,
order: index,
overlayBounds: {
height: win.bounds.height,
width: win.bounds.width,
x: win.bounds.x - displayBounds.x,
y: win.bounds.y - displayBounds.y,
},
title: win.title,
windowId: win.windowId,
}));
logger.info(`Enumerated ${results.length} windows for display`);
return results;
}
export function findWindowById(windowId: number): Window | undefined {
return Window.all().find((w) => w.id() === windowId);
}
@@ -1,58 +1,107 @@
import { exec } from 'node:child_process';
import { execFile } from 'node:child_process';
import { platform } from 'node:os';
import { promisify } from 'node:util';
import type { IToolDetector, ToolStatus } from '@/core/infrastructure/ToolDetectorManager';
import { createCommandDetector } from '@/core/infrastructure/ToolDetectorManager';
const execPromise = promisify(exec);
const execFilePromise = promisify(execFile);
type HeterogeneousCliAgentType = 'claude-code' | 'codex';
interface ValidatedDetectorOptions {
description: string;
name: string;
priority: number;
validateFlag?: string;
validateKeywords: string[];
}
const resolveCommandPath = async (command: string): Promise<string | undefined> => {
const trimmedCommand = command.trim();
if (!trimmedCommand) return;
const whichCommand = platform() === 'win32' ? 'where' : 'which';
try {
const { stdout } = await execFilePromise(whichCommand, [trimmedCommand], { timeout: 3000 });
return stdout.trim().split(/\r?\n/)[0] || trimmedCommand;
} catch {
return trimmedCommand;
}
};
const detectValidatedCommand = async (
command: string,
options: Pick<ValidatedDetectorOptions, 'validateFlag' | 'validateKeywords'>,
): Promise<ToolStatus> => {
const trimmedCommand = command.trim();
if (!trimmedCommand) return { available: false };
const { validateFlag = '--version', validateKeywords } = options;
try {
const { stderr, stdout } = await execFilePromise(trimmedCommand, [validateFlag], {
timeout: 5000,
windowsHide: true,
});
const output = `${stdout}\n${stderr}`.trim();
const loweredOutput = output.toLowerCase();
if (!validateKeywords.some((keyword) => loweredOutput.includes(keyword.toLowerCase()))) {
return { available: false };
}
return {
available: true,
path: await resolveCommandPath(trimmedCommand),
version: output.split(/\r?\n/)[0],
};
} catch {
return { available: false };
}
};
const HETEROGENEOUS_CLI_AGENT_OPTIONS = {
'claude-code': {
validateKeywords: ['claude code'],
},
'codex': {
validateKeywords: ['codex'],
},
} as const satisfies Record<
HeterogeneousCliAgentType,
Pick<ValidatedDetectorOptions, 'validateKeywords'>
>;
export const detectHeterogeneousCliCommand = async (
agentType: HeterogeneousCliAgentType,
command: string,
): Promise<ToolStatus> => {
const validator = HETEROGENEOUS_CLI_AGENT_OPTIONS[agentType];
if (!validator) return { available: false };
return detectValidatedCommand(command, validator);
};
/**
* Detector that resolves a command path via which/where, then validates
* the binary by matching `--version` (or `--help`) output against a keyword
* to avoid collisions with unrelated executables of the same name.
*/
const createValidatedDetector = (options: {
candidates: string[];
description: string;
name: string;
priority: number;
validateFlag?: string;
validateKeywords: string[];
}): IToolDetector => {
const {
name,
description,
priority,
candidates,
validateFlag = '--version',
validateKeywords,
} = options;
const createValidatedDetector = (
options: ValidatedDetectorOptions & {
candidates: string[];
},
): IToolDetector => {
const { candidates, description, name, priority, ...validation } = options;
return {
description,
async detect(): Promise<ToolStatus> {
const whichCmd = platform() === 'win32' ? 'where' : 'which';
for (const cmd of candidates) {
try {
const { stdout: pathOut } = await execPromise(`${whichCmd} ${cmd}`, { timeout: 3000 });
const toolPath = pathOut.trim().split('\n')[0];
if (!toolPath) continue;
const { stdout: out } = await execPromise(`${cmd} ${validateFlag}`, { timeout: 5000 });
const output = out.trim();
const lowered = output.toLowerCase();
if (!validateKeywords.some((kw) => lowered.includes(kw.toLowerCase()))) continue;
return {
available: true,
path: toolPath,
version: output.split('\n')[0],
};
} catch {
continue;
}
const status = await detectValidatedCommand(cmd, validation);
if (status.available) return status;
}
return { available: false };
@@ -6,7 +6,7 @@
*/
export { browserAutomationDetectors } from './agentBrowserDetectors';
export { cliAgentDetectors } from './cliAgentDetectors';
export { cliAgentDetectors, detectHeterogeneousCliCommand } from './cliAgentDetectors';
export { astSearchDetectors, contentSearchDetectors } from './contentSearchDetectors';
export { fileSearchDetectors } from './fileSearchDetectors';
export { runtimeEnvironmentDetectors } from './runtimeEnvironmentDetectors';
@@ -432,9 +432,9 @@ describe('FileService', () => {
});
it('should handle partial failures in batch deletion', async () => {
let callCount = 0;
let _callCount = 0;
mockFsUnlink.mockImplementation((path: any, callback: any) => {
callCount++;
_callCount++;
// Fail on a specific file
if (path.includes('file2.txt') && !path.includes('.meta')) {
callback(new Error('Permission denied'));

Some files were not shown because too many files have changed in this diff Show More