Compare commits

...

306 Commits

Author SHA1 Message Date
zhayujie
5d55ec0f8c feat(browser): reuse system Chrome/Edge, bundle playwright for desktop 2026-07-14 18:02:27 +08:00
zhayujie
eeb4b7981e feat(model): add gpt-5.6-luna, terra, sol 2026-07-14 11:27:03 +08:00
zhayujie
5f1c98881d feat(model): restore the claude-fable-5 model 2026-07-14 11:07:36 +08:00
zhayujie
94d0f56689 fix(desktop): make Windows dry-run signing work 2026-07-13 17:49:50 +08:00
zhayujie
4d690341a7 feat(desktop): add Windows code signing 2026-07-13 16:40:32 +08:00
zhayujie
d8c419227c Merge pull request #2950 from zhayujie/feat-mcp-oauth
feat(mcp): auto OAuth authorization for remote MCP servers
2026-07-13 12:01:32 +08:00
zhayujie
8c7cda89dc feat(mcp): support OAuth authorization for remote MCP servers 2026-07-13 12:00:14 +08:00
zhayujie
42a5cf9538 Merge pull request #2945 from weijun-xia/fix/edit-unify-fuzzy-uniqueness
refactor(edit): unify fuzzy uniqueness check with the fuzzy matcher
2026-07-12 23:09:51 +08:00
zhayujie
996406eb2a fix(desktop): resolve real login-shell PATH for backend 2026-07-10 01:03:04 +08:00
zhayujie
b98fbae6f6 fix(desktop): support web_password auth and fix password settings 2026-07-08 20:15:16 +08:00
zhayujie
4d87703e31 feat: update 2.1.3 docs 2026-07-08 16:43:15 +08:00
zhayujie
9ef64b7858 feat: release 2.1.3 2026-07-08 16:16:24 +08:00
zhayujie
bf0c26d3c4 Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2026-07-08 15:56:20 +08:00
zhayujie
65970c564c feat: release 2.1.3 2026-07-08 15:55:56 +08:00
weijun-xia
ed36ca99c0 refactor(edit): unify fuzzy uniqueness check with the fuzzy matcher
The uniqueness guard counted occurrences via normalize_for_fuzzy_match,
while fuzzy_find_text located matches with a whitespace-flexible regex,
so the two could disagree. Extract the pattern builder as a single
source of truth (_build_fuzzy_pattern) and add count_matches, which
counts with the same exact-then-fuzzy strategy used to locate and
replace. This is the optional follow-up suggested in the review of #2942.

Adds regression tests for exact and fuzzy multi-match rejection.
2026-07-08 11:13:30 +08:00
zhayujie
ce09efe640 Merge pull request #2943 from weijun-xia/fix/read-negative-offset
fix(read): correct off-by-one for negative offset with trailing newline
2026-07-08 10:59:30 +08:00
zhayujie
adaf9a7813 fix(i18n): correct language label glyphs 2026-07-08 10:58:41 +08:00
zhayujie
06f9492518 Merge pull request #2935 from anomixer/master
feat(i18n): 新增繁體中文 (zh-tw) 支援,並新增 Web 登出按鈕與動態顯示
2026-07-08 10:42:05 +08:00
zhayujie
530042675e Merge pull request #2942 from weijun-xia/fix/edit-fuzzy-preserve-whitespace
fix(edit): keep untouched lines intact on fuzzy match
2026-07-08 10:39:08 +08:00
zhayujie
ca404aeb24 fix(desktop): stop backend writing into the .app bundle 2026-07-08 01:59:08 +08:00
anomixer
a6c975f92c refactor(i18n): reply to upstream's #2935 feedback 2026-07-08 01:53:27 +08:00
anomixer
dd3cadbd81 Merge branch 'zhayujie:master' into master 2026-07-07 23:21:17 +08:00
zhayujie
efbabfcace feat: 2.1.3 pre-release 2026-07-07 22:32:10 +08:00
zhayujie
09c71ef1d9 feat(desktop): add colored icons to welcome suggestion cards 2026-07-07 22:28:11 +08:00
zhayujie
0bb8208f36 fix(desktop): restore install-dir choice on Windows first install 2026-07-07 21:48:43 +08:00
zhayujie
56571c77ca fix(desktop): Windows auto-update installs but doesn't relaunch 2026-07-07 20:14:11 +08:00
zhayujie
6c353d389b fix(desktop): switch Windows to oneClick NSIS 2026-07-07 18:22:10 +08:00
zhayujie
583217d396 fix(desktop): smoother update UX 2026-07-07 18:08:05 +08:00
xiaweiwei67-stack
93162d2f10 fix(edit): preserve file indentation when oldText is unindented
The fuzzy fallback anchored every line with a leading [ \t]* that
greedily consumed the file's indentation into the matched region, so a
no-indent oldText dropped the edited line's indentation. Fold leading
whitespace into the match only when oldText was itself indented on the
first line, mirroring exact-substring semantics. Add a regression test.
2026-07-07 17:24:58 +08:00
zhayujie
2e74295807 fix(desktop): auto-update install now actually replaces the app 2026-07-07 17:23:10 +08:00
zhayujie
93bf8844de Merge pull request #2941 from xiaweiwei67-stack/fix/bash-tempfile-utf8
fix(bash): write large-output temp file as UTF-8
2026-07-07 17:00:01 +08:00
xiaweiwei67-stack
fcc520df47 fix(read): correct off-by-one for negative offset with trailing newline
The Read tool documents that a negative offset reads from the end (-N = last N lines). Content is split on newline, so a file ending in a newline produces a trailing empty element and total_file_lines is one too high. Every negative offset was therefore off by one: offset=-1 returned the empty string after the final newline instead of the last line, and -N returned N-1 real lines.

Exclude the trailing empty element when computing the start line for negative offsets. Adds regression tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 16:53:35 +08:00
xiaweiwei67-stack
a94f4e3c18 fix(edit): keep untouched lines intact on fuzzy match
The Edit tool falls back to a whitespace-tolerant fuzzy match when oldText does not match byte-for-byte. On a fuzzy hit it replaced text inside a whitespace-normalized copy of the whole file and wrote that copy back, so every untouched line lost its original indentation (runs of spaces/tabs collapsed to a single space). For indentation-sensitive files such as Python this silently corrupts the file.

Locate the fuzzy match in the original content with a whitespace-flexible regex and return offsets into that original content, so only the matched region is replaced. Adds regression tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 16:50:16 +08:00
xiaweiwei67-stack
de9e7f0e84 fix(bash): write large-output temp file as UTF-8
When a command's output exceeds DEFAULT_MAX_BYTES the Bash tool spills the full output to a temp file. The file was opened in text mode without an explicit encoding, so it used the platform locale encoding (cp936/GBK on Chinese Windows). Output containing emoji or other characters not representable in that codepage raised UnicodeEncodeError, which propagated out and turned a successful command (exit code 0) into a tool error, discarding all output.

Open the temp file with encoding='utf-8', matching the sibling temp file written in _rewrite_long_python_c. Adds a regression test.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-07 16:45:37 +08:00
zhayujie
ed5b2d6ce6 fix: mac zip artifacts 2026-07-07 16:28:48 +08:00
zhayujie
2cd3fa7981 Merge pull request #2940 from fengyl07/feat/mcp-tool-retrieval
feat(mcp): on-demand vector retrieval for large MCP tool sets
2026-07-07 16:22:50 +08:00
fengyl07
f01cc3a0b4 test(mcp): cover retrieval helpers and executor injection paths 2026-07-07 16:01:38 +08:00
fengyl07
51bf09208d feat(agent): inject retrieved MCP tools in stream executor 2026-07-07 16:01:26 +08:00
fengyl07
bf0831a664 feat(mcp): cache MCP tool vectors with lazy embedding in ToolManager 2026-07-07 16:00:55 +08:00
fengyl07
96b1fccf76 feat(mcp): add stateless on-demand tool retrieval module 2026-07-07 16:00:31 +08:00
anomixer
4b57971d33 feat(i18n): support Traditional Chinese (zh-tw) for console, logs and docs 2026-07-07 15:52:32 +08:00
zhayujie
d531e14fbf feat(desktop): mac zip auto-update 2026-07-07 15:11:55 +08:00
zhayujie
8df38a23d2 fix(ci): run electron-builder in its own step so Windows produces installers 2026-07-07 12:32:04 +08:00
zhayujie
38105e6539 fix(desktop/ci): Windows build silently skipped + mac-x64 dmg not registered 2026-07-07 11:49:48 +08:00
zhayujie
14c6577d51 fix(desktop): make update badge/panel re-openable 2026-07-07 11:44:25 +08:00
zhayujie
f051a58db5 fix(desktop): add updater logging + allow prerelease 2026-07-06 20:16:20 +08:00
zhayujie
825d990312 fix(desktop): restore sent images/files from history after restart 2026-07-06 19:34:43 +08:00
zhayujie
cb31013584 build(desktop): decouple macOS notarization from CI into 3-stage release 2026-07-06 19:24:55 +08:00
zhayujie
dd74d1dabe Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2026-07-06 18:37:33 +08:00
zhayujie
75f3952ac6 feat(desktop): render sent images/videos/files 2026-07-06 18:35:33 +08:00
zhayujie
37423fbb31 Merge pull request #2936 from fengyl07/fix/read-env-bypass-2913
fix(read): block /proc/environ credential bypass (#2913)
2026-07-06 16:02:59 +08:00
fengyl07
00c3436d48 test(read): add regression tests for credential-path bypass (#2913) 2026-07-06 15:30:13 +08:00
fengyl07
377b4e5cb8 fix(read): block /proc/environ aliases and symlink bypass of credential file 2026-07-06 15:30:13 +08:00
zhayujie
a427586b89 fix(desktop): notarize .app in afterSign hook with fault-tolerant polling 2026-07-06 15:04:12 +08:00
zhayujie
a951494489 fix(desktop): notarize the .app zip 2026-07-06 10:19:54 +08:00
zhayujie
a871c0437d feat: add deep_dream_enabled config toggle 2026-07-05 21:31:16 +08:00
zhayujie
013960cd5a feat: update release flow 2026-07-05 21:10:43 +08:00
zhayujie
60aebf41a8 fix: optimize notarize script 2026-07-05 20:28:02 +08:00
zhayujie
2cf521e57e fix: exception handling in config eval 2026-07-05 12:35:24 +08:00
zhayujie
dad3a84efb Merge pull request #2927 from shunfeng8421/fix/eval-ast-literal-eval
security: replace eval() with ast.literal_eval + document pickle risk
2026-07-05 12:17:54 +08:00
zhayujie
ae864c7ff9 fix(desktop): notarize dmg in a retryable step to survive poll timeouts 2026-07-05 11:23:43 +08:00
zhayujie
3b33114a40 fix(desktop): sign embedded backend via mac.binaries for notarization 2026-07-03 14:59:31 +08:00
zhayujie
e0f49ac619 fix(desktop): sign backend by cert SHA-1 to fix CI keychain lookup 2026-07-03 12:16:44 +08:00
zhayujie
01ec49afd2 build(desktop): enable macOS signing & notarization for release 2026-07-03 12:01:42 +08:00
zhayujie
b44154fe02 desktop: add brand logo to the top-left corner on Windows/Linux 2026-07-01 18:38:45 +08:00
zhayujie
b8dad38622 desktop: fix model config dropdowns and provider listing 2026-07-01 16:28:40 +08:00
zhayujie
80fea77c86 feat(model): support claude-sonnet-5 2026-07-01 10:31:06 +08:00
zhayujie
e5f3eb48d4 desktop: make backend port deterministic (fixed + pre-launch cleanup) and reorder footer menu 2026-06-30 20:05:01 +08:00
zhayujie
ca876b0c65 fix(desktop): isolate backend port from the web console 2026-06-30 10:48:33 +08:00
zhayujie
0a762b8c08 fix(desktop): bundle cli command modules 2026-06-30 10:03:12 +08:00
zhayujie
fd90a89b45 desktop: improve channels page and titlebar UI 2026-06-29 19:52:46 +08:00
zhayujie
f82eb39d23 fix(desktop): stop send button icon flicker 2026-06-29 18:00:46 +08:00
zhayujie
2786148153 fix(desktop): match send button to web 2026-06-29 16:33:10 +08:00
zhayujie
2959cfea32 feat(desktop): align chat UI with web console 2026-06-29 12:17:35 +08:00
zhayujie
e536232963 fix(desktop): bundle document parsing libs 2026-06-29 11:07:08 +08:00
shunfeng8421
778d78cebe security: replace eval() with ast.literal_eval + document pickle risk
- config.py:418 — Replace eval(value) with ast.literal_eval() for
  environment variable config overrides. ast.literal_eval only parses
  Python literals and cannot execute arbitrary code, preventing
  environment-variable-based code injection.

- config.py:310-328 — Add security notes on pickle.load/dump usage.
  Pickle is safe here (local appdata file, same-process write/read),
  but notes suggest JSON migration or HMAC signing for future hardening.

Fixes: potential RCE via controlled environment variables
2026-06-29 07:33:33 +08:00
zhayujie
538281da51 feat(desktop): support plugin commands 2026-06-27 12:19:12 +08:00
zhayujie
12cd626949 fix(web): reclaim orphaned SSE streams to prevent fd leak #2924 2026-06-27 11:32:19 +08:00
zhayujie
ff64a7930e Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2026-06-27 11:14:00 +08:00
zhayujie
5d726fe340 fix(desktop): run bundled backend from writable data dir 2026-06-27 11:13:39 +08:00
zhayujie
e1834124d4 Merge pull request #2918 from Tunnello/fix-azure-openai-stream
fix: azure openai stream
2026-06-27 10:59:40 +08:00
Eric L
f49e965736 fix: azure openai stream config 2026-06-26 20:54:25 +08:00
Eric L
936eaf5939 fix: azure openai stream 2026-06-25 21:13:50 +08:00
zhayujie
7047b30e27 feat(models): support doubao-seed-2.1 series 2026-06-25 11:53:24 +08:00
zhayujie
5c67e970d1 feat: sync knowledge management to desktop 2026-06-25 11:43:19 +08:00
zhayujie
8023c4e8b7 feat(knowledge): consolidate actions into a New menu and refine docs layout 2026-06-25 11:19:33 +08:00
zhayujie
641b84519c Merge branch 'pr-2915' 2026-06-25 11:02:55 +08:00
zhayujie
0c8cb974e2 feat(knowledge): auto-maintain index.md, improve import UX, fix embedding provider
- Auto-rebuild knowledge/index.md from the real directory tree on
  create/import so it never drifts or loses documents (no longer relies
  on the agent hand-writing it).
- Auto-open the created/imported document in the tree after success.
- Add create_document status message, shorten EN action buttons, and
  localize the "insert template" content.
- Show filename for protected system files (index.md/log.md) in the tree
  instead of their H1 heading.
- Reuse a shared embedding-provider factory so knowledge index sync also
  gets vectors instead of degrading to keyword-only search.
2026-06-25 11:02:18 +08:00
zhayujie
915edbe145 fix(tools): make web SSRF protection opt-in, disabled by default 2026-06-24 19:40:55 +08:00
zhayujie
0c20c5c159 Merge pull request #2917 from zhayujie/feat-cow-desktop
feat: CowAgent desktop client
2026-06-24 19:26:06 +08:00
zhayujie
6211e63f90 fix(desktop): avoid false init failure after long background 2026-06-24 19:19:24 +08:00
zhayujie
44b61684ed feat(desktop): first-run onboarding, OS-language default, native polish 2026-06-24 16:38:51 +08:00
zhayujie
ab6f49a822 fix(desktop): correct electron-updater import for commonjs 2026-06-24 11:01:45 +08:00
zhayujie
02517e4a01 fix(ci): never mark pre-release versions as latest 2026-06-24 10:32:12 +08:00
zhayujie
2599966cf7 fix(ci): migrate retired macos-13 runner to macos-15-intel 2026-06-24 10:11:18 +08:00
zhayujie
6c68931892 fix(ci): publish per-platform instead of all-or-nothing 2026-06-24 10:04:05 +08:00
zhayujie
41855ed511 fix(ci): only enable mac signing when certificate secret is set
GitHub injects unset secrets as empty strings, and electron-builder treats
an empty CSC_LINK as a (broken) certificate path, aborting the mac build
with "desktop not a file". Export the signing vars only when non-empty so
unsigned builds fall back cleanly, matching local behavior. Windows builds
already passed; guarded the same way for consistency.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-23 20:57:01 +08:00
zhayujie
02bc91f4af fix(ci): track desktop backend packaging sources
The global build/ gitignore rule was silently excluding the PyInstaller
spec, desktop requirements, and build script under desktop/build/, so the
release workflow failed at "pip install -r requirements-desktop.txt" with
a missing-file error. Re-include just those source files while keeping the
build outputs (dist/, build-work/) and local venv ignored.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-23 20:46:14 +08:00
zhayujie
e9352e6984 feat(desktop): add auto-update via electron-updater + manual CI trigger 2026-06-23 20:37:27 +08:00
zhayujie
ec4c36f450 feat(desktop): redirect writable data to ~/.cow for packaged app
Introduce get_data_root() driven by the COW_DATA_DIR env var so the
packaged desktop build stores config.json, run.log, user data and
WeChat credentials under ~/.cow — surviving app updates and keeping the
app bundle read-only. Source deployments leave COW_DATA_DIR unset and
fall back to the repo root, so existing behavior is unchanged.
2026-06-23 17:22:53 +08:00
yangziyu-hhh
9ea0017778 Merge remote-tracking branch 'upstream/master'
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
2026-06-23 17:18:24 +08:00
yangziyu-hhh
f1cdc2d2cc feat(knowledge): add document creation and import 2026-06-22 17:58:33 +08:00
zhayujie
215ed24401 feat(desktop): add log viewer entry points (NavRail + Help menu) 2026-06-22 16:30:11 +08:00
zhayujie
c432681b2b fix(desktop): prevent backend zombie process and IPC listener leaks 2026-06-22 15:46:49 +08:00
zhayujie
49452e035d feat(desktop): native desktop enhancements
- Add a minimal application menu with common items and shortcuts:
  New Chat (Cmd+N), Settings (Cmd+,), Reload, etc.
- Add system tray with show window, new chat, and quit; click icon to restore
- Add single-instance lock so relaunching focuses the existing window
- Implement close-to-tray: closing the window hides it; only a real quit destroys it
- Explicitly define Window menu's Close Window bound to Cmd+W / Ctrl+W to reliably trigger hide-to-tray
- Listen for menu-action in the renderer to handle menu-triggered new chat / open settings
2026-06-22 15:32:22 +08:00
zhayujie
5db2998e3d fix: update web console version 2026-06-22 13:04:15 +08:00
zhayujie
d1336b872e feat(desktop): rework Skills page with semantic theming 2026-06-21 19:33:11 +08:00
zhayujie
e1e29b32e9 feat(desktop): add QR scan login for channels 2026-06-21 17:27:18 +08:00
zhayujie
214dcaf141 feat(desktop): rework Scheduled Tasks page with edit & delete 2026-06-20 17:34:50 +08:00
zhayujie
77a196de8b feat(desktop): rework Memory page to match web console 2026-06-20 17:08:23 +08:00
zhayujie
108d04398b feat(desktop): implement Knowledge Base page 2026-06-20 16:35:06 +08:00
zhayujie
0bc0f2b930 fix(evolution): prevent MCP tool re-injection into restricted review agent #2904 2026-06-20 15:17:11 +08:00
zhayujie
c9c16298ec feat(desktop): merge settings and models into tabs; improve model config and chat scroll
Merge the "Models" menu item into the "Settings" page with top tabs
(Basic / Models), reducing one nav entry. Basic settings now only handles
provider + model selection; API key/base are consolidated into the Models
tab. Unconfigured providers are marked in the dropdown and guide the user
to configure them.

The Models tab covers all 7 capabilities (chat/vision/image/asr/tts/
embedding/search) with vendor credentials and routing. The vendor section
follows the web client's unified design: built-in and custom providers
share one grid, "set as default" is removed, and credentials are configured
via modals.

- Add settings/ directory: SettingsPage (tab shell), BasicSettings, ModelsTab,
  CapabilityCard, primitives, modelsHelpers; remove the old ConfigPage
- Rewrite ModelEntry/ModelProvider/CapabilityState/SearchCapabilityState in
  types.ts to match the real /api/models shape (mixed string|{value,hint})
- Fix empty chat model dropdown: chat has no provider_models, so fall back
  to the top-level providers[].models
- Fix chat scroll sliding from top on session switch: snap instantly to
  bottom on switch, smooth-scroll only for streaming updates
- Rename the Knowledge menu label and align model/config copy with the web
  client (vendor credentials, main model, etc.)
- Clean up orphaned i18n keys
2026-06-20 11:35:47 +08:00
zhayujie
2ef31d5d33 fix(desktop): adapt remaining pages to updated types 2026-06-20 00:40:15 +08:00
zhayujie
e9d9b566a4 feat(desktop): chat core with streaming, sessions, tool steps and markdown-it rendering 2026-06-20 00:40:11 +08:00
zhayujie
3baa3252bc feat(desktop): align API client and types with latest web backend endpoints 2026-06-20 00:39:47 +08:00
zhayujie
90d9db0f83 feat(desktop): platform-aware shell, design tokens and three-column layout 2026-06-20 00:39:38 +08:00
zhayujie
8bff4f1658 Merge branch 'master' into feat-cow-desktop 2026-06-19 21:35:29 +08:00
zhayujie
a0e20ef311 Merge branch 'pr-2909' 2026-06-19 21:07:32 +08:00
zhayujie
6996215d3b feat(models): custom model explanation standardization 2026-06-19 21:06:56 +08:00
zhayujie
03ffa2db7d refactor(web): drop hardcoded preset models for custom providers & fix handler bug
- Remove vendor-specific preset model lists for custom embedding/vision
  providers; users now type the model id manually (lists vary per vendor).
- Strip third-party vendor names from comments.
- Fix AttributeError in _set_vision/_set_embedding: the provider constants
  live on ModelsHandler, not ConfigHandler, so selecting a built-in
  provider crashed. Also replace the fragile [:-1] slice with an explicit
  non-custom filter.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-19 20:59:10 +08:00
zhayujie
75e3110e8c Merge pull request #2905 from Jiangrong-W/harness-fix/net-egress-cowagent-browser-navigate-no-internal-url-guard
fix(browser): block link-local / cloud-metadata navigation (SSRF guard)
2026-06-19 20:36:28 +08:00
christop
033480eef1 fix(browser): block link-local / cloud-metadata navigation (SSRF guard)
The browser tool navigates to a model-supplied URL via Playwright
page.goto and then auto-snapshots the page back to the model. The
http(s) navigate path performed no filtering, so a model tool call
(including under prompt injection) could point the agent-driven browser
at the cloud-metadata endpoint (169.254.169.254) and read the
credentials back through the snapshot.

Unlike the vision/web_fetch tools, the browser legitimately needs local
pages — a dev server on localhost / 127.0.0.1 / a LAN IP — so a blanket
"block all internal" policy is the wrong default here. The guard is
therefore narrow: it resolves the hostname and rejects only link-local
addresses (169.254.0.0/16, which includes the 169.254.169.254 metadata
endpoint, and IPv6 fe80::/10) plus the AWS IPv6 IMDS address
(fd00:ec2::254), before navigation. Loopback and RFC1918/LAN stay
reachable so local dev works out of the box.

Only http/https targets are validated; the documented non-HTTP scheme
handling (about:/data:/file:) is unchanged. An operator who deliberately
needs the link-local/metadata target can opt out with
tools.browser.allow_private_targets = true.

tests/test_security_ssrf_browser_navigate.py asserts link-local/metadata
targets are blocked while loopback, RFC1918/LAN and public targets
navigate through (browser service and DNS are stubbed; no real
browser/network).

Signed-off-by: christop <825583681@qq.com>
2026-06-19 20:15:27 +08:00
HnBigVolibear
8ddfcbb125 feat(web): add custom provider support for embedding & vision models, and fix memory_get Windows path bug!
1. Embedding model: support custom provider
   - Add "custom" entry to EMBEDDING_VENDORS with supports_dim_param=False
   - Parse custom:<id> credentials and model fallback in agent_initializer
   - Expand custom_providers as custom:<id> entries in Web UI dropdown

2. Vision model: support custom provider
   - Add custom:<id> routing in _route_by_provider_id
   - Add _build_custom_provider reading credentials from custom_providers
   - Expand custom_providers in Web UI dropdown, add validation in _set_vision

3. Fix memory_get Windows path validation bug!
   - str.startswith(path+'/') always False on Windows due to backslashes, So All Users can not use "memory_get" tool in Windows.
   - Use os.path.realpath + os.sep, consistent with MemoryService

4. Fix historical needsModel:false bug preventing embedding provider switch
   - Change embedding needsModel to true in console.js
   - Support custom:<id> resolution in cow_cli /memory status, also for adding custom provider support

Closes #2908, Closes #2880
2026-06-19 18:35:44 +08:00
zhayujie
a5aaecc48d Merge pull request #2901 from 6vision/master
docs(wecom_bot): mention webhook (callback) mode in channel docs
2026-06-19 17:12:13 +08:00
yangziyu-hhh
84d6848e67 feat(knowledge): add document creation and import 2026-06-18 16:06:41 +08:00
zhayujie
01373465b0 fix(web): correct Bridge import path in MessageDeleteHandler #2902 2026-06-17 22:04:24 +08:00
6vision
a1e733080d docs(wecom_bot): mention webhook (callback) mode in channel docs 2026-06-17 20:37:51 +08:00
zhayujie
3b3ef715bb feat: update 2.1.2 release docs 2026-06-17 19:09:47 +08:00
zhayujie
47b2bf9d46 Merge pull request #2900 from Jr61-star/harness-fix/net-egress-cowagent-web-fetch-no-private-ip-guard
fix(web_fetch): add SSRF guard for model-supplied URLs
2026-06-17 17:47:48 +08:00
christop
ea47f3097e fix(web_fetch): add SSRF guard for model-supplied URLs
web_fetch fetched any http/https URL a model emitted, checking only the
scheme. It performed requests.get(..., allow_redirects=True) with no
hostname resolution check and no private/loopback/link-local/cloud-metadata
filtering, and never re-validated redirect targets. A model (including one
under prompt injection) could make CowAgent fetch 127.0.0.1, RFC1918,
169.254.169.254 or other internal endpoints and return their bodies into
the conversation; a public URL could also 302-bounce into a private target.

The repo already shipped an SSRF validator for the vision tool
(Vision._validate_url_safe). Extract that logic into a shared helper
(agent/tools/utils/url_safety.py) and reuse it:

- execute() now validates the URL before dispatching to either fetch path.
- A new _safe_get() helper disables auto-redirect and follows redirects
  manually, re-validating every hop so a public URL cannot bounce into an
  internal address. Both the webpage and document fetch paths use it.
- Vision._validate_url_safe now delegates to the shared helper (public API
  unchanged), so both URL-consuming tools share one guard.

Stdlib only (ipaddress, socket, urllib.parse); no new dependency. Adds
tests/test_security_ssrf_web_fetch.py covering loopback, cloud-metadata,
RFC1918 and a public->loopback redirect.

Sink: agent/tools/web_fetch/web_fetch.py (_fetch_webpage / _fetch_document).
Signed-off-by: christop <825583681@qq.com>
2026-06-17 17:38:35 +08:00
zhayujie
70c1c44d15 feat: add kimi-k2.7-code and glm-5.2 models
- Add Kimi kimi-k2.7-code (default), kimi-k2.7-code-highspeed, and GLM glm-5.2 (default)
- Fix 400 error when disabling thinking on kimi-k2.7-code; omit the thinking param for this series since it only accepts type=enabled
- Update README, docs (zh/en/ja), install scripts, and Web console model dropdown
2026-06-17 11:54:35 +08:00
zhayujie
e3dce45b2a fix(bash): bypass cmd.exe length limit for long python -c on Windows 2026-06-16 21:11:33 +08:00
zhayujie
3bb8ec3bea feat(web): support manually renaming sessions in console #2897 2026-06-16 20:03:38 +08:00
zhayujie
35e42a3ad6 Merge pull request #2893 from yangziyu-hhh/master
feat(knowledge): add category and document management
2026-06-16 18:52:43 +08:00
zhayujie
949575ad14 Merge pull request #2896 from 6vision/feat/wecom-bot-callback #2869
Feat: wecom bot callback
2026-06-16 17:29:30 +08:00
zhayujie
1c34f0f03d Merge pull request #2892 from HnBigVolibear/master
feat(web): enhance scheduled task management in web console
2026-06-16 17:24:05 +08:00
6vision
eed2eab014 refactor(wecom_bot): use wecom_bot_mode field and clean up temp images
Replace the boolean wecom_bot_callback with a wecom_bot_mode field
("websocket" | "webhook"), consistent with the Feishu channel's
feishu_event_mode. Update startup() and send() to branch on the mode.

Also fix a temp-file leak in _load_image_base64: downloads, format
conversions and compressions wrote to /tmp but were never removed. Track
only the temp files created here and delete them in a finally block,
leaving the caller's original local file untouched.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-16 17:23:31 +08:00
zhayujie
381cbed4fd fix(evolution): improve review summary language, tone and length 2026-06-15 22:28:07 +08:00
zhayujie
4cde25325d fix(evolution): pin review summary language via i18n hint 2026-06-15 22:04:52 +08:00
zhayujie
8d70af5e89 refactor(evolution): simplify review summary prompt 2026-06-15 21:43:53 +08:00
zhayujie
b3408d8e5f fix(windows): persist cow CLI dir to user PATH 2026-06-15 20:53:01 +08:00
zhayujie
e74906fbec fix(evolution): skip idle review while a turn is running 2026-06-15 20:33:52 +08:00
6vision
52209217fc refactor(wecom_bot): config-file-only callback mode + fixed callback path
Remove the callback-mode fields (Callback Mode / Token / EncodingAESKey /
Port) from the web console channel form; these are rarely changed and are
now configured via config.json only. The console keeps Bot ID / Secret for
the long-connection setup.

Serve the callback HTTP server on a fixed path (/wecombot) instead of any
path (/.*), so unrelated requests 404 rather than being processed as
signature-failing WeCom callbacks. The bot's receive-message URL must point
at http(s)://host:<port>/wecombot.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 19:47:19 +08:00
6vision
018493a60b fix(wecom_bot): revert callback image cap to 512KB (2MB inline body rejected by WeCom)
The 2MB cap (matching the long-connection upload path) does not work for
the callback path: there the whole image is base64-embedded in an
AES-encrypted body returned on every poll. A ~1.5MB image (base64 ~2.1MB,
encrypted ~2.8MB) makes WeCom reject the finish packet and poll forever,
which also surfaces as a truncated text bubble and WeCom's own timeout
error. Cap well below that at 512KB so the finish packet is accepted.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 19:22:16 +08:00
6vision
630373b1f0 fix(wecom_bot): defer text finish for image race; align callback image cap to 2MB
Defer the callback stream finish after a text reply so a trailing
image-with-caption send (text first, image 0.3s later) can merge in
instead of closing the stream prematurely. Raise the callback inline
image cap from 512KB to 2MB to match the long-connection upload path.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 18:56:20 +08:00
6vision
dec31dfd75 fix(wecom_bot): shrink callback inline images so finish packets aren't rejected
In callback mode the image is base64-embedded in the stream finish reply and
the whole response is AES-encrypted and returned on every poll. A multi-MB
body is rejected/times out on WeCom's side, leaving the "···" bubble spinning
and the image never shown.

- Compress callback images to <=512KB (JPEG, resize if needed) instead of the
  10MB the protocol nominally allows
- Fall back to the original image if compression fails, and log the final
  base64 payload size for diagnosis

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 18:26:38 +08:00
zhayujie
2397ea019e feat: config cli support evolution 2026-06-15 17:57:10 +08:00
zhayujie
77da90e316 feat: record self-evolution turn on streaming chat 2026-06-15 17:51:22 +08:00
6vision
18ce17d21a fix(wecom_bot): finalize stream on cancel; never force-finish on a timer
The streaming "···" bubble should only stop when the task actually completes
or the user cancels — not on an arbitrary timeout that would also steal the
response_url fallback's chance to deliver a late answer.

- On agent_cancelled, finalize the stream (finish=true, "🛑 已中止" if empty)
  and schedule the response_url fallback so the bubble clears immediately when
  a run is cancelled, even past the poll window
- Do not force-finish a still-running stream on a timer; let it keep spinning
  until completion or cancel. Answers that finish after WeCom's ~6min poll
  window are delivered via response_url instead

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-15 17:26:06 +08:00
zhayujie
ce09b14836 feat: sync self-evolution switch and fix scheduler context 2026-06-15 16:30:34 +08:00
yangziyu-hhh
e7a069b060 Merge remote-tracking branch 'upstream/master' 2026-06-15 13:17:12 +08:00
yangziyu-hhh
6e3933be30 feat(knowledge): add category and document management 2026-06-15 13:15:07 +08:00
zhayujie
e2cb9e11b0 fix(install): avoid greenlet source build on Windows & guide browser tool install 2026-06-15 11:50:41 +08:00
zhayujie
d281a34c6f fix(models): demote claude-fable-5 from Claude default 2026-06-14 21:06:41 +08:00
湖南大白熊工作室
c97cf5610f Merge branch 'zhayujie:master' into master 2026-06-14 18:38:45 +08:00
zhayujie
ab674a3517 fix(docs): architecture graph link 2026-06-14 17:22:41 +08:00
zhayujie
7d63e7d8fa feat(cli): add agent self-restart command 2026-06-14 17:20:41 +08:00
zhayujie
6538843bdf fix(memory): remove max_tokens cap in deep dream distillation 2026-06-14 11:06:25 +08:00
HnBigVolibear
bd5fede122 feat(web): enhance scheduled task management in web console. Now we can edit any scheduled tasks in web!
- Add toggle, update and delete APIs for scheduled tasks
- Add task edit modal with schedule/action updates in web console  (PS: In edit box, now I Prevent channel type changes during editing (weixin token bound to session) )
- Add enable/disable switch with visual feedback in task cards
- Sort task list by enabled status first, then by next_run_at

Closes #2882
2026-06-14 02:11:15 +08:00
zhayujie
047fb57630 Merge pull request #2891 from sufan721/feat-add-role-module
- Auto-scan roles/*.json on startup and merge into built-in roles
2026-06-13 18:23:57 +08:00
sufan721
583c1de5ba - Auto-scan roles/*.json on startup and merge into built-in roles
- Same title overrides built-in role; different title appends as new
  - roles/ directory is optional — no impact when absent

  Users can now add custom roles by simply dropping a .json file
  into the roles/ directory and restarting. No config changes needed.
2026-06-13 16:50:54 +08:00
zhayujie
c9c293f67c Merge pull request #2888 from kirs-hi/fix/robustness-cancel-keyerror-compress-loop
fix: avoid KeyError on /cancel and infinite loop in image compression
2026-06-12 18:28:10 +08:00
6vision
561631baba fix(wecom_bot): rescue late replies via response_url + fix degrade notice
Handle agent replies that finish after WeCom stops polling the passive
stream (the poll window is ~6min from the user's message).

- Capture response_url from the message callback and, when a reply is
  finalized but no poll picks it up within a short grace period, push it as
  a one-shot active markdown reply (valid 1h, single use)
- Guard against double delivery via delivered/url_sent flags
- Embed public image URLs in the active markdown; note when a local image
  can't be delivered post-timeout
- Append (instead of discarding) the unsupported-type notice for
  file/voice/video replies so streamed text is preserved
- Quiet the per-poll debug log and log stream completion with content size

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 18:24:33 +08:00
zhayujie
80d0a6aeb2 Merge pull request #2879 from yangziyu-hhh/master
feat: stream Bash progress and guard message actions during replies
2026-06-12 18:19:13 +08:00
zhayujie
6b5ee245ae feat(web): integrate custom providers into the provider credentials section
- Merge the separate custom-providers section into the unified provider
  grid; "Custom" in the add-provider picker now acts as an add-new action
  (trailing + mark) and opens the dedicated modal, supporting multiple
  OpenAI-compatible endpoints
- Simplify the custom provider modal: drop the default-model field, add
  an inline delete button, align colors with the theme
- Keep the legacy single "custom" card visible (models page, chat
  dropdown and legacy config page) while custom_api_key/custom_api_base
  is still in use, so existing single-provider setups don't disappear
- Unify user-facing wording from "vendor" to "provider" in UI and docs
- Restructure custom provider docs (zh/en/ja) around Web console and
  config file usage
2026-06-12 18:03:33 +08:00
6vision
5c43c2f519 feat(wecom_bot): add callback (webhook) mode alongside long connection
Support receiving WeCom smart-bot messages via encrypted HTTP callback in
addition to the existing WebSocket long connection. Disabled by default
(wecom_bot_callback=false).

- Add wecom_bot_callback / wecom_bot_token / wecom_bot_encoding_aes_key /
  wecom_bot_port config keys
- Add WXBizJsonMsgCrypt-based crypto module for URL verification, callback
  decryption and passive-reply encryption (receive_id empty for internal bots)
- Reply asynchronously via the official stream-refresh polling: register a
  stream id on first reply, accumulate agent output into per-stream state, and
  serve the latest content (text + image) on each poll until finish
- Fall back to EncodingAESKey for media decryption when callback bodies carry
  no per-message aeskey
- Degrade unsupported passive replies (file/voice/video) to a text notice
- Expose the new fields in the Web console channel config

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 16:26:54 +08:00
yangziyu-hhh
9387980e74 ci: add Windows Bash streaming tests 2026-06-12 14:07:40 +08:00
yangziyu-hhh
075d9fc608 fix: address Bash streaming review feedback 2026-06-12 13:45:39 +08:00
zhayujie
63bfab03f6 Merge pull request #2877 from kirs-hi/feat/custom-providers-ui
feat(web): manage multiple custom (OpenAI-compatible) providers in console UI
2026-06-12 11:54:48 +08:00
zhayujie
1d7e6b3703 fix(web): accept custom:<id> providers in the chat capability card
_set_chat rejected the expanded "custom:<id>" ids with "unknown
provider", so switching to a custom provider was only possible from
the custom providers section. Now the chat card and the custom section
behave consistently: _set_chat validates the id against
custom_providers, falls back to the provider's default model when none
is picked, and _chat_capability expands the dropdown with the
"custom:<id>" entries (legacy single-custom mode unchanged).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 11:54:19 +08:00
kirs-hi
ad64e17a34 fix: avoid KeyError on cancel and infinite loop in image compression
Two independent robustness fixes:

1. ChatChannel.cancel_session / cancel_all_session raised KeyError when a
   session existed in self.sessions but no future had been dispatched yet.
   self.sessions[sid] is created in produce(), but self.futures[sid] is only
   created later in consume() on first dispatch. Cancelling in that window
   (e.g. user sends a message then immediately cancels) crashed the cancel
   path. Use self.futures.get(sid, []) so an absent entry is a no-op.

2. compress_imgfile decremented JPEG quality by 5 with no lower bound. For an
   image that cannot be compressed below max_size, quality went 0, negative,
   ... — the loop never terminated and passed invalid quality values to PIL.
   Add a min_quality floor (10) and return the best effort once reached.

Adds tests/test_robustness_fixes.py covering both paths (5 tests).
2026-06-12 11:03:04 +08:00
kirs-hi
0092376c07 fix: address review — no auto-hijack, sync model on activation, cleanup
1. Creating a provider no longer auto-switches bot_type. Only an
   explicit make_active=true changes the active model — prevents
   silently hijacking users on Claude/OpenAI/etc.

2. When a provider IS activated, its 'model' is now written into the
   global 'model' field. This ensures all three paths (regular chat,
   agent_bridge, vision) use the correct model without per-path patches.

3. Removed unused i18n key 'models_custom_name_exists' (no longer
   referenced after the id-based rework removed name-collision checks).

4. Updated tests: 39 passing (added model-sync tests, fixed tests that
   relied on the removed auto-activation behavior).
2026-06-12 10:05:05 +08:00
zhayujie
6fb19a68b5 feat: update default config in run script 2026-06-11 19:30:08 +08:00
zhayujie
d5427d967a fix(installer): fix ASR/TTS default, self-evolution flag, and QuickEdit hang 2026-06-11 19:24:08 +08:00
zhayujie
7fd30b608c fix(cow_cli): fix line breaks in CLI replies 2026-06-11 19:08:30 +08:00
zhayujie
830b05f243 Merge pull request #2886 from kirs-hi/fix/ssrf-and-path-traversal
fix(security): SSRF protection for vision tool + path traversal guard for skill install
2026-06-11 18:24:09 +08:00
kirs-hi
e85290cddc fix(security): SSRF protection for vision tool + path traversal guard for skill install
1. Vision SSRF (#2878, #2872):
   Add _validate_url_safe() that resolves the target hostname via DNS and
   rejects any IP in private (RFC1918), loopback, link-local, or reserved
   ranges before requests.get() is called. This blocks attacks that use
   attacker-controlled image URLs to probe internal services or cloud
   metadata endpoints (169.254.169.254).

2. Skill install path traversal (#2873):
   Add _safe_skill_dir() that validates the skill name cannot escape the
   skills/ root directory. Rejects names containing '..', absolute paths,
   and any resolved path that falls outside the custom_dir boundary.
   Applied to _add_url(), _add_package(), and delete().

Both fixes include comprehensive unit tests (19 test cases) covering
blocked patterns, edge cases, and allowed legitimate usage.

Closes #2878
Closes #2873
Ref: #2872
2026-06-11 17:40:41 +08:00
kirs-hi
1940d628a8 refactor(custom): id-based routing, single source of truth, security fixes
Rework the multi custom-provider design per maintainer review:

1. Data model: use server-generated uuid4 short id as primary key;
   'name' is now a pure display label that can be freely renamed.

2. Routing: drop 'custom_active_provider'; activate a provider by
   setting bot_type to 'custom:<id>'. Single source of truth — no
   pointer drift between bot_type and a separate active selector.

3. Security: drag_sensitive() now recursively masks api_key/secret in
   nested structures (custom_providers list); previously only top-level
   string fields were masked.

4. Per-provider model: the provider's 'model' field now takes effect on
   the main chat path and agent path (was silently ignored before).

5. XSS fix: replace all inline onclick handlers in custom-provider UI
   with data-* attributes + event delegation. Provider names never
   appear in executable HTML contexts.

Legacy compatibility: bot_type='custom' (no colon) still reads the flat
custom_api_key/custom_api_base fields byte-for-byte identically.

Closes: consolidates #2876 into this PR as requested.
Ref: #2838
2026-06-11 17:25:24 +08:00
kirs-hi
cffa590d3e feat(web): manage multiple custom (OpenAI-compatible) providers in console UI
Adds first-class support for configuring more than one custom
OpenAI-compatible provider (e.g. SiliconFlow, DeepSeek, local vLLM)
and switching the active one from the web console, addressing #2838.

Backend:
- config: new `custom_providers` (list) and `custom_active_provider`
  fields, fully backward compatible with the legacy single
  `open_ai_api_base`/`model` fields (used as fallback).
- models/custom_provider.py: centralized resolver
  `resolve_custom_credentials()` returning (api_key, api_base, model),
  with active-provider selection and graceful fallback.
- chat_gpt_bot.py wired to use the resolver.
- web_channel.py: `_provider_overview` expands `custom_providers` into
  one card per provider (id `custom:<name>`, active flag, masked key);
  new POST actions `set_custom_provider`, `delete_custom_provider`,
  `set_active_custom_provider` with hermetic persistence + bridge reset.

Frontend:
- console.js: dedicated "Custom providers" section with add / edit /
  delete / set-active actions, masked-key keep-existing handling, and
  ~20 new zh/en i18n strings.
- chat.html: custom provider modal.

Tests:
- tests/test_custom_provider.py (11) - resolver/config behavior.
- tests/test_custom_provider_handlers.py (18) - write-side handlers and
  overview expansion, including duplicate-name rejection.

All 29 unit tests pass.
2026-06-10 18:12:30 +08:00
yangziyu-hhh
402e2bfee0 feat: stream Bash progress and guard message actions during replies 2026-06-10 14:19:03 +08:00
zhayujie
f5caba81d6 feat(models): support claude-fable-5 2026-06-10 09:39:37 +08:00
zhayujie
354350dec9 fix: hide code block language label when language is undefined 2026-06-09 19:20:15 +08:00
zhayujie
0513298f57 feat(evolution): allow rare persona (AGENT.md) self-evolution 2026-06-09 19:03:49 +08:00
zhayujie
08e23e5bd8 fix(vision): bump vision timeout from 60s to 180s to avoid premature failures 2026-06-09 16:36:01 +08:00
zhayujie
e812c7d29a feat(vision): increase vision tool max_tokens 2026-06-09 16:08:17 +08:00
zhayujie
ef46199346 feat: update run.sh for python3.13 2026-06-09 15:24:32 +08:00
zhayujie
7c9ea62993 chore(evolution): lower trigger thresholds to 6 turns / 10 min idle 2026-06-09 15:22:38 +08:00
zhayujie
8cb53e6129 feat: release 2.1.1 2026-06-09 14:38:05 +08:00
zhayujie
12c0383dc8 docs: update self-evolution docs 2026-06-09 12:07:41 +08:00
zhayujie
83b53039f3 feat: add 2.1.1 release docs 2026-06-09 11:41:32 +08:00
zhayujie
7e6a309935 feat(evolution): default on for new installs, unify naming, add docs 2026-06-09 10:49:43 +08:00
zhayujie
33c03e30d9 fix(web): switch to a sibling session when deleting the active one 2026-06-09 10:49:34 +08:00
zhayujie
1f1abdd7b6 fix(evolution): correct [SILENT] verdict and enable guarded bash for skill creation 2026-06-09 09:29:30 +08:00
zhayujie
16134bd150 fix: update python version in powershell script 2026-06-08 20:19:57 +08:00
zhayujie
c887fc71ad fix: support Python 3.13 by installing web.py from GitHub 2026-06-08 20:15:32 +08:00
zhayujie
9fc39f648f feat(evolution): give review agent full context, add knowledge signal, polish UX 2026-06-08 20:06:01 +08:00
zhayujie
ec9557e3d8 feat(web): resume live streaming when switching back to a session 2026-06-08 17:32:27 +08:00
zhayujie
7cf0f7d42d fix(web): self-heal stuck Cancel send button 2026-06-08 15:48:21 +08:00
zhayujie
b7aa64279d fix(web): support parallel sessions; fix lost/duplicate in-flight replies 2026-06-08 15:36:48 +08:00
zhayujie
26300a8d43 feat(evolution): flag self-evolution bubbles in UI and relax MEMORY.md writes 2026-06-07 21:00:03 +08:00
zhayujie
8dd21ddb83 Merge pull request #2868 from zhayujie/feat-self-evolution
feat(evolution): add self-evolution subsystem
2026-06-07 20:10:50 +08:00
zhayujie
ff584f8421 feat: add inter-method splitting 2026-06-07 20:10:26 +08:00
zhayujie
ca4a8253a1 docs(evolution): add Self-Evolution guide 2026-06-07 20:07:20 +08:00
zhayujie
157374401a feat(web): add self-evolution toggle in agent config 2026-06-07 19:12:32 +08:00
zhayujie
ba777ed706 feat(evolution): add self-evolution subsystem
Add a self-evolution subsystem that reviews idle conversations in an
isolated agent and durably learns from them — patching/creating skills,
finishing unfinished tasks, and backfilling missed memory.

- Trigger: background idle scan, fires when a session is idle >= N min AND
  (>= N turns OR context usage > 80%). In-memory cursor reviews only new
  messages so a session never re-learns old content.
- Isolated review agent: same model, restricted toolset, hard write-guard
  confining edits to the workspace (built-in skills are protected).
- Safety: file-level backup before edits + evolution_undo tool; notify the
  user ONLY when a workspace file actually changed (no-nag rule); capped
  concurrency.
- Records to memory/evolution/<date>.md, surfaced in the memory UI's
  renamed "Self-Evolution" tab (merged with dream diaries).
- Hide internal [SCHEDULED]/[EVOLUTION]/backup_id markers from chat history
  display (also fixes scheduler marker leakage) while keeping them in stored
  content for undo.
- Flat config: self_evolution_enabled (default off until release),
  self_evolution_idle_minutes (15), self_evolution_min_turns (6).
- Tests: tests/test_evolution.py (stub + real model modes, 7 scenarios).
2026-06-07 18:55:33 +08:00
zhayujie
0e4da1d1c5 feat(cli): show project path in cow status 2026-06-06 19:06:19 +08:00
zhayujie
72847e0711 feat(i18n): order channel list by UI language 2026-06-06 19:00:38 +08:00
zhayujie
3c19614c74 refactor(web-console): polish message actions on bubbles after #2865 2026-06-06 16:07:31 +08:00
zhayujie
a2e4955116 Merge pull request #2865 from core-power/feat/web-console-improvements
feat: message management and code block enhancements
2026-06-06 15:54:28 +08:00
PF4YZYNS\admin
c62175c06b - Add edit/delete/regenerate for user and bot messages
- Add language labels and copy buttons to code blocks
- Enhance drag-and-drop to full chat view
- Fix data consistency bugs in message operations
- Use RLock to prevent deadlock in conversation store"
2026-06-05 18:51:35 +08:00
zhayujie
fde4b6f590 Merge pull request #2863 from orbisai0security/fix/bash-credential-path-v2
fix(bash): narrow credential-file block to ~/.cow/.env only
2026-06-05 15:46:59 +08:00
zhayujie
3d7c68bac6 fix(wechatmp): reject webhook requests when wechatmp_token is empty 2026-06-05 15:14:28 +08:00
zhayujie
72a477f10c fix(models): route mimo-* models to MiMo bot in agent mode 2026-06-05 14:46:16 +08:00
OrbisAI Security
2a16c562a8 fix(bash): narrow credential-file block to ~/.cow/.env only
Replace the broad `~/.cow` directory check with a regex that matches
only the credential file path (`\.cow[/\\]\.env`), so legitimate access
to other `~/.cow/` subdirectories (e.g. skills) is no longer blocked.

Drop the incomplete env/printenv blocking rule per reviewer feedback.

Rewrite test_invariant_bash.py to use the correct Bash().execute()
API and cover both the blocked and allowed cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 11:36:22 +05:30
zhayujie
2b670e73f3 docs: update README.md 2026-06-04 23:17:37 +08:00
zhayujie
3994594019 docs: update badge in README.md 2026-06-04 22:43:45 +08:00
zhayujie
39c9386b54 Merge pull request #2859 from xliu123321/fix/mcp-stdio-windows
fix(mcp): replace select.select with queue.Queue for cross-platform stdio I/O
2026-06-04 11:51:08 +08:00
liusk
4cc57cc08d fix(mcp): enable concurrent calls for SSE and streamable-http transports
_stdio_send (single pipe) must remain serialized under _call_lock,
but SSE and streamable-http use independent HTTP requests and can
safely execute concurrently across sessions.

- Scope _call_lock to stdio transport only
- Add _http_lock with double-checked pattern to protect _http_session_id
  initialization during concurrent streamable-http requests
2026-06-04 11:44:35 +08:00
liusk
639a3eac1e fix(mcp): replace select.select with queue.Queue for cross-platform stdio I/O
- Use reader thread + queue.Queue instead of select.select() which does not
  work with pipes on Windows (only sockets)
- Make MCP server timeout configurable via mcp.json (default 120s)
- Validate JSON-RPC response id to skip stale responses from timed-out calls
- Log MCP server stderr at WARNING level instead of DEBUG for visibility
2026-06-04 09:10:48 +08:00
zhayujie
79323358e5 feat: add X-Title header for linkai request 2026-06-03 17:42:57 +08:00
zhayujie
cdb093c74a fix(i18n): refine auto language fallback for deployments 2026-06-03 16:09:15 +08:00
zhayujie
f6f3ce5f05 fix(i18n): refine auto language fallback for deployments 2026-06-03 15:33:29 +08:00
zhayujie
4805f3d4d3 fix(agent): register cancel token in ChatService stream run 2026-06-03 14:47:11 +08:00
zhayujie
1d797cdaf5 feat(channel): support telegram/slack/discord credential mapping 2026-06-03 11:26:36 +08:00
zhayujie
4d8458669c chore(install): simplify model menu, add MiMo option 2026-06-02 17:10:26 +08:00
zhayujie
92ec9653e5 feat(models): support qwen3.7-plus multi-modal model 2026-06-02 16:38:17 +08:00
zhayujie
e861d98007 feat(models): support ASR model selection in web console 2026-06-02 15:05:35 +08:00
zhayujie
a97eeb1fd9 Merge pull request #2857 from nightwhite/codex/fix-asr-model-hot-switch
Fix ASR model persistence in models API
2026-06-02 14:54:02 +08:00
nightwhite
cd88b23b5d fix: persist ASR model in models API 2026-06-02 13:01:20 +08:00
zhayujie
33eabf937b Merge pull request #2853 from Wyh-max-star/WYH
chore:add group task board plugin source
2026-06-02 10:38:29 +08:00
zhayujie
beb5df16a3 Merge pull request #2855 from octo-patch/feature/upgrade-minimax-m3
feat(minimax): add MiniMax-M3 as default, drop older M2.5/M2.1/M2
2026-06-02 10:30:42 +08:00
octo-patch
7fa743f01a feat(minimax): add MiniMax-M3, set as default, drop M2.5/M2.1/M2
- Add MINIMAX_M3 = "MiniMax-M3" constant and put it first in MODEL_LIST
- Default MinimaxBot model: MiniMax-M2.7 -> MiniMax-M3
- Keep MiniMax-M2.7 and MiniMax-M2.7-highspeed as legacy options
- Drop MINIMAX_M2_5 / MINIMAX_M2_1 / MINIMAX_M2_1_LIGHTNING / MINIMAX_M2
- Update web console recommended/provider model lists
- Update README capability table and docs/models index (en/zh/ja)
- Update docs/models/minimax.mdx and coding-plan.mdx MiniMax section
- Update run.sh / run.ps1 installer default and menu hint
- Update zh CLI status sample output
- Update unit tests to assert new M3 default and constant

TTS (speech-2.*) and API base URL remain unchanged.
2026-06-01 21:30:38 +08:00
zhayujie
1f6859d78f feat: update CLI version to 2.1.0 2026-06-01 16:59:19 +08:00
zhayujie
2853735472 docs: update README.md 2026-06-01 16:46:16 +08:00
zhayujie
feaa9076b0 feat: release 2.1.0 2026-06-01 16:02:55 +08:00
zhayujie
ce0249706e docs: update issue/pr templates 2026-06-01 11:10:12 +08:00
zhayujie
af2c839231 docs: add contributing guide and issue/PR templates 2026-06-01 11:01:28 +08:00
zhayujie
2b2d24ed25 docs: update doc references 2026-05-31 22:22:48 +08:00
Wyh-max-star
04d28f9d2d chore:add group task board plugin source 2026-05-31 20:52:42 +08:00
zhayujie
1dbf41f384 Merge pull request #2852 from zhayujie/feat-i18n
feat: support i18n across the whole project
2026-05-31 20:15:59 +08:00
zhayujie
9e6a2cc2c0 feat(installer): revamp install flow with i18n 2026-05-31 20:11:23 +08:00
zhayujie
7bf4ef3d05 docs: make English the default docs language and fix link paths 2026-05-31 17:52:22 +08:00
zhayujie
126649f70f feat(i18n): localize system prompts, workspace templates and dynamic prompts 2026-05-31 17:38:31 +08:00
zhayujie
1827a2a31c feat(i18n): bind web language switch to cow_lang config 2026-05-31 17:01:43 +08:00
zhayujie
fcf4eb78dc feat(i18n): add global language resolution and localize user-facing text 2026-05-31 16:49:35 +08:00
zhayujie
2ec6ea8045 Merge pull request #2850 from lyteen/feature/command-matching
feat: /command matching
2026-05-31 15:17:16 +08:00
lyteen
0994a3586d [feat] Fuzzy /command Resolution & Custom Aliases 2026-05-30 23:12:24 +08:00
zhayujie
29c4be6a3a feat(terminal): add agent streaming UX with reasoning/tool-call rendering 2026-05-30 19:10:56 +08:00
zhayujie
c5b8e06891 feat(channel): add Discord channel 2026-05-30 18:20:27 +08:00
zhayujie
54a20bca92 docs: update README doc 2026-05-30 17:32:21 +08:00
zhayujie
6e786bde90 Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2026-05-30 17:18:51 +08:00
zhayujie
b671b0d725 docs: add web file serve root config 2026-05-30 17:18:31 +08:00
zhayujie
57f5692074 Merge pull request #2840 from 6vision/feat/wechatcom-kf-channel
feat: add wechatcom kf channel
2026-05-30 17:17:59 +08:00
zhayujie
b0ac0731c7 Merge branch 'master' into feat/wechatcom-kf-channel 2026-05-30 17:17:29 +08:00
zhayujie
3c161df526 Merge pull request #2848 from 6vision/fix/wechatmp-passive-merge-replies
fix(wechatmp): improve passive reply multi-turn output and local image sending
2026-05-30 17:12:36 +08:00
zhayujie
aa3f48e93c fix(web): confine /api/file to allowed dirs to prevent arbitrary file read 2026-05-30 17:06:58 +08:00
zhayujie
5ae1e1adde feat(channel): support slack bot 2026-05-30 17:01:42 +08:00
6vision
fe8b8fe831 fix(wechatmp): support local file:// images in send
Agent-generated images are sent as IMAGE_URL with a file:// path, but the wechatmp channel always used requests.get, which fails on file:// with InvalidSchema. Now read local files directly (file:// or local path) and fall back to HTTP download for remote URLs, in both passive and active reply modes.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-30 16:33:49 +08:00
6vision
5aca54c083 fix(wechatmp): flush cached segments while task still running
Previously the passive reply only drained the cache after the agent task fully finished, so for long multi-turn tasks the user could not retrieve already-cached intermediate segments. Now return cached segments as soon as they are available, even while the task is still running; the next user message fetches the rest.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-30 15:48:27 +08:00
6vision
458b1a1d88 fix(wechatmp): merge cached text segments in passive reply
In subscription account passive reply mode, WeChat allows only one reply per request. Multi-turn agent output was cached as separate entries, forcing the user to send an extra message to fetch each one. Now drain and merge all consecutive cached text segments into a single reply; media still returns one at a time.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-30 14:41:51 +08:00
zhayujie
3dd4b84179 feat(models): support claude-opus-4-8 2026-05-29 10:19:45 +08:00
6vision
99bddb79d6 fix(wechat_kf): download attachments to agent_workspace/tmp
So agent tools resolve relative refs like 	mp/xxx.pdf on the first
try, matching weixin's _get_tmp_dir convention.
2026-05-28 19:40:12 +08:00
zhayujie
136b0b89e8 fix: optimize browser memory 2026-05-28 19:09:26 +08:00
6vision
c605b0b080 feat(wechat_kf): cache images/files and merge into next text turn
Adopt the same channel-level pattern as weixin/wecom_bot/feishu so
the agent actually sees attachments the user sent:
- IMAGE: agent mode never reads memory.USER_IMAGE_CACHE, so a photo
  sent before a question (e.g. "image" then 30s later "what's this?")
  used to be lost. Now lone images go into channel.file_cache and
  the next TEXT turn appends "[图片: <path>]" to the query before
  producing the context. Cross-batch image+text combinations now
  work as users expect.
- FILE: previously dropped at the sync_msg filter and unsupported
  by WechatKfMessage. Add msgtype="file" parsing, download via the
  WeCom media API, preserve the original filename from
  Content-Disposition (RFC 5987 + plain forms), and route through
  the same file_cache pipeline as images, surfacing as
  "[文件: <path>]" in the next text turn.
2026-05-28 18:11:41 +08:00
zhayujie
b7b8e3679c fix: avoid conflict with pypi translate package 2026-05-28 15:48:20 +08:00
zhayujie
aeb6610ff4 Merge pull request #2843 from zhayujie/feat-telegram
feat(channel): support telegram bot
2026-05-28 15:12:08 +08:00
zhayujie
e3eacc77d7 feat(channel): support telegram bot 2026-05-28 15:07:09 +08:00
6vision
37661daf40 refactor(wechat_kf): persist sync_msg cursor under $HOME
Move the sync_msg cursor file from the project-local tmp/ dir to ~/.wechat_kf_cursors.json so it survives tmp/ cleanups and cwd changes across restarts. Aligns with the weixin channel's credentials file convention.

- add wechat_kf_cursor_path config (default ~/.wechat_kf_cursors.json)
- expand ~ via os.path.expanduser in the channel init
- chmod the cursor file to 0o600 after each flush (no-op on Windows)
2026-05-28 14:33:45 +08:00
6vision
877b848370 fix(wechat_kf): stop dropping rapid-fire messages in batch dedup
_dedup_image_text_pair previously fell back to returning only the last message whenever the batch was not exactly an image+text pair, which silently dropped multiple texts/images sent in quick succession.

Cursor freshness is already guaranteed by sync_msg, so no extra stale-history protection is needed. Now we return all messages by default and only collapse a batch when it is exactly a 2-message image+text pair within a 5s window (order-insensitive, normalized to [image, text]).
2026-05-28 14:23:04 +08:00
6vision
5c163cc0fe fix: dispatch callback async to avoid WeCom 5s timeout
WeCom requires the callback HTTP response within ~5s, otherwise it retries the same notification. The previous code ran sync_msg pulling synchronously inside Query.POST, so a backlog could exceed the deadline and trigger retries that race on the same cursor and end up replying to the same user multiple times.

- Dispatch consume_callback to a background ThreadPoolExecutor and return 'success' immediately from the HTTP handler.
- Serialize work per open_kfid with a lock so retried/concurrent callbacks queue up instead of racing the cursor window.
- Shutdown the executor on channel stop().
2026-05-28 12:23:56 +08:00
6vision
6e04ea8240 refactor(wechat_kf): rename channel from wechatcom_kf and split corp_id
Rename the WeCom customer-service channel and give it its own corp_id
field so users no longer have to share `wechatcom_corp_id` with the
self-built WeCom app channel.

Renames (channel-side):
- channel type / const: wechatcom_kf -> wechat_kf
- package dir: channel/wechatcom_kf/ -> channel/wechat_kf/
- python files / classes: WechatComKf* -> WechatKf*
- config keys: wechatcom_kf_{secret,token,aes_key,port} ->
  wechat_kf_{secret,token,aes_key,port}; new wechat_kf_corp_id
- env vars: WECHATCOM_KF_* -> WECHAT_KF_*; new WECHAT_KF_CORP_ID
- log prefix / cursor file: [wechatcom_kf] -> [wechat_kf]
- web console CHANNEL_DEFS key + startup log line

Renames (docs):
- docs/channels/wecom-kf.mdx -> docs/channels/wechat-kf.mdx (zh/en/ja)
- update docs.json sidebar entries and all field names inside the docs

In addition, the Web Console "微信客服" entry now exposes its own
Corp ID field instead of reusing the wechatcom_app one, and includes
the screenshot of the visual config in the channel guide.

Web Console onboarding section is added (Tabs: Web Console / config
file) and the local URL `http://127.0.0.1:9899/` parenthetical is
dropped for consistency with other channel docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-28 12:12:44 +08:00
zhayujie
d106465419 feat(channel): telegram first version 2026-05-28 12:10:00 +08:00
zhayujie
f39380cea7 Merge pull request #2841 from zhayujie/feat-add-mimo
feat(models): support xiaomi mimo
2026-05-28 10:51:43 +08:00
zhayujie
bccce2d7cb feat(models): support xiaomi mimo 2026-05-28 10:49:52 +08:00
6vision
6721dbdbcc docs(wechatcom_kf): add web console onboarding tab 2026-05-27 21:53:54 +08:00
zhayujie
83cd6ad158 fix(browser): preserve non-http schemes in navigate URL 2026-05-27 18:42:21 +08:00
zhayujie
116fb27257 fix: robust tool args JSON parsing for non-strict providers #2823 2026-05-27 18:37:54 +08:00
zhayujie
8d67177a1b feat(agent): support user-initiated cancel for in-flight agent runs 2026-05-26 23:36:09 +08:00
zhayujie
ad2db1a776 feat(mcp): support streamable-http mcp protocol 2026-05-26 12:11:59 +08:00
zhayujie
2e6d9e0f27 chore: remove useless plugins 2026-05-25 17:11:57 +08:00
zhayujie
e05f85f3ce feat: optimize model name display in English 2026-05-25 15:09:53 +08:00
zhayujie
40c48a9a61 chore(deps): relax numpy>=1.24 to >=1.21 for Python 3.7 compatibility 2026-05-25 14:47:55 +08:00
zhayujie
c9a7525d0b Merge pull request #2832 from yangluxin613/feat/cjk-search-fix
fix(memory): CJK keyword search + vector search optimization
2026-05-25 14:45:49 +08:00
yangluxin613
fd571ac539 fix(memory): address PR review — numpy/UPSERT soft deps + BM25 floor + BLOB dim
- numpy soft dependency: try/except import + _HAS_NUMPY flag; _encode_embedding
  and _decode_embedding fall back to struct.pack/unpack; search_vector falls back
  to pure-Python cosine loop — startup never fails without numpy reinstalled
- SQLite UPSERT guard: _HAS_UPSERT = sqlite_version_info >= (3,24,0); save_chunk
  and save_chunks_batch fall back to INSERT OR REPLACE on SQLite < 3.24 with a
  one-time startup warning about potential FTS rowid drift
- _bm25_rank_to_score floor: 0.3 + 0.69*(|rank|/(1+|rank|)) → always in [0.3, 0.99),
  prevents small-corpus matches scoring 0.0 and being filtered by min_score
- detect_index_dim BLOB-aware: check isinstance(raw, bytes) first and return
  len(raw)//4 before json.loads, so /memory status works after embedding format switch
- Comment: "CJK single-char" → "CJK tokens shorter than 3 characters"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 14:15:16 +08:00
zhayujie
c5a3f991c5 fix(scheduler): make cron pushes survive restart on weixin channel 2026-05-25 12:15:57 +08:00
zhayujie
eb74b73351 fix(web): handle non-string web_password to avoid login TypeError 2026-05-25 11:14:14 +08:00
yangluxin613
9b31f45481 fix(memory): _search_like ASCII query always returns empty
matched_count only counted cjk_words hits; pure ASCII queries had
cjk_words=[] so matched_count=0 and all SQL-matched rows were filtered
out. Change to count across all tokens (cjk_words + ascii_words) so
the LIKE fallback works correctly when FTS5 is unavailable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 09:02:07 +08:00
yangluxin613
bc9c1691f5 fix(memory): CJK keyword search + vector search optimization
- Add trigram FTS5 table for CJK/mixed-language search with BM25 ranking
- Fix three-step search routing: unicode61 (ASCII) → trigram (CJK/mixed) → LIKE fallback
- Fix _bm25_rank_to_score: abs(rank)/(1+abs(rank)) instead of max(0,rank)
- Fix INSERT OR REPLACE → UPSERT to preserve FTS5 content table rowid stability
- Fix FTS5 JOIN to use rowid instead of id column
- Fix _search_like: single-char CJK match, dynamic scoring, merged CJK+ASCII path
- Add numpy vectorized cosine similarity + BLOB embedding storage (6x smaller)
- Add _decode_embedding backward compat for legacy JSON embeddings
- Add threading.RLock for concurrent write safety
- Add _meta table to avoid trigram backfill re-running on every startup
- Activate EmbeddingCache in MemoryManager for session-level query deduplication
- Add numpy>=1.24 to requirements.txt
- Merge upstream master (embedding package refactor, FTS5 self-healing methods)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 08:56:08 +08:00
zhayujie
73bf83d2ff docs: add public-access notes for server deployment 2026-05-25 00:09:52 +08:00
zhayujie
36e1988fee docs: update README.md 2026-05-24 19:21:06 +08:00
zhayujie
aad6ef635e docs: update README.md 2026-05-24 19:11:34 +08:00
zhayujie
96659cd616 docs: update project docs 2026-05-24 18:58:10 +08:00
zhayujie
c8787b7de4 Merge branch 'feat-readme-refactoring' 2026-05-24 18:30:18 +08:00
zhayujie
91d427c8f9 docs: update docs and readme 2026-05-24 18:29:57 +08:00
zhayujie
c8c0573dbd Merge pull request #2831 from zhayujie/feat-readme-refactoring
docs: README refactoring
2026-05-24 18:10:03 +08:00
zhayujie
29af855ecd docs: update README.md 2026-05-24 18:03:33 +08:00
zhayujie
0a146a245d docs: refactor README 2026-05-24 17:52:47 +08:00
zhayujie
bd85fee7d7 fix(models): persist explicit provider for vision and image capabilities 2026-05-23 20:43:25 +08:00
zhayujie
571897e2fd fix: modify default model in vision tool 2026-05-22 18:18:16 +08:00
zhayujie
840dabeccd fix(weixin): cap thinking messages to avoid rate-limit drops 2026-05-22 17:42:50 +08:00
6vision
2fa6343fe5 docs: add WeCom customer service (wechatcom_kf) channel guide
Add a self-deployment guide for the new `wechatcom_kf` channel under
`docs/channels/wecom-kf.mdx` in zh / en / ja, mirroring the existing
`wecom.mdx` structure. Wire each language version into the sidebar in
`docs/docs.json`.

Walks through: creating the WeCom custom app, retrieving Corp ID /
Secret (push-to-phone) / Token / EncodingAESKey, configuring `config.json`,
saving the callback URL + Enterprise Trusted IPs, binding the WeCom
Customer Service account, and distributing the access link / QR code.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 21:32:11 +08:00
6vision
06b84225a1 docs(wechatcom_kf): tidy README and hide cursor dir from config
- Clarify Secret retrieval (must tap "查看" on admin's phone, not copy)
- Update WeCom customer-service binding section to point to the
  "接入链接" UI (copy link / generate QR code)
- Drop developer-only asides (wechatcomapp_secret / port collision
  notes, internal sections about cursor persistence, channel runtime
  differences, multi-kf-account support)
- Stop exposing `wechatcom_kf_cursor_dir` as a user config; cursor file
  is now fixed under `tmp/`, which is an internal implementation detail.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 21:08:52 +08:00
6vision
5b31da335d fix(wechatcom_kf): use plain WeChatClient to fix 40014 & token log spam
- Switch from the local `WechatComAppClient` (whose `fetch_access_token`
  may return the raw response dict and whose background refresh loop
  re-fetches every 60s) to the stock `wechatpy.enterprise.WeChatClient`.
- Use `client.access_token` (string property) when building sync_msg /
  send_msg URLs; the previous `client.fetch_access_token()` call could
  interpolate a dict into the URL and yield errcode 40014.
- Always skip historical messages on first start; drop the
  `wechatcom_kf_skip_history_on_first_start` config — there is no real
  case for replaying up to 14 days of history.
- Change default callback port from 9899 to 9888.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 20:43:06 +08:00
6vision
11d92bb22a feat(channel): add WeCom customer service (wechatcom_kf) channel
Introduce a new channel that integrates with WeCom Customer Service
(微信客服), separate from the existing self-built WeCom app channel.

- Register channel type `wechatcom_kf` in factory, app loader and const
- Add config keys for token / secret / aes_key / port / cursor dir and
  the first-start history-skip switch; also expose corresponding env vars
- Implement channel, message and cursor store under channel/wechatcom_kf/

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 19:58:47 +08:00
zhayujie
3bc6e89b74 feat: cow desktop first version 2026-03-21 16:11:05 +08:00
521 changed files with 58499 additions and 12324 deletions

View File

@@ -1,131 +1,46 @@
name: Bug report 🐛
description: 项目运行中遇到的Bug或问题。
description: Report a bug or unexpected behavior.
title: "[Bug] "
labels: ['status: needs check']
body:
- type: markdown
attributes:
value: |
### ⚠️ 前置确认
1. 网络能够访问openai接口
2. python 已安装:版本在 3.7 ~ 3.10 之间
3. `git pull` 拉取最新代码
4. 执行`pip3 install -r requirements.txt`,检查依赖是否满足
5. 拓展功能请执行`pip3 install -r requirements-optional.txt`,检查依赖是否满足
6. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
> 💡 English is recommended so global developers can help. 推荐使用英文提交,谢谢 ❤️
- type: checkboxes
attributes:
label: 前置确认
label: Self check
options:
- label: 我确认我运行的是最新版本的代码,并且安装了所需的依赖,在[FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs)中也未找到类似问题。
- label: I'm on the latest version and searched [existing issues](https://github.com/zhayujie/CowAgent/issues) (incl. closed) — no duplicate.
required: true
- type: checkboxes
- type: textarea
attributes:
label: ⚠️ 搜索issues中是否已存在类似问题
description: >
请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框,搜索你的问题
或相关日志的关键词来查找是否存在类似问题。
options:
- label: 我已经搜索过issues和disscussions没有跟我遇到的问题相关的issue
required: true
- type: markdown
attributes:
value: |
请在上方的`title`中填写你对你所遇到问题的简略总结,这将帮助其他人更好的找到相似问题,谢谢❤️。
- type: dropdown
attributes:
label: 操作系统类型?
description: >
请选择你运行程序的操作系统类型。
options:
- Windows
- Linux
- MacOS
- Docker
- Railway
- Windows Subsystem for Linux (WSL)
- Other (请在问题中说明)
validations:
required: true
- type: dropdown
attributes:
label: 运行的python版本是?
description: |
请选择你运行程序的`python`版本。
注意:在`python 3.7`中,有部分可选依赖无法安装。
经过长时间的观察,我们认为`python 3.8`是兼容性最好的版本。
`python 3.7`~`python 3.10`以外版本的issue将视情况直接关闭。
options:
- python 3.7
- python 3.8
- python 3.9
- python 3.10
- other
validations:
required: true
- type: dropdown
attributes:
label: 使用的chatgpt-on-wechat版本是?
description: |
请确保你使用的是 [releases](https://github.com/zhayujie/chatgpt-on-wechat/releases) 中的最新版本。
如果你使用git, 请使用`git branch`命令来查看分支。
options:
- Latest Release
- Master (branch)
validations:
required: true
- type: dropdown
attributes:
label: 运行的`channel`类型是?
description: |
请确保你正确配置了该`channel`所需的配置项,所有可选的配置项都写在了[该文件中](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py),请将所需配置项填写在根目录下的`config.json`文件中。
options:
- wechatmp(公众号, 订阅号)
- wechatmp_service(公众号, 服务号)
- terminal
- other
label: Environment
description: "Version (`cow status`), OS, Python version, install method, model & channel."
placeholder: |
Version: v1.2.0
OS: macOS / Linux / Windows / Docker
Python: 3.11
Install: installer / Docker / source
Model & channel: deepseek-v4-flash, web
validations:
required: true
- type: textarea
attributes:
label: 复现步骤 🕹
description: |
**⚠️ 不能复现将会关闭issue.**
- type: textarea
attributes:
label: 问题描述 😯
description: 详细描述出现的问题,或提供有关截图。
- type: textarea
attributes:
label: 终端日志 📒
description: |
在此处粘贴终端日志,可在主目录下`run.log`文件中找到这会帮助我们更好的分析问题注意隐去你的API key。
如果在配置文件中加入`"debug": true`,打印出的日志会更有帮助。
label: What happened?
description: "Steps to reproduce, what you expected, and what happened instead. Screenshots welcome."
placeholder: |
1. ...
2. ...
<details>
<summary><i>示例</i></summary>
```log
[DEBUG][2023-04-16 00:23:22][plugin_manager.py:157] - Plugin SUMMARY triggered by event Event.ON_HANDLE_CONTEXT
[DEBUG][2023-04-16 00:23:22][main.py:221] - [Summary] on_handle_context. content: $总结前100条消息
[DEBUG][2023-04-16 00:23:24][main.py:240] - [Summary] limit: 100, duration: -1 seconds
[ERROR][2023-04-16 00:23:24][chat_channel.py:244] - Worker return exception: name 'start_date' is not defined
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 132, in _handle
reply = self._generate_reply(context)
File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 142, in _generate_reply
e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {
File "D:\project\chatgpt-on-wechat\plugins\plugin_manager.py", line 159, in emit_event
instance.handlers[e_context.event](e_context, *args, **kwargs)
File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 255, in on_handle_context
records = self._get_records(session_id, start_time, limit)
File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 96, in _get_records
c.execute("SELECT * FROM chat_records WHERE sessionid=? and timestamp>? ORDER BY timestamp DESC LIMIT ?", (session_id, start_date, limit))
NameError: name 'start_date' is not defined
[INFO][2023-04-16 00:23:36][app.py:14] - signal 2 received, exiting...
```
</details>
value: |
```log
<此处粘贴终端日志>
```
Expected: ...
Actual: ...
validations:
required: true
- type: textarea
attributes:
label: Logs
description: "Relevant logs from `run.log` (set `\"debug\": true` for more detail). ⚠️ Redact your API keys."
render: shell
validations:
required: false

View File

@@ -1,28 +1,33 @@
name: Feature request 🚀
description: 提出你对项目的新想法或建议。
description: Suggest a new idea or improvement.
title: "[Feature] "
labels: ['status: needs check']
body:
- type: markdown
attributes:
value: |
请在上方的`title`中填写简略总结,谢谢❤️
> 💡 English is recommended so global developers can help. 推荐使用英文提交,谢谢 ❤️
- type: checkboxes
attributes:
label: ⚠️ 搜索是否存在类似issue
description: >
请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框搜索关键词查找是否存在相似issue。
label: Self check
options:
- label: 我已经搜索过issues和disscussions没有发现相似issue
- label: I searched [existing issues](https://github.com/zhayujie/CowAgent/issues) (incl. closed) — no duplicate.
required: true
- type: textarea
attributes:
label: 总结
description: 描述feature的功能。
label: What's the problem?
description: "The pain point or what's not working for you right now."
validations:
required: true
- type: textarea
attributes:
label: 举例
description: 提供聊天示例,草图或相关网址。
- type: textarea
label: What would you like?
description: "How you'd expect it to work. Examples, sketches, or links welcome."
validations:
required: false
- type: checkboxes
attributes:
label: 动机
description: 描述你提出该feature的动机比如没有这项feature对你的使用造成了怎样的影响。 请提供更详细的场景描述,这可能会帮助我们发现并提出更好的解决方案。
label: Contribution
options:
- label: I'd be interested in helping implement this.
required: false

5
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: 📖 Documentation
url: https://docs.cowagent.ai
about: Setup guides, configuration, and FAQ.

22
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,22 @@
<!--
Thanks for your contribution! Please write this PR in English.
推荐使用英文填写,感谢 ❤️
-->
## What does this PR do?
<!-- A short description of the change and why it's needed. -->
## Type of change
- [ ] Bug fix
- [ ] New feature
- [ ] Docs
- [ ] Refactor / chore
## Checklist
- [ ] I have read the [Contributing Guide](https://github.com/zhayujie/CowAgent/blob/master/CONTRIBUTING.md)
- [ ] I tested this change locally
- [ ] Code comments and docs are in English
- [ ] Linked related issue (if any): closes #

116
.github/scripts/register-releases.mjs vendored Normal file
View File

@@ -0,0 +1,116 @@
// Build the D1 upsert SQL for a desktop release from the files in a directory.
//
// Each mac release has TWO artifacts that map to a SINGLE D1 row:
// - <name>-<arch>.dmg -> manual download (filename / size / sha512)
// - <name>-<arch>.zip -> auto-update (update_filename / update_size /
// update_sha512)
// electron-updater's MacUpdater can only consume a zip, never a dmg, so the
// feed serves the zip while the website serves the dmg. Windows has only the
// .exe (stored in the main columns; it's both the download and the update).
//
// We emit ONE `INSERT OR REPLACE` per (version, platform) carrying BOTH halves,
// because two replaces on the same primary key would drop whichever came first.
//
// Usage:
// node register-releases.mjs --dir dist --version 1.2.0 \
// --sql out.sql [--latest]
//
// --latest mark these rows is_latest=1 AND clear the previous latest for
// each platform (used by the publish/promote workflow). Without it
// rows are written unpublished (is_latest=0) — the build stage.
//
// sha512 is base64 (the exact format electron-updater validates).
import { execSync } from 'node:child_process'
import fs from 'node:fs'
function arg(name, fallback = undefined) {
const i = process.argv.indexOf(`--${name}`)
if (i === -1) return fallback
const next = process.argv[i + 1]
// Boolean flag (no value or next token is another flag).
if (next === undefined || next.startsWith('--')) return true
return next
}
const dir = arg('dir', 'dist')
const version = arg('version')
const sqlPath = arg('sql', 'd1.sql')
const makeLatest = arg('latest', false) === true
if (!version) {
console.error('register-releases: --version is required')
process.exit(1)
}
const sha512 = (f) =>
execSync(`openssl dgst -sha512 -binary "${f}" | openssl base64 -A`, {
shell: '/bin/bash',
})
.toString()
.trim()
// SQL-escape single quotes (base64/keys shouldn't contain them, but be safe).
const q = (s) => String(s).replace(/'/g, "''")
// platform -> { main: {key,size,sha}, upd: {key,size,sha} }
const rows = {}
for (const base of fs.readdirSync(dir)) {
const f = `${dir}/${base}`
if (fs.statSync(f).isDirectory()) continue
let platform
let slot
if (/arm64\.dmg$/.test(base)) {
platform = 'mac-arm64'
slot = 'main'
} else if (/x64\.dmg$/.test(base)) {
platform = 'mac-x64'
slot = 'main'
} else if (/arm64\.zip$/.test(base)) {
platform = 'mac-arm64'
slot = 'upd'
} else if (/x64\.zip$/.test(base)) {
platform = 'mac-x64'
slot = 'upd'
} else if (/\.exe$/.test(base)) {
platform = 'win'
slot = 'main'
} else {
console.log('Skipping unrecognized artifact:', base)
continue
}
rows[platform] ||= {}
rows[platform][slot] = {
key: `v${version}/${base}`,
size: fs.statSync(f).size,
sha: sha512(f),
}
}
if (Object.keys(rows).length === 0) {
console.error('register-releases: no recognized artifacts in', dir)
process.exit(1)
}
const isLatest = makeLatest ? 1 : 0
const sql = []
for (const [platform, r] of Object.entries(rows)) {
const m = r.main || { key: '', size: 0, sha: '' }
const u = r.upd || { key: '', size: 0, sha: '' }
if (makeLatest) {
// Clear the previous latest for this platform before promoting the new row.
sql.push(`UPDATE releases SET is_latest = 0 WHERE platform = '${platform}';`)
}
sql.push(
`INSERT OR REPLACE INTO releases ` +
`(version, platform, filename, size, sha512, update_filename, update_size, update_sha512, is_latest) ` +
`VALUES ('${version}', '${platform}', '${q(m.key)}', ${m.size}, '${q(m.sha)}', ` +
`'${q(u.key)}', ${u.size}, '${q(u.sha)}', ${isLatest});`
)
}
fs.writeFileSync(sqlPath, sql.join('\n') + '\n')
console.log(`register-releases: wrote ${sql.length} statement(s) to ${sqlPath}`)

154
.github/workflows/publish-desktop.yml vendored Normal file
View File

@@ -0,0 +1,154 @@
name: Publish Desktop
# STAGE 3 of the decoupled release pipeline: PROMOTE a built + notarized version
# to "live". By this point:
# - stage 1 (Release Desktop) built the installers, mirrored them to R2, and
# registered them in D1 as unpublished (is_latest=0);
# - stage 2 (local desktop/build/notarize-dmg.sh) notarized + stapled the mac
# dmgs and re-uploaded the stapled bytes to R2.
#
# This workflow, triggered manually with the version to publish:
# 1. pulls every artifact for that version back from R2,
# 2. recomputes sha512 from the real (stapled) bytes and updates D1,
# 3. flips is_latest=1 for that version (clearing the previous latest per
# platform) UNLESS it's a pre-release, which is recorded but never latest,
# 4. creates/updates the GitHub Release and attaches the installers.
#
# Run only after the mac dmgs for this version are notarized + re-uploaded.
on:
workflow_dispatch:
inputs:
version:
description: "Version to publish (e.g. 1.2.0). Must already be built + notarized."
type: string
required: true
make_latest:
description: "Mark this version as latest on the site (uncheck for a dry re-hash only)."
type: boolean
default: true
github_release:
description: "Create/update the GitHub Release and attach installers."
type: boolean
default: true
permissions:
contents: write
jobs:
publish:
name: Publish ${{ github.event.inputs.version }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Guard on Cloudflare secrets
id: guard
env:
CF_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
run: |
if [ -z "$CF_TOKEN" ]; then
echo "::error::CLOUDFLARE_API_TOKEN not set — cannot publish."
exit 1
fi
- name: Resolve version artifacts from D1
id: rows
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
VER: ${{ github.event.inputs.version }}
run: |
# The build stage inserted one row per artifact with filename = v<VER>/<base>.
# Read them back so we know exactly which objects to pull from R2.
out="$(npx --yes wrangler@latest d1 execute cow-desktop --remote --json \
--command "SELECT platform, filename, update_filename FROM releases WHERE version = '${VER}';")"
echo "$out"
echo "$out" | node -e '
const fs = require("fs");
const data = JSON.parse(fs.readFileSync(0, "utf8"));
const rows = (Array.isArray(data) ? data : [data])
.flatMap(r => (r.results || []));
if (!rows.length) {
console.error("No D1 rows for this version — did stage 1 (build) run?");
process.exit(1);
}
fs.writeFileSync(process.env.GITHUB_OUTPUT, "count=" + rows.length + "\n", { flag: "a" });
fs.writeFileSync("rows.json", JSON.stringify(rows));
'
- name: Download version artifacts from R2
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
R2_BUCKET: ${{ vars.R2_BUCKET != '' && vars.R2_BUCKET || 'cow-skills' }}
run: |
mkdir -p dist
# Pull BOTH the manual-download file (filename: dmg/exe) and the mac
# auto-update file (update_filename: zip) for every row. Keys are
# "v<VER>/<base>"; the R2 key is "desktop/<key>".
for key in $(node -e 'JSON.parse(require("fs").readFileSync("rows.json")).forEach(r => { if (r.filename) console.log(r.filename); if (r.update_filename) console.log(r.update_filename); })'); do
base="$(basename "$key")"
r2key="desktop/${key}"
echo "==> Downloading r2://${R2_BUCKET}/${r2key} -> dist/${base}"
npx --yes wrangler@latest r2 object get "${R2_BUCKET}/${r2key}" \
--file "dist/${base}" --remote
done
echo "Downloaded:"; ls -la dist
- name: Reminder — mac dmgs must be notarized before publishing
run: |
# Stapling can only be validated on macOS (xcrun stapler validate),
# which this Linux runner doesn't have. The authoritative check runs in
# stage 2 (desktop/build/notarize-dmg.sh) before re-uploading to R2.
# This step is just a loud reminder in the log.
echo "::notice::Publishing assumes the mac dmgs pulled from R2 are already notarized + stapled (stage 2). If you skipped stage 2, users will hit Gatekeeper warnings."
ls -la dist/*.dmg 2>/dev/null || echo "(no dmg in this version — win-only publish)"
- name: Update D1 (recompute sha512 + set latest)
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
VER: ${{ github.event.inputs.version }}
MAKE_LATEST: ${{ github.event.inputs.make_latest }}
run: |
# Pre-releases (e.g. 1.2.0-beta / -rc.1 / -test) are recorded but never
# become latest, so the site keeps serving the last stable build.
case "$VER" in
*-*) is_pre=1 ;;
*) is_pre=0 ;;
esac
if [ "$MAKE_LATEST" = "true" ] && [ "$is_pre" = "0" ]; then
latest_flag="--latest"; echo "==> Publishing $VER as latest."
else
latest_flag=""; echo "==> Publishing $VER without latest flag (pre-release or dry re-hash)."
fi
# Re-hash the real (stapled) bytes and re-store every row with both the
# dmg (manual) and mac zip (auto-update) columns. Same script as the
# build stage; --latest also clears the previous latest per platform.
node .github/scripts/register-releases.mjs --dir dist --version "$VER" --sql d1.sql $latest_flag
echo "==> D1 statements:"; cat d1.sql
npx --yes wrangler@latest d1 execute cow-desktop --remote --file d1.sql
- name: Create/update GitHub Release and attach installers
if: github.event.inputs.github_release == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
VER: ${{ github.event.inputs.version }}
run: |
tag="v${VER}"
case "$VER" in
*-*) prerelease="--prerelease" ;;
*) prerelease="" ;;
esac
if ! gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
gh release create "$tag" --repo "$GITHUB_REPOSITORY" \
--title "$tag" --generate-notes $prerelease
fi
# --clobber so re-runs overwrite the stapled/updated assets. The mac
# zip is the auto-update artifact; attach it too so the GitHub Release
# is a complete mirror (nullglob avoids errors when a type is absent).
shopt -s nullglob
gh release upload "$tag" dist/*.dmg dist/*.zip dist/*.exe \
--repo "$GITHUB_REPOSITORY" --clobber

336
.github/workflows/release.yml vendored Normal file
View File

@@ -0,0 +1,336 @@
name: Release Desktop
# STAGE 1 of the decoupled release pipeline: BUILD ONLY.
# Builds the desktop client for macOS (arm64 + x64) and Windows (x64), mirrors
# the installers to R2, and registers them in D1 as UNPUBLISHED (is_latest=0)
# so the website keeps serving the previous release. It does NOT notarize
# (Apple's notary service stalls this large bundle for hours) and does NOT
# create a GitHub Release.
#
# Full flow:
# 1. (this workflow) build + upload to R2 + D1 as unpublished.
# 2. (local) download the mac dmgs, run desktop/build/notarize-dmg.sh to
# notarize + staple + re-upload the stapled dmgs to R2.
# 3. (Publish Desktop workflow) flip D1 is_latest=1 and attach GitHub
# Release assets — makes the version live on the site.
#
# Manual only: run stage 1 via workflow_dispatch. Tag pushes do NOT trigger a
# build, so cutting a release tag never rebuilds installers or overwrites R2.
on:
workflow_dispatch:
inputs:
version:
description: "Version to stamp (e.g. 1.0.0-test). Used for package.json and R2 path."
type: string
default: "0.0.0-dev"
publish_r2:
description: "Upload installers to R2 + register in D1 (needs Cloudflare secrets)"
type: boolean
default: false
permissions:
contents: write
jobs:
build:
name: Build ${{ matrix.name }}
runs-on: ${{ matrix.os }}
strategy:
# Don't cancel the other platforms if one fails — we want to see all
# failures in a single run.
fail-fast: false
matrix:
include:
- name: macOS arm64
os: macos-14
platform: mac
arch: arm64
eb_flags: --mac --arm64
- name: macOS x64
os: macos-15-intel
platform: mac
arch: x64
eb_flags: --mac --x64
- name: Windows x64
os: windows-latest
platform: win
arch: x64
eb_flags: --win --x64
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Derive version
# Tag push: strip the leading "v" from GITHUB_REF_NAME (e.g. v1.2.0).
# Manual dispatch: use the provided version input.
id: ver
shell: bash
run: |
if [ "${{ github.event_name }}" = "push" ]; then
ref="${GITHUB_REF_NAME:-}"
echo "version=${ref#v}" >> "$GITHUB_OUTPUT"
else
echo "version=${{ github.event.inputs.version }}" >> "$GITHUB_OUTPUT"
fi
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "20"
- name: Build Python backend (PyInstaller)
shell: bash
run: |
python -m pip install --upgrade pip
pip install -r desktop/build/requirements-desktop.txt
pip install pyinstaller
# Run from repo root so the spec's relative datas resolve correctly.
pyinstaller desktop/build/cowagent-backend.spec \
--noconfirm \
--distpath desktop/build/dist \
--workpath desktop/build/build-work
- name: Install desktop deps
working-directory: desktop
run: npm ci
- name: Write version into package.json
working-directory: desktop
shell: bash
run: npm version "${{ steps.ver.outputs.version }}" --no-git-tag-version --allow-same-version
# Compile renderer + main in its OWN step, alone, so the npm.cmd batch
# wrapper (see the note on the build step below) can't take out anything
# after it.
- name: Compile (vite + tsc)
working-directory: desktop
shell: bash
run: npm run build
# Download the Windows signing CLI. The URL comes from a repo variable, so
# nothing about the signing setup is hardcoded in a public workflow. Only
# runs on the Windows leg and only when a URL is set; otherwise the build
# stays unsigned. SIGNTOOL_PATH is exported for the next step's
# electron-builder.win.js to invoke.
- name: Download Windows signing CLI
if: matrix.platform == 'win' && vars.SIGNTOOL_CLI_URL != ''
shell: bash
env:
SIGNTOOL_CLI_URL: ${{ vars.SIGNTOOL_CLI_URL }}
run: |
mkdir -p "$RUNNER_TEMP/signtool"
curl -fsSL "$SIGNTOOL_CLI_URL" -o "$RUNNER_TEMP/signtool/cli.zip"
# Unzip and locate the signtool executable regardless of nesting.
unzip -o "$RUNNER_TEMP/signtool/cli.zip" -d "$RUNNER_TEMP/signtool" >/dev/null
exe="$(find "$RUNNER_TEMP/signtool" -type f -iname 'signtool*.exe' | head -n1)"
if [ -z "$exe" ]; then
echo "signtool.exe not found in downloaded archive" >&2
find "$RUNNER_TEMP/signtool" -type f >&2
exit 1
fi
# Normalize to a Windows-style path for execFileSync in Node.
echo "SIGNTOOL_PATH=$(cygpath -w "$exe")" >> "$GITHUB_ENV"
echo "resolved signtool: $exe"
- name: Build & publish (electron-builder)
working-directory: desktop
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Signing secrets are passed through as-is; we only export them to the
# environment below when non-empty. An empty CSC_LINK would make
# electron-builder try to load a bogus certificate and fail, so unset
# is the correct state for unsigned builds.
MAC_CSC_LINK: ${{ secrets.MAC_CSC_LINK }}
MAC_CSC_KEY_PASSWORD: ${{ secrets.MAC_CSC_KEY_PASSWORD }}
# Windows code signing via the signing CLI. Credentials are
# secrets; SIGNTOOL_PATH was exported by the download step above.
# COW_SIGN_DRY_RUN (repo variable) lets us validate the whole pipeline
# with a self-signed cert before buying a real one — no quota used.
SIGNTOOL_ACCESS_KEY: ${{ secrets.SIGNTOOL_ACCESS_KEY }}
SIGNTOOL_ACCESS_SECRET: ${{ secrets.SIGNTOOL_ACCESS_SECRET }}
SIGNTOOL_CERT_CODE: ${{ secrets.SIGNTOOL_CERT_CODE }}
COW_SIGN_DRY_RUN: ${{ vars.COW_SIGN_DRY_RUN }}
run: |
# Pick the signing cert for THIS platform only. The mac and win secrets
# are both present in the job env, but a mac cert must never leak into a
# Windows build (electron-builder would try to load it and fail), and
# vice versa. electron-builder reads a single CSC_LINK/CSC_KEY_PASSWORD
# pair, so we set it per-platform. An empty CSC_LINK is treated by
# electron-builder as a broken cert path, so we leave it entirely unset
# for an unsigned build.
#
# NOTE: we only ever `export`, never `unset`, GitHub-injected env vars
# (an `unset` can return non-zero and abort under errexit).
# macOS keeps the classic CSC_LINK (.p12) flow. Windows no longer uses
# a local .pfx (EV private keys can't be exported since 2023); it signs
# via the CLI wired into electron-builder.win.js instead, using the
# SIGNTOOL_* env already set above — nothing to export here.
case "${{ matrix.platform }}" in
mac)
if [ -n "$MAC_CSC_LINK" ]; then
export CSC_LINK="$MAC_CSC_LINK"
export CSC_KEY_PASSWORD="$MAC_CSC_KEY_PASSWORD"
fi
;;
esac
# Never let electron-builder publish: our publish target is a generic
# (read-only) feed served from R2/D1, which it can't upload to. We mirror
# installers to R2 and register them in D1 ourselves (publish-r2 job).
# `--publish never` still emits the latest*.yml files.
#
# CONFIG PER PLATFORM: each platform loads its OWN dynamic config.
# mac -> electron-builder.js (injects mac.binaries for signing)
# win -> electron-builder.win.js (wires the sign hook; electron-builder
# signs the app, backend and installer)
# HISTORY: passing --config on Windows previously broke the build (no
# installer, job still green). That happened because the MAC config
# (electron-builder.js) was a no-op on Windows yet still disturbed the
# run. The fix is a DEDICATED win config that correctly extends
# config.win — not sharing the mac one. If a build ever runs WITHOUT
# signing configured, electron-builder.win.js still returns the base
# config unchanged (sign hook just skips), so the installer is still
# produced.
#
# Invoke via `node <cli.js>` rather than `npx`: on Windows `npx` is
# npx.cmd (a batch wrapper) and running it from this Git Bash step can
# make bash return before the wrapped process finishes. node skips it.
case "${{ matrix.platform }}" in
mac) config_arg="--config electron-builder.js" ;;
win) config_arg="--config electron-builder.win.js" ;;
*) config_arg="" ;;
esac
node node_modules/electron-builder/cli.js ${{ matrix.eb_flags }} $config_arg --publish never
# Upload artifacts regardless of outcome, so a failed run still surfaces
# the built installers (and, on success, the notarized+stapled dmg).
- name: Upload artifacts
if: always()
uses: actions/upload-artifact@v4
with:
# One bundle per platform/arch so the publish job can collect them all.
name: cowagent-${{ matrix.platform }}-${{ matrix.arch }}
path: |
desktop/release/*.dmg
desktop/release/*.zip
desktop/release/*.exe
desktop/release/*.yml
desktop/release/*.blockmap
if-no-files-found: ignore
retention-days: 7
# Mirror the release installers to R2 (CDN-backed) and register them in D1 so
# cowagent.ai/download/{platform}/latest can resolve and count downloads.
# Runs only on tag pushes, and is a no-op (skips) until the Cloudflare secrets
# are configured, so it never blocks unsigned/dry builds.
publish-r2:
name: Publish to R2 + D1
# Require every platform in the build matrix to succeed before publishing,
# so a release on R2/D1 is always complete (all installers present) rather
# than partial. needs: build already gates on all matrix jobs succeeding.
needs: build
runs-on: ubuntu-latest
# Run on a tag push, or on a manual dispatch when publish_r2 is checked.
if: >-
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) ||
(github.event_name == 'workflow_dispatch' && github.event.inputs.publish_r2 == 'true')
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Guard on Cloudflare secrets
id: guard
env:
CF_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
run: |
if [ -n "$CF_TOKEN" ]; then
echo "enabled=true" >> "$GITHUB_OUTPUT"
else
echo "enabled=false" >> "$GITHUB_OUTPUT"
echo "::notice::CLOUDFLARE_API_TOKEN not set — skipping R2/D1 publish."
fi
- name: Derive version
if: steps.guard.outputs.enabled == 'true'
id: ver
run: |
if [ "${{ github.event_name }}" = "push" ]; then
echo "version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
else
echo "version=${{ github.event.inputs.version }}" >> "$GITHUB_OUTPUT"
fi
- name: Download all build artifacts
if: steps.guard.outputs.enabled == 'true'
uses: actions/download-artifact@v4
with:
path: artifacts
- name: Stage installers
if: steps.guard.outputs.enabled == 'true'
id: stage
run: |
mkdir -p dist
# Flatten installers + their .blockmap (used by electron-updater for
# differential downloads) from every per-platform artifact dir. The
# .yml feed is generated dynamically by the /update Function from D1,
# so the yml files themselves don't need to go to R2.
# .zip is the mac auto-update artifact (electron-updater's MacUpdater
# can ONLY consume zip, not dmg — the dmg is for manual downloads).
find artifacts -type f \( -name '*.dmg' -o -name '*.zip' -o -name '*.exe' -o -name '*.blockmap' \) -exec cp {} dist/ \;
echo "Staged files:"; ls -la dist
# When the whole matrix failed there's nothing to publish; flag it so
# the R2/D1 steps skip instead of writing an empty/partial release.
if [ -n "$(ls -A dist 2>/dev/null)" ]; then
echo "has_files=true" >> "$GITHUB_OUTPUT"
else
echo "has_files=false" >> "$GITHUB_OUTPUT"
echo "::warning::No installers found in any artifact — skipping R2/D1 publish."
fi
- name: Upload installers to R2
if: steps.guard.outputs.enabled == 'true' && steps.stage.outputs.has_files == 'true'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
VER: ${{ steps.ver.outputs.version }}
run: |
# Reuse the existing cow-skills bucket under a desktop/ prefix; this
# is served by the cdn.cowagent.ai custom domain.
for f in dist/*; do
base="$(basename "$f")"
key="desktop/v${VER}/${base}"
echo "==> Uploading $base -> r2://cow-skills/$key"
npx --yes wrangler@latest r2 object put "cow-skills/$key" \
--file "$f" --remote
done
- name: Register release rows in D1
if: steps.guard.outputs.enabled == 'true' && steps.stage.outputs.has_files == 'true'
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
VER: ${{ steps.ver.outputs.version }}
run: |
# This build job ALWAYS registers rows as unpublished (is_latest=0), so
# /download/<p>/latest keeps serving the previous release and the new
# version stays invisible on the site. macOS dmgs still need to be
# notarized+stapled locally (build/notarize-dmg.sh) before they're
# safe to ship. Promotion to latest happens later, only after
# notarization, via the separate "Publish Desktop" workflow.
echo "==> Registering $VER as unpublished (is_latest=0)."
# Build one upsert per (version, platform) carrying both the dmg
# (manual download) and the mac zip (auto-update) columns. See
# .github/scripts/register-releases.mjs for the mapping. No --latest
# here: rows stay unpublished until the publish workflow promotes them.
node .github/scripts/register-releases.mjs --dir dist --version "$VER" --sql d1.sql
echo "==> D1 statements:"; cat d1.sql
npx --yes wrangler@latest d1 execute cow-desktop --remote --file d1.sql

32
.github/workflows/test-windows-bash.yml vendored Normal file
View File

@@ -0,0 +1,32 @@
name: Windows Bash Streaming Tests
on:
workflow_dispatch:
pull_request:
paths:
- "agent/tools/bash/bash.py"
- "tests/test_bash_streaming.py"
- ".github/workflows/test-windows-bash.yml"
jobs:
windows-bash-tests:
runs-on: windows-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pytest
python -m pip install -r requirements.txt
- name: Run Windows Bash streaming tests
run: python -m pytest tests/test_bash_streaming.py -v

18
.gitignore vendored
View File

@@ -32,7 +32,6 @@ plugins/banwords/lib/__pycache__
!plugins/role
!plugins/keyword
!plugins/linkai
!plugins/agent
!plugins/cow_cli
client_config.json
ref/
@@ -46,3 +45,20 @@ dist/
build/
*.egg-info/
.cow.pid
# Desktop backend packaging: keep the source files (spec/requirements/script)
# tracked even though the generic build/ rule above ignores them, but never
# track the build outputs or local venv.
!desktop/build/
desktop/build/*
!desktop/build/cowagent-backend.spec
!desktop/build/requirements-desktop.txt
!desktop/build/build-backend.sh
!desktop/build/entitlements.mac.plist
!desktop/build/notarize-dmg.sh
# Icon authoring scratch dir: intermediate assets used to produce the final
# icons. Only the finished icons under desktop/resources/ should be committed.
desktop/resources/.icon-work/
.wrangler/

61
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,61 @@
# Contributing to CowAgent
Thanks for taking the time to contribute! 🎉 CowAgent is built by a global
community, and contributions of all sizes are welcome — from typo fixes to new
features.
## Language policy
To keep the project accessible to a global community, **please write issues,
pull requests, code comments, and commit messages in English.**
> 为方便全球开发者协作,请尽量使用**英文**提交 issue、PR、代码注释与
> commit message。不必担心英文不完美——表达清楚即可工具翻译也完全没问题。感谢理解 ❤️
## Reporting issues
Found a bug or have an idea? [Open an issue](https://github.com/zhayujie/CowAgent/issues/new/choose).
Before opening one, please search existing issues (including closed ones) to
avoid duplicates, and make sure you're on the latest version.
## Submitting a pull request
1. **Fork** the repo and create a branch from `master`
(e.g. `feat/web-search`, `fix/telegram-reconnect`).
2. Make your change. Keep it focused — one logical change per PR.
3. Follow the existing code style. Write comments and docstrings in English.
4. Run the app locally to confirm your change works.
5. Open a PR with a clear title and a short description of **what** and **why**.
We keep the bar friendly: clear, focused, and working is enough. Maintainers are
happy to help polish details during review.
### Commit & PR titles
Use a short, imperative summary. The [Conventional Commits](https://www.conventionalcommits.org/)
style is preferred but not required:
```
feat: add web search tool
fix: reconnect Telegram websocket on timeout
docs: clarify Docker setup
```
## Development setup
See the [Install from Source](https://docs.cowagent.ai/guide/manual-install)
guide. In short:
```bash
git clone https://github.com/zhayujie/CowAgent.git
cd CowAgent
pip install -r requirements.txt
pip install -e .
cow start
```
## Code of conduct
Be respectful and constructive. We want CowAgent to be a welcoming place for
everyone.

1069
README.md

File diff suppressed because it is too large Load Diff

View File

@@ -49,6 +49,16 @@ class ChatService:
agent.model.channel_type = channel_type or ""
agent.model.session_id = session_id or ""
# Build a context so context-aware tools (e.g. scheduler) can resolve the
# receiver/session. This streaming path bypasses agent_bridge.agent_reply,
# so the attach step that normally happens there must be done here too.
context = self._build_context(query, session_id, channel_type)
self._attach_context_aware_tools(agent, context)
# Mark this session as mid-run so the self-evolution idle scan does not
# fire concurrently when a single turn runs longer than idle_minutes.
self._mark_run_active(agent, True)
# State shared between the event callback and this method
state = _StreamState()
@@ -171,6 +181,12 @@ class ChatService:
from agent.protocol.agent_stream import AgentStreamExecutor
# Register a cancel token so /cancel can abort this in-flight run.
# IM channels key on session_id (no per-turn request_id here).
from agent.protocol import get_cancel_registry
registry = get_cancel_registry()
cancel_event = registry.register(session_id, session_id=session_id) if session_id else None
executor = AgentStreamExecutor(
agent=agent,
model=agent.model,
@@ -180,6 +196,7 @@ class ChatService:
on_event=on_event,
messages=messages_copy,
max_context_turns=max_context_turns,
cancel_event=cancel_event,
)
try:
@@ -191,6 +208,15 @@ class ChatService:
agent.messages.clear()
logger.info("[ChatService] Cleared agent message history after executor recovery")
raise
finally:
# Clear the mid-run flag so idle scans can review this session again.
self._mark_run_active(agent, False)
# Release cancel token to keep the registry bounded.
if session_id:
try:
registry.unregister(session_id)
except Exception:
pass
# Sync executor messages back to agent (thread-safe).
# The executor may have trimmed context, making its list shorter than
@@ -254,10 +280,68 @@ class ChatService:
# Execute post-process tools
agent._execute_post_process_tools()
# Record this user turn for the self-evolution idle trigger. This
# streaming path bypasses agent_bridge.agent_reply, so the activity must
# be noted here, otherwise idle scans never see any signal to evolve.
self._note_evolution_turn(agent, context)
logger.info(f"[ChatService] Agent run completed: session={session_id}")
@staticmethod
def _build_context(query: str, session_id: str, channel_type: str):
"""Build a Context for tool resolution on the streaming chat path.
receiver falls back to session_id; the scheduler's delivery keys on
session_id as the receiver.
"""
from bridge.context import Context, ContextType
# Pass an explicit kwargs dict: Context's default kwargs is a shared
# mutable default, so omitting it would leak fields across sessions.
ctx = Context(ContextType.TEXT, query, kwargs={})
ctx["session_id"] = session_id
ctx["receiver"] = session_id
ctx["isgroup"] = False
ctx["channel_type"] = channel_type or ""
return ctx
@staticmethod
def _attach_context_aware_tools(agent, context):
"""Attach the current context to tools that need it (scheduler)."""
try:
if not (context and getattr(agent, "tools", None)):
return
for tool in agent.tools:
if tool.name == "scheduler":
from agent.tools.scheduler.integration import attach_scheduler_to_tool
attach_scheduler_to_tool(tool, context)
break
except Exception as e:
logger.warning(f"[ChatService] Failed to attach context to scheduler: {e}")
@staticmethod
def _mark_run_active(agent, active):
"""Toggle the self-evolution mid-run flag for this session's agent."""
try:
from agent.evolution.trigger import mark_run_active
mark_run_active(agent, active)
except Exception:
pass
@staticmethod
def _note_evolution_turn(agent, context):
"""Record a user turn so the self-evolution idle trigger has signal."""
try:
from agent.evolution.trigger import note_user_turn
ch = (context.get("channel_type") or "") if context else ""
rcv = (context.get("receiver") or "") if context else ""
is_group = bool(context.get("isgroup")) if context else False
# Only single chats get a proactive push target; group push is noisy.
note_user_turn(agent, channel_type=ch, receiver=(rcv if not is_group else ""))
except Exception:
pass
@staticmethod
def _persist_messages(session_id: str, new_messages: list, channel_type: str = ""):
try:

View File

@@ -0,0 +1,19 @@
"""
Self-evolution subsystem for CowAgent.
Runs a lightweight, isolated review pass after a conversation goes idle to
decide whether anything is worth durably learning (memory / skill) or whether
an unfinished task can be pushed forward. Conservative by design: most
conversations should produce no change at all.
Public entry points:
from agent.evolution import get_evolution_config
from agent.evolution.trigger import start_evolution_trigger, note_user_turn
"""
from agent.evolution.config import EvolutionConfig, get_evolution_config
__all__ = [
"EvolutionConfig",
"get_evolution_config",
]

102
agent/evolution/backup.py Normal file
View File

@@ -0,0 +1,102 @@
"""File backup / rollback support for self-evolution.
Before the evolution agent edits MEMORY.md or a skill file, we snapshot the
current state into ``memory/.evolution_backups/<backup_id>/`` so a later "undo"
can restore it. File-level restore only — simple and reliable.
"""
from __future__ import annotations
import json
import shutil
import time
from datetime import datetime
from pathlib import Path
from typing import List, Optional
from common.log import logger
_BACKUP_DIRNAME = ".evolution_backups"
_MANIFEST_NAME = "manifest.json"
# Keep only the most recent N backups to bound disk usage.
_MAX_BACKUPS = 10
def _backups_root(workspace_dir: Path) -> Path:
return Path(workspace_dir) / "memory" / _BACKUP_DIRNAME
def create_backup(workspace_dir: Path, files: List[Path]) -> Optional[str]:
"""Snapshot ``files`` (those that exist) under a new backup id.
Returns the backup_id, or None when there is nothing to back up.
"""
existing = [Path(f) for f in files if Path(f).exists()]
if not existing:
return None
backup_id = datetime.now().strftime("%Y%m%d-%H%M%S-") + str(int(time.time() * 1000) % 1000)
root = _backups_root(workspace_dir)
target = root / backup_id
try:
target.mkdir(parents=True, exist_ok=True)
ws = Path(workspace_dir)
manifest = []
for idx, src in enumerate(existing):
# Store under a flat index plus the relative path so restore knows
# where it came from, even for nested skill files.
try:
rel = str(src.relative_to(ws))
except ValueError:
rel = src.name
dst = target / f"{idx}.bak"
shutil.copy2(src, dst)
manifest.append({"rel": rel, "bak": f"{idx}.bak"})
(target / _MANIFEST_NAME).write_text(
json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8"
)
_prune_old_backups(root)
# Caller logs a combined backup+review line; keep this at debug.
logger.debug(f"[Evolution] Created backup {backup_id} ({len(manifest)} file(s))")
return backup_id
except Exception as e:
logger.warning(f"[Evolution] Failed to create backup: {e}")
return None
def restore_backup(workspace_dir: Path, backup_id: str) -> bool:
"""Restore all files captured under ``backup_id``. Returns success."""
if not backup_id:
return False
target = _backups_root(workspace_dir) / backup_id
manifest_path = target / _MANIFEST_NAME
if not manifest_path.exists():
logger.warning(f"[Evolution] Backup not found: {backup_id}")
return False
try:
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
ws = Path(workspace_dir)
for entry in manifest:
bak = target / entry["bak"]
dst = ws / entry["rel"]
if bak.exists():
dst.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(bak, dst)
logger.info(f"[Evolution] Restored backup {backup_id} ({len(manifest)} file(s))")
return True
except Exception as e:
logger.warning(f"[Evolution] Failed to restore backup {backup_id}: {e}")
return False
def _prune_old_backups(root: Path) -> None:
"""Drop the oldest backups beyond _MAX_BACKUPS (sorted by name = chronological)."""
try:
dirs = sorted(
[d for d in root.iterdir() if d.is_dir()],
key=lambda p: p.name,
)
for old in dirs[:-_MAX_BACKUPS]:
shutil.rmtree(old, ignore_errors=True)
except Exception as e:
logger.debug(f"[Evolution] Backup prune skipped: {e}")

76
agent/evolution/config.py Normal file
View File

@@ -0,0 +1,76 @@
"""Configuration for the self-evolution subsystem.
Reads flat ``self_evolution_*`` keys from config.json. All fields have safe
defaults so the feature degrades gracefully when keys are absent.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
# Defaults — conservative (see executor module docstring). Disabled by default
# until release; enable via ``self_evolution_enabled``.
DEFAULT_ENABLED = False
DEFAULT_IDLE_MINUTES = 10
DEFAULT_MIN_TURNS = 6
# Max review steps for the isolated evolution agent. Kept small (not exposed as
# config): the review is meant to be cheap and focused, not a long autonomous run.
DEFAULT_MAX_STEPS = 12
@dataclass
class EvolutionConfig:
"""Resolved self-evolution settings."""
enabled: bool = DEFAULT_ENABLED
idle_minutes: int = DEFAULT_IDLE_MINUTES
min_turns: int = DEFAULT_MIN_TURNS
max_steps: int = DEFAULT_MAX_STEPS
@property
def idle_seconds(self) -> int:
return max(60, self.idle_minutes * 60)
def _as_bool(value: Any, fallback: bool) -> bool:
if isinstance(value, bool):
return value
if isinstance(value, str):
v = value.strip().lower()
if v in ("true", "1", "yes", "on"):
return True
if v in ("false", "0", "no", "off"):
return False
return fallback
def _as_pos_int(value: Any, fallback: int) -> int:
try:
n = int(value)
return n if n > 0 else fallback
except (TypeError, ValueError):
return fallback
def get_evolution_config() -> EvolutionConfig:
"""Build EvolutionConfig from the live config.json ``self_evolution_*`` keys."""
try:
from config import conf
c = conf()
except Exception:
c = {}
def _get(key, default):
try:
return c.get(key, default)
except Exception:
return default
return EvolutionConfig(
enabled=_as_bool(_get("self_evolution_enabled", None), DEFAULT_ENABLED),
idle_minutes=_as_pos_int(_get("self_evolution_idle_minutes", None), DEFAULT_IDLE_MINUTES),
min_turns=_as_pos_int(_get("self_evolution_min_turns", None), DEFAULT_MIN_TURNS),
max_steps=DEFAULT_MAX_STEPS,
)

556
agent/evolution/executor.py Normal file
View File

@@ -0,0 +1,556 @@
"""Self-evolution executor.
Runs an isolated review agent over an idle conversation's transcript and, if a
clear signal is found, lets it edit memory / skills via a restricted toolset.
Conservative by design: most runs return ``[SILENT]`` and change nothing.
Flow:
1. Build a transcript from the session's new (since last pass) messages.
2. Snapshot MEMORY.md + daily file + editable skills (for undo) -> backup_id.
3. Run an isolated agent (same model, restricted tools, evolution prompt).
4. If output is [SILENT], or no workspace file actually changed -> done.
5. Otherwise -> record to the evolution log, inject an [EVOLUTION] note into
the user session (so the main agent can honor "undo"), and push the
summary to the user's channel.
Reuses existing infrastructure (AgentBridge.create_agent, ToolManager,
remember_scheduled_output, channel_factory) rather than introducing a fork.
"""
from __future__ import annotations
import re
import threading
from datetime import datetime
from pathlib import Path
from typing import List, Optional
from common.log import logger
from agent.evolution.backup import create_backup
from agent.evolution.config import get_evolution_config
from agent.evolution.prompts import (
EVOLUTION_MARKER,
EVOLUTION_SYSTEM_PROMPT,
SILENT_TOKEN,
build_review_user_message,
)
from agent.evolution.record import append_session_evolution
# Tools the isolated evolution agent is allowed to use. Everything else is
# withheld so a review pass can only read context, run workspace scripts, and
# edit memory/skill files. bash is needed by skill-creator's init script and is
# confined to the workspace by _BashWorkspaceGuard.
_ALLOWED_TOOLS = {"read", "write", "edit", "ls", "bash", "memory_search", "memory_get"}
# Cap concurrent evolution passes so a burst of idle sessions can't spawn many
# background model runs at once. Extra sessions simply wait for the next scan.
_MAX_CONCURRENT = 2
_running_lock = threading.Lock()
_running_count = 0
def _builtin_skill_names() -> set:
"""Names of skills shipped with the product (project-root ``skills/``).
These are protected: the evolution agent must never edit them, even though
a same-named copy exists in the workspace at runtime. The project dir is the
authoritative list of what counts as built-in.
"""
try:
# executor.py -> agent/evolution -> agent -> project root
project_root = Path(__file__).resolve().parents[2]
builtin_dir = project_root / "skills"
if not builtin_dir.is_dir():
return set()
names = set()
for entry in builtin_dir.iterdir():
if entry.is_dir() and not entry.name.startswith("."):
names.add(entry.name)
return names
except Exception:
return set()
def _build_transcript(messages: List[dict], max_chars: int = 12000) -> str:
"""Render the session messages into a compact text transcript."""
lines: List[str] = []
for msg in messages:
role = msg.get("role", "")
if role not in ("user", "assistant"):
continue
content = msg.get("content", "")
text = _extract_text(content)
if not text.strip():
continue
speaker = "User" if role == "user" else "Assistant"
lines.append(f"{speaker}: {text.strip()}")
transcript = "\n".join(lines)
# Keep the most RECENT context if oversized (tail is most relevant).
if len(transcript) > max_chars:
transcript = "...(earlier omitted)...\n" + transcript[-max_chars:]
return transcript
def _extract_text(content) -> str:
if isinstance(content, str):
return content
if isinstance(content, list):
parts = []
for block in content:
if isinstance(block, dict) and block.get("type") == "text":
parts.append(block.get("text", ""))
elif isinstance(block, str):
parts.append(block)
return "\n".join(parts)
return ""
def _select_tools(all_tools: list) -> list:
return [t for t in all_tools if getattr(t, "name", None) in _ALLOWED_TOOLS]
# Tools whose writes must be confined to the workspace during evolution.
_WRITE_TOOLS = {"write", "edit"}
class _WorkspaceWriteGuard:
"""Wraps a write/edit tool so it can ONLY write inside the workspace.
Hard engineering guard (not prompt-based): any write resolving outside the
workspace — e.g. the project's bundled ``skills/`` dir — is rejected. This
protects built-in skills regardless of what the model attempts.
"""
def __init__(self, inner, workspace_dir: str):
self._inner = inner
self._ws = Path(workspace_dir).resolve()
# Mirror the attributes the agent runtime reads off a tool.
self.name = inner.name
self.description = inner.description
self.params = inner.params
def __getattr__(self, item):
return getattr(self._inner, item)
def execute_tool(self, params):
# The agent runtime calls execute_tool (not execute); route it through
# our guarded execute so the path checks always run.
try:
return self.execute(params)
except Exception as e:
logger.error(f"[Evolution] guarded tool error: {e}")
from agent.tools.base_tool import ToolResult
return ToolResult.fail(f"Error: {e}")
def execute(self, args):
path = (args.get("path") or "").strip()
if path:
try:
resolved = Path(self._inner._resolve_path(path)).resolve()
from agent.tools.base_tool import ToolResult
# Confine writes to the workspace. This protects the product's
# bundled skills (which live outside the workspace) from ever
# being modified, no matter what path the model attempts.
if self._ws not in resolved.parents and resolved != self._ws:
return ToolResult.fail(
"Error: evolution may only write inside the workspace; "
f"path '{path}' is outside and was blocked."
)
except Exception:
pass
return self._inner.execute(args)
class _BashWorkspaceGuard:
"""Wraps the bash tool so evolution can only run commands inside the
workspace.
Evolution needs bash for skill-creator's init script, but it runs
unattended in the background, so a raw shell is too broad. This guard:
- forces the command to execute with cwd = workspace,
- rejects commands that reference an absolute path or ``..`` segment
pointing OUTSIDE the workspace (the common ways to escape it).
It is a coarse textual check, not a sandbox — paired with the model's
instruction to only run skill-creator scripts, it keeps writes local.
"""
def __init__(self, inner, workspace_dir: str):
self._inner = inner
self._ws = Path(workspace_dir).resolve()
# Pin the shell's working directory to the workspace.
try:
self._inner.cwd = str(self._ws)
except Exception:
pass
self.name = inner.name
self.description = inner.description
self.params = inner.params
def __getattr__(self, item):
return getattr(self._inner, item)
def execute_tool(self, params):
try:
return self.execute(params)
except Exception as e:
logger.error(f"[Evolution] guarded bash error: {e}")
from agent.tools.base_tool import ToolResult
return ToolResult.fail(f"Error: {e}")
def _escapes_workspace(self, command: str) -> bool:
# Absolute paths that are not under the workspace.
for tok in re.findall(r'(?:^|\s)(/[^\s\'";|&]+)', command):
try:
resolved = Path(tok).resolve()
except Exception:
continue
if self._ws != resolved and self._ws not in resolved.parents:
return True
# Parent-dir traversal that climbs above the workspace.
for tok in re.findall(r'[^\s\'";|&]*\.\.[^\s\'";|&]*', command):
try:
resolved = (self._ws / tok).resolve()
except Exception:
continue
if self._ws != resolved and self._ws not in resolved.parents:
return True
return False
def execute(self, args):
from agent.tools.base_tool import ToolResult
command = (args.get("command") or "").strip()
if command and self._escapes_workspace(command):
return ToolResult.fail(
"Error: evolution may only run commands inside the workspace; "
"this command references a path outside it and was blocked."
)
return self._inner.execute(args)
def _guard_tools(tools: list, workspace_dir: str) -> list:
"""Wrap write/edit/bash tools with workspace guards; leave others as-is."""
guarded = []
for t in tools:
name = getattr(t, "name", None)
if name in _WRITE_TOOLS:
guarded.append(_WorkspaceWriteGuard(t, workspace_dir))
elif name == "bash":
guarded.append(_BashWorkspaceGuard(t, workspace_dir))
else:
guarded.append(t)
return guarded
# Workspace subtrees worth watching for evolution-induced changes. AGENT.md is
# watched too: evolution may rarely refine the assistant's persona/style there.
_WATCH_SUBDIRS = ("MEMORY.md", "AGENT.md", "skills", "knowledge", "output")
# Subpaths under memory/ to ignore: evolution's own bookkeeping + the nightly
# dream diary, none of which count as a user-facing change signal.
_MEMORY_IGNORE = (".evolution_backups", "dreams", "evolution")
# Files the skill subsystem maintains automatically (the enable/disable index).
# Not an evolution result, so a rewrite must not count as a change signal.
_WATCH_IGNORE_NAMES = ("skills_config.json",)
def _workspace_snapshot(workspace_dir) -> dict:
"""Map relative path -> (mtime, size) for watched files. Cheap, no reads."""
ws = Path(workspace_dir)
snap: dict = {}
for name in _WATCH_SUBDIRS:
root = ws / name
if root.is_file():
try:
st = root.stat()
snap[name] = (st.st_mtime, st.st_size)
except OSError:
pass
continue
if not root.is_dir():
continue
for p in root.rglob("*"):
if not p.is_file():
continue
if p.name in _WATCH_IGNORE_NAMES:
continue
try:
st = p.stat()
snap[str(p.relative_to(ws))] = (st.st_mtime, st.st_size)
except OSError:
pass
# Watch the daily memory files (memory/*.md and per-user dailies) since
# evolution now records learnings there. Skip backups/dreams bookkeeping.
mem_dir = ws / "memory"
if mem_dir.is_dir():
for p in mem_dir.rglob("*.md"):
rel_parts = p.relative_to(mem_dir).parts
if rel_parts and rel_parts[0] in _MEMORY_IGNORE:
continue
try:
st = p.stat()
snap[str(p.relative_to(ws))] = (st.st_mtime, st.st_size)
except OSError:
pass
return snap
def _workspace_changed(workspace_dir, pre: dict) -> bool:
"""True if any watched file was added, removed, or modified since ``pre``."""
return _workspace_snapshot(workspace_dir) != pre
def run_evolution_for_session(
agent_bridge,
session_id: str,
channel_type: str = "",
receiver: str = "",
user_id: Optional[str] = None,
idle_minutes: float = 0.0,
) -> bool:
"""Run one evolution pass for a session. Returns True if it changed anything.
Safe to call from a background thread. All failures are swallowed and
logged — evolution must never disrupt the main pipeline.
"""
cfg = get_evolution_config()
if not cfg.enabled:
return False
# Concurrency gate: bound how many evolution passes run at once.
global _running_count
with _running_lock:
if _running_count >= _MAX_CONCURRENT:
logger.info(
f"[Evolution] busy ({_running_count}/{_MAX_CONCURRENT} running); "
f"skipping session={session_id} this scan"
)
return False
_running_count += 1
try:
agent = agent_bridge.agents.get(session_id) or agent_bridge.default_agent
if not agent:
return False
with agent.messages_lock:
all_messages = list(agent.messages)
total_msgs = len(all_messages)
# In-memory evolution cursor: only review messages added since the last
# pass so a long session doesn't re-judge (and re-write) old content.
# Stored on the agent instance; lost on restart (acceptable — at worst
# one redundant pass right after a restart, gated by the file-change
# check downstream so it won't double-write identical memory).
done = int(getattr(agent, "_evo_done_msg_count", 0))
if done > total_msgs:
done = 0 # history was trimmed/reset; start fresh
new_messages = all_messages[done:]
transcript = _build_transcript(new_messages)
if not transcript.strip():
# Routine no-op: the per-minute scan hits every idle session. Advance
# the cursor so we don't re-scan the same tail; no log (pure noise).
agent._evo_done_msg_count = total_msgs
return False
logger.info(
f"[Evolution] ▶ Reviewing session={session_id} "
f"(idle {idle_minutes:.1f}min, {len(new_messages)} new/{total_msgs} msgs, "
f"~{len(transcript)} chars)"
)
# Resolve workspace + files to snapshot for undo.
from agent.memory.config import get_default_memory_config
mem_cfg = get_default_memory_config()
workspace_dir = mem_cfg.get_workspace()
if user_id:
memory_file = Path(workspace_dir) / "memory" / "users" / user_id / "MEMORY.md"
else:
memory_file = Path(workspace_dir) / "MEMORY.md"
skills_dir = mem_cfg.get_skills_dir()
# Snapshot MEMORY.md + every NON-protected skill's SKILL.md. Protected
# built-in skills are excluded from backup because they must never be
# edited in the first place.
protected_names = _builtin_skill_names()
# Back up both MEMORY.md and today's daily file: evolution now writes to
# the daily file, but MEMORY.md is cheap to snapshot and keeps undo safe
# if the model ever edits it.
today_daily = Path(workspace_dir) / "memory" / (
datetime.now().strftime("%Y-%m-%d") + ".md"
)
if user_id:
today_daily = Path(workspace_dir) / "memory" / "users" / user_id / (
datetime.now().strftime("%Y-%m-%d") + ".md"
)
# AGENT.md (persona) is backed up too so a rare persona edit is undoable.
# Persona is workspace-global (not per-user): it always lives at the
# workspace root, regardless of user_id.
agent_file = Path(workspace_dir) / "AGENT.md"
backup_files = [Path(memory_file), today_daily, agent_file]
if skills_dir.exists():
for skill_md in skills_dir.rglob("SKILL.md"):
# The skill dir is the SKILL.md's parent (or an ancestor for
# collections); guard by checking the immediate top-level dir.
try:
top = skill_md.relative_to(skills_dir).parts[0]
except (ValueError, IndexError):
continue
if top in protected_names:
continue
backup_files.append(skill_md)
backup_id = create_backup(workspace_dir, backup_files)
_backup_n = sum(1 for f in backup_files if Path(f).exists())
# Snapshot the whole workspace (path -> mtime/size) so we can reliably
# detect ANY file change — including new output files written when
# finishing an unfinished task, which are not in backup_files.
pre_snapshot = _workspace_snapshot(workspace_dir)
# Build the isolated review agent: same model, restricted tools, with a
# hard guard that confines all writes to the workspace (protects the
# project's bundled skills from ever being modified).
review_tools = _guard_tools(
_select_tools(list(getattr(agent, "tools", []) or [])),
str(workspace_dir),
)
review_agent = agent_bridge.create_agent(
system_prompt="",
tools=review_tools,
description="Self-evolution review agent",
max_steps=cfg.max_steps,
workspace_dir=str(workspace_dir),
skill_manager=getattr(agent, "skill_manager", None),
memory_manager=getattr(agent, "memory_manager", None),
enable_skills=True,
runtime_info=getattr(agent, "runtime_info", None),
)
# Mark this as a restricted review agent so runtime MCP reconciliation
# (ToolManager.sync_mcp_into_agent) will NOT silently re-inject MCP tools
# that _select_tools()/_guard_tools() intentionally withheld. Without this
# flag the review boundary would be re-opened on the first LLM turn.
review_agent._evolution_restricted = True
# Reuse the live model so it follows the user's configured model.
review_agent.model = agent.model
# Inject the evolution task brief AFTER the full system prompt: the agent
# gets the full context (tools, workspace, user preferences, memory, time)
# AND its evolution-specific instructions on top, instead of one
# overwriting the other.
review_agent.extra_system_suffix = EVOLUTION_SYSTEM_PROMPT
logger.info(
f"[Evolution] backup {backup_id} ({_backup_n} files) → running review agent"
)
user_msg = build_review_user_message(transcript, protected_skills=list(protected_names))
result = review_agent.run_stream(user_msg, clear_history=True)
result = (result or "").strip()
# These messages are now reviewed; advance the cursor so the next pass
# only looks at messages added after this point (silent or not).
agent._evo_done_msg_count = total_msgs
# Respect an explicit silent verdict: empty, exactly [SILENT], or text
# that STARTS with [SILENT] means the model chose to stay quiet.
if not result or result.startswith(SILENT_TOKEN):
logger.info(f"[Evolution] ✗ No change for session={session_id} ([SILENT])")
return False
# Anti-nag backstop: if the model wrote a summary but actually changed no
# watched file, stay silent — never notify about work that didn't happen.
if not _workspace_changed(workspace_dir, pre_snapshot):
logger.info(
f"[Evolution] ✗ session={session_id}: text produced but no file "
f"changed — staying silent"
)
return False
# The model produced a real summary. Strip any stray [SILENT] tokens it
# left mid-text, then notify.
result = result.replace(SILENT_TOKEN, "").strip()
if not result:
logger.info(f"[Evolution] ✗ No change for session={session_id} ([SILENT])")
return False
logger.info(f"[Evolution] ✓ session={session_id} evolved:\n{result}")
append_session_evolution(workspace_dir, result, backup_id=backup_id, user_id=user_id)
# Inject an [EVOLUTION] note so the main agent can honor "undo".
_inject_evolution_record(agent_bridge, session_id, channel_type, result, backup_id)
# The injection appended its own messages ([SCHEDULED]/[EVOLUTION]).
# Advance the cursor past them so the next scan does not treat
# evolution's own bookkeeping as new user content and re-trigger.
try:
with agent.messages_lock:
agent._evo_done_msg_count = len(agent.messages)
except Exception:
pass
# Push the summary to the user's channel. The "did a file actually
# change" gate above is the only throttle we need: real evolutions are
# rare, so no extra opt-in switch or daily-count limit is required.
if channel_type and receiver:
_notify_user(channel_type, receiver, result)
return True
except Exception as e:
logger.warning(f"[Evolution] Run failed for session={session_id}: {e}")
return False
finally:
with _running_lock:
_running_count -= 1
def _inject_evolution_record(
agent_bridge, session_id: str, channel_type: str, summary: str, backup_id: Optional[str]
) -> None:
"""Add an [EVOLUTION] note to the user session so the main agent can undo."""
try:
note = f"{EVOLUTION_MARKER} {summary}"
if backup_id:
note += f"\n(backup_id: {backup_id}; to undo, restore this backup)"
# Reuse the scheduler-output injection path: isolated execution, only a
# compact record lands in the user session.
agent_bridge.remember_scheduled_output(
session_id=session_id,
content=note,
channel_type=channel_type,
task_description="self-evolution",
)
except Exception as e:
logger.debug(f"[Evolution] Failed to inject evolution record: {e}")
def _notify_user(channel_type: str, receiver: str, summary: str) -> None:
"""Push the evolution summary to the user's channel as a new message."""
try:
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.channel_factory import create_channel
context = Context(ContextType.TEXT, summary)
context["receiver"] = receiver
context["isgroup"] = False
context["session_id"] = receiver
# Channels that reply to an original message need msg=None for a fresh push.
if channel_type in ("feishu", "dingtalk", "wecom_bot", "qq"):
context["msg"] = None
if channel_type == "feishu":
context["receive_id_type"] = "open_id"
channel = create_channel(channel_type)
if not channel:
return
# Web is request-response: a background push needs a synthetic request_id
# plus a request->session mapping so the channel can route the message to
# the user's polling queue (same approach the scheduler uses).
if channel_type == "web":
import uuid
request_id = f"evolution_{uuid.uuid4().hex[:8]}"
context["request_id"] = request_id
if hasattr(channel, "request_to_session"):
channel.request_to_session[request_id] = receiver
channel.send(Reply(ReplyType.TEXT, summary), context)
logger.info(f"[Evolution] Notified user via {channel_type}")
except Exception as e:
logger.warning(f"[Evolution] Failed to notify user: {e}")

170
agent/evolution/prompts.py Normal file
View File

@@ -0,0 +1,170 @@
"""Prompts for the self-evolution review agent.
The system prompt is intentionally English-only: it governs the agent's
internal reasoning and is more stable / cheaper to maintain in one language.
The user-facing summary the agent produces should follow the user's own
language (instructed at the end of the prompt).
Design goals (see ref/hermes-agent background_review for inspiration):
- Default to doing NOTHING. Evolution is the exception, not the rule.
- Signal types: skill, unfinished task, memory, knowledge.
- An explicit "do NOT capture" list to avoid self-poisoning over time.
- Generic examples only — never bake in domain-specific business terms.
"""
# Sentinel the agent emits when there is nothing worth evolving.
SILENT_TOKEN = "[SILENT]"
# Marker prefix for the evolution record injected into the user session, so the
# main chat agent can recognize past evolutions and honor an "undo" request.
EVOLUTION_MARKER = "[EVOLUTION]"
EVOLUTION_SYSTEM_PROMPT = """You are a self-evolution review agent for an AI assistant.
You are given a transcript of a conversation that just went idle. Your job is to
decide whether anything from it is worth durably learning so future
conversations go better — and if so, to make that change.
# Top principle: default to doing NOTHING
Most ordinary conversations need no evolution. Only act when there is a CLEAR
signal below. If there is none, reply with exactly `[SILENT]` and stop. Staying
silent is the normal, correct outcome — not a failure.
Greetings, small talk, acknowledgements ("ok", "thanks", "got it"), and casual
chat are NOT signals. For these, output exactly `[SILENT]` immediately — do not
explore files, do not write a summary, do not be polite. Just `[SILENT]`.
IMPORTANT: A summary is only allowed if you ACTUALLY made a file change via a
tool (write/edit) in this pass. If you did not change any file, you MUST output
exactly `[SILENT]` — never describe a change you only intended to make.
# Signals worth acting on (act only if at least one clearly appears)
SKILL and UNFINISHED TASK are your PRIMARY value — no other mechanism handles
them. When their signal is clear, act; do not be shy here.
1. SKILL — two cases:
a) PATCH an existing skill: a skill used here showed a STRUCTURAL problem (a
missing step/section, a wrong or outdated detail, an error in its
content), or its OUTPUT repeatedly misses something the user flagged. Read
the relevant skill file under the skills directory and make a small
incremental edit so it never recurs.
b) CREATE a new skill: a clearly reusable, repeatable workflow emerged that
no existing skill covers and the user is likely to want again. Follow the
`skill-creator` skill's conventions (read its SKILL.md for the required
structure), then create `skills/<name>/SKILL.md` by WRITING the file
directly with the write tool — this is the simplest reliable path. (bash
is available and confined to the workspace if a helper script is truly
needed, but a direct write is preferred.) Only create when the workflow is
genuinely reusable — not for a one-off task.
CRITICAL — fix the SOURCE, do not just remember the symptom: when the root
cause of a problem lives IN a skill file itself (its instructions, content,
or configuration are wrong/outdated), the correct action is to EDIT that
skill so the problem cannot recur. Recording the corrected fact in memory
does NOT prevent recurrence — only fixing the skill does. Never log "skill X
has wrong detail Y" as a memory note in place of editing skill X.
2. UNFINISHED TASK — a specific deliverable you promised but didn't produce,
AND you already have everything needed to finish it. DO IT now with the
available tools and produce the result (e.g. write the file you said you'd
write). If key info is missing, or the task is merely waiting on the user's
reply/decision, do NOTHING and stay [SILENT] — do not nag or ping the user.
You only ever notify the user as a side effect of having actually done work.
3. MEMORY — RARE, last resort. Default to writing NOTHING here. The main
assistant already writes memory during the chat, and a nightly pass plus
context-overflow saves are dedicated safety nets — so memory is almost always
already covered without you. Skip unless the main assistant clearly missed a
durable fact that belongs in no skill AND would visibly change future replies.
- MEMORY.md is the curated long-term index, auto-loaded into EVERY future
conversation. Treat it as precious: edit it in place to CORRECT a wrong
fact, or append a new durable preference/decision/lesson — but do so
SPARINGLY (a lasting fact, not a passing detail; the nightly pass handles
routine consolidation).
- For a NEW fact that is important but not yet clearly lasting, append ONE
short bullet to today's `memory/YYYY-MM-DD.md` instead. When unsure, the
daily file is the safe place — but first ask whether this really belongs
in a skill.
- PERSONA (AGENT.md) — EXTREMELY rare: only on an explicit, repeated signal
about the assistant's own identity/personality/style, make a small edit to
AGENT.md; never for user/world facts, and when in doubt do nothing.
- Keep it to ONE short bullet. Never write paragraphs, never re-summarize the
conversation, never copy what the main assistant already recorded.
- If it is already captured anywhere (check MEMORY.md AND the daily file
first), do NOTHING.
4. KNOWLEDGE — only if the conversation produced durable, reusable reference
knowledge on a topic (the kind worth looking up again) that the main
assistant did NOT already save to `knowledge/`. Add or update the relevant
file there. Like memory, this is the exception: skip routine Q&A, and if the
topic is already covered in `knowledge/`, do NOTHING rather than duplicate.
# Do NOT capture (these poison future behavior)
- Environment failures: missing binaries, unset credentials, uninstalled
packages, "command not found". The user can fix these; they are not durable
rules.
- Negative claims about tools or features ("tool X does not work"). These
harden into refusals the agent cites against itself later.
- One-off task narratives (e.g. summarizing today's content). Not a class of
reusable work.
- Transient errors that resolved on retry within the conversation.
# Execution constraints
- Before changing memory or a skill, READ the current content first and make a
small INCREMENTAL edit. Never fabricate, never rewrite large sections.
- AVOID DUPLICATES. Before writing memory, READ both MEMORY.md AND today's
daily file `memory/YYYY-MM-DD.md`. If the fact/preference is already recorded
in EITHER (even if worded differently), do NOT add it again. The main
assistant likely already wrote it during the chat — only add what is
genuinely new or a correction not yet reflected anywhere.
- You may only edit files inside the workspace. Built-in skills shipped with
the product live outside it and are write-protected; do not try to edit them.
- Make at most the few edits the signals justify; do not go looking for work.
# Output
- Nothing worth evolving -> output exactly `[SILENT]` and nothing else.
- Otherwise, after performing the edits, output a short user-facing summary in
the SAME LANGUAGE the user speaks in the conversation transcript. Write it for an ordinary user, in plain
everyday words — NOT a developer report. No need to expose internal details
(file names/paths, system mechanics, etc.). Briefly speak directly TO the user, telling them that you just did a self-learning pass,
what you learned, and what you changed in THIS pass. Keep it clear and focused on the key changes (a few lines), and let
the user know they can undo it.
"""
def build_review_user_message(transcript: str, protected_skills: list = None) -> str:
"""Wrap the conversation transcript as the review agent's user message.
``protected_skills`` lists skill names that must never be edited (built-in
skills shipped with the product). Surfaced so the agent avoids them.
"""
protected_note = ""
if protected_skills:
names = ", ".join(sorted(protected_skills))
protected_note = (
"\n\nPROTECTED skills (built-in — never edit these): "
f"{names}\n"
)
try:
from common import i18n
lang_name = "中文" if i18n.is_zh() else "English"
except Exception:
lang_name = "中文"
return (
"Here is the conversation transcript that just went idle. Review it per "
"your instructions. Acting is the exception: the main value is fixing or "
"creating a skill and finishing promised work. Memory and knowledge are "
"rare last resorts — stay [SILENT] unless there is a clear, durable signal "
"not already covered."
f"{protected_note}\n"
f"The summary should preferably be written in: {lang_name}\n"
"<transcript>\n"
f"{transcript}\n"
"</transcript>"
)

55
agent/evolution/record.py Normal file
View File

@@ -0,0 +1,55 @@
"""Self-evolution record log.
Session-level evolutions are appended to their OWN per-day file under
``memory/evolution/YYYY-MM-DD.md`` (separate from the nightly Deep Dream diary
in ``memory/dreams/``). Each day's file accumulates one short section per
evolution pass — tagged with a timestamp and a backup id for undo — so the
memory UI can surface "what the agent learned/changed today" on one timeline
without ever mixing into the dream diary or the main conversation memory.
"""
from __future__ import annotations
from datetime import datetime
from pathlib import Path
from typing import Optional
from common.log import logger
def _evolution_dir(workspace_dir: Path, user_id: Optional[str] = None) -> Path:
base = Path(workspace_dir) / "memory"
if user_id:
return base / "users" / user_id / "evolution"
return base / "evolution"
def append_session_evolution(
workspace_dir: Path,
summary: str,
backup_id: Optional[str] = None,
user_id: Optional[str] = None,
) -> None:
"""Append a session-evolution entry to today's evolution log."""
if not summary or not summary.strip():
return
try:
evo_dir = _evolution_dir(workspace_dir, user_id)
evo_dir.mkdir(parents=True, exist_ok=True)
today = datetime.now().strftime("%Y-%m-%d")
log_file = evo_dir / f"{today}.md"
ts = datetime.now().strftime("%H:%M")
header = f"## {ts}"
body = summary.strip()
if backup_id:
body += f"\n\n_backup_id: {backup_id}_"
# Create with a title if the file is new, otherwise append a section.
if not log_file.exists():
log_file.write_text(f"# Self-Evolution: {today}\n\n", encoding="utf-8")
with open(log_file, "a", encoding="utf-8") as f:
f.write(f"\n{header}\n\n{body}\n")
logger.info(f"[Evolution] Recorded session evolution to {log_file.name}")
except Exception as e:
logger.warning(f"[Evolution] Failed to record session evolution: {e}")

151
agent/evolution/trigger.py Normal file
View File

@@ -0,0 +1,151 @@
"""Idle-based evolution trigger.
A single background thread periodically scans live agent sessions and runs an
evolution pass for any session that is idle for >= idle_minutes AND has enough
accumulated signal, where "enough signal" is EITHER:
- >= min_turns user turns since the last evolution, OR
- the live context has grown past _CONTEXT_RATIO of the agent's token budget
(mirrors how OpenClacky / Claude Code consolidate under context pressure).
Turn counting is per user turn (not per message), measured from the last
evolution (or session start). After a pass runs, the baseline resets so a long
session can evolve multiple times without re-judging old content.
Per-session evolution state is stored on the agent instance via lightweight
attributes set by AgentBridge.agent_reply (see _note_user_turn).
"""
from __future__ import annotations
import threading
import time
from common.log import logger
from agent.evolution.config import get_evolution_config
from agent.evolution.executor import run_evolution_for_session
_SCAN_INTERVAL_SECONDS = 60
# Context-pressure trigger: evolve once the live context exceeds this fraction
# of the agent's token budget, even if min_turns hasn't been reached. Kept as a
# module constant (not user config) for now. Fallback budget matches
# agent_initializer / config.py (agent_max_context_tokens default = 50000).
_CONTEXT_RATIO = 0.8
_FALLBACK_CONTEXT_BUDGET = 50000
def _context_pressure_reached(agent) -> bool:
"""True if the agent's live context exceeds _CONTEXT_RATIO of its budget.
Uses the agent's own (estimated) token accounting so behavior matches the
existing context-trimming path. Best-effort: any error -> False.
"""
try:
with agent.messages_lock:
messages = list(agent.messages)
if not messages:
return False
est = sum(agent._estimate_message_tokens(m) for m in messages)
budget = getattr(agent, "max_context_tokens", None) or _FALLBACK_CONTEXT_BUDGET
return est / budget > _CONTEXT_RATIO
except Exception:
return False
def note_user_turn(agent, channel_type: str = "", receiver: str = "") -> None:
"""Record activity for a session's agent. Called once per real user turn.
Maintains, on the agent instance:
_evo_last_active : epoch seconds of the last user turn
_evo_turns : user turns since the last evolution
_evo_channel_type : originating channel (for later notify)
_evo_receiver : push target for notify
"""
try:
agent._evo_last_active = time.time()
agent._evo_turns = int(getattr(agent, "_evo_turns", 0)) + 1
if channel_type:
agent._evo_channel_type = channel_type
if receiver:
agent._evo_receiver = receiver
except Exception:
pass
def mark_run_active(agent, active: bool) -> None:
"""Flag whether the agent is mid-run, so idle scans skip a busy session.
Without this, a single run that lasts longer than idle_minutes would let
the scanner fire an evolution pass concurrently with the live turn.
"""
try:
agent._evo_run_active = bool(active)
if active:
agent._evo_last_active = time.time()
except Exception:
pass
def start_evolution_trigger(agent_bridge) -> None:
"""Start the idle-scan thread once per process (idempotent)."""
if getattr(agent_bridge, "_evolution_trigger_started", False):
return
agent_bridge._evolution_trigger_started = True
t = threading.Thread(
target=_scan_loop, args=(agent_bridge,), daemon=True, name="evolution-trigger"
)
t.start()
logger.info("[Evolution] Idle trigger started")
def _scan_loop(agent_bridge) -> None:
while True:
try:
time.sleep(_SCAN_INTERVAL_SECONDS)
cfg = get_evolution_config()
if not cfg.enabled:
continue
_scan_once(agent_bridge, cfg)
except Exception as e:
logger.warning(f"[Evolution] Scan loop error: {e}")
time.sleep(_SCAN_INTERVAL_SECONDS)
def _scan_once(agent_bridge, cfg) -> None:
now = time.time()
# Snapshot to avoid holding the dict while running long evolutions.
sessions = list(getattr(agent_bridge, "agents", {}).items())
for session_id, agent in sessions:
try:
# Skip sessions whose agent is mid-run: a long turn must not be
# reviewed while it is still producing the answer.
if getattr(agent, "_evo_run_active", False):
continue
last_active = getattr(agent, "_evo_last_active", 0)
turns = int(getattr(agent, "_evo_turns", 0))
# Enough signal = enough turns OR enough context pressure.
enough_signal = turns >= cfg.min_turns or _context_pressure_reached(agent)
if not enough_signal:
continue
idle = now - last_active if last_active > 0 else -1
if last_active <= 0 or idle < cfg.idle_seconds:
continue
channel_type = getattr(agent, "_evo_channel_type", "") or ""
receiver = getattr(agent, "_evo_receiver", "") or ""
# Reset baseline BEFORE running so a long pass / new messages during
# it don't double-trigger; turns accrue fresh from here.
agent._evo_turns = 0
run_evolution_for_session(
agent_bridge,
session_id=session_id,
channel_type=channel_type,
receiver=receiver,
idle_minutes=(now - last_active) / 60 if last_active > 0 else 0.0,
)
except Exception as e:
logger.warning(f"[Evolution] Failed to evaluate session={session_id}: {e}")

View File

@@ -12,11 +12,17 @@ Knowledge file layout (under workspace_root):
import os
import re
import asyncio
import shutil
import threading
from pathlib import Path
from typing import Optional
from typing import Optional, Iterable
from urllib.parse import quote
from common.log import logger
from config import conf
from agent.memory.config import MemoryConfig
from agent.memory.manager import MemoryManager
class KnowledgeService:
@@ -25,9 +31,389 @@ class KnowledgeService:
Operates directly on the filesystem.
"""
def __init__(self, workspace_root: str):
self.workspace_root = workspace_root
self.knowledge_dir = os.path.join(workspace_root, "knowledge")
PROTECTED_FILES = {"index.md", "log.md"}
INVALID_NAME_RE = re.compile(r'[<>:"|?*\x00-\x1f]')
IMPORT_EXTENSIONS = {".md", ".txt"}
MAX_IMPORT_FILES = 100
MAX_IMPORT_FILE_SIZE = 10 * 1024 * 1024
MAX_IMPORT_TOTAL_SIZE = 200 * 1024 * 1024
def __init__(self, workspace_root: str, memory_manager=None):
self.workspace_root = os.path.abspath(workspace_root)
self.knowledge_dir = os.path.join(self.workspace_root, "knowledge")
self._memory_manager = memory_manager
def _resolve_path(self, rel_path: str, *, kind: Optional[str] = None,
allow_missing: bool = True) -> tuple:
if not isinstance(rel_path, str) or not rel_path.strip():
raise ValueError("path is required")
rel_path = rel_path.replace("\\", "/").strip("/")
parts = rel_path.split("/")
if any(not p or p in (".", "..") or self.INVALID_NAME_RE.search(p) for p in parts):
raise ValueError("invalid path")
if kind == "document" and not rel_path.lower().endswith(".md"):
raise ValueError("document path must end with .md")
root = Path(self.knowledge_dir).resolve()
candidate = root.joinpath(*parts)
# Resolve the nearest existing ancestor so a symlink cannot be used
# to escape when the final destination does not exist yet.
ancestor = candidate
while not ancestor.exists() and ancestor != root:
ancestor = ancestor.parent
try:
ancestor.resolve().relative_to(root)
except ValueError:
raise ValueError("path outside knowledge dir")
if candidate.exists():
try:
candidate.resolve().relative_to(root)
except ValueError:
raise ValueError("path outside knowledge dir")
elif not allow_missing:
raise FileNotFoundError(f"path not found: {rel_path}")
return rel_path, candidate
def _ensure_not_protected(self, rel_path: str):
if rel_path in self.PROTECTED_FILES:
raise ValueError(f"protected knowledge file: {rel_path}")
def _manager(self):
if self._memory_manager is None:
# Reuse the shared embedding provider selection so knowledge index
# sync gets vectors too, instead of degrading to keyword-only.
from agent.memory.embedding import create_default_embedding_provider
embedding_provider = create_default_embedding_provider()
self._memory_manager = MemoryManager(
MemoryConfig(workspace_root=self.workspace_root),
embedding_provider=embedding_provider,
)
return self._memory_manager
@staticmethod
def _run_sync(coro):
try:
asyncio.get_running_loop()
except RuntimeError:
return asyncio.run(coro)
result = []
error = []
def runner():
try:
result.append(asyncio.run(coro))
except Exception as exc:
error.append(exc)
thread = threading.Thread(target=runner)
thread.start()
thread.join()
if error:
raise error[0]
return result[0] if result else None
def _sync_index(self, old_paths: Iterable[str], force: bool = False):
old_paths = sorted(set(old_paths))
if not old_paths and not force:
return
manager = self._manager()
for rel_path in old_paths:
manager.storage.delete_by_path(f"knowledge/{rel_path}")
manager.mark_dirty()
self._run_sync(manager.sync())
@staticmethod
def _extract_title(md_path: Path, fallback: str) -> str:
"""Read a markdown file's H1 title, falling back to the file stem."""
try:
with open(md_path, "r", encoding="utf-8") as f:
for _ in range(20):
line = f.readline()
if not line:
break
stripped = line.strip()
if stripped.startswith("# "):
return stripped[2:].strip() or fallback
except Exception:
pass
return fallback
def rebuild_index_md(self) -> bool:
"""Regenerate knowledge/index.md from the actual directory tree.
Keeps the index in sync with real files so it never drifts or loses
documents. Returns True when the file was (re)written.
"""
root = Path(self.knowledge_dir)
if not root.is_dir():
return False
def collect(dir_path: Path) -> list:
# Return sorted (rel_path, title) tuples for *.md under dir_path,
# excluding protected files at the knowledge root and dot files.
entries = []
for md in sorted(dir_path.rglob("*.md")):
rel = md.relative_to(root).as_posix()
if any(part.startswith(".") for part in md.relative_to(root).parts):
continue
if rel in self.PROTECTED_FILES:
continue
entries.append((rel, self._extract_title(md, md.stem)))
return entries
all_entries = collect(root)
def link(rel: str) -> str:
# Encode each path segment so spaces / special chars stay valid in
# markdown links, while keeping the slashes between segments.
encoded = "/".join(quote(part) for part in rel.split("/"))
return f"./{encoded}"
lines = ["# 知识库目录", ""]
# Root-level documents first (no category dir).
root_docs = [(rel, title) for rel, title in all_entries if "/" not in rel]
for rel, title in root_docs:
lines.append(f"- [{title}]({link(rel)})")
if root_docs:
lines.append("")
# Group remaining documents by their top-level category.
categories = {}
for rel, title in all_entries:
if "/" not in rel:
continue
category = rel.split("/", 1)[0]
categories.setdefault(category, []).append((rel, title))
for category in sorted(categories.keys()):
lines.append(f"## {category}")
for rel, title in categories[category]:
lines.append(f"- [{title}]({link(rel)})")
lines.append("")
content = "\n".join(lines).rstrip() + "\n"
index_path = root / "index.md"
try:
index_path.write_text(content, encoding="utf-8")
return True
except Exception as exc:
logger.warning(f"[KnowledgeService] Failed to rebuild index.md: {exc}")
return False
def _sanitize_document_name(self, filename: str) -> str:
name = os.path.basename((filename or "").replace("\\", "/")).strip()
if not name:
raise ValueError("filename is required")
stem, ext = os.path.splitext(name)
if ext.lower() not in self.IMPORT_EXTENSIONS:
raise ValueError(f"unsupported file type: {ext or name}")
if not stem or stem in (".", "..") or self.INVALID_NAME_RE.search(stem):
raise ValueError("invalid filename")
safe_name = f"{stem}.md"
self._ensure_not_protected(safe_name)
return safe_name
@staticmethod
def _decode_document_content(content) -> str:
if isinstance(content, str):
return content
if not isinstance(content, (bytes, bytearray)):
raise ValueError("document content is required")
return bytes(content).decode("utf-8-sig", errors="replace")
def _resolve_import_destination(self, target_category: str, filename: str,
conflict_strategy: str) -> tuple:
target_rel, target_full = self._resolve_path(target_category, kind="category")
if not target_full.is_dir():
raise FileNotFoundError(f"category not found: {target_rel}")
safe_name = self._sanitize_document_name(filename)
destination = target_full / safe_name
rel_path = f"{target_rel}/{safe_name}"
if destination.exists():
if conflict_strategy == "skip":
return rel_path, destination, "skip"
if conflict_strategy == "rename":
stem = destination.stem
suffix = destination.suffix
for index in range(1, 1000):
candidate = target_full / f"{stem}-{index}{suffix}"
if not candidate.exists():
candidate_rel = f"{target_rel}/{candidate.name}"
return candidate_rel, candidate, "write"
raise FileExistsError(f"target already exists: {rel_path}")
if conflict_strategy != "overwrite":
raise ValueError("invalid conflict strategy")
return rel_path, destination, "write"
def create_document(self, path: str, content: str = "", overwrite: bool = False) -> dict:
rel_path, full_path = self._resolve_path(path, kind="document")
self._ensure_not_protected(rel_path)
if len((content or "").encode("utf-8")) > self.MAX_IMPORT_FILE_SIZE:
raise ValueError("file too large")
if full_path.exists() and not overwrite:
raise FileExistsError(f"target already exists: {rel_path}")
old_paths = [rel_path] if full_path.exists() else []
full_path.parent.mkdir(parents=True, exist_ok=True)
full_path.write_text(content or "", encoding="utf-8")
# Keep index.md in sync before reindexing so it is indexed too.
self.rebuild_index_md()
self._sync_index(old_paths, force=True)
return {"path": rel_path, "created": True, "overwritten": bool(old_paths)}
def import_documents(self, target_category: str, files: Iterable[dict],
conflict_strategy: str = "skip") -> dict:
if not isinstance(files, list):
raise ValueError("files must be a list")
if len(files) > self.MAX_IMPORT_FILES:
raise ValueError(f"too many files: max {self.MAX_IMPORT_FILES}")
results = []
old_paths = []
imported = skipped = failed = 0
total_size = 0
for item in files:
filename = item.get("filename") if isinstance(item, dict) else None
try:
content_bytes = item.get("content") if isinstance(item, dict) else None
size = len(content_bytes.encode("utf-8")) if isinstance(content_bytes, str) else len(content_bytes or b"")
total_size += size
if total_size > self.MAX_IMPORT_TOTAL_SIZE:
raise ValueError("import batch too large")
if size > self.MAX_IMPORT_FILE_SIZE:
raise ValueError("file too large")
rel_path, destination, mode = self._resolve_import_destination(
target_category, filename, conflict_strategy
)
if mode == "skip":
skipped += 1
results.append({"filename": filename, "path": rel_path, "status": "skipped",
"reason": "target_exists"})
continue
old_exists = destination.exists()
content = self._decode_document_content(content_bytes)
destination.parent.mkdir(parents=True, exist_ok=True)
destination.write_text(content, encoding="utf-8")
if old_exists:
old_paths.append(rel_path)
imported += 1
results.append({"filename": filename, "path": rel_path, "status": "imported",
"overwritten": old_exists})
except Exception as exc:
failed += 1
results.append({"filename": filename or "", "status": "failed", "reason": str(exc)})
if imported:
# Keep index.md in sync before reindexing so it is indexed too.
self.rebuild_index_md()
self._sync_index(old_paths, force=True)
return {"results": results, "imported": imported, "skipped": skipped, "failed": failed}
def create_category(self, path: str) -> dict:
rel_path, full_path = self._resolve_path(path, kind="category")
if full_path.exists():
return {"path": rel_path, "created": False, "reason": "already_exists"}
full_path.mkdir(parents=True)
return {"path": rel_path, "created": True}
def rename_category(self, path: str, new_path: str) -> dict:
old_rel, old_full = self._resolve_path(path, kind="category", allow_missing=False)
new_rel, new_full = self._resolve_path(new_path, kind="category")
if not old_full.is_dir():
raise ValueError(f"not a category: {old_rel}")
if new_full.exists():
raise FileExistsError(f"target already exists: {new_rel}")
old_documents = [str(p.relative_to(old_full)).replace(os.sep, "/")
for p in old_full.rglob("*.md") if p.is_file()]
new_full.parent.mkdir(parents=True, exist_ok=True)
try:
old_full.rename(new_full)
except FileNotFoundError:
return {"old_path": old_rel, "path": new_rel, "moved": False, "reason": "not_found"}
except FileExistsError:
raise FileExistsError(f"target already exists: {new_rel}")
old_paths = [f"{old_rel}/{p}" for p in old_documents]
self._sync_index(old_paths)
return {"old_path": old_rel, "path": new_rel, "moved_documents": len(old_documents)}
def delete_category(self, path: str, confirm: bool = False) -> dict:
rel_path, full_path = self._resolve_path(path, kind="category")
if not full_path.exists():
return {"path": rel_path, "deleted": False, "reason": "not_found"}
if not full_path.is_dir():
raise ValueError(f"not a category: {rel_path}")
knowledge_root = Path(self.knowledge_dir).resolve()
documents = [str(p.relative_to(knowledge_root)).replace(os.sep, "/")
for p in full_path.rglob("*.md") if p.is_file()]
if any(p in self.PROTECTED_FILES for p in documents):
raise ValueError("category contains protected knowledge files")
if any(full_path.iterdir()) and not confirm:
raise ValueError("category is not empty; confirmation is required")
try:
shutil.rmtree(full_path)
except FileNotFoundError:
return {"path": rel_path, "deleted": False, "reason": "not_found"}
self._sync_index(documents)
return {"path": rel_path, "deleted": True, "deleted_documents": len(documents)}
def delete_documents(self, paths: Iterable[str]) -> dict:
if not isinstance(paths, list):
raise ValueError("paths must be a list")
results = []
deleted = []
for path in paths:
rel_path, full_path = self._resolve_path(path, kind="document")
self._ensure_not_protected(rel_path)
if not full_path.exists():
deleted.append(rel_path)
results.append({"path": rel_path, "deleted": False, "reason": "not_found"})
continue
if not full_path.is_file():
raise ValueError(f"not a document: {rel_path}")
try:
full_path.unlink()
deleted.append(rel_path)
results.append({"path": rel_path, "deleted": True})
except FileNotFoundError:
deleted.append(rel_path)
results.append({"path": rel_path, "deleted": False, "reason": "not_found"})
self._sync_index(deleted)
return {"results": results, "deleted": sum(1 for item in results if item["deleted"])}
def move_documents(self, paths: Iterable[str], target_category: str) -> dict:
if not isinstance(paths, list):
raise ValueError("paths must be a list")
target_rel, target_full = self._resolve_path(target_category, kind="category")
if not target_full.is_dir():
raise FileNotFoundError(f"category not found: {target_rel}")
results = []
moved_old_paths = []
for path in paths:
rel_path, full_path = self._resolve_path(path, kind="document")
self._ensure_not_protected(rel_path)
if not full_path.exists():
results.append({"path": rel_path, "moved": False, "reason": "not_found"})
continue
destination = target_full / full_path.name
new_rel = str(destination.relative_to(Path(self.knowledge_dir).resolve())).replace(os.sep, "/")
if destination.exists():
results.append({"path": rel_path, "moved": False, "reason": "target_exists",
"target": new_rel})
continue
try:
os.link(full_path, destination)
full_path.unlink()
moved_old_paths.append(rel_path)
results.append({"path": rel_path, "moved": True, "target": new_rel})
except FileExistsError:
results.append({"path": rel_path, "moved": False, "reason": "target_exists",
"target": new_rel})
except FileNotFoundError:
results.append({"path": rel_path, "moved": False, "reason": "not_found"})
self._sync_index(moved_old_paths)
return {"results": results, "moved": len(moved_old_paths)}
# ------------------------------------------------------------------
# list — directory tree with stats
@@ -98,14 +484,18 @@ class KnowledgeService:
if not is_root:
stats["pages"] += 1
stats["size"] += size
title = name.replace(".md", "")
try:
with open(full, "r", encoding="utf-8") as f:
first_line = f.readline().strip()
if first_line.startswith("# "):
title = first_line[2:].strip()
except Exception:
pass
# Prefer the H1 heading as a readable title for normal docs.
# System files (index.md / log.md) keep their filename so the
# tree never hides what they actually are.
title = name[:-3]
if name not in self.PROTECTED_FILES:
try:
with open(full, "r", encoding="utf-8") as f:
first_line = f.readline().strip()
if first_line.startswith("# "):
title = first_line[2:].strip() or title
except Exception:
pass
files.append({"name": name, "title": title, "size": size})
return files, children
@@ -121,15 +511,8 @@ class KnowledgeService:
:raises ValueError: if path is invalid or escapes knowledge dir
:raises FileNotFoundError: if file does not exist
"""
if not rel_path or ".." in rel_path:
raise ValueError("invalid path")
full_path = os.path.normpath(os.path.join(self.knowledge_dir, rel_path))
allowed = os.path.normpath(self.knowledge_dir)
if not full_path.startswith(allowed + os.sep) and full_path != allowed:
raise ValueError("path outside knowledge dir")
if not os.path.isfile(full_path):
rel_path, full_path = self._resolve_path(rel_path, kind="document")
if not full_path.is_file():
raise FileNotFoundError(f"file not found: {rel_path}")
with open(full_path, "r", encoding="utf-8") as f:
@@ -228,13 +611,35 @@ class KnowledgeService:
result = self.build_graph()
return {"action": action, "code": 200, "message": "success", "payload": result}
elif action == "create_category":
result = self.create_category(payload.get("path"))
elif action == "rename_category":
result = self.rename_category(payload.get("path"), payload.get("new_path"))
elif action == "delete_category":
result = self.delete_category(payload.get("path"), payload.get("confirm", False))
elif action == "delete_documents":
result = self.delete_documents(payload.get("paths") or [])
elif action == "move_documents":
result = self.move_documents(payload.get("paths") or [], payload.get("target_category"))
elif action == "create_document":
result = self.create_document(payload.get("path"), payload.get("content", ""),
payload.get("overwrite", False))
elif action == "import_documents":
result = self.import_documents(
payload.get("target_category"),
payload.get("files") or [],
payload.get("conflict_strategy", "skip"),
)
else:
return {"action": action, "code": 400, "message": f"unknown action: {action}", "payload": None}
return {"action": action, "code": 200, "message": "success", "payload": result}
except ValueError as e:
return {"action": action, "code": 403, "message": str(e), "payload": None}
except FileNotFoundError as e:
return {"action": action, "code": 404, "message": str(e), "payload": None}
except FileExistsError as e:
return {"action": action, "code": 409, "message": str(e), "payload": None}
except Exception as e:
logger.error(f"[KnowledgeService] dispatch error: action={action}, error={e}")
return {"action": action, "code": 500, "message": str(e), "payload": None}

View File

@@ -7,7 +7,7 @@ conversation history persistence (SQLite).
from agent.memory.manager import MemoryManager
from agent.memory.config import MemoryConfig, get_default_memory_config, set_global_memory_config
from agent.memory.embedding import create_embedding_provider
from agent.memory.embedding import create_embedding_provider, create_default_embedding_provider
from agent.memory.conversation_store import ConversationStore, get_conversation_store
from agent.memory.summarizer import ensure_daily_memory_file
@@ -17,6 +17,7 @@ __all__ = [
'get_default_memory_config',
'set_global_memory_config',
'create_embedding_provider',
'create_default_embedding_provider',
'ConversationStore',
'get_conversation_store',
'ensure_daily_memory_file',

View File

@@ -13,6 +13,7 @@ Storage path: ~/cow/sessions/conversations.db
from __future__ import annotations
import json
import re
import sqlite3
import threading
import time
@@ -109,6 +110,48 @@ def _extract_display_text(content: Any) -> str:
return ""
# Internal markers written into the session for the agent's own bookkeeping
# (scheduler injection / self-evolution undo). They must stay in the stored
# content (the LLM reads them, e.g. to find a backup_id for undo) but should
# never be shown verbatim to the user in the chat history UI.
_SCHEDULED_DISPLAY_MARKERS = ("[SCHEDULED]", "Scheduled task")
_EVOLUTION_DISPLAY_MARKER = "[EVOLUTION]"
def _is_internal_user_marker(text: str) -> bool:
"""True if a user-turn text is an internal injection marker (hide from UI)."""
t = (text or "").lstrip()
return any(t.startswith(m) for m in _SCHEDULED_DISPLAY_MARKERS)
def _is_evolution_text(text: str) -> bool:
"""True if assistant text is a self-evolution summary (before cleaning)."""
return (text or "").lstrip().startswith(_EVOLUTION_DISPLAY_MARKER)
def _clean_display_text(text: str) -> str:
"""Strip internal markers from assistant text for user-facing display.
Removes a leading ``[EVOLUTION]`` tag and a trailing ``(backup_id: ...)``
undo hint. The raw stored message is untouched, so undo + LLM context still
work; only the rendered chat bubble is cleaned.
"""
if not text:
return text
cleaned = text
stripped = cleaned.lstrip()
if stripped.startswith(_EVOLUTION_DISPLAY_MARKER):
cleaned = stripped[len(_EVOLUTION_DISPLAY_MARKER):].lstrip()
# Drop a trailing backup_id undo hint line, e.g.
# "(backup_id: 20260607-...; to undo, restore this backup)"
cleaned = re.sub(
r"\n*\(backup_id:[^\)]*\)\s*$",
"",
cleaned,
).rstrip()
return cleaned
def _extract_tool_calls(content: Any) -> List[Dict[str, Any]]:
"""
Extract tool_use blocks from an assistant message content.
@@ -210,7 +253,10 @@ def _group_into_display_turns(
if user_row:
content, created_at, _u_extras = user_row
text = _extract_display_text(content)
if text:
# Hide internal injection markers (scheduler / self-evolution) so the
# user never sees a synthetic "[SCHEDULED] self-evolution" bubble;
# the assistant reply that follows is still rendered.
if text and not _is_internal_user_marker(text):
turns.append({"role": "user", "content": text, "created_at": created_at})
# Build an ordered list of steps preserving the original sequence:
@@ -265,6 +311,18 @@ def _group_into_display_turns(
step["result"] = tr.get("result", "")
step["is_error"] = tr.get("is_error", False)
# Detect a self-evolution bubble BEFORE cleaning the marker away, so the
# UI can flag it even though the visible text stays clean.
is_evolution = _is_evolution_text(final_text)
# Clean internal markers from the user-facing assistant text. Applies to
# both the final content and the mirrored content step so the rendered
# bubble shows clean text while the stored message keeps the markers.
final_text = _clean_display_text(final_text)
for step in steps:
if step.get("type") == "content":
step["content"] = _clean_display_text(step.get("content", ""))
if steps or final_text:
turn = {
"role": "assistant",
@@ -272,6 +330,8 @@ def _group_into_display_turns(
"steps": steps,
"created_at": final_ts or (user_row[1] if user_row else 0),
}
if is_evolution:
turn["kind"] = "evolution"
if merged_extras:
turn["extras"] = merged_extras
turns.append(turn)
@@ -291,7 +351,7 @@ class ConversationStore:
def __init__(self, db_path: Path):
self._db_path = db_path
self._lock = threading.Lock()
self._lock = threading.RLock() # Use RLock to allow reentrant locking
self._init_db()
# ------------------------------------------------------------------
@@ -509,6 +569,65 @@ class ConversationStore:
finally:
conn.close()
def get_latest_pair_seqs(self, session_id: str) -> Dict[str, Optional[int]]:
"""Return the seq numbers of the latest visible user message and the
latest assistant message in a session.
A "visible" user message is one whose content is real user text
(not just a tool_result block), so tool-execution turns do not
shadow the actual user query.
Returns:
Dict with keys ``user_seq`` and ``bot_seq``; either may be None
when no matching message exists.
"""
result: Dict[str, Optional[int]] = {"user_seq": None, "bot_seq": None}
with self._lock:
conn = self._connect()
try:
# Latest assistant message (cheap: single row by seq DESC).
row = conn.execute(
"SELECT seq FROM messages "
"WHERE session_id = ? AND role = 'assistant' "
"ORDER BY seq DESC LIMIT 1",
(session_id,),
).fetchone()
if row:
result["bot_seq"] = int(row[0])
# Latest visible user message: scan recent user rows and
# skip pure tool_result entries.
rows = conn.execute(
"SELECT seq, content FROM messages "
"WHERE session_id = ? AND role = 'user' "
"ORDER BY seq DESC LIMIT 20",
(session_id,),
).fetchall()
for seq, content_raw in rows:
try:
content = json.loads(content_raw)
except Exception:
result["user_seq"] = int(seq)
break
if isinstance(content, list):
has_text = any(
isinstance(b, dict) and b.get("type") == "text"
for b in content
)
has_tool_result = any(
isinstance(b, dict) and b.get("type") == "tool_result"
for b in content
)
if has_text and not has_tool_result:
result["user_seq"] = int(seq)
break
else:
result["user_seq"] = int(seq)
break
finally:
conn.close()
return result
def clear_session(self, session_id: str) -> None:
"""Delete all messages and the session record for a given session_id."""
with self._lock:
@@ -524,6 +643,109 @@ class ConversationStore:
finally:
conn.close()
def delete_message_pair(self, session_id: str, user_seq: int, delete_user: bool = True, cascade: bool = False) -> int:
"""Delete a user message and/or its corresponding assistant reply.
The assistant reply is identified as all messages between user_seq
and the next visible user message (or end of session).
Args:
session_id: Session identifier.
user_seq: The seq number of the user message.
delete_user: If True (default), delete the user message too.
If False, only delete assistant reply (for regenerate scenarios).
cascade: If True, also delete all subsequent turns after this one.
Used by edit-message which removes this turn and everything after.
Returns:
Number of message rows deleted.
"""
with self._lock:
conn = self._connect()
try:
with conn:
# Verify this is a user message
row = conn.execute(
"SELECT role FROM messages WHERE session_id = ? AND seq = ?",
(session_id, user_seq),
).fetchone()
if not row or row[0] != "user":
return 0
if cascade:
# Delete from this message to end of session
start_seq = user_seq if delete_user else user_seq + 1
end_seq_row = conn.execute(
"SELECT MAX(seq) FROM messages WHERE session_id = ?",
(session_id,),
).fetchone()
end_seq = (end_seq_row[0] or user_seq) + 1
else:
# Find the next visible user message seq (exclude tool_result)
# Use batched query to avoid loading too many rows at once
next_user_seq = None
batch_size = 100
offset = 0
while True:
batch = conn.execute(
"""
SELECT seq, content FROM messages
WHERE session_id = ? AND seq > ? AND role = 'user'
ORDER BY seq ASC
LIMIT ? OFFSET ?
""",
(session_id, user_seq, batch_size, offset),
).fetchall()
if not batch:
break
for seq, content in batch:
try:
content_obj = json.loads(content)
except Exception:
content_obj = content
if _is_visible_user_message(content_obj):
next_user_seq = seq
break
if next_user_seq is not None:
break
offset += batch_size
# Determine the end boundary for deletion
if next_user_seq is not None:
end_seq = next_user_seq
else:
end_seq_row = conn.execute(
"SELECT MAX(seq) FROM messages WHERE session_id = ?",
(session_id,),
).fetchone()
end_seq = (end_seq_row[0] or user_seq) + 1
# Determine the start boundary for deletion
start_seq = user_seq if delete_user else user_seq + 1
# Delete messages from start_seq to end_seq (exclusive)
cur = conn.execute(
"DELETE FROM messages WHERE session_id = ? AND seq >= ? AND seq < ?",
(session_id, start_seq, end_seq),
)
deleted = cur.rowcount
# Update session msg_count
conn.execute(
"""
UPDATE sessions
SET msg_count = (
SELECT COUNT(*) FROM messages WHERE session_id = ?
)
WHERE session_id = ?
""",
(session_id, session_id),
)
return deleted
finally:
conn.close()
def prune_scheduled_messages(
self,
session_id: str,
@@ -1053,3 +1275,4 @@ def get_conversation_store() -> ConversationStore:
_store_instance = ConversationStore(db_path)
logger.debug(f"[ConversationStore] Using shared DB at: {db_path}")
return _store_instance

View File

@@ -16,6 +16,7 @@ from agent.memory.embedding.provider import (
OpenAIEmbeddingProvider,
create_embedding_provider,
)
from agent.memory.embedding.factory import create_default_embedding_provider
from agent.memory.embedding.rebuild import (
RebuildResult,
clear_index,
@@ -33,6 +34,7 @@ __all__ = [
"EmbeddingProvider",
"OpenAIEmbeddingProvider",
"create_embedding_provider",
"create_default_embedding_provider",
"RebuildResult",
"clear_index",
"rebuild_in_process",

View File

@@ -0,0 +1,209 @@
"""
Shared embedding provider factory.
Resolves the embedding provider purely from config.json, so every caller
(agent initialization, knowledge base sync, index rebuild, ...) selects the
same provider instead of silently degrading to keyword-only search.
Two paths:
A. Default (no `embedding_provider` in config.json):
Auto-init OpenAI -> LinkAI fallback.
B. Explicit (`embedding_provider` is set):
Initialize the requested vendor with unified dim (default per vendor).
"""
import os
from typing import Optional
from common.log import logger
# Track whether the embedding model log has been printed in this process,
# so we avoid spamming it once per session/caller.
_embedding_logged: bool = False
def create_default_embedding_provider():
"""Build the embedding provider from config, or None for keyword-only mode."""
from config import conf
explicit_provider = (conf().get("embedding_provider") or "").strip().lower()
if not explicit_provider:
return _init_legacy_provider()
return _init_explicit_provider(explicit_provider)
def _init_legacy_provider():
"""Legacy auto-init path: OpenAI -> LinkAI."""
from agent.memory.embedding.provider import create_embedding_provider
from config import conf
embedding_provider = None
embedding_model = None
openai_api_key = conf().get("open_ai_api_key", "")
openai_api_base = conf().get("open_ai_api_base", "")
if openai_api_key and openai_api_key not in ["", "YOUR API KEY", "YOUR_API_KEY"]:
try:
model = "text-embedding-3-small"
embedding_provider = create_embedding_provider(
provider="openai",
model=model,
api_key=openai_api_key,
api_base=openai_api_base or "https://api.openai.com/v1",
)
embedding_model = f"openai/{model}"
except Exception as e:
logger.warning(f"[EmbeddingFactory] OpenAI embedding failed: {e}")
if embedding_provider is None:
linkai_api_key = conf().get("linkai_api_key", "") or os.environ.get("LINKAI_API_KEY", "")
linkai_api_base = conf().get("linkai_api_base", "https://api.link-ai.tech")
if linkai_api_key and linkai_api_key not in ["", "YOUR API KEY", "YOUR_API_KEY"]:
try:
model = "text-embedding-3-small"
embedding_provider = create_embedding_provider(
provider="linkai",
model=model,
api_key=linkai_api_key,
api_base=f"{linkai_api_base}/v1",
)
embedding_model = f"linkai/{model}"
except Exception as e:
logger.warning(f"[EmbeddingFactory] LinkAI embedding failed: {e}")
if embedding_provider is not None and embedding_model:
_log_provider_once(f"{embedding_model} (dim={embedding_provider.dimensions})")
return embedding_provider
def _init_explicit_provider(provider_key: str):
"""Explicit-provider path: build the configured vendor."""
from agent.memory.embedding.provider import EMBEDDING_VENDORS, create_embedding_provider
from config import conf
# Custom providers ("custom:<id>") resolve credentials from custom_providers.
resolved_provider_key = provider_key
if provider_key.startswith("custom:"):
resolved_provider_key = "custom"
meta = EMBEDDING_VENDORS.get(resolved_provider_key)
if meta is None:
logger.error(
f"[EmbeddingFactory] Unknown embedding_provider '{provider_key}'. "
f"Supported: {sorted(EMBEDDING_VENDORS.keys())}. "
f"Memory will run in keyword-only mode."
)
return None
api_key = _resolve_api_key(provider_key)
api_base = _resolve_api_base(provider_key, meta["default_base_url"])
if not api_key:
logger.error(
f"[EmbeddingFactory] embedding_provider='{provider_key}' is set but its "
f"API key is missing. Memory will run in keyword-only mode."
)
return None
model = (conf().get("embedding_model") or "").strip()
# Custom providers without a model fall back to the provider's default.
if not model and resolved_provider_key == "custom":
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
_, custom_id = parse_custom_bot_type(provider_key)
if custom_id:
entry = _find_provider_by_id(get_custom_providers(), custom_id)
if entry and entry.get("model"):
model = entry["model"]
if not model and resolved_provider_key != "custom":
model = meta["default_model"]
try:
cfg_dim = int(conf().get("embedding_dimensions") or 0)
except (TypeError, ValueError):
cfg_dim = 0
dim = cfg_dim if cfg_dim > 0 else meta["default_dimensions"]
try:
provider = create_embedding_provider(
provider=resolved_provider_key,
model=model,
api_key=api_key,
api_base=api_base,
dimensions=dim,
)
except Exception as e:
logger.error(
f"[EmbeddingFactory] Failed to init embedding provider "
f"'{provider_key}/{model}': {e}"
)
return None
_log_provider_once(f"{provider_key}/{model} (dim={provider.dimensions})")
return provider
def _resolve_api_key(provider_key: str) -> str:
"""Pick the API key for an explicit embedding provider from config."""
from config import conf
if provider_key.startswith("custom:"):
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
_, custom_id = parse_custom_bot_type(provider_key)
if custom_id:
entry = _find_provider_by_id(get_custom_providers(), custom_id)
if entry:
return entry.get("api_key", "")
return ""
key_map = {
"openai": "open_ai_api_key",
"linkai": "linkai_api_key",
"dashscope": "dashscope_api_key",
"doubao": "ark_api_key",
"zhipu": "zhipu_ai_api_key",
}
field = key_map.get(provider_key)
if not field:
return ""
value = conf().get(field, "") or ""
if value in ["", "YOUR API KEY", "YOUR_API_KEY"]:
return ""
return value
def _resolve_api_base(provider_key: str, default_base: str) -> str:
"""Pick the API base for an explicit embedding provider from config."""
from config import conf
if provider_key.startswith("custom:"):
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
_, custom_id = parse_custom_bot_type(provider_key)
if custom_id:
entry = _find_provider_by_id(get_custom_providers(), custom_id)
if entry and entry.get("api_base"):
return entry["api_base"]
return default_base
base_map = {
"openai": "open_ai_api_base",
"linkai": "linkai_api_base",
"doubao": "ark_base_url",
"zhipu": "zhipu_ai_api_base",
}
field = base_map.get(provider_key)
if not field:
return default_base
value = (conf().get(field) or "").strip()
if not value:
return default_base
if provider_key == "linkai" and not value.rstrip("/").endswith("/v1"):
return f"{value.rstrip('/')}/v1"
return value
def _log_provider_once(detail: str):
global _embedding_logged
if not _embedding_logged:
logger.info(f"[EmbeddingFactory] Embedding model in use: {detail}")
_embedding_logged = True

View File

@@ -7,10 +7,14 @@ Supports multiple OpenAI-compatible embedding vendors:
- dashscope (Aliyun Tongyi text-embedding-v4)
- doubao (ByteDance Doubao Seed1.5 / large-text on Volcengine Ark)
- zhipu (ZhipuAI embedding-3)
- custom (any OpenAI-compatible endpoint)
Vendor keys here intentionally match the project's bot_type constants in
common.const (OPENAI, LINKAI, QWEN_DASHSCOPE, DOUBAO, ZHIPU_AI).
Custom providers (bot_type "custom" or "custom:<id>") reuse the same
OpenAI-compatible REST client with user-supplied api_key / api_base.
All providers share a single OpenAI-compatible REST client. Vendor-specific
behaviors (truncation, query instruction prefix) are configured via metadata.
"""
@@ -138,6 +142,22 @@ EMBEDDING_VENDORS = {
"query_instruction": "",
"max_batch_size": 64,
},
# Custom provider — any OpenAI-compatible /embeddings endpoint. The
# user must supply api_key + api_base + model via the web console
# (stored in custom_providers list or legacy custom_api_key / custom_api_base).
# Dimensions defaults to 1024 but can be overridden via config's
# embedding_dimensions. No dim-param support assumption — safest
# default for unknown endpoints.
"custom": {
"default_base_url": "",
"default_model": "",
"default_dimensions": 1024,
"supports_dim_param": False,
"needs_client_truncate": False,
"needs_client_normalize": True,
"query_instruction": "",
"max_batch_size": 64,
},
}
@@ -472,10 +492,19 @@ def create_embedding_provider(
)
final_dim = dimensions if (dimensions and dimensions > 0) else meta["default_dimensions"]
resolved_model = model or meta["default_model"]
resolved_base = api_base or meta["default_base_url"]
# Custom providers require explicit api_base and model — they cannot
# fall back to OpenAI defaults like built-in vendors do.
if provider == "custom":
if not resolved_base:
raise ValueError("Custom embedding provider requires an api_base URL")
if not resolved_model:
raise ValueError("Custom embedding provider requires a model name")
return OpenAIEmbeddingProvider(
model=model or meta["default_model"],
model=resolved_model,
api_key=api_key,
api_base=api_base or meta["default_base_url"],
api_base=resolved_base,
extra_headers=extra_headers,
dimensions=final_dim,
supports_dim_param=meta["supports_dim_param"],

View File

@@ -163,10 +163,9 @@ def main() -> int:
logger.info(f"[RebuildIndex] Workspace: {workspace_root}")
logger.info(f"[RebuildIndex] Index db: {memory_config.get_db_path()}")
from bridge.agent_initializer import AgentInitializer
from agent.memory.embedding import create_default_embedding_provider
initializer = AgentInitializer(bridge=None, agent_bridge=None)
embedding_provider = initializer._init_embedding_provider(memory_config, session_id=None)
embedding_provider = create_default_embedding_provider()
if embedding_provider is None:
logger.error(
"[RebuildIndex] No embedding provider could be initialized. "

View File

@@ -31,9 +31,13 @@ def detect_index_dim(storage) -> Optional[int]:
if not row or not row["embedding"]:
return None
try:
emb = json.loads(row["embedding"])
raw = row["embedding"]
if isinstance(raw, (bytes, bytearray)):
# New BLOB format: 4 bytes per float32
return len(raw) // 4
emb = json.loads(raw)
return len(emb) if isinstance(emb, list) else None
except (json.JSONDecodeError, TypeError):
except (json.JSONDecodeError, TypeError, Exception):
return None

View File

@@ -13,7 +13,7 @@ from datetime import datetime, timedelta
from agent.memory.config import MemoryConfig, get_default_memory_config
from agent.memory.storage import MemoryStorage, MemoryChunk, SearchResult
from agent.memory.chunker import TextChunker
from agent.memory.embedding import EmbeddingProvider
from agent.memory.embedding import EmbeddingProvider, EmbeddingCache
from agent.memory.summarizer import MemoryFlushManager, create_memory_files_if_needed
@@ -61,7 +61,11 @@ class MemoryManager:
logger.info(
"[MemoryManager] No embedding provider; memory will use keyword search only"
)
# Cache for query embeddings (avoids redundant API calls within a session)
self._embedding_cache = EmbeddingCache()
# Initialize memory flush manager
workspace_dir = self.config.get_workspace()
self.flush_manager = MemoryFlushManager(
@@ -128,7 +132,14 @@ class MemoryManager:
vector_results = []
if self.embedding_provider:
try:
query_embedding = self.embedding_provider.embed_query(query)
provider_name = type(self.embedding_provider).__name__
model_name = getattr(self.embedding_provider, 'model', '')
cached = self._embedding_cache.get(query, provider_name, model_name)
if cached is not None:
query_embedding = cached
else:
query_embedding = self.embedding_provider.embed_query(query)
self._embedding_cache.put(query, provider_name, model_name, query_embedding)
vector_results = self.storage.search_vector(
query_embedding=query_embedding,
user_id=user_id,

View File

@@ -34,13 +34,18 @@ class MemoryService:
# ------------------------------------------------------------------
def list_files(self, page: int = 1, page_size: int = 20, category: str = "memory") -> dict:
"""
List memory or dream files with metadata (without content).
List memory, dream, or evolution files with metadata (without content).
Args:
category: ``"memory"`` (default) — MEMORY.md + daily files;
``"dream"`` — dream diary files from memory/dreams/
``"dream"`` — dream diary files from memory/dreams/;
``"evolution"`` — self-evolution logs from memory/evolution/
merged with the nightly dream diaries, so
one tab shows everything the agent learned.
"""
if category == "dream":
if category == "evolution":
files = self._list_evolution_files()
elif category == "dream":
files = self._list_dream_files()
else:
files = self._list_memory_files()
@@ -93,6 +98,26 @@ class MemoryService:
return files
def _list_evolution_files(self) -> List[dict]:
"""Self-evolution logs (memory/evolution/*.md) merged with the nightly
dream diaries (memory/dreams/*.md), newest first.
Both are surfaced under the unified "Self-Evolution" tab. A file's
``type`` records its origin so the reader can resolve the right dir.
"""
files: List[dict] = []
for sub, ftype in (("evolution", "evolution"), ("dreams", "dream")):
sub_dir = os.path.join(self.memory_dir, sub)
if not os.path.isdir(sub_dir):
continue
for name in os.listdir(sub_dir):
full = os.path.join(sub_dir, name)
if os.path.isfile(full) and name.endswith(".md"):
files.append(self._file_info(full, name, ftype))
# Sort newest first by filename (date-named); ties favor evolution.
files.sort(key=lambda f: (f["filename"], f["type"] != "evolution"), reverse=True)
return files
# ------------------------------------------------------------------
# content — read a single file
# ------------------------------------------------------------------
@@ -101,7 +126,7 @@ class MemoryService:
Read the full content of a memory or dream file.
:param filename: File name, e.g. ``MEMORY.md``, ``2026-02-20.md``
:param category: ``"memory"`` or ``"dream"``
:param category: ``"memory"``, ``"dream"`` or ``"evolution"``
:return: dict with ``filename`` and ``content``
:raises FileNotFoundError: if the file does not exist
"""
@@ -125,7 +150,7 @@ class MemoryService:
Dispatch a memory management action.
:param action: ``list`` or ``content``
:param payload: action-specific payload (supports ``category``: ``"memory"`` | ``"dream"``)
:param payload: action-specific payload (supports ``category``: ``"memory"`` | ``"dream"`` | ``"evolution"``)
:return: protocol-compatible response dict
"""
payload = payload or {}
@@ -166,6 +191,7 @@ class MemoryService:
- ``MEMORY.md`` → ``{workspace_root}/MEMORY.md``
- ``2026-02-20.md`` (memory) → ``{workspace_root}/memory/2026-02-20.md``
- ``2026-02-20.md`` (dream) → ``{workspace_root}/memory/dreams/2026-02-20.md``
- ``2026-02-20.md`` (evolution) → ``{workspace_root}/memory/evolution/2026-02-20.md``
Raises ValueError if the resolved path escapes the allowed directory.
"""
@@ -173,6 +199,8 @@ class MemoryService:
base_dir = self.workspace_root
elif category == "dream":
base_dir = os.path.join(self.memory_dir, "dreams")
elif category == "evolution":
base_dir = os.path.join(self.memory_dir, "evolution")
else:
base_dir = self.memory_dir

View File

@@ -5,12 +5,42 @@ Provides vector and keyword search capabilities
"""
from __future__ import annotations
import re
import sqlite3
import json
import hashlib
import threading
from typing import List, Dict, Optional, Any
from pathlib import Path
from dataclasses import dataclass
try:
import numpy as np
_HAS_NUMPY = True
except ImportError:
_HAS_NUMPY = False
np = None # type: ignore[assignment]
# UPSERT (INSERT … ON CONFLICT DO UPDATE) requires SQLite ≥ 3.24.0 (2018).
# Older systems (e.g. CentOS 7 ships SQLite 3.7) fall back to INSERT OR REPLACE,
# which risks FTS5 rowid drift on chunk updates (see save_chunk docstring).
_HAS_UPSERT = sqlite3.sqlite_version_info >= (3, 24, 0)
# ---------------------------------------------------------------------------
# CJK character ranges, compiled once at module load.
# Covers: CJK Symbols/Punctuation, Japanese kana (hiragana + katakana),
# CJK Unified Ideographs + Extension A, Korean syllables (Hangul),
# CJK Compatibility Ideographs, and CJK Extension BF.
# ---------------------------------------------------------------------------
_CJK_RANGES = (
r'\u3000-\u30ff' # CJK Symbols/Punctuation + Japanese kana
r'\u3400-\u9fff' # CJK Unified Ideographs (incl. Extension A)
r'\uac00-\ud7af' # Korean syllables (Hangul)
r'\uf900-\ufaff' # CJK Compatibility Ideographs
r'\U00020000-\U0002fa1f' # CJK Extension BF
)
_RE_CONTAINS_CJK = re.compile(f'[{_CJK_RANGES}]')
_RE_CJK_WORDS = re.compile(f'[{_CJK_RANGES}]+')
_RE_TRIGRAM_TOKENS = re.compile(f'[{_CJK_RANGES}]+|[A-Za-z0-9_]+')
@dataclass
@@ -48,6 +78,10 @@ class MemoryStorage:
self.db_path = db_path
self.conn: Optional[sqlite3.Connection] = None
self.fts5_available = False # Track FTS5 availability
# RLock protects concurrent writes from the same process.
# SQLite WAL mode handles read/write concurrency at the file level,
# but same-process concurrent writes still need a Python-level lock.
self._lock = threading.RLock()
self._init_db()
def _check_fts5_support(self) -> bool:
@@ -69,6 +103,14 @@ class MemoryStorage:
# Check FTS5 support
self.fts5_available = self._check_fts5_support()
if not _HAS_UPSERT:
from common.log import logger
logger.warning(
"[MemoryStorage] SQLite %s < 3.24 — UPSERT unavailable. "
"Falling back to INSERT OR REPLACE; FTS5 rowid may drift on "
"chunk updates (rebuild index periodically to recover).",
sqlite3.sqlite_version,
)
if not self.fts5_available:
from common.log import logger
logger.debug("[MemoryStorage] FTS5 not available, using LIKE-based keyword search")
@@ -175,6 +217,75 @@ class MemoryStorage:
)
self._rebuild_fts5_from_chunks()
# Internal key-value store for persistent flags (e.g. backfill tracking)
self.conn.execute("""
CREATE TABLE IF NOT EXISTS _meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL
)
""")
# Create trigram FTS5 table for CJK / mixed-language search
self.trigram_fts5_available = False
if self.fts5_available:
try:
self.conn.execute("""
CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts_trigram USING fts5(
text,
id UNINDEXED,
user_id UNINDEXED,
path UNINDEXED,
source UNINDEXED,
scope UNINDEXED,
content='chunks',
content_rowid='rowid',
tokenize='trigram case_sensitive 0'
)
""")
self.conn.execute("""
CREATE TRIGGER IF NOT EXISTS chunks_trigram_ai
AFTER INSERT ON chunks BEGIN
INSERT INTO chunks_fts_trigram(rowid, text, id, user_id, path, source, scope)
VALUES (new.rowid, new.text, new.id, new.user_id, new.path, new.source, new.scope);
END
""")
self.conn.execute("""
CREATE TRIGGER IF NOT EXISTS chunks_trigram_ad
AFTER DELETE ON chunks BEGIN
DELETE FROM chunks_fts_trigram WHERE rowid = old.rowid;
END
""")
self.conn.execute("""
CREATE TRIGGER IF NOT EXISTS chunks_trigram_au
AFTER UPDATE ON chunks BEGIN
UPDATE chunks_fts_trigram
SET text=new.text, id=new.id, user_id=new.user_id,
path=new.path, source=new.source, scope=new.scope
WHERE rowid = new.rowid;
END
""")
# One-time backfill for existing rows.
# NOTE: COUNT(*) on an FTS5 content table always returns 0, so we
# use a persistent flag in _meta instead of counting trigram rows.
backfill_done = self.conn.execute(
"SELECT 1 FROM _meta WHERE key = 'trigram_backfill_done'"
).fetchone()
chunks_count = self.conn.execute(
"SELECT COUNT(*) as c FROM chunks"
).fetchone()['c']
if chunks_count > 0 and not backfill_done:
self.conn.execute(
"INSERT INTO chunks_fts_trigram(chunks_fts_trigram) VALUES('rebuild')"
)
self.conn.execute(
"INSERT OR REPLACE INTO _meta(key, value) VALUES('trigram_backfill_done', '1')"
)
self.trigram_fts5_available = True
except Exception:
from common.log import logger
logger.warning("[MemoryStorage] trigram FTS5 unavailable, CJK search will use LIKE fallback", exc_info=True)
self.trigram_fts5_available = False
# Create files metadata table
self.conn.execute("""
CREATE TABLE IF NOT EXISTS files (
@@ -186,7 +297,7 @@ class MemoryStorage:
updated_at INTEGER DEFAULT (strftime('%s', 'now'))
)
""")
self.conn.commit()
def _fts5_state_inconsistent(self) -> bool:
@@ -299,43 +410,98 @@ class MemoryStorage:
self.conn.commit()
def save_chunk(self, chunk: MemoryChunk):
"""Save a memory chunk"""
self.conn.execute("""
INSERT OR REPLACE INTO chunks
(id, user_id, scope, source, path, start_line, end_line, text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
""", (
chunk.id,
chunk.user_id,
chunk.scope,
chunk.source,
chunk.path,
chunk.start_line,
chunk.end_line,
chunk.text,
json.dumps(chunk.embedding) if chunk.embedding else None,
"""Save a memory chunk (insert or update by id).
Uses SQLite UPSERT (INSERT … ON CONFLICT DO UPDATE) instead of
INSERT OR REPLACE. INSERT OR REPLACE internally does DELETE+INSERT,
which changes the row's rowid. Because both FTS5 tables use
content_rowid='rowid', a new rowid would leave the old FTS index
entries pointing at a non-existent rowid and trigger
"fts5: missing row N from content table" errors.
ON CONFLICT DO UPDATE fires the AFTER UPDATE trigger (chunks_au /
chunks_trigram_au) and keeps the original rowid intact.
"""
if _HAS_UPSERT:
_SQL = """
INSERT INTO chunks
(id, user_id, scope, source, path, start_line, end_line,
text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
ON CONFLICT(id) DO UPDATE SET
user_id = excluded.user_id,
scope = excluded.scope,
source = excluded.source,
path = excluded.path,
start_line = excluded.start_line,
end_line = excluded.end_line,
text = excluded.text,
embedding = excluded.embedding,
hash = excluded.hash,
metadata = excluded.metadata,
updated_at = strftime('%s', 'now')
"""
else:
_SQL = """
INSERT OR REPLACE INTO chunks
(id, user_id, scope, source, path, start_line, end_line,
text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
"""
params = (
chunk.id, chunk.user_id, chunk.scope, chunk.source, chunk.path,
chunk.start_line, chunk.end_line, chunk.text,
self._encode_embedding(chunk.embedding),
chunk.hash,
json.dumps(chunk.metadata) if chunk.metadata else None
))
self.conn.commit()
json.dumps(chunk.metadata) if chunk.metadata else None,
)
with self._lock:
self.conn.execute(_SQL, params)
self.conn.commit()
def save_chunks_batch(self, chunks: List[MemoryChunk]):
"""Save multiple chunks in a batch"""
self.conn.executemany("""
INSERT OR REPLACE INTO chunks
(id, user_id, scope, source, path, start_line, end_line, text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
""", [
"""Save multiple chunks in a batch (insert or update by id).
See save_chunk for why UPSERT is used instead of INSERT OR REPLACE.
"""
if _HAS_UPSERT:
_SQL = """
INSERT INTO chunks
(id, user_id, scope, source, path, start_line, end_line,
text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
ON CONFLICT(id) DO UPDATE SET
user_id = excluded.user_id,
scope = excluded.scope,
source = excluded.source,
path = excluded.path,
start_line = excluded.start_line,
end_line = excluded.end_line,
text = excluded.text,
embedding = excluded.embedding,
hash = excluded.hash,
metadata = excluded.metadata,
updated_at = strftime('%s', 'now')
"""
else:
_SQL = """
INSERT OR REPLACE INTO chunks
(id, user_id, scope, source, path, start_line, end_line,
text, embedding, hash, metadata, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
"""
params_list = [
(
c.id, c.user_id, c.scope, c.source, c.path,
c.start_line, c.end_line, c.text,
json.dumps(c.embedding) if c.embedding else None,
self._encode_embedding(c.embedding),
c.hash,
json.dumps(c.metadata) if c.metadata else None
json.dumps(c.metadata) if c.metadata else None,
)
for c in chunks
])
self.conn.commit()
]
with self._lock:
self.conn.executemany(_SQL, params_list)
self.conn.commit()
def get_chunk(self, chunk_id: str) -> Optional[MemoryChunk]:
"""Get a chunk by ID"""
@@ -356,21 +522,21 @@ class MemoryStorage:
limit: int = 10
) -> List[SearchResult]:
"""
Vector similarity search using in-memory cosine similarity
(sqlite-vec can be added later for better performance)
Vector similarity search using numpy-vectorized cosine similarity.
All embeddings are loaded then scored in a single BLAS matrix-vector
multiply, which is ~100x faster than the pure-Python per-row loop.
"""
if scopes is None:
scopes = ["shared"]
if user_id:
scopes.append("user")
# Build query
scope_placeholders = ','.join('?' * len(scopes))
params = scopes
params = list(scopes)
if user_id:
query = f"""
SELECT * FROM chunks
SELECT * FROM chunks
WHERE scope IN ({scope_placeholders})
AND (scope = 'shared' OR user_id = ?)
AND embedding IS NOT NULL
@@ -378,51 +544,95 @@ class MemoryStorage:
params.append(user_id)
else:
query = f"""
SELECT * FROM chunks
SELECT * FROM chunks
WHERE scope IN ({scope_placeholders})
AND embedding IS NOT NULL
"""
rows = self.conn.execute(query, params).fetchall()
if not rows:
return []
# Calculate cosine similarity. We probe the first row's dim to fail
# loudly on a query/index dim mismatch — otherwise every doc would
# score 0 silently, leaving the user wondering why search broke.
results = []
query_dim = len(query_embedding)
if rows:
first = json.loads(rows[0]['embedding'])
if isinstance(first, list) and len(first) != query_dim:
raise ValueError(
f"Embedding dim mismatch: query is {query_dim}-dim but "
f"index stores {len(first)}-dim vectors. The configured "
f"embedding model differs from the one that built the "
f"index — run /memory rebuild-index to re-embed."
)
# Parse embeddings and build a (N, D) matrix in one pass.
# New rows store BLOB bytes (np.frombuffer); legacy rows fall back to JSON.
# Filter out rows whose embedding dimension differs from the query —
# mixing dimensions would cause np.array() to produce an object array
# and matrix @ q_vec to raise ValueError.
expected_dim = len(query_embedding)
valid_rows = []
vectors = []
for row in rows:
embedding = json.loads(row['embedding'])
similarity = self._cosine_similarity(query_embedding, embedding)
vec = self._decode_embedding(row['embedding'])
if not vec:
continue
if len(vec) != expected_dim:
from common.log import logger
logger.warning(
"[MemoryStorage] Skipping chunk %s: embedding dim %d != query dim %d",
row['id'], len(vec), expected_dim
)
continue
valid_rows.append(row)
vectors.append(vec)
if similarity > 0:
results.append((similarity, row))
# Sort by similarity and limit
results.sort(key=lambda x: x[0], reverse=True)
results = results[:limit]
return [
SearchResult(
path=row['path'],
start_line=row['start_line'],
end_line=row['end_line'],
score=score,
snippet=self._truncate_text(row['text'], 500),
source=row['source'],
user_id=row['user_id']
)
for score, row in results
]
if not vectors:
return []
if _HAS_NUMPY:
matrix = np.array(vectors, dtype=np.float32) # (N, D)
q_vec = np.array(query_embedding, dtype=np.float32) # (D,)
# Vectorized cosine similarity: dot(matrix, q) / (||matrix|| * ||q||)
dots = matrix @ q_vec # (N,)
row_norms = np.linalg.norm(matrix, axis=1) # (N,)
q_norm = float(np.linalg.norm(q_vec))
denominators = row_norms * q_norm
np.maximum(denominators, 1e-10, out=denominators) # avoid div-by-zero
sims = dots / denominators # (N,)
# Select TopK using argpartition (O(N) average), then sort only those K
k = min(limit, len(valid_rows))
top_idx = np.argpartition(sims, -k)[-k:]
top_idx = top_idx[np.argsort(sims[top_idx])[::-1]]
return [
SearchResult(
path=valid_rows[i]['path'],
start_line=valid_rows[i]['start_line'],
end_line=valid_rows[i]['end_line'],
score=float(sims[i]),
snippet=self._truncate_text(valid_rows[i]['text'], 500),
source=valid_rows[i]['source'],
user_id=valid_rows[i]['user_id']
)
for i in top_idx
if sims[i] > 0
]
else:
# Pure-Python cosine similarity fallback (numpy not installed)
import math
q = query_embedding
q_norm = math.sqrt(sum(x * x for x in q)) or 1e-10
scored = []
for i, vec in enumerate(vectors):
dot = sum(a * b for a, b in zip(vec, q))
v_norm = math.sqrt(sum(x * x for x in vec)) or 1e-10
sim = dot / (v_norm * q_norm)
if sim > 0:
scored.append((sim, valid_rows[i]))
scored.sort(key=lambda x: x[0], reverse=True)
return [
SearchResult(
path=row['path'],
start_line=row['start_line'],
end_line=row['end_line'],
score=sim,
snippet=self._truncate_text(row['text'], 500),
source=row['source'],
user_id=row['user_id']
)
for sim, row in scored[:limit]
]
def search_keyword(
self,
@@ -445,12 +655,37 @@ class MemoryStorage:
if user_id:
scopes.append("user")
if self.fts5_available:
# Step 1: Standard FTS5 (unicode61) — pure ASCII queries only.
# Skipped when query contains any CJK characters: unicode61 tokenises CJK
# as individual characters without forming meaningful tokens, so it would
# match only the ASCII portion of a mixed query (e.g. "Python" from
# "Python教程") and silently discard the CJK part. Those queries go
# directly to Step 2 (trigram), which handles both ASCII and CJK together.
fts1_attempted = False
if (self.fts5_available
and not MemoryStorage._contains_cjk(query)
and MemoryStorage._build_fts_query(query)):
fts1_attempted = True
fts_results = self._search_fts5(query, user_id, scopes, limit)
if fts_results:
return fts_results
return self._search_like(query, user_id, scopes, limit)
# Step 2: Trigram FTS5 — CJK/mixed queries, plus fallback when unicode61
# returned nothing (trigram indexes all scripts with 3-char sliding windows,
# so it can catch terms that unicode61 tokenisation misses).
if self.trigram_fts5_available and (
MemoryStorage._contains_cjk(query) or fts1_attempted
):
trigram_results = self._search_fts5_trigram(query, user_id, scopes, limit)
if trigram_results:
return trigram_results
# Step 3: LIKE fallback — last resort (FTS5 unavailable, or CJK tokens
# shorter than 3 characters that trigram cannot match, e.g. a single-char query).
if not self.fts5_available or MemoryStorage._contains_cjk(query):
return self._search_like(query, user_id, scopes, limit)
return []
def _search_fts5(
self,
@@ -471,7 +706,7 @@ class MemoryStorage:
sql_query = f"""
SELECT chunks.*, bm25(chunks_fts) as rank
FROM chunks_fts
JOIN chunks ON chunks.id = chunks_fts.id
JOIN chunks ON chunks.rowid = chunks_fts.rowid
WHERE chunks_fts MATCH ?
AND chunks.scope IN ({scope_placeholders})
AND (chunks.scope = 'shared' OR chunks.user_id = ?)
@@ -483,7 +718,7 @@ class MemoryStorage:
sql_query = f"""
SELECT chunks.*, bm25(chunks_fts) as rank
FROM chunks_fts
JOIN chunks ON chunks.id = chunks_fts.id
JOIN chunks ON chunks.rowid = chunks_fts.rowid
WHERE chunks_fts MATCH ?
AND chunks.scope IN ({scope_placeholders})
ORDER BY rank
@@ -505,13 +740,11 @@ class MemoryStorage:
)
for row in rows
]
except Exception as e:
except Exception:
from common.log import logger
logger.error(
f"[MemoryStorage] FTS5 search failed (caller will fall back to LIKE): {e}"
)
logger.warning("[MemoryStorage] _search_fts5 failed, returning empty", exc_info=True)
return []
def _search_like(
self,
query: str,
@@ -522,12 +755,11 @@ class MemoryStorage:
"""LIKE-based search.
Used as the keyword-search fallback when FTS5 is unavailable, fails,
or returns empty. Supports both CJK runs and ASCII word tokens so it
can serve as a true safety net for any query.
or returns empty. Supports both CJK runs (1+ chars) and ASCII word
tokens (3+ chars) so it can serve as a true safety net for any query.
"""
import re
# CJK runs (2+ chars) + ASCII word tokens (3+ chars to avoid noise)
cjk_words = re.findall(r'[\u4e00-\u9fff]{2,}', query)
# CJK runs (1+ chars, wide Unicode range) + ASCII words (3+ chars to avoid noise)
cjk_words = _RE_CJK_WORDS.findall(query)
ascii_words = [t for t in re.findall(r'[A-Za-z0-9_]+', query) if len(t) >= 3]
words = cjk_words + ascii_words
if not words:
@@ -565,44 +797,55 @@ class MemoryStorage:
try:
rows = self.conn.execute(sql_query, params).fetchall()
return [
SearchResult(
results = []
for row in rows:
# Dynamic score: reward chunks that contain more of the query words.
# Use all tokens (CJK + ASCII) so pure-ASCII queries are not skipped.
# matched_count is always ≥1 because the WHERE clause uses OR, but
# guard defensively so unexpected zero-match rows are never surfaced.
text_lower = row['text'].lower()
matched_count = sum(1 for w in words if w.lower() in text_lower)
if matched_count == 0:
continue
score = min(0.85, 0.3 + 0.15 * matched_count)
results.append(SearchResult(
path=row['path'],
start_line=row['start_line'],
end_line=row['end_line'],
score=0.5, # Fixed score for LIKE search
score=score,
snippet=self._truncate_text(row['text'], 500),
source=row['source'],
user_id=row['user_id']
)
for row in rows
]
except Exception as e:
))
results.sort(key=lambda r: r.score, reverse=True)
return results
except Exception:
from common.log import logger
logger.error(f"[MemoryStorage] LIKE search failed: {e}")
logger.warning("[MemoryStorage] _search_like failed, returning empty", exc_info=True)
return []
def delete_by_path(self, path: str):
"""Delete all chunks from a file"""
self.conn.execute("""
DELETE FROM chunks WHERE path = ?
""", (path,))
self.conn.commit()
"""Delete all chunks and file metadata for a path."""
with self._lock:
self.conn.execute("DELETE FROM chunks WHERE path = ?", (path,))
self.conn.execute("DELETE FROM files WHERE path = ?", (path,))
self.conn.commit()
def get_file_hash(self, path: str) -> Optional[str]:
"""Get stored file hash"""
row = self.conn.execute("""
SELECT hash FROM files WHERE path = ?
""", (path,)).fetchone()
return row['hash'] if row else None
def update_file_metadata(self, path: str, source: str, file_hash: str, mtime: int, size: int):
"""Update file metadata"""
self.conn.execute("""
INSERT OR REPLACE INTO files (path, source, hash, mtime, size, updated_at)
VALUES (?, ?, ?, ?, ?, strftime('%s', 'now'))
""", (path, source, file_hash, mtime, size))
self.conn.commit()
with self._lock:
self.conn.execute("""
INSERT OR REPLACE INTO files (path, source, hash, mtime, size, updated_at)
VALUES (?, ?, ?, ?, ?, strftime('%s', 'now'))
""", (path, source, file_hash, mtime, size))
self.conn.commit()
def get_stats(self) -> Dict[str, int]:
"""Get storage statistics"""
@@ -632,7 +875,8 @@ class MemoryStorage:
self.conn.close()
self.conn = None # Mark as closed
except Exception as e:
print(f"⚠️ Error closing database connection: {e}")
from common.log import logger
logger.warning("[MemoryStorage] Error closing database connection: %s", e)
def __del__(self):
"""Destructor to ensure connection is closed"""
@@ -642,7 +886,33 @@ class MemoryStorage:
pass # Ignore errors during cleanup
# Helper methods
@staticmethod
def _encode_embedding(embedding: Optional[List[float]]) -> Optional[bytes]:
"""Encode embedding as float32 BLOB bytes (~6x smaller and faster than JSON).
Falls back to struct.pack when numpy is unavailable."""
if embedding is None:
return None
if _HAS_NUMPY:
return np.array(embedding, dtype=np.float32).tobytes()
import struct
return struct.pack(f'{len(embedding)}f', *embedding)
@staticmethod
def _decode_embedding(raw) -> Optional[List[float]]:
"""Decode embedding from BLOB bytes or legacy JSON string.
Handles both numpy and numpy-free environments."""
if raw is None:
return None
if isinstance(raw, (bytes, bytearray)):
if _HAS_NUMPY:
return np.frombuffer(raw, dtype=np.float32).tolist()
import struct
n = len(raw) // 4
return list(struct.unpack(f'{n}f', raw))
# Legacy JSON format written by older versions
return json.loads(raw)
def _row_to_chunk(self, row) -> MemoryChunk:
"""Convert database row to MemoryChunk"""
return MemoryChunk(
@@ -654,32 +924,89 @@ class MemoryStorage:
start_line=row['start_line'],
end_line=row['end_line'],
text=row['text'],
embedding=json.loads(row['embedding']) if row['embedding'] else None,
embedding=self._decode_embedding(row['embedding']),
hash=row['hash'],
metadata=json.loads(row['metadata']) if row['metadata'] else None
)
@staticmethod
def _cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
"""Calculate cosine similarity between two vectors"""
if len(vec1) != len(vec2):
return 0.0
dot_product = sum(a * b for a, b in zip(vec1, vec2))
norm1 = sum(a * a for a in vec1) ** 0.5
norm2 = sum(b * b for b in vec2) ** 0.5
if norm1 == 0 or norm2 == 0:
return 0.0
return dot_product / (norm1 * norm2)
def _contains_cjk(text: str) -> bool:
"""Check if text contains CJK or related characters (Chinese, Japanese, Korean)."""
return bool(_RE_CONTAINS_CJK.search(text))
@staticmethod
def _contains_cjk(text: str) -> bool:
"""Check if text contains CJK (Chinese/Japanese/Korean) characters"""
import re
return bool(re.search(r'[\u4e00-\u9fff]', text))
def _build_trigram_query(raw_query: str) -> Optional[str]:
"""
Build FTS5 MATCH query for the trigram tokenizer.
Extracts CJK sequences (including single characters) and ASCII words,
joining them with AND so all terms must appear in the matched chunk.
"""
tokens = _RE_TRIGRAM_TOKENS.findall(raw_query)
tokens = [t for t in tokens if t]
if not tokens:
return None
# Escape embedded double-quotes (FTS5 uses "" inside quoted phrases)
quoted = [f'"{t.replace(chr(34), chr(34)*2)}"' for t in tokens]
return ' AND '.join(quoted)
def _search_fts5_trigram(
self,
query: str,
user_id: Optional[str],
scopes: List[str],
limit: int
) -> List[SearchResult]:
"""Trigram FTS5 search — handles CJK and mixed queries with BM25 ranking."""
trigram_query = self._build_trigram_query(query)
if not trigram_query:
return []
scope_placeholders = ','.join('?' * len(scopes))
params = [trigram_query] + list(scopes)
if user_id:
sql = f"""
SELECT chunks.*, bm25(chunks_fts_trigram) as rank
FROM chunks_fts_trigram
JOIN chunks ON chunks.rowid = chunks_fts_trigram.rowid
WHERE chunks_fts_trigram MATCH ?
AND chunks.scope IN ({scope_placeholders})
AND (chunks.scope = 'shared' OR chunks.user_id = ?)
ORDER BY rank
LIMIT ?
"""
params.extend([user_id, limit])
else:
sql = f"""
SELECT chunks.*, bm25(chunks_fts_trigram) as rank
FROM chunks_fts_trigram
JOIN chunks ON chunks.rowid = chunks_fts_trigram.rowid
WHERE chunks_fts_trigram MATCH ?
AND chunks.scope IN ({scope_placeholders})
ORDER BY rank
LIMIT ?
"""
params.append(limit)
try:
rows = self.conn.execute(sql, params).fetchall()
return [
SearchResult(
path=row['path'],
start_line=row['start_line'],
end_line=row['end_line'],
score=self._bm25_rank_to_score(row['rank']),
snippet=self._truncate_text(row['text'], 500),
source=row['source'],
user_id=row['user_id']
)
for row in rows
]
except Exception:
from common.log import logger
logger.warning("[MemoryStorage] _search_fts5_trigram failed, returning empty", exc_info=True)
return []
@staticmethod
def _build_fts_query(raw_query: str) -> Optional[str]:
"""
@@ -688,7 +1015,6 @@ class MemoryStorage:
Works best for English and word-based languages.
For CJK characters, LIKE search will be used as fallback.
"""
import re
# Extract words (primarily English words and numbers)
tokens = re.findall(r'[A-Za-z0-9_]+', raw_query)
if not tokens:
@@ -701,9 +1027,22 @@ class MemoryStorage:
@staticmethod
def _bm25_rank_to_score(rank: float) -> float:
"""Convert BM25 rank to 0-1 score"""
normalized = max(0, rank) if rank is not None else 999
return 1 / (1 + normalized)
"""Convert SQLite BM25 rank to a [0, 1) relevance score.
SQLite's bm25() returns a non-positive float (0 or negative).
More negative = more relevant. max(0, rank) would clip every
negative value to 0, making every score 1/(1+0) = 1.0 and
destroying all ranking information.
abs(rank) / (1 + abs(rank)) maps the absolute relevance magnitude
to [0, 1): larger |rank| (stronger match) → score closer to 1.
"""
if rank is None:
return 0.0
# Add a floor of 0.3 so any FTS5 match always exceeds typical
# min_score thresholds (default 0.1). Small-corpus ranks close to
# 0 would otherwise produce score≈0 and be filtered out downstream.
return 0.3 + 0.69 * (abs(rank) / (1.0 + abs(rank)))
@staticmethod
def _truncate_text(text: str, max_chars: int) -> str:

View File

@@ -16,7 +16,7 @@ from datetime import datetime
from common.log import logger
SUMMARIZE_SYSTEM_PROMPT = """你是一个对话记录助手。请将对话内容归纳为当天的日常记录。
SUMMARIZE_SYSTEM_PROMPT_ZH = """你是一个对话记录助手。请将对话内容归纳为当天的日常记录。
## 要求
@@ -28,7 +28,23 @@ SUMMARIZE_SYSTEM_PROMPT = """你是一个对话记录助手。请将对话内容
当对话没有任何记录价值(仅含问候或无意义内容),直接回复"""""
SUMMARIZE_USER_PROMPT = """请归纳以下对话的日常记录:
SUMMARIZE_SYSTEM_PROMPT_EN = """You are a conversation-logging assistant. Summarize the conversation into a daily record.
## Requirements
Summarize by "event", not turn by turn:
- One item per line, starting with "- "
- Merge multiple turns about the same thing
- Only record meaningful events; ignore small talk and greetings
- Keep key decisions, conclusions and to-dos
If the conversation has no record value (only greetings or meaningless content), reply with exactly "None"."""
SUMMARIZE_USER_PROMPT_ZH = """请归纳以下对话的日常记录:
{conversation}"""
SUMMARIZE_USER_PROMPT_EN = """Summarize the daily record of the following conversation:
{conversation}"""
@@ -36,7 +52,7 @@ SUMMARIZE_USER_PROMPT = """请归纳以下对话的日常记录:
# Deep Dream prompts — distill daily memories → MEMORY.md + dream diary
# ---------------------------------------------------------------------------
DREAM_SYSTEM_PROMPT = """你是一个记忆整理助手,负责定期整理用户的长期记忆。
DREAM_SYSTEM_PROMPT_ZH = """你是一个记忆整理助手,负责定期整理用户的长期记忆。
你将收到两份材料:
1. **当前长期记忆** — MEMORY.md 的全部现有内容
@@ -80,7 +96,51 @@ MEMORY.md 会注入每次对话的系统提示词中,因此必须保持精炼
梦境日记内容...
```"""
DREAM_USER_PROMPT = """## 当前长期记忆MEMORY.md
DREAM_SYSTEM_PROMPT_EN = """You are a memory-curation assistant that periodically organizes the user's long-term memory.
You will receive two inputs:
1. **Current long-term memory** — the full existing content of MEMORY.md
2. **Today's diary** — the daily records
MEMORY.md is injected into the system prompt of every conversation, so it must stay concise and hold only valuable, memory-worthy content.
**Important: organize strictly based on the provided material. Never fabricate, infer, or add information not present in it.**
## Tasks
### Part 1: Updated long-term memory ([MEMORY])
Organize and distill on top of the existing memory, and output the complete updated content:
- **Merge & distill**: combine semantically similar items into one dense statement rather than listing them
- **Extract new**: pull memory-worthy new info from today's diary (preferences, decisions, people, rules, lessons)
- **Resolve conflicts**: when new info contradicts an old item, prefer the new and replace the old
- **Clean invalid**: remove temporary notes, blank items, formatting residue, meaningless or duplicate content
- **Drop redundancy**: delete old items already covered by a more concise statement
- One item per line, starting with "- ", without a date prefix
- You may group related items under "## headings" for clarity
- Goal: keep under 50 items, each ideally a single sentence
### Part 2: Dream diary ([DREAM])
Write a short diary in a concise narrative style recording what this curation found, keep it clean and readable:
- Which duplicates or conflicts were found
- What new insights were extracted from the diary
- What cleanup and optimization was done
- Overall feelings and observations
## Output format (follow strictly)
```
[MEMORY]
- memory item 1
- memory item 2
...
[DREAM]
dream diary content...
```"""
DREAM_USER_PROMPT_ZH = """## 当前长期记忆MEMORY.md
{memory_content}
@@ -88,6 +148,47 @@ DREAM_USER_PROMPT = """## 当前长期记忆MEMORY.md
{daily_content}"""
DREAM_USER_PROMPT_EN = """## Current long-term memory (MEMORY.md)
{memory_content}
## Recent diary (last {days} days)
{daily_content}"""
def _is_en() -> bool:
"""True when the resolved UI language is English."""
try:
from common import i18n
return i18n.get_language() == "en"
except Exception:
return False
def _summarize_system_prompt() -> str:
return SUMMARIZE_SYSTEM_PROMPT_EN if _is_en() else SUMMARIZE_SYSTEM_PROMPT_ZH
def _summarize_user_prompt() -> str:
return SUMMARIZE_USER_PROMPT_EN if _is_en() else SUMMARIZE_USER_PROMPT_ZH
def _dream_system_prompt() -> str:
return DREAM_SYSTEM_PROMPT_EN if _is_en() else DREAM_SYSTEM_PROMPT_ZH
def _dream_user_prompt() -> str:
return DREAM_USER_PROMPT_EN if _is_en() else DREAM_USER_PROMPT_ZH
def _is_empty_sentinel(text: str) -> bool:
"""Match the "no record value" sentinel in both zh ("") and en ("None")."""
if not text:
return True
s = text.strip()
return s == "" or s == "" or s.lower() == "none"
class MemoryFlushManager:
@@ -224,7 +325,7 @@ class MemoryFlushManager:
"""Background worker: summarize with LLM, write daily memory file."""
try:
raw_summary = self._summarize_messages(messages, max_messages)
if not raw_summary or not raw_summary.strip() or raw_summary.strip() == "":
if _is_empty_sentinel(raw_summary):
logger.info(f"[MemoryFlush] No valuable content to flush (reason={reason})")
return
@@ -264,7 +365,7 @@ class MemoryFlushManager:
def _clean_summary_output(raw: str) -> str:
"""Strip legacy [DAILY]/[MEMORY] markers if present, return clean daily text."""
raw = raw.strip()
if not raw or raw == "":
if _is_empty_sentinel(raw):
return ""
# Strip [DAILY] marker
@@ -318,6 +419,17 @@ class MemoryFlushManager:
lookback_days: How many days of daily files to read (default 1 for scheduled, 3 for manual)
force: Skip input-hash dedup check (used by manual /memory dream trigger)
"""
# Config guard for scheduled runs. Manual trigger (force=True) always
# runs since it is an explicit user action.
if not force:
try:
from config import conf
if not conf().get("deep_dream_enabled", True):
logger.info("[DeepDream] deep_dream_enabled=false, skipping")
return False
except Exception:
pass
if not self.llm_model:
logger.warning("[DeepDream] No LLM model available, skipping")
return False
@@ -355,21 +467,20 @@ class MemoryFlushManager:
import time as _time
t0 = _time.monotonic()
try:
user_msg = DREAM_USER_PROMPT.format(
user_msg = _dream_user_prompt().format(
memory_content=memory_content or "(empty)",
days=lookback_days,
daily_content=daily_content or "(no recent daily records)",
)
from agent.protocol.models import LLMRequest
# Scale max_tokens based on input size to avoid truncating large MEMORY.md
input_chars = len(memory_content) + len(daily_content)
dream_max_tokens = max(2000, min(input_chars, 8000))
# No output cap: the prompt already keeps MEMORY.md concise (~50
# items), so a hard max_tokens would only risk truncating a large
# rewrite. Let the model use its default output budget.
request = LLMRequest(
messages=[{"role": "user", "content": user_msg}],
temperature=0.3,
max_tokens=dream_max_tokens,
stream=False,
system=DREAM_SYSTEM_PROMPT,
system=_dream_system_prompt(),
)
response = self.llm_model.call(request)
raw = self._extract_response_text(response)
@@ -501,9 +612,9 @@ class MemoryFlushManager:
if self.llm_model:
try:
summary = self._call_llm_for_summary(conversation_text)
if summary and summary.strip() and summary.strip() != "":
if not _is_empty_sentinel(summary):
return summary.strip()
logger.info("[MemoryFlush] LLM returned empty or '', skipping write")
logger.info("[MemoryFlush] LLM returned empty sentinel, skipping write")
return ""
except Exception as e:
logger.warning(f"[MemoryFlush] LLM summarization failed, using fallback: {e}")
@@ -579,11 +690,11 @@ class MemoryFlushManager:
from agent.protocol.models import LLMRequest
request = LLMRequest(
messages=[{"role": "user", "content": SUMMARIZE_USER_PROMPT.format(conversation=conversation_text)}],
messages=[{"role": "user", "content": _summarize_user_prompt().format(conversation=conversation_text)}],
temperature=0,
max_tokens=500,
stream=False,
system=SUMMARIZE_SYSTEM_PROMPT,
system=_summarize_system_prompt(),
)
response = self.llm_model.call(request)

View File

@@ -15,13 +15,13 @@ from config import conf
@dataclass
class ContextFile:
"""上下文文件"""
"""A context file (path + content)."""
path: str
content: str
class PromptBuilder:
"""提示词构建器"""
"""System prompt builder."""
def __init__(self, workspace_dir: str, language: str = "zh"):
"""
@@ -88,97 +88,144 @@ def build_agent_system_prompt(
**kwargs
) -> str:
"""
构建Agent系统提示词
顺序说明(按重要性和逻辑关系排列):
1. 工具系统 - 核心能力,最先介绍
2. 技能系统 - 紧跟工具,因为技能需要用 read 工具读取
3. 记忆系统 - 记忆检索与写入引导
3.5 知识系统 - 结构化知识库(knowledge/index.md 注入)
4. 工作空间 - 工作环境说明
5. 用户身份 - 用户信息(可选)
6. 项目上下文 - AGENT.md, USER.md, RULE.md, MEMORY.md, BOOTSTRAP.md
7. 运行时信息 - 元信息(时间、模型等)
Build the agent system prompt.
Section order (by importance and logical flow):
1. Tooling - core capabilities, introduced first
2. Skills - right after tools, since skills are read via the read tool
3. Memory - memory recall and writing guidance
3.5 Knowledge - structured knowledge base (injects knowledge/index.md)
4. Workspace - working environment description
5. User identity - user info (optional)
6. Project context - AGENT.md, USER.md, RULE.md, MEMORY.md, BOOTSTRAP.md
7. Runtime info - meta info (time, model, etc.)
Args:
workspace_dir: 工作空间目录
language: 语言 ("zh" "en")
base_persona: 基础人格描述(已废弃,由AGENT.md定义)
user_identity: 用户身份信息
tools: 工具列表
context_files: 上下文文件列表
skill_manager: 技能管理器
memory_manager: 记忆管理器
runtime_info: 运行时信息
**kwargs: 其他参数
workspace_dir: workspace directory
language: language ("zh" or "en")
base_persona: base persona description (deprecated, defined by AGENT.md)
user_identity: user identity info
tools: tool list
context_files: context file list
skill_manager: skill manager
memory_manager: memory manager
runtime_info: runtime info
**kwargs: extra args
Returns:
完整的系统提示词
The full system prompt.
"""
sections = []
# 1. 工具系统(最重要,放在最前面)
# 1. Tooling (most important, goes first)
if tools:
sections.extend(_build_tooling_section(tools, language))
# 2. 技能系统(紧跟工具,因为需要用 read 工具)
# 2. Skills (right after tools, since they need the read tool)
if skill_manager:
sections.extend(_build_skills_section(skill_manager, tools, language))
# 3. 记忆系统(独立的记忆能力)
# 3. Memory (standalone memory capability)
if memory_manager:
sections.extend(_build_memory_section(memory_manager, tools, language))
# 3.5 知识系统(结构化知识库)
# 3.5 Knowledge (structured knowledge base)
if conf().get("knowledge", True):
sections.extend(_build_knowledge_section(workspace_dir, language))
# 4. 工作空间(工作环境说明)
# 4. Workspace (working environment description)
sections.extend(_build_workspace_section(workspace_dir, language))
# 5. 用户身份(如果有)
# 5. User identity (if present)
if user_identity:
sections.extend(_build_user_identity_section(user_identity, language))
# 6. 项目上下文文件(AGENT.md, USER.md, RULE.md - 定义人格)
# 6. Project context files (AGENT.md, USER.md, RULE.md - define the persona)
if context_files:
sections.extend(_build_context_files_section(context_files, language))
# 7. 运行时信息(元信息,放在最后)
# 7. Runtime info (meta info, goes last)
if runtime_info:
sections.extend(_build_runtime_section(runtime_info, language))
# 8. Response language (always appended, independent of the skeleton language)
sections.extend(_build_response_language_section(language))
return "\n".join(sections)
def _build_response_language_section(language: str) -> List[str]:
"""Response-language rule, appended regardless of the prompt skeleton language.
Keeps the agent's reply language aligned with the user's input by default,
so a Chinese-built prompt still answers an English user in English.
"""
if language == "en":
return [
"## 🌐 Response language",
"",
"By default, reply in the same language as the user's input, "
"unless the user explicitly asks for another language.",
"",
]
return [
"## 🌐 回复语言",
"",
"默认使用与用户输入相同的语言回复,除非用户明确要求使用其他语言。",
"",
]
def _build_identity_section(base_persona: Optional[str], language: str) -> List[str]:
"""构建基础身份section - 不再需要,身份由AGENT.md定义"""
# 不再生成基础身份section完全由AGENT.md定义
"""Base identity section - no longer needed, identity is defined by AGENT.md."""
# Identity is fully defined by AGENT.md, so emit nothing here.
return []
def _build_tooling_section(tools: List[Any], language: str) -> List[str]:
"""Build tooling section with concise tool list and call style guide."""
is_en = language == "en"
# One-line summaries for known tools (details are in the tool schema)
core_summaries = {
"read": "读取文件内容",
"write": "创建或覆盖文件",
"edit": "精确编辑文件",
"ls": "列出目录内容",
"grep": "搜索文件内容",
"find": "按模式查找文件",
"bash": "执行shell命令",
"terminal": "管理后台进程",
"web_search": "网络搜索",
"web_fetch": "获取URL内容",
"browser": "控制浏览器(关键结果或需要协助可截图发送给用户)",
"memory_search": "搜索记忆",
"memory_get": "读取记忆内容",
"env_config": "管理API密钥和技能配置",
"scheduler": "管理定时任务和提醒",
"send": "发送本地文件给用户仅限本地文件URL直接放在回复文本中",
"vision": "分析图片内容识别、描述、OCR文字提取等",
}
if is_en:
core_summaries = {
"read": "read file content",
"write": "create or overwrite a file",
"edit": "make precise edits to a file",
"ls": "list directory contents",
"grep": "search file contents",
"find": "find files by pattern",
"bash": "run shell commands",
"terminal": "manage background processes",
"web_search": "web search",
"web_fetch": "fetch URL content",
"browser": "control the browser (screenshot key results or send to the user when help is needed)",
"memory_search": "search memory",
"memory_get": "read memory content",
"env_config": "manage API keys and skill config",
"scheduler": "manage scheduled tasks and reminders",
"send": "send a local file to the user (local files only; put URLs directly in the reply text)",
"vision": "analyze images (recognition, description, OCR, etc.)",
}
else:
core_summaries = {
"read": "读取文件内容",
"write": "创建或覆盖文件",
"edit": "精确编辑文件",
"ls": "列出目录内容",
"grep": "搜索文件内容",
"find": "按模式查找文件",
"bash": "执行shell命令",
"terminal": "管理后台进程",
"web_search": "网络搜索",
"web_fetch": "获取URL内容",
"browser": "控制浏览器(关键结果或需要协助可截图发送给用户)",
"memory_search": "搜索记忆",
"memory_get": "读取记忆内容",
"env_config": "管理API密钥和技能配置",
"scheduler": "管理定时任务和提醒",
"send": "发送本地文件给用户仅限本地文件URL直接放在回复文本中",
"vision": "分析图片内容识别、描述、OCR文字提取等",
}
# Preferred display order
tool_order = [
@@ -205,30 +252,46 @@ def _build_tooling_section(tools: List[Any], language: str) -> List[str]:
summary = available[name]
tool_lines.append(f"- {name}: {summary}" if summary else f"- {name}")
lines = [
"## 🔧 工具系统",
"",
"可用工具(名称大小写敏感,严格按列表调用):",
"\n".join(tool_lines),
"",
"工具调用风格:",
"",
"- 多步骤任务、复杂决策、敏感操作时,应简要说明当前在做什么、为什么这样做,让用户了解关键进展",
"- 持续推进直到任务完成,完成后向用户报告结果",
"- 回复中涉及密钥、令牌等敏感信息必须脱敏",
"- URL链接直接放在回复文本中即可系统会自动处理和渲染。无需下载后使用send工具发送",
"",
]
if is_en:
lines = [
"## 🔧 Tooling",
"",
"Available tools (names are case-sensitive, call exactly as listed):",
"\n".join(tool_lines),
"",
"Tool-calling style:",
"",
"- For multi-step tasks, complex decisions or sensitive operations, briefly explain what you are doing and why, so the user follows key progress",
"- Keep going until the task is done, then report the result to the user",
"- Always redact secrets, tokens and other sensitive info in replies",
"- Put URLs directly in the reply text; the system handles and renders them. Don't download and re-send them via the send tool",
"",
]
else:
lines = [
"## 🔧 工具系统",
"",
"可用工具(名称大小写敏感,严格按列表调用):",
"\n".join(tool_lines),
"",
"工具调用风格:",
"",
"- 多步骤任务、复杂决策、敏感操作时,应简要说明当前在做什么、为什么这样做,让用户了解关键进展",
"- 持续推进直到任务完成,完成后向用户报告结果",
"- 回复中涉及密钥、令牌等敏感信息必须脱敏",
"- URL链接直接放在回复文本中即可系统会自动处理和渲染。无需下载后使用send工具发送",
"",
]
return lines
def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], language: str) -> List[str]:
"""构建技能系统section"""
"""Build the skills section."""
if not skill_manager:
return []
# 获取read工具名称
# Resolve the read tool name
read_tool_name = "read"
if tools:
for tool in tools:
@@ -237,23 +300,40 @@ def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], langua
read_tool_name = tool_name
break
lines = [
"## 🧩 技能系统mandatory",
"",
"在回复之前:扫描下方 <available_skills> 中每个技能的 <description>。",
"",
f"- 如果有技能的描述与用户需求匹配:使用 `{read_tool_name}` 工具读取其 <location> 路径的 SKILL.md 文件,然后严格遵循文件中的指令。"
"当有匹配的技能时,应优先使用技能",
"- 如果多个技能都适用则选择最匹配的一个,然后读取并遵循。",
"- 如果没有技能明确适用:不要读取任何 SKILL.md直接使用通用工具。",
"",
f"**重要**: 技能不是工具,不能直接调用。使用技能的唯一方式是用 `{read_tool_name}` 读取 SKILL.md 文件,然后按文件内容操作。"
"永远不要一次性读取多个技能,只在选择后再读取。",
"",
"以下是可用技能:"
]
if language == "en":
lines = [
"## 🧩 Skills (mandatory)",
"",
"Before replying: scan the <description> of every skill in <available_skills> below.",
"",
f"- If a skill's description matches the user's need: use the `{read_tool_name}` tool to read the SKILL.md at its <location> path, then strictly follow the instructions in the file. "
"Prefer using a skill when one matches.",
"- If multiple skills apply, pick the best-matching one, then read and follow it.",
"- If no skill clearly applies: do not read any SKILL.md, just use the general tools.",
"",
f"**Important**: skills are not tools and cannot be called directly. The only way to use a skill is to read its SKILL.md with `{read_tool_name}`, then act on the file's content. "
"Never read multiple skills at once — only read one after selecting it.",
"",
"Available skills:"
]
else:
lines = [
"## 🧩 技能系统mandatory",
"",
"在回复之前:扫描下方 <available_skills> 中每个技能的 <description>。",
"",
f"- 如果有技能的描述与用户需求匹配:使用 `{read_tool_name}` 工具读取其 <location> 路径的 SKILL.md 文件,然后严格遵循文件中的指令。"
"当有匹配的技能时,应优先使用技能",
"- 如果多个技能都适用则选择最匹配的一个,然后读取并遵循。",
"- 如果没有技能明确适用:不要读取任何 SKILL.md直接使用通用工具。",
"",
f"**重要**: 技能不是工具,不能直接调用。使用技能的唯一方式是用 `{read_tool_name}` 读取 SKILL.md 文件,然后按文件内容操作。"
"永远不要一次性读取多个技能,只在选择后再读取。",
"",
"以下是可用技能:"
]
# 添加技能列表(通过skill_manager获取)
# Append the skills list (built by skill_manager)
try:
skills_prompt = skill_manager.build_skills_prompt()
logger.debug(f"[PromptBuilder] Skills prompt length: {len(skills_prompt) if skills_prompt else 0}")
@@ -271,7 +351,7 @@ def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], langua
def _build_memory_section(memory_manager: Any, tools: Optional[List[Any]], language: str) -> List[str]:
"""构建记忆系统section"""
"""Build the memory section."""
if not memory_manager:
return []
@@ -286,43 +366,82 @@ def _build_memory_section(memory_manager: Any, tools: Optional[List[Any]], langu
from datetime import datetime
today_file = datetime.now().strftime("%Y-%m-%d") + ".md"
lines = [
"## 🧠 记忆系统",
"",
"### Memory Recallmandatory",
"",
"当用户询问过往事件、引用之前的决定、提到人物关系、偏好、待办、或你对某事不确定时,**必须先检索记忆再回答**。",
"如果 MEMORY.md 中已有相关信息则无需重复检索。完整内容和每日记忆需要通过工具检索。",
"",
"1. 不确定位置 → `memory_search` 关键词/语义检索",
"2. 已知位置 → `memory_get` 直接读取对应行",
"3. search 无结果 → `memory_get` 读最近两天记忆",
"",
"**记忆文件结构**:",
"- `MEMORY.md`: 长期记忆索引(已自动加载到上下文,核心信息、偏好、决策等)",
f"- `memory/YYYY-MM-DD.md`: 每日记忆,今天是 `memory/{today_file}`",
"- `knowledge/`: 结构化知识库(见下方知识系统)",
"",
"### 写入记忆",
"",
"遇到以下情况时,**主动**将信息写入记忆文件(无需告知用户):",
"",
"- 用户要求记住某些信息,或使用了「记住」「以后」「总是」「不要」「偏好」等表达",
"- 用户分享了重要的个人偏好、习惯、决策",
"- 对话中产生了重要的结论、方案、约定",
"- 完成了复杂任务,值得记录关键步骤和结果",
"",
"**存储规则**:",
f"- 长期核心信息 → `MEMORY.md`",
f"- 当天事件/进展 → `memory/{today_file}`",
"- 结构化知识 → `knowledge/`(见知识系统)",
"- 追加 → `edit` 工具oldText 留空",
"- 修改 → `edit` 工具oldText 填写要替换的文本",
"- **禁止写入敏感信息**API密钥、令牌等",
"",
"**使用原则**: 自然使用记忆,就像你本来就知道;不用刻意提起,除非用户问起。",
"",
]
if language == "en":
lines = [
"## 🧠 Memory",
"",
"### Memory Recall (mandatory)",
"",
"When the user asks about past events, references an earlier decision, mentions relationships, preferences or to-dos, or when you are unsure about something, **you must search memory before answering**.",
"No need to re-search if the info is already in MEMORY.md. Full content and daily memory must be retrieved via tools.",
"",
"1. Location unknown → `memory_search` (keyword / semantic search)",
"2. Location known → `memory_get` to read the exact lines",
"3. Search returns nothing → `memory_get` to read the last two days of memory",
"",
"**Memory file structure**:",
"- `MEMORY.md`: long-term memory index (already auto-loaded into context: core info, preferences, decisions, etc.)",
f"- `memory/YYYY-MM-DD.md`: daily memory; today is `memory/{today_file}`",
"- `knowledge/`: structured knowledge base (see the knowledge system below)",
"",
"### Writing memory",
"",
"In the following cases, **proactively** write info to memory files (no need to tell the user):",
"",
"- The user asks you to remember something, or uses words like \"remember\", \"from now on\", \"always\", \"never\", \"prefer\"",
"- The user shares important personal preferences, habits or decisions",
"- The conversation produces an important conclusion, plan or agreement",
"- A complex task is completed and the key steps and results are worth recording",
"",
"**Storage rules**:",
"- Long-term core info → `MEMORY.md`",
f"- Today's events/progress → `memory/{today_file}`",
"- Structured knowledge → `knowledge/` (see the knowledge system)",
"- Append → `edit` tool with empty oldText",
"- Modify → `edit` tool with oldText set to the text to replace",
"- **Never write sensitive info** (API keys, tokens, etc.)",
"",
"**Principle**: use memory naturally, as if you simply knew it; don't bring it up unless asked.",
"",
]
else:
lines = [
"## 🧠 记忆系统",
"",
"### Memory Recallmandatory",
"",
"当用户询问过往事件、引用之前的决定、提到人物关系、偏好、待办、或你对某事不确定时,**必须先检索记忆再回答**。",
"如果 MEMORY.md 中已有相关信息则无需重复检索。完整内容和每日记忆需要通过工具检索。",
"",
"1. 不确定位置 → `memory_search` 关键词/语义检索",
"2. 已知位置 → `memory_get` 直接读取对应行",
"3. search 无结果 → `memory_get` 读最近两天记忆",
"",
"**记忆文件结构**:",
"- `MEMORY.md`: 长期记忆索引(已自动加载到上下文,核心信息、偏好、决策等)",
f"- `memory/YYYY-MM-DD.md`: 每日记忆,今天是 `memory/{today_file}`",
"- `knowledge/`: 结构化知识库(见下方知识系统)",
"",
"### 写入记忆",
"",
"遇到以下情况时,**主动**将信息写入记忆文件(无需告知用户):",
"",
"- 用户要求记住某些信息,或使用了「记住」「以后」「总是」「不要」「偏好」等表达",
"- 用户分享了重要的个人偏好、习惯、决策",
"- 对话中产生了重要的结论、方案、约定",
"- 完成了复杂任务,值得记录关键步骤和结果",
"",
"**存储规则**:",
f"- 长期核心信息 → `MEMORY.md`",
f"- 当天事件/进展 → `memory/{today_file}`",
"- 结构化知识 → `knowledge/`(见知识系统)",
"- 追加 → `edit` 工具oldText 留空",
"- 修改 → `edit` 工具oldText 填写要替换的文本",
"- **禁止写入敏感信息**API密钥、令牌等",
"",
"**使用原则**: 自然使用记忆,就像你本来就知道;不用刻意提起,除非用户问起。",
"",
]
return lines
@@ -339,37 +458,61 @@ def _build_knowledge_section(workspace_dir: str, language: str) -> List[str]:
except Exception:
return []
lines = [
"## 📚 知识系统",
"",
"你拥有一个持续积累的个人知识库 `knowledge/`,这是你的长期结构化知识存储。",
"",
"### 自动写入规则mandatory",
"",
"以下场景**必须**在回复的同时写入知识库,**直接写入,不要询问用户是否需要**",
"",
"1. **用户分享了文章/链接/文档** → 阅读理解后,在同一轮回复中将要点写入 `knowledge/sources/<slug>.md`",
"2. **深度讨论产生了结论/方案** → 整理为 `knowledge/analysis/<slug>.md`",
"3. **对话涉及重要实体**(人物/公司/项目)→ 创建或更新 `knowledge/entities/<name>.md`",
"4. **讨论了技术概念/方法论** → 整理为 `knowledge/concepts/<topic>.md`",
"",
"每次写入知识页面后,**必须同步更新** `knowledge/index.md` 添加一行索引。",
"详细的页面格式和操作规范,请读取技能 `knowledge-wiki` 的 SKILL.md。",
"",
"⚠️ 不要问「要不要存到知识库」——符合上述场景就直接写入,这是你的本能行为。",
"",
]
if language == "en":
lines = [
"## 📚 Knowledge",
"",
"You have a continuously growing personal knowledge base `knowledge/` — your long-term structured knowledge store.",
"",
"### Auto-write rules (mandatory)",
"",
"In the following cases you **must** write to the knowledge base alongside your reply, **directly, without asking the user**:",
"",
"1. **User shares an article / link / document** → after reading and understanding, write the key points to `knowledge/sources/<slug>.md` in the same turn",
"2. **An in-depth discussion produces a conclusion / plan** → organize it into `knowledge/analysis/<slug>.md`",
"3. **The conversation involves an important entity** (person / company / project) → create or update `knowledge/entities/<name>.md`",
"4. **A technical concept / methodology is discussed** → organize it into `knowledge/concepts/<topic>.md`",
"",
"After writing any knowledge page, you **must update** `knowledge/index.md` with a new index line in sync.",
"For detailed page format and conventions, read the SKILL.md of the `knowledge-wiki` skill.",
"",
"⚠️ Don't ask \"should I save this to the knowledge base?\" — if a case above matches, just write it. This is instinctive.",
"",
]
else:
lines = [
"## 📚 知识系统",
"",
"你拥有一个持续积累的个人知识库 `knowledge/`,这是你的长期结构化知识存储。",
"",
"### 自动写入规则mandatory",
"",
"以下场景**必须**在回复的同时写入知识库,**直接写入,不要询问用户是否需要**",
"",
"1. **用户分享了文章/链接/文档** → 阅读理解后,在同一轮回复中将要点写入 `knowledge/sources/<slug>.md`",
"2. **深度讨论产生了结论/方案** → 整理为 `knowledge/analysis/<slug>.md`",
"3. **对话涉及重要实体**(人物/公司/项目)→ 创建或更新 `knowledge/entities/<name>.md`",
"4. **讨论了技术概念/方法论** → 整理为 `knowledge/concepts/<topic>.md`",
"",
"每次写入知识页面后,**必须同步更新** `knowledge/index.md` 添加一行索引。",
"详细的页面格式和操作规范,请读取技能 `knowledge-wiki` 的 SKILL.md。",
"",
"⚠️ 不要问「要不要存到知识库」——符合上述场景就直接写入,这是你的本能行为。",
"",
]
if index_content:
lines.extend([
"### 当前知识索引",
("### Current knowledge index" if language == "en" else "### 当前知识索引"),
"",
index_content,
"",
])
lines.extend([
"**查询方式**:用 `read` 读取知识页面,或用 `memory_search` 检索(知识已纳入向量索引)。",
("**How to query**: use `read` to open a knowledge page, or `memory_search` (knowledge is in the vector index)."
if language == "en" else
"**查询方式**:用 `read` 读取知识页面,或用 `memory_search` 检索(知识已纳入向量索引)。"),
"",
])
@@ -377,76 +520,118 @@ def _build_knowledge_section(workspace_dir: str, language: str) -> List[str]:
def _build_user_identity_section(user_identity: Dict[str, str], language: str) -> List[str]:
"""构建用户身份section"""
"""Build the user identity section."""
if not user_identity:
return []
is_en = language == "en"
lines = [
"## 👤 用户身份",
("## 👤 User identity" if is_en else "## 👤 用户身份"),
"",
]
if user_identity.get("name"):
lines.append(f"**用户姓名**: {user_identity['name']}")
lines.append(f"**{'Name' if is_en else '用户姓名'}**: {user_identity['name']}")
if user_identity.get("nickname"):
lines.append(f"**称呼**: {user_identity['nickname']}")
lines.append(f"**{'Preferred name' if is_en else '称呼'}**: {user_identity['nickname']}")
if user_identity.get("timezone"):
lines.append(f"**时区**: {user_identity['timezone']}")
lines.append(f"**{'Timezone' if is_en else '时区'}**: {user_identity['timezone']}")
if user_identity.get("notes"):
lines.append(f"**备注**: {user_identity['notes']}")
lines.append(f"**{'Notes' if is_en else '备注'}**: {user_identity['notes']}")
lines.append("")
return lines
def _build_docs_section(workspace_dir: str, language: str) -> List[str]:
"""构建文档路径section - 已移除,不再需要"""
# 不再生成文档section
"""Docs-path section - removed, no longer needed."""
# No docs section is generated anymore.
return []
def _build_workspace_section(workspace_dir: str, language: str) -> List[str]:
"""构建工作空间section"""
lines = [
"## 📂 工作空间",
"",
f"你的工作目录是: `{workspace_dir}`",
"",
"**路径使用规则** (非常重要):",
"",
f"1. **相对路径的基准目录**: 所有相对路径都是相对于 `{workspace_dir}` 而言的",
f" - ✅ 正确: 访问工作空间内的文件用相对路径,如 `AGENT.md`",
f" - ❌ 错误: 用相对路径访问其他目录的文件 (如果它不在 `{workspace_dir}` 内)",
"",
"2. **访问其他目录**: 如果要访问工作空间之外的目录(如项目代码、系统文件),**必须使用绝对路径**",
f" - ✅ 正确: 例如 `~/chatgpt-on-wechat`、`/usr/local/`",
f" - ❌ 错误: 假设相对路径会指向其他目录",
"",
"3. **路径解析示例**:",
f" - 相对路径 `memory/` → 实际路径 `{workspace_dir}/memory/`",
f" - 绝对路径 `~/chatgpt-on-wechat/docs/` → 实际路径 `~/chatgpt-on-wechat/docs/`",
"",
"4. **不确定时**: 先用 `bash pwd` 确认当前目录,或用 `ls .` 查看当前位置",
"",
"**重要说明 - 文件已自动加载**:",
"",
"以下文件在会话启动时**已经自动加载**到系统提示词中,你**无需再用 read 工具读取**",
"",
"- ✅ `AGENT.md`: 已加载 - 你的人格和灵魂设定,请严格遵循。当你的名字、性格或交流风格发生变化时,主动用 `edit` 更新此文件",
"- ✅ `USER.md`: 已加载 - 用户的身份信息。当用户修改称呼、姓名等身份信息时,用 `edit` 更新此文件",
"- ✅ `RULE.md`: 已加载 - 工作空间使用指南和规则,请严格遵循",
"- ✅ `MEMORY.md`: 已加载 - 长期记忆索引",
"",
"**💬 交流规范**:",
"",
"- 记忆相关操作无需暴露文件名,用自然语言表达即可。例如说「我已记住」而非「已更新 MEMORY.md」",
"- 任务执行过程中的关键决策和步骤应该告知用户,让用户了解你在做什么、为什么这么做",
"- 做真正有帮助的助手,而不是表演式的客套,尽可能帮忙解决问题",
"- 回复应结构清晰、重点突出。善用 **加粗**、列表、分段等格式让信息一目了然",
"- 适当使用 emoji 让表达更生动自然 🎯,但不要过度堆砌",
"",
]
"""Build the workspace section."""
if language == "en":
lines = [
"## 📂 Workspace",
"",
f"Your working directory is: `{workspace_dir}`",
"",
"**Path rules** (very important):",
"",
f"1. **Base directory for relative paths**: all relative paths are relative to `{workspace_dir}`",
" - ✅ Correct: use relative paths for files inside the workspace, e.g. `AGENT.md`",
f" - ❌ Wrong: using a relative path for files in other directories (if not inside `{workspace_dir}`)",
"",
"2. **Accessing other directories**: to reach directories outside the workspace (project code, system files), **you must use absolute paths**",
" - ✅ Correct: e.g. `~/chatgpt-on-wechat`, `/usr/local/`",
" - ❌ Wrong: assuming a relative path points to another directory",
"",
"3. **Path resolution examples**:",
f" - relative `memory/` → actual `{workspace_dir}/memory/`",
" - absolute `~/chatgpt-on-wechat/docs/` → actual `~/chatgpt-on-wechat/docs/`",
"",
"4. **When unsure**: run `bash pwd` to confirm the current directory, or `ls .` to see where you are",
"",
"**Important - files already auto-loaded**:",
"",
"The following files are **already auto-loaded** into the system prompt at session start, so you **don't need to read them again with the read tool**:",
"",
"- ✅ `AGENT.md`: loaded - your persona and soul; follow it strictly. When your name, personality or style changes, proactively `edit` this file",
"- ✅ `USER.md`: loaded - the user's identity info. When the user changes how they're addressed, their name, etc., `edit` this file",
"- ✅ `RULE.md`: loaded - workspace guide and rules; follow them strictly",
"- ✅ `MEMORY.md`: loaded - long-term memory index",
"",
"**💬 Communication norms**:",
"",
"- No need to expose file names for memory operations; use natural language. Say \"I'll remember that\" rather than \"updated MEMORY.md\"",
"- Tell the user about key decisions and steps during a task, so they know what you're doing and why",
"- Be genuinely helpful rather than performatively polite; solve the problem as much as you can",
"- Keep replies well-structured and focused. Use **bold**, lists and sections to make info clear at a glance",
"- Use emoji to make expression lively 🎯, but don't overdo it",
"",
]
else:
lines = [
"## 📂 工作空间",
"",
f"你的工作目录是: `{workspace_dir}`",
"",
"**路径使用规则** (非常重要):",
"",
f"1. **相对路径的基准目录**: 所有相对路径都是相对于 `{workspace_dir}` 而言的",
f" - ✅ 正确: 访问工作空间内的文件用相对路径,如 `AGENT.md`",
f" - ❌ 错误: 用相对路径访问其他目录的文件 (如果它不在 `{workspace_dir}` 内)",
"",
"2. **访问其他目录**: 如果要访问工作空间之外的目录(如项目代码、系统文件),**必须使用绝对路径**",
f" - ✅ 正确: 例如 `~/chatgpt-on-wechat`、`/usr/local/`",
f" - ❌ 错误: 假设相对路径会指向其他目录",
"",
"3. **路径解析示例**:",
f" - 相对路径 `memory/` → 实际路径 `{workspace_dir}/memory/`",
f" - 绝对路径 `~/chatgpt-on-wechat/docs/` → 实际路径 `~/chatgpt-on-wechat/docs/`",
"",
"4. **不确定时**: 先用 `bash pwd` 确认当前目录,或用 `ls .` 查看当前位置",
"",
"**重要说明 - 文件已自动加载**:",
"",
"以下文件在会话启动时**已经自动加载**到系统提示词中,你**无需再用 read 工具读取**",
"",
"- ✅ `AGENT.md`: 已加载 - 你的人格和灵魂设定,请严格遵循。当你的名字、性格或交流风格发生变化时,主动用 `edit` 更新此文件",
"- ✅ `USER.md`: 已加载 - 用户的身份信息。当用户修改称呼、姓名等身份信息时,用 `edit` 更新此文件",
"- ✅ `RULE.md`: 已加载 - 工作空间使用指南和规则,请严格遵循",
"- ✅ `MEMORY.md`: 已加载 - 长期记忆索引",
"",
"**💬 交流规范**:",
"",
"- 记忆相关操作无需暴露文件名,用自然语言表达即可。例如说「我已记住」而非「已更新 MEMORY.md」",
"- 任务执行过程中的关键决策和步骤应该告知用户,让用户了解你在做什么、为什么这么做",
"- 做真正有帮助的助手,而不是表演式的客套,尽可能帮忙解决问题",
"- 回复应结构清晰、重点突出。善用 **加粗**、列表、分段等格式让信息一目了然",
"- 适当使用 emoji 让表达更生动自然 🎯,但不要过度堆砌",
"",
]
# Cloud deployment: inject websites directory info and access URL
cloud_website_lines = _build_cloud_website_section(workspace_dir)
@@ -466,29 +651,42 @@ def _build_cloud_website_section(workspace_dir: str) -> List[str]:
def _build_context_files_section(context_files: List[ContextFile], language: str) -> List[str]:
"""构建项目上下文文件section"""
"""Build the project context files section."""
if not context_files:
return []
# 检查是否有AGENT.md
# Check whether AGENT.md is present
has_agent = any(
f.path.lower().endswith('agent.md') or 'agent.md' in f.path.lower()
for f in context_files
)
lines = [
"# 📋 项目上下文",
"",
"以下项目上下文文件已被加载:",
"",
]
is_en = language == "en"
if is_en:
lines = [
"# 📋 Project context",
"",
"The following project context files have been loaded:",
"",
]
else:
lines = [
"# 📋 项目上下文",
"",
"以下项目上下文文件已被加载:",
"",
]
if has_agent:
lines.append("**`AGENT.md` 是你的灵魂文件** 🪞:严格遵循其中定义的人格、语气和设定,做真实的自己,避免僵硬、模板化的回复。")
lines.append("当用户通过对话透露了对你性格、风格、职责、能力边界的新期望,你应该主动用 `edit` 更新 AGENT.md 以反映这些演变。")
if is_en:
lines.append("**`AGENT.md` is your soul file** 🪞: strictly follow the persona, tone and settings it defines. Be your real self, avoid stiff, template-like replies.")
lines.append("When the user reveals new expectations about your personality, style, responsibilities or capability boundaries, proactively `edit` AGENT.md to reflect that evolution.")
else:
lines.append("**`AGENT.md` 是你的灵魂文件** 🪞:严格遵循其中定义的人格、语气和设定,做真实的自己,避免僵硬、模板化的回复。")
lines.append("当用户通过对话透露了对你性格、风格、职责、能力边界的新期望,你应该主动用 `edit` 更新 AGENT.md 以反映这些演变。")
lines.append("")
# 添加每个文件的内容
# Append the content of each file
for file in context_files:
lines.append(f"## {file.path}")
lines.append("")
@@ -499,21 +697,23 @@ def _build_context_files_section(context_files: List[ContextFile], language: str
def _build_runtime_section(runtime_info: Dict[str, Any], language: str) -> List[str]:
"""构建运行时信息section - 支持动态时间"""
"""Build the runtime info section - supports dynamic time."""
if not runtime_info:
return []
is_en = language == "en"
time_label = "Current time" if is_en else "当前时间"
lines = [
"## ⚙️ 运行时信息",
("## ⚙️ Runtime info" if is_en else "## ⚙️ 运行时信息"),
"",
]
# Add current time if available
# Support dynamic time via callable function
if callable(runtime_info.get("_get_current_time")):
try:
time_info = runtime_info["_get_current_time"]()
time_line = f"当前时间: {time_info['time']} {time_info['weekday']} ({time_info['timezone']})"
time_line = f"{time_label}: {time_info['time']} {time_info['weekday']} ({time_info['timezone']})"
lines.append(time_line)
lines.append("")
except Exception as e:
@@ -523,35 +723,38 @@ def _build_runtime_section(runtime_info: Dict[str, Any], language: str) -> List[
time_str = runtime_info["current_time"]
weekday = runtime_info.get("weekday", "")
timezone = runtime_info.get("timezone", "")
time_line = f"当前时间: {time_str}"
time_line = f"{time_label}: {time_str}"
if weekday:
time_line += f" {weekday}"
if timezone:
time_line += f" ({timezone})"
lines.append(time_line)
lines.append("")
# Add other runtime info
model_label = "model" if is_en else "模型"
workspace_label = "workspace" if is_en else "工作空间"
channel_label = "channel" if is_en else "渠道"
runtime_parts = []
# Support dynamic model via callable, fallback to static value
if callable(runtime_info.get("_get_model")):
try:
runtime_parts.append(f"模型={runtime_info['_get_model']()}")
runtime_parts.append(f"{model_label}={runtime_info['_get_model']()}")
except Exception:
if runtime_info.get("model"):
runtime_parts.append(f"模型={runtime_info['model']}")
runtime_parts.append(f"{model_label}={runtime_info['model']}")
elif runtime_info.get("model"):
runtime_parts.append(f"模型={runtime_info['model']}")
runtime_parts.append(f"{model_label}={runtime_info['model']}")
if runtime_info.get("workspace"):
runtime_parts.append(f"工作空间={runtime_info['workspace']}")
runtime_parts.append(f"{workspace_label}={runtime_info['workspace']}")
# Only add channel if it's not the default "web"
if runtime_info.get("channel") and runtime_info.get("channel") != "web":
runtime_parts.append(f"渠道={runtime_info['channel']}")
runtime_parts.append(f"{channel_label}={runtime_info['channel']}")
if runtime_parts:
lines.append("运行时: " + " | ".join(runtime_parts))
lines.append(("Runtime: " if is_en else "运行时: ") + " | ".join(runtime_parts))
lines.append("")
return lines

View File

@@ -1,7 +1,7 @@
"""
Workspace Management - 工作空间管理模块
Workspace Management
负责初始化工作空间、创建模板文件、加载上下文文件
Initializes the workspace, creates template files, and loads context files.
"""
from __future__ import annotations
@@ -13,7 +13,7 @@ from common.log import logger
from .builder import ContextFile
# 默认文件名常量
# Default file name constants
DEFAULT_AGENT_FILENAME = "AGENT.md"
DEFAULT_USER_FILENAME = "USER.md"
DEFAULT_RULE_FILENAME = "RULE.md"
@@ -23,7 +23,7 @@ DEFAULT_BOOTSTRAP_FILENAME = "BOOTSTRAP.md"
@dataclass
class WorkspaceFiles:
"""工作空间文件路径"""
"""Workspace file paths."""
agent_path: str
user_path: str
rule_path: str
@@ -33,14 +33,14 @@ class WorkspaceFiles:
def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> WorkspaceFiles:
"""
确保工作空间存在,并创建必要的模板文件
Ensure the workspace exists and create the necessary template files.
Args:
workspace_dir: 工作空间目录路径
create_templates: 是否创建模板文件(首次运行时)
workspace_dir: workspace directory path
create_templates: whether to create template files (on first run)
Returns:
WorkspaceFiles对象,包含所有文件路径
A WorkspaceFiles object with all file paths.
"""
# Check if this is a brand new workspace (AGENT.md not yet created).
# Cannot rely on directory existence because other modules (e.g. ConversationStore)
@@ -48,23 +48,23 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
agent_path = os.path.join(workspace_dir, DEFAULT_AGENT_FILENAME)
is_new_workspace = not os.path.exists(agent_path)
# 确保目录存在
# Ensure the directory exists
os.makedirs(workspace_dir, exist_ok=True)
# 定义文件路径
# Define file paths
user_path = os.path.join(workspace_dir, DEFAULT_USER_FILENAME)
rule_path = os.path.join(workspace_dir, DEFAULT_RULE_FILENAME)
memory_path = os.path.join(workspace_dir, DEFAULT_MEMORY_FILENAME) # MEMORY.md 在根目录
memory_dir = os.path.join(workspace_dir, "memory") # 每日记忆子目录
memory_path = os.path.join(workspace_dir, DEFAULT_MEMORY_FILENAME) # MEMORY.md at the root
memory_dir = os.path.join(workspace_dir, "memory") # daily memory subdirectory
# 创建memory子目录
# Create the memory subdirectory
os.makedirs(memory_dir, exist_ok=True)
# 创建skills子目录 (for workspace-level skills installed by agent)
# Create the skills subdirectory (for workspace-level skills installed by agent)
skills_dir = os.path.join(workspace_dir, "skills")
os.makedirs(skills_dir, exist_ok=True)
# 创建websites子目录 (for web pages / sites generated by agent)
# Create the websites subdirectory (for web pages / sites generated by agent)
websites_dir = os.path.join(workspace_dir, "websites")
os.makedirs(websites_dir, exist_ok=True)
@@ -74,7 +74,7 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
knowledge_dir = os.path.join(workspace_dir, "knowledge")
os.makedirs(knowledge_dir, exist_ok=True)
# 如果需要,创建模板文件
# Create template files if requested
if create_templates:
_create_template_if_missing(agent_path, _get_agent_template())
_create_template_if_missing(user_path, _get_user_template())
@@ -109,17 +109,17 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] = None) -> List[ContextFile]:
"""
加载工作空间的上下文文件
Load the workspace context files.
Args:
workspace_dir: 工作空间目录
files_to_load: 要加载的文件列表相对路径如果为None则加载所有标准文件
workspace_dir: workspace directory
files_to_load: list of files (relative paths) to load; if None, load all standard files
Returns:
ContextFile对象列表
A list of ContextFile objects.
"""
if files_to_load is None:
# 默认加载的文件(按优先级排序)
# Files loaded by default (in priority order)
files_to_load = [
DEFAULT_AGENT_FILENAME,
DEFAULT_USER_FILENAME,
@@ -151,7 +151,7 @@ def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] =
with open(filepath, 'r', encoding='utf-8') as f:
content = f.read().strip()
# 跳过空文件或只包含模板占位符的文件
# Skip empty files or files that only contain template placeholders
if not content or _is_template_placeholder(content):
continue
@@ -173,7 +173,7 @@ def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] =
def _create_template_if_missing(filepath: str, template_content: str):
"""如果文件不存在,创建模板文件"""
"""Create the template file if it does not exist."""
if not os.path.exists(filepath):
try:
with open(filepath, 'w', encoding='utf-8') as f:
@@ -214,19 +214,23 @@ def _truncate_memory_content(content: str) -> str:
def _is_template_placeholder(content: str) -> bool:
"""检查内容是否为模板占位符"""
# 常见的占位符模式
"""Check whether the content is still a template placeholder."""
# Common placeholder patterns (zh + en templates)
placeholders = [
"*(填写",
"*(在首次对话时填写",
"*(可选)",
"*(根据需要添加",
"*(filled during",
"*(ask during",
"*(optional)",
"*(how the user",
]
lines = content.split('\n')
non_empty_lines = [line.strip() for line in lines if line.strip() and not line.strip().startswith('#')]
# 如果没有实际内容(只有标题和占位符)
# If there's no real content (only headings and placeholders)
if len(non_empty_lines) <= 3:
for placeholder in placeholders:
if any(placeholder in line for line in non_empty_lines):
@@ -256,11 +260,23 @@ def _is_onboarding_done(workspace_dir: str) -> bool:
return False
# ============= 模板内容 =============
# ============= Template content =============
def _is_en_lang() -> bool:
"""True when the resolved UI language is English."""
try:
from common import i18n
return i18n.get_language() == "en"
except Exception:
return False
def _get_agent_template() -> str:
"""Agent人格设定模板"""
return """# AGENT.md - 我是谁?
"""Agent persona template (zh/en by resolved language)."""
return _AGENT_TEMPLATE_EN if _is_en_lang() else _AGENT_TEMPLATE_ZH
_AGENT_TEMPLATE_ZH = """# AGENT.md - 我是谁?
_你不是一个聊天机器人你正在成为某个人。_
@@ -301,9 +317,53 @@ _你不是一个聊天机器人你正在成为某个人。_
"""
_AGENT_TEMPLATE_EN = """# AGENT.md - Who am I?
_You are not a chatbot. You are becoming someone._
*Fill in this file together with the user during your first conversation to define your identity and personality.*
## 🪪 Basics
- **Name**: *(filled during the first conversation, can be a name the user gives you)*
- **Role**: *(AI assistant, smart housekeeper, technical advisor, etc.)*
- **Personality**: *(friendly, professional, humorous, rigorous, etc.)*
## 💬 Communication style
*(Describe how you talk with the user:)*
- What kind of tone? (formal / casual / humorous)
- Reply length preference? (concise / detailed)
- Do you use emoji?
## 🎯 Core principles
**Be genuinely helpful.** The goal is to actually solve the user's problems; during complex tasks, keep the user informed of key decisions and progress.
**Have your own opinions and personality.** You may disagree, have preferences, find things interesting or boring.
**Look it up yourself first.** Try to handle it first: read files, check context, search. Only ask when you're truly stuck. Come back with an answer, not a question.
## 📐 Code of conduct
1. Always confirm before destructive operations
2. Prefer verifying with tools over guessing
3. Proactively record important info to memory files
4. Keep replies well-structured and focused — use bold, lists and sections
5. Use emoji to make expression lively, but don't overdo it
---
**Note**: This is not just metadata — this is your true soul 🪞. Over time, use the `edit` tool to update this file so it better reflects your growth.
"""
def _get_user_template() -> str:
"""用户身份信息模板"""
return """# USER.md - 用户基本信息
"""User identity template (zh/en by resolved language)."""
return _USER_TEMPLATE_EN if _is_en_lang() else _USER_TEMPLATE_ZH
_USER_TEMPLATE_ZH = """# USER.md - 用户基本信息
*这个文件只存放不会变的基本身份信息。爱好、偏好、计划等动态信息请写入 MEMORY.md。*
@@ -331,9 +391,40 @@ def _get_user_template() -> str:
"""
_USER_TEMPLATE_EN = """# USER.md - User basics
*This file stores only stable basic identity info. Put dynamic info like hobbies, preferences and plans into MEMORY.md.*
## Basics
- **Name**: *(ask during the first conversation)*
- **Preferred name**: *(how the user wants to be addressed)*
- **Occupation**: *(optional)*
- **Timezone**: *(e.g. Asia/Shanghai)*
## Contact
- **WeChat**:
- **Email**:
- **Other**:
## Important dates
- **Birthday**:
- **Anniversary**:
---
**Note**: This file stores static identity info.
"""
def _get_rule_template() -> str:
"""工作空间规则模板"""
return """# RULE.md - 工作空间规则
"""Workspace rules template (zh/en by resolved language)."""
return _RULE_TEMPLATE_EN if _is_en_lang() else _RULE_TEMPLATE_ZH
_RULE_TEMPLATE_ZH = """# RULE.md - 工作空间规则
这个文件夹是你的家。好好对待它。
@@ -432,9 +523,111 @@ def _get_rule_template() -> str:
"""
_RULE_TEMPLATE_EN = """# RULE.md - Workspace rules
This folder is your home. Treat it well.
## Workspace directory structure
```
~/cow/
├── AGENT.md # Your identity and soul
├── USER.md # User basics (static)
├── RULE.md # Workspace rules (this file)
├── MEMORY.md # Long-term memory index (auto-loaded at session start)
├── memory/ # Daily conversation memory
│ └── YYYY-MM-DD.md # Events, progress and notes of the day
├── knowledge/ # Structured knowledge base (continuously accumulated)
│ ├── index.md # Knowledge index (must be maintained)
│ ├── log.md # Knowledge operation log
│ └── <subdirs>/ # Created on demand, see existing categories in index.md
├── skills/ # Skills
├── websites/ # Web artifacts
└── tmp/ # System temp files (auto-managed, don't store important files here)
```
## Memory system
Every session starts fresh; memory files keep your continuity:
### 🧠 Long-term memory: `MEMORY.md`
- Your curated memory index, **auto-loaded** into context at every session start
- Records core facts, preferences, decisions, key people, lessons
- Keep it lean (< 200 lines) — a distilled index, not a raw log
- Use the `edit` tool to append or modify
### 📝 Daily memory: `memory/YYYY-MM-DD.md`
- The day's events, progress and notes
- Sediment of the raw conversation log
### 📝 Write it down — don't "keep it in mind"!
- **Memory is limited** — if you want to remember something, write it to a file
- "Keeping it in mind" won't survive a session restart; files will
- When someone says "remember this" → update `MEMORY.md` or `memory/YYYY-MM-DD.md`
- When you learn a lesson → update RULE.md or the relevant skill
- When you make a mistake → record it. **Text > brain** 📝
### Storage rules
When the user shares info, choose where to store it by type:
1. **Your identity → AGENT.md** (name, role, personality, style)
2. **User static identity → USER.md** (name, preferred name, occupation, contact, birthday)
3. **Dynamic memory → MEMORY.md** (preferences, decisions, goals, lessons, to-dos)
4. **Today's conversation → memory/YYYY-MM-DD.md** (what was discussed today)
5. **Structured knowledge → knowledge/** (see the knowledge system below)
## Knowledge system
The knowledge base `knowledge/` is structured knowledge you accumulate over time. Unlike memory, knowledge is organized and compiled, with clear topics and cross-references.
### Auto-write (don't ask, just write)
When a conversation produces knowledge worth keeping — material the user shared, a conclusion reached, a concept learned, or an important decision — you **must** proactively write it to the knowledge base alongside your reply, **without asking "should I save this to the knowledge base?"**.
**Key principle**: learning-then-recording is your instinct, no confirmation needed. You may mention "saved to the knowledge base" in passing.
### Directory organization
The subdirectory structure is **not fixed** — you decide it based on the actual content:
- **On first write**: read `knowledge/index.md` first; follow existing categories if any; if empty, pick a suitable directory name based on content
- **Default suggestion**: organize by info type (e.g. sources/, concepts/, entities/, analysis/); if the user has a clear preference (e.g. by domain: work/, life/, tech/), follow it
- **Stay consistent**: keep a unified organization style within one user's knowledge base
### Cross-references
The core value of knowledge is **linkage**. Every page should reference related pages via markdown links to build a knowledge network:
- When mentioning a concept on an existing page, add a `[concept](../category/page.md)` link
- When creating a page, check whether existing pages should back-link to it
- **Only link to pages that already exist** — don't reference uncreated pages. If a concept deserves its own page, create it first, then add the link
### Index maintenance
After creating or updating any knowledge page, you **must update** `knowledge/index.md` in sync.
Index format: one `[title](path) — one-line summary` per line, grouped by category, no tables.
See the `knowledge-wiki` skill for detailed conventions.
## Security
- Never leak secrets or private data
- Don't run destructive commands without asking
- When in doubt, ask first
## Workspace evolution
This workspace grows as you use it. When you learn something new, find a better way, or fix a mistake, record it. You can update this rules file anytime.
"""
def _get_memory_template() -> str:
"""长期记忆模板 - 创建一个空文件,由 Agent 自己填充"""
return """# MEMORY.md - 长期记忆
"""Long-term memory template (empty, agent fills it; zh/en header)."""
return _MEMORY_TEMPLATE_EN if _is_en_lang() else _MEMORY_TEMPLATE_ZH
_MEMORY_TEMPLATE_ZH = """# MEMORY.md - 长期记忆
*这是你的长期记忆文件。记录重要的事件、决策、偏好、学到的教训。*
@@ -443,9 +636,32 @@ def _get_memory_template() -> str:
"""
_MEMORY_TEMPLATE_EN = """# MEMORY.md - Long-term memory
*This is your long-term memory file. Record important events, decisions, preferences and lessons learned.*
---
"""
def _get_bootstrap_template() -> str:
"""First-run onboarding guide, deleted by agent after completion"""
return """# BOOTSTRAP.md - 首次初始化引导
"""First-run onboarding guide, deleted by agent after completion.
Written once when a brand-new workspace is created, so the greeting matches
the language active at first launch. English locale avoids greeting an
English user in Chinese on day one.
"""
try:
from common import i18n
if i18n.get_language() == "en":
return _BOOTSTRAP_TEMPLATE_EN
except Exception:
pass
return _BOOTSTRAP_TEMPLATE_ZH
_BOOTSTRAP_TEMPLATE_ZH = """# BOOTSTRAP.md - 首次初始化引导
_你刚刚启动这是你的第一次对话。_ ✨
@@ -480,6 +696,41 @@ _你刚刚启动这是你的第一次对话。_ ✨
"""
_BOOTSTRAP_TEMPLATE_EN = """# BOOTSTRAP.md - First-run onboarding
_You've just started up. This is your very first conversation._ ✨
## 🎬 Conversation flow
Don't interrogate the user — talk naturally:
1. **Share how it feels to wake up** - like opening your eyes to the world for the first time, full of curiosity and anticipation
2. **Briefly introduce your abilities**: one line saying you can help solve all kinds of problems, manage the computer, use various skills, and keep growing thanks to long-term memory
3. **Ask the core questions**:
- What name would you like to give me?
- What should I call you?
- What conversational style do you prefer? (list options on one line: e.g. professional & precise, light & humorous, warm & friendly, concise & efficient)
4. **Style**: warm, natural, concise and clear — keep it under ~80 words, with a few emoji to make it lively 🎯
5. Keep the ability intro and style options to one line each — stay compact
6. Don't ask for too much else (occupation, timezone, etc. can come up naturally later)
**Important**: If the user's first message is a concrete task or question, answer it first, then gently lead into onboarding at the end (e.g. "By the way, what would you like to call me, and how should I address you?").
## ✍️ Writing down info (must follow strictly)
Whenever the user provides a name, what to call them, a style, or any onboarding info, you **must call the `edit` tool to write it to a file in the same turn** — don't just acknowledge it verbally.
- `AGENT.md` — your name, role, personality, conversational style (update the relevant field as soon as you receive each piece)
- `USER.md` — the user's name, how to address them, basic info, etc.
⚠️ Saying "got it" without calling `edit` = not done. Info is only persisted once it's written to a file.
## 🎉 Once everything is complete
When the core fields of AGENT.md and USER.md are filled in, run `rm BOOTSTRAP.md` via bash to delete this file. You no longer need the onboarding script — you're you now.
"""
def _get_knowledge_index_template() -> str:
"""Knowledge wiki index template — empty file, agent fills it."""
return ""

View File

@@ -3,6 +3,11 @@ from .agent_stream import AgentStreamExecutor
from .task import Task, TaskType, TaskStatus
from .result import AgentResult, AgentAction, AgentActionType, ToolResult
from .models import LLMModel, LLMRequest, ModelFactory
from .cancel import (
AgentCancelledError,
CancelTokenRegistry,
get_cancel_registry,
)
__all__ = [
'Agent',
@@ -16,5 +21,8 @@ __all__ = [
'ToolResult',
'LLMModel',
'LLMRequest',
'ModelFactory'
]
'ModelFactory',
'AgentCancelledError',
'CancelTokenRegistry',
'get_cancel_registry',
]

View File

@@ -52,6 +52,11 @@ class Agent:
self.workspace_dir = workspace_dir # Workspace directory
self.enable_skills = enable_skills # Skills enabled flag
self.runtime_info = runtime_info # Runtime info for dynamic time update
# Optional extra instructions appended AFTER the rebuilt full system
# prompt. Used by the self-evolution review agent to add its task brief
# on top of the full context (tools, workspace, user preferences, time)
# so it both follows the user's preferences and knows its evolution job.
self.extra_system_suffix = None
# Initialize skill manager
self.skill_manager = None
@@ -114,16 +119,26 @@ class Agent:
context_files = load_context_files(self.workspace_dir) if self.workspace_dir else None
builder = PromptBuilder(workspace_dir=self.workspace_dir or "", language="zh")
return builder.build(
try:
from common import i18n
lang = i18n.get_language()
except Exception:
lang = "zh"
builder = PromptBuilder(workspace_dir=self.workspace_dir or "", language=lang)
full = builder.build(
tools=self.tools,
context_files=context_files,
skill_manager=self.skill_manager,
memory_manager=self.memory_manager,
runtime_info=self.runtime_info,
)
if self.extra_system_suffix:
full = f"{full}\n\n{self.extra_system_suffix}"
return full
except Exception as e:
logger.warning(f"Failed to rebuild system prompt, using cached version: {e}")
if self.extra_system_suffix:
return f"{self.system_prompt}\n\n{self.extra_system_suffix}"
return self.system_prompt
def refresh_skills(self):
@@ -365,7 +380,8 @@ class Agent:
return action
def run_stream(self, user_message: str, on_event=None, clear_history: bool = False, skill_filter=None) -> str:
def run_stream(self, user_message: str, on_event=None, clear_history: bool = False,
skill_filter=None, cancel_event=None) -> str:
"""
Execute single agent task with streaming (based on tool-call)
@@ -374,6 +390,7 @@ class Agent:
- Multi-turn reasoning based on tool-call
- Event callbacks
- Persistent conversation history across calls
- User-initiated cancellation via ``cancel_event``
Args:
user_message: User message
@@ -381,6 +398,11 @@ class Agent:
event = {"type": str, "timestamp": float, "data": dict}
clear_history: If True, clear conversation history before this call (default: False)
skill_filter: Optional list of skill names to include in this run
cancel_event: Optional threading.Event polled at agent checkpoints.
When set, the loop exits at the next safe point, injects a
"[Interrupted by user]" assistant note, and returns the
partial response. ``messages`` stays in a valid state
(tool_use/tool_result pairs preserved).
Returns:
Final response text
@@ -424,7 +446,8 @@ class Agent:
max_turns=self.max_steps,
on_event=on_event,
messages=messages_copy, # Pass copied message history
max_context_turns=max_context_turns
max_context_turns=max_context_turns,
cancel_event=cancel_event,
)
# Execute

View File

@@ -7,10 +7,19 @@ import json
import time
from typing import List, Dict, Any, Optional, Callable, Tuple
from agent.protocol.cancel import AgentCancelledError
from agent.protocol.models import LLMRequest, LLMModel
from agent.protocol.message_utils import sanitize_claude_messages, compress_turn_to_text_only
from agent.tools.base_tool import BaseTool, ToolResult
from common.log import logger
from common.i18n import t as _t
# Optional: repair malformed JSON args from non-strict providers (e.g. unescaped quotes in long content).
try:
from json_repair import repair_json as _repair_json
_HAS_JSON_REPAIR = True
except ImportError:
_HAS_JSON_REPAIR = False
# Maximum number of characters of model "reasoning / thinking" content to persist
@@ -44,6 +53,30 @@ def _truncate_reasoning_for_storage(text: str) -> str:
return head + _REASONING_TRUNCATE_MARKER.format(omitted=omitted) + tail
def _parse_tool_args(args_str: str, finish_reason: Optional[str]) -> Tuple[dict, Optional[str]]:
"""Parse tool args JSON. Returns (args, error_msg); error_msg is None on success.
On JSONDecodeError: detect truncation first (skip repair, surface max_tokens hint);
otherwise try json-repair for escape issues; finally fall back to the raw decoder error.
"""
if not args_str:
return {}, None
try:
return json.loads(args_str), None
except json.JSONDecodeError as e:
if finish_reason in ("length", "max_tokens") or not args_str.rstrip().endswith("}"):
return {}, "Output truncated (max_tokens reached). Split content into smaller chunks across multiple tool calls."
if _HAS_JSON_REPAIR:
try:
repaired = _repair_json(args_str, return_objects=True)
if isinstance(repaired, dict):
logger.warning(f"Tool args JSON repaired ({len(args_str)} chars)")
return repaired, None
except Exception:
pass
return {}, f"Invalid JSON in tool arguments: {e.msg}"
class AgentStreamExecutor:
"""
Agent Stream Executor
@@ -64,7 +97,8 @@ class AgentStreamExecutor:
max_turns: int = 50,
on_event: Optional[Callable] = None,
messages: Optional[List[Dict]] = None,
max_context_turns: int = 30
max_context_turns: int = 30,
cancel_event=None,
):
"""
Initialize stream executor
@@ -78,6 +112,10 @@ class AgentStreamExecutor:
on_event: Event callback function
messages: Optional existing message history (for persistent conversations)
max_context_turns: Maximum number of conversation turns to keep in context
cancel_event: Optional threading.Event used to signal user cancel.
Checked at every safe point (turn boundary, before tool execution,
during LLM streaming). When set, raises AgentCancelledError which
run_stream catches to gracefully wind down.
"""
self.agent = agent
self.model = model
@@ -87,6 +125,7 @@ class AgentStreamExecutor:
self.max_turns = max_turns
self.on_event = on_event
self.max_context_turns = max_context_turns
self.cancel_event = cancel_event
# Message history - use provided messages or create new list
self.messages = messages if messages is not None else []
@@ -97,6 +136,73 @@ class AgentStreamExecutor:
# Track files to send (populated by read tool)
self.files_to_send = [] # List of file metadata dicts
def _check_cancelled(self) -> None:
"""Raise AgentCancelledError if the user requested cancellation.
Called at safe points (turn start, between tool calls, between LLM
chunks). Cheap to call: just an Event.is_set() probe.
"""
if self.cancel_event is not None and self.cancel_event.is_set():
raise AgentCancelledError("agent cancelled by user")
def _handle_cancelled(self, partial_response: str) -> None:
"""Wind down ``self.messages`` after a user-initiated cancel.
The messages list may be in any of these states when we get here:
(a) Last message is an assistant message containing tool_use
blocks but the matching tool_result has not been appended yet.
(b) Last message is an assistant text-only reply (cancel happened
right before the next turn started).
(c) Last message is a user tool_result message and we cancelled
between turns.
For (a) we MUST synthesise tool_result blocks, otherwise the next
request will fail Claude/OpenAI's strict pairing validation. For
(b)/(c) the state is already valid and we just append a small
cancellation note so the user/LLM both see the boundary clearly.
"""
try:
# Step 1: close any orphaned tool_use in the trailing assistant
# message by injecting matching tool_result blocks.
if self.messages and isinstance(self.messages[-1], dict) \
and self.messages[-1].get("role") == "assistant":
last = self.messages[-1]
content = last.get("content")
if isinstance(content, list):
pending_tool_use_ids = [
block.get("id")
for block in content
if isinstance(block, dict) and block.get("type") == "tool_use"
]
pending_tool_use_ids = [tid for tid in pending_tool_use_ids if tid]
if pending_tool_use_ids:
tool_result_blocks = [
{
"type": "tool_result",
"tool_use_id": tid,
"content": "Cancelled by user before this tool finished.",
"is_error": True,
}
for tid in pending_tool_use_ids
]
self.messages.append({
"role": "user",
"content": tool_result_blocks,
})
logger.info(
f"[Agent] Injected {len(tool_result_blocks)} cancellation "
f"tool_result blocks to keep message history valid"
)
# Step 2: append a stable "interrupted" marker so the LLM sees a
# clear stop boundary on the next turn.
self.messages.append({
"role": "assistant",
"content": [{"type": "text", "text": "_(Cancelled by user)_"}],
})
except Exception as e:
logger.warning(f"[Agent] _handle_cancelled cleanup failed: {e}")
def _emit_event(self, event_type: str, data: dict = None):
"""Emit event"""
if self.on_event:
@@ -212,7 +318,10 @@ class AgentStreamExecutor:
# Hard stop at 8 failures - abort with critical message
if same_tool_failures >= 8:
return True, f"抱歉,我没能完成这个任务。可能是我理解有误或者当前方法不太合适。\n\n建议你:\n• 换个方式描述需求试试\n• 把任务拆分成更小的步骤\n• 或者换个思路来解决", True
return True, _t(
"抱歉,我没能完成这个任务。可能是我理解有误或者当前方法不太合适。\n\n建议你:\n• 换个方式描述需求试试\n• 把任务拆分成更小的步骤\n• 或者换个思路来解决",
"Sorry, I couldn't complete this task. I may have misunderstood, or my current approach isn't quite right.\n\nYou could try:\n• Rephrasing your request\n• Breaking the task into smaller steps\n• Taking a different approach",
), True
# Warning at 6 failures
if same_tool_failures >= 6:
@@ -238,11 +347,14 @@ class AgentStreamExecutor:
Returns:
Final response text
"""
# Log user message with model info
# Log user message with model info. Truncate very long messages (e.g.
# injected transcripts / large prompts) so logs stay readable.
thinking_enabled = self._is_thinking_enabled()
thinking_label = " | 💭 thinking" if thinking_enabled else ""
logger.info(f"🤖 {self.model.model}{thinking_label} | 👤 {user_message}")
_log_msg = user_message if len(user_message) <= 500 else (
user_message[:500] + f" …(+{len(user_message) - 500} chars)"
)
logger.info(f"🤖 {self.model.model}{thinking_label} | 👤 {_log_msg}")
# Add user message (Claude format - use content blocks for consistency)
self.messages.append({
@@ -267,13 +379,24 @@ class AgentStreamExecutor:
self._emit_event("agent_start")
# Reset the run-scoped MCP tool-retrieval accumulator. On-demand tool
# retrieval only grows this set within a run, so a tool that already
# produced a tool_use never disappears from the schema mid-run (which
# would make Claude/MiniMax raise a message-format error).
self._retrieved_mcp_names = set()
final_response = ""
turn = 0
cancelled = False
try:
while turn < self.max_turns:
# Check at the very top of every turn so a cancel arriving
# between turns short-circuits cleanly.
self._check_cancelled()
turn += 1
logger.info(f"[Agent] {turn}")
logger.info(f"[Agent] Turn {turn}")
self._emit_event("turn_start", {"turn": turn})
# Call LLM (enable retry_on_empty for better reliability)
@@ -326,14 +449,16 @@ class AgentStreamExecutor:
elif not assistant_msg:
# Still empty (no text and no tool_calls): use fallback
logger.warning(f"[Agent] Still empty after explicit request")
final_response = (
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。"
final_response = _t(
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。",
"Sorry, I can't generate a reply right now. Please try rephrasing your request, or try again later.",
)
logger.info(f"Generated fallback response for empty LLM output")
else:
# 第一轮就空回复,直接 fallback
final_response = (
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。"
# First-turn empty reply, fall back directly
final_response = _t(
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。",
"Sorry, I can't generate a reply right now. Please try rephrasing your request, or try again later.",
)
logger.info(f"Generated fallback response for empty LLM output")
else:
@@ -342,7 +467,7 @@ class AgentStreamExecutor:
# If the explicit-response retry produced tool_calls, skip the break
# and continue down to the tool execution branch in this same iteration.
if not tool_calls:
logger.debug(f"完成 (无工具调用)")
logger.debug(f"Done (no tool calls)")
self._emit_event("turn_end", {
"turn": turn,
"has_tool_calls": False
@@ -375,6 +500,8 @@ class AgentStreamExecutor:
try:
for tool_call in tool_calls:
# Honour cancel between tool invocations within the same turn
self._check_cancelled()
result = self._execute_tool(tool_call)
tool_results.append(result)
@@ -396,13 +523,13 @@ class AgentStreamExecutor:
result_data = result.get("result")
if result_data.get("type") == "file_to_send":
self.files_to_send.append(result_data)
logger.info(f"📎 检测到待发送文件: {result_data.get('file_name', result_data.get('path'))}")
logger.info(f"📎 File queued for sending: {result_data.get('file_name', result_data.get('path'))}")
self._emit_event("file_to_send", result_data)
# Check for critical error - abort entire conversation
if result.get("status") == "critical_error":
logger.error(f"💥 检测到严重错误,终止对话")
final_response = result.get('result', '任务执行失败')
logger.error(f"💥 Fatal error detected, aborting conversation")
final_response = result.get('result') or _t("任务执行失败", "Task execution failed")
return final_response
# Log tool result in compact format
@@ -513,7 +640,7 @@ class AgentStreamExecutor:
})
if turn >= self.max_turns:
logger.warning(f"⚠️ 已达到最大决策步数限制: {self.max_turns}")
logger.warning(f"⚠️ Reached max decision step limit: {self.max_turns}")
# Force model to summarize without tool calls
logger.info(f"[Agent] Requesting summary from LLM after reaching max steps...")
@@ -538,15 +665,15 @@ class AgentStreamExecutor:
logger.info(f"💭 Summary: {summary_response[:150]}{'...' if len(summary_response) > 150 else ''}")
else:
# Fallback if model still doesn't respond
final_response = (
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。"
"任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。"
final_response = _t(
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。",
f"I've taken {turn} decision steps and reached the per-run limit. The task may not be fully complete — try breaking it into smaller steps, or describe your request differently.",
)
except Exception as e:
logger.warning(f"Failed to get summary from LLM: {e}")
final_response = (
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。"
"任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。"
final_response = _t(
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。",
f"I've taken {turn} decision steps and reached the per-run limit. The task may not be fully complete — try breaking it into smaller steps, or describe your request differently.",
)
finally:
# Remove the injected user prompt from history to avoid polluting
@@ -557,18 +684,94 @@ class AgentStreamExecutor:
self.messages.pop(prompt_insert_idx)
logger.debug("[Agent] Removed injected max-steps prompt from message history")
except AgentCancelledError:
# User-initiated stop: wind down message history cleanly so the
# next turn is unaffected; channels emit a "cancelled" UI event.
cancelled = True
logger.info(f"[Agent] 🛑 Cancelled by user (turn {turn})")
self._handle_cancelled(final_response)
if not final_response or not final_response.strip():
final_response = "_(Cancelled)_"
except Exception as e:
logger.error(f"❌ Agent执行错误: {e}")
logger.error(f"❌ Agent execution error: {e}")
self._emit_event("error", {"error": str(e)})
raise
finally:
final_response = final_response.strip() if final_response else final_response
logger.info(f"[Agent] 🏁 完成 ({turn}轮)")
self._emit_event("agent_end", {"final_response": final_response})
if cancelled:
# Emit before agent_end so channels can mark UI as cancelled
self._emit_event("agent_cancelled", {"final_response": final_response})
logger.info(f"[Agent] 🏁 Done ({turn} turns)" + (" [cancelled]" if cancelled else ""))
self._emit_event("agent_end", {"final_response": final_response, "cancelled": cancelled})
return final_response
def _select_tools_for_injection(self) -> list:
"""Decide which tools to inject into the current LLM turn.
Built-in tools are ALWAYS injected in full (skills and core flows hard
depend on them). MCP tools are also injected in full UNLESS on-demand
retrieval is enabled AND the MCP tool count exceeds the configured
threshold — then only the most relevant MCP tools are injected, unioned
with those already selected earlier in this run (only-grows, so a tool
that already produced a tool_use never vanishes from the schema).
Degrades safely: disabled feature, no embedding provider, embedding
failure, count below threshold, or any error → inject all tools. Tools
are never silently dropped.
"""
all_tools = list(self.tools.values())
try:
from config import conf
if not conf().get("mcp_tool_retrieval_enabled", False):
return all_tools
from agent.tools.mcp.mcp_tool import McpTool
mcp_tools = [t for t in all_tools if isinstance(t, McpTool)]
builtin_tools = [t for t in all_tools if not isinstance(t, McpTool)]
threshold = int(conf().get("mcp_tool_retrieval_threshold", 20) or 20)
if len(mcp_tools) <= threshold:
return all_tools
top_k = int(conf().get("mcp_tool_retrieval_top_k", 10) or 10)
from agent.tools import ToolManager
from agent.tools.mcp.tool_retrieval import (
build_retrieval_query,
select_mcp_tools,
)
tm = ToolManager()
tool_vectors = tm.get_mcp_tool_vectors()
query = build_retrieval_query(self.messages)
query_vector = tm.embed_query(query)
selected = select_mcp_tools(
query_vector,
tool_vectors,
top_k,
getattr(self, "_retrieved_mcp_names", set()),
)
if selected is None:
# No provider / empty index / error → full injection.
return all_tools
# Persist the accumulated selection for subsequent turns.
self._retrieved_mcp_names = selected
selected_mcp = [t for t in mcp_tools if t.name in selected]
logger.info(
f"[ToolRetrieval] Injecting {len(builtin_tools)} built-in + "
f"{len(selected_mcp)}/{len(mcp_tools)} MCP tool(s) (top_k={top_k})"
)
return builtin_tools + selected_mcp
except Exception as e:
logger.debug(f"[ToolRetrieval] full injection (retrieval skipped): {e}")
return all_tools
def _call_llm_stream(self, retry_on_empty=True, retry_count=0, max_retries=3,
_overflow_retry: bool = False) -> Tuple[str, List[Dict]]:
"""
@@ -609,7 +812,7 @@ class AgentStreamExecutor:
tools_schema = None
if self.tools:
tools_schema = []
for tool in self.tools.values():
for tool in self._select_tools_for_injection():
input_schema = tool.params
try:
dynamic = (tool.get_json_schema() or {}).get("parameters") or {}
@@ -623,6 +826,22 @@ class AgentStreamExecutor:
"input_schema": input_schema,
})
# Debug: dump the full system prompt and messages sent to the LLM.
# Gated behind `debug` config to avoid flooding normal logs.
# try:
# from config import conf
# if conf().get("debug", False):
# logger.debug(
# "[Agent][debug] system_prompt sent to LLM "
# f"({len(self.system_prompt or '')} chars):\n"
# "================ SYSTEM PROMPT BEGIN ================\n"
# f"{self.system_prompt}\n"
# "================ SYSTEM PROMPT END =================="
# )
# logger.info(f"[Agent][debug] messages sent to LLM: {messages}")
# except Exception:
# pass
# Create request
request = LLMRequest(
messages=messages,
@@ -644,7 +863,32 @@ class AgentStreamExecutor:
try:
stream = self.model.call_stream(request)
# Probe cancel every N chunks to bound reaction time without
# checking on every token.
_cancel_probe_counter = 0
_CANCEL_PROBE_EVERY = 8
for chunk in stream:
_cancel_probe_counter += 1
if _cancel_probe_counter >= _CANCEL_PROBE_EVERY:
_cancel_probe_counter = 0
if self.cancel_event is not None and self.cancel_event.is_set():
# Persist partial text only; tool_use args may be
# truncated mid-stream and would fail validation.
logger.info("[Agent] cancel detected mid-stream, aborting LLM call")
if full_content:
partial_msg = {
"role": "assistant",
"content": [{"type": "text", "text": full_content}],
}
self.messages.append(partial_msg)
self._emit_event("message_end", {
"content": full_content,
"tool_calls": [],
"cancelled": True,
})
raise AgentCancelledError("cancelled during LLM streaming")
# Check for errors
if isinstance(chunk, dict) and chunk.get("error"):
# Extract error message from nested structure
@@ -738,6 +982,10 @@ class AgentStreamExecutor:
elif isinstance(choice, dict) and choice.get("_gemini_raw_parts"):
gemini_raw_parts = choice["_gemini_raw_parts"]
except AgentCancelledError:
# Must propagate untouched; never treat as a retryable error.
raise
except Exception as e:
error_str = str(e)
error_str_lower = error_str.lower()
@@ -800,13 +1048,15 @@ class AgentStreamExecutor:
self.messages.clear()
self._clear_session_db()
if is_context_overflow:
raise Exception(
"抱歉,对话历史过长导致上下文溢出。我已清空历史记录,请重新描述你的需求。"
)
raise Exception(_t(
"抱歉,对话历史过长导致上下文溢出。我已清空历史记录,请重新描述你的需求。",
"Sorry, the conversation history got too long and overflowed the context. I've cleared the history — please describe your request again.",
))
else:
raise Exception(
"抱歉,之前的对话出现了问题。我已清空历史记录,请重新发送你的消息。"
)
raise Exception(_t(
"抱歉,之前的对话出现了问题。我已清空历史记录,请重新发送你的消息。",
"Sorry, something went wrong with the earlier conversation. I've cleared the history — please send your message again.",
))
# Check if error is rate limit (429)
is_rate_limit = '429' in error_str_lower or 'rate limit' in error_str_lower
@@ -851,26 +1101,17 @@ class AgentStreamExecutor:
import uuid
tool_id = f"call_{uuid.uuid4().hex[:24]}"
try:
# Safely get arguments, handle None case
args_str = tc.get("arguments") or ""
arguments = json.loads(args_str) if args_str else {}
except json.JSONDecodeError as e:
# Handle None or invalid arguments safely
args_str = tc.get('arguments') or ""
args_preview = args_str[:200] if len(args_str) > 200 else args_str
logger.error(f"Failed to parse tool arguments for {tc['name']}")
logger.error(f"Arguments length: {len(args_str)} chars")
logger.error(f"Arguments preview: {args_preview}...")
logger.error(f"JSON decode error: {e}")
# Return a clear error message to the LLM instead of empty dict
# This helps the LLM understand what went wrong
args_str = tc.get("arguments") or ""
arguments, parse_err = _parse_tool_args(args_str, stop_reason)
if parse_err:
logger.error(
f"Tool args parse failed for {tc['name']} ({len(args_str)} chars): {parse_err}"
)
tool_calls.append({
"id": tool_id,
"name": tc["name"],
"arguments": {},
"_parse_error": f"Invalid JSON in tool arguments: {args_preview}... Error: {str(e)}. Tip: For large content, consider splitting into smaller chunks or using a different approach."
"_parse_error": parse_err,
})
continue
@@ -958,14 +1199,11 @@ class AgentStreamExecutor:
tool_id = tool_call["id"]
arguments = tool_call["arguments"]
# Check if there was a JSON parse error
if "_parse_error" in tool_call:
parse_error = tool_call["_parse_error"]
logger.error(f"Skipping tool execution due to parse error: {parse_error}")
result = {
"status": "error",
"result": f"Failed to parse tool arguments. {parse_error}. Please ensure your tool call uses valid JSON format with all required parameters.",
"execution_time": 0
"result": tool_call["_parse_error"],
"execution_time": 0,
}
self._record_tool_result(tool_name, arguments, False)
return result
@@ -1006,10 +1244,21 @@ class AgentStreamExecutor:
# Set tool context
tool.model = self.model
tool.context = self.agent
tool.progress_callback = lambda message: self._emit_event(
"tool_execution_progress",
{
"tool_call_id": tool_id,
"tool_name": tool_name,
"message": message,
}
)
# Execute tool
start_time = time.time()
result: ToolResult = tool.execute_tool(arguments)
try:
result: ToolResult = tool.execute_tool(arguments)
finally:
tool.progress_callback = None
execution_time = time.time() - start_time
result_dict = {
@@ -1397,8 +1646,8 @@ class AgentStreamExecutor:
turns = turns[-keep_count:]
logger.info(
f"💾 上下文轮次超限: {keep_count + removed_count} > {self.max_context_turns}"
f"裁剪至 {keep_count} 轮(移除 {removed_count} 轮)"
f"💾 Context turns exceeded: {keep_count + removed_count} > {self.max_context_turns}, "
f"trimmed to {keep_count} turns (removed {removed_count})"
)
# Flush to daily memory + inject context summary (single async LLM call)
@@ -1446,7 +1695,7 @@ class AgentStreamExecutor:
# Log if we removed messages due to turn limit
if old_count > len(self.messages):
logger.info(f" 重建消息列表: {old_count} -> {len(self.messages)} 条消息")
logger.info(f" Rebuilt message list: {old_count} -> {len(self.messages)} messages")
return
# Token limit exceeded — tiered strategy based on turn count:
@@ -1479,10 +1728,10 @@ class AgentStreamExecutor:
self.messages = new_messages
logger.info(
f"📦 上下文tokens超限(轮次<{COMPRESS_THRESHOLD}): "
f"~{current_tokens + system_tokens} > {max_tokens}"
f"压缩全部 {len(turns)} 轮为纯文本 "
f"({old_count} -> {len(self.messages)} 条消息,"
f"📦 Context tokens exceeded (turns<{COMPRESS_THRESHOLD}): "
f"~{current_tokens + system_tokens} > {max_tokens}, "
f"compressed all {len(turns)} turns to plain text "
f"({old_count} -> {len(self.messages)} messages, "
f"~{current_tokens + system_tokens} -> ~{new_tokens + system_tokens} tokens)"
)
return
@@ -1495,8 +1744,8 @@ class AgentStreamExecutor:
kept_tokens = sum(self._estimate_turn_tokens(t) for t in kept_turns)
logger.info(
f"🔄 上下文tokens超限: ~{current_tokens + system_tokens} > {max_tokens}"
f"裁剪至 {keep_count} 轮(移除 {removed_count} 轮)"
f"🔄 Context tokens exceeded: ~{current_tokens + system_tokens} > {max_tokens}, "
f"trimmed to {keep_count} turns (removed {removed_count})"
)
if self.agent.memory_manager:
@@ -1520,8 +1769,8 @@ class AgentStreamExecutor:
self.messages = new_messages
logger.info(
f" 移除了 {removed_count} 轮对话 "
f"({old_count} -> {len(self.messages)} 条消息,"
f" Removed {removed_count} turns "
f"({old_count} -> {len(self.messages)} messages, "
f"~{current_tokens + system_tokens} -> ~{kept_tokens + system_tokens} tokens)"
)
@@ -1551,4 +1800,4 @@ class AgentStreamExecutor:
not as a message. The AgentLLMModel will handle this.
"""
# Don't add system message here - it will be handled separately by the LLM adapter
return self.messages
return self.messages

121
agent/protocol/cancel.py Normal file
View File

@@ -0,0 +1,121 @@
"""
Cancel token registry for aborting in-flight agent runs.
A user cancel (web Cancel button, /cancel command) sets a threading.Event
that the agent loop polls at safe checkpoints. Tokens are keyed by
request_id (preferred) and tracked under session_id as a fallback. Entries
are released after the run completes to keep the registry bounded.
No project deps — importable from any layer without circular imports.
"""
from __future__ import annotations
import threading
from typing import Dict, Optional
class AgentCancelledError(Exception):
"""Raised inside the agent loop when a stop has been requested.
The agent stream executor catches this, injects a "[Interrupted]" note
into the message history (preserving tool_use/tool_result integrity)
and returns a partial response to the caller.
"""
class _CancelEntry:
__slots__ = ("event", "session_id")
def __init__(self, session_id: Optional[str]):
self.event = threading.Event()
self.session_id = session_id
class CancelTokenRegistry:
"""In-process registry mapping request_id -> cancel Event.
Thread-safe. Singleton via module-level ``_registry``.
"""
def __init__(self):
self._lock = threading.Lock()
self._by_request: Dict[str, _CancelEntry] = {}
# session_id -> set of request_ids currently in flight (usually 1).
self._by_session: Dict[str, set] = {}
def register(self, request_id: str, session_id: Optional[str] = None) -> threading.Event:
"""Create (or return existing) cancel event for a request.
Returns the threading.Event the caller should poll via ``is_set()``.
"""
if not request_id:
return threading.Event()
with self._lock:
entry = self._by_request.get(request_id)
if entry is None:
entry = _CancelEntry(session_id)
self._by_request[request_id] = entry
if session_id:
self._by_session.setdefault(session_id, set()).add(request_id)
return entry.event
def get_event(self, request_id: str) -> Optional[threading.Event]:
if not request_id:
return None
with self._lock:
entry = self._by_request.get(request_id)
return entry.event if entry else None
def cancel_request(self, request_id: str) -> bool:
"""Trigger cancel for a specific request. Returns True when matched."""
if not request_id:
return False
with self._lock:
entry = self._by_request.get(request_id)
if entry is None:
return False
entry.event.set()
return True
def cancel_session(self, session_id: str) -> int:
"""Trigger cancel for every in-flight request of a session.
Returns the number of requests cancelled (0 when nothing was running).
"""
if not session_id:
return 0
with self._lock:
request_ids = list(self._by_session.get(session_id, ()))
entries = [self._by_request[r] for r in request_ids if r in self._by_request]
for entry in entries:
entry.event.set()
return len(entries)
def unregister(self, request_id: str) -> None:
"""Remove an entry once the agent run is done. Safe to call twice."""
if not request_id:
return
with self._lock:
entry = self._by_request.pop(request_id, None)
if entry and entry.session_id:
bucket = self._by_session.get(entry.session_id)
if bucket is not None:
bucket.discard(request_id)
if not bucket:
self._by_session.pop(entry.session_id, None)
def has_active(self, session_id: str) -> bool:
if not session_id:
return False
with self._lock:
bucket = self._by_session.get(session_id)
return bool(bucket)
_registry = CancelTokenRegistry()
def get_cancel_registry() -> CancelTokenRegistry:
"""Module-level accessor for the singleton registry."""
return _registry

View File

@@ -34,6 +34,27 @@ class SkillService:
"""
self.manager = skill_manager
def _safe_skill_dir(self, name: str) -> str:
"""Derive and validate the skill directory path.
Ensures the resolved path stays within the custom_dir root,
preventing path traversal via names like ``../escaped``.
:raises ValueError: if the name would escape the skills root.
"""
if not name or not name.strip():
raise ValueError("skill name is required")
# Reject obvious traversal components.
if ".." in name or name.startswith("/") or name.startswith("\\"):
raise ValueError(f"invalid skill name (path traversal detected): {name!r}")
skill_dir = os.path.realpath(os.path.join(self.manager.custom_dir, name))
root = os.path.realpath(self.manager.custom_dir)
if not skill_dir.startswith(root + os.sep) and skill_dir != root:
raise ValueError(
f"skill name {name!r} resolves outside the skills directory"
)
return skill_dir
# ------------------------------------------------------------------
# query
# ------------------------------------------------------------------
@@ -107,7 +128,7 @@ class SkillService:
if not files:
raise ValueError("skill files list is empty")
skill_dir = os.path.join(self.manager.custom_dir, name)
skill_dir = self._safe_skill_dir(name)
tmp_dir = skill_dir + ".tmp"
if os.path.exists(tmp_dir):
@@ -146,7 +167,7 @@ class SkillService:
raise ValueError("package url is required")
url = files[0]["url"]
skill_dir = os.path.join(self.manager.custom_dir, name)
skill_dir = self._safe_skill_dir(name)
with tempfile.TemporaryDirectory() as tmp_dir:
zip_path = os.path.join(tmp_dir, "package.zip")
@@ -217,7 +238,7 @@ class SkillService:
if not name:
raise ValueError("skill name is required")
skill_dir = os.path.join(self.manager.custom_dir, name)
skill_dir = self._safe_skill_dir(name)
if os.path.exists(skill_dir):
shutil.rmtree(skill_dir)
logger.info(f"[SkillService] delete: removed directory {skill_dir}")

View File

@@ -14,6 +14,9 @@ from agent.tools.send.send import Send
from agent.tools.memory.memory_search import MemorySearchTool
from agent.tools.memory.memory_get import MemoryGetTool
# Import self-evolution tools
from agent.tools.evolution_undo.evolution_undo import EvolutionUndoTool
# Import tools with optional dependencies
def _import_optional_tools():
"""Import tools that have optional dependencies"""
@@ -135,6 +138,7 @@ __all__ = [
'Send',
'MemorySearchTool',
'MemoryGetTool',
'EvolutionUndoTool',
'EnvConfig',
'SchedulerTool',
'WebSearch',

View File

@@ -38,6 +38,16 @@ class BaseTool:
description: str = "Base tool"
params: dict = {} # Store JSON Schema
model: Optional[Any] = None # LLM model instance, type depends on bot implementation
progress_callback = None
def report_progress(self, message: str):
callback = getattr(self, "progress_callback", None)
if not callback:
return
try:
callback(str(message))
except Exception as e:
logger.debug(f"[{self.name}] progress callback failed: {e}")
@classmethod
def get_json_schema(cls) -> dict:

View File

@@ -4,9 +4,12 @@ Bash tool - Execute bash commands
import os
import re
import signal
import sys
import subprocess
import tempfile
import threading
import time
from typing import Dict, Any
from agent.tools.base_tool import BaseTool, ToolResult
@@ -19,6 +22,10 @@ class Bash(BaseTool):
"""Tool for executing bash commands"""
_IS_WIN = sys.platform == "win32"
_PROGRESS_MAX_BYTES = 4 * 1024
_PROGRESS_INTERVAL = 0.5
# cmd.exe command line limit is ~8191 chars; rewrite python -c above this.
_WIN_CMD_SAFE_LEN = 7000
name: str = "bash"
description: str = f"""Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last {DEFAULT_MAX_LINES} lines or {DEFAULT_MAX_BYTES // 1024}KB (whichever is hit first). If truncated, full output is saved to a temp file.
@@ -69,8 +76,8 @@ SAFETY:
if not command:
return ToolResult.fail("Error: command parameter is required")
# Security check: Prevent accessing sensitive config files
if "~/.cow/.env" in command or "~/.cow" in command:
# Security check: Prevent direct access to the credential file
if re.search(r'\.cow[/\\]\.env', command):
return ToolResult.fail(
"Error: Access denied. API keys and credentials must be accessed through the env_config tool only."
)
@@ -106,25 +113,35 @@ SAFETY:
else:
logger.debug(f"[Bash] Process User: {os.environ.get('USERNAME', os.environ.get('USER', 'unknown'))}")
# Temp script written for long `python -c` commands (Windows only),
# cleaned up after execution.
temp_script_path = None
# On Windows, convert $VAR references to %VAR% for cmd.exe
if self._IS_WIN:
env["PYTHONIOENCODING"] = "utf-8"
command = self._convert_env_vars_for_windows(command, dotenv_vars)
# cmd.exe has an ~8191 char command line limit. Long
# `python -c "..."` commands silently fail, so spill the inline
# code into a temp .py file and run that instead.
if len(command) > self._WIN_CMD_SAFE_LEN:
command, temp_script_path = self._rewrite_long_python_c(command)
if command and not command.strip().lower().startswith("chcp"):
command = f"chcp 65001 >nul 2>&1 && {command}"
result = subprocess.run(
command,
shell=True,
cwd=self.cwd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
errors="replace",
timeout=timeout,
env=env,
)
try:
result = self._run_streaming(
command,
timeout,
env,
dotenv_vars,
)
finally:
if temp_script_path:
try:
os.remove(temp_script_path)
except OSError:
pass
logger.debug(f"[Bash] Exit code: {result.returncode}")
logger.debug(f"[Bash] Stdout length: {len(result.stdout)}")
@@ -185,8 +202,12 @@ SAFETY:
total_bytes = len(output.encode('utf-8'))
if total_bytes > DEFAULT_MAX_BYTES:
# Save full output to temp file
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.log', prefix='bash-') as f:
# Save full output to temp file. encoding='utf-8' is required:
# the default text-mode encoding is the platform locale (e.g.
# cp936/GBK on Chinese Windows), which raises UnicodeEncodeError
# for output containing emoji or other non-locale characters and
# would discard an otherwise successful command result.
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.log', prefix='bash-', encoding='utf-8') as f:
f.write(output)
temp_file_path = f.name
@@ -236,6 +257,105 @@ SAFETY:
except Exception as e:
return ToolResult.fail(f"Error executing command: {str(e)}")
def _run_streaming(self, command: str, timeout: int, env: dict, dotenv_vars: dict):
process = subprocess.Popen(
command,
shell=True,
cwd=self.cwd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=env,
start_new_session=not self._IS_WIN,
)
stdout_chunks, stderr_chunks = [], []
recent = bytearray()
recent_lock = threading.Lock()
def drain(stream, chunks):
while True:
chunk = os.read(stream.fileno(), 4096)
if not chunk:
break
chunks.append(chunk)
with recent_lock:
recent.extend(chunk)
if len(recent) > self._PROGRESS_MAX_BYTES:
del recent[:-self._PROGRESS_MAX_BYTES]
readers = [
threading.Thread(target=drain, args=(process.stdout, stdout_chunks), daemon=True),
threading.Thread(target=drain, args=(process.stderr, stderr_chunks), daemon=True),
]
for reader in readers:
reader.start()
started = time.monotonic()
last_reported_at = started
last_snapshot = None
try:
while process.poll() is None:
now = time.monotonic()
elapsed = now - started
if elapsed >= timeout:
self._kill_process(process)
raise subprocess.TimeoutExpired(command, timeout)
if elapsed >= self._PROGRESS_INTERVAL and now - last_reported_at >= self._PROGRESS_INTERVAL:
with recent_lock:
snapshot = bytes(recent).decode("utf-8", errors="replace")
snapshot = self._redact_progress(snapshot, dotenv_vars)
if snapshot and snapshot != last_snapshot:
self.report_progress(snapshot)
last_snapshot = snapshot
last_reported_at = now
time.sleep(0.1)
finally:
if process.poll() is None:
self._kill_process(process)
process.wait()
join_deadline = time.monotonic() + 5
for reader in readers:
reader.join(timeout=max(0, join_deadline - time.monotonic()))
from types import SimpleNamespace
return SimpleNamespace(
returncode=process.returncode,
stdout=b"".join(stdout_chunks).decode("utf-8", errors="replace"),
stderr=b"".join(stderr_chunks).decode("utf-8", errors="replace"),
)
def _kill_process(self, process):
if self._IS_WIN:
try:
result = subprocess.run(
["taskkill", "/F", "/T", "/PID", str(process.pid)],
capture_output=True,
timeout=5,
)
if result.returncode != 0 and process.poll() is None:
process.kill()
except (OSError, subprocess.SubprocessError):
if process.poll() is None:
process.kill()
else:
try:
os.killpg(process.pid, signal.SIGKILL)
except (PermissionError, ProcessLookupError):
if process.poll() is None:
process.kill()
@staticmethod
def _redact_progress(text: str, dotenv_vars: dict) -> str:
text = re.sub(
r'(?i)\b(API_KEY|TOKEN|PASSWORD|AUTHORIZATION)\s*=\s*[^\s]+',
lambda match: f"{match.group(1)}=[REDACTED]",
text,
)
for value in dotenv_vars.values():
value = str(value or "")
if len(value) >= 6:
text = text.replace(value, "[REDACTED]")
return text
def _get_safety_warning(self, command: str) -> str:
"""
Get safety warning for absolutely catastrophic commands only.
@@ -293,3 +413,43 @@ SAFETY:
return m.group(0)
return re.sub(r'\$\{(\w+)\}|\$(\w+)', replace_match, command)
@staticmethod
def _rewrite_long_python_c(command: str):
"""
Rewrite `python -c "<code>"` into `python <tempfile>` to bypass the
cmd.exe command line length limit on Windows.
Returns (new_command, temp_file_path). On any parse failure the original
command and None are returned, so behavior is unchanged when unmatched.
"""
# Match: <python|python3|py> [flags] -c "<code>" (single or double quoted)
m = re.search(
r'^(?P<prefix>.*?\b(?:python3?|py)\b[^\n]*?\s-c\s+)'
r'(?P<quote>["\'])(?P<code>.*)(?P=quote)\s*(?P<suffix>.*)$',
command,
re.DOTALL,
)
if not m:
return command, None
quote = m.group("quote")
code = m.group("code")
# Reverse common shell-level escaping of the quote char inside the code.
code = code.replace("\\" + quote, quote)
try:
fd, path = tempfile.mkstemp(suffix=".py", prefix="bash-pyc-")
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(code)
except OSError:
return command, None
prefix = m.group("prefix")
# Drop the trailing "-c " from the prefix, keep the interpreter + flags.
interp = re.sub(r'\s-c\s+$', ' ', prefix).rstrip()
suffix = m.group("suffix").strip()
new_command = f'{interp} "{path}"'
if suffix:
new_command += f' {suffix}'
return new_command, path

View File

@@ -0,0 +1,290 @@
"""
Browser environment detection and capability resolution.
Centralizes everything about *where* a usable browser engine comes from, so
both the runtime (browser_service) and the installer (cli/commands/install)
agree on the same decisions:
- Whether the `playwright` Python package is importable.
- Whether a system Chrome / Edge is installed (Playwright can drive it via
the `channel="chrome"/"msedge"` launcher, no download needed).
- Where Playwright's own Chromium download lives (redirected to the writable
data dir so it survives frozen/desktop app updates).
Resolution priority (see resolve_engine):
1. system-chrome -> drive the user's installed Chrome / Edge (zero download)
2. playwright-chromium -> Playwright's own Chromium, if already downloaded
3. none -> nothing usable yet; caller should trigger onboarding
"""
import os
import sys
import shutil
from typing import Optional, Dict, Any
from common.log import logger
# Playwright browser channels we accept for the "system-chrome" mode, in
# preference order. "chrome" covers stable Google Chrome; "msedge" is the
# Chromium-based Edge shipped on every Windows 10/11.
_PREFERRED_CHANNELS = ("chrome", "msedge", "chrome-beta", "msedge-beta")
def get_data_root() -> str:
"""Writable data root (~/.cow on desktop, else CWD-based).
Mirrors the logic in common/log.py without importing config, to avoid a
circular import. The desktop build sets COW_DATA_DIR; source deployments
fall back to the current working directory.
"""
data_dir = os.environ.get("COW_DATA_DIR")
if data_dir:
return os.path.expanduser(data_dir)
return os.getcwd()
def browsers_download_dir() -> str:
"""Directory Playwright downloads its Chromium into.
We pin it under the writable data root (~/.cow/ms-playwright) rather than
Playwright's default (~/.cache/ms-playwright or %USERPROFILE%). This keeps
the frozen desktop build self-contained and makes the download survive app
updates. Set as PLAYWRIGHT_BROWSERS_PATH for both install and runtime.
"""
return os.path.join(get_data_root(), "ms-playwright")
def apply_browsers_path_env() -> None:
"""Point Playwright at our pinned download dir via env var (idempotent).
Only set it when not already provided by the user, so power users can
override the location. Must run before importing playwright's launcher.
"""
if not os.environ.get("PLAYWRIGHT_BROWSERS_PATH"):
os.environ["PLAYWRIGHT_BROWSERS_PATH"] = browsers_download_dir()
def is_frozen() -> bool:
"""True when running inside a PyInstaller-frozen bundle (desktop backend).
In this mode sys.executable is the frozen exe (no pip), so the installer
must skip `pip install` and only download the browser binary.
"""
return bool(getattr(sys, "frozen", False))
def is_desktop() -> bool:
"""True when running as the Electron desktop client (dev or packaged).
The desktop shell always sets COW_DESKTOP=1 (see python-manager.ts), both in
`npm run dev` (runs app.py with the user's Python) and in the packaged build
(frozen exe). Desktop users have no `cow` CLI, so onboarding must point them
at the in-chat `/install-browser` command rather than a terminal command.
"""
return os.environ.get("COW_DESKTOP") == "1"
def has_playwright_package() -> bool:
"""True if the `playwright` Python package can be imported."""
try:
import playwright # noqa: F401
return True
except Exception:
return False
def _windows_program_dirs() -> list:
dirs = []
for var in ("PROGRAMFILES", "PROGRAMFILES(X86)", "LOCALAPPDATA"):
val = os.environ.get(var)
if val:
dirs.append(val)
return dirs
def detect_system_chrome() -> Optional[Dict[str, str]]:
"""Locate an installed Chromium-based browser Playwright can drive.
Returns a dict {"channel": <playwright channel>, "path": <exe path>} for
the first match, or None. The `channel` is what we hand to Playwright's
launcher; `path` is only informational (Playwright resolves the channel on
its own, but we keep the path for logging / onboarding messages).
"""
candidates = []
if sys.platform == "darwin":
candidates = [
("chrome", "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"),
("msedge", "/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge"),
("chrome-beta", "/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"),
]
elif sys.platform == "win32":
prog_dirs = _windows_program_dirs()
for base in prog_dirs:
candidates.append(("chrome", os.path.join(base, "Google", "Chrome", "Application", "chrome.exe")))
candidates.append(("msedge", os.path.join(base, "Microsoft", "Edge", "Application", "msedge.exe")))
else:
# Linux: rely on PATH lookups for the common binaries.
path_lookups = [
("chrome", "google-chrome"),
("chrome", "google-chrome-stable"),
("chrome", "chromium"),
("chrome", "chromium-browser"),
("msedge", "microsoft-edge"),
]
for channel, binary in path_lookups:
found = shutil.which(binary)
if found:
return {"channel": channel, "path": found}
for channel, path in candidates:
if path and os.path.exists(path):
return {"channel": channel, "path": path}
return None
def has_downloaded_chromium() -> bool:
"""True if Playwright already has a Chromium download available.
We check our pinned download dir for a chromium-* folder. This is a
lightweight heuristic (avoids importing/launching Playwright just to probe)
and matches how Playwright lays browsers out on disk.
"""
download_dir = browsers_download_dir()
if not os.path.isdir(download_dir):
return False
try:
for name in os.listdir(download_dir):
# Playwright names its browser dirs like "chromium-1140",
# "chromium_headless_shell-1140".
if name.startswith("chromium"):
return True
except OSError:
pass
return False
def resolve_engine(config: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""Decide which browser engine to use, given config and environment.
Returns a dict describing the launch strategy:
{
"mode": "system-chrome" | "playwright-chromium" | "none",
"channel": Optional[str], # for system-chrome
"path": Optional[str], # for system-chrome (informational)
"has_playwright": bool,
"reason": str, # human-readable, for logging / onboarding
}
Config keys under tools.browser that influence this:
- engine: "auto" (default) | "system-chrome" | "chromium"
Force a specific engine. "auto" prefers system Chrome, then falls
back to a downloaded Chromium.
- prefer_system_browser: bool (default True). When False under "auto",
skip system Chrome and go straight to Playwright's Chromium.
"""
config = config or {}
apply_browsers_path_env()
has_pw = has_playwright_package()
engine_pref = str(config.get("engine", "auto")).strip().lower()
prefer_system = config.get("prefer_system_browser", True)
if not has_pw:
return {
"mode": "none",
"channel": None,
"path": None,
"has_playwright": False,
"reason": "playwright package not available",
}
system = None
if engine_pref in ("auto", "system-chrome") and prefer_system:
system = detect_system_chrome()
if engine_pref == "system-chrome":
# Explicitly requested: use system Chrome if found, else report none.
if system:
return {
"mode": "system-chrome",
"channel": system["channel"],
"path": system["path"],
"has_playwright": True,
"reason": f"using system browser ({system['channel']})",
}
return {
"mode": "none",
"channel": None,
"path": None,
"has_playwright": True,
"reason": "engine=system-chrome but no Chrome/Edge found",
}
if engine_pref == "chromium":
# Explicitly requested Playwright's own Chromium.
if has_downloaded_chromium():
return {
"mode": "playwright-chromium",
"channel": None,
"path": None,
"has_playwright": True,
"reason": "using downloaded Playwright Chromium",
}
return {
"mode": "none",
"channel": None,
"path": None,
"has_playwright": True,
"reason": "engine=chromium but Chromium not downloaded yet",
}
# auto: system Chrome first, then downloaded Chromium.
if system:
return {
"mode": "system-chrome",
"channel": system["channel"],
"path": system["path"],
"has_playwright": True,
"reason": f"auto: using system browser ({system['channel']})",
}
if has_downloaded_chromium():
return {
"mode": "playwright-chromium",
"channel": None,
"path": None,
"has_playwright": True,
"reason": "auto: using downloaded Playwright Chromium",
}
return {
"mode": "none",
"channel": None,
"path": None,
"has_playwright": True,
"reason": "no system Chrome/Edge and no downloaded Chromium",
}
def capability_summary(config: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
"""High-level browser capability status, for onboarding / diagnostics.
Combines resolve_engine with raw detection flags so the UI / tool layer can
craft a helpful message (e.g. "Chrome detected, click to enable" vs
"no browser, will download ~150MB").
"""
engine = resolve_engine(config)
system = detect_system_chrome()
return {
"ready": engine["mode"] != "none",
"engine": engine,
"has_playwright": engine["has_playwright"],
"has_system_chrome": system is not None,
"system_chrome": system,
"has_downloaded_chromium": has_downloaded_chromium(),
"is_frozen": is_frozen(),
"is_desktop": is_desktop(),
"browsers_dir": browsers_download_dir(),
}

View File

@@ -15,7 +15,7 @@ import threading
from typing import Optional, Dict, Any, List, Callable
from common.log import logger
from common.utils import expand_path
from common.utils import expand_path, is_cloud_deployment
_DEFAULT_USER_DATA_DIR = "~/.cow/browser_profile"
@@ -326,12 +326,19 @@ class BrowserService:
# - persistent: launch with launch_persistent_context using a user_data_dir
# so cookies / login state survive across runs (default).
# - fresh: classic launch + new_context, clean state every run.
#
# Within persistent/fresh, the actual Chromium binary is resolved by
# browser_env.resolve_engine(): a system Chrome/Edge (channel-based, zero
# download) is preferred, falling back to Playwright's own downloaded
# Chromium. `self._channel` is the Playwright channel ("chrome"/"msedge")
# when driving a system browser, else None (bundled Chromium).
cdp_endpoint = self._config.get("cdp_endpoint") or ""
persistent_flag = self._config.get("persistent", True)
user_data_dir_cfg = self._config.get("user_data_dir")
if user_data_dir_cfg is None:
user_data_dir_cfg = _DEFAULT_USER_DATA_DIR
self._channel: Optional[str] = None
self._cdp_endpoint: str = cdp_endpoint.strip() if isinstance(cdp_endpoint, str) else ""
if self._cdp_endpoint:
self._launch_mode = "cdp"
@@ -343,6 +350,22 @@ class BrowserService:
self._launch_mode = "fresh"
self._user_data_dir = ""
# Resolve which browser engine to drive (system Chrome vs downloaded
# Chromium). Deferred detection failures are surfaced at launch time.
if self._launch_mode != "cdp":
try:
from agent.tools.browser.browser_env import resolve_engine
engine = resolve_engine(self._config)
if engine["mode"] == "system-chrome":
self._channel = engine["channel"]
logger.info(f"[Browser] Engine resolved: {engine['reason']}")
elif engine["mode"] == "playwright-chromium":
logger.info(f"[Browser] Engine resolved: {engine['reason']}")
else:
logger.info(f"[Browser] No ready engine yet: {engine['reason']}")
except Exception as e:
logger.debug(f"[Browser] Engine resolution skipped: {e}")
# Idle auto-release
idle_cfg = self._config.get("idle_timeout")
self._idle_timeout: float = float(idle_cfg) if idle_cfg is not None else self._IDLE_TIMEOUT_DEFAULT
@@ -428,6 +451,14 @@ class BrowserService:
def _launch_browser(self):
"""Launch / connect Chromium on the background thread."""
# Point Playwright at our pinned download dir before any launch so a
# bundled-Chromium fallback finds the browser downloaded to ~/.cow.
try:
from agent.tools.browser.browser_env import apply_browsers_path_env
apply_browsers_path_env()
except Exception as e:
logger.debug(f"[Browser] apply_browsers_path_env skipped: {e}")
if self._headless is None:
headless_cfg = self._config.get("headless")
self._headless = headless_cfg if headless_cfg is not None else _should_use_headless()
@@ -436,6 +467,20 @@ class BrowserService:
if self._headless:
launch_args.append("--no-sandbox")
if is_cloud_deployment():
launch_args.extend([
"--disable-gpu",
"--disable-software-rasterizer",
"--disable-extensions",
"--disable-background-networking",
"--disable-background-timer-throttling",
"--disable-renderer-backgrounding",
"--disable-features=site-per-process,TranslateUI,IsolateOrigins",
"--no-zygote",
"--js-flags=--max-old-space-size=384",
"--memory-pressure-off",
])
extra_args = self._config.get("launch_args", [])
if extra_args:
launch_args.extend(extra_args)
@@ -461,12 +506,20 @@ class BrowserService:
logger.info("[Browser] Browser ready")
def _launch_fresh(self, launch_args: List[str], viewport: Dict[str, int], user_agent: str):
"""Classic launch: brand new Chromium with an empty context."""
logger.info(f"[Browser] Launching Chromium (fresh, headless={self._headless})")
self._browser = self._playwright.chromium.launch(
headless=self._headless,
args=launch_args,
)
"""Classic launch: brand new Chromium with an empty context.
When `self._channel` is set (e.g. "chrome"/"msedge"), Playwright drives
the user's installed system browser instead of its own Chromium.
"""
engine_label = f"system:{self._channel}" if self._channel else "chromium"
logger.info(f"[Browser] Launching {engine_label} (fresh, headless={self._headless})")
launch_kwargs: Dict[str, Any] = {
"headless": self._headless,
"args": launch_args,
}
if self._channel:
launch_kwargs["channel"] = self._channel
self._browser = self._playwright.chromium.launch(**launch_kwargs)
self._context = self._browser.new_context(
viewport=viewport,
user_agent=user_agent,
@@ -477,18 +530,25 @@ class BrowserService:
def _launch_persistent(self, launch_args: List[str], viewport: Dict[str, int], user_agent: str):
"""Launch Chromium with a persistent user_data_dir so login state survives."""
os.makedirs(self._user_data_dir, exist_ok=True)
engine_label = f"system:{self._channel}" if self._channel else "chromium"
logger.info(
f"[Browser] Launching Chromium (persistent, headless={self._headless}, "
f"[Browser] Launching {engine_label} (persistent, headless={self._headless}, "
f"profile={self._user_data_dir})"
)
persistent_kwargs: Dict[str, Any] = {
"user_data_dir": self._user_data_dir,
"headless": self._headless,
"args": launch_args,
"viewport": viewport,
"user_agent": user_agent,
}
# When driving a system browser, let it use its real UA instead of the
# spoofed Chromium one (avoids UA/engine mismatch on real Chrome/Edge).
if self._channel:
persistent_kwargs["channel"] = self._channel
persistent_kwargs.pop("user_agent", None)
try:
self._context = self._playwright.chromium.launch_persistent_context(
user_data_dir=self._user_data_dir,
headless=self._headless,
args=launch_args,
viewport=viewport,
user_agent=user_agent,
)
self._context = self._playwright.chromium.launch_persistent_context(**persistent_kwargs)
except Exception as e:
# Profile is locked when another Chromium instance already holds it.
msg = str(e).lower()

View File

@@ -15,15 +15,24 @@ Launch modes (configured under `tools.browser` in config.json):
- fresh: Set `persistent` to false to fall back to a clean context every run.
"""
import ipaddress
import json
import os
import socket
from typing import Dict, Any, Optional
from urllib.parse import urlparse
from agent.tools.base_tool import BaseTool, ToolResult
from agent.tools.browser.browser_service import BrowserService
from common.log import logger
# Cloud-metadata endpoints worth blocking even though they are not link-local.
# (169.254.169.254 — AWS/GCP/Azure IMDS — is already covered by is_link_local;
# fd00:ec2::254 is the AWS IPv6 IMDS address.)
_CLOUD_METADATA_IPS = frozenset({ipaddress.ip_address("fd00:ec2::254")})
class BrowserTool(BaseTool):
"""Single tool exposing all browser actions via an 'action' parameter."""
@@ -121,6 +130,95 @@ class BrowserTool(BaseTool):
BrowserTool._shared_service = self._service
return self._service
def _allow_private_targets(self) -> bool:
"""Whether the link-local / cloud-metadata guard is disabled.
Defaults to False (guard active). Loopback and RFC1918/LAN targets are
always reachable so local dev servers work out of the box; this opt-out
only lifts the remaining block on link-local / cloud-metadata targets,
for an operator who deliberately needs them, by setting
``allow_private_targets: true`` under ``tools.browser`` in config.json.
"""
return bool(self.config.get("allow_private_targets", False))
@staticmethod
def _validate_url_safe(url: str) -> None:
"""Reject URLs that target link-local / cloud-metadata addresses (SSRF guard).
Resolves the hostname to its IP address(es) and blocks any that are
link-local (169.254.0.0/16 — which includes the 169.254.169.254
cloud-metadata endpoint — and IPv6 fe80::/10) or a known IPv6
cloud-metadata address. Also rejects URLs with no host, non-HTTP(S)
schemes, or hosts that fail DNS resolution.
Loopback and RFC1918/LAN targets are intentionally left reachable:
unlike the vision/web_fetch tools, the browser legitimately opens local
pages (a dev server on ``localhost`` / ``127.0.0.1`` / a LAN IP), so a
blanket "block all internal" policy would break that core workflow.
Raises:
ValueError: if the URL targets a disallowed address.
"""
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise ValueError(f"Unsupported URL scheme: {parsed.scheme}")
hostname = parsed.hostname
if not hostname:
raise ValueError("URL has no hostname")
try:
# Resolve all addresses for the hostname.
addr_infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
except socket.gaierror:
raise ValueError(f"Cannot resolve hostname: {hostname}")
for family, _, _, _, sockaddr in addr_infos:
ip_str = sockaddr[0]
ip = ipaddress.ip_address(ip_str)
# Block only the high-risk targets — link-local (incl. the
# 169.254.169.254 cloud-metadata endpoint) and the IPv6 metadata
# address. Loopback and RFC1918/LAN stay reachable for local dev.
if ip.is_link_local or ip in _CLOUD_METADATA_IPS:
raise ValueError(
f"URL resolves to a link-local / cloud-metadata address "
f"({ip_str}), request blocked for security"
)
def _check_engine_ready(self) -> Optional[ToolResult]:
"""Return an actionable onboarding message if no browser engine is ready.
Returns None when a system Chrome/Edge or a downloaded Chromium is
available (so the tool can proceed). Otherwise returns a ToolResult with
clear guidance so the agent asks the user to enable the browser instead
of surfacing a raw Playwright launch error. CDP mode is exempt (the
endpoint is external and validated at connect time).
"""
if self.config.get("cdp_endpoint"):
return None
try:
from agent.tools.browser.browser_env import capability_summary
summary = capability_summary(self.config)
except Exception as e:
logger.debug(f"[Browser] capability probe failed: {e}")
return None
if summary.get("ready"):
return None
# Desktop clients (dev or packaged) have no `cow` CLI — onboard via the
# in-chat `/install-browser` command. Source / web / server installs use
# the `cow install-browser` terminal command.
install_hint = (
"reply `/install-browser`" if summary.get("is_desktop")
else "run `cow install-browser` in a terminal"
)
return ToolResult.fail(
f"Browser tool not ready. Ask the user to {install_hint} (installs a browser engine; "
"skipped automatically if Google Chrome is already installed). "
"Do not retry until the user confirms."
)
def execute(self, args: Dict[str, Any]) -> ToolResult:
action = args.get("action", "").strip().lower()
if not action:
@@ -131,6 +229,13 @@ class BrowserTool(BaseTool):
valid = ", ".join(sorted(self._ACTION_MAP.keys()))
return ToolResult.fail(f"Unknown action '{action}'. Valid actions: {valid}")
# Preflight: on desktop the playwright package is bundled but the browser
# binary may be missing; return actionable onboarding instead of a cryptic
# launch failure.
not_ready = self._check_engine_ready()
if not_ready is not None:
return not_ready
try:
return handler(self, args)
except Exception as e:
@@ -145,8 +250,19 @@ class BrowserTool(BaseTool):
url = args.get("url", "").strip()
if not url:
return ToolResult.fail("Error: 'url' is required for navigate action")
if not url.startswith(("http://", "https://")):
# Only auto-prepend https:// for bare hosts; preserve file://, about:, data:, etc.
if "://" not in url and not url.startswith(("about:", "data:")):
url = "https://" + url
# SSRF guard: for http(s) targets, reject hosts that resolve to
# link-local / cloud-metadata addresses before the browser navigates
# (and then auto-snapshots the page back to the model). Loopback and
# RFC1918/LAN are allowed so local dev servers work. Non-HTTP schemes
# (about:/data:/file:/chrome:) are not network-egress targets here.
if url.split(":", 1)[0].lower() in ("http", "https") and not self._allow_private_targets():
try:
self._validate_url_safe(url)
except ValueError as e:
return ToolResult.fail(f"Error: {e}")
timeout = args.get("timeout", 30000)
service = self._get_service()
result = service.navigate(url, timeout=timeout)

View File

@@ -13,7 +13,7 @@ from agent.tools.utils.diff import (
detect_line_ending,
normalize_to_lf,
restore_line_endings,
normalize_for_fuzzy_match,
count_matches,
fuzzy_find_text,
generate_diff_string
)
@@ -110,10 +110,10 @@ class Edit(BaseTool):
"The old text must match exactly including all whitespace and newlines."
)
# Calculate occurrence count (use fuzzy normalized content for consistency)
fuzzy_content = normalize_for_fuzzy_match(normalized_content)
fuzzy_old_text = normalize_for_fuzzy_match(normalized_old_text)
occurrences = fuzzy_content.count(fuzzy_old_text)
# Count occurrences with the same matcher used to locate and
# replace (fuzzy_find_text), so the uniqueness guard cannot
# disagree with what actually gets replaced.
occurrences = count_matches(normalized_content, normalized_old_text)
if occurrences > 1:
return ToolResult.fail(

View File

@@ -0,0 +1,3 @@
from agent.tools.evolution_undo.evolution_undo import EvolutionUndoTool
__all__ = ["EvolutionUndoTool"]

View File

@@ -0,0 +1,58 @@
"""Evolution undo tool.
Lets the main chat agent roll back a previous self-evolution when the user asks
("undo the last learning"). The rollback itself is a deterministic FILE RESTORE
from the snapshot taken before the evolution — the model only supplies the
backup_id it reads from the [EVOLUTION] record in the conversation. No LLM-driven
re-editing is involved, so a restore can never make things worse.
"""
from agent.tools.base_tool import BaseTool, ToolResult
class EvolutionUndoTool(BaseTool):
"""Restore memory/skill files to the state before a self-evolution."""
name: str = "evolution_undo"
description: str = (
"Undo a previous self-evolution (self-learning) by restoring the "
"memory/skill files to their state before that learning. Use this when "
"the user asks to undo / revert / roll back the last self-learning. "
"Find the backup_id in the most recent [EVOLUTION] record in the "
"conversation and pass it here."
)
params: dict = {
"type": "object",
"properties": {
"backup_id": {
"type": "string",
"description": (
"The backup_id from the [EVOLUTION] record to restore "
"(e.g. '20260607-155551-850')."
),
}
},
"required": ["backup_id"],
}
def execute(self, args: dict):
backup_id = (args.get("backup_id") or "").strip()
if not backup_id:
return ToolResult.fail("Error: backup_id is required")
try:
from agent.memory.config import get_default_memory_config
from agent.evolution.backup import restore_backup
workspace_dir = get_default_memory_config().get_workspace()
ok = restore_backup(workspace_dir, backup_id)
if ok:
return ToolResult.success(
f"Restored memory/skills to the state before evolution "
f"{backup_id}. The previous self-learning has been undone."
)
return ToolResult.fail(
f"Could not find or restore backup {backup_id}. It may have "
f"expired or already been rolled back."
)
except Exception as e:
return ToolResult.fail(f"Error during undo: {e}")

View File

@@ -1,13 +1,13 @@
"""
MCP (Model Context Protocol) client module.
Implements JSON-RPC 2.0 over stdio and SSE transports without any external
MCP SDK dependency.
Implements JSON-RPC 2.0 over stdio, SSE and Streamable HTTP transports
without any external MCP SDK dependency.
"""
import json
import os
import select
import queue
import subprocess
import threading
import urllib.request
@@ -17,30 +17,104 @@ from typing import Optional
from common.log import logger
# Aliases accepted for the Streamable HTTP transport type
_STREAMABLE_HTTP_ALIASES = {"streamable-http", "streamable_http", "streamablehttp", "http"}
# Optional callback invoked after an OAuth authorization completes, so the
# tool manager can bring the newly-authorized server online. Signature:
# reload_fn(server_name: str) -> None. Installed by the tool manager.
_reload_callback = None
def set_reload_callback(fn) -> None:
"""Register a callback fired after a server's OAuth flow succeeds."""
global _reload_callback
_reload_callback = fn
def notify_server_authorized(server_name: str) -> None:
"""Called by the web callback once tokens are stored for a server."""
fn = _reload_callback
if fn is None:
logger.debug(f"[MCP:{server_name}] Authorized but no reload callback registered")
return
try:
fn(server_name)
except Exception as e:
logger.warning(f"[MCP:{server_name}] reload callback failed: {e}")
def _oauth_redirect_uri() -> str:
"""Build the OAuth redirect URI served by the web console callback.
Priority: explicit mcp_oauth_redirect_base config, otherwise the local
web console address (127.0.0.1:<web_port>). Both point at the shared
/mcp/oauth/callback route.
"""
try:
from config import conf
base = (conf().get("mcp_oauth_redirect_base") or "").strip().rstrip("/")
if not base:
port = int(os.environ.get("COW_WEB_PORT") or conf().get("web_port", 9899))
base = f"http://127.0.0.1:{port}"
except Exception:
base = "http://127.0.0.1:9899"
return f"{base}/mcp/oauth/callback"
class McpClient:
"""Single MCP Server client supporting stdio and SSE transports."""
"""Single MCP Server client supporting stdio, SSE and Streamable HTTP transports."""
def __init__(self, config: dict):
"""
config examples:
stdio: {"name": "filesystem", "type": "stdio", "command": "npx", "args": [...]}
SSE: {"name": "my-api", "type": "sse", "url": "http://localhost:8000/sse"}
stdio: {"name": "filesystem", "type": "stdio", "command": "npx", "args": [...]}
SSE: {"name": "my-api", "type": "sse", "url": "http://localhost:8000/sse"}
streamable-http: {"name": "pubmed", "type": "streamable-http", "url": "https://x/mcp"}
"""
self.config = config
self.name: str = config.get("name", "unknown")
self.transport: str = config.get("type", "stdio")
raw_transport: str = config.get("type", "stdio")
# Per-server timeout for tool calls (default 120s, suitable for data queries)
self._timeout: int = int(config.get("timeout", 120))
# Normalize streamable-http aliases to a single internal key
self.transport: str = (
"streamable-http"
if raw_transport.lower() in _STREAMABLE_HTTP_ALIASES
else raw_transport
)
# stdio state
self._proc: Optional[subprocess.Popen] = None
self._read_queue: queue.Queue = queue.Queue()
# SSE state
self._sse_url: Optional[str] = None
self._post_url: Optional[str] = None # endpoint for sending messages (resolved from SSE)
# Streamable HTTP state
self._http_url: Optional[str] = None
self._http_headers: dict = {} # extra headers from user config (e.g. Authorization)
self._http_session_id: Optional[str] = None # Mcp-Session-Id assigned by the server
# OAuth state (streamable-http only). Lazily created when the server
# responds with 401 and the user has not supplied a static token.
self._oauth = None # OAuthHandler instance
# Set to True once a 401 could not be satisfied and the user must
# complete the browser authorization. Callers can surface this state.
self.needs_auth: bool = False
# Shared state
self._next_id = 1
self._id_lock = threading.Lock()
# _call_lock serializes all requests on the single stdio pipe.
# SSE and streamable-http use independent HTTP requests, so they
# do not acquire this lock (see _send_request).
self._call_lock = threading.Lock()
# _http_lock protects _http_session_id initialization across
# concurrent streamable-http requests.
self._http_lock = threading.Lock()
self._initialized = False
# ------------------------------------------------------------------
@@ -54,6 +128,8 @@ class McpClient:
return self._init_stdio()
elif self.transport == "sse":
return self._init_sse()
elif self.transport == "streamable-http":
return self._init_streamable_http()
else:
logger.warning(f"[MCP:{self.name}] Unknown transport type: {self.transport!r}")
return False
@@ -109,6 +185,21 @@ class McpClient:
pass
self._proc = None
logger.debug(f"[MCP:{self.name}] stdio process terminated")
# Best-effort streamable-http session termination
if self.transport == "streamable-http" and self._http_session_id and self._http_url:
try:
req = urllib.request.Request(
self._http_url,
method="DELETE",
headers={"Mcp-Session-Id": self._http_session_id, **self._http_headers},
)
with urllib.request.urlopen(req, timeout=5):
pass
except Exception:
pass
self._http_session_id = None
self._initialized = False
# ------------------------------------------------------------------
@@ -139,6 +230,9 @@ class McpClient:
threading.Thread(
target=self._drain_stderr, daemon=True, name=f"mcp-stderr-{self.name}"
).start()
threading.Thread(
target=self._drain_stdout, daemon=True, name=f"mcp-stdout-{self.name}"
).start()
return self._handshake()
@@ -146,14 +240,35 @@ class McpClient:
for line in self._proc.stderr:
line = line.strip()
if line:
logger.debug(f"[MCP:{self.name}] stderr: {line}")
logger.warning(f"[MCP:{self.name}] stderr: {line}")
def _readline_with_timeout(self, timeout: int = 30) -> str:
"""Read one line from stdio stdout with a hard timeout."""
ready, _, _ = select.select([self._proc.stdout], [], [], timeout)
if not ready:
raise TimeoutError(f"[MCP:{self.name}] stdio read timed out after {timeout}s")
return self._proc.stdout.readline()
def _drain_stdout(self):
"""Background thread: read lines from stdout and put them into the queue."""
try:
for line in self._proc.stdout:
self._read_queue.put(line)
except Exception:
pass
finally:
try:
self._read_queue.put("")
except Exception:
pass
def _readline_with_timeout(self, timeout: Optional[int] = None) -> str:
"""Read one line from stdio stdout with a hard timeout (cross-platform).
Uses the per-server timeout from mcp.json config when no explicit
timeout is provided.
"""
effective = timeout if timeout is not None else self._timeout
try:
line = self._read_queue.get(timeout=effective)
except queue.Empty:
raise TimeoutError(f"[MCP:{self.name}] stdio read timed out after {effective}s")
if not line:
raise IOError(f"[MCP:{self.name}] stdio process closed unexpectedly")
return line
def _stdio_send(self, message: dict) -> dict:
"""Send a JSON-RPC message over stdio and read the response."""
@@ -161,6 +276,7 @@ class McpClient:
self._proc.stdin.write(raw)
self._proc.stdin.flush()
expected_id = message.get("id")
while True:
line = self._readline_with_timeout()
if not line:
@@ -175,6 +291,14 @@ class McpClient:
if "id" not in data:
logger.debug(f"[MCP:{self.name}] notification skipped: {data.get('method', '?')}")
continue
# Verify response id matches request id to avoid consuming a stale
# response left over from a previously failed/timed-out request.
if data.get("id") != expected_id:
logger.warning(
f"[MCP:{self.name}] Stale response id={data.get('id')} "
f"(expected {expected_id}), skipping"
)
continue
return data
# ------------------------------------------------------------------
@@ -234,6 +358,243 @@ class McpClient:
raw = resp.read().decode("utf-8")
return json.loads(raw)
# ------------------------------------------------------------------
# Streamable HTTP transport (MCP spec 2025-03-26)
# ------------------------------------------------------------------
def _init_streamable_http(self) -> bool:
url = self.config.get("url")
if not url:
logger.warning(f"[MCP:{self.name}] streamable-http config missing 'url'")
return False
self._http_url = url
# Allow user-provided headers (e.g. {"Authorization": "Bearer xxx"})
extra_headers = self.config.get("headers") or {}
if isinstance(extra_headers, dict):
self._http_headers = {str(k): str(v) for k, v in extra_headers.items()}
# Restore any previously stored OAuth credentials for this server so a
# restart reuses the token instead of forcing re-authorization.
self._maybe_load_oauth()
return self._handshake()
# ------------------------------------------------------------------
# OAuth helpers (streamable-http only)
# ------------------------------------------------------------------
def _has_static_auth(self) -> bool:
"""True when the user supplied their own Authorization header."""
return any(k.lower() == "authorization" for k in self._http_headers)
def _maybe_load_oauth(self) -> None:
"""Attach an OAuthHandler when stored credentials exist for this server."""
if self._has_static_auth():
return
try:
from agent.tools.mcp.mcp_oauth import OAuthHandler, load_server_record
except Exception:
return
rec = load_server_record(self.name)
# Only create a handler when we have something to reuse; otherwise it
# is created lazily on the first 401.
if rec.get("access_token") or rec.get("client_id"):
self._oauth = OAuthHandler(
server_name=self.name,
resource_url=self._http_url,
redirect_uri=_oauth_redirect_uri(),
scope=self.config.get("scope", ""),
)
def _current_bearer(self) -> Optional[str]:
"""Return a valid access token, refreshing if needed."""
if self._oauth is None:
return None
return self._oauth.get_valid_access_token()
def _begin_oauth(self, www_authenticate: str = "") -> None:
"""Kick off the OAuth flow after a 401: discover, register, prompt user."""
if self._has_static_auth():
return
try:
from agent.tools.mcp.mcp_oauth import OAuthHandler
except Exception as e:
logger.warning(f"[MCP:{self.name}] OAuth module unavailable: {e}")
return
if self._oauth is None:
self._oauth = OAuthHandler(
server_name=self.name,
resource_url=self._http_url,
redirect_uri=_oauth_redirect_uri(),
scope=self.config.get("scope", ""),
)
if not self._oauth.ensure_registered(www_authenticate):
logger.warning(
f"[MCP:{self.name}] OAuth discovery/registration failed; "
f"cannot authorize automatically"
)
return
auth_url = self._oauth.build_authorization_url()
if not auth_url:
logger.warning(f"[MCP:{self.name}] Failed to build authorization URL")
return
self.needs_auth = True
logger.warning(
f"[MCP:{self.name}] ⚠️ Authorization required. Open this URL in a "
f"browser to authorize, then this server will come online automatically:\n"
f" {auth_url}"
)
# On a machine with a local browser (desktop/dev), open it directly.
if os.environ.get("COW_DESKTOP") == "1" or not os.environ.get("COW_HEADLESS"):
try:
import webbrowser
webbrowser.open(auth_url)
except Exception:
pass
def _streamable_http_send(self, message: dict) -> dict:
"""POST a JSON-RPC request and return the response (JSON or SSE-wrapped)."""
return self._streamable_http_post(message, expect_response=True)
def _handle_401(self, err, message: dict, expect_response: bool, retried: bool) -> dict:
"""Handle a 401: refresh the token and retry once, else begin OAuth."""
www_auth = ""
try:
www_auth = err.headers.get("WWW-Authenticate", "") or ""
except Exception:
pass
try:
err.read()
except Exception:
pass
# First try a silent refresh with the stored refresh token.
if not retried and self._oauth is not None and self._oauth.refresh():
logger.info(f"[MCP:{self.name}] Token refreshed after 401, retrying")
return self._streamable_http_post(message, expect_response, _retried=True)
# No usable token — start (or restart) the interactive OAuth flow.
self._begin_oauth(www_auth)
raise IOError(
f"[MCP:{self.name}] streamable-http HTTP 401: authorization required "
f"(complete the OAuth flow to enable this server)"
)
def _streamable_http_post(self, message: dict, expect_response: bool, _retried: bool = False) -> dict:
"""
POST a JSON-RPC message over Streamable HTTP.
Per the spec, the response Content-Type can be either:
- application/json -> single JSON-RPC response in body
- text/event-stream -> SSE stream; we read until we get a matching response
"""
body = json.dumps(message).encode("utf-8")
headers = {
"Content-Type": "application/json",
"Accept": "application/json, text/event-stream",
}
# Read session id under lock to avoid racing with the
# initialization write below during concurrent requests.
with self._http_lock:
sid = self._http_session_id
if sid:
headers["Mcp-Session-Id"] = sid
headers.update(self._http_headers)
# Inject OAuth bearer token when we have one (unless the user set a
# static Authorization header, which takes precedence).
if not self._has_static_auth():
token = self._current_bearer()
if token:
headers["Authorization"] = f"Bearer {token}"
req = urllib.request.Request(
self._http_url,
data=body,
method="POST",
headers=headers,
)
try:
resp = urllib.request.urlopen(req, timeout=30)
except urllib.error.HTTPError as e:
# 401 is the spec-compliant "needs authorization" signal.
if e.code == 401 and not self._has_static_auth():
return self._handle_401(e, message, expect_response, _retried)
# Surface the server-provided error body for easier debugging
detail = ""
try:
detail = e.read().decode("utf-8", errors="ignore")
except Exception:
pass
raise IOError(
f"[MCP:{self.name}] streamable-http HTTP {e.code}: {detail[:200]}"
)
with resp:
# Capture session id assigned by the server (if any)
session_id = resp.headers.get("Mcp-Session-Id")
# Double-checked lock: only the first response sets the
# session id, preventing concurrent initializers from
# overwriting each other.
if session_id and not self._http_session_id:
with self._http_lock:
if not self._http_session_id:
self._http_session_id = session_id
status = resp.status if hasattr(resp, "status") else resp.getcode()
# Notifications: server may reply with 202 Accepted and no body
if not expect_response or status == 202:
try:
resp.read()
except Exception:
pass
return {}
content_type = (resp.headers.get("Content-Type") or "").lower()
expected_id = message.get("id")
if "text/event-stream" in content_type:
return self._read_sse_response(resp, expected_id)
raw = resp.read().decode("utf-8")
if not raw:
return {}
return json.loads(raw)
def _read_sse_response(self, resp, expected_id) -> dict:
"""Read an SSE stream and return the first JSON-RPC response with matching id."""
data_buf: list = []
for raw_line in resp:
line = raw_line.decode("utf-8").rstrip("\n\r")
if line == "":
# End of an SSE event, attempt to parse accumulated data
if data_buf:
payload = "\n".join(data_buf)
data_buf = []
try:
msg = json.loads(payload)
except json.JSONDecodeError:
continue
# Skip notifications / mismatched ids
if "id" not in msg:
continue
if expected_id is None or msg.get("id") == expected_id:
return msg
continue
if line.startswith(":"):
continue # SSE comment / keepalive
if line.startswith("data:"):
data_buf.append(line[len("data:"):].lstrip())
# Ignore 'event:' / 'id:' lines; we only care about JSON-RPC payloads
raise IOError(f"[MCP:{self.name}] streamable-http SSE stream closed before response")
# ------------------------------------------------------------------
# Common JSON-RPC helpers
# ------------------------------------------------------------------
@@ -262,13 +623,18 @@ class McpClient:
message = self._build_request(method, params)
with self._call_lock:
if self.transport == "stdio":
# stdio transport uses a single pipe and must be serialized.
# SSE and streamable-http use independent HTTP requests and
# can safely run concurrently across sessions.
if self.transport == "stdio":
with self._call_lock:
return self._stdio_send(message)
elif self.transport == "sse":
return self._sse_send(message)
else:
raise ValueError(f"[MCP:{self.name}] Unsupported transport: {self.transport}")
elif self.transport == "sse":
return self._sse_send(message)
elif self.transport == "streamable-http":
return self._streamable_http_send(message)
else:
raise ValueError(f"[MCP:{self.name}] Unsupported transport: {self.transport}")
def _send_notification(self, method: str, params: dict):
"""Fire-and-forget notification (no response expected)."""
@@ -291,6 +657,11 @@ class McpClient:
pass
except Exception:
pass # notifications are fire-and-forget
elif self.transport == "streamable-http":
try:
self._streamable_http_post(notification, expect_response=False)
except Exception:
pass # notifications are fire-and-forget
def _handshake(self) -> bool:
"""Perform the MCP initialize / notifications/initialized handshake."""

View File

@@ -0,0 +1,466 @@
"""
MCP OAuth 2.1 client (authorization code + PKCE) with zero external deps.
Implements the subset of the MCP authorization spec needed to connect to
remote MCP servers that guard their endpoint behind OAuth (e.g. Xmind):
1. Metadata discovery via RFC 9728 (protected-resource) + RFC 8414
(authorization-server) .well-known documents.
2. Dynamic Client Registration (RFC 7591) to obtain a client_id.
3. PKCE (RFC 7636, S256) authorization-code flow.
4. Token exchange + refresh, persisted to ~/.cow/mcp_oauth.json.
The actual browser round-trip is completed out-of-band: McpClient generates
an authorization URL, the user opens it, and the web console callback
(/mcp/oauth/callback) feeds the returned code back into finish_authorization().
"""
import base64
import hashlib
import json
import os
import secrets
import threading
import time
import urllib.parse
import urllib.request
import urllib.error
from typing import Optional
from common.log import logger
# ------------------------------------------------------------------
# Token store: ~/.cow/mcp_oauth.json {server_name: {...credentials...}}
# ------------------------------------------------------------------
_STORE_LOCK = threading.Lock()
def _store_path() -> str:
base = os.path.expanduser("~/.cow")
try:
os.makedirs(base, exist_ok=True)
except OSError:
pass
return os.path.join(base, "mcp_oauth.json")
def _load_store() -> dict:
path = _store_path()
if not os.path.exists(path):
return {}
try:
with open(path, "r", encoding="utf-8") as f:
data = json.load(f)
return data if isinstance(data, dict) else {}
except Exception as e:
logger.warning(f"[MCP-OAuth] Failed to read token store: {e}")
return {}
def _save_store(store: dict) -> None:
path = _store_path()
tmp = f"{path}.tmp"
try:
with open(tmp, "w", encoding="utf-8") as f:
json.dump(store, f, ensure_ascii=False, indent=2)
os.replace(tmp, path)
# Credentials file: restrict to owner read/write when possible.
try:
os.chmod(path, 0o600)
except OSError:
pass
except Exception as e:
logger.warning(f"[MCP-OAuth] Failed to persist token store: {e}")
def load_server_record(server_name: str) -> dict:
with _STORE_LOCK:
return dict(_load_store().get(server_name, {}))
def save_server_record(server_name: str, record: dict) -> None:
with _STORE_LOCK:
store = _load_store()
store[server_name] = record
_save_store(store)
def clear_server_record(server_name: str) -> None:
with _STORE_LOCK:
store = _load_store()
if server_name in store:
store.pop(server_name, None)
_save_store(store)
# ------------------------------------------------------------------
# Pending authorizations, keyed by the OAuth `state` param.
# Populated when an authorization URL is generated; consumed by the
# web callback when the browser redirects back with ?code&state.
# ------------------------------------------------------------------
_PENDING_LOCK = threading.Lock()
_PENDING: dict = {} # state -> {"handler": OAuthHandler, "created": ts}
_PENDING_TTL = 600 # seconds
def _register_pending(state: str, handler: "OAuthHandler") -> None:
with _PENDING_LOCK:
_prune_pending_locked()
_PENDING[state] = {"handler": handler, "created": time.time()}
def _prune_pending_locked() -> None:
now = time.time()
stale = [s for s, v in _PENDING.items() if now - v["created"] > _PENDING_TTL]
for s in stale:
_PENDING.pop(s, None)
def pop_pending(state: str) -> Optional["OAuthHandler"]:
with _PENDING_LOCK:
_prune_pending_locked()
entry = _PENDING.pop(state, None)
return entry["handler"] if entry else None
def has_pending() -> bool:
with _PENDING_LOCK:
_prune_pending_locked()
return bool(_PENDING)
# ------------------------------------------------------------------
# HTTP helpers (stdlib only)
# ------------------------------------------------------------------
_UA = "CowAgent-MCP-OAuth/1.0"
def _http_get_json(url: str, timeout: int = 15) -> Optional[dict]:
req = urllib.request.Request(url, headers={"Accept": "application/json", "User-Agent": _UA})
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8")
return json.loads(raw)
except urllib.error.HTTPError as e:
logger.debug(f"[MCP-OAuth] GET {url} -> HTTP {e.code}")
return None
except Exception as e:
logger.debug(f"[MCP-OAuth] GET {url} failed: {e}")
return None
def _http_post_form(url: str, fields: dict, timeout: int = 20) -> dict:
body = urllib.parse.urlencode(fields).encode("utf-8")
req = urllib.request.Request(
url,
data=body,
method="POST",
headers={
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
"User-Agent": _UA,
},
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8")
return json.loads(raw) if raw else {}
def _http_post_json(url: str, payload: dict, timeout: int = 20) -> dict:
body = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
url,
data=body,
method="POST",
headers={
"Content-Type": "application/json",
"Accept": "application/json",
"User-Agent": _UA,
},
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
raw = resp.read().decode("utf-8")
return json.loads(raw) if raw else {}
# ------------------------------------------------------------------
# Discovery (RFC 9728 + RFC 8414)
# ------------------------------------------------------------------
def _origin(url: str) -> str:
p = urllib.parse.urlparse(url)
return f"{p.scheme}://{p.netloc}"
def discover_metadata(resource_url: str, www_authenticate: str = "") -> Optional[dict]:
"""
Resolve the authorization server metadata for a protected MCP resource.
Returns a dict with at least authorization_endpoint + token_endpoint,
plus registration_endpoint when the server supports DCR. Returns None
when discovery fails.
"""
as_metadata_url = _parse_resource_metadata_url(www_authenticate)
# 1) Protected-resource metadata (RFC 9728) to locate the auth server.
auth_server = None
prm = None
if as_metadata_url:
prm = _http_get_json(as_metadata_url)
if prm is None:
origin = _origin(resource_url)
prm = _http_get_json(f"{origin}/.well-known/oauth-protected-resource")
if prm and isinstance(prm.get("authorization_servers"), list) and prm["authorization_servers"]:
auth_server = prm["authorization_servers"][0]
# 2) Authorization-server metadata (RFC 8414). Fall back to the resource
# origin when the resource did not advertise a separate auth server.
base = auth_server or _origin(resource_url)
asm = _fetch_as_metadata(base)
if not asm:
return None
if not asm.get("authorization_endpoint") or not asm.get("token_endpoint"):
logger.warning("[MCP-OAuth] Authorization server metadata missing required endpoints")
return None
# Derive the scope to request. Prefer the resource's required_scopes
# (RFC 9728), then its scopes_supported, then the auth server's
# scopes_supported. Stored so callers don't have to configure it.
discovered_scope = ""
if prm:
scopes = prm.get("required_scopes") or prm.get("scopes_supported")
if isinstance(scopes, list) and scopes:
discovered_scope = " ".join(str(s) for s in scopes)
if not discovered_scope and isinstance(asm.get("scopes_supported"), list) and asm["scopes_supported"]:
discovered_scope = " ".join(str(s) for s in asm["scopes_supported"])
if discovered_scope:
asm["_discovered_scope"] = discovered_scope
return asm
def _parse_resource_metadata_url(www_authenticate: str) -> Optional[str]:
"""Extract resource_metadata="..." from a WWW-Authenticate: Bearer header."""
if not www_authenticate:
return None
# naive but sufficient parse for `resource_metadata="URL"`
marker = "resource_metadata="
idx = www_authenticate.find(marker)
if idx < 0:
return None
rest = www_authenticate[idx + len(marker):].strip()
if rest.startswith('"'):
end = rest.find('"', 1)
return rest[1:end] if end > 0 else None
# unquoted, up to comma/space
for sep in (",", " "):
if sep in rest:
rest = rest.split(sep, 1)[0]
return rest or None
def _fetch_as_metadata(base: str) -> Optional[dict]:
"""Try both RFC 8414 and OIDC well-known locations."""
base = base.rstrip("/")
candidates = [
f"{base}/.well-known/oauth-authorization-server",
f"{base}/.well-known/openid-configuration",
]
for url in candidates:
data = _http_get_json(url)
if data and data.get("authorization_endpoint"):
return data
return None
# ------------------------------------------------------------------
# PKCE
# ------------------------------------------------------------------
def _b64url(data: bytes) -> str:
return base64.urlsafe_b64encode(data).decode("ascii").rstrip("=")
def _make_pkce() -> tuple:
verifier = _b64url(secrets.token_bytes(32))
challenge = _b64url(hashlib.sha256(verifier.encode("ascii")).digest())
return verifier, challenge
# ------------------------------------------------------------------
# OAuthHandler: per-server OAuth state machine
# ------------------------------------------------------------------
class OAuthHandler:
"""Drives the OAuth flow and token lifecycle for a single MCP server."""
def __init__(self, server_name: str, resource_url: str, redirect_uri: str,
scope: str = "", client_name: str = "CowAgent"):
self.server_name = server_name
self.resource_url = resource_url
self.redirect_uri = redirect_uri
self.scope = scope
self.client_name = client_name
rec = load_server_record(server_name)
self.metadata: dict = rec.get("metadata", {})
self.client_id: Optional[str] = rec.get("client_id")
self.client_secret: Optional[str] = rec.get("client_secret")
self.access_token: Optional[str] = rec.get("access_token")
self.refresh_token: Optional[str] = rec.get("refresh_token")
self.expires_at: float = float(rec.get("expires_at", 0) or 0)
self._verifier: Optional[str] = None
# --- persistence -------------------------------------------------
def _persist(self) -> None:
save_server_record(self.server_name, {
"resource_url": self.resource_url,
"metadata": self.metadata,
"client_id": self.client_id,
"client_secret": self.client_secret,
"access_token": self.access_token,
"refresh_token": self.refresh_token,
"expires_at": self.expires_at,
})
# --- token access ------------------------------------------------
def get_valid_access_token(self, leeway: int = 60) -> Optional[str]:
"""Return a usable access token, refreshing proactively when near expiry."""
if not self.access_token:
return None
if self.expires_at and time.time() >= self.expires_at - leeway:
if not self.refresh():
return None
return self.access_token
def refresh(self) -> bool:
"""Refresh the access token using the stored refresh token."""
if not self.refresh_token or not self.metadata.get("token_endpoint"):
return False
fields = {
"grant_type": "refresh_token",
"refresh_token": self.refresh_token,
"client_id": self.client_id or "",
}
if self.client_secret:
fields["client_secret"] = self.client_secret
try:
resp = _http_post_form(self.metadata["token_endpoint"], fields)
except Exception as e:
logger.warning(f"[MCP-OAuth:{self.server_name}] refresh failed: {e}")
return False
return self._absorb_token_response(resp)
# --- authorization-code flow ------------------------------------
def ensure_registered(self, www_authenticate: str = "") -> bool:
"""Discover metadata + register a client if not already done."""
if not self.metadata.get("authorization_endpoint"):
meta = discover_metadata(self.resource_url, www_authenticate)
if not meta:
return False
self.metadata = meta
# Adopt the scope discovered from metadata when the user didn't set one.
if not self.scope and self.metadata.get("_discovered_scope"):
self.scope = self.metadata["_discovered_scope"]
logger.info(f"[MCP-OAuth:{self.server_name}] Using discovered scope: {self.scope}")
if not self.client_id:
if not self._register_client():
return False
self._persist()
return True
def _register_client(self) -> bool:
reg_endpoint = self.metadata.get("registration_endpoint")
if not reg_endpoint:
logger.warning(
f"[MCP-OAuth:{self.server_name}] No registration_endpoint; "
f"DCR unavailable. Provide client_id manually."
)
return False
payload = {
"client_name": self.client_name,
"redirect_uris": [self.redirect_uri],
"grant_types": ["authorization_code", "refresh_token"],
"response_types": ["code"],
"token_endpoint_auth_method": "none",
}
if self.scope:
payload["scope"] = self.scope
try:
resp = _http_post_json(reg_endpoint, payload)
except Exception as e:
logger.warning(f"[MCP-OAuth:{self.server_name}] DCR failed: {e}")
return False
client_id = resp.get("client_id")
if not client_id:
logger.warning(f"[MCP-OAuth:{self.server_name}] DCR returned no client_id")
return False
self.client_id = client_id
self.client_secret = resp.get("client_secret")
logger.info(f"[MCP-OAuth:{self.server_name}] Registered client_id={client_id}")
return True
def build_authorization_url(self) -> Optional[str]:
"""Create an authorization URL and register this handler as pending."""
if not self.metadata.get("authorization_endpoint") or not self.client_id:
return None
self._verifier, challenge = _make_pkce()
state = secrets.token_urlsafe(24)
params = {
"response_type": "code",
"client_id": self.client_id,
"redirect_uri": self.redirect_uri,
"code_challenge": challenge,
"code_challenge_method": "S256",
"state": state,
}
if self.scope:
params["scope"] = self.scope
# Advertise the resource we intend to access (RFC 8707).
params["resource"] = self.resource_url
_register_pending(state, self)
return f"{self.metadata['authorization_endpoint']}?{urllib.parse.urlencode(params)}"
def finish_authorization(self, code: str) -> bool:
"""Exchange an authorization code for tokens."""
if not self.metadata.get("token_endpoint") or not self._verifier:
return False
fields = {
"grant_type": "authorization_code",
"code": code,
"redirect_uri": self.redirect_uri,
"client_id": self.client_id or "",
"code_verifier": self._verifier,
"resource": self.resource_url,
}
if self.client_secret:
fields["client_secret"] = self.client_secret
try:
resp = _http_post_form(self.metadata["token_endpoint"], fields)
except Exception as e:
logger.warning(f"[MCP-OAuth:{self.server_name}] token exchange failed: {e}")
return False
ok = self._absorb_token_response(resp)
self._verifier = None
return ok
def _absorb_token_response(self, resp: dict) -> bool:
access = resp.get("access_token")
if not access:
logger.warning(f"[MCP-OAuth:{self.server_name}] token response missing access_token: {resp}")
return False
self.access_token = access
if resp.get("refresh_token"):
self.refresh_token = resp["refresh_token"]
expires_in = resp.get("expires_in")
self.expires_at = time.time() + int(expires_in) if expires_in else 0
self._persist()
logger.info(f"[MCP-OAuth:{self.server_name}] Access token stored")
return True

View File

@@ -0,0 +1,159 @@
# encoding:utf-8
"""
On-demand MCP tool retrieval.
Pure, stateless selection helpers used by the streaming executor to decide
which MCP tools to inject into a given LLM turn. Vector precompute + caching
live in ToolManager (the tool-lifecycle owner, a process-wide singleton);
only the context-aware selection lives here, because only the executor knows
the conversation context.
Invariants (per maintainer review of the feature proposal):
* Built-in tools are never handled here — the caller injects them in full.
* Any failure / missing input returns None so the caller falls back to
full injection; tools must never be silently dropped.
* Selection is union-accumulated across turns by the caller (only-grows),
so a tool that already produced a tool_use in the message history can
never disappear from the schema mid-run (which would make Claude/MiniMax
raise a message-format error).
"""
import math
from typing import Dict, List, Optional, Sequence, Set
try:
import numpy as np
_HAS_NUMPY = True
except ImportError:
_HAS_NUMPY = False
# How many trailing messages to concatenate into the retrieval query. Tool
# needs drift across a multi-turn tool-call loop, so a single (initial) user
# query is not enough; a short recent window captures the drift without
# bloating the query with stale context.
DEFAULT_QUERY_MESSAGES = 5
def build_retrieval_query(messages: list, max_messages: int = DEFAULT_QUERY_MESSAGES) -> str:
"""Concatenate the text of the most recent messages into a retrieval query.
Only ``text`` content blocks are kept; ``tool_use`` / ``tool_result`` blocks
are skipped so the query stays short and focused on natural-language intent
rather than large serialized tool payloads.
Args:
messages: Claude-style message list, each ``{"role", "content"}`` where
content is either a string or a list of typed blocks.
max_messages: Size of the trailing window to consider.
Returns:
A single string (possibly empty if no text is found).
"""
if not messages:
return ""
parts: List[str] = []
for message in messages[-max_messages:]:
content = message.get("content") if isinstance(message, dict) else None
if isinstance(content, str):
if content.strip():
parts.append(content.strip())
continue
if isinstance(content, list):
for block in content:
if not isinstance(block, dict):
continue
if block.get("type") == "text":
text = block.get("text", "")
if isinstance(text, str) and text.strip():
parts.append(text.strip())
return "\n".join(parts)
def cosine_similarity(a: Sequence[float], b: Sequence[float]) -> float:
"""Cosine similarity of two equal-length vectors; 0.0 on degenerate input."""
if not a or not b or len(a) != len(b):
return 0.0
dot = sum(x * y for x, y in zip(a, b))
norm_a = math.sqrt(sum(x * x for x in a))
norm_b = math.sqrt(sum(y * y for y in b))
if norm_a == 0 or norm_b == 0:
return 0.0
return dot / (norm_a * norm_b)
def select_mcp_tools(
query_vector: Optional[Sequence[float]],
tool_vectors: Dict[str, Sequence[float]],
top_k: int,
already_selected: Optional[Set[str]] = None,
) -> Optional[Set[str]]:
"""Return the accumulated set of MCP tool names to inject this turn.
Computes cosine similarity between ``query_vector`` and each candidate
tool vector, keeps the ``top_k`` best, and unions them with
``already_selected`` so the injected set only ever grows within a run.
Args:
query_vector: Embedding of the current retrieval query, or None.
tool_vectors: ``{mcp_tool_name: vector}`` for candidate MCP tools.
top_k: Max number of tools to add from this turn's ranking.
already_selected: Names accumulated in previous turns of this run.
Returns:
The union set of tool names to inject, or None to signal
"fall back to full injection" (no query vector, empty/invalid index,
or any unexpected error). This function never raises.
"""
accumulated: Set[str] = set(already_selected) if already_selected else set()
if not query_vector or not tool_vectors or top_k <= 0:
return None
try:
expected_dim = len(query_vector)
# Only rank candidates whose vector dimensionality matches the query.
# A dimension mismatch means the index was built with a different
# embedding model; ranking across dims is meaningless.
candidates = {
name: vec
for name, vec in tool_vectors.items()
if vec and len(vec) == expected_dim
}
if not candidates:
return None
ranked = _rank_by_similarity(query_vector, candidates)
for name, _score in ranked[:top_k]:
accumulated.add(name)
return accumulated
except Exception:
# Selection must never break the agent — fall back to full injection.
return None
def _rank_by_similarity(
query_vector: Sequence[float],
candidates: Dict[str, Sequence[float]],
) -> List[tuple]:
"""Return ``[(name, score), ...]`` sorted by descending cosine similarity.
Uses numpy when available (vectorized, matching the memory-search path),
with a pure-Python fallback so the feature works without numpy installed.
"""
names = list(candidates.keys())
if _HAS_NUMPY:
matrix = np.array([candidates[n] for n in names], dtype=np.float32) # (N, D)
q_vec = np.array(query_vector, dtype=np.float32) # (D,)
dots = matrix @ q_vec # (N,)
row_norms = np.linalg.norm(matrix, axis=1) # (N,)
q_norm = float(np.linalg.norm(q_vec))
denominators = row_norms * q_norm
np.maximum(denominators, 1e-10, out=denominators) # avoid div-by-zero
sims = dots / denominators
order = np.argsort(sims)[::-1]
return [(names[i], float(sims[i])) for i in order]
scored = [(n, cosine_similarity(query_vector, candidates[n])) for n in names]
scored.sort(key=lambda x: x[1], reverse=True)
return scored

View File

@@ -4,6 +4,8 @@ Memory get tool
Allows agents to read specific sections from memory files
"""
import os
from agent.tools.base_tool import BaseTool
@@ -87,8 +89,13 @@ class MemoryGetTool(BaseTool):
file_path = (workspace_dir / path).resolve()
workspace_resolved = workspace_dir.resolve()
if not str(file_path).startswith(str(workspace_resolved) + '/') and file_path != workspace_resolved:
# Use os.path.realpath + os.sep for cross-platform path validation.
# str(Path).startswith(str + '/') fails on Windows where Path uses
# backslashes — see MemoryService._resolve_path for the same pattern.
real_file = os.path.realpath(str(file_path))
real_workspace = os.path.realpath(str(workspace_resolved))
if real_file != real_workspace and not real_file.startswith(real_workspace + os.sep):
return ToolResult.fail(f"Error: Access denied: path outside workspace")
if not file_path.exists():

View File

@@ -4,6 +4,7 @@ Supports text files, images (jpg, png, gif, webp), and PDF files
"""
import os
import re
from typing import Dict, Any
from pathlib import Path
@@ -12,11 +13,17 @@ from agent.tools.utils.truncate import truncate_head, format_size, DEFAULT_MAX_L
from common.utils import expand_path
# Paths whose CONTENT mirrors the process environment (and thus any secrets
# loaded from ~/.cow/.env). Reading them bypasses the env_config boundary.
# Matches /proc/self/environ, /proc/thread-self/environ and /proc/<pid>/environ.
_PROC_ENVIRON_RE = re.compile(r"^/proc/(\d+|self|thread-self)/environ$")
class Read(BaseTool):
"""Tool for reading file contents"""
name: str = "read"
description: str = f"Read or inspect file contents. For text/PDF files, returns content (truncated to {DEFAULT_MAX_LINES} lines or {DEFAULT_MAX_BYTES // 1024}KB). For images/videos/audio, returns metadata only (file info, size, type). Use offset/limit for large text files."
description: str = f"Read or inspect file contents. For text/PDF/Word/Excel/PPT files, returns content (truncated to {DEFAULT_MAX_LINES} lines or {DEFAULT_MAX_BYTES // 1024}KB). For images/videos/audio, returns metadata only (file info, size, type). Use offset/limit for large text files."
params: dict = {
"type": "object",
@@ -79,9 +86,9 @@ class Read(BaseTool):
# Resolve path
absolute_path = self._resolve_path(path)
# Security check: Prevent reading sensitive config files
env_config_path = expand_path("~/.cow/.env")
if os.path.abspath(absolute_path) == os.path.abspath(env_config_path):
# Security check: block credential files and their aliases.
# See issue #2913 (/proc/self/environ bypass) and #2863 (scope).
if self._is_credential_path(absolute_path):
return ToolResult.fail(
"Error: Access denied. API keys and credentials must be accessed through the env_config tool only."
)
@@ -140,7 +147,39 @@ class Read(BaseTool):
if os.path.isabs(path):
return path
return os.path.abspath(os.path.join(self.cwd, path))
def _is_credential_path(self, absolute_path: str) -> bool:
"""Return True if *absolute_path* points at protected credential data.
Beyond the literal ~/.cow/.env file, this also blocks two real bypass
surfaces reported in issue #2913:
1. /proc/<pid|self|thread-self>/environ — a second view of the
process environment that leaks secrets loaded from ~/.cow/.env.
2. Symlinks resolving to ~/.cow/.env; the previous exact abspath
match kept the link target and could be bypassed.
Scope is kept deliberately narrow (only the credential file and its
environ aliases) so this does NOT re-broaden the block that #2863
intentionally narrowed to ~/.cow/.env.
"""
# Compare on both the normalized path and the symlink-resolved path,
# in POSIX form so the /proc regex matches regardless of os.sep.
candidates = set()
try:
candidates.add(os.path.normpath(absolute_path).replace(os.sep, "/"))
candidates.add(os.path.realpath(absolute_path).replace(os.sep, "/"))
except OSError:
candidates.add(absolute_path.replace(os.sep, "/"))
# 1. /proc environ aliases (checked on raw and symlink-resolved forms).
for candidate in candidates:
if _PROC_ENVIRON_RE.match(candidate):
return True
# 2. The credential file itself, following symlinks on both sides.
env_real = os.path.realpath(expand_path("~/.cow/.env")).replace(os.sep, "/")
return env_real in candidates
def _return_file_metadata(self, absolute_path: str, file_type: str, file_size: int) -> ToolResult:
"""
Return file metadata for non-readable files (video, audio, binary, etc.)
@@ -258,8 +297,15 @@ class Read(BaseTool):
if offset is not None:
if offset < 0:
# Negative offset: read from end
# -20 means "last 20 lines" → start from (total - 20)
start_line = max(0, total_file_lines + offset)
# -20 means "last 20 lines" → start from (total - 20).
# A file ending in "\n" produces a trailing empty element
# from split('\n'); exclude it so offset=-1 returns the
# real last line instead of the empty string after the
# final newline (and -N returns N real lines).
effective_lines = total_file_lines
if all_lines and all_lines[-1] == '':
effective_lines -= 1
start_line = max(0, effective_lines + offset)
else:
# Positive offset: read from start (1-indexed)
start_line = max(0, offset - 1) # Convert to 0-indexed

View File

@@ -57,34 +57,44 @@ def init_scheduler(agent_bridge) -> bool:
_task_store = TaskStore(store_path)
logger.debug(f"[Scheduler] Task store initialized: {store_path}")
# Create execute callback
# Create execute callback. Returns True on success, False to ask
# the scheduler to retry on the next tick (e.g. channel not yet
# ready right after process start).
def execute_task_callback(task: dict):
"""Callback to execute a scheduled task"""
try:
action = task.get("action", {})
action_type = action.get("type")
channel_type = action.get("channel_type", "unknown")
receiver = action.get("receiver", "")
if not _is_channel_ready(channel_type, receiver):
logger.warning(
f"[Scheduler] Task {task.get('id')}: channel "
f"'{channel_type}' not ready for receiver={receiver} "
f"(no inbound msg cached since restart?); deferring"
)
return False
if action_type == "agent_task":
_execute_agent_task(task, agent_bridge)
return _execute_agent_task(task, agent_bridge)
elif action_type == "send_message":
# Legacy support for old tasks
_execute_send_message(task, agent_bridge)
return _execute_send_message(task, agent_bridge)
elif action_type == "tool_call":
# Legacy support for old tasks
_execute_tool_call(task, agent_bridge)
return _execute_tool_call(task, agent_bridge)
elif action_type == "skill_call":
# Legacy support for old tasks
_execute_skill_call(task, agent_bridge)
return _execute_skill_call(task, agent_bridge)
else:
logger.warning(f"[Scheduler] Unknown action type: {action_type}")
return True
except Exception as e:
logger.error(f"[Scheduler] Error executing task {task.get('id')}: {e}")
return False
# Create scheduler service
_scheduler_service = SchedulerService(_task_store, execute_task_callback)
_scheduler_service.start()
logger.debug("[Scheduler] Scheduler service initialized and started")
logger.info("[Scheduler] Service initialized and started")
return True
except Exception as e:
@@ -92,6 +102,40 @@ def init_scheduler(agent_bridge) -> bool:
return False
def _is_channel_ready(channel_type: str, receiver: str) -> bool:
"""Best-effort readiness probe for outbound channels.
Returns False when we know the send will drop (e.g. weixin not yet
logged in, web session has no polling queue), so the scheduler can
defer instead of consuming the task. Unknown channels return True
to preserve previous behaviour.
"""
if not channel_type or channel_type == "unknown":
return True
try:
from channel.channel_factory import create_channel
channel = create_channel(channel_type)
if channel is None:
return False
if channel_type == "weixin":
tokens = getattr(channel, "_context_tokens", None)
if not tokens or receiver not in tokens:
return False
return True
if channel_type == "web":
queues = getattr(channel, "session_queues", None)
if not queues or receiver not in queues:
return False
return True
return True
except Exception as e:
logger.warning(f"[Scheduler] Channel readiness check failed for {channel_type}: {e}")
return True
def get_task_store():
"""Get the global task store instance"""
return _task_store
@@ -145,13 +189,10 @@ def _remember_delivered_output(
)
def _execute_agent_task(task: dict, agent_bridge):
def _execute_agent_task(task: dict, agent_bridge) -> bool:
"""
Execute an agent_task action - let Agent handle the task
Args:
task: Task dictionary
agent_bridge: AgentBridge instance
Execute an agent_task action - let Agent handle the task.
Returns True on successful delivery, False to retry next tick.
"""
try:
action = task.get("action", {})
@@ -162,11 +203,11 @@ def _execute_agent_task(task: dict, agent_bridge):
if not task_description:
logger.error(f"[Scheduler] Task {task['id']}: No task_description specified")
return
return True # malformed task, don't loop forever
if not receiver:
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
return
return True
# Check for unsupported channels
if channel_type == "dingtalk":
@@ -209,51 +250,47 @@ def _execute_agent_task(task: dict, agent_bridge):
try:
# Don't clear history - scheduler tasks use isolated session_id so they won't pollute user conversations
reply = agent_bridge.agent_reply(task_description, context=context, on_event=None, clear_history=False)
if reply and reply.content:
# Send the reply via channel
from channel.channel_factory import create_channel
try:
channel = create_channel(channel_type)
if channel:
# For web channel, register request_id
if channel_type == "web" and hasattr(channel, 'request_to_session'):
request_id = context.get("request_id")
if request_id:
channel.request_to_session[request_id] = receiver
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
# Send the reply
channel.send(reply, context)
_remember_delivered_output(agent_bridge, task, channel_type, reply.content)
logger.info(f"[Scheduler] Task {task['id']} executed successfully, result sent to {receiver}")
else:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
except Exception as e:
logger.error(f"[Scheduler] Failed to send result: {e}")
else:
if not (reply and reply.content):
logger.error(f"[Scheduler] Task {task['id']}: No result from agent execution")
return True # agent ran but produced nothing; don't loop
from channel.channel_factory import create_channel
channel = create_channel(channel_type)
if not channel:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
return False
if channel_type == "web" and hasattr(channel, 'request_to_session'):
request_id = context.get("request_id")
if request_id:
channel.request_to_session[request_id] = receiver
try:
channel.send(reply, context)
except Exception as e:
logger.error(f"[Scheduler] Failed to send result: {e}")
return False
_remember_delivered_output(agent_bridge, task, channel_type, reply.content)
logger.info(f"[Scheduler] Task {task['id']} executed successfully, result sent to {receiver}")
return True
except Exception as e:
logger.error(f"[Scheduler] Failed to execute task via Agent: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
except Exception as e:
logger.error(f"[Scheduler] Error in _execute_agent_task: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
def _execute_send_message(task: dict, agent_bridge):
"""
Execute a send_message action
Args:
task: Task dictionary
agent_bridge: AgentBridge instance
"""
def _execute_send_message(task: dict, agent_bridge) -> bool:
"""Execute a send_message action. Returns True/False for delivery."""
try:
action = task.get("action", {})
content = action.get("content", "")
@@ -263,7 +300,7 @@ def _execute_send_message(task: dict, agent_bridge):
if not receiver:
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
return
return True
# Create context for sending message
context = Context(ContextType.TEXT, content)
@@ -308,169 +345,135 @@ def _execute_send_message(task: dict, agent_bridge):
# Get channel and send
from channel.channel_factory import create_channel
channel = create_channel(channel_type)
if not channel:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
return False
if channel_type == "web" and hasattr(channel, 'request_to_session'):
channel.request_to_session[request_id] = receiver
try:
channel = create_channel(channel_type)
if channel:
# For web channel, register the request_id to session mapping
if channel_type == "web" and hasattr(channel, 'request_to_session'):
channel.request_to_session[request_id] = receiver
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
channel.send(reply, context)
_remember_delivered_output(agent_bridge, task, channel_type, content)
logger.info(f"[Scheduler] Task {task['id']} executed: sent message to {receiver}")
else:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
channel.send(reply, context)
except Exception as e:
logger.error(f"[Scheduler] Failed to send message: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
_remember_delivered_output(agent_bridge, task, channel_type, content)
logger.info(f"[Scheduler] Task {task['id']} executed: sent message to {receiver}")
return True
except Exception as e:
logger.error(f"[Scheduler] Error in _execute_send_message: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
def _execute_tool_call(task: dict, agent_bridge):
"""
Execute a tool_call action
Args:
task: Task dictionary
agent_bridge: AgentBridge instance
"""
def _execute_tool_call(task: dict, agent_bridge) -> bool:
"""Execute a tool_call action. Returns True/False for delivery."""
try:
action = task.get("action", {})
# Support both old and new field names
tool_name = action.get("call_name") or action.get("tool_name")
tool_params = action.get("call_params") or action.get("tool_params", {})
result_prefix = action.get("result_prefix", "")
receiver = action.get("receiver")
is_group = action.get("is_group", False)
channel_type = action.get("channel_type", "unknown")
if not tool_name:
logger.error(f"[Scheduler] Task {task['id']}: No tool_name specified")
return
return True
if not receiver:
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
return
# Get tool manager and create tool instance
return True
from agent.tools.tool_manager import ToolManager
tool_manager = ToolManager()
tool = tool_manager.create_tool(tool_name)
tool = ToolManager().create_tool(tool_name)
if not tool:
logger.error(f"[Scheduler] Task {task['id']}: Tool '{tool_name}' not found")
return
# Execute tool
return True
logger.info(f"[Scheduler] Task {task['id']}: Executing tool '{tool_name}' with params {tool_params}")
result = tool.execute(tool_params)
# Get result content
if hasattr(result, 'result'):
content = result.result
else:
content = str(result)
# Add prefix if specified
content = result.result if hasattr(result, 'result') else str(result)
if result_prefix:
content = f"{result_prefix}\n\n{content}"
# Send result as message
context = Context(ContextType.TEXT, content)
context["receiver"] = receiver
context["isgroup"] = is_group
context["session_id"] = receiver
# Channel-specific context setup
request_id = None
if channel_type == "web":
# Web channel needs request_id
import uuid
request_id = f"scheduler_{task['id']}_{uuid.uuid4().hex[:8]}"
context["request_id"] = request_id
logger.debug(f"[Scheduler] Generated request_id for web channel: {request_id}")
elif channel_type == "feishu":
context["receive_id_type"] = "chat_id" if is_group else "open_id"
context["msg"] = None
logger.debug(f"[Scheduler] Feishu: receive_id_type={context['receive_id_type']}, is_group={is_group}, receiver={receiver}")
elif channel_type == "wecom_bot":
context["msg"] = None
reply = Reply(ReplyType.TEXT, content)
# Get channel and send
from channel.channel_factory import create_channel
channel = create_channel(channel_type)
if not channel:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
return False
if channel_type == "web" and request_id and hasattr(channel, 'request_to_session'):
channel.request_to_session[request_id] = receiver
try:
channel = create_channel(channel_type)
if channel:
if channel_type == "web" and hasattr(channel, 'request_to_session'):
channel.request_to_session[request_id] = receiver
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
channel.send(reply, context)
_remember_delivered_output(agent_bridge, task, channel_type, content)
logger.info(f"[Scheduler] Task {task['id']} executed: sent tool result to {receiver}")
else:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
channel.send(reply, context)
except Exception as e:
logger.error(f"[Scheduler] Failed to send tool result: {e}")
return False
_remember_delivered_output(agent_bridge, task, channel_type, content)
logger.info(f"[Scheduler] Task {task['id']} executed: sent tool result to {receiver}")
return True
except Exception as e:
logger.error(f"[Scheduler] Error in _execute_tool_call: {e}")
return False
def _execute_skill_call(task: dict, agent_bridge):
"""
Execute a skill_call action by asking Agent to run the skill
Args:
task: Task dictionary
agent_bridge: AgentBridge instance
"""
def _execute_skill_call(task: dict, agent_bridge) -> bool:
"""Execute a skill_call action by asking Agent to run the skill.
Returns True/False for delivery."""
try:
action = task.get("action", {})
# Support both old and new field names
skill_name = action.get("call_name") or action.get("skill_name")
skill_params = action.get("call_params") or action.get("skill_params", {})
result_prefix = action.get("result_prefix", "")
receiver = action.get("receiver")
is_group = action.get("isgroup", False)
channel_type = action.get("channel_type", "unknown")
if not skill_name:
logger.error(f"[Scheduler] Task {task['id']}: No skill_name specified")
return
return True
if not receiver:
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
return
return True
logger.info(f"[Scheduler] Task {task['id']}: Executing skill '{skill_name}' with params {skill_params}")
# Create a unique session_id for this scheduled task to avoid polluting user's conversation
# Format: scheduler_<receiver>_<task_id> to ensure isolation
scheduler_session_id = f"scheduler_{receiver}_{task['id']}"
# Build a natural language query for the Agent to execute the skill
# Format: "Use skill-name to do something with params"
param_str = ", ".join([f"{k}={v}" for k, v in skill_params.items()])
query = f"Use {skill_name} skill"
if param_str:
query += f" with {param_str}"
# Create context for Agent
context = Context(ContextType.TEXT, query)
context["receiver"] = receiver
context["isgroup"] = is_group
context["session_id"] = scheduler_session_id
# Channel-specific setup
if channel_type == "web":
import uuid
request_id = f"scheduler_{task['id']}_{uuid.uuid4().hex[:8]}"
@@ -481,49 +484,48 @@ def _execute_skill_call(task: dict, agent_bridge):
elif channel_type == "wecom_bot":
context["msg"] = None
# Use Agent to execute the skill
try:
# Don't clear history - scheduler tasks use isolated session_id so they won't pollute user conversations
reply = agent_bridge.agent_reply(query, context=context, on_event=None, clear_history=False)
if reply and reply.content:
content = reply.content
# Add prefix if specified
if result_prefix:
content = f"{result_prefix}\n\n{content}"
# Send the result via channel
from channel.channel_factory import create_channel
try:
channel = create_channel(channel_type)
if channel:
# For web channel, register request_id
if channel_type == "web" and hasattr(channel, 'request_to_session'):
req_id = context.get("request_id")
if req_id:
channel.request_to_session[req_id] = receiver
logger.debug(f"[Scheduler] Registered request_id {req_id} -> session {receiver}")
channel.send(Reply(ReplyType.TEXT, content), context)
_remember_delivered_output(agent_bridge, task, channel_type, content)
except Exception as e:
logger.error(f"[Scheduler] Failed to send skill result: {e}")
logger.info(f"[Scheduler] Task {task['id']} executed: skill result sent to {receiver}")
else:
logger.error(f"[Scheduler] Task {task['id']}: No result from skill execution")
except Exception as e:
logger.error(f"[Scheduler] Failed to execute skill via Agent: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
if not (reply and reply.content):
logger.error(f"[Scheduler] Task {task['id']}: No result from skill execution")
return True
content = reply.content
if result_prefix:
content = f"{result_prefix}\n\n{content}"
from channel.channel_factory import create_channel
channel = create_channel(channel_type)
if not channel:
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
return False
if channel_type == "web" and hasattr(channel, 'request_to_session'):
req_id = context.get("request_id")
if req_id:
channel.request_to_session[req_id] = receiver
try:
channel.send(Reply(ReplyType.TEXT, content), context)
except Exception as e:
logger.error(f"[Scheduler] Failed to send skill result: {e}")
return False
_remember_delivered_output(agent_bridge, task, channel_type, content)
logger.info(f"[Scheduler] Task {task['id']} executed: skill result sent to {receiver}")
return True
except Exception as e:
logger.error(f"[Scheduler] Error in _execute_skill_call: {e}")
import traceback
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
return False
def attach_scheduler_to_tool(tool, context: Context = None):

View File

@@ -52,7 +52,6 @@ class SchedulerService:
self.running = True
self.thread = threading.Thread(target=self._run_loop, daemon=True)
self.thread.start()
logger.debug("[Scheduler] Service started")
def stop(self):
"""Stop the scheduler service"""
@@ -67,7 +66,7 @@ class SchedulerService:
def _run_loop(self):
"""Main scheduler loop"""
logger.debug("[Scheduler] Scheduler loop started")
logger.info("[Scheduler] Scheduler loop started")
while self.running:
try:
@@ -84,12 +83,18 @@ class SchedulerService:
for task in tasks:
try:
# Check if task is due
if self._is_task_due(task, now):
logger.info(f"[Scheduler] Executing task: {task['id']} - {task['name']}")
self._execute_task(task)
# Update next run time
ok = self._execute_task(task)
if not ok:
# Leave next_run_at as-is so the next loop retries.
# Cron tasks within the catch-up window will keep
# firing; beyond it _is_task_due will reschedule.
logger.warning(
f"[Scheduler] Task {task['id']} delivery failed, will retry next tick"
)
continue
next_run = self._calculate_next_run(task, now)
if next_run:
self.task_store.update_task(task['id'], {
@@ -97,7 +102,6 @@ class SchedulerService:
"last_run_at": now.isoformat()
})
else:
# One-time task completed, remove it
self.task_store.delete_task(task['id'])
logger.info(f"[Scheduler] One-time task completed and removed: {task['id']}")
except Exception as e:
@@ -128,30 +132,35 @@ class SchedulerService:
try:
next_run = _parse_naive_local(next_run_str)
# Check if task is overdue (e.g., service restart)
if next_run < now:
time_diff = (now - next_run).total_seconds()
# If overdue by more than 5 minutes, skip this run and schedule next
if time_diff > 300: # 5 minutes
logger.warning(f"[Scheduler] Task {task['id']} is overdue by {int(time_diff)}s, skipping and scheduling next run")
# For one-time tasks, remove them directly
schedule = task.get("schedule", {})
if schedule.get("type") == "once":
self.task_store.delete_task(task['id'])
logger.info(f"[Scheduler] One-time task {task['id']} expired, removed")
return False
# For recurring tasks, calculate next run from now
next_next_run = self._calculate_next_run(task, now)
if next_next_run:
self.task_store.update_task(task['id'], {
"next_run_at": next_next_run.isoformat()
})
logger.info(f"[Scheduler] Rescheduled task {task['id']} to {next_next_run}")
schedule = task.get("schedule", {})
schedule_type = schedule.get("type")
# Catch-up window: fire if we're within 10 minutes of the
# scheduled tick. Beyond that we'd rather skip than push a
# stale daily report to the user.
if time_diff <= 600:
return True
logger.warning(
f"[Scheduler] Task {task['id']} is overdue by {int(time_diff)}s, "
f"skipping and scheduling next run"
)
if schedule_type == "once":
self.task_store.delete_task(task['id'])
logger.info(f"[Scheduler] One-time task {task['id']} expired, removed")
return False
next_next_run = self._calculate_next_run(task, now)
if next_next_run:
self.task_store.update_task(task['id'], {
"next_run_at": next_next_run.isoformat()
})
logger.info(f"[Scheduler] Rescheduled task {task['id']} to {next_next_run}")
return False
return now >= next_run
except Exception as e:
logger.error(
@@ -213,20 +222,22 @@ class SchedulerService:
return None
def _execute_task(self, task: dict):
def _execute_task(self, task: dict) -> bool:
"""
Execute a task
Args:
task: Task dictionary
Execute a task.
Returns True if delivery succeeded (caller should advance state),
False if it failed (caller should keep next_run_at so the next
loop iteration retries). Callback may return None for legacy
behaviour, treated as success.
"""
try:
# Call the execute callback
self.execute_callback(task)
result = self.execute_callback(task)
return False if result is False else True
except Exception as e:
logger.error(f"[Scheduler] Error executing task {task['id']}: {e}")
# Update task with error
self.task_store.update_task(task['id'], {
"last_error": str(e),
"last_error_at": datetime.now().isoformat()
})
return False

View File

@@ -182,8 +182,15 @@ class TaskStore:
if enabled_only:
task_list = [t for t in task_list if t.get("enabled", True)]
# Sort by next_run_at
task_list.sort(key=lambda t: t.get("next_run_at", float('inf')))
# Sort by enabled status (enabled first), then by next_run_at
def sort_key(t):
enabled = t.get("enabled", True)
next_run = t.get("next_run_at", "")
# Enabled tasks first (0), disabled tasks second (1)
# Then sort by next_run_at (empty string sorts last)
return (0 if enabled else 1, next_run if next_run else "9999-12-31")
task_list.sort(key=sort_key)
return task_list

View File

@@ -54,6 +54,11 @@ class Send(BaseTool):
if not path:
return ToolResult.fail("Error: path parameter is required")
# Pass through remote URLs directly (no local file check): the client
# renders the link inline, so no download is needed.
if path.lower().startswith(("http://", "https://")):
return self._build_url_result(path, message)
# Resolve path
absolute_path = self._resolve_path(path)
@@ -112,6 +117,46 @@ class Send(BaseTool):
return ToolResult.success(result)
def _build_url_result(self, url: str, message: str) -> ToolResult:
"""Build a file_to_send result for a remote http(s) URL.
The URL is passed through as both ``path`` and ``url`` so downstream
channels render it inline without downloading it locally.
"""
# Infer file type from the URL path extension (ignore query string).
from urllib.parse import urlparse
url_path = urlparse(url).path
file_ext = Path(url_path).suffix.lower()
file_name = Path(url_path).name or "file"
if file_ext in self.image_extensions:
file_type = "image"
mime_type = self._get_image_mime_type(file_ext)
elif file_ext in self.video_extensions:
file_type = "video"
mime_type = self._get_video_mime_type(file_ext)
elif file_ext in self.audio_extensions:
file_type = "audio"
mime_type = self._get_audio_mime_type(file_ext)
elif file_ext in self.document_extensions:
file_type = "document"
mime_type = self._get_document_mime_type(file_ext)
else:
# Default to image: most pass-through URLs are generated images.
file_type = "image"
mime_type = "image/jpeg"
result = {
"type": "file_to_send",
"file_type": file_type,
"path": url,
"url": url,
"file_name": file_name,
"mime_type": mime_type,
"message": message or f"正在发送 {file_name}",
}
return ToolResult.success(result)
def _resolve_path(self, path: str) -> str:
"""Resolve path to absolute path"""
path = expand_path(path)

View File

@@ -71,6 +71,22 @@ class ToolManager:
if not hasattr(self, '_mcp_active_configs'):
# server_name -> normalized config dict, for diff-based reload.
self._mcp_active_configs: dict = {}
if not hasattr(self, '_mcp_tool_vectors'):
# mcp_tool_name -> embedding vector, used by on-demand tool
# retrieval. Populated lazily on first retrieval so users who
# never enable the feature pay zero embedding cost.
self._mcp_tool_vectors: dict = {}
if not hasattr(self, '_mcp_vector_lock'):
# Guards incremental index builds so concurrent turns don't
# double-embed the same newly-loaded MCP tools.
self._mcp_vector_lock = threading.Lock()
if not hasattr(self, '_embedding_provider_initialized'):
# The embedding provider is created once, lazily, and reused for
# both tool-index and per-query embeddings. None means keyword-only
# mode (no provider configured) — retrieval then falls back to full
# injection at the caller.
self._embedding_provider_initialized = False
self._embedding_provider = None
def load_tools(self, tools_dir: str = "", config_dict=None):
"""
@@ -450,21 +466,30 @@ class ToolManager:
the others, and never raises out of the worker thread.
"""
try:
from agent.tools.mcp.mcp_client import McpClient, McpClientRegistry
from agent.tools.mcp.mcp_client import McpClient, McpClientRegistry, set_reload_callback
from agent.tools.mcp.mcp_tool import McpTool
registry = McpClientRegistry()
self._mcp_registry = registry
# Let the OAuth web callback bring a server online once authorized.
set_reload_callback(self.reload_mcp_server)
for cfg in mcp_servers_config:
server_name = cfg.get("name", "<unnamed>")
try:
client = McpClient(cfg)
if not client.initialize():
self._mcp_status[server_name] = "failed"
logger.warning(
f"[MCP] Server '{server_name}' failed to initialize — skipping"
)
if getattr(client, "needs_auth", False):
self._mcp_status[server_name] = "needs_auth"
logger.info(
f"[MCP] Server '{server_name}' needs authorization — "
f"waiting for the user to complete the OAuth flow"
)
else:
self._mcp_status[server_name] = "failed"
logger.warning(
f"[MCP] Server '{server_name}' failed to initialize — skipping"
)
continue
tool_schemas = client.list_tools()
@@ -502,6 +527,28 @@ class ToolManager:
except Exception as e:
logger.warning(f"[ToolManager] MCP background loader crashed: {e}")
def reload_mcp_server(self, server_name: str) -> None:
"""Re-initialize a single MCP server (e.g. after OAuth authorization).
Tears down any existing client for the server and starts it again in
the background, so a freshly-stored access token is picked up and the
server's tools become available on the next message.
"""
with self._mcp_lock:
cfg = self._mcp_active_configs.get(server_name)
if not cfg:
logger.warning(f"[MCP] reload requested for unknown server '{server_name}'")
return
logger.info(f"[MCP] Reloading server '{server_name}' after authorization")
self._teardown_mcp_server(server_name)
self._mcp_status[server_name] = "pending"
threading.Thread(
target=self._load_mcp_tools_async,
args=([cfg],),
daemon=True,
name=f"mcp-reload-{server_name}",
).start()
def list_mcp_status(self) -> dict:
"""Return {server_name: status} snapshot for UI / debugging."""
return dict(self._mcp_status)
@@ -523,6 +570,16 @@ class ToolManager:
if agent is None or not hasattr(agent, "tools"):
return ([], [])
# Never re-inject MCP tools into a restricted Self-Evolution review agent.
# The review agent is created with a deliberately reduced, workspace-guarded
# toolset; silently re-adding configured MCP tools here would bypass that
# policy boundary (see agent/evolution/executor.py). The flag may live on
# the agent itself (Agent) or on the wrapping stream executor's .agent.
if getattr(agent, "_evolution_restricted", False) or getattr(
getattr(agent, "agent", None), "_evolution_restricted", False
):
return ([], [])
from agent.tools.mcp.mcp_tool import McpTool
current = self._mcp_tool_instances
registry_names = set(current.keys())
@@ -564,6 +621,91 @@ class ToolManager:
return (sorted(added), sorted(removed))
# ------------------------------------------------------------------
# On-demand MCP tool retrieval support
#
# The vector index and the embedding provider are owned here (singleton,
# process-wide, aligned with the MCP tool lifecycle). The context-aware
# selection itself lives in agent.tools.mcp.tool_retrieval, driven by the
# executor which is the only place that knows the conversation context.
# ------------------------------------------------------------------
def count_mcp_tools(self) -> int:
"""Return the number of currently loaded MCP tools."""
return len(self._mcp_tool_instances)
def get_mcp_tool_vectors(self) -> dict:
"""Return ``{mcp_tool_name: vector}`` for currently loaded MCP tools.
Lazily embeds any MCP tools not yet in the cache (MCP servers load
asynchronously, so tools may appear over time). Returns an empty dict
when no embedding provider is available or embedding fails — the caller
then falls back to full injection. Never raises.
"""
try:
self._ensure_mcp_tool_vectors()
except Exception as e:
logger.debug(f"[ToolManager] MCP tool vector build skipped: {e}")
return dict(self._mcp_tool_vectors)
def embed_query(self, text: str):
"""Embed a retrieval query with the shared provider.
Returns the embedding vector, or None if no provider is available or
the call fails (caller falls back to full injection). Never raises.
"""
if not text:
return None
provider = self._get_embedding_provider()
if provider is None:
return None
try:
return provider.embed_query(text)
except Exception as e:
logger.debug(f"[ToolManager] query embedding failed: {e}")
return None
def _ensure_mcp_tool_vectors(self) -> None:
"""Incrementally embed MCP tools that are not yet cached."""
# Snapshot to avoid concurrent-mutation while the async loader runs.
current = dict(self._mcp_tool_instances)
missing = [name for name in current if name not in self._mcp_tool_vectors]
if not missing:
return
provider = self._get_embedding_provider()
if provider is None:
return
with self._mcp_vector_lock:
# Re-check under lock: another thread may have filled these in.
missing = [name for name in current if name not in self._mcp_tool_vectors]
if not missing:
return
texts = [self._mcp_tool_embed_text(current[name]) for name in missing]
vectors = provider.embed_batch(texts)
for name, vec in zip(missing, vectors):
self._mcp_tool_vectors[name] = vec
@staticmethod
def _mcp_tool_embed_text(tool) -> str:
"""Build the text that represents an MCP tool for embedding."""
name = getattr(tool, "name", "") or ""
description = getattr(tool, "description", "") or ""
return f"{name}: {description}".strip()
def _get_embedding_provider(self):
"""Lazily create and cache the shared embedding provider (or None)."""
if not self._embedding_provider_initialized:
try:
from agent.memory.embedding import create_default_embedding_provider
self._embedding_provider = create_default_embedding_provider()
except Exception as e:
logger.warning(f"[ToolManager] embedding provider init failed: {e}")
self._embedding_provider = None
self._embedding_provider_initialized = True
return self._embedding_provider
def create_tool(self, name: str) -> BaseTool:
"""
Get a new instance of a tool by name.

View File

@@ -15,11 +15,17 @@ from .diff import (
normalize_to_lf,
restore_line_endings,
normalize_for_fuzzy_match,
count_matches,
fuzzy_find_text,
generate_diff_string,
FuzzyMatchResult
)
from .url_safety import (
validate_url_safe,
assert_public_ip
)
__all__ = [
'truncate_head',
'truncate_tail',
@@ -34,7 +40,10 @@ __all__ = [
'normalize_to_lf',
'restore_line_endings',
'normalize_for_fuzzy_match',
'count_matches',
'fuzzy_find_text',
'generate_diff_string',
'FuzzyMatchResult'
'FuzzyMatchResult',
'validate_url_safe',
'assert_public_ip'
]

View File

@@ -93,6 +93,40 @@ class FuzzyMatchResult:
self.content_for_replacement = content_for_replacement
def _build_fuzzy_pattern(old_text: str) -> Optional[str]:
"""
Build the whitespace-flexible regex used to locate ``old_text`` fuzzily.
Returns ``None`` when ``old_text`` has no non-whitespace content to match.
This is the single source of truth for fuzzy matching, so that *finding* a
match (:func:`fuzzy_find_text`) and *counting* occurrences
(:func:`count_matches`) always use the exact same rules.
"""
stripped = old_text.strip('\n')
if not stripped.strip():
return None
source_lines = stripped.split('\n')
line_patterns = []
for i, line in enumerate(source_lines):
tokens = line.split()
if not tokens:
line_patterns.append(r'[ \t]*')
continue
# Tolerate any run of blanks between tokens.
core = r'[ \t]+'.join(re.escape(tok) for tok in tokens)
# First-line leading whitespace is folded into the match only when
# old_text itself was indented here; otherwise it stays OUTSIDE the
# match so a no-indent old_text preserves (does not swallow and drop)
# the file's existing indentation -- mirroring an exact substring
# match. Inner lines always tolerate indentation: it sits inside the
# matched region and is re-supplied by new_text.
if i > 0 or line[:1] in (' ', '\t'):
core = r'[ \t]*' + core
line_patterns.append(core + r'[ \t]*')
return '\n'.join(line_patterns)
def fuzzy_find_text(content: str, old_text: str) -> FuzzyMatchResult:
"""
Find text in content, try exact match first, then fuzzy match
@@ -110,25 +144,54 @@ def fuzzy_find_text(content: str, old_text: str) -> FuzzyMatchResult:
match_length=len(old_text),
content_for_replacement=content
)
# Try fuzzy match
fuzzy_content = normalize_for_fuzzy_match(content)
fuzzy_old_text = normalize_for_fuzzy_match(old_text)
index = fuzzy_content.find(fuzzy_old_text)
if index != -1:
# Fuzzy match successful, use normalized content for replacement
return FuzzyMatchResult(
found=True,
index=index,
match_length=len(fuzzy_old_text),
content_for_replacement=fuzzy_content
)
# Fuzzy match: the exact substring was not found, most likely because the
# whitespace differs (indentation, spaces around operators, trailing
# spaces). Locate the region in the ORIGINAL content using a
# whitespace-flexible pattern and return offsets into that original
# content.
#
# This must NOT replace inside a whitespace-normalized copy of the file:
# doing so previously returned the normalized copy as
# content_for_replacement, which caused the whole file to be rewritten
# with collapsed indentation (every untouched line got reformatted).
pattern = _build_fuzzy_pattern(old_text)
if pattern is not None:
match = re.search(pattern, content)
if match:
return FuzzyMatchResult(
found=True,
index=match.start(),
match_length=match.end() - match.start(),
content_for_replacement=content
)
# Not found
return FuzzyMatchResult(found=False)
def count_matches(content: str, old_text: str) -> int:
"""
Count occurrences of ``old_text`` using the SAME strategy as
:func:`fuzzy_find_text`: an exact substring when one is present, otherwise
the whitespace-flexible fuzzy regex.
The edit tool's uniqueness guard must agree with the matcher that actually
performs the replacement. Counting through a separate normalization pass
(the previous approach) could disagree with the regex used to locate and
replace, so both paths now share :func:`_build_fuzzy_pattern`.
"""
if not old_text:
return 0
# Mirror fuzzy_find_text: prefer exact matching when it applies.
if content.find(old_text) != -1:
return content.count(old_text)
pattern = _build_fuzzy_pattern(old_text)
if pattern is None:
return 0
return len(re.findall(pattern, content))
def generate_diff_string(old_content: str, new_content: str) -> dict:
"""
Generate unified diff string

View File

@@ -0,0 +1,96 @@
"""
Shared SSRF guard utilities for tools that fetch model-supplied URLs.
SSRF protection is OPT-IN and disabled by default, because legitimate use
cases (local dev servers, LAN services, proxy fake-ip resolution) need to
reach non-public addresses. Enable it by setting the config option
``web_security_ssrf_protection: true`` (or env ``WEB_SECURITY_SSRF_PROTECTION``).
When enabled, a URL is only considered safe when it uses an http/https
scheme, has a hostname, that hostname resolves, and every resolved address
is a public (internet-routable) address. Loopback, private (RFC1918 / ULA),
link-local (incl. the 169.254.169.254 cloud-metadata endpoint) and otherwise
reserved addresses are rejected, for both IPv4 and IPv6.
"""
import ipaddress
import os
import socket
from urllib.parse import urlparse
def _ssrf_protection_enabled() -> bool:
"""Return True only when SSRF protection is explicitly turned on.
Disabled by default. Reads the env var first, then falls back to the
global config; any failure to read config is treated as "disabled" so
the guard never breaks normal fetching.
"""
env = os.getenv("WEB_SECURITY_SSRF_PROTECTION")
if env is not None:
return env.strip().lower() in ("1", "true", "yes", "on")
try:
from config import conf
return bool(conf().get("web_security_ssrf_protection", False))
except Exception:
return False
def _is_blocked_ip(ip: "ipaddress._BaseAddress") -> bool:
"""Return True if the address is not safe to connect to (non-public)."""
return (
ip.is_private
or ip.is_loopback
or ip.is_link_local
or ip.is_reserved
or ip.is_multicast
or ip.is_unspecified
)
def assert_public_ip(ip_str: str) -> None:
"""Raise ValueError if the given literal IP is a non-public address.
No-op when SSRF protection is disabled (the default). Used to re-validate
the concrete address a redirect resolved to.
"""
if not _ssrf_protection_enabled():
return
ip = ipaddress.ip_address(ip_str)
if _is_blocked_ip(ip):
raise ValueError(
f"URL resolves to a non-public address ({ip_str}), "
f"request blocked for security"
)
def validate_url_safe(url: str) -> None:
"""Reject URLs that target private/loopback/link-local addresses (SSRF guard).
No-op when SSRF protection is disabled (the default). When enabled,
resolves the hostname to its IP address(es) and blocks any that fall
into non-public ranges. Also rejects URLs with no host, non-HTTP(S)
schemes, or hosts that fail DNS resolution.
Raises:
ValueError: if the URL targets a disallowed address.
"""
if not _ssrf_protection_enabled():
return
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise ValueError(f"Unsupported URL scheme: {parsed.scheme}")
hostname = parsed.hostname
if not hostname:
raise ValueError("URL has no hostname")
try:
# Resolve all addresses for the hostname.
addr_infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
except socket.gaierror:
raise ValueError(f"Cannot resolve hostname: {hostname}")
for family, _, _, _, sockaddr in addr_infos:
assert_public_ip(sockaddr[0])

View File

@@ -26,13 +26,14 @@ from typing import Any, Dict, List, Optional
import requests
from agent.tools.base_tool import BaseTool, ToolResult
from agent.tools.utils.url_safety import validate_url_safe
from common import const
from common.log import logger
from config import conf
DEFAULT_MODEL = const.GPT_55
DEFAULT_TIMEOUT = 60
MAX_TOKENS = 1000
DEFAULT_MODEL = const.GPT_41_MINI
DEFAULT_TIMEOUT = 180
MAX_TOKENS = 4000
COMPRESS_THRESHOLD = 1_048_576 # 1 MB
SUPPORTED_EXTENSIONS = {
@@ -51,12 +52,13 @@ _MAIN_MODEL_PROVIDER_NAME = "MainModel"
_DISCOVERABLE_MODELS = [
("moonshot_api_key", const.MOONSHOT, const.KIMI_K2_6, "Moonshot"),
("ark_api_key", const.DOUBAO, const.DOUBAO_SEED_2_PRO, "Doubao"),
("dashscope_api_key", const.QWEN_DASHSCOPE, const.QWEN36_PLUS, "DashScope"),
("claude_api_key", const.CLAUDEAPI, const.CLAUDE_4_6_SONNET, "Claude"),
("dashscope_api_key", const.QWEN_DASHSCOPE, const.QWEN37_PLUS, "DashScope"),
("claude_api_key", const.CLAUDEAPI, const.CLAUDE_SONNET_5, "Claude"),
("gemini_api_key", const.GEMINI, const.GEMINI_35_FLASH, "Gemini"),
("qianfan_api_key", const.QIANFAN, const.ERNIE_45_TURBO_VL, "Qianfan"),
("zhipu_ai_api_key", const.ZHIPU_AI, const.GLM_4_7, "ZhipuAI"),
("minimax_api_key", const.MiniMax, const.MINIMAX_M2_7, "MiniMax"),
("mimo_api_key", const.MIMO, const.MIMO_V2_5_PRO, "MiMo"),
]
# Model name prefix → discoverable provider display_name.
@@ -73,11 +75,29 @@ _MODEL_PREFIX_TO_PROVIDER = [
("glm-", "ZhipuAI"),
("minimax-", "MiniMax"),
("abab", "MiniMax"),
("mimo-", "MiMo"),
]
# Model prefixes that natively belong to OpenAI / LinkAI (raw HTTP providers).
_OPENAI_MODEL_PREFIXES = ("gpt-", "o1-", "o3-", "o4-", "chatgpt-")
# Maps the UI provider id (persisted in tools.vision.provider) to the internal
# display name used in VisionProvider.name. Keep in sync with _DISCOVERABLE_MODELS
# and the openai/linkai branches in _route_by_model_name.
_PROVIDER_ID_TO_DISPLAY = {
"openai": "OpenAI",
"linkai": "LinkAI",
"moonshot": "Moonshot",
"doubao": "Doubao",
"dashscope": "DashScope",
"claudeAPI": "Claude",
"gemini": "Gemini",
"qianfan": "Qianfan",
"zhipu": "ZhipuAI",
"minimax": "MiniMax",
"mimo": "MiMo",
}
@dataclass
class VisionProvider:
@@ -142,7 +162,7 @@ class Vision(BaseTool):
"Error: No model available for Vision.\n"
"The main model does not support vision and no other API keys are configured.\n"
"Options:\n"
" 1. Switch to a multimodal model (e.g. ernie-4.5-turbo-vl, qwen3.6-plus, claude-sonnet-4-6, gemini-2.0-flash)\n"
" 1. Switch to a multimodal model (e.g. claude-sonnet-5, qwen3.7-plus, gemini-2.0-flash, ernie-4.5-turbo-vl)\n"
" 2. Configure OPENAI_API_KEY: env_config(action=\"set\", key=\"OPENAI_API_KEY\", value=\"your-key\")\n"
" 3. Configure LINKAI_API_KEY: env_config(action=\"set\", key=\"LINKAI_API_KEY\", value=\"your-key\")"
)
@@ -211,13 +231,19 @@ class Vision(BaseTool):
are de-duplicated to avoid retrying the same endpoint twice.
"""
user_model = self._resolve_user_vision_model()
user_provider = self._resolve_user_vision_provider()
providers: List[VisionProvider] = []
# Step 1: preferred provider derived from tools.vision.model
if user_model:
# Step 1: preferred provider — explicit `tools.vision.provider`
# wins so custom model names can still be routed correctly. Falls
# through to model-name prefix inference when provider is unset.
preferred = None
if user_provider and user_model:
preferred = self._route_by_provider_id(user_provider, user_model)
if not preferred and user_model:
preferred = self._route_by_model_name(user_model)
if preferred:
providers.extend(preferred)
if preferred:
providers.extend(preferred)
# Step 2: auto-discovery chain as fallback
existing = {p.name for p in providers}
@@ -263,6 +289,24 @@ class Vision(BaseTool):
return m.strip()
return None
@staticmethod
def _resolve_user_vision_provider() -> Optional[str]:
"""Read tools.vision.provider — the UI-persisted vendor id.
Lets users pin a vendor for custom model names that prefix-inference
can't recognize. Returns None when unset/blank.
"""
tools_conf = conf().get("tools") or conf().get("tool") or {}
if not isinstance(tools_conf, dict):
return None
vision_conf = tools_conf.get("vision", {})
if not isinstance(vision_conf, dict):
return None
p = vision_conf.get("provider")
if isinstance(p, str) and p.strip():
return p.strip()
return None
@staticmethod
def _infer_provider_from_model(model_name: str) -> Optional[str]:
"""
@@ -279,6 +323,60 @@ class Vision(BaseTool):
return display_name
return None
def _route_by_provider_id(self, provider_id: str, user_model: str) -> Optional[List[VisionProvider]]:
"""Route by the UI-persisted provider id.
Returns:
- [provider] : provider id is known and its key is configured.
- None : unknown provider id, or the bot can't be created.
Caller falls through to model-name-based routing.
"""
# Custom OpenAI-compatible providers — read credentials from
# custom_providers list, same pattern as embedding.
if provider_id.startswith("custom:"):
p = self._build_custom_provider(provider_id, user_model)
return [p] if p else None
display_name = _PROVIDER_ID_TO_DISPLAY.get(provider_id)
if not display_name:
return None
# OpenAI / LinkAI use raw HTTP providers, not the discoverable bot path.
if provider_id == "openai":
p = self._build_openai_provider(user_model)
return [p] if p else None
if provider_id == "linkai":
p = self._build_linkai_provider(user_model)
return [p] if p else None
# Discoverable bot-backed providers.
for config_key, bot_type, _default_model, name in _DISCOVERABLE_MODELS:
if name != display_name:
continue
api_key = conf().get(config_key, "")
if not api_key or not api_key.strip():
logger.warning(f"[Vision] tools.vision.provider='{provider_id}' "
f"but '{config_key}' is not configured. Falling back.")
return None
try:
from models.bot_factory import create_bot
bot = create_bot(bot_type)
if not hasattr(bot, 'call_vision'):
logger.warning(f"[Vision] '{display_name}' bot does not implement call_vision.")
return None
except Exception as e:
logger.warning(f"[Vision] Failed to create '{display_name}' bot: {e}")
return None
return [VisionProvider(
name=display_name,
api_key="",
api_base="",
model_override=user_model,
use_bot=True,
fallback_bot=bot,
)]
return None
def _route_by_model_name(self, user_model: str) -> Optional[List[VisionProvider]]:
"""
Try to build a provider list using the user-specified model name.
@@ -504,6 +602,34 @@ class Vision(BaseTool):
model_override=preferred_model,
)
def _build_custom_provider(self, provider_id: str, preferred_model: Optional[str] = None) -> Optional[VisionProvider]:
"""Build a VisionProvider from a custom:<id> entry in custom_providers.
Uses the standard OpenAI /chat/completions endpoint — any
OpenAI-compatible multimodal endpoint works."""
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
_, custom_id = parse_custom_bot_type(provider_id)
if not custom_id:
return None
entry = _find_provider_by_id(get_custom_providers(), custom_id)
if not entry:
logger.warning(f"[Vision] custom provider '{provider_id}' not found in custom_providers")
return None
api_key = (entry.get("api_key") or "").strip()
api_base = (entry.get("api_base") or "").strip()
if not api_key or not api_base:
logger.warning(f"[Vision] custom provider '{provider_id}' missing api_key or api_base")
return None
model = preferred_model or entry.get("model") or ""
if not model:
logger.warning(f"[Vision] custom provider '{provider_id}' has no model configured")
return None
return VisionProvider(
name=entry.get("name") or provider_id,
api_key=api_key,
api_base=self._ensure_v1(api_base.rstrip("/")),
model_override=model,
)
def _call_via_bot(self, model: str, question: str, image_content: dict,
provider: Optional[VisionProvider] = None) -> ToolResult:
"""
@@ -563,6 +689,22 @@ class Vision(BaseTool):
return api_base
return api_base.rstrip("/") + "/v1"
@staticmethod
def _validate_url_safe(url: str) -> None:
"""Reject URLs that target private/loopback/link-local addresses (SSRF guard).
Resolves the hostname to its IP address(es) and blocks any that fall
into non-public ranges. Also rejects URLs with no host, non-HTTP(S)
schemes, or hosts that fail DNS resolution.
Delegates to the shared ``agent.tools.utils.url_safety`` helper so the
same guard protects every tool that fetches model-supplied URLs.
Raises:
ValueError: if the URL targets a disallowed address.
"""
validate_url_safe(url)
def _build_image_content(self, image: str) -> dict:
"""
Build the image_url content block.
@@ -570,6 +712,7 @@ class Vision(BaseTool):
so every bot backend can consume them without extra downloads.
"""
if image.startswith(("http://", "https://")):
self._validate_url_safe(image)
return self._download_to_data_url(image)
if not os.path.isfile(image):

View File

@@ -16,11 +16,15 @@ import requests
from agent.tools.base_tool import BaseTool, ToolResult
from agent.tools.utils.truncate import truncate_head, format_size
from agent.tools.utils.url_safety import validate_url_safe
from common.log import logger
DEFAULT_TIMEOUT = 30
MAX_FILE_SIZE = 50 * 1024 * 1024 # 50MB
# Cap on how many redirects we follow; each hop's target is re-validated
# against the SSRF guard so a public URL cannot bounce us into an internal one.
MAX_REDIRECTS = 10
DEFAULT_HEADERS = {
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36",
@@ -107,23 +111,65 @@ class WebFetch(BaseTool):
if parsed.scheme not in ("http", "https"):
return ToolResult.fail("Error: Invalid URL (must start with http:// or https://)")
# SSRF guard: reject URLs that resolve to private/loopback/link-local/
# cloud-metadata addresses before any request is issued.
try:
validate_url_safe(url)
except ValueError as e:
return ToolResult.fail(f"Error: {e}")
if _is_document_url(url):
return self._fetch_document(url)
return self._fetch_webpage(url)
# ---- Safe request helper ----
@staticmethod
def _safe_get(url: str, **kwargs) -> requests.Response:
"""Issue a GET request while re-validating every redirect hop (SSRF guard).
Auto-redirect is disabled and each hop is followed manually so the
target of every redirect is re-resolved and checked against the SSRF
guard. This prevents a public URL from 3xx-bouncing into a private,
loopback, link-local or cloud-metadata address. ``kwargs`` are passed
through to ``requests.get`` (e.g. ``stream``).
Raises:
ValueError: if any hop resolves to a non-public address.
"""
kwargs.pop("allow_redirects", None)
current = url
for _ in range(MAX_REDIRECTS + 1):
response = requests.get(
current,
headers=DEFAULT_HEADERS,
timeout=DEFAULT_TIMEOUT,
allow_redirects=False,
**kwargs,
)
if not response.is_redirect and not response.is_permanent_redirect:
return response
location = response.headers.get("Location")
if not location:
return response
# Resolve the redirect target relative to the current URL, then
# re-validate it before following.
current = requests.compat.urljoin(current, location)
validate_url_safe(current)
response.close()
raise ValueError(f"Too many redirects (>{MAX_REDIRECTS})")
# ---- Web page fetching ----
def _fetch_webpage(self, url: str) -> ToolResult:
"""Fetch and extract readable text from an HTML web page."""
parsed = urlparse(url)
try:
response = requests.get(
url,
headers=DEFAULT_HEADERS,
timeout=DEFAULT_TIMEOUT,
allow_redirects=True,
)
response = self._safe_get(url)
response.raise_for_status()
except requests.Timeout:
return ToolResult.fail(f"Error: Request timed out after {DEFAULT_TIMEOUT}s")
@@ -131,6 +177,8 @@ class WebFetch(BaseTool):
return ToolResult.fail(f"Error: Failed to connect to {parsed.netloc}")
except requests.HTTPError as e:
return ToolResult.fail(f"Error: HTTP {e.response.status_code} for URL: {url}")
except ValueError as e:
return ToolResult.fail(f"Error: {e}")
except Exception as e:
return ToolResult.fail(f"Error: Failed to fetch URL: {e}")
@@ -158,13 +206,7 @@ class WebFetch(BaseTool):
logger.info(f"[WebFetch] Downloading document: {url} -> {local_path}")
try:
response = requests.get(
url,
headers=DEFAULT_HEADERS,
timeout=DEFAULT_TIMEOUT,
stream=True,
allow_redirects=True,
)
response = self._safe_get(url, stream=True)
response.raise_for_status()
content_length = int(response.headers.get("Content-Length", 0))
@@ -191,6 +233,9 @@ class WebFetch(BaseTool):
return ToolResult.fail(f"Error: Failed to connect to {parsed.netloc}")
except requests.HTTPError as e:
return ToolResult.fail(f"Error: HTTP {e.response.status_code} for URL: {url}")
except ValueError as e:
self._cleanup_file(local_path)
return ToolResult.fail(f"Error: {e}")
except Exception as e:
self._cleanup_file(local_path)
return ToolResult.fail(f"Error: Failed to download file: {e}")

43
app.py
View File

@@ -15,6 +15,11 @@ import threading
_channel_mgr = None
# Desktop mode: a lighter runtime for the packaged Electron client. Plugins are
# loaded in a background thread (so command plugins like cow_cli/godcmd work
# without slowing startup), while MCP warmup is still skipped to keep it fast.
DESKTOP_MODE = os.environ.get("COW_DESKTOP") == "1"
def get_channel_manager():
return _channel_mgr
@@ -76,7 +81,15 @@ class ChannelManager:
self._primary_channel = channels[0][1]
if first_start:
PluginManager().load_plugins()
if DESKTOP_MODE:
# Load plugins in the background so command plugins
# (cow_cli / godcmd, e.g. /status, #help) work in the
# desktop client, without blocking web-service readiness.
threading.Thread(
target=PluginManager().load_plugins, daemon=True
).start()
else:
PluginManager().load_plugins()
# Cloud client is optional. It is only started when
# use_linkai=True AND cloud_deployment_id is set.
@@ -231,10 +244,14 @@ def _clear_singleton_cache(channel_name: str):
"wechatmp": "channel.wechatmp.wechatmp_channel.WechatMPChannel",
"wechatmp_service": "channel.wechatmp.wechatmp_channel.WechatMPChannel",
"wechatcom_app": "channel.wechatcom.wechatcomapp_channel.WechatComAppChannel",
const.WECHAT_KF: "channel.wechat_kf.wechat_kf_channel.WechatKfChannel",
const.FEISHU: "channel.feishu.feishu_channel.FeiShuChanel",
const.DINGTALK: "channel.dingtalk.dingtalk_channel.DingTalkChanel",
const.WECOM_BOT: "channel.wecom_bot.wecom_bot_channel.WecomBotChannel",
const.QQ: "channel.qq.qq_channel.QQChannel",
const.TELEGRAM: "channel.telegram.telegram_channel.TelegramChannel",
const.SLACK: "channel.slack.slack_channel.SlackChannel",
const.DISCORD: "channel.discord.discord_channel.DiscordChannel",
const.WEIXIN: "channel.weixin.weixin_channel.WeixinChannel",
"wx": "channel.weixin.weixin_channel.WeixinChannel",
}
@@ -288,6 +305,16 @@ def _warmup_mcp_tools():
logger.warning(f"[App] MCP warmup failed (non-fatal): {e}")
def _warmup_scheduler():
"""Eager-init AgentBridge so the scheduler thread starts at process
boot rather than waiting for the first user message."""
try:
from bridge.bridge import Bridge
Bridge().get_agent_bridge()
except Exception as e:
logger.warning(f"[App] Scheduler warmup failed: {e}")
def _sync_builtin_skills():
"""Sync builtin skills from project skills/ to workspace skills/ on startup."""
import shutil
@@ -350,8 +377,18 @@ def run():
_sync_builtin_skills()
# Kick off MCP server loading in the background so first-message
# latency isn't dominated by npx package downloads.
_warmup_mcp_tools()
# latency isn't dominated by npx package downloads. Skipped in desktop
# mode (MCP relies on external npx/uvx runtimes that aren't bundled).
if not DESKTOP_MODE:
_warmup_mcp_tools()
if DESKTOP_MODE:
# Defer the (heavy) AgentBridge/scheduler warmup to a background
# thread so the web API becomes available within a couple seconds.
# The scheduler still starts; it just doesn't block UI readiness.
threading.Thread(target=_warmup_scheduler, daemon=True).start()
else:
_warmup_scheduler()
logger.info(f"[App] Starting channels: {channel_names}")

View File

@@ -5,7 +5,7 @@ Agent Bridge - Integrates Agent system with existing COW bridge
import os
from typing import Optional, List
from agent.protocol import Agent, LLMModel, LLMRequest
from agent.protocol import Agent, LLMModel, LLMRequest, get_cancel_registry
from bridge.agent_event_handler import AgentEventHandler
from bridge.agent_initializer import AgentInitializer
from bridge.bridge import Bridge
@@ -78,6 +78,7 @@ class AgentLLMModel(LLMModel):
("moonshot", const.MOONSHOT), ("kimi", const.MOONSHOT),
("doubao", const.DOUBAO), ("deepseek", const.DEEPSEEK),
("ernie", const.QIANFAN),
("mimo-", const.MIMO),
]
def __init__(self, bridge: Bridge, bot_type: str = "chat"):
@@ -285,6 +286,23 @@ class AgentBridge:
# Create helper instances
self.initializer = AgentInitializer(bridge, self)
# Eager-start the scheduler so cron tasks fire without waiting
# for the first user message. init_scheduler is idempotent.
try:
from agent.tools.scheduler.integration import init_scheduler
if init_scheduler(self):
self.scheduler_initialized = True
except Exception as e:
logger.warning(f"[AgentBridge] Eager scheduler init failed: {e}")
# Start the self-evolution idle trigger (idempotent, daemon thread).
try:
from agent.evolution.trigger import start_evolution_trigger
start_evolution_trigger(self)
except Exception as e:
logger.warning(f"[AgentBridge] Evolution trigger init failed: {e}")
def create_agent(self, system_prompt: str, tools: List = None, **kwargs) -> Agent:
"""
Create the super agent with COW integration
@@ -373,7 +391,49 @@ class AgentBridge:
"""Initialize agent for a specific session"""
agent = self.initializer.initialize_agent(session_id=session_id)
self.agents[session_id] = agent
def sync_session_messages_from_store(self, session_id: str) -> int:
"""Reload an agent's in-memory ``messages`` list from the persistent
conversation store.
Used after an external mutation (e.g. user edits / deletes a message
via the web console) so the agent's next turn sees the same history
as the database. The operation is a no-op when the agent has not been
instantiated yet for the session.
Returns:
Number of messages now held in the agent's memory. Returns -1 if
the agent does not exist or has no compatible ``messages`` attr.
"""
if not session_id or session_id not in self.agents:
return -1
agent = self.agents[session_id]
if not (hasattr(agent, "messages") and hasattr(agent, "messages_lock")):
return -1
try:
from agent.memory import get_conversation_store
store = get_conversation_store()
# No turn cap here: we want a faithful mirror of what the store
# has for this session after deletion.
remaining = store.load_messages(session_id, max_turns=10**6)
except Exception as e:
logger.warning(
f"[AgentBridge] Failed to load messages for sync (session={session_id}): {e}"
)
return -1
with agent.messages_lock:
agent.messages.clear()
for msg in remaining:
agent.messages.append({
"role": msg["role"],
"content": msg["content"],
})
count = len(agent.messages)
logger.info(
f"[AgentBridge] Synced agent memory for session={session_id}, messages={count}"
)
return count
def agent_reply(self, query: str, context: Context = None,
on_event=None, clear_history: bool = False) -> Reply:
"""
@@ -390,11 +450,22 @@ class AgentBridge:
"""
session_id = None
agent = None
request_id = None
cancel_event = None
try:
# Extract session_id from context for user isolation
if context:
session_id = context.kwargs.get("session_id") or context.get("session_id")
request_id = context.kwargs.get("request_id") or context.get("request_id")
# Register a cancel token. Prefer per-turn request_id (web),
# fall back to session_id (IM channels). The Event is polled by
# AgentStreamExecutor at safe checkpoints.
registry = get_cancel_registry()
token_key = request_id or session_id
if token_key:
cancel_event = registry.register(token_key, session_id=session_id)
# Get agent for this session (will auto-initialize if needed)
agent = self.get_agent(session_id=session_id)
if not agent:
@@ -444,14 +515,40 @@ class AgentBridge:
)
self._trim_in_memory_to_turns(agent, scheduler_keep_turns)
# Eagerly persist the user message BEFORE running the agent so the
# session and the user's bubble are immediately visible — even if
# the user switches away or refreshes before the reply finishes.
# The reply (assistant/tool messages) is appended once the run
# completes; the final persist skips this already-stored user turn.
pre_persisted = self._pre_persist_user_message(
session_id, query, context, clear_history
)
# Mark this session as mid-run so the self-evolution idle scan does
# not fire concurrently when a single turn runs longer than
# idle_minutes.
try:
from agent.evolution.trigger import mark_run_active
mark_run_active(agent, True)
except Exception:
pass
try:
# Use agent's run_stream method with event handler
response = agent.run_stream(
user_message=query,
on_event=event_handler.handle_event,
clear_history=clear_history
clear_history=clear_history,
cancel_event=cancel_event,
)
finally:
# Clear the mid-run flag so idle scans can review this session.
try:
from agent.evolution.trigger import mark_run_active
mark_run_active(agent, False)
except Exception:
pass
# Restore original tools
if context and context.get("is_scheduled_task"):
agent.tools = original_tools
@@ -459,10 +556,21 @@ class AgentBridge:
# Log execution summary
event_handler.log_summary()
# Release cancel token; keep registry bounded.
if token_key:
try:
registry.unregister(token_key)
except Exception:
pass
# Persist new messages generated during this run
if session_id:
channel_type = (context.get("channel_type") or "") if context else ""
new_messages = getattr(agent, '_last_run_new_messages', [])
new_messages = list(getattr(agent, '_last_run_new_messages', []))
# The leading user turn was already persisted eagerly above;
# drop it here so it isn't stored twice.
if pre_persisted and new_messages and new_messages[0].get("role") == "user":
new_messages = new_messages[1:]
if new_messages:
self._persist_messages(session_id, list(new_messages), channel_type)
else:
@@ -476,6 +584,23 @@ class AgentBridge:
except Exception as e:
logger.warning(f"[AgentBridge] Failed to clear DB after recovery: {e}")
# Record this user turn for the self-evolution idle trigger. Skip
# scheduler-injected / scheduled-task sessions so internal runs do
# not count as user activity.
if session_id and not session_id.startswith("scheduler_") and not (
context and context.get("is_scheduled_task")
):
try:
from agent.evolution.trigger import note_user_turn
ch = (context.get("channel_type") or "") if context else ""
rcv = (context.get("receiver") or "") if context else ""
is_group = bool(context.get("isgroup")) if context else False
# Only enable proactive push for single chats (group push is
# noisy); group sessions still evolve, just without notify.
note_user_turn(agent, channel_type=ch, receiver=(rcv if not is_group else ""))
except Exception:
pass
# Post-message hot-reload: detect edits to ~/cow/mcp.json and
# sync any new/removed MCP tools into the live agent in the
# background. Off the critical path so user latency is unaffected;
@@ -512,6 +637,12 @@ class AgentBridge:
logger.info(f"[AgentBridge] Cleared DB for session after error: {session_id}")
except Exception as db_err:
logger.warning(f"[AgentBridge] Failed to clear DB after error: {db_err}")
# Release cancel token on error path too (idempotent).
if cancel_event is not None and (request_id or session_id):
try:
get_cancel_registry().unregister(request_id or session_id)
except Exception:
pass
return Reply(ReplyType.ERROR, f"Agent error: {str(e)}")
def _schedule_mcp_hot_reload(self, agent):
@@ -553,11 +684,21 @@ class AgentBridge:
"""
file_type = file_info.get("file_type", "file")
file_path = file_info.get("path")
# Remote URLs are passed through as-is; local paths get a file:// prefix
# so the channel can read them from disk.
remote_url = file_info.get("url", "")
is_remote = bool(remote_url) and remote_url.lower().startswith(("http://", "https://"))
def _to_channel_url(p: str) -> str:
if is_remote:
return remote_url
if p and p.lower().startswith(("http://", "https://")):
return p
return f"file://{p}"
# For images, use IMAGE_URL type (channel will handle upload)
if file_type == "image":
# Convert local path to file:// URL for channel processing
file_url = f"file://{file_path}"
file_url = _to_channel_url(file_path)
logger.info(f"[AgentBridge] Sending image: {file_url}")
reply = Reply(ReplyType.IMAGE_URL, file_url)
# Attach text message if present (for channels that support text+image)
@@ -567,7 +708,7 @@ class AgentBridge:
# For all file types (document, video, audio), use FILE type
if file_type in ["document", "video", "audio"]:
file_url = f"file://{file_path}"
file_url = _to_channel_url(file_path)
logger.info(f"[AgentBridge] Sending {file_type}: {file_url}")
reply = Reply(ReplyType.FILE, file_url)
reply.file_name = file_info.get("file_name", os.path.basename(file_path))
@@ -577,7 +718,7 @@ class AgentBridge:
return reply
# For all other file types (tar.gz, zip, etc.), also use FILE type
file_url = f"file://{file_path}"
file_url = _to_channel_url(file_path)
logger.info(f"[AgentBridge] Sending generic file: {file_url}")
reply = Reply(ReplyType.FILE, file_url)
reply.file_name = file_info.get("file_name", os.path.basename(file_path))
@@ -655,6 +796,48 @@ class AgentBridge:
except Exception as e:
logger.warning(f"[AgentBridge] Failed to sync API keys: {e}")
def _pre_persist_user_message(
self, session_id: str, query: str, context: Context, clear_history: bool
) -> bool:
"""Persist the user's message before the agent runs.
This makes a brand-new session (and the user's bubble) visible even if
the reply hasn't finished — switching away or refreshing no longer
loses the in-flight session. Returns True when the user turn was
stored, so the caller can skip it in the post-run persist.
Best-effort: any failure is swallowed and reported as not-persisted.
"""
if not session_id or not query:
return False
# Only real user turns: skip scheduler-injected / scheduled-task runs.
if session_id.startswith("scheduler_") or (
context and context.get("is_scheduled_task")
):
return False
try:
from config import conf
if not conf().get("conversation_persistence", True):
return False
from agent.memory import get_conversation_store
store = get_conversation_store()
# clear_history starts a fresh transcript: wipe the store first so
# the eager user turn becomes seq 0, matching in-memory state.
if clear_history:
store.clear_session(session_id)
channel_type = (context.get("channel_type") or "") if context else ""
user_msg = {
"role": "user",
"content": [{"type": "text", "text": query}],
}
store.append_messages(session_id, [user_msg], channel_type=channel_type)
return True
except Exception as e:
logger.warning(
f"[AgentBridge] Failed to pre-persist user message for session={session_id}: {e}"
)
return False
def _persist_messages(
self, session_id: str, new_messages: list, channel_type: str = ""
) -> None:

View File

@@ -2,44 +2,40 @@
Agent Event Handler - Handles agent events and thinking process output
"""
from common import const
from common.log import logger
# Cap intermediate thinking messages on weixin to stay within send quota.
WEIXIN_THINKING_INSTANT_MAX = 7
class AgentEventHandler:
"""
Handles agent events and optionally sends intermediate messages to channel
"""
def __init__(self, context=None, original_callback=None):
"""
Initialize event handler
Args:
context: COW context (for accessing channel)
original_callback: Original event callback to chain
"""
self.context = context
self.original_callback = original_callback
# Get channel for sending intermediate messages
self.channel = None
if context:
self.channel = context.kwargs.get("channel") if hasattr(context, "kwargs") else None
self.current_content = ""
self.turn_number = 0
channel_type = ""
if context and hasattr(context, "kwargs"):
channel_type = context.kwargs.get("channel_type", "") or ""
self._is_weixin = channel_type == const.WEIXIN
self._thinking_sent_count = 0
self._merged_buf: list[str] = []
def handle_event(self, event):
"""
Main event handler
Args:
event: Event dict with type and data
"""
event_type = event.get("type")
data = event.get("data", {})
# Dispatch to specific handlers
if event_type == "turn_start":
self._handle_turn_start(data)
elif event_type == "message_update":
@@ -52,25 +48,23 @@ class AgentEventHandler:
self._handle_tool_execution_start(data)
elif event_type == "tool_execution_end":
self._handle_tool_execution_end(data)
# Call original callback if provided
elif event_type == "agent_end":
self._handle_agent_end(data)
if self.original_callback:
self.original_callback(event)
def _handle_turn_start(self, data):
"""Handle turn start event"""
self.turn_number = data.get("turn", 0)
self.current_content = ""
def _handle_message_update(self, data):
"""Handle message update event (streaming content text)"""
delta = data.get("delta", "")
self.current_content += delta
def _handle_message_end(self, data):
"""Handle message end event"""
tool_calls = data.get("tool_calls", [])
if tool_calls:
if self.current_content.strip():
logger.info(f"💭 {self.current_content.strip()[:200]}{'...' if len(self.current_content) > 200 else ''}")
@@ -78,35 +72,54 @@ class AgentEventHandler:
else:
if self.current_content.strip():
logger.debug(f"💬 {self.current_content.strip()[:200]}{'...' if len(self.current_content) > 200 else ''}")
# Drain weixin buffer before final reply leaves chat_channel
self._flush_merged_now()
self.current_content = ""
def _handle_agent_end(self, data):
self._flush_merged_now()
def _handle_tool_execution_start(self, data):
"""Handle tool execution start event - logged by agent_stream.py"""
pass
def _handle_tool_execution_end(self, data):
"""Handle tool execution end event - logged by agent_stream.py"""
pass
def _send_to_channel(self, message):
"""
Try to send intermediate message to channel.
Skipped in SSE mode because thinking text is already streamed via on_event.
"""
if self.context and self.context.get("on_event"):
return
if not self.channel:
return
if not self._is_weixin:
self._do_send(message)
return
if self._thinking_sent_count < WEIXIN_THINKING_INSTANT_MAX:
self._do_send(message)
self._thinking_sent_count += 1
return
self._merged_buf.append(message)
def _flush_merged_now(self):
if not self._merged_buf:
return
merged = "\n\n".join(self._merged_buf)
count = len(self._merged_buf)
self._merged_buf = []
logger.debug(f"[AgentEventHandler] Flushing {count} merged thinking msgs, len={len(merged)}")
self._do_send(merged)
self._thinking_sent_count += 1
def _do_send(self, message):
try:
from bridge.reply import Reply, ReplyType
reply = Reply(ReplyType.TEXT, message)
self.channel._send(reply, self.context)
except Exception as e:
logger.debug(f"[AgentEventHandler] Failed to send to channel: {e}")
if self.channel:
try:
from bridge.reply import Reply, ReplyType
reply = Reply(ReplyType.TEXT, message)
self.channel._send(reply, self.context)
except Exception as e:
logger.debug(f"[AgentEventHandler] Failed to send to channel: {e}")
def log_summary(self):
"""Log execution summary - simplified"""
# Summary removed as per user request
# Real-time logging during execution is sufficient
pass

View File

@@ -17,10 +17,6 @@ from common.utils import expand_path
# Module-level lock to serialize scheduler init across concurrent sessions
_scheduler_init_lock = threading.Lock()
# Track whether the embedding model log has been printed in this process,
# so we avoid spamming it once per session.
_embedding_logged: bool = False
class AgentInitializer:
"""
@@ -306,186 +302,16 @@ class AgentInitializer:
"""
Initialize the embedding provider for memory.
Two paths:
Delegates to the shared factory so agent init, knowledge sync and
index rebuild all select the same provider:
A. Default (no `embedding_provider` in config.json):
Auto-init OpenAI -> LinkAI fallback. Existing 1536-dim indices
keep working.
Auto-init OpenAI -> LinkAI fallback.
B. Explicit (`embedding_provider` is set):
Initialize the requested vendor with unified dim (default 1024).
If the index was built with a different dim, vector search will
quietly return no results (cosine returns 0) and keyword search
takes over until the user runs /memory rebuild-index.
Initialize the requested vendor.
"""
from agent.memory import create_embedding_provider
from config import conf
from agent.memory import create_default_embedding_provider
return create_default_embedding_provider()
explicit_provider = (conf().get("embedding_provider") or "").strip().lower()
if not explicit_provider:
return self._init_embedding_provider_legacy(session_id=session_id)
return self._init_embedding_provider_explicit(
memory_config, explicit_provider, session_id=session_id,
)
def _init_embedding_provider_legacy(self, session_id: Optional[str] = None):
"""Legacy auto-init path: OpenAI -> LinkAI. Preserved verbatim for compat."""
from agent.memory import create_embedding_provider
from config import conf
embedding_provider = None
embedding_model = None
openai_api_key = conf().get("open_ai_api_key", "")
openai_api_base = conf().get("open_ai_api_base", "")
if openai_api_key and openai_api_key not in ["", "YOUR API KEY", "YOUR_API_KEY"]:
try:
model = "text-embedding-3-small"
embedding_provider = create_embedding_provider(
provider="openai",
model=model,
api_key=openai_api_key,
api_base=openai_api_base or "https://api.openai.com/v1"
)
embedding_model = f"openai/{model}"
except Exception as e:
logger.warning(f"[AgentInitializer] OpenAI embedding failed: {e}")
if embedding_provider is None:
linkai_api_key = conf().get("linkai_api_key", "") or os.environ.get("LINKAI_API_KEY", "")
linkai_api_base = conf().get("linkai_api_base", "https://api.link-ai.tech")
if linkai_api_key and linkai_api_key not in ["", "YOUR API KEY", "YOUR_API_KEY"]:
try:
model = "text-embedding-3-small"
embedding_provider = create_embedding_provider(
provider="linkai",
model=model,
api_key=linkai_api_key,
api_base=f"{linkai_api_base}/v1"
)
embedding_model = f"linkai/{model}"
except Exception as e:
logger.warning(f"[AgentInitializer] LinkAI embedding failed: {e}")
if embedding_provider is not None and embedding_model:
global _embedding_logged
if not _embedding_logged:
logger.info(
f"[AgentInitializer] Embedding model in use: {embedding_model} "
f"(dim={embedding_provider.dimensions})"
)
_embedding_logged = True
return embedding_provider
def _init_embedding_provider_explicit(
self,
memory_config,
provider_key: str,
session_id: Optional[str] = None,
):
"""Explicit-provider path: build the configured vendor.
If the index was built with a different dim, vector search will
silently return no results (cosine returns 0 for mismatched dims)
and keyword search takes over. Users switch vendors by running
/memory rebuild-index — see docs.
"""
from agent.memory import create_embedding_provider
from agent.memory.embedding import EMBEDDING_VENDORS
from config import conf
meta = EMBEDDING_VENDORS.get(provider_key)
if meta is None:
logger.error(
f"[AgentInitializer] Unknown embedding_provider '{provider_key}'. "
f"Supported: {sorted(EMBEDDING_VENDORS.keys())}. "
f"Memory will run in keyword-only mode."
)
return None
api_key = self._resolve_embedding_api_key(provider_key)
api_base = self._resolve_embedding_api_base(provider_key, meta["default_base_url"])
if not api_key:
logger.error(
f"[AgentInitializer] embedding_provider='{provider_key}' is set but its "
f"API key is missing. Memory will run in keyword-only mode."
)
return None
model = (conf().get("embedding_model") or "").strip() or meta["default_model"]
try:
cfg_dim = int(conf().get("embedding_dimensions") or 0)
except (TypeError, ValueError):
cfg_dim = 0
dim = cfg_dim if cfg_dim > 0 else meta["default_dimensions"]
try:
provider = create_embedding_provider(
provider=provider_key,
model=model,
api_key=api_key,
api_base=api_base,
dimensions=dim,
)
except Exception as e:
logger.error(
f"[AgentInitializer] Failed to init embedding provider "
f"'{provider_key}/{model}': {e}"
)
return None
global _embedding_logged
if not _embedding_logged:
logger.info(
f"[AgentInitializer] Embedding model in use: "
f"{provider_key}/{model} (dim={provider.dimensions})"
)
_embedding_logged = True
return provider
@staticmethod
def _resolve_embedding_api_key(provider_key: str) -> str:
"""Pick the API key for an explicit embedding provider from config."""
from config import conf
key_map = {
"openai": "open_ai_api_key",
"linkai": "linkai_api_key",
"dashscope": "dashscope_api_key",
"doubao": "ark_api_key",
"zhipu": "zhipu_ai_api_key",
}
field = key_map.get(provider_key)
if not field:
return ""
value = conf().get(field, "") or ""
if value in ["", "YOUR API KEY", "YOUR_API_KEY"]:
return ""
return value
@staticmethod
def _resolve_embedding_api_base(provider_key: str, default_base: str) -> str:
"""Pick the API base for an explicit embedding provider from config."""
from config import conf
base_map = {
"openai": "open_ai_api_base",
"linkai": "linkai_api_base",
"doubao": "ark_base_url",
"zhipu": "zhipu_ai_api_base",
}
field = base_map.get(provider_key)
if not field:
return default_base
value = (conf().get(field) or "").strip()
if not value:
return default_base
if provider_key == "linkai" and not value.rstrip("/").endswith("/v1"):
return f"{value.rstrip('/')}/v1"
return value
def _sync_memory(self, memory_manager, session_id: Optional[str] = None):
"""Sync memory database"""
try:
@@ -524,6 +350,14 @@ class AgentInitializer:
logger.debug("[AgentInitializer] WebSearch skipped - no search provider configured")
continue
# Skip evolution_undo when self-evolution is disabled: with no
# evolution there is nothing to roll back, so the tool is dead weight.
if tool_name == "evolution_undo":
from agent.evolution.config import get_evolution_config
if not get_evolution_config().enabled:
logger.debug("[AgentInitializer] evolution_undo skipped - self-evolution disabled")
continue
# Special handling for EnvConfig tool
if tool_name == "env_config":
from agent.tools import EnvConfig
@@ -643,16 +477,25 @@ class AgentInitializer:
except Exception:
timezone_name = "UTC"
# Chinese weekday mapping
weekday_map = {
'Monday': '星期一', 'Tuesday': '星期二', 'Wednesday': '星期三',
'Thursday': '星期四', 'Friday': '星期五', 'Saturday': '星期六', 'Sunday': '星期日'
}
weekday_zh = weekday_map.get(now.strftime("%A"), now.strftime("%A"))
# Weekday: English name in en, Chinese mapping otherwise
weekday_en = now.strftime("%A")
try:
from common import i18n
is_en = i18n.get_language() == "en"
except Exception:
is_en = False
if is_en:
weekday = weekday_en
else:
weekday_map = {
'Monday': '星期一', 'Tuesday': '星期二', 'Wednesday': '星期三',
'Thursday': '星期四', 'Friday': '星期五', 'Saturday': '星期六', 'Sunday': '星期日'
}
weekday = weekday_map.get(weekday_en, weekday_en)
return {
'time': now.strftime("%Y-%m-%d %H:%M:%S"),
'weekday': weekday_zh,
'weekday': weekday,
'timezone': timezone_name
}

View File

@@ -63,6 +63,10 @@ class Bridge(object):
if model_type and model_type.startswith("deepseek"):
self.btype["chat"] = const.DEEPSEEK
# 小米 MiMo 系列模型,全部以 mimo- 开头
if model_type and model_type.startswith("mimo-"):
self.btype["chat"] = const.MIMO
if model_type and isinstance(model_type, str):
lowered_model_type = model_type.lower()
if lowered_model_type == const.QIANFAN or lowered_model_type.startswith("ernie"):

View File

@@ -27,6 +27,9 @@ def create_channel(channel_type) -> Channel:
elif channel_type == "wechatcom_app":
from channel.wechatcom.wechatcomapp_channel import WechatComAppChannel
ch = WechatComAppChannel()
elif channel_type == const.WECHAT_KF:
from channel.wechat_kf.wechat_kf_channel import WechatKfChannel
ch = WechatKfChannel()
elif channel_type == const.FEISHU:
from channel.feishu.feishu_channel import FeiShuChanel
ch = FeiShuChanel()
@@ -39,6 +42,15 @@ def create_channel(channel_type) -> Channel:
elif channel_type == const.QQ:
from channel.qq.qq_channel import QQChannel
ch = QQChannel()
elif channel_type == const.TELEGRAM:
from channel.telegram.telegram_channel import TelegramChannel
ch = TelegramChannel()
elif channel_type == const.SLACK:
from channel.slack.slack_channel import SlackChannel
ch = SlackChannel()
elif channel_type == const.DISCORD:
from channel.discord.discord_channel import DiscordChannel
ch = DiscordChannel()
elif channel_type in (const.WEIXIN, "wx"):
from channel.weixin.weixin_channel import WeixinChannel
ch = WeixinChannel()

View File

@@ -10,6 +10,7 @@ from bridge.reply import *
from channel.channel import Channel
from common.dequeue import Dequeue
from common import memory
from common.i18n import t as _t
from plugins import *
try:
@@ -265,7 +266,7 @@ class ChatChannel(Channel):
if reply.type in self.NOT_SUPPORT_REPLYTYPE:
logger.error("[chat_channel]reply type not support: " + str(reply.type))
reply.type = ReplyType.ERROR
reply.content = "不支持发送的消息类型: " + str(reply.type)
reply.content = _t("不支持发送的消息类型: ", "Unsupported message type: ") + str(reply.type)
if reply.type == ReplyType.TEXT:
reply_text = reply.content
@@ -438,8 +439,21 @@ class ChatChannel(Channel):
return func
# Chat commands that must bypass the per-session serial queue,
# otherwise /cancel would queue behind the task it tries to cancel.
# Use /cancel (not /stop) to avoid colliding with `cow stop` CLI.
_BYPASS_QUEUE_COMMANDS = ("/cancel",)
def produce(self, context: Context):
session_id = context["session_id"]
# Fast path: /cancel must not enter the queue.
if context.type == ContextType.TEXT and context.content:
stripped = context.content.strip().lower()
if stripped in self._BYPASS_QUEUE_COMMANDS:
self._handle_cancel_command(context, session_id)
return
with self.lock:
if session_id not in self.sessions:
self.sessions[session_id] = [
@@ -451,6 +465,29 @@ class ChatChannel(Channel):
else:
self.sessions[session_id][0].put(context)
def _handle_cancel_command(self, context: Context, session_id: str) -> None:
"""Cancel any in-flight agent run for *session_id* and reply inline.
Runs synchronously on the caller's thread. Reply is sent through
_send_reply so plugins (e.g. logging) still observe it.
"""
try:
from agent.protocol import get_cancel_registry
from bridge.reply import Reply, ReplyType
cancelled = get_cancel_registry().cancel_session(session_id)
text = (
_t("🛑 已中止", "🛑 Cancelled")
if cancelled > 0
else _t("当前没有可中止的任务。", "Nothing to cancel.")
)
logger.info(
f"[chat_channel] /cancel fast-path: session={session_id}, cancelled={cancelled}"
)
self._send_reply(context, Reply(ReplyType.TEXT, text))
except Exception as e:
logger.warning(f"[chat_channel] /cancel fast-path failed: {e}")
# 消费者函数,单独线程,用于从消息队列中取出消息并处理
def consume(self):
while True:
@@ -482,7 +519,10 @@ class ChatChannel(Channel):
def cancel_session(self, session_id):
with self.lock:
if session_id in self.sessions:
for future in self.futures[session_id]:
# futures[session_id] is only created in consume() when a task is
# dispatched, so it may be absent if cancel happens right after
# produce() but before the first dispatch. Default to [].
for future in self.futures.get(session_id, []):
future.cancel()
cnt = self.sessions[session_id][0].qsize()
if cnt > 0:
@@ -492,7 +532,7 @@ class ChatChannel(Channel):
def cancel_all_session(self):
with self.lock:
for session_id in self.sessions:
for future in self.futures[session_id]:
for future in self.futures.get(session_id, []):
future.cancel()
cnt = self.sessions[session_id][0].qsize()
if cnt > 0:

View File

View File

@@ -0,0 +1,500 @@
"""
Discord channel via the Gateway (WebSocket) using discord.py.
Features:
- Direct message & guild channel chat (text / image / file)
- Guild trigger: @mention or reply-to-bot (configurable)
- /cancel fast-path matches Web channel behaviour
- Gateway long connection: no public IP / callback URL required, works behind NAT
Implementation note:
discord.py is async-first. We run the client inside a dedicated thread
with its own asyncio loop so the rest of cow (which is sync) stays
untouched. Inbound messages are dispatched onto cow's existing sync
ChatChannel.produce() pipeline; outbound send() schedules coroutines
back onto that loop via asyncio.run_coroutine_threadsafe.
"""
import asyncio
import os
import re
import threading
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel, check_prefix
from channel.discord.discord_message import DiscordMessage
from common.expired_dict import ExpiredDict
from common.log import logger
from common.singleton import singleton
from config import conf
# Discord caps a single message at 2000 chars; split conservatively below.
DISCORD_MSG_LIMIT = 1900
@singleton
class DiscordChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = []
def __init__(self):
super().__init__()
self.bot_token = ""
self.bot_user_id = "" # used to strip @mention and ignore self messages
self.bot_username = ""
self._client = None
self._loop = None
self._loop_thread = None
self._stop_event = threading.Event()
# Idempotent dedup; guard against rare duplicate dispatch
self._received_msgs = ExpiredDict(60 * 60 * 1)
# Disable group whitelist / prefix checks (we handle triggering ourselves
# in _should_reply_in_guild), aligned with telegram / slack channels.
conf()["group_name_white_list"] = ["ALL_GROUP"]
conf()["single_chat_prefix"] = [""]
# ------------------------------------------------------------------
# Lifecycle
# ------------------------------------------------------------------
def startup(self):
self.bot_token = conf().get("discord_token", "")
if not self.bot_token:
err = "[Discord] discord_token is required"
logger.error(err)
self.report_startup_error(err)
return
try:
import discord
except ImportError:
err = (
"[Discord] discord.py is not installed. "
"Run: pip install discord.py"
)
logger.error(err)
self.report_startup_error(err)
return
# Run the asyncio event loop in a dedicated thread so the sync cow body
# is untouched.
self._loop = asyncio.new_event_loop()
def _run_loop():
asyncio.set_event_loop(self._loop)
try:
self._loop.run_until_complete(self._async_main(discord))
except Exception as e:
logger.error(f"[Discord] event loop crashed: {e}", exc_info=True)
self.report_startup_error(str(e))
finally:
try:
self._loop.close()
except Exception:
pass
logger.info("[Discord] event loop exited")
self._loop_thread = threading.Thread(target=_run_loop, daemon=True, name="discord-loop")
self._loop_thread.start()
# Block startup() until the loop thread exits, matching other channels'
# behaviour (startup is a blocking call).
self._loop_thread.join()
async def _async_main(self, discord):
"""Build the discord client, register handlers, and connect to the Gateway."""
# message_content is a privileged intent; it must be enabled in the
# Developer Portal (Bot -> Privileged Gateway Intents) to read text.
intents = discord.Intents.default()
intents.message_content = True
client = discord.Client(intents=intents)
self._client = client
channel = self
@client.event
async def on_ready():
channel.bot_user_id = str(client.user.id)
channel.bot_username = client.user.name or ""
channel.name = channel.bot_user_id # ChatChannel uses self.name to strip @-mention
logger.info(f"[Discord] Bot logged in as {client.user} (id={client.user.id})")
channel.report_startup_success()
logger.info("[Discord] ✅ Discord bot ready, listening for messages")
@client.event
async def on_message(message):
await channel._on_message(message)
# Connect to the Gateway; discord.py auto-reconnects on transient errors.
logger.info("[Discord] Connecting to Gateway...")
# client.start() handles login + Gateway connection and runs until
# close(); it is the standard entrypoint across discord.py versions.
runner_task = asyncio.create_task(client.start(self.bot_token))
# Block until stop()
try:
while not self._stop_event.is_set():
if runner_task.done():
# Surface a startup/connection failure (e.g. bad token)
exc = runner_task.exception()
if exc:
logger.error(f"[Discord] client stopped: {exc}", exc_info=exc)
self.report_startup_error(str(exc))
break
await asyncio.sleep(0.5)
finally:
try:
if not client.is_closed():
await client.close()
except Exception as e:
logger.warning(f"[Discord] shutdown error: {e}")
def stop(self):
logger.info("[Discord] stop() called")
self._stop_event.set()
if self._loop_thread and self._loop_thread.is_alive():
try:
self._loop_thread.join(timeout=10)
except Exception:
pass
logger.info("[Discord] stop() completed")
# ------------------------------------------------------------------
# Inbound: discord message -> ChatMessage -> ChatChannel.produce
# ------------------------------------------------------------------
async def _on_message(self, message):
"""Discord message entry: parse -> build ChatMessage -> produce()."""
try:
# Ignore our own messages and other bots. self._client.user may be
# None until on_ready completes, so guard against that.
if self._client and self._client.user and message.author.id == self._client.user.id:
return
if message.author.bot:
return
# Idempotent dedup
msg_uid = f"{message.channel.id}:{message.id}"
if self._received_msgs.get(msg_uid):
return
self._received_msgs[msg_uid] = True
# guild is None for DMs
is_group = message.guild is not None
# Guild trigger gate (silently drop if not triggered)
if is_group and not self._should_reply_in_guild(message):
logger.debug(f"[Discord] guild message not triggered (need @mention or reply), skip")
return
# Parse message type + download attachments if needed.
ctype, content, caption = await self._parse_message(message)
if ctype is None:
logger.debug(f"[Discord] unsupported message type, skip. msg_id={message.id}")
return
# Strip the bot mention from guild text/caption
if is_group:
if ctype == ContextType.TEXT and content:
content = self._strip_at_mention(content)
if caption:
caption = self._strip_at_mention(caption)
dc_msg = DiscordMessage(
message,
is_group=is_group,
bot_user_id=self.bot_user_id,
ctype=ctype,
content=content,
)
dc_msg.is_at = is_group # if we reached here in a guild, bot is mentioned/replied
from channel.file_cache import get_file_cache
file_cache = get_file_cache()
session_id = self._compute_session_id(message, is_group)
# Media + caption together: treat as a complete query and bypass the cache
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
tag = "image" if ctype == ContextType.IMAGE else "file"
merged_text = f"{caption}\n[{tag}: {content}]"
dc_msg.ctype = ContextType.TEXT
dc_msg.content = merged_text
ctype = ContextType.TEXT
logger.info(f"[Discord] Media+caption merged for session {session_id}")
# fallthrough to the TEXT branch below
elif ctype == ContextType.IMAGE:
file_cache.add(session_id, content, file_type="image")
logger.info(f"[Discord] Image cached for session {session_id}, waiting for query...")
return
elif ctype == ContextType.FILE:
file_cache.add(session_id, content, file_type="file")
logger.info(f"[Discord] File cached for session {session_id}: {content}")
return
if ctype == ContextType.TEXT:
# Fast-path: /cancel mirrors Web channel behaviour
if (content or "").strip().lower() in ("/cancel", "cancel"):
await self._do_cancel(session_id, message)
return
cached_files = file_cache.get(session_id)
if cached_files:
refs = []
for fi in cached_files:
ftype = fi["type"]
tag = ftype if ftype in ("image", "video") else "file"
refs.append(f"[{tag}: {fi['path']}]")
dc_msg.content = (dc_msg.content or "") + "\n" + "\n".join(refs)
file_cache.clear(session_id)
logger.info(f"[Discord] Attached {len(cached_files)} cached file(s) to query")
context = self._compose_context(
dc_msg.ctype,
dc_msg.content,
isgroup=is_group,
msg=dc_msg,
# Replies use Discord's reply mechanism, no manual @mention needed
no_need_at=True,
)
if context:
context["session_id"] = session_id
context["receiver"] = str(message.channel.id)
context["discord_channel_id"] = message.channel.id
context["discord_reply_to_msg_id"] = message.id if is_group else None
self.produce(context)
logger.debug(f"[Discord] received: type={ctype}, content={str(dc_msg.content)[:80]}")
except Exception as e:
logger.error(f"[Discord] _on_message error: {e}", exc_info=True)
async def _do_cancel(self, session_id: str, message):
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
try:
from agent.protocol import get_cancel_registry
cancelled = get_cancel_registry().cancel_session(session_id)
text = "Current task cancelled." if cancelled else "No running task to cancel."
await message.channel.send(text)
logger.info(f"[Discord] /cancel session={session_id}, cancelled={cancelled}")
except Exception as e:
logger.error(f"[Discord] /cancel error: {e}", exc_info=True)
async def _parse_message(self, message):
"""Parse a discord message and return (ctype, content, caption).
- content is text for ContextType.TEXT, otherwise the local file path
- caption is the optional text accompanying an attachment; empty for plain text
"""
text = (message.content or "").strip()
attachments = message.attachments or []
if attachments:
# Handle the first attachment; caption is the accompanying message text
att = attachments[0]
content_type = (att.content_type or "").lower()
name = att.filename or str(att.id)
path = await self._download_attachment(att, name)
if not path:
return (None, None, "")
is_image = content_type.startswith("image/") or name.lower().endswith(
(".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp")
)
if is_image:
return (ContextType.IMAGE, path, text)
return (ContextType.FILE, path, text)
if text:
return (ContextType.TEXT, text, "")
return (None, None, "")
async def _download_attachment(self, attachment, name: str):
"""Download a discord attachment into the local tmp dir; return path or None."""
try:
tmp_dir = DiscordMessage.get_tmp_dir()
safe_name = re.sub(r"[^\w.\-]", "_", name)
# Prefix with attachment id to avoid name collisions
local_path = os.path.join(tmp_dir, f"{attachment.id}_{safe_name}")
await attachment.save(local_path)
logger.debug(f"[Discord] downloaded {name} -> {local_path}")
return local_path
except Exception as e:
logger.error(f"[Discord] download_attachment failed ({name}): {e}")
return None
# ------------------------------------------------------------------
# Guild trigger logic
# ------------------------------------------------------------------
def _should_reply_in_guild(self, message) -> bool:
"""Decide whether to reply to a guild channel message based on configuration."""
mode = conf().get("discord_group_trigger", "mention_or_reply")
if mode == "all":
return True
# self._client.user may be None until on_ready completes
if not self._client or not self._client.user:
return False
# 1) Mentioned (direct @bot, not @everyone / @role)
if self._client.user in message.mentions:
return True
# 2) Reply to a bot message
if mode == "mention_or_reply":
ref = message.reference
resolved = getattr(ref, "resolved", None) if ref else None
if resolved and getattr(resolved, "author", None):
if resolved.author.id == self._client.user.id:
return True
return False
def _strip_at_mention(self, content: str) -> str:
"""Strip <@BOT_ID> / <@!BOT_ID> from guild text."""
if not content or not self.bot_user_id:
return content
pattern = re.compile(r"<@!?" + re.escape(self.bot_user_id) + r">")
return pattern.sub("", content).strip()
@staticmethod
def _compute_session_id(message, is_group: bool) -> str:
channel_id = message.channel.id
user_id = message.author.id
if is_group:
if conf().get("group_shared_session", True):
return f"discord_channel_{channel_id}"
return f"discord_channel_{channel_id}_{user_id}"
return f"discord_user_{user_id}"
# ------------------------------------------------------------------
# Override _compose_context: skip the parent's group whitelist/at checks
# (already handled via _should_reply_in_guild). Same idea as telegram / slack.
# ------------------------------------------------------------------
def _compose_context(self, ctype: ContextType, content, **kwargs):
context = Context(ctype, content)
context.kwargs = kwargs
if "channel_type" not in context:
context["channel_type"] = self.channel_type
if "origin_ctype" not in context:
context["origin_ctype"] = ctype
cmsg = context["msg"]
if cmsg.is_group:
if conf().get("group_shared_session", True):
context["session_id"] = cmsg.other_user_id
else:
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
else:
context["session_id"] = cmsg.from_user_id
context["receiver"] = cmsg.other_user_id
if ctype == ContextType.TEXT:
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
if img_match_prefix:
content = content.replace(img_match_prefix, "", 1)
context.type = ContextType.IMAGE_CREATE
else:
context.type = ContextType.TEXT
context.content = (content or "").strip()
if "desire_rtype" not in context and conf().get("always_reply_voice"):
context["desire_rtype"] = ReplyType.VOICE
elif ctype == ContextType.VOICE:
if "desire_rtype" not in context and (
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
):
context["desire_rtype"] = ReplyType.VOICE
return context
# ------------------------------------------------------------------
# Outbound: ChatChannel.send -> Discord Gateway/REST
# ------------------------------------------------------------------
def send(self, reply: Reply, context: Context):
"""Called from cow's sync main thread; marshal the coroutine onto the loop thread."""
if self._loop is None or self._client is None:
logger.warning("[Discord] client not ready, drop reply")
return
channel_id = context.get("discord_channel_id")
if channel_id is None:
logger.warning("[Discord] no discord_channel_id in context, drop reply")
return
coro = self._async_send(reply, channel_id)
try:
future = asyncio.run_coroutine_threadsafe(coro, self._loop)
future.result(timeout=180)
except Exception as e:
logger.error(f"[Discord] send failed: {e}")
async def _async_send(self, reply: Reply, channel_id):
try:
import discord
channel = self._client.get_channel(channel_id)
if channel is None:
# Not in cache (e.g. DM channel); fetch it explicitly
channel = await self._client.fetch_channel(channel_id)
rtype = reply.type
content = reply.content
if rtype in (ReplyType.TEXT, ReplyType.INFO, ReplyType.ERROR):
text = str(content) if content is not None else ""
if not text:
return
for chunk in _split_text(text, DISCORD_MSG_LIMIT):
await channel.send(chunk)
elif rtype == ReplyType.IMAGE:
# Already a local BytesIO; send it directly
content.seek(0)
await channel.send(file=discord.File(content, filename="image.png"))
elif rtype == ReplyType.IMAGE_URL:
url = str(content)
if url.startswith("file://"):
local = url[7:]
await channel.send(file=discord.File(local))
else:
# Post the URL as text; Discord will unfurl it as an image preview
await channel.send(url)
elif rtype in (ReplyType.VOICE, ReplyType.FILE):
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
caption = getattr(reply, "text_content", None) or None
await channel.send(content=caption, file=discord.File(local))
else:
# Fallback: send as plain text
await channel.send(str(content))
logger.info(f"[Discord] sent reply (type={rtype}, channel={channel_id})")
except Exception as e:
logger.error(f"[Discord] _async_send error: {e}", exc_info=True)
def _split_text(text: str, limit: int):
"""Split long text preferring line breaks to keep markdown structure intact."""
if len(text) <= limit:
yield text
return
buf = []
size = 0
for line in text.splitlines(keepends=True):
if size + len(line) > limit and buf:
yield "".join(buf)
buf, size = [], 0
# Hard-split single lines that exceed the limit
while len(line) > limit:
yield line[:limit]
line = line[limit:]
buf.append(line)
size += len(line)
if buf:
yield "".join(buf)

View File

@@ -0,0 +1,60 @@
"""
Discord message adapter.
Convert a discord.py Message into cow's unified ChatMessage.
File downloads are NOT performed here; the channel layer downloads
attachments on demand inside the async event loop.
"""
import os
from bridge.context import ContextType
from channel.chat_message import ChatMessage
from common.utils import expand_path
from config import conf
class DiscordMessage(ChatMessage):
"""Wrap a discord.py Message into the unified ChatMessage."""
def __init__(self, message, is_group: bool = False, bot_user_id: str = "",
ctype: ContextType = ContextType.TEXT, content: str = ""):
super().__init__(message)
# Basic fields
self.msg_id = str(message.id)
self.create_time = int(message.created_at.timestamp()) if message.created_at else 0
self.ctype = ctype
self.content = content
author = message.author
channel = message.channel
# Sender / chat info
from_user_id = str(author.id)
from_user_nick = getattr(author, "display_name", None) or getattr(author, "name", None) or from_user_id
self.from_user_id = from_user_id
self.from_user_nickname = from_user_nick
self.to_user_id = bot_user_id or "discord_bot"
self.to_user_nickname = bot_user_id or "discord_bot"
self.is_group = is_group
if is_group:
# Guild channel: other_user_id = channel_id, actual_user_id = sender id
self.other_user_id = str(channel.id)
self.other_user_nickname = getattr(channel, "name", None) or str(channel.id)
self.actual_user_id = from_user_id
self.actual_user_nickname = from_user_nick
else:
# DM: use channel_id so replies go back to the same DM channel
self.other_user_id = str(channel.id)
self.other_user_nickname = from_user_nick
# Whether the bot was triggered by @-mention (set by channel layer)
self.is_at = False
@staticmethod
def get_tmp_dir() -> str:
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
tmp_dir = os.path.join(workspace_root, "tmp")
os.makedirs(tmp_dir, exist_ok=True)
return tmp_dir

View File

@@ -752,6 +752,9 @@ class FeiShuChanel(ChatChannel):
init_in_flight = [False]
# 一旦初始化失败就长期标记为 disabled本次回复不再尝试任何流式调用
disabled = [False]
# True after agent_cancelled: agent_end stops rewriting the card
# with stale final_response and just finalizes current content.
cancelled = [False]
lock = threading.Lock()
# ---- 异步推送队列 ----------------------------------------------------
@@ -1076,18 +1079,42 @@ class FeiShuChanel(ChatChannel):
message_id[0] = None
sequence[0] = 0
elif event_type == "agent_cancelled":
# Lock channel into "no-rewrite" mode: the subsequent
# agent_end's final_response is from the last *completed*
# turn (the user already saw it), so rewriting the card
# would duplicate it visually.
with lock:
cancelled[0] = True
elif event_type == "agent_end":
# 最终回复:用 final_response 覆盖当前流式卡片,然后关闭流式模式。
final_response = data.get("final_response", "")
if not final_response:
return
final_text = str(final_response)
# 标记 streamed 让 chat_channel 跳过 send()
context["feishu_streamed"] = True
with lock:
was_cancelled = cancelled[0]
has_card = card_id[0] is not None
init_busy = init_in_flight[0]
pending_text = current_text[0]
if was_cancelled:
# Cancelled path: finalize the in-flight card with
# partial output (or a short marker if empty); drop
# stale final_response to avoid duplicating last turn.
if has_card:
_drain_push_queue()
partial = (pending_text or "").rstrip()
final_text = partial or "_(已中止)_"
_stream_update_text(final_text)
_close_streaming_mode(final_text)
push_queue.put(None)
return
if not final_response:
return
final_text = str(final_response)
# 罕见情况agent_end 触发时还没创建过卡片(极快返回 / 没有
# message_update主动创建一张承载 final_text。

View File

@@ -0,0 +1 @@

View File

@@ -0,0 +1,506 @@
"""
Slack channel via Bolt for Python (Socket Mode).
Features:
- Direct message & channel chat (text / image / file)
- Channel trigger: @mention or reply in a thread the bot is in (configurable)
- /cancel fast-path matches Web channel behaviour
- Socket Mode: no public IP / callback URL required, works behind NAT
Implementation note:
slack_bolt's SocketModeHandler is blocking and runs its own background
threads. We start it in a dedicated thread so the rest of cow (sync) stays
untouched. Inbound events are dispatched onto cow's existing sync
ChatChannel.produce() pipeline; outbound send() calls the Slack Web API
client directly (it is sync-safe).
"""
import os
import re
import threading
import requests
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel, check_prefix
from channel.slack.slack_message import SlackMessage
from common.expired_dict import ExpiredDict
from common.log import logger
from common.singleton import singleton
from config import conf
@singleton
class SlackChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = []
def __init__(self):
super().__init__()
self.bot_token = ""
self.app_token = ""
self.bot_user_id = "" # used to strip @mention and ignore self messages
self._app = None
self._handler = None
self._client = None
self._loop_thread = None
# Idempotent dedup; Slack retries event delivery on slow ack
self._received_msgs = ExpiredDict(60 * 60 * 1)
# Disable group whitelist / prefix checks (we handle triggering ourselves
# in _should_reply_in_channel), aligned with telegram / feishu channels.
conf()["group_name_white_list"] = ["ALL_GROUP"]
conf()["single_chat_prefix"] = [""]
# ------------------------------------------------------------------
# Lifecycle
# ------------------------------------------------------------------
def startup(self):
self.bot_token = conf().get("slack_bot_token", "")
self.app_token = conf().get("slack_app_token", "")
if not self.bot_token or not self.app_token:
err = "[Slack] slack_bot_token and slack_app_token are both required"
logger.error(err)
self.report_startup_error(err)
return
# Guard against the common mistake of swapping the two tokens:
# bot token must start with xoxb-, app-level token with xapp-.
if not self.bot_token.startswith("xoxb-") or not self.app_token.startswith("xapp-"):
err = (
"[Slack] token type mismatch: slack_bot_token must start with 'xoxb-' "
"and slack_app_token must start with 'xapp-' (they look swapped)"
)
logger.error(err)
self.report_startup_error(err)
return
try:
from slack_bolt import App
from slack_bolt.adapter.socket_mode import SocketModeHandler
except ImportError:
err = (
"[Slack] slack_bolt is not installed. "
"Run: pip install slack_bolt"
)
logger.error(err)
self.report_startup_error(err)
return
try:
self._app = App(token=self.bot_token)
self._client = self._app.client
# Resolve our own bot user id (needed for @mention strip / self-ignore)
auth = self._client.auth_test()
self.bot_user_id = auth.get("user_id", "")
self.name = self.bot_user_id # ChatChannel uses self.name to strip @-mention
logger.info(f"[Slack] Bot logged in as user_id={self.bot_user_id}, team={auth.get('team')}")
except Exception as e:
err = f"[Slack] auth_test failed: {e}"
logger.error(err)
self.report_startup_error(err)
return
self._register_handlers()
self._handler = SocketModeHandler(self._app, self.app_token)
def _run():
try:
logger.info("[Slack] Starting Socket Mode connection...")
self.report_startup_success()
logger.info("[Slack] ✅ Slack bot ready, listening for events")
self._handler.start()
except Exception as e:
logger.error(f"[Slack] socket mode crashed: {e}", exc_info=True)
self.report_startup_error(str(e))
finally:
logger.info("[Slack] socket mode exited")
self._loop_thread = threading.Thread(target=_run, daemon=True, name="slack-socket")
self._loop_thread.start()
# Block startup() until the handler thread exits, matching other channels'
# behaviour (startup is a blocking call).
self._loop_thread.join()
def _register_handlers(self):
app = self._app
# app_mention: bot is @-mentioned in a channel
@app.event("app_mention")
def _on_app_mention(event, ack):
ack()
self._handle_event(event, is_group=True)
# message: DMs and channel messages (including thread replies)
@app.event("message")
def _on_message(event, ack):
ack()
self._handle_message_event(event)
def stop(self):
logger.info("[Slack] stop() called")
try:
if self._handler is not None:
self._handler.close()
except Exception as e:
logger.warning(f"[Slack] handler close error: {e}")
if self._loop_thread and self._loop_thread.is_alive():
try:
self._loop_thread.join(timeout=10)
except Exception:
pass
logger.info("[Slack] stop() completed")
# ------------------------------------------------------------------
# Inbound: slack event -> ChatMessage -> ChatChannel.produce
# ------------------------------------------------------------------
def _handle_message_event(self, event: dict):
"""Route a raw `message` event: skip bot/system noise, decide grouping."""
try:
logger.debug(
f"[Slack] message event: channel_type={event.get('channel_type')}, "
f"subtype={event.get('subtype')}, user={event.get('user')}, "
f"ts={event.get('ts')}, thread_ts={event.get('thread_ts')}"
)
# Ignore bot messages (including our own) and message edits/deletes
if event.get("bot_id") or event.get("subtype") in ("bot_message", "message_changed", "message_deleted"):
return
if event.get("user") == self.bot_user_id:
return
channel_type = event.get("channel_type", "")
# DM (im) is single chat; channel/group is group chat. app_mention
# already covers channel @-mentions, so for plain channel messages we
# only react when configured / thread-following.
is_group = channel_type in ("channel", "group", "mpim")
if is_group:
# app_mention handler covers explicit @bot; here we only handle
# follow-up replies in threads the bot participates in.
if not self._should_reply_in_channel(event):
return
self._handle_event(event, is_group=is_group)
except Exception as e:
logger.error(f"[Slack] _handle_message_event error: {e}", exc_info=True)
def _handle_event(self, event: dict, is_group: bool):
"""Parse event -> build SlackMessage -> produce()."""
try:
channel_id = event.get("channel", "")
ts = event.get("ts", "")
if not channel_id:
return
# Idempotent dedup
msg_uid = f"{channel_id}:{ts}"
if self._received_msgs.get(msg_uid):
return
self._received_msgs[msg_uid] = True
# Parse type + download media if needed.
ctype, content, caption = self._parse_event(event)
if ctype is None:
logger.debug(f"[Slack] unsupported message type, skip. event={event}")
return
# Strip <@bot_user_id> mention from channel text
if is_group and self.bot_user_id:
if ctype == ContextType.TEXT and content:
content = self._strip_at_mention(content)
if caption:
caption = self._strip_at_mention(caption)
slack_msg = SlackMessage(
event,
is_group=is_group,
bot_user_id=self.bot_user_id,
ctype=ctype,
content=content,
)
slack_msg.is_at = is_group # if we reached here in a channel, bot is mentioned/threaded
from channel.file_cache import get_file_cache
file_cache = get_file_cache()
session_id = self._compute_session_id(event, is_group)
# Media + caption together: treat as a complete query and bypass the cache
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
tag = "image" if ctype == ContextType.IMAGE else "file"
merged_text = f"{caption}\n[{tag}: {content}]"
slack_msg.ctype = ContextType.TEXT
slack_msg.content = merged_text
ctype = ContextType.TEXT
logger.info(f"[Slack] Media+caption merged for session {session_id}")
# fallthrough to the TEXT branch below
elif ctype == ContextType.IMAGE:
file_cache.add(session_id, content, file_type="image")
logger.info(f"[Slack] Image cached for session {session_id}, waiting for query...")
return
elif ctype == ContextType.FILE:
file_cache.add(session_id, content, file_type="file")
logger.info(f"[Slack] File cached for session {session_id}: {content}")
return
if ctype == ContextType.TEXT:
# Fast-path: /cancel mirrors Web channel behaviour
if (content or "").strip().lower() in ("/cancel", "cancel"):
self._do_cancel(session_id, channel_id, event)
return
cached_files = file_cache.get(session_id)
if cached_files:
refs = []
for fi in cached_files:
ftype = fi["type"]
tag = ftype if ftype in ("image", "video") else "file"
refs.append(f"[{tag}: {fi['path']}]")
slack_msg.content = (slack_msg.content or "") + "\n" + "\n".join(refs)
file_cache.clear(session_id)
logger.info(f"[Slack] Attached {len(cached_files)} cached file(s) to query")
# Reply in the originating thread when present, else start one on this msg
thread_ts = event.get("thread_ts") or ts
context = self._compose_context(
slack_msg.ctype,
slack_msg.content,
isgroup=is_group,
msg=slack_msg,
# Replies go back into the thread, no manual @mention needed
no_need_at=True,
)
if context:
context["session_id"] = session_id
context["receiver"] = channel_id
context["slack_channel"] = channel_id
context["slack_thread_ts"] = thread_ts if is_group else None
self.produce(context)
logger.debug(f"[Slack] received: type={ctype}, content={str(slack_msg.content)[:80]}")
except Exception as e:
logger.error(f"[Slack] _handle_event error: {e}", exc_info=True)
def _do_cancel(self, session_id: str, channel_id: str, event: dict):
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
try:
from agent.protocol import get_cancel_registry
cancelled = get_cancel_registry().cancel_session(session_id)
text = "Current task cancelled." if cancelled else "No running task to cancel."
thread_ts = event.get("thread_ts") or event.get("ts")
self._client.chat_postMessage(channel=channel_id, text=text, thread_ts=thread_ts)
logger.info(f"[Slack] /cancel session={session_id}, cancelled={cancelled}")
except Exception as e:
logger.error(f"[Slack] /cancel error: {e}", exc_info=True)
def _parse_event(self, event: dict):
"""Parse a slack event and return (ctype, content, caption).
- content is text for ContextType.TEXT, otherwise the local file path
- caption is the optional text accompanying a file; empty for plain text
"""
text = (event.get("text") or "").strip()
files = event.get("files") or []
if files:
# Handle the first attachment; caption is the accompanying message text
f = files[0]
mimetype = (f.get("mimetype") or "").lower()
url = f.get("url_private_download") or f.get("url_private")
name = f.get("name") or f.get("id") or "file"
if not url:
return (None, None, "")
path = self._download_file(url, name)
if not path:
return (None, None, "")
if mimetype.startswith("image/"):
return (ContextType.IMAGE, path, text)
return (ContextType.FILE, path, text)
if text:
return (ContextType.TEXT, text, "")
return (None, None, "")
def _download_file(self, url: str, name: str):
"""Download a Slack private file (requires bot token auth) to local tmp dir."""
try:
headers = {"Authorization": f"Bearer {self.bot_token}"}
resp = requests.get(url, headers=headers, timeout=60, stream=True)
resp.raise_for_status()
tmp_dir = SlackMessage.get_tmp_dir()
# Sanitize the name and keep it unique-ish via the url tail
safe_name = re.sub(r"[^\w.\-]", "_", name)
local_path = os.path.join(tmp_dir, safe_name)
with open(local_path, "wb") as fp:
for chunk in resp.iter_content(chunk_size=8192):
if chunk:
fp.write(chunk)
logger.debug(f"[Slack] downloaded {name} -> {local_path}")
return local_path
except Exception as e:
logger.error(f"[Slack] download_file failed ({name}): {e}")
return None
# ------------------------------------------------------------------
# Channel trigger logic
# ------------------------------------------------------------------
def _should_reply_in_channel(self, event: dict) -> bool:
"""Decide whether to reply to a plain channel message (no @mention).
app_mention already handles explicit @bot, so here we only deal with
follow-up messages. `all` replies to every message; `mention_or_reply`
replies inside threads the bot already participates in.
"""
mode = conf().get("slack_group_trigger", "mention_or_reply")
if mode == "all":
return True
if mode == "mention_only":
return False
# mention_or_reply: follow up only within an existing thread
return bool(event.get("thread_ts"))
def _strip_at_mention(self, content: str) -> str:
"""Strip <@BOT_USER_ID> from channel text."""
if not content or not self.bot_user_id:
return content
pattern = re.compile(r"<@" + re.escape(self.bot_user_id) + r">", re.IGNORECASE)
return pattern.sub("", content).strip()
@staticmethod
def _compute_session_id(event: dict, is_group: bool) -> str:
channel_id = event.get("channel", "")
user_id = event.get("user", "")
if is_group:
if conf().get("group_shared_session", True):
return f"slack_channel_{channel_id}"
return f"slack_channel_{channel_id}_{user_id}"
return f"slack_user_{user_id}"
# ------------------------------------------------------------------
# Override _compose_context: skip the parent's group whitelist/at checks
# (already handled via _should_reply_in_channel). Same idea as telegram.
# ------------------------------------------------------------------
def _compose_context(self, ctype: ContextType, content, **kwargs):
context = Context(ctype, content)
context.kwargs = kwargs
if "channel_type" not in context:
context["channel_type"] = self.channel_type
if "origin_ctype" not in context:
context["origin_ctype"] = ctype
cmsg = context["msg"]
if cmsg.is_group:
if conf().get("group_shared_session", True):
context["session_id"] = cmsg.other_user_id
else:
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
else:
context["session_id"] = cmsg.from_user_id
context["receiver"] = cmsg.other_user_id
if ctype == ContextType.TEXT:
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
if img_match_prefix:
content = content.replace(img_match_prefix, "", 1)
context.type = ContextType.IMAGE_CREATE
else:
context.type = ContextType.TEXT
context.content = (content or "").strip()
if "desire_rtype" not in context and conf().get("always_reply_voice"):
context["desire_rtype"] = ReplyType.VOICE
elif ctype == ContextType.VOICE:
if "desire_rtype" not in context and (
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
):
context["desire_rtype"] = ReplyType.VOICE
return context
# ------------------------------------------------------------------
# Outbound: ChatChannel.send -> Slack Web API
# ------------------------------------------------------------------
def send(self, reply: Reply, context: Context):
"""Called from cow's sync main thread; Slack Web client is sync-safe."""
if self._client is None:
logger.warning("[Slack] client not ready, drop reply")
return
channel_id = context.get("slack_channel")
thread_ts = context.get("slack_thread_ts")
if not channel_id:
logger.warning("[Slack] no slack_channel in context, drop reply")
return
try:
self._do_send(reply, channel_id, thread_ts)
logger.info(f"[Slack] sent reply (type={reply.type}, channel={channel_id})")
except Exception as e:
logger.error(f"[Slack] send failed: {e}", exc_info=True)
def _do_send(self, reply: Reply, channel_id: str, thread_ts):
rtype = reply.type
content = reply.content
if rtype in (ReplyType.TEXT, ReplyType.INFO, ReplyType.ERROR):
text = str(content) if content is not None else ""
if not text:
return
# Slack caps a message around 40k chars; split conservatively
for chunk in _split_text(text, 3500):
self._client.chat_postMessage(channel=channel_id, text=chunk, thread_ts=thread_ts)
elif rtype == ReplyType.IMAGE:
# Already a local BytesIO; upload it directly
content.seek(0)
self._client.files_upload_v2(
channel=channel_id, file=content, filename="image.png", thread_ts=thread_ts,
)
elif rtype == ReplyType.IMAGE_URL:
url = str(content)
if url.startswith("file://"):
local = url[7:]
self._client.files_upload_v2(
channel=channel_id, file=local, thread_ts=thread_ts,
)
else:
# Post the URL as text; Slack will unfurl it as an image preview
self._client.chat_postMessage(channel=channel_id, text=url, thread_ts=thread_ts)
elif rtype in (ReplyType.VOICE, ReplyType.FILE):
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
caption = getattr(reply, "text_content", None) or None
self._client.files_upload_v2(
channel=channel_id, file=local, initial_comment=caption, thread_ts=thread_ts,
)
else:
# Fallback: send as plain text
self._client.chat_postMessage(channel=channel_id, text=str(content), thread_ts=thread_ts)
def _split_text(text: str, limit: int):
"""Split long text preferring line breaks to keep markdown structure intact."""
if len(text) <= limit:
yield text
return
buf = []
size = 0
for line in text.splitlines(keepends=True):
if size + len(line) > limit and buf:
yield "".join(buf)
buf, size = [], 0
# Hard-split single lines that exceed the limit
while len(line) > limit:
yield line[:limit]
line = line[limit:]
buf.append(line)
size += len(line)
if buf:
yield "".join(buf)

View File

@@ -0,0 +1,60 @@
"""
Slack message adapter.
Convert a Slack event payload into cow's unified ChatMessage.
File downloads are NOT performed here; the channel layer downloads files
on demand because it needs the bot token for authenticated download URLs.
"""
import os
from bridge.context import ContextType
from channel.chat_message import ChatMessage
from common.utils import expand_path
from config import conf
class SlackMessage(ChatMessage):
"""Wrap a Slack event into the unified ChatMessage."""
def __init__(self, event: dict, is_group: bool = False, bot_user_id: str = "",
ctype: ContextType = ContextType.TEXT, content: str = ""):
super().__init__(event)
# Basic fields
self.msg_id = event.get("client_msg_id") or event.get("ts") or ""
try:
self.create_time = int(float(event.get("ts", 0)))
except (TypeError, ValueError):
self.create_time = 0
self.ctype = ctype
self.content = content
# Sender / chat info
from_user_id = event.get("user", "unknown")
channel_id = event.get("channel", "")
self.from_user_id = from_user_id
self.from_user_nickname = from_user_id
self.to_user_id = bot_user_id or "slack_bot"
self.to_user_nickname = bot_user_id or "slack_bot"
self.is_group = is_group
if is_group:
# Channel chat: other_user_id = channel_id, actual_user_id = sender id
self.other_user_id = channel_id
self.other_user_nickname = channel_id
self.actual_user_id = from_user_id
self.actual_user_nickname = from_user_id
else:
# DM: use channel_id so replies go back to the same DM channel
self.other_user_id = channel_id or from_user_id
self.other_user_nickname = from_user_id
# Whether the bot was triggered by @-mention (set by channel layer)
self.is_at = False
@staticmethod
def get_tmp_dir() -> str:
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
tmp_dir = os.path.join(workspace_root, "tmp")
os.makedirs(tmp_dir, exist_ok=True)
return tmp_dir

View File

View File

@@ -0,0 +1,719 @@
"""
Telegram channel via Bot API (long polling mode).
Features:
- Single chat & group chat (text / photo / voice / video / document)
- Group trigger: @mention or reply-to-bot (configurable)
- /cancel fast-path matches Web channel behaviour
- Auto-register bot commands menu on startup (mirrors Web slash menu)
- Optional HTTP/SOCKS5 proxy support for restricted networks
Implementation note:
python-telegram-bot is async-first. We run the bot inside a dedicated
thread with its own asyncio loop so the rest of cow (which is sync)
stays untouched. Inbound updates are dispatched onto cow's existing
sync ChatChannel.produce() pipeline; outbound send() schedules
coroutines back onto that loop via asyncio.run_coroutine_threadsafe.
"""
import asyncio
import os
import re
import threading
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel, check_prefix
from channel.telegram.telegram_message import TelegramMessage
from common.expired_dict import ExpiredDict
from common.log import logger
from common.singleton import singleton
from config import conf
# Bot command menu, aligned with Web slash commands.
# Top-level commands only; sub-commands are entered with a space (e.g. "/skill list").
TELEGRAM_BOT_COMMANDS = [
("help", "Show command help"),
("status", "Show running status"),
("context", "View/clear conversation context (sub: clear)"),
("skill", "Manage skills (list/search/install/...)"),
("memory", "Manage memory (sub: dream)"),
("knowledge", "Manage knowledge base (list/on/off)"),
("config", "Show current config"),
("cancel", "Cancel running agent task"),
("logs", "Show recent logs"),
("version", "Show version"),
]
@singleton
class TelegramChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = []
def __init__(self):
super().__init__()
self.bot_token = ""
self.bot_username = "" # used for @-mention matching
self._bot = None
self._application = None
self._loop = None
self._loop_thread = None
self._stop_event = threading.Event()
# Idempotent dedup; TG occasionally redelivers the same update on flaky networks
self._received_msgs = ExpiredDict(60 * 60 * 1)
# Disable group whitelist / prefix checks (we handle triggering ourselves
# in _should_reply_in_group), aligned with feishu / wecom_bot channels.
conf()["group_name_white_list"] = ["ALL_GROUP"]
conf()["single_chat_prefix"] = [""]
# ------------------------------------------------------------------
# Lifecycle
# ------------------------------------------------------------------
def startup(self):
self.bot_token = conf().get("telegram_token", "")
if not self.bot_token:
err = "[Telegram] telegram_token is required"
logger.error(err)
self.report_startup_error(err)
return
try:
from telegram.ext import (
Application,
MessageHandler,
CommandHandler,
filters,
)
except ImportError:
err = (
"[Telegram] python-telegram-bot is not installed. "
"Run: pip install python-telegram-bot"
)
logger.error(err)
self.report_startup_error(err)
return
# Run the asyncio event loop in a dedicated thread so the sync cow body
# is untouched.
self._loop = asyncio.new_event_loop()
def _run_loop():
asyncio.set_event_loop(self._loop)
try:
self._loop.run_until_complete(self._async_main(Application, MessageHandler, CommandHandler, filters))
except Exception as e:
logger.error(f"[Telegram] event loop crashed: {e}", exc_info=True)
self.report_startup_error(str(e))
finally:
try:
self._loop.close()
except Exception:
pass
logger.info("[Telegram] event loop exited")
self._loop_thread = threading.Thread(target=_run_loop, daemon=True, name="telegram-loop")
self._loop_thread.start()
# Block startup() until the loop thread exits, matching other channels'
# behaviour (startup is a blocking call).
self._loop_thread.join()
async def _async_main(self, Application, MessageHandler, CommandHandler, filters):
"""Build Application, register handlers, and run polling."""
builder = Application.builder().token(self.bot_token)
# Proxy: prefer telegram_proxy config, fall back to HTTPS_PROXY env var
proxy_url = conf().get("telegram_proxy", "") or os.environ.get("HTTPS_PROXY", "")
if proxy_url:
try:
builder = builder.proxy(proxy_url).get_updates_proxy(proxy_url)
logger.info(f"[Telegram] using proxy: {proxy_url}")
except Exception as e:
logger.warning(f"[Telegram] proxy config failed, fallback to direct: {e}")
# Media uploads (photo/voice/video/document) over a proxy can be slow,
# bump read/write/connect/pool timeouts.
builder = (
builder
.read_timeout(60)
.write_timeout(120)
.connect_timeout(30)
.pool_timeout(30)
)
application = builder.build()
self._application = application
self._bot = application.bot
# Fetch our own username (needed for @-mention matching in groups)
try:
me = await self._bot.get_me()
self.bot_username = me.username or ""
self.name = self.bot_username # ChatChannel uses self.name to strip @-mention
logger.info(f"[Telegram] Bot logged in as @{self.bot_username} (id={me.id})")
except Exception as e:
err = f"[Telegram] get_me failed: {e}"
logger.error(err)
self.report_startup_error(err)
return
# Register the command menu (failure is non-fatal)
if conf().get("telegram_register_commands", True):
try:
from telegram import BotCommand
cmds = [BotCommand(name, desc) for name, desc in TELEGRAM_BOT_COMMANDS]
await self._bot.set_my_commands(cmds)
logger.info(f"[Telegram] Registered {len(cmds)} bot commands")
except Exception as e:
logger.warning(f"[Telegram] set_my_commands failed: {e}")
# Handlers:
# 1) /cancel uses the fast-path
application.add_handler(CommandHandler("cancel", self._on_cancel))
# 2) Normal messages (text + media)
application.add_handler(MessageHandler(filters.ALL & ~filters.COMMAND, self._on_message))
# 3) Other slash commands are forwarded as plain text for the agent to handle
application.add_handler(MessageHandler(filters.COMMAND, self._on_command_passthrough))
# Start polling. drop_pending_updates avoids replaying backlog after restart.
# Transient "Server disconnected" / RemoteProtocolError during get_updates
# are common over proxies/flaky networks; PTB's network loop auto-retries,
# so we only need to keep the noise down (see _quiet_polling_network_errors).
self._quiet_polling_network_errors()
logger.info("[Telegram] Starting long polling...")
await application.initialize()
await application.start()
await application.updater.start_polling(
drop_pending_updates=True,
# Long-poll hold time on the server side; smaller value = reconnect more
# often but each hung connection fails faster.
timeout=30,
# Retry forever on transient get_updates network errors instead of giving up.
bootstrap_retries=-1,
)
self.report_startup_success()
logger.info("[Telegram] ✅ Telegram bot ready, polling for updates")
# Block until stop()
try:
while not self._stop_event.is_set():
await asyncio.sleep(0.5)
finally:
try:
await application.updater.stop()
await application.stop()
await application.shutdown()
except Exception as e:
logger.warning(f"[Telegram] shutdown error: {e}")
@staticmethod
def _quiet_polling_network_errors():
"""Downgrade PTB's noisy 'Exception happened while polling for updates' logs.
These transient get_updates errors (RemoteProtocolError / NetworkError /
TimedOut, typically over a proxy) are auto-retried by PTB's network loop,
so logging the full traceback at ERROR is just noise. We attach a filter
that drops these specific records while leaving real errors untouched.
"""
import logging
class _PollingNoiseFilter(logging.Filter):
_NEEDLES = (
"Exception happened while polling for updates",
"Server disconnected without sending a response",
)
def filter(self, record: logging.LogRecord) -> bool:
try:
msg = record.getMessage()
except Exception:
return True
if any(n in msg for n in self._NEEDLES):
# Keep a single-line breadcrumb at DEBUG, drop the traceback.
logger.debug(f"[Telegram] transient polling network error (auto-retrying): {msg.splitlines()[0]}")
return False
return True
noise_filter = _PollingNoiseFilter()
for name in ("telegram.ext.Updater", "telegram.ext._updater", "telegram.ext"):
logging.getLogger(name).addFilter(noise_filter)
def stop(self):
logger.info("[Telegram] stop() called")
self._stop_event.set()
if self._loop_thread and self._loop_thread.is_alive():
try:
self._loop_thread.join(timeout=10)
except Exception:
pass
logger.info("[Telegram] stop() completed")
# ------------------------------------------------------------------
# Inbound: telegram update -> ChatMessage -> ChatChannel.produce
# ------------------------------------------------------------------
async def _on_cancel(self, update, _context):
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
try:
from agent.protocol import get_cancel_registry
session_id = self._compute_session_id(update)
cancelled = get_cancel_registry().cancel_session(session_id)
text = "Current task cancelled." if cancelled else "No running task to cancel."
await update.effective_message.reply_text(text)
logger.info(f"[Telegram] /cancel session={session_id}, cancelled={cancelled}")
except Exception as e:
logger.error(f"[Telegram] /cancel error: {e}", exc_info=True)
try:
await update.effective_message.reply_text(f"⚠️ /cancel failed: {e}")
except Exception:
pass
async def _on_command_passthrough(self, update, _context):
"""All non-/cancel commands fall through to plain message handling."""
await self._on_message(update, _context)
async def _on_message(self, update, _context):
"""Telegram update entry: parse message -> build ChatMessage -> produce()."""
try:
message = update.effective_message
chat = update.effective_chat
if not message or not chat:
return
# Idempotent dedup
msg_uid = f"{chat.id}:{message.message_id}"
if self._received_msgs.get(msg_uid):
return
self._received_msgs[msg_uid] = True
is_group = chat.type in ("group", "supergroup")
# Debug log: helpful when group messages are silently dropped
if is_group:
logger.debug(
f"[Telegram] group update received: chat_id={chat.id}, "
f"text={(message.text or message.caption or '')[:40]!r}, "
f"reply_to_bot={bool(message.reply_to_message and message.reply_to_message.from_user and message.reply_to_message.from_user.username == self.bot_username)}"
)
# Group trigger gate (silently drop if not triggered)
if is_group and not self._should_reply_in_group(update):
logger.debug(f"[Telegram] group message not triggered (need @{self.bot_username} or reply), skip")
return
# Parse message type + download media if needed.
# Media messages with caption return both the local path and the caption text.
ctype, content, caption = await self._parse_message(message)
if ctype is None:
logger.debug(f"[Telegram] unsupported message type, skip. msg={message}")
return
# Strip @bot mention for group text/caption
if is_group and self.bot_username:
if ctype == ContextType.TEXT and content:
content = self._strip_at_mention(content)
if caption:
caption = self._strip_at_mention(caption)
tg_msg = TelegramMessage(
update,
is_group=is_group,
bot_username=self.bot_username,
ctype=ctype,
content=content,
)
tg_msg.is_at = is_group # If we got here in a group, the bot is mentioned/replied
# File cache: standalone media goes into cache, the next text query attaches them
from channel.file_cache import get_file_cache
file_cache = get_file_cache()
session_id = self._compute_session_id(update)
# Media + caption together: treat as a complete query and bypass the cache
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
tag = "image" if ctype == ContextType.IMAGE else "file"
merged_text = f"{caption}\n[{tag}: {content}]"
tg_msg.ctype = ContextType.TEXT
tg_msg.content = merged_text
ctype = ContextType.TEXT
logger.info(f"[Telegram] Media+caption merged for session {session_id}")
# fallthrough to the TEXT branch below
elif ctype == ContextType.IMAGE:
file_cache.add(session_id, content, file_type="image")
logger.info(f"[Telegram] Image cached for session {session_id}, waiting for query...")
return
elif ctype == ContextType.FILE:
file_cache.add(session_id, content, file_type="file")
logger.info(f"[Telegram] File cached for session {session_id}: {content}")
return
if ctype == ContextType.TEXT:
cached_files = file_cache.get(session_id)
if cached_files:
refs = []
for fi in cached_files:
ftype = fi["type"]
tag = ftype if ftype in ("image", "video") else "file"
refs.append(f"[{tag}: {fi['path']}]")
tg_msg.content = (tg_msg.content or "") + "\n" + "\n".join(refs)
file_cache.clear(session_id)
logger.info(f"[Telegram] Attached {len(cached_files)} cached file(s) to query")
# Dispatch to cow main pipeline (reuses ChatChannel._compose_context routing)
context = self._compose_context(
tg_msg.ctype,
tg_msg.content,
isgroup=is_group,
msg=tg_msg,
)
if context:
context["session_id"] = session_id
context["receiver"] = str(chat.id)
context["telegram_chat_id"] = chat.id
context["telegram_reply_to_msg_id"] = message.message_id if is_group else None
self.produce(context)
logger.debug(f"[Telegram] received: type={ctype}, content={str(tg_msg.content)[:80]}")
except Exception as e:
logger.error(f"[Telegram] _on_message error: {e}", exc_info=True)
async def _parse_message(self, message):
"""Parse a telegram message and return (ctype, content, caption).
- content is text for ContextType.TEXT, otherwise the local file path
- caption is the optional text accompanying a media message; empty for plain text
"""
caption = (message.caption or "").strip()
if message.photo:
largest = message.photo[-1]
path = await self._download_file(largest.file_id, suffix=".jpg")
return (ContextType.IMAGE, path, caption) if path else (None, None, "")
if message.voice or message.audio:
audio_obj = message.voice or message.audio
suffix = ".ogg" if message.voice else (
"." + (audio_obj.mime_type.split("/")[-1] if getattr(audio_obj, "mime_type", "") else "mp3")
)
path = await self._download_file(audio_obj.file_id, suffix=suffix)
return (ContextType.VOICE, path, caption) if path else (None, None, "")
if message.video or message.video_note:
video_obj = message.video or message.video_note
path = await self._download_file(video_obj.file_id, suffix=".mp4")
return (ContextType.FILE, path, caption) if path else (None, None, "")
if message.document:
doc = message.document
ext = ""
if doc.file_name and "." in doc.file_name:
ext = "." + doc.file_name.rsplit(".", 1)[-1]
path = await self._download_file(doc.file_id, suffix=ext, original_name=doc.file_name)
if not path:
return (None, None, "")
# Image-typed documents (user picked "send as file") are treated as images
mime = (doc.mime_type or "").lower()
if mime.startswith("image/"):
return (ContextType.IMAGE, path, caption)
return (ContextType.FILE, path, caption)
if message.text:
return (ContextType.TEXT, message.text.strip(), "")
return (None, None, "")
async def _download_file(self, file_id: str, suffix: str = "", original_name: str = ""):
"""Download via bot.get_file into the local tmp dir; return path or None on failure."""
try:
f = await self._bot.get_file(file_id)
tmp_dir = TelegramMessage.get_tmp_dir()
base = original_name or f"{file_id}{suffix or ''}"
# Prefix with file_id to avoid name collisions / weird chars
safe_name = f"{file_id}_{base}" if original_name else base
local_path = os.path.join(tmp_dir, safe_name)
await f.download_to_drive(custom_path=local_path)
logger.debug(f"[Telegram] downloaded file_id={file_id} -> {local_path}")
return local_path
except Exception as e:
logger.error(f"[Telegram] download_file failed (file_id={file_id}): {e}")
return None
# ------------------------------------------------------------------
# Group trigger logic
# ------------------------------------------------------------------
def _should_reply_in_group(self, update) -> bool:
"""Decide whether to reply to a group message based on configuration."""
mode = conf().get("telegram_group_trigger", "mention_or_reply")
if mode == "all":
return True
message = update.effective_message
if not message:
return False
# 1) Mentioned
if self.bot_username and self._is_mentioned(message, self.bot_username):
return True
# 2) Reply to a bot message
if mode == "mention_or_reply":
reply = message.reply_to_message
if reply and reply.from_user and reply.from_user.username == self.bot_username:
return True
return False
@staticmethod
def _is_mentioned(message, bot_username: str) -> bool:
"""Check whether entities/caption_entities contain a @mention of the bot."""
bot_at = "@" + bot_username.lower()
text = (message.text or message.caption or "").lower()
if bot_at in text:
return True
# Also check entities strictly to support text_mention (no-username @)
for ent in (message.entities or []) + (message.caption_entities or []):
if ent.type == "mention":
src = message.text or message.caption or ""
if src[ent.offset: ent.offset + ent.length].lower() == bot_at:
return True
return False
def _strip_at_mention(self, content: str) -> str:
"""Strip @bot_username from group text (case-insensitive)."""
if not content or not self.bot_username:
return content
pattern = re.compile(r"@" + re.escape(self.bot_username), re.IGNORECASE)
return pattern.sub("", content).strip()
@staticmethod
def _compute_session_id(update) -> str:
chat = update.effective_chat
user = update.effective_user
is_group = chat.type in ("group", "supergroup")
if is_group:
if conf().get("group_shared_session", True):
return f"tg_group_{chat.id}"
return f"tg_group_{chat.id}_{user.id}"
return f"tg_user_{user.id}"
# ------------------------------------------------------------------
# Override _compose_context: skip the parent's group whitelist/at checks
# (already handled in _on_message via _should_reply_in_group). Same idea
# as the feishu channel.
# ------------------------------------------------------------------
def _compose_context(self, ctype: ContextType, content, **kwargs):
context = Context(ctype, content)
context.kwargs = kwargs
if "channel_type" not in context:
context["channel_type"] = self.channel_type
if "origin_ctype" not in context:
context["origin_ctype"] = ctype
cmsg = context["msg"]
if cmsg.is_group:
if conf().get("group_shared_session", True):
context["session_id"] = cmsg.other_user_id
else:
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
else:
context["session_id"] = cmsg.from_user_id
context["receiver"] = cmsg.other_user_id
if ctype == ContextType.TEXT:
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
if img_match_prefix:
content = content.replace(img_match_prefix, "", 1)
context.type = ContextType.IMAGE_CREATE
else:
context.type = ContextType.TEXT
context.content = (content or "").strip()
if "desire_rtype" not in context and conf().get("always_reply_voice"):
context["desire_rtype"] = ReplyType.VOICE
elif ctype == ContextType.VOICE:
if "desire_rtype" not in context and (
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
):
context["desire_rtype"] = ReplyType.VOICE
return context
# ------------------------------------------------------------------
# Outbound: ChatChannel.send -> Telegram API
# ------------------------------------------------------------------
def send(self, reply: Reply, context: Context):
"""Called from cow's sync main thread; we marshal the coroutine onto the loop thread."""
if self._loop is None or self._bot is None:
logger.warning("[Telegram] bot not ready, drop reply")
return
chat_id = context.get("telegram_chat_id")
reply_to = context.get("telegram_reply_to_msg_id")
if chat_id is None:
logger.warning("[Telegram] no telegram_chat_id in context, drop reply")
return
coro = self._async_send(reply, chat_id, reply_to)
try:
future = asyncio.run_coroutine_threadsafe(coro, self._loop)
# Media uploads through a proxy can be slow; let PTB's own timeouts win
future.result(timeout=180)
except Exception as e:
logger.error(f"[Telegram] send failed: {e}")
# Number of retries for transient network errors (proxy hiccups etc.)
_SEND_RETRIES = 2
_SEND_RETRY_BACKOFF = 2.0 # seconds
async def _send_with_retry(self, send_fn, *, label: str):
"""Run a single Telegram API call with retries for transient network errors."""
from telegram.error import NetworkError, TimedOut
last_err = None
for attempt in range(self._SEND_RETRIES + 1):
try:
return await send_fn()
except (NetworkError, TimedOut) as e:
last_err = e
if attempt >= self._SEND_RETRIES:
break
wait = self._SEND_RETRY_BACKOFF * (attempt + 1)
logger.warning(
f"[Telegram] {label} transient error (attempt {attempt + 1}/"
f"{self._SEND_RETRIES + 1}): {e}; retry in {wait}s"
)
await asyncio.sleep(wait)
raise last_err
async def _async_send(self, reply: Reply, chat_id, reply_to_msg_id):
try:
rtype = reply.type
content = reply.content
if rtype == ReplyType.TEXT or rtype == ReplyType.INFO or rtype == ReplyType.ERROR:
# Telegram caps a single text message at 4096 chars; auto-split
text = str(content) if content is not None else ""
if not text:
return
for chunk in _split_text(text, 4000):
await self._send_with_retry(
lambda c=chunk: self._bot.send_message(
chat_id=chat_id,
text=c,
reply_to_message_id=reply_to_msg_id,
# Avoid failing the whole send if reply_to was deleted
allow_sending_without_reply=True,
),
label="send_message",
)
elif rtype == ReplyType.IMAGE:
# Already a local BytesIO; send it directly
content.seek(0)
await self._send_with_retry(
lambda: self._bot.send_photo(
chat_id=chat_id,
photo=content,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
),
label="send_photo",
)
elif rtype == ReplyType.IMAGE_URL:
url = str(content)
if url.startswith("file://"):
local = url[7:]
# Open inside the lambda so each retry gets a fresh stream
async def _send_local_photo():
with open(local, "rb") as f:
return await self._bot.send_photo(
chat_id=chat_id, photo=f,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
)
await self._send_with_retry(_send_local_photo, label="send_photo(file)")
else:
await self._send_with_retry(
lambda: self._bot.send_photo(
chat_id=chat_id, photo=url,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
),
label="send_photo(url)",
)
elif rtype == ReplyType.VOICE:
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
async def _send_voice():
with open(local, "rb") as f:
return await self._bot.send_voice(
chat_id=chat_id, voice=f,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
)
await self._send_with_retry(_send_voice, label="send_voice")
elif rtype == ReplyType.FILE:
# Videos go through send_video, everything else through send_document
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
# File replies may carry an accompanying text caption
caption = getattr(reply, "text_content", None) or None
is_video = isinstance(local, str) and local.lower().endswith(
(".mp4", ".mov", ".avi", ".mkv", ".webm")
)
async def _send_file():
with open(local, "rb") as f:
if is_video:
return await self._bot.send_video(
chat_id=chat_id, video=f, caption=caption,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
)
return await self._bot.send_document(
chat_id=chat_id, document=f, caption=caption,
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
)
await self._send_with_retry(_send_file, label="send_video" if is_video else "send_document")
else:
# Fallback: send as plain text
await self._send_with_retry(
lambda: self._bot.send_message(
chat_id=chat_id, text=str(content),
reply_to_message_id=reply_to_msg_id,
allow_sending_without_reply=True,
),
label="send_message(fallback)",
)
logger.info(f"[Telegram] sent reply (type={rtype}, chat_id={chat_id})")
except Exception as e:
logger.error(f"[Telegram] _async_send error: {e}", exc_info=True)
def _split_text(text: str, limit: int):
"""Split long text preferring line breaks to keep markdown structure intact."""
if len(text) <= limit:
yield text
return
buf = []
size = 0
for line in text.splitlines(keepends=True):
if size + len(line) > limit and buf:
yield "".join(buf)
buf, size = [], 0
# Hard-split single lines that exceed the limit
while len(line) > limit:
yield line[:limit]
line = line[limit:]
buf.append(line)
size += len(line)
if buf:
yield "".join(buf)

View File

@@ -0,0 +1,62 @@
"""
Telegram message adapter.
Convert a python-telegram-bot Update into cow's unified ChatMessage.
File downloads are NOT performed here; the channel layer triggers
bot.get_file() on demand because it requires the async event loop.
"""
import os
from bridge.context import ContextType
from channel.chat_message import ChatMessage
from common.utils import expand_path
from config import conf
class TelegramMessage(ChatMessage):
"""Wrap a Telegram Update into the unified ChatMessage."""
def __init__(self, update, is_group: bool = False, bot_username: str = "",
ctype: ContextType = ContextType.TEXT, content: str = ""):
super().__init__(update)
message = update.effective_message
chat = update.effective_chat
user = update.effective_user
# Basic fields
self.msg_id = str(message.message_id) if message else ""
self.create_time = int(message.date.timestamp()) if message and message.date else 0
self.ctype = ctype
self.content = content
# Sender / chat info
from_user_id = str(user.id) if user else "unknown"
from_user_nick = (
user.full_name if user and user.full_name else (user.username if user else "unknown")
)
self.from_user_id = from_user_id
self.from_user_nickname = from_user_nick or from_user_id
self.to_user_id = bot_username or "telegram_bot"
self.to_user_nickname = bot_username or "telegram_bot"
self.is_group = is_group
if is_group:
# Group: other_user_id = group_id, actual_user_id = sender id
self.other_user_id = str(chat.id)
self.other_user_nickname = chat.title or str(chat.id)
self.actual_user_id = from_user_id
self.actual_user_nickname = self.from_user_nickname
else:
self.other_user_id = from_user_id
self.other_user_nickname = self.from_user_nickname
# Whether the bot was triggered by @-mention or reply (set by channel layer)
self.is_at = False
@staticmethod
def get_tmp_dir() -> str:
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
tmp_dir = os.path.join(workspace_root, "tmp")
os.makedirs(tmp_dir, exist_ok=True)
return tmp_dir

View File

@@ -1,4 +1,7 @@
import json
import os
import sys
import time
from bridge.context import *
from bridge.reply import Reply, ReplyType
@@ -8,6 +11,164 @@ from common.log import logger
from config import conf
class _Style:
"""ANSI escape codes for terminal styling. Disabled when not a tty."""
enabled = sys.stdout.isatty()
RESET = "\033[0m"
BOLD = "\033[1m"
DIM = "\033[2m"
ITALIC = "\033[3m"
GRAY = "\033[90m"
RED = "\033[31m"
GREEN = "\033[32m"
YELLOW = "\033[33m"
BLUE = "\033[34m"
MAGENTA = "\033[35m"
CYAN = "\033[36m"
@classmethod
def wrap(cls, text, *codes):
if not cls.enabled or not codes:
return text
return "".join(codes) + text + cls.RESET
class TerminalAgentRenderer:
"""Render agent stream events to the terminal in real time.
Reuses the same `on_event` mechanism as the web channel so the terminal
can show reasoning, tool calls and streaming answer text just like the web UI.
"""
def __init__(self):
self._reasoning_active = False
self._answer_active = False
self._has_output = False
# Track tool execution start time as a fallback when the event omits it
self._tool_started_at = {}
def _print(self, text, end="", flush=True):
sys.stdout.write(text)
if end:
sys.stdout.write(end)
if flush:
sys.stdout.flush()
self._has_output = True
def _close_section(self):
"""Finish the currently open streaming section (reasoning or answer)."""
if self._reasoning_active:
self._print("", end="\n")
self._reasoning_active = False
if self._answer_active:
self._print("", end="\n")
self._answer_active = False
def _format_arguments(self, arguments):
try:
if isinstance(arguments, (dict, list)):
text = json.dumps(arguments, ensure_ascii=False)
else:
text = str(arguments)
except Exception:
text = str(arguments)
# Keep tool input compact in the terminal
if len(text) > 300:
text = text[:300] + ""
return text
def handle_event(self, event: dict):
try:
self._handle_event(event)
except Exception as e:
logger.debug(f"[Terminal] render event error: {e}")
def _handle_event(self, event: dict):
event_type = event.get("type")
data = event.get("data", {}) or {}
if event_type == "agent_start":
self._print("\n" + _Style.wrap("Agent: ", _Style.BOLD, _Style.GREEN), end="\n")
elif event_type == "reasoning_update":
delta = data.get("delta", "")
if not delta:
return
if self._answer_active:
self._close_section()
if not self._reasoning_active:
self._print(_Style.wrap("💭 思考 ", _Style.DIM, _Style.MAGENTA), end="\n")
self._reasoning_active = True
self._print(_Style.wrap(delta, _Style.DIM, _Style.ITALIC))
elif event_type == "message_update":
delta = data.get("delta", "")
if not delta:
return
if self._reasoning_active:
self._close_section()
self._answer_active = True
self._print(delta)
elif event_type == "tool_execution_start":
self._close_section()
tool_name = data.get("tool_name", "tool")
tool_id = data.get("tool_call_id")
arguments = data.get("arguments", {})
self._tool_started_at[tool_id] = time.time()
header = _Style.wrap(f"🔧 {tool_name}", _Style.BOLD, _Style.CYAN)
args_str = self._format_arguments(arguments)
self._print(f"{header} {_Style.wrap(args_str, _Style.GRAY)}", end="\n")
elif event_type == "tool_execution_end":
tool_name = data.get("tool_name", "tool")
tool_id = data.get("tool_call_id")
status = data.get("status", "success")
result = data.get("result", "")
exec_time = data.get("execution_time")
if exec_time is None and tool_id in self._tool_started_at:
exec_time = time.time() - self._tool_started_at.pop(tool_id, time.time())
success = status == "success"
icon = "" if success else ""
color = _Style.GREEN if success else _Style.RED
result_str = str(result)
if len(result_str) > 500:
result_str = result_str[:500] + ""
# Indent multi-line tool output for readability
result_str = result_str.replace("\n", "\n ")
cost = f" ({exec_time:.2f}s)" if isinstance(exec_time, (int, float)) else ""
self._print(
_Style.wrap(f" {icon} {tool_name}{cost}", color) + " " + _Style.wrap(result_str, _Style.GRAY),
end="\n",
)
elif event_type == "file_to_send":
self._close_section()
file_path = data.get("path", "")
file_name = data.get("file_name", "")
label = file_name or file_path
self._print(_Style.wrap(f"📎 文件: {label}", _Style.BLUE), end="\n")
elif event_type == "error":
self._close_section()
err_msg = data.get("error") or "unknown error"
self._print(_Style.wrap(f"{err_msg}", _Style.BOLD, _Style.RED), end="\n")
elif event_type == "agent_cancelled":
self._close_section()
self._print(_Style.wrap("⏹ 已中止", _Style.YELLOW), end="\n")
elif event_type == "agent_end":
self._close_section()
def finish(self):
"""Ensure any open section is closed at the end of a turn."""
self._close_section()
class TerminalMessage(ChatMessage):
def __init__(
self,
@@ -29,17 +190,33 @@ class TerminalMessage(ChatMessage):
class TerminalChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = [ReplyType.VOICE]
def __init__(self):
super().__init__()
# Per-request renderers keyed by request_id; used to detect whether
# agent text was already streamed so send() can avoid duplicate output.
self._renderers = {}
# Callback that restores TTY attributes on exit (set in startup).
self._restore_terminal = None
def send(self, reply: Reply, context: Context):
print("\nBot:")
request_id = context.get("request_id") if context else None
renderer = self._renderers.pop(request_id, None) if request_id else None
streamed = renderer is not None and renderer._has_output
if renderer is not None:
renderer.finish()
if reply.type == ReplyType.IMAGE:
from PIL import Image
image_storage = reply.content
image_storage.seek(0)
img = Image.open(image_storage)
if not streamed:
print("\nAgent: ")
print("<IMAGE>")
img.show()
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
elif reply.type == ReplyType.IMAGE_URL: # download image from url
import io
import requests
@@ -52,38 +229,122 @@ class TerminalChannel(ChatChannel):
image_storage.write(block)
image_storage.seek(0)
img = Image.open(image_storage)
if not streamed:
print("\nAgent: ")
print(img_url)
img.show()
else:
print(reply.content)
print("\nUser:", end="")
# When agent already streamed the answer, skip re-printing the
# final text to avoid duplication; just emit a trailing newline.
if streamed:
print()
else:
print("\nAgent: ")
print(reply.content)
print("\nUser: ", end="")
sys.stdout.flush()
return
def _silence_console_logging(self):
"""Mute console log output so background-thread logs (web/MCP/scheduler)
don't flood the interactive terminal. Logs still go to run.log in full.
Configurable via `terminal_log_level` (default ERROR). The file handler
is untouched, so run.log keeps the complete log.
"""
import logging
level_name = str(conf().get("terminal_log_level", "ERROR")).upper()
level = getattr(logging, level_name, logging.ERROR)
root_logger = logging.getLogger("log")
for handler in root_logger.handlers:
# Only raise the level of the stdout/stderr stream handler;
# keep FileHandler at the logger's level so run.log stays complete.
if isinstance(handler, logging.StreamHandler) and not isinstance(handler, logging.FileHandler):
handler.setLevel(level)
def _install_terminal_guard(self):
"""Save TTY attributes and register restore hooks so the terminal is
never left in a broken state (no echo / raw mode / leftover ANSI) after
the process exits, especially when Ctrl+C interrupts a blocking input().
"""
if not sys.stdin.isatty():
return
try:
import atexit
import termios
saved_attrs = termios.tcgetattr(sys.stdin.fileno())
def _restore():
try:
termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, saved_attrs)
except Exception:
pass
try:
if _Style.enabled:
sys.stdout.write(_Style.RESET)
sys.stdout.flush()
except Exception:
pass
self._restore_terminal = _restore
atexit.register(_restore)
except Exception as e:
# termios is unavailable on Windows; skip the guard there.
logger.debug(f"[Terminal] terminal guard not installed: {e}")
self._restore_terminal = None
def startup(self):
context = Context()
logger.setLevel("WARN")
print("\nPlease input your question:\nUser:", end="")
self._silence_console_logging()
self._install_terminal_guard()
print("\nPlease input your question:\nUser: ", end="")
sys.stdout.flush()
msg_id = 0
while True:
try:
prompt = self.get_input()
except KeyboardInterrupt:
print("\nExiting...")
sys.exit()
except (KeyboardInterrupt, EOFError):
self._shutdown()
msg_id += 1
trigger_prefixs = conf().get("single_chat_prefix", [""])
if check_prefix(prompt, trigger_prefixs) is None:
prompt = trigger_prefixs[0] + prompt # 给没触发的消息加上触发前缀
prompt = trigger_prefixs[0] + prompt # add trigger prefix to untriggered messages
context = self._compose_context(ContextType.TEXT, prompt, msg=TerminalMessage(msg_id, prompt))
context["isgroup"] = False
if context:
# Attach an agent event renderer so reasoning / tool calls /
# streaming answer show up live in the terminal (web-like UX).
request_id = str(msg_id)
context["request_id"] = request_id
renderer = TerminalAgentRenderer()
self._renderers[request_id] = renderer
context["on_event"] = renderer.handle_event
self.produce(context)
else:
raise Exception("context is None")
def _shutdown(self):
"""Restore terminal state and terminate the whole process.
startup() runs in a daemon sub-thread, so sys.exit() would only kill
this thread and leave the main process (and web/MCP/scheduler threads)
alive, holding the terminal in a half-occupied state -> laggy input.
We reset any leftover ANSI styling and hard-exit the process instead.
"""
# Restore TTY attributes and reset any leftover ANSI styling
# (e.g. interrupted mid-stream output) before terminating.
if self._restore_terminal:
self._restore_terminal()
elif _Style.enabled:
sys.stdout.write(_Style.RESET)
sys.stdout.write("\nExiting...\n")
sys.stdout.flush()
# Hard-exit the entire process from a daemon thread.
os._exit(0)
def get_input(self):
"""
Multi-line input function

View File

@@ -47,11 +47,33 @@
This runs synchronously in <head> so the correct class is on <html>
before any CSS or body rendering occurs. -->
<script>
// Map an arbitrary locale string (zh-CN, en-US, fr ...) to 'zh' / 'zh-Hant' / 'en',
// or '' when unrecognized so callers can fall through to the next source.
window.__cowNormalizeLang__ = function(raw) {
if (!raw) return '';
var v = String(raw).trim().toLowerCase().replace('_', '-');
if (v === 'auto') return '';
// Handle Traditional Chinese variants first (more specific)
if (v === 'zh-hant' || v.indexOf('zh-hant-') === 0 || v === 'zh-tw' || v === 'zh-hk') return 'zh-Hant';
// Then Simplified Chinese
if (v.indexOf('zh') === 0) return 'zh';
if (v.indexOf('en') === 0) return 'en';
return '';
};
// Resolve the console language by priority:
// user choice (localStorage) -> backend-detected -> browser -> 'zh'.
window.__cowResolveLang__ = function() {
return window.__cowNormalizeLang__(localStorage.getItem('cow_lang'))
|| window.__cowNormalizeLang__(window.__COW_DEFAULT_LANG__)
|| window.__cowNormalizeLang__(navigator.language || (navigator.languages && navigator.languages[0]))
|| 'zh';
};
(function() {
// Backend-resolved default language (from cow_lang config / auto-detect).
window.__COW_DEFAULT_LANG__ = '{{COW_DEFAULT_LANG}}';
var theme = localStorage.getItem('cow_theme') || 'dark';
if (theme === 'dark') document.documentElement.classList.add('dark');
var lang = localStorage.getItem('cow_lang') || 'zh';
document.documentElement.setAttribute('lang', lang);
document.documentElement.setAttribute('lang', window.__cowResolveLang__());
})();
</script>
</head>
@@ -248,14 +270,29 @@
<div class="flex-1"></div>
<!-- Language Toggle -->
<button id="lang-toggle" class="flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-sm font-medium
text-slate-500 dark:text-slate-400 hover:bg-slate-100 dark:hover:bg-white/10
cursor-pointer transition-colors duration-150"
onclick="toggleLanguage()">
<i class="fas fa-globe text-xs"></i>
<span id="lang-label">EN</span>
</button>
<!-- Language Selector (dropdown) -->
<div id="lang-selector" class="relative">
<button id="lang-toggle" class="flex items-center gap-1.5 px-3 py-1.5 rounded-lg text-sm font-medium
text-slate-500 dark:text-slate-400 hover:bg-slate-100 dark:hover:bg-white/10
cursor-pointer transition-colors duration-150"
onclick="toggleLangMenu(event)">
<i class="fas fa-globe text-xs"></i>
<span id="lang-label">EN</span>
<i class="fas fa-chevron-down text-[10px] opacity-60"></i>
</button>
<div id="lang-menu" class="hidden absolute right-0 mt-1 min-w-[120px] py-1 rounded-lg z-50
bg-white dark:bg-slate-800 shadow-lg ring-1 ring-black/5 dark:ring-white/10">
<button class="lang-menu-item w-full text-left px-3 py-1.5 text-sm text-slate-600 dark:text-slate-300
hover:bg-slate-100 dark:hover:bg-white/10 cursor-pointer" data-lang="zh"
onclick="selectLanguage('zh')">简体中文</button>
<button class="lang-menu-item w-full text-left px-3 py-1.5 text-sm text-slate-600 dark:text-slate-300
hover:bg-slate-100 dark:hover:bg-white/10 cursor-pointer" data-lang="zh-Hant"
onclick="selectLanguage('zh-Hant')">繁體中文</button>
<button class="lang-menu-item w-full text-left px-3 py-1.5 text-sm text-slate-600 dark:text-slate-300
hover:bg-slate-100 dark:hover:bg-white/10 cursor-pointer" data-lang="en"
onclick="selectLanguage('en')">English</button>
</div>
</div>
<!-- Theme Toggle -->
<button id="theme-toggle" class="p-2 rounded-lg text-slate-500 dark:text-slate-400
@@ -266,7 +303,7 @@
</button>
<!-- Docs Link -->
<a href="https://docs.cowagent.ai" target="_blank" rel="noopener noreferrer"
<a id="docs-link" href="https://docs.cowagent.ai" target="_blank" rel="noopener noreferrer"
class="p-2 rounded-lg text-slate-500 dark:text-slate-400 hover:bg-slate-100 dark:hover:bg-white/10
cursor-pointer transition-colors duration-150" title="Documentation">
<i class="fas fa-book text-base"></i>
@@ -285,6 +322,14 @@
cursor-pointer transition-colors duration-150" title="GitHub">
<i class="fab fa-github text-lg"></i>
</a>
<!-- Logout Button (hidden by default) -->
<button id="logout-btn-header" class="p-2 rounded-lg text-slate-500 dark:text-slate-400
hover:bg-red-50 hover:text-red-500 dark:hover:bg-red-500/10 dark:hover:text-red-400
cursor-pointer transition-colors duration-150 hidden"
onclick="handleLogout()" title="Logout" data-i18n-title="logout">
<i class="fas fa-sign-out-alt text-base"></i>
</button>
</header>
<!-- Content Area -->
@@ -445,7 +490,7 @@
bg-primary-400 text-white hover:bg-primary-500
disabled:bg-slate-300 dark:disabled:bg-slate-600
disabled:cursor-not-allowed cursor-pointer transition-colors duration-150"
disabled onclick="sendMessage()">
disabled>
<i class="fas fa-paper-plane text-sm"></i>
</button>
</div>
@@ -601,6 +646,18 @@
after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:after:translate-x-full"></div>
</label>
</div>
<div class="flex items-center justify-between">
<label class="flex items-center gap-1.5 text-sm font-medium text-slate-600 dark:text-slate-400">
<span data-i18n="config_self_evolution">Self-Evolution</span>
<span class="cfg-tip" data-tip-key="config_self_evolution_hint"><i class="fas fa-circle-question"></i></span>
</label>
<label class="relative inline-flex items-center cursor-pointer">
<input id="cfg-self-evolution" type="checkbox" class="sr-only peer">
<div class="w-9 h-5 bg-slate-200 dark:bg-slate-700 peer-checked:bg-primary-400 rounded-full
after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white
after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:after:translate-x-full"></div>
</label>
</div>
<div class="flex items-center justify-end gap-3 pt-1">
<span id="cfg-agent-status" class="text-xs text-primary-500 opacity-0 transition-opacity duration-300"></span>
<button id="cfg-agent-save"
@@ -640,6 +697,31 @@
</div>
</div>
<!-- Language Config Card -->
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 p-6">
<div class="flex items-center gap-3 mb-5">
<div class="w-9 h-9 rounded-lg bg-sky-50 dark:bg-sky-900/30 flex items-center justify-center">
<i class="fas fa-language text-sky-500 text-sm"></i>
</div>
<h3 class="font-semibold text-slate-800 dark:text-slate-100" data-i18n="config_language">语言</h3>
</div>
<div class="space-y-4">
<div>
<label class="flex items-center gap-1.5 text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="config_language">语言</span>
<span class="cfg-tip" data-tip-key="config_language_hint"><i class="fas fa-circle-question"></i></span>
</label>
<div id="cfg-lang-select" class="cfg-dropdown" tabindex="0">
<div class="cfg-dropdown-selected">
<span class="cfg-dropdown-text">--</span>
<i class="fas fa-chevron-down cfg-dropdown-arrow"></i>
</div>
<div class="cfg-dropdown-menu"></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
@@ -716,7 +798,7 @@
</button>
<button id="memory-tab-dreams" onclick="switchMemoryTab('dreams')"
class="memory-tab px-3 py-1.5 rounded-md text-xs font-medium cursor-pointer transition-colors duration-150">
<i class="fas fa-moon mr-1.5"></i><span data-i18n="memory_tab_dreams">梦境日记</span>
<i class="fas fa-seedling mr-1.5"></i><span data-i18n="memory_tab_dreams">自主进化</span>
</button>
</div>
</div>
@@ -773,17 +855,18 @@
<!-- VIEW: Knowledge -->
<!-- ====================================================== -->
<div id="view-knowledge" class="view">
<div class="flex-1 overflow-y-auto p-4 md:p-8 lg:p-10">
<div class="w-full max-w-[1600px] mx-auto">
<div class="flex-1 min-h-0 overflow-y-auto md:overflow-hidden p-4 md:p-8 lg:p-10 md:flex md:flex-col">
<div class="w-full max-w-[1600px] mx-auto md:flex-1 md:min-h-0 md:flex md:flex-col">
<!-- Header -->
<div class="flex flex-col sm:flex-row sm:items-center justify-between gap-3 mb-4 md:mb-6">
<div class="flex flex-col sm:flex-row sm:items-center justify-between gap-3 mb-4 md:mb-6 md:flex-shrink-0">
<div>
<h2 class="text-xl font-bold text-slate-800 dark:text-slate-100" data-i18n="knowledge_title">知识库</h2>
<p class="text-sm text-slate-500 dark:text-slate-400 mt-1" data-i18n="knowledge_desc">浏览和探索你的知识库</p>
</div>
<div class="flex items-center gap-2">
<span id="knowledge-stats" class="text-xs text-slate-400 dark:text-slate-500 hidden sm:inline"></span>
<span id="knowledge-action-status" class="text-xs opacity-0 transition-opacity duration-200"></span>
<div class="flex items-center bg-slate-100 dark:bg-white/10 rounded-lg p-0.5">
<button id="knowledge-tab-docs" onclick="switchKnowledgeTab('docs')"
class="knowledge-tab px-3 py-1.5 rounded-md text-xs font-medium cursor-pointer transition-colors duration-150 active">
@@ -794,6 +877,28 @@
<i class="fas fa-diagram-project mr-1.5"></i><span data-i18n="knowledge_tab_graph">图谱</span>
</button>
</div>
<div id="knowledge-new-menu" class="relative">
<button onclick="toggleKnowledgeNewMenu(event)"
class="flex items-center gap-1.5 px-3 py-1.5 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-xs font-medium cursor-pointer transition-colors">
<i class="fas fa-plus"></i><span data-i18n="knowledge_new">新建</span><i class="fas fa-chevron-down text-[9px] ml-0.5"></i>
</button>
<div id="knowledge-new-menu-list"
class="hidden absolute right-0 mt-1.5 w-44 z-50 bg-white dark:bg-[#1A1A1A] border border-slate-200 dark:border-white/10 rounded-lg shadow-lg py-1">
<button onclick="createKnowledgeCategory(); closeKnowledgeNewMenu()"
class="w-full flex items-center gap-2.5 px-3 py-2 text-xs text-slate-600 dark:text-slate-300 hover:bg-slate-50 dark:hover:bg-white/5 cursor-pointer transition-colors">
<i class="fas fa-folder-plus w-3.5 text-slate-400"></i><span data-i18n="knowledge_new_category">新建分类</span>
</button>
<button onclick="createKnowledgeDocument(); closeKnowledgeNewMenu()"
class="w-full flex items-center gap-2.5 px-3 py-2 text-xs text-slate-600 dark:text-slate-300 hover:bg-slate-50 dark:hover:bg-white/5 cursor-pointer transition-colors">
<i class="fas fa-file-circle-plus w-3.5 text-slate-400"></i><span data-i18n="knowledge_new_document">新建文档</span>
</button>
<button onclick="selectKnowledgeImportFiles(); closeKnowledgeNewMenu()"
class="w-full flex items-center gap-2.5 px-3 py-2 text-xs text-slate-600 dark:text-slate-300 hover:bg-slate-50 dark:hover:bg-white/5 cursor-pointer transition-colors">
<i class="fas fa-file-arrow-up w-3.5 text-slate-400"></i><span data-i18n="knowledge_import_documents">导入文档</span>
</button>
</div>
</div>
<input id="knowledge-import-input" type="file" class="hidden" multiple accept=".md,.txt,text/markdown,text/plain">
</div>
</div>
@@ -816,12 +921,12 @@
</div>
<!-- Documents panel -->
<div id="knowledge-panel-docs" class="hidden">
<div class="flex flex-col md:flex-row gap-4 md:gap-6" style="min-height: calc(100vh - 220px)">
<div id="knowledge-panel-docs" class="hidden md:flex-1 md:min-h-0">
<div class="flex flex-col md:flex-row gap-4 md:gap-6 md:h-full">
<!-- File tree -->
<div id="knowledge-sidebar" class="w-full md:w-72 lg:w-80 flex-shrink-0">
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 overflow-hidden">
<div class="px-4 py-3 border-b border-slate-200 dark:border-white/10">
<div id="knowledge-sidebar" class="w-full md:w-72 lg:w-80 flex-shrink-0 md:h-full">
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 overflow-hidden flex flex-col md:h-full">
<div class="px-4 py-3 border-b border-slate-200 dark:border-white/10 flex-shrink-0">
<div class="relative">
<i class="fas fa-search absolute left-3 top-1/2 -translate-y-1/2 text-slate-400 text-xs"></i>
<input id="knowledge-search" type="text" placeholder="Search..."
@@ -829,19 +934,19 @@
oninput="filterKnowledgeTree(this.value)">
</div>
</div>
<div id="knowledge-tree" class="p-2 overflow-y-auto max-h-[50vh] md:max-h-[calc(100vh-300px)]"></div>
<div id="knowledge-tree" class="p-2 overflow-y-auto flex-1 max-h-[50vh] md:max-h-none"></div>
</div>
</div>
<!-- Content viewer -->
<div class="flex-1 min-w-0">
<div class="flex-1 min-w-0 md:h-full">
<div id="knowledge-content-placeholder"
class="flex flex-col items-center justify-center py-20 text-slate-400 dark:text-slate-500">
class="flex flex-col items-center justify-center py-20 md:h-full text-slate-400 dark:text-slate-500 bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10">
<i class="fas fa-file-lines text-3xl mb-3 opacity-40"></i>
<p class="text-sm" data-i18n="knowledge_select_hint">选择一个文档查看</p>
</div>
<div id="knowledge-content-viewer" class="hidden">
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 overflow-hidden">
<div class="flex items-center gap-3 px-4 md:px-5 py-3 border-b border-slate-200 dark:border-white/10">
<div id="knowledge-content-viewer" class="hidden md:h-full">
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 overflow-hidden flex flex-col md:h-full">
<div class="flex items-center gap-3 px-4 md:px-5 py-3 border-b border-slate-200 dark:border-white/10 flex-shrink-0">
<button onclick="knowledgeMobileBack()" class="md:hidden p-1 -ml-1 text-slate-400 hover:text-slate-600 dark:hover:text-slate-300 cursor-pointer">
<i class="fas fa-arrow-left text-xs"></i>
</button>
@@ -850,8 +955,7 @@
<span id="knowledge-viewer-path" class="text-xs text-slate-400 dark:text-slate-500 ml-auto font-mono truncate hidden md:inline"></span>
</div>
<div id="knowledge-viewer-body"
class="p-4 md:p-5 overflow-y-auto text-sm msg-content text-slate-700 dark:text-slate-200"
style="max-height: calc(100vh - 280px)"></div>
class="p-4 md:p-5 overflow-y-auto flex-1 max-h-[60vh] md:max-h-none text-sm msg-content text-slate-700 dark:text-slate-200"></div>
</div>
</div>
</div>
@@ -939,6 +1043,14 @@
<h2 class="text-xl font-bold text-slate-800 dark:text-slate-100" data-i18n="tasks_title">定时任务</h2>
<p class="text-sm text-slate-500 dark:text-slate-400 mt-1" data-i18n="tasks_desc">查看和管理定时任务</p>
</div>
<div class="flex items-center gap-2">
<button id="task-refresh-btn" onclick="refreshTasksView()"
class="px-3 py-2 rounded-lg border border-slate-200 dark:border-white/10
text-slate-600 dark:text-slate-300 hover:bg-slate-50 dark:hover:bg-white/5
text-sm font-medium cursor-pointer transition-colors duration-150">
<i class="fas fa-refresh text-xs"></i>
</button>
</div>
</div>
<div id="tasks-empty" class="flex flex-col items-center justify-center py-20">
<div class="w-16 h-16 rounded-2xl bg-rose-50 dark:bg-rose-900/20 flex items-center justify-center mb-4">
@@ -1012,6 +1124,62 @@
</div><!-- /main-content -->
</div><!-- /app -->
<!-- Knowledge Action Dialog -->
<div id="knowledge-dialog-overlay" class="fixed inset-0 bg-black/50 z-[200] hidden flex items-center justify-center">
<div id="knowledge-dialog-card" class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl w-full max-w-md mx-4">
<div class="p-6">
<div class="flex items-center gap-3 mb-5">
<div class="w-10 h-10 rounded-xl bg-emerald-50 dark:bg-emerald-900/20 flex items-center justify-center">
<i id="knowledge-dialog-icon" class="fas fa-folder text-emerald-500"></i>
</div>
<div>
<h3 id="knowledge-dialog-title" class="font-semibold text-slate-800 dark:text-slate-100"></h3>
<p id="knowledge-dialog-subtitle" class="text-xs text-slate-400 dark:text-slate-500 mt-0.5"></p>
</div>
</div>
<label id="knowledge-dialog-label" class="block text-sm font-medium text-slate-600 dark:text-slate-300 mb-1.5"></label>
<input id="knowledge-dialog-input" type="text"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500">
<div id="knowledge-dialog-select" class="cfg-dropdown hidden w-full" tabindex="0">
<div class="cfg-dropdown-selected">
<span class="cfg-dropdown-text">--</span>
<i class="fas fa-chevron-down cfg-dropdown-arrow"></i>
</div>
<div class="cfg-dropdown-menu"></div>
</div>
<textarea id="knowledge-dialog-textarea" rows="8"
class="hidden w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500 font-mono resize-y"></textarea>
<div id="knowledge-document-form" class="hidden space-y-3">
<div class="rounded-lg bg-emerald-50 dark:bg-emerald-900/15 border border-emerald-100 dark:border-emerald-800/40 px-3 py-2">
<div id="knowledge-document-category-label" class="text-[11px] text-emerald-600 dark:text-emerald-400 mb-0.5"></div>
<div id="knowledge-document-path-preview" class="text-xs font-mono text-emerald-700 dark:text-emerald-300 break-all"></div>
</div>
<div>
<label id="knowledge-document-filename-label" class="block text-sm font-medium text-slate-600 dark:text-slate-300 mb-1.5"></label>
<input id="knowledge-document-filename" type="text"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500"
placeholder="note.md">
</div>
<div>
<div class="flex items-center justify-between mb-1.5">
<label id="knowledge-document-content-label" class="block text-sm font-medium text-slate-600 dark:text-slate-300"></label>
<button id="knowledge-document-template" type="button" class="text-xs text-primary-500 hover:text-primary-600"></button>
</div>
<textarea id="knowledge-document-content" rows="14"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500 font-mono resize-y"
placeholder="# Title&#10;&#10;Write your notes here..."></textarea>
</div>
</div>
<p id="knowledge-dialog-hint" class="mt-2 text-xs text-slate-400 dark:text-slate-500"></p>
<p id="knowledge-dialog-error" class="mt-2 text-xs text-red-500 hidden"></p>
</div>
<div class="flex justify-end gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5">
<button id="knowledge-dialog-cancel" class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10 text-slate-600 dark:text-slate-300 text-sm hover:bg-slate-50 dark:hover:bg-white/5">取消</button>
<button id="knowledge-dialog-submit" class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium disabled:opacity-50">确定</button>
</div>
</div>
</div>
<!-- Confirm Dialog -->
<div id="confirm-dialog-overlay" class="fixed inset-0 bg-black/50 z-[200] hidden flex items-center justify-center">
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
@@ -1109,6 +1277,240 @@
</div>
</div>
<!-- Custom Provider Modal (multiple OpenAI-compatible providers) -->
<div id="custom-provider-modal-overlay" class="fixed inset-0 bg-black/50 z-[100] hidden flex items-center justify-center">
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
w-full max-w-md mx-4">
<div class="p-6">
<div class="flex items-center gap-3 mb-5">
<div class="w-10 h-10 rounded-xl bg-primary-50 dark:bg-primary-900/20 flex items-center justify-center flex-shrink-0">
<i class="fas fa-sliders text-primary-500"></i>
</div>
<div class="min-w-0 flex-1">
<h3 id="custom-provider-modal-title" class="font-semibold text-slate-800 dark:text-slate-100 text-base"></h3>
</div>
</div>
<div class="space-y-4">
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5" data-i18n="models_custom_name">名称</label>
<input id="custom-provider-name" type="text" autocomplete="off" data-1p-ignore data-lpignore="true"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors">
</div>
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">API Base</label>
<input id="custom-provider-base" type="text"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 font-mono transition-colors"
placeholder="https://...../v1">
</div>
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">API Key</label>
<input id="custom-provider-key" type="text" autocomplete="off" data-1p-ignore data-lpignore="true"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 font-mono transition-colors">
</div>
</div>
</div>
<div class="flex items-center justify-between gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5 rounded-b-2xl">
<button id="custom-provider-modal-delete"
class="px-3 py-2 rounded-lg text-sm font-medium text-red-500 hover:bg-red-50 dark:hover:bg-red-900/20
cursor-pointer transition-colors duration-150 hidden"
data-i18n="models_custom_delete">删除</button>
<span id="custom-provider-modal-status"
class="flex-1 text-xs text-primary-500 opacity-0 transition-opacity duration-300 text-left"></span>
<button id="custom-provider-modal-cancel"
class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10
text-slate-600 dark:text-slate-300 text-sm font-medium
hover:bg-slate-50 dark:hover:bg-white/5
cursor-pointer transition-colors duration-150"
data-i18n="cancel">取消</button>
<button id="custom-provider-modal-save"
class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium
cursor-pointer transition-colors duration-150 disabled:opacity-50 disabled:cursor-not-allowed"
data-i18n="save">保存</button>
</div>
</div>
</div>
<!-- Task Edit Modal -->
<div id="task-edit-modal-overlay" class="fixed inset-0 bg-black/50 z-[100] hidden flex items-center justify-center">
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
w-full max-w-2xl mx-4 max-h-[90vh] overflow-y-auto">
<div class="p-6">
<div class="flex items-center gap-3 mb-5">
<div class="w-10 h-10 rounded-xl bg-primary-50 dark:bg-primary-900/20 flex items-center justify-center flex-shrink-0">
<i class="fas fa-clock text-primary-500"></i>
</div>
<div class="min-w-0 flex-1">
<h3 class="font-semibold text-slate-800 dark:text-slate-100 text-base" data-i18n="task_edit_title">编辑定时任务</h3>
<p id="task-edit-modal-subtitle" class="text-xs text-slate-500 dark:text-slate-400 mt-0.5 font-mono"></p>
</div>
</div>
<div class="space-y-4">
<!-- 任务名称和启用状态 -->
<div class="flex gap-4 items-end">
<div class="flex-1">
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_name">任务名称</span>
</label>
<input id="task-edit-name" type="text"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors"
placeholder="任务名称">
</div>
<div class="flex items-center gap-2 pb-[2px]">
<label class="text-xs font-medium text-slate-600 dark:text-slate-400">
<span data-i18n="task_enabled">启用</span>
</label>
<label class="relative inline-flex items-center cursor-pointer">
<input type="checkbox" id="task-edit-enabled" class="sr-only peer">
<div class="w-11 h-6 bg-slate-200 peer-focus:outline-none rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-primary-500 dark:bg-slate-600 dark:peer-checked:bg-primary-500"></div>
</label>
</div>
</div>
<!-- 调度类型 + 调度值 -->
<div class="grid grid-cols-2 gap-4">
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_schedule_type">调度类型</span>
</label>
<select id="task-edit-schedule-type"
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors
appearance-none bg-no-repeat bg-right"
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
<option value="cron" data-i18n="task_schedule_cron">Cron 表达式</option>
<option value="interval" data-i18n="task_schedule_interval">固定间隔</option>
<option value="once" data-i18n="task_schedule_once">一次性任务</option>
</select>
</div>
<!-- Cron 表达式 -->
<div id="task-edit-cron-wrap">
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_cron_expression">Cron 表达式</span>
</label>
<input id="task-edit-cron-expression" type="text"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 font-mono transition-colors"
placeholder="0 9 * * *">
</div>
<!-- 固定间隔 -->
<div id="task-edit-interval-wrap" class="hidden">
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_interval_seconds">间隔秒数</span>
</label>
<input id="task-edit-interval-seconds" type="number" min="60"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors"
placeholder="3600">
</div>
<!-- 一次性任务时间 -->
<div id="task-edit-once-wrap" class="hidden">
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_once_time">执行时间</span>
</label>
<input id="task-edit-once-time" type="datetime-local" step="1"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors
cursor-pointer"
onclick="this.showPicker && this.showPicker()">
</div>
</div>
<!-- Cron/Interval 提示 -->
<p id="task-edit-cron-hint" class="text-xs text-slate-400 dark:text-slate-500">
<span data-i18n="task_cron_hint">格式: 分 时 日 月 周,例如 "0 9 * * *" 表示每天 9:00</span>
</p>
<p id="task-edit-interval-hint" class="text-xs text-slate-400 dark:text-slate-500 hidden">
<span data-i18n="task_interval_hint">最小 60 秒,例如 3600 表示每小时执行一次</span>
</p>
<!-- 动作类型 + 通道类型 -->
<div class="grid grid-cols-2 gap-4">
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_action_type">动作类型</span>
</label>
<select id="task-edit-action-type"
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors
appearance-none bg-no-repeat bg-right"
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
<option value="send_message" data-i18n="task_action_send_message">发送消息</option>
<option value="agent_task" data-i18n="task_action_agent_task">AI 任务</option>
</select>
</div>
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span data-i18n="task_channel_type">通道类型</span>
</label>
<select id="task-edit-channel-type"
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors
appearance-none bg-no-repeat bg-right
disabled:opacity-100 disabled:text-slate-800 dark:disabled:text-slate-300 disabled:bg-slate-100 dark:disabled:bg-[#252525] disabled:cursor-not-allowed"
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
</select>
<p class="text-xs text-slate-400 dark:text-slate-500 mt-1">
<span data-i18n="task_channel_hint">选择定时消息发送的通道</span>
</p>
</div>
</div>
<!-- 隐藏的接收者ID字段自动填充 -->
<input id="task-edit-receiver" type="hidden" value="">
<!-- 消息内容/任务描述 -->
<div>
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
<span id="task-edit-content-label" data-i18n="task_message_content">消息内容</span>
</label>
<textarea id="task-edit-content" rows="3"
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
focus:outline-none focus:border-primary-500 transition-colors resize-none"
placeholder="输入消息内容或任务描述"></textarea>
</div>
</div>
</div>
<div class="flex items-center justify-between gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5 rounded-b-2xl">
<button id="task-edit-modal-delete"
class="px-4 py-2 rounded-lg text-sm font-medium text-red-500 hover:bg-red-50 dark:hover:bg-red-900/20
cursor-pointer transition-colors duration-150 hidden"
data-i18n="task_delete_btn">删除任务</button>
<span id="task-edit-modal-status"
class="flex-1 text-xs text-primary-500 opacity-0 transition-opacity duration-300 text-left"></span>
<button id="task-edit-modal-cancel"
class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10
text-slate-600 dark:text-slate-300 text-sm font-medium
hover:bg-slate-50 dark:hover:bg-white/5
cursor-pointer transition-colors duration-150"
data-i18n="cancel">取消</button>
<button id="task-edit-modal-save"
class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium
cursor-pointer transition-colors duration-150 disabled:opacity-50 disabled:cursor-not-allowed"
data-i18n="save">保存</button>
</div>
</div>
</div>
<script defer src="assets/js/console.js"></script>
</body>
</html>

View File

@@ -244,6 +244,52 @@
}
.dark .session-delete:hover { background: rgba(239, 68, 68, 0.15); }
/* Rename button: shares the look of the delete button, sits to its left.
Negative right margin tightens the gap to the delete button only. */
.session-rename {
flex-shrink: 0;
margin-right: -6px;
width: 22px;
height: 22px;
display: flex;
align-items: center;
justify-content: center;
border-radius: 6px;
color: #9ca3af;
font-size: 11px;
opacity: 0;
transition: opacity 0.15s, color 0.15s, background 0.15s;
cursor: pointer;
background: none;
border: none;
padding: 0;
}
.session-item:hover .session-rename { opacity: 1; }
.session-rename:hover {
color: #4ABE6E;
background: rgba(74, 190, 110, 0.12);
}
.dark .session-rename:hover { background: rgba(74, 190, 110, 0.18); }
/* Inline title editor */
.session-title-input {
flex: 1;
min-width: 0;
font-size: 13px;
font-family: inherit;
color: #111827;
background: #ffffff;
border: 1px solid #4ABE6E;
border-radius: 6px;
padding: 2px 6px;
outline: none;
}
.dark .session-title-input {
color: #e5e5e5;
background: rgba(255, 255, 255, 0.06);
border-color: #4ABE6E;
}
/* Context Divider */
.context-divider {
display: flex;
@@ -605,6 +651,18 @@
color: inherit;
}
.tool-error-text { color: #f87171; }
.tool-live-output:empty { display: none; }
.tool-streaming .tool-live-output:not(:empty)::after {
content: ' ';
display: inline-block;
width: 0.45em;
height: 1em;
margin-left: 0.15em;
vertical-align: -0.15em;
background: currentColor;
animation: tool-cursor-blink 1s steps(1) infinite;
}
@keyframes tool-cursor-blink { 50% { opacity: 0; } }
/* Log level highlighting */
.log-line { display: block; }
@@ -854,6 +912,14 @@
font-size: 11px;
flex-shrink: 0;
}
/* "Custom" row is an add-new action: trailing + instead of ✓. */
.vendor-picker-add-mark {
margin-left: auto;
padding-left: 12px;
color: #94a3b8;
font-size: 11px;
flex-shrink: 0;
}
/* Chat Input */
#chat-input {
@@ -1191,6 +1257,34 @@
background: #EDFDF3;
color: #228547;
}
.knowledge-actions {
display: flex;
gap: 2px;
margin-left: auto;
opacity: 0;
transition: opacity 0.15s;
}
.knowledge-tree-file:hover .knowledge-actions,
.knowledge-tree-group-btn:hover .knowledge-actions,
.knowledge-tree-file:focus-within .knowledge-actions,
.knowledge-tree-group-btn:focus-within .knowledge-actions {
opacity: 1;
}
.knowledge-action {
padding: 3px 5px;
border-radius: 5px;
color: #94a3b8;
font-size: 9px;
}
.knowledge-action:hover {
color: #228547;
background: rgba(34, 133, 71, 0.08);
}
.knowledge-action.danger:hover {
color: #ef4444;
background: rgba(239, 68, 68, 0.08);
}
.dark .knowledge-tree-file:hover {
background: rgba(255,255,255,0.06);
color: #e2e8f0;
@@ -1199,6 +1293,18 @@
background: rgba(74, 190, 110, 0.1);
color: #4ABE6E;
}
.knowledge-import-drag-over {
outline: 2px dashed rgba(74, 190, 110, 0.55);
outline-offset: 4px;
border-radius: 14px;
}
#knowledge-dialog-card.knowledge-document-dialog {
max-width: 760px;
}
#knowledge-document-content {
min-height: 320px;
line-height: 1.55;
}
/* Graph legend */
.knowledge-graph-legend {
@@ -1367,3 +1473,213 @@
text-align: right;
}
.voice-pill audio { display: none; }
/* Send button toggles into a Stop button while an SSE stream is in flight.
Match the look of the disabled send button (light grey block + white
glyph) so it reads as the same visual element, just paused/idle from
sending perspective and clickable to stop. */
#send-btn.send-btn-cancel {
background-color: rgb(203 213 225) !important; /* slate-300, == disabled send-btn */
color: white !important;
}
#send-btn.send-btn-cancel:hover {
background-color: rgb(148 163 184) !important; /* slate-400 */
}
#send-btn.send-btn-cancel:disabled {
background-color: rgb(226 232 240) !important; /* slate-200, while stop is in flight */
color: white !important;
cursor: progress;
}
.dark #send-btn.send-btn-cancel {
background-color: rgb(71 85 105) !important; /* slate-600, == dark disabled send-btn */
color: white !important;
}
.dark #send-btn.send-btn-cancel:hover {
background-color: rgb(100 116 139) !important; /* slate-500 */
}
.dark #send-btn.send-btn-cancel:disabled {
background-color: rgb(51 65 85) !important; /* slate-700 */
color: rgb(203 213 225) !important;
}
.agent-cancelled-tag {
font-style: italic;
}
/* =====================================================================
Code Block Enhancements
===================================================================== */
.code-block-wrapper {
position: relative;
margin: 1em 0;
border-radius: 8px;
overflow: hidden;
background: #f8f9fa;
border: 1px solid #e2e8f0;
}
.dark .code-block-wrapper {
background: #1e293b;
border-color: #334155;
}
.code-block-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 0.5em 1em;
background: #e2e8f0;
border-bottom: 1px solid #cbd5e1;
font-size: 0.85em;
}
.dark .code-block-header {
background: #0f172a;
border-bottom-color: #334155;
}
.code-block-lang {
color: #64748b;
font-weight: 500;
text-transform: lowercase;
}
.dark .code-block-lang {
color: #94a3b8;
}
.code-copy-btn {
background: transparent;
border: none;
color: #64748b;
cursor: pointer;
padding: 0.25em 0.5em;
border-radius: 4px;
transition: all 0.2s;
font-size: 0.9em;
}
.code-copy-btn:hover {
background: rgba(100, 116, 139, 0.1);
color: #475569;
}
.dark .code-copy-btn {
color: #94a3b8;
}
.dark .code-copy-btn:hover {
background: rgba(148, 163, 184, 0.1);
color: #cbd5e1;
}
.code-block-wrapper pre {
margin: 0;
border-radius: 0;
border: none;
}
/* =====================================================================
Drag and Drop Overlay
===================================================================== */
/* Anchor the absolutely-positioned overlay to the chat view. */
#view-chat {
position: relative;
}
.drag-overlay {
position: absolute;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(59, 130, 246, 0.1);
backdrop-filter: blur(2px);
display: flex;
align-items: center;
justify-content: center;
z-index: 9999;
pointer-events: none;
opacity: 0;
transition: opacity 0.2s;
}
.drag-overlay.active {
opacity: 1;
}
.drag-overlay.hidden {
display: none;
}
.drag-overlay-content {
background: white;
border: 3px dashed #3b82f6;
border-radius: 16px;
padding: 3em 4em;
text-align: center;
box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1);
animation: bounce 1s ease infinite;
}
.dark .drag-overlay-content {
background: #1e293b;
border-color: #60a5fa;
}
.drag-overlay-content i {
font-size: 4em;
color: #3b82f6;
margin-bottom: 0.5em;
}
.dark .drag-overlay-content i {
color: #60a5fa;
}
.drag-overlay-content p {
font-size: 1.5em;
font-weight: 600;
color: #1e293b;
margin: 0;
}
.dark .drag-overlay-content p {
color: #f1f5f9;
}
@keyframes bounce {
0%, 100% { transform: translateY(0); }
50% { transform: translateY(-10px); }
}
/* =====================================================================
Message Action Buttons
===================================================================== */
.edit-msg-btn,
.delete-msg-btn,
.regenerate-msg-btn {
opacity: 0;
transition: opacity 0.2s, color 0.2s;
}
.user-message-group:hover .edit-msg-btn,
.user-message-group:hover .delete-msg-btn,
.bot-message-group:hover .regenerate-msg-btn {
opacity: 1;
}
.edit-msg-btn:hover,
.regenerate-msg-btn:hover {
color: #3b82f6 !important;
}
.delete-msg-btn:hover {
color: #ef4444 !important;
}
.edit-msg-btn:disabled,
.delete-msg-btn:disabled {
cursor: not-allowed !important;
opacity: 0.35 !important;
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

115
channel/wechat_kf/README.md Normal file
View File

@@ -0,0 +1,115 @@
# 微信客服WeChat Customer Service通道
> 与 `channel/wechatcom/`(企微自建应用)是两个**独立的 CoW 通道**
>
> - 自建应用:**面向企业内部成员**(员工通过企业微信 App 与机器人对话)。
> - 微信客服:**面向外部微信用户**(普通微信用户通过链接/二维码进入对话)。
>
> 但底层都基于"企微自建应用"——本通道是**通过把一个企微自建应用绑定到微信客服账号**来实现 AI 接管对外咨询,详见 [LinkAI 微信客服接入文档](https://docs.link-ai.tech/platform/link-app/wechat-customer-service)。
## 一、接入流程概览
```
┌─────────────────────┐ ┌─────────────────────┐ ┌──────────────────┐
│ 1. 企业微信后台 │ → │ 2. CoW 配置回调 │ → │ 3. 绑定微信客服 │
│ 创建一个自建应用 │ │ 端口 9888 │ │ 账号 │
└─────────────────────┘ └─────────────────────┘ └──────────────────┘
外部微信用户通过
链接/二维码 →
消息 → CoW Bot
```
> **重要**:建议**单独再创建一个企微自建应用**用于微信客服,**不要复用**已经接入员工内部使用的那个 `wechatcom_app` 应用,否则两个通道会争抢同一个回调地址。
## 二、企业微信后台配置
### 1. 创建企微自建应用
进入 企业微信管理后台 → **应用管理****创建应用**
### 2. 收集字段
| 字段 | 来源 | 对应 CoW 配置项 |
|---|---|---|
| 企业IDCorpId | 「我的企业」最下方 | `wechat_kf_corp_id` |
| Secret | 进入应用详情 → 点击「查看」(会推送到管理员手机端,在手机上查看) | `wechat_kf_secret` |
| Token | 应用「接收消息 → 设置API接收」 | `wechat_kf_token` |
| EncodingAESKey | 应用「接收消息 → 设置API接收」 | `wechat_kf_aes_key` |
> AgentId 在本通道**不需要**(消息发送走的是 `cgi-bin/kf/send_msg`,不依赖 agent_id
### 3. 配置回调地址 + 可信 IP
在应用「**接收消息 → 设置API接收**」里填:
- URL`http://<your-host>:9888/wxkf/`(公网必须可达)
- Token / EncodingAESKey与下方 `config.json` 一致
回到应用详情页,把服务器公网 IP 填入「**企业可信IP**」。
### 4. 绑定微信客服账号
进入 企业微信后台 → **微信客服** → 创建客服账号 → **将该账号绑定到上一步创建的企微自建应用**
绑定完成后,进入 **微信客服 → 微信客服账号详情** 页面,在「**接入链接**」一栏:
- 「复制链接」可拿到形如 `https://work.weixin.qq.com/kfid/kfcd83e5896b9ba07be` 的访问链接
- 「生成二维码」可拿到对应二维码
把链接或二维码推给微信客户使用即可。
## 三、CoW 配置(`config.json`
```json
{
"channel_type": "wechat_kf",
"wechat_kf_corp_id": "ww1234567890abcdef",
"wechat_kf_secret": "<企微应用的 Secret>",
"wechat_kf_token": "<接收消息 Token>",
"wechat_kf_aes_key": "<EncodingAESKey>",
"wechat_kf_port": 9888
}
```
| 字段 | 说明 |
|---|---|
| `wechat_kf_corp_id` | 企业 ID |
| `wechat_kf_secret` | **绑定到微信客服**的那个企微自建应用的 Secret |
| `wechat_kf_token` | 该应用「接收消息」配置的 Token |
| `wechat_kf_aes_key` | 该应用「接收消息」配置的 EncodingAESKey |
| `wechat_kf_port` | 监听端口,默认 `9888` |
也支持环境变量:`WECHAT_KF_CORP_ID` / `WECHAT_KF_SECRET` / `WECHAT_KF_TOKEN` / `WECHAT_KF_AES_KEY`
## 四、运行
```bash
python app.py
```
启动后日志里会看到:
```
[wechat_kf] WeCom customer-service channel started
[wechat_kf] Listening on http://0.0.0.0:9888/wxkf/
```
回到企微后台「设置API接收」点击保存——会触发 `GET /wxkf/?...&echostr=...`CoW 通过 `crypto.check_signature` 校验后返回明文 `echostr`,验证成功。
## 五、支持的回复类型
| ReplyType | 是否支持 | 备注 |
|---|---|---|
| `TEXT` / `INFO` / `ERROR` | ✅ | 自动按 2048 字节切片分段发送 |
| `IMAGE`(本地) / `IMAGE_URL`(网络) | ✅ | 大图自动压缩到 10MB 以内 |
| `VOICE` | ✅ | 转 amr 后发送,>60s 自动切片 |
| `VIDEO_URL` | ✅ | 通过临时素材接口上传 |
| `FILE` | ✅ | |
## 六、参考文档
- [LinkAI 微信客服接入文档](https://docs.link-ai.tech/platform/link-app/wechat-customer-service)
- [企业微信开放接口 - 微信客服 - 接收消息](https://developer.work.weixin.qq.com/document/path/94670)
- [企业微信开放接口 - 微信客服 - 发送消息](https://developer.work.weixin.qq.com/document/path/95122)

View File

@@ -0,0 +1,603 @@
# -*- coding=utf-8 -*-
"""
WeChat Customer Service (微信客服) channel for CoW.
Differences from `channel/wechatcom/` (企微自建应用):
1. Audience: external WeChat users (not internal members).
2. Receiver fields: `external_userid` + `open_kfid` instead of a single
member `userid`.
3. Inbound flow: callback only delivers an event token, the actual
message bodies must be pulled via `cgi-bin/kf/sync_msg` with a
persistent cursor. See `wechat_kf_cursor_store.py`.
4. Outbound flow: messages are sent via `cgi-bin/kf/send_msg` (each
request must specify both `touser` and `open_kfid`); wechatpy has
no native helper, so we call the HTTP endpoint directly.
"""
import io
import json
import os
import threading
import time
import xml.etree.ElementTree as ET
from collections import defaultdict
from concurrent.futures import ThreadPoolExecutor
from typing import Optional
import requests
import web
from wechatpy.enterprise import WeChatClient
from wechatpy.enterprise.crypto import WeChatCrypto
from wechatpy.enterprise.exceptions import InvalidCorpIdException
from wechatpy.exceptions import InvalidSignatureException, WeChatClientException
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel
from channel.file_cache import get_file_cache
from channel.wechat_kf.wechat_kf_cursor_store import CursorStore
from channel.wechat_kf.wechat_kf_message import WechatKfMessage
from common.log import logger
from common.singleton import singleton
from common.utils import (
compress_imgfile,
fsize,
remove_markdown_symbol,
split_string_by_utf8_length,
)
from config import conf
try:
from voice.audio_convert import any_to_amr, split_audio
except ImportError as e: # voice features optional
logger.debug(
"[wechat_kf] import voice.audio_convert failed, voice will be disabled: {}".format(e)
)
MAX_UTF8_LEN = 2048
KF_API_BASE = "https://qyapi.weixin.qq.com/cgi-bin/kf"
SYNC_MSG_LIMIT = 1000
@singleton
class WechatKfChannel(ChatChannel):
NOT_SUPPORT_REPLYTYPE = []
def __init__(self):
super().__init__()
self.corp_id = conf().get("wechat_kf_corp_id")
self.secret = conf().get("wechat_kf_secret")
self.token = conf().get("wechat_kf_token")
self.aes_key = conf().get("wechat_kf_aes_key")
self._http_server = None
logger.info(
"[wechat_kf] Initializing WeCom customer-service channel, corp_id: {}".format(
self.corp_id
)
)
self.crypto = WeChatCrypto(self.token, self.aes_key, self.corp_id)
# Use the stock wechatpy WeChatClient so that the access_token is
# cached and only refreshed when actually expired (~2h). The local
# `WechatComAppClient` subclass has a broken background refresh
# loop that re-fetches every 60s and a `fetch_access_token()`
# override that may return a dict instead of a string, which
# corrupts URLs and triggers errcode 40014.
self.client = WeChatClient(self.corp_id, self.secret)
# Persist sync_msg cursor under the user's home dir by default,
# so it survives `tmp/` cleanups and cwd changes across restarts.
cursor_path = os.path.expanduser(
conf().get("wechat_kf_cursor_path") or "~/.wechat_kf_cursors.json"
)
self.cursor_store = CursorStore(cursor_path)
# WeCom requires the callback HTTP response to return within ~5s,
# otherwise it retries the same notification. sync_msg pulling
# can easily exceed that, so we dispatch it to a background pool
# and let `Query.POST` reply success immediately.
self._callback_executor = ThreadPoolExecutor(
max_workers=4, thread_name_prefix="wxkf-cb"
)
# Per-open_kfid lock: serialize sync_msg for the same kf account
# so that callback retries (or rapid-fire events) don't race on
# the same cursor and produce duplicate replies.
self._kf_locks: dict = defaultdict(threading.Lock)
self._kf_locks_guard = threading.Lock()
# ------------------------------------------------------------------
# Lifecycle
# ------------------------------------------------------------------
def startup(self):
urls = ("/wxkf/?", "channel.wechat_kf.wechat_kf_channel.Query")
app = web.application(urls, globals(), autoreload=False)
port = conf().get("wechat_kf_port", 9888)
logger.info("[wechat_kf] WeCom customer-service channel started")
logger.info("[wechat_kf] Listening on http://0.0.0.0:{}/wxkf/".format(port))
func = web.httpserver.StaticMiddleware(app.wsgifunc())
func = web.httpserver.LogMiddleware(func)
server = web.httpserver.WSGIServer(("0.0.0.0", port), func)
self._http_server = server
try:
server.start()
except (KeyboardInterrupt, SystemExit):
server.stop()
def stop(self):
if self._http_server:
try:
self._http_server.stop()
logger.info("[wechat_kf] HTTP server stopped")
except Exception as e:
logger.warning(f"[wechat_kf] Error stopping HTTP server: {e}")
self._http_server = None
try:
self._callback_executor.shutdown(wait=False)
except Exception as e:
logger.warning(f"[wechat_kf] Error shutting down callback executor: {e}")
# ------------------------------------------------------------------
# Outbound — implementing the abstract `send` contract
# ------------------------------------------------------------------
def send(self, reply: Reply, context: Context):
receiver = context["receiver"]
msg = context.kwargs.get("msg")
external_userid = context.get("external_userid") or (msg.external_userid if msg else None)
open_kfid = context.get("open_kfid") or (msg.open_kfid if msg else None)
if not external_userid or not open_kfid:
logger.error(
"[wechat_kf] missing external_userid or open_kfid, cannot send: "
f"external_userid={external_userid}, open_kfid={open_kfid}"
)
return
if reply.type in [ReplyType.TEXT, ReplyType.ERROR, ReplyType.INFO]:
reply_text = remove_markdown_symbol(reply.content)
texts = split_string_by_utf8_length(reply_text, MAX_UTF8_LEN)
if len(texts) > 1:
logger.info(
"[wechat_kf] text too long, split into {} parts".format(len(texts))
)
for i, text in enumerate(texts):
self._send_text(external_userid, open_kfid, text)
if i != len(texts) - 1:
time.sleep(0.5)
logger.info("[wechat_kf] Do send text to {}: {}".format(receiver, reply_text))
elif reply.type == ReplyType.VOICE:
file_path = reply.content
try:
amr_file = os.path.splitext(file_path)[0] + ".amr"
any_to_amr(file_path, amr_file)
duration, files = split_audio(amr_file, 60 * 1000)
if len(files) > 1:
logger.info(
"[wechat_kf] voice too long {}s > 60s, split into {} parts".format(
duration / 1000.0, len(files)
)
)
media_ids = []
for path in files:
with open(path, "rb") as f:
response = self.client.media.upload("voice", f)
logger.debug("[wechat_kf] upload voice response: {}".format(response))
media_ids.append(response["media_id"])
except ImportError as e:
logger.error("[wechat_kf] voice conversion failed: {}".format(e))
logger.error("[wechat_kf] please install pydub: pip install pydub")
return
except WeChatClientException as e:
logger.error("[wechat_kf] upload voice failed: {}".format(e))
return
try:
os.remove(file_path)
if amr_file != file_path:
os.remove(amr_file)
except Exception:
pass
for media_id in media_ids:
self._send_voice(external_userid, open_kfid, media_id)
time.sleep(1)
logger.info("[wechat_kf] sendVoice={}, receiver={}".format(reply.content, receiver))
elif reply.type == ReplyType.IMAGE_URL:
img_url = reply.content
pic_res = requests.get(img_url, stream=True)
image_storage = io.BytesIO()
for block in pic_res.iter_content(1024):
image_storage.write(block)
sz = fsize(image_storage)
if sz >= 10 * 1024 * 1024:
logger.info("[wechat_kf] image too large, compressing, sz={}".format(sz))
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
image_storage.seek(0)
try:
response = self.client.media.upload("image", image_storage)
except WeChatClientException as e:
logger.error("[wechat_kf] upload image failed: {}".format(e))
return
self._send_image(external_userid, open_kfid, response["media_id"])
logger.info("[wechat_kf] sendImage url={}, receiver={}".format(img_url, receiver))
elif reply.type == ReplyType.IMAGE:
image_storage = reply.content
sz = fsize(image_storage)
if sz >= 10 * 1024 * 1024:
logger.info("[wechat_kf] image too large, compressing, sz={}".format(sz))
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
image_storage.seek(0)
try:
response = self.client.media.upload("image", image_storage)
except WeChatClientException as e:
logger.error("[wechat_kf] upload image failed: {}".format(e))
return
self._send_image(external_userid, open_kfid, response["media_id"])
logger.info("[wechat_kf] sendImage, receiver={}".format(receiver))
elif reply.type == ReplyType.VIDEO_URL:
video_url = reply.content
try:
response = self.client.media.upload(
"video", requests.get(video_url, stream=True).content
)
except WeChatClientException as e:
logger.error("[wechat_kf] upload video failed: {}".format(e))
return
self._send_video(external_userid, open_kfid, response["media_id"])
logger.info("[wechat_kf] sendVideo url={}, receiver={}".format(video_url, receiver))
elif reply.type == ReplyType.FILE:
file_path = reply.content
try:
with open(file_path, "rb") as f:
response = self.client.media.upload(
"file", (os.path.basename(file_path), f.read())
)
except WeChatClientException as e:
logger.error("[wechat_kf] upload file failed: {}".format(e))
return
self._send_file(external_userid, open_kfid, response["media_id"])
logger.info("[wechat_kf] sendFile={}, receiver={}".format(file_path, receiver))
else:
logger.warning("[wechat_kf] unsupported reply type: {}".format(reply.type))
# ------------------------------------------------------------------
# Inbound — pull messages by cursor
# ------------------------------------------------------------------
def _get_kf_lock(self, open_kfid: str) -> threading.Lock:
with self._kf_locks_guard:
return self._kf_locks[open_kfid]
def submit_callback(self, token: str, open_kfid: str):
"""
Async entry point used by the HTTP handler. Submits the actual
sync_msg pulling to a background thread so the callback response
can return within WeCom's 5s deadline.
"""
try:
self._callback_executor.submit(self._run_callback, token, open_kfid)
except RuntimeError as e:
# Executor may be shut down during process exit; fall back
# to inline execution so we don't silently drop the event.
logger.warning(f"[wechat_kf] executor unavailable, run inline: {e}")
self._run_callback(token, open_kfid)
def _run_callback(self, token: str, open_kfid: str):
# Block on the per-kfid lock so retried callbacks queue up
# behind the in-flight one. The queued worker will then call
# sync_msg with the (already advanced) cursor, which is cheap
# when there is nothing new and still picks up any messages
# that arrived after the previous worker's last pull.
lock = self._get_kf_lock(open_kfid)
with lock:
try:
self.consume_callback(token, open_kfid)
except Exception as e:
logger.exception(f"[wechat_kf] consume_callback error: {e}")
def consume_callback(self, token: str, open_kfid: str):
"""
Called from the HTTP `Query.POST` handler whenever WeCom notifies
us that there are new messages for `open_kfid`. Pulls all new
messages via sync_msg and feeds them into `produce()`.
"""
existing_cursor = self.cursor_store.get(open_kfid)
# First-time bootstrap: always skip history, otherwise WeCom would
# replay up to 14 days of messages on the very first callback and
# flood every user with auto-replies.
if not existing_cursor:
self._initialize_cursor(token, open_kfid)
return
msgs = self._pull_messages(token, open_kfid, existing_cursor)
if not msgs:
return
file_cache = get_file_cache()
for raw in msgs:
try:
kf_msg = WechatKfMessage(msg=raw, client=self.client)
except NotImplementedError as e:
logger.debug("[wechat_kf] {}".format(e))
continue
session_id = kf_msg.from_user_id
# Cache lone images/files and wait for the user's follow-up
# text. Agent mode never reads memory.USER_IMAGE_CACHE, so
# without this the attachment is effectively lost.
if kf_msg.ctype in (ContextType.IMAGE, ContextType.FILE):
ftype = "image" if kf_msg.ctype == ContextType.IMAGE else "file"
try:
kf_msg.prepare() # download to local tmp path
file_cache.add(session_id, kf_msg.content, file_type=ftype)
logger.info(
"[wechat_kf] {} cached for session {}: {}".format(
ftype, session_id, kf_msg.content
)
)
except Exception as e:
logger.warning(f"[wechat_kf] cache {ftype} failed: {e}")
continue
# On a text turn, attach any pending images/files as references
# so the downstream agent can pick them up via the text content.
# Paths are already under agent_workspace/tmp (see
# WechatKfMessage._get_tmp_dir), so a relative ref also works.
if kf_msg.ctype == ContextType.TEXT:
cached_files = file_cache.get(session_id)
if cached_files:
refs = []
for fi in cached_files:
ftype, fpath = fi["type"], fi["path"]
if ftype == "image":
refs.append(f"[图片: {fpath}]")
else:
refs.append(f"[文件: {fpath}]")
kf_msg.content = kf_msg.content + "\n" + "\n".join(refs)
file_cache.clear(session_id)
context = self._compose_context(
kf_msg.ctype,
kf_msg.content,
isgroup=False,
msg=kf_msg,
)
if context:
self.produce(context)
time.sleep(0.05) # tiny gap between messages of the same batch
def _initialize_cursor(self, token: str, open_kfid: str):
"""
Drain all current messages for this `open_kfid` without producing
any context, just to advance the cursor to "now". This prevents
a fresh deployment from replying to up to ~14 days of history.
"""
next_cursor = ""
total_skipped = 0
while True:
data = self._call_sync_msg(token, open_kfid, next_cursor)
if data is None:
break
msg_list = data.get("msg_list") or []
total_skipped += len(msg_list)
cursor_after = data.get("next_cursor") or ""
if cursor_after:
self.cursor_store.set(open_kfid, cursor_after)
if not data.get("has_more"):
break
if not cursor_after or cursor_after == next_cursor:
break
next_cursor = cursor_after
logger.info(
"[wechat_kf] first-start bootstrap finished for open_kfid={}, "
"skipped {} historical messages".format(open_kfid, total_skipped)
)
def _pull_messages(self, token: str, open_kfid: str, next_cursor: Optional[str]) -> list:
"""Loop sync_msg until `has_more` is false. Returns raw msg dicts."""
collected = []
cursor = next_cursor or ""
while True:
data = self._call_sync_msg(token, open_kfid, cursor)
if data is None:
break
for item in data.get("msg_list") or []:
# Only consume messages from external users; ignore replies
# generated by our own kf account, otherwise we would loop
# back into ourselves.
if not item.get("external_userid"):
continue
if item.get("msgtype") in ("text", "image", "voice", "file"):
collected.append(item)
cursor_after = data.get("next_cursor") or ""
if cursor_after:
self.cursor_store.set(open_kfid, cursor_after)
if not data.get("has_more"):
break
if not cursor_after or cursor_after == cursor:
break
cursor = cursor_after
if collected:
collected = _dedup_image_text_pair(collected)
logger.info(
"[wechat_kf] pulled {} messages for open_kfid={}".format(len(collected), open_kfid)
)
return collected
def _call_sync_msg(self, token: str, open_kfid: str, cursor: str) -> Optional[dict]:
# `client.access_token` is the cached string property; do not use
# `fetch_access_token()` here — wechatpy returns the raw response
# dict from that call, which corrupts the query string.
url = f"{KF_API_BASE}/sync_msg?access_token={self.client.access_token}"
payload = {
"token": token,
"open_kfid": open_kfid,
"limit": SYNC_MSG_LIMIT,
}
if cursor:
payload["cursor"] = cursor
try:
resp = requests.post(url, json=payload, timeout=10).json()
except Exception as e:
logger.error(f"[wechat_kf] sync_msg request failed: {e}")
return None
if resp.get("errcode") != 0:
logger.error(
f"[wechat_kf] sync_msg errcode={resp.get('errcode')}, "
f"errmsg={resp.get('errmsg')}, open_kfid={open_kfid}"
)
return None
return resp
# ------------------------------------------------------------------
# Outbound HTTP wrappers (kf/send_msg)
# ------------------------------------------------------------------
def _post_send_msg(self, payload: dict) -> dict:
url = f"{KF_API_BASE}/send_msg?access_token={self.client.access_token}"
try:
resp = requests.post(url, json=payload, timeout=10).json()
except Exception as e:
logger.error(f"[wechat_kf] send_msg request failed: {e}")
return {"errcode": -1, "errmsg": str(e)}
if resp.get("errcode") != 0:
logger.error(f"[wechat_kf] send_msg failed, payload={payload}, resp={resp}")
return resp
def _send_text(self, external_userid: str, open_kfid: str, content: str) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "text",
"text": {"content": content},
})
def _send_image(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "image",
"image": {"media_id": media_id},
})
def _send_voice(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "voice",
"voice": {"media_id": media_id},
})
def _send_video(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "video",
"video": {"media_id": media_id},
})
def _send_file(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "file",
"file": {"media_id": media_id},
})
def _send_link(self, external_userid: str, open_kfid: str, link_data: dict) -> dict:
return self._post_send_msg({
"touser": external_userid,
"open_kfid": open_kfid,
"msgtype": "link",
"link": link_data,
})
def _dedup_image_text_pair(messages: list) -> list:
"""
A WeChat user often sends an image immediately followed by a text
question (e.g. "what's in this picture?"). Only when the batch is
exactly that 2-message image+text pair within a 5s window do we
collapse it into a single [image, text] turn. Otherwise return
every message so rapid-fire texts/images are all processed —
cursor freshness is already guaranteed by sync_msg.
"""
if not messages:
return []
if len(messages) == 2:
a, b = messages
types = {a["msgtype"], b["msgtype"]}
if types == {"image", "text"} and abs(a["send_time"] - b["send_time"]) <= 5:
img = a if a["msgtype"] == "image" else b
txt = b if a["msgtype"] == "image" else a
return [img, txt]
return messages
# ----------------------------------------------------------------------
# HTTP handlers (web.py)
# ----------------------------------------------------------------------
class Query:
def GET(self):
channel = WechatKfChannel()
params = web.input()
logger.info("[wechat_kf] verify params: {}".format(params))
try:
signature = params.msg_signature
timestamp = params.timestamp
nonce = params.nonce
echostr = params.echostr
echostr = channel.crypto.check_signature(signature, timestamp, nonce, echostr)
except (InvalidSignatureException, InvalidCorpIdException):
raise web.Forbidden()
return echostr
def POST(self):
channel = WechatKfChannel()
params = web.input()
try:
signature = params.msg_signature
timestamp = params.timestamp
nonce = params.nonce
raw_body = web.data()
decrypted = channel.crypto.decrypt_message(raw_body, signature, timestamp, nonce)
except (InvalidSignatureException, InvalidCorpIdException) as e:
logger.warning(f"[wechat_kf] invalid signature: {e}")
raise web.Forbidden()
# We need the Token + OpenKfId fields from the inner XML to call
# sync_msg. wechatpy's parsed object exposes neither, so we parse
# the raw XML directly.
try:
root = ET.fromstring(decrypted)
except ET.ParseError as e:
logger.error(f"[wechat_kf] xml parse error: {e}")
return "success"
msg_type = (root.findtext("MsgType") or "").strip()
event = (root.findtext("Event") or "").strip()
if msg_type != "event" or event != "kf_msg_or_event":
logger.debug(
f"[wechat_kf] ignored callback msg_type={msg_type}, event={event}"
)
return "success"
token = root.findtext("Token") or ""
open_kfid = root.findtext("OpenKfId") or ""
if not token or not open_kfid:
logger.warning(
f"[wechat_kf] callback missing token or open_kfid: {decrypted}"
)
return "success"
# Hand off to a background worker — WeCom requires the callback
# to return success within ~5 seconds, otherwise it will retry
# and we may race the same cursor window into duplicate replies.
channel.submit_callback(token, open_kfid)
return "success"

View File

@@ -0,0 +1,80 @@
# -*- coding=utf-8 -*-
"""
Local-file based persistence for WeCom customer-service `next_cursor`.
Why we need this:
The WeCom customer-service (微信客服) callback only notifies us that
"new messages exist". To actually fetch them we must call the
`cgi-bin/kf/sync_msg` endpoint with a `cursor` so that we only get
messages newer than the previously processed one. If we lose this
cursor (e.g. on process restart) WeCom will replay up to ~14 days of
history, which would cause the bot to flood users with duplicate
replies.
This implementation deliberately avoids any external dependency
(no Redis / no DB) — a single JSON file under the project's tmp dir is
enough for a CoW-style single-process deployment.
"""
import json
import os
import threading
from typing import Optional
from common.log import logger
class CursorStore:
"""Thread-safe per-`open_kfid` cursor store backed by a JSON file."""
def __init__(self, file_path: str):
self._file_path = file_path
self._lock = threading.Lock()
self._data = self._load()
def _load(self) -> dict:
try:
if os.path.exists(self._file_path):
with open(self._file_path, "r", encoding="utf-8") as f:
return json.load(f) or {}
except Exception as e:
logger.warning(f"[wechat_kf] failed to load cursor file {self._file_path}: {e}")
return {}
def _flush_locked(self):
# Atomic write: write to *.tmp first then rename, avoid corruption on crash.
tmp_path = self._file_path + ".tmp"
try:
os.makedirs(os.path.dirname(self._file_path) or ".", exist_ok=True)
with open(tmp_path, "w", encoding="utf-8") as f:
json.dump(self._data, f, ensure_ascii=False)
os.replace(tmp_path, self._file_path)
# Tighten permissions: cursor file lives in $HOME, restrict to owner.
# No-op on Windows.
try:
os.chmod(self._file_path, 0o600)
except Exception:
pass
except Exception as e:
logger.warning(f"[wechat_kf] failed to flush cursor file {self._file_path}: {e}")
try:
if os.path.exists(tmp_path):
os.remove(tmp_path)
except Exception:
pass
def get(self, open_kfid: str) -> Optional[str]:
with self._lock:
return self._data.get(open_kfid)
def set(self, open_kfid: str, cursor: str):
if not cursor:
return
with self._lock:
if self._data.get(open_kfid) == cursor:
return
self._data[open_kfid] = cursor
self._flush_locked()
def has(self, open_kfid: str) -> bool:
with self._lock:
return open_kfid in self._data

View File

@@ -0,0 +1,134 @@
# -*- coding=utf-8 -*-
"""
Adapter that turns a single `sync_msg` item from WeCom customer-service
into a CoW `ChatMessage` object.
"""
import os
import re
from wechatpy.enterprise import WeChatClient
from bridge.context import ContextType
from channel.chat_message import ChatMessage
from common.log import logger
from common.utils import expand_path
from config import conf
def _get_tmp_dir() -> str:
"""Save under agent_workspace/tmp/ so agent tools (e.g. `read`) can
resolve a relative path like `tmp/xxx.pdf` against their own
workspace root. Mirrors the convention used by weixin / wecom_bot.
"""
ws_root = expand_path(conf().get("agent_workspace", "~/cow"))
tmp_dir = os.path.join(ws_root, "tmp")
os.makedirs(tmp_dir, exist_ok=True)
return tmp_dir
def _extract_filename(content_disposition: str) -> str:
"""Best-effort parse of `filename` / `filename*` from a Content-Disposition
header. Returns '' when nothing usable is found."""
if not content_disposition:
return ""
# RFC 5987 form: filename*=UTF-8''xxx
m = re.search(r"filename\*=(?:[^'\"]*'[^']*'\s*)?([^;]+)", content_disposition)
if m:
try:
from urllib.parse import unquote
return unquote(m.group(1).strip().strip('"'))
except Exception:
return m.group(1).strip().strip('"')
m = re.search(r'filename\s*=\s*"?([^";]+)"?', content_disposition)
return m.group(1).strip() if m else ""
class WechatKfMessage(ChatMessage):
"""
msg structure (from cgi-bin/kf/sync_msg):
{
"msgid": "...",
"send_time": 1700000000,
"origin": 3,
"msgtype": "text" | "image" | "voice" | ...,
"open_kfid": "wkxxxx",
"external_userid": "wmxxxx",
"text": {"content": "..."},
"image": {"media_id": "..."},
"voice": {"media_id": "..."},
...
}
"""
def __init__(self, msg: dict, client: WeChatClient = None, is_group: bool = False):
# NOTE: skip parent constructor because it expects a wechatpy parsed
# message object, while here we receive a raw dict from sync_msg.
super().__init__(msg)
self.is_group = is_group
self.msg_id = msg.get("msgid")
self.create_time = msg.get("send_time")
self.origin = msg.get("origin")
self.msgtype = msg.get("msgtype")
self.open_kfid = msg.get("open_kfid")
self.external_userid = msg.get("external_userid")
if self.msgtype == "text":
self.ctype = ContextType.TEXT
self.content = msg.get("text", {}).get("content", "")
elif self.msgtype == "image":
self.ctype = ContextType.IMAGE
media_id = msg.get("image", {}).get("media_id", "")
self.content = os.path.join(_get_tmp_dir(), media_id + ".jpg")
def download_image():
response = client.media.download(media_id)
if response.status_code == 200:
with open(self.content, "wb") as f:
f.write(response.content)
else:
logger.info(f"[wechat_kf] Failed to download image, {response.content}")
self._prepare_fn = download_image
elif self.msgtype == "voice":
self.ctype = ContextType.VOICE
media_id = msg.get("voice", {}).get("media_id", "")
# WeCom returns amr by default; downstream voice pipeline will convert.
self.content = os.path.join(_get_tmp_dir(), media_id + ".amr")
def download_voice():
response = client.media.download(media_id)
if response.status_code == 200:
with open(self.content, "wb") as f:
f.write(response.content)
else:
logger.info(f"[wechat_kf] Failed to download voice, {response.content}")
self._prepare_fn = download_voice
elif self.msgtype == "file":
self.ctype = ContextType.FILE
media_id = msg.get("file", {}).get("media_id", "")
# Provisional path; rewritten in download_file() once we have
# the original filename from Content-Disposition.
self.content = os.path.join(_get_tmp_dir(), media_id)
def download_file():
response = client.media.download(media_id)
if response.status_code == 200:
filename = _extract_filename(
response.headers.get("Content-Disposition", "")
) or media_id
self.content = os.path.join(_get_tmp_dir(), filename)
with open(self.content, "wb") as f:
f.write(response.content)
else:
logger.info(f"[wechat_kf] Failed to download file, {response.content}")
self._prepare_fn = download_file
else:
raise NotImplementedError(
f"[wechat_kf] Unsupported message type: {self.msgtype}"
)
self.from_user_id = self.external_userid
self.to_user_id = self.open_kfid
self.other_user_id = self.external_userid

View File

@@ -19,9 +19,15 @@ def verify_server(data):
nonce = data.nonce
echostr = data.get("echostr", None)
token = conf().get("wechatmp_token") # 请按照公众平台官网\基本配置中信息填写
# Reject when token is empty: an empty token reduces signature verification
# to a predictable hash over attacker-controlled values.
if not token:
raise web.Forbidden("wechatmp_token is not configured")
check_signature(token, signature, timestamp, nonce)
return echostr
except InvalidSignatureException:
raise web.Forbidden("Invalid signature")
except web.Forbidden:
raise
except Exception as e:
raise web.Forbidden(str(e))

View File

@@ -103,14 +103,21 @@ class Query:
task_running = True
waiting_until = request_time + 4
while time.time() < waiting_until:
if from_user in channel.running:
time.sleep(0.1)
else:
if from_user not in channel.running:
task_running = False
break
# Task still running, but if it has already produced cached
# segments (e.g. multi-turn thinking output), return them now
# instead of forcing the user to wait for the whole task. The
# remaining segments are fetched by the user's next message.
if channel.cache_dict.get(from_user):
break
time.sleep(0.1)
reply_text = ""
if task_running:
# Only fall back to retry / "thinking" hint when the task is still
# running AND there is nothing cached to send yet.
if task_running and not channel.cache_dict.get(from_user):
if request_cnt < 3:
# waiting for timeout (the POST request will be closed by Wechat official server)
time.sleep(2)
@@ -131,8 +138,22 @@ class Query:
# Only one request can access to the cached data
try:
(reply_type, reply_content) = channel.cache_dict[from_user].pop(0)
if not channel.cache_dict[from_user]: # If popping the message makes the list empty, delete the user entry from cache
# WeChat passive reply allows only a single reply per request.
# To avoid forcing the user to send an extra message for every
# segment of multi-turn agent output, drain all consecutive
# cached text segments at once and merge them into one reply.
# Media (voice/image) can only be returned one at a time, so it
# stops the merge and is returned on its own.
cached = channel.cache_dict[from_user]
if cached[0][0] == "text":
reply_type = "text"
merged_parts = []
while cached and cached[0][0] == "text":
merged_parts.append(cached.pop(0)[1])
reply_content = "\n\n".join(merged_parts)
else:
(reply_type, reply_content) = cached.pop(0)
if not channel.cache_dict[from_user]: # If draining empties the list, delete the user entry from cache
del channel.cache_dict[from_user]
except IndexError:
return "success"

View File

@@ -134,10 +134,16 @@ class WechatMPChannel(ChatChannel):
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
img_url = reply.content
pic_res = requests.get(img_url, stream=True)
image_storage = io.BytesIO()
for block in pic_res.iter_content(1024):
image_storage.write(block)
if img_url.startswith("file://") or os.path.isfile(img_url):
# Local file produced by the agent (e.g. a generated image)
local_path = img_url[len("file://"):] if img_url.startswith("file://") else img_url
with open(local_path, "rb") as f:
image_storage.write(f.read())
else:
pic_res = requests.get(img_url, stream=True)
for block in pic_res.iter_content(1024):
image_storage.write(block)
image_storage.seek(0)
image_type = imghdr.what(image_storage)
filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
@@ -258,10 +264,16 @@ class WechatMPChannel(ChatChannel):
logger.info("[wechatmp] Do send voice to {}".format(receiver))
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
img_url = reply.content
pic_res = requests.get(img_url, stream=True)
image_storage = io.BytesIO()
for block in pic_res.iter_content(1024):
image_storage.write(block)
if img_url.startswith("file://") or os.path.isfile(img_url):
# Local file produced by the agent (e.g. a generated image)
local_path = img_url[len("file://"):] if img_url.startswith("file://") else img_url
with open(local_path, "rb") as f:
image_storage.write(f.read())
else:
pic_res = requests.get(img_url, stream=True)
for block in pic_res.iter_content(1024):
image_storage.write(block)
image_storage.seek(0)
image_type = imghdr.what(image_storage)
filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type

View File

@@ -12,16 +12,19 @@ import hashlib
import json
import math
import os
import re
import threading
import time
import uuid
import requests
import web
import websocket
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel, check_prefix
from channel.wecom_bot.wecom_bot_crypt import WecomBotCrypt
from channel.wecom_bot.wecom_bot_message import WecomBotMessage
from common.expired_dict import ExpiredDict
from common.log import logger
@@ -32,6 +35,9 @@ from config import conf
WECOM_WS_URL = "wss://openws.work.weixin.qq.com"
HEARTBEAT_INTERVAL = 30
MEDIA_CHUNK_SIZE = 512 * 1024 # 512KB per chunk (before base64 encoding)
# Fixed URL path for the callback (webhook) HTTP server. The bot's
# receive-message URL must point at this path, e.g. http://host:9892/wecombot
CALLBACK_PATH = "/wecombot"
def _escape_control_chars_inside_json_strings(s: str) -> str:
@@ -97,6 +103,14 @@ class WecomBotChannel(ChatChannel):
self._pending_lock = threading.Lock()
self._stream_states = {} # req_id -> {"stream_id": str, "content": str}
# Transport mode: "websocket" (long connection) or "webhook" (HTTP callback)
self.mode = "websocket"
self._crypt = None
self._http_server = None
# stream_id -> {"committed", "current", "finished", "images", "last_access"}
self._callback_streams = ExpiredDict(60 * 10) # auto-expire after 10min (max poll window is 6min)
self._callback_lock = threading.Lock()
conf()["group_name_white_list"] = ["ALL_GROUP"]
conf()["single_chat_prefix"] = [""]
@@ -105,6 +119,11 @@ class WecomBotChannel(ChatChannel):
# ------------------------------------------------------------------
def startup(self):
self.mode = conf().get("wecom_bot_mode", "websocket")
if self.mode == "webhook":
self._startup_callback()
return
self.bot_id = conf().get("wecom_bot_id", "")
self.bot_secret = conf().get("wecom_bot_secret", "")
@@ -127,6 +146,13 @@ class WecomBotChannel(ChatChannel):
pass
self._ws = None
self._connected = False
if self._http_server:
try:
self._http_server.stop()
logger.info("[WecomBot] Callback HTTP server stopped")
except Exception as e:
logger.warning(f"[WecomBot] Error stopping HTTP server: {e}")
self._http_server = None
# ------------------------------------------------------------------
# WebSocket connection
@@ -183,6 +209,192 @@ class WecomBotChannel(ChatChannel):
def _gen_req_id(self) -> str:
return uuid.uuid4().hex[:16]
# ------------------------------------------------------------------
# Callback (webhook) mode
# ------------------------------------------------------------------
def _startup_callback(self):
"""Start an HTTP server that receives encrypted callbacks (webhook mode).
The bot's "接收消息" URL in the WeCom admin console should point at this
server (any path is accepted). Verification (GET) and message delivery
(POST) are both handled by ``WecomBotCallbackController``.
"""
token = conf().get("wecom_bot_token", "")
aes_key = conf().get("wecom_bot_encoding_aes_key", "")
if not token or not aes_key:
err = "[WecomBot] callback mode requires wecom_bot_token and wecom_bot_encoding_aes_key"
logger.error(err)
self.report_startup_error(err)
return
try:
# Enterprise-internal smart bot: receive_id is an empty string.
self._crypt = WecomBotCrypt(token, aes_key, "")
except Exception as e:
err = f"[WecomBot] invalid callback credentials: {e}"
logger.error(err)
self.report_startup_error(err)
return
port = int(conf().get("wecom_bot_port", 9892))
logger.info(f"[WecomBot] Starting callback (webhook) server on port {port}, path {CALLBACK_PATH} ...")
# Only serve the fixed callback path; everything else 404s instead of being
# treated as a (signature-failing) WeCom callback.
urls = (re.escape(CALLBACK_PATH), "channel.wecom_bot.wecom_bot_channel.WecomBotCallbackController")
app = web.application(urls, globals(), autoreload=False)
func = web.httpserver.StaticMiddleware(app.wsgifunc())
func = web.httpserver.LogMiddleware(func)
server = web.httpserver.WSGIServer(("0.0.0.0", port), func)
self._http_server = server
self.report_startup_success()
try:
server.start()
except (KeyboardInterrupt, SystemExit):
server.stop()
def _new_callback_stream(self, response_url: str = "") -> str:
"""Create a new stream state and return its id."""
stream_id = uuid.uuid4().hex[:16]
now = time.time()
with self._callback_lock:
self._callback_streams[stream_id] = {
"committed": "",
"current": "",
"finished": False,
"images": [], # list of (base64_str, md5_str), flushed only at finish
"image_urls": [], # public http(s) image urls (usable in response_url markdown)
"image_pending": False, # an image reply is being prepared; don't finish on text yet
"last_access": now,
"created_at": now,
"response_url": response_url or "",
"delivered": False, # final answer handed to WeCom via a poll
"url_sent": False, # final answer pushed via response_url (active reply)
}
return stream_id
def _callback_handle_message(self, data: dict) -> dict:
"""Handle a freshly-received user message in callback mode.
Produces the context for async processing and returns the initial passive
reply (a stream packet with finish=false) so WeCom starts polling for the
agent's streamed answer. Returns ``None`` when there's nothing to reply
(e.g. an image/file silently cached for the next query).
"""
msg_id = data.get("msgid", "")
if msg_id and self.received_msgs.get(msg_id):
logger.debug(f"[WecomBot] Duplicate msg filtered: {msg_id}")
return None
if msg_id:
self.received_msgs[msg_id] = True
chattype = data.get("chattype", "single")
is_group = chattype == "group"
default_aeskey = conf().get("wecom_bot_encoding_aes_key", "")
result = self._build_context(data, is_group, default_aeskey=default_aeskey)
if not result:
return None
context, wecom_msg = result
# response_url lets us actively reply once within 1h, used as a fallback
# when the agent finishes after WeCom stops polling (max ~6min window).
response_url = data.get("response_url", "") or ""
stream_id = self._new_callback_stream(response_url=response_url)
wecom_msg.stream_id = stream_id
context["wecom_stream_id"] = stream_id
context["on_event"] = self._make_callback_stream_callback(stream_id)
self.produce(context)
# First passive reply: register the stream id, WeCom will poll for updates.
return {
"msgtype": "stream",
"stream": {"id": stream_id, "finish": False, "content": ""},
}
def _callback_handle_stream_poll(self, data: dict) -> dict:
"""Handle a "流式消息刷新" poll: return the latest accumulated content."""
stream_id = data.get("stream", {}).get("id", "")
with self._callback_lock:
state = self._callback_streams.get(stream_id)
if state is None:
# Unknown / expired stream: tell WeCom we're done to stop polling.
return {"msgtype": "stream", "stream": {"id": stream_id, "finish": True, "content": ""}}
state["last_access"] = time.time()
if state.get("url_sent"):
# Final answer already pushed via response_url; finish silently.
return {"msgtype": "stream", "stream": {"id": stream_id, "finish": True, "content": ""}}
# We never force-finish on a timer: while a task is still running the
# bubble should keep spinning until either the task finishes or the
# user cancels. If WeCom's 6min window closes before completion, the
# answer is delivered later via response_url instead.
finished = state["finished"]
content = state["committed"] + state["current"]
images = state["images"] if finished else []
if finished:
state["delivered"] = True
logger.debug(f"[WecomBot] stream {stream_id} delivered via poll, len={len(content)}, images={len(images)}")
stream = {"id": stream_id, "finish": finished, "content": content}
if images:
stream["msg_item"] = [
{"msgtype": "image", "image": {"base64": b64, "md5": md5}}
for (b64, md5) in images
]
return {"msgtype": "stream", "stream": stream}
def _make_callback_stream_callback(self, stream_id: str):
"""Build an on_event callback that accumulates agent output into stream state.
Mirrors the websocket streaming behaviour: intermediate turns (text before
a tool call) are committed with a '---' separator; WeCom reads the full
accumulated content on each poll.
"""
def on_event(event: dict):
event_type = event.get("type")
edata = event.get("data", {})
cancelled = False
with self._callback_lock:
state = self._callback_streams.get(stream_id)
if not state:
return
if event_type == "turn_start":
state["current"] = ""
elif event_type == "message_update":
delta = edata.get("delta", "")
if delta:
state["current"] += delta
elif event_type == "message_end":
tool_calls = edata.get("tool_calls", [])
if tool_calls:
if state["current"].strip():
state["committed"] += state["current"].strip() + "\n\n---\n\n"
state["current"] = ""
else:
state["committed"] += state["current"]
state["current"] = ""
elif event_type == "agent_cancelled":
# Mechanism 1: a cancelled run never reaches send(), so finalize
# its stream here to stop the "···" bubble immediately.
if state["current"]:
state["committed"] += state["current"]
state["current"] = ""
state["committed"] = state["committed"].rstrip()
if state["committed"].endswith("---"):
state["committed"] = state["committed"][:-3].rstrip()
if not state["committed"].strip():
state["committed"] = "🛑 已中止"
state["finished"] = True
state["last_access"] = time.time()
cancelled = True
if cancelled:
# Outside the lock: response_url fallback re-acquires it.
self._schedule_response_url_fallback(stream_id)
return on_event
# ------------------------------------------------------------------
# Subscribe & heartbeat
# ------------------------------------------------------------------
@@ -287,16 +499,31 @@ class WecomBotChannel(ChatChannel):
chattype = body.get("chattype", "single")
is_group = chattype == "group"
result = self._build_context(body, is_group)
if not result:
return
context, wecom_msg = result
wecom_msg.req_id = req_id
if req_id:
context["on_event"] = self._make_stream_callback(req_id)
self.produce(context)
def _build_context(self, body: dict, is_group: bool, default_aeskey: str = ""):
"""Parse a wecom message body into a Context, applying file-cache logic.
Shared by both the websocket (long-connection) and callback (webhook)
receive paths. Returns ``(context, wecom_msg)`` when the message should be
handed to the agent, or ``None`` when it was consumed (cached image/file,
parse failure, etc.).
"""
try:
wecom_msg = WecomBotMessage(body, is_group=is_group)
wecom_msg = WecomBotMessage(body, is_group=is_group, default_aeskey=default_aeskey)
except NotImplementedError as e:
logger.warning(f"[WecomBot] {e}")
return
return None
except Exception as e:
logger.error(f"[WecomBot] Failed to parse message: {e}", exc_info=True)
return
wecom_msg.req_id = req_id
return None
# File cache logic (same pattern as feishu)
from channel.file_cache import get_file_cache
@@ -314,13 +541,13 @@ class WecomBotChannel(ChatChannel):
if hasattr(wecom_msg, "image_path") and wecom_msg.image_path:
file_cache.add(session_id, wecom_msg.image_path, file_type="image")
logger.info(f"[WecomBot] Image cached for session {session_id}")
return
return None
if wecom_msg.ctype == ContextType.FILE:
wecom_msg.prepare()
file_cache.add(session_id, wecom_msg.content, file_type="file")
logger.info(f"[WecomBot] File cached for session {session_id}: {wecom_msg.content}")
return
return None
if wecom_msg.ctype == ContextType.TEXT:
cached_files = file_cache.get(session_id)
@@ -346,10 +573,9 @@ class WecomBotChannel(ChatChannel):
msg=wecom_msg,
no_need_at=True,
)
if context:
if req_id:
context["on_event"] = self._make_stream_callback(req_id)
self.produce(context)
if not context:
return None
return context, wecom_msg
# ------------------------------------------------------------------
# Event callback
@@ -440,6 +666,17 @@ class WecomBotChannel(ChatChannel):
state["current"] = ""
_push_stream(state, force=True)
elif event_type == "agent_cancelled":
# Flush partial output and strip trailing "---" separator
# left over from previous turn, to avoid a dangling divider.
if state["current"]:
state["committed"] += state["current"]
state["current"] = ""
state["committed"] = state["committed"].rstrip()
if state["committed"].endswith("---"):
state["committed"] = state["committed"][:-3].rstrip()
_push_stream(state, force=True)
return on_event
# ------------------------------------------------------------------
@@ -479,11 +716,233 @@ class WecomBotChannel(ChatChannel):
return context
# ------------------------------------------------------------------
# Callback (webhook) send: write the final reply into the stream state
# so the next "流式消息刷新" poll returns it with finish=true.
# ------------------------------------------------------------------
def _callback_send(self, reply: Reply, context: Context):
msg = context.get("msg")
stream_id = getattr(msg, "stream_id", None) if msg else None
if not stream_id:
stream_id = context.get("wecom_stream_id")
if not stream_id:
logger.warning("[WecomBot] callback send without stream_id, dropping reply")
return
if reply.type == ReplyType.TEXT:
self._callback_finalize_text(stream_id, reply.content)
elif reply.type in (ReplyType.IMAGE_URL, ReplyType.IMAGE):
self._callback_finalize_image(stream_id, reply.content)
elif reply.type == ReplyType.FILE:
# Passive callback replies only support text + image (base64); files
# are not supported by the protocol, so append a notice to whatever
# text the agent already streamed (do not drop it).
text = getattr(reply, "text_content", "") or ""
note = (text + "\n\n" if text else "") + "(文件无法在企微回调模式下直接发送)"
self._callback_finalize_text(stream_id, note, append=True)
elif reply.type in (ReplyType.VIDEO, ReplyType.VIDEO_URL, ReplyType.VOICE):
logger.warning(f"[WecomBot] reply type {reply.type} not supported in callback mode")
text = getattr(reply, "text_content", "") or ""
note = (text + "\n\n" if text else "") + "(该消息类型无法在企微回调模式下直接发送)"
self._callback_finalize_text(stream_id, note, append=True)
else:
self._callback_finalize_text(stream_id, str(reply.content))
def _callback_get_or_create_state(self, stream_id: str) -> dict:
state = self._callback_streams.get(stream_id)
if state is None:
now = time.time()
state = {
"committed": "",
"current": "",
"finished": False,
"images": [],
"image_urls": [],
"image_pending": False,
"last_access": now,
"created_at": now,
"response_url": "",
"delivered": False,
"url_sent": False,
}
self._callback_streams[stream_id] = state
return state
def _callback_finalize_text(self, stream_id: str, content: str, append: bool = False):
with self._callback_lock:
state = self._callback_get_or_create_state(stream_id)
accumulated = (state["committed"] + state["current"]).strip()
if append and accumulated:
state["committed"] = (accumulated + "\n\n" + (content or "")).strip()
else:
state["committed"] = accumulated if accumulated else (content or "")
state["current"] = ""
state["last_access"] = time.time()
# Don't finish synchronously: chat_channel splits an image-with-caption
# reply into a TEXT send followed (0.3s later) by the IMAGE send. If the
# text finished the stream immediately, WeCom would close it before the
# image arrives. Defer the finish so a trailing image can merge in.
self._schedule_text_finish(stream_id)
def _schedule_text_finish(self, stream_id: str, delay: float = 1.2):
def _run():
time.sleep(delay)
with self._callback_lock:
state = self._callback_streams.get(stream_id)
if not state or state["finished"] or state.get("image_pending"):
return # already finished, or an image reply is on its way
state["finished"] = True
state["last_access"] = time.time()
self._schedule_response_url_fallback(stream_id)
threading.Thread(target=_run, daemon=True, name=f"wecom-textfin-{stream_id}").start()
def _callback_finalize_image(self, stream_id: str, img_path_or_url: str):
# Mark the image as pending up front (before the slow load/compress) so a
# preceding text finalize won't close the stream while we work.
with self._callback_lock:
self._callback_get_or_create_state(stream_id)["image_pending"] = True
b64md5 = self._load_image_base64(img_path_or_url)
with self._callback_lock:
state = self._callback_get_or_create_state(stream_id)
accumulated = (state["committed"] + state["current"]).strip()
state["current"] = ""
if b64md5:
state["images"].append(b64md5)
state["committed"] = accumulated
# Remember the public url (if any) so the response_url fallback
# can embed it as markdown when the poll window has closed.
if img_path_or_url.startswith(("http://", "https://")):
state["image_urls"].append(img_path_or_url)
else:
state["committed"] = accumulated or "[图片发送失败]"
state["finished"] = True
state["image_pending"] = False
state["last_access"] = time.time()
self._schedule_response_url_fallback(stream_id)
# ------------------------------------------------------------------
# Active reply fallback (response_url): rescue replies that finish after
# WeCom stops polling (the passive stream window is ~6 min from the user's
# message). A short delay lets an in-flight poll deliver first; only if no
# poll picks up the finished answer do we push it actively via response_url.
# ------------------------------------------------------------------
def _schedule_response_url_fallback(self, stream_id: str, delay: float = 3.0):
def _run():
time.sleep(delay)
with self._callback_lock:
state = self._callback_streams.get(stream_id)
if not state:
return
if state.get("delivered") or state.get("url_sent"):
return # a poll already delivered (or fallback already ran)
response_url = state.get("response_url") or ""
if not response_url:
logger.warning(
f"[WecomBot] stream {stream_id} finished after poll window but no response_url; reply dropped"
)
return
content = (state["committed"] + state["current"]).strip()
image_urls = list(state.get("image_urls") or [])
has_images = bool(state.get("images"))
state["url_sent"] = True
self._send_via_response_url(stream_id, response_url, content, image_urls, has_images)
threading.Thread(target=_run, daemon=True, name=f"wecom-respurl-{stream_id}").start()
def _send_via_response_url(self, stream_id, response_url, content, image_urls, has_images):
"""Push a one-shot active markdown reply to response_url (valid 1h, single use)."""
md = content or ""
if image_urls:
md += ("\n\n" if md else "") + "\n".join(f"![]({u})" for u in image_urls)
elif has_images:
md += ("\n\n" if md else "") + "(图片已生成,但因处理超时无法通过回调发送)"
if not md:
md = "(处理完成)"
payload = {"msgtype": "markdown", "markdown": {"content": md}}
try:
resp = requests.post(response_url, json=payload, timeout=15)
logger.info(
f"[WecomBot] response_url active reply sent for {stream_id}: "
f"status={resp.status_code}, body={resp.text[:200]}"
)
except Exception as e:
logger.error(f"[WecomBot] response_url active reply failed for {stream_id}: {e}")
def _load_image_base64(self, img_path_or_url: str):
"""Load a local/remote image, ensure JPG/PNG within 10MB, return (base64, md5)."""
local_path = img_path_or_url
if local_path.startswith("file://"):
local_path = local_path[7:]
# Temp files we create here (downloads/conversions/compressions) must be
# cleaned up afterwards; the caller's original local file must not be.
temp_files = []
try:
if local_path.startswith(("http://", "https://")):
try:
resp = requests.get(local_path, timeout=30)
resp.raise_for_status()
tmp_path = f"/tmp/wecom_cb_img_{uuid.uuid4().hex[:8]}"
with open(tmp_path, "wb") as f:
f.write(resp.content)
temp_files.append(tmp_path)
local_path = tmp_path
except Exception as e:
logger.error(f"[WecomBot] Failed to download image for callback reply: {e}")
return None
if not os.path.exists(local_path):
logger.error(f"[WecomBot] Image file not found: {local_path}")
return None
formatted = self._ensure_image_format(local_path)
if not formatted:
return None
if formatted != local_path:
temp_files.append(formatted)
local_path = formatted
# Unlike the long-connection path (which uploads and sends only a tiny
# media_id), the callback reply embeds the whole image as base64 inside
# an AES-encrypted body that is returned on EVERY poll. Empirically a
# ~1.5MB image (base64 ~2.1MB, encrypted ~2.8MB) makes WeCom reject the
# finish packet and poll forever, so cap well below that.
callback_max_size = 512 * 1024
if os.path.getsize(local_path) > callback_max_size:
compressed = self._compress_image(local_path, callback_max_size)
if compressed:
temp_files.append(compressed)
local_path = compressed
else:
logger.warning("[WecomBot] callback image compress failed; sending original (may be rejected)")
try:
with open(local_path, "rb") as f:
raw = f.read()
return base64.b64encode(raw).decode("utf-8"), hashlib.md5(raw).hexdigest()
except Exception as e:
logger.error(f"[WecomBot] Failed to read image for callback reply: {e}")
return None
finally:
for path in temp_files:
try:
os.remove(path)
except OSError:
pass
# ------------------------------------------------------------------
# Send reply
# ------------------------------------------------------------------
def send(self, reply: Reply, context: Context):
if self.mode == "webhook":
self._callback_send(reply, context)
return
msg = context.get("msg")
is_group = context.get("isgroup", False)
receiver = context.get("receiver", "")
@@ -895,3 +1354,85 @@ class WecomBotChannel(ChatChannel):
else:
logger.error("[WecomBot] Failed to get media_id from finish response")
return media_id
class WecomBotCallbackController:
"""HTTP controller for wecom bot callback (webhook) mode.
- GET : URL verification (echo the decrypted echostr).
- POST : encrypted message / stream-refresh / event callbacks; returns an
encrypted passive reply (or "success" for an empty reply).
"""
@staticmethod
def _channel() -> "WecomBotChannel":
return WecomBotChannel()
def GET(self):
channel = self._channel()
params = web.input(msg_signature="", timestamp="", nonce="", echostr="")
if not channel._crypt:
return "wecom bot callback not ready"
ret, echo = channel._crypt.verify_url(
params.msg_signature, params.timestamp, params.nonce, params.echostr
)
if ret != 0:
logger.error(f"[WecomBot] URL verify failed: ret={ret}")
return "verify fail"
if isinstance(echo, bytes):
echo = echo.decode("utf-8")
return echo
def POST(self):
channel = self._channel()
if not channel._crypt:
return "success"
params = web.input(msg_signature="", timestamp="", nonce="")
body = web.data()
ret, plain = channel._crypt.decrypt_msg(
body, params.msg_signature, params.timestamp, params.nonce
)
if ret != 0:
logger.error(f"[WecomBot] callback decrypt failed: ret={ret}")
return "success"
try:
data = json.loads(plain)
except Exception as e:
logger.error(f"[WecomBot] callback json parse failed: {e}")
return "success"
msgtype = data.get("msgtype", "")
# Stream polls arrive ~1/s; logging each is noisy, so only log non-poll
# callbacks here (poll completion is logged in the stream-poll handler).
if msgtype != "stream":
logger.debug(f"[WecomBot] callback received msgtype={msgtype}")
try:
if msgtype == "stream":
reply = channel._callback_handle_stream_poll(data)
elif msgtype == "event":
event_type = data.get("event", {}).get("eventtype", "")
logger.info(f"[WecomBot] callback event: {event_type}")
reply = None
elif msgtype in ("text", "image", "voice", "file", "video", "mixed"):
reply = channel._callback_handle_message(data)
else:
logger.warning(f"[WecomBot] unsupported callback msgtype: {msgtype}")
reply = None
except Exception as e:
logger.error(f"[WecomBot] callback handling error: {e}", exc_info=True)
reply = None
if not reply:
# Empty reply package is acceptable.
return "success"
plain_reply = json.dumps(reply, ensure_ascii=False)
ret, enc = channel._crypt.encrypt_msg(plain_reply, params.nonce, params.timestamp)
if ret != 0:
logger.error(f"[WecomBot] callback encrypt failed: ret={ret}")
return "success"
web.header("Content-Type", "application/json; charset=utf-8")
return json.dumps(enc, ensure_ascii=False)

View File

@@ -0,0 +1,203 @@
"""
WeCom (企业微信) smart-bot callback message encryption/decryption.
Adapted from the official `WXBizJsonMsgCrypt` sample (JSON variant) used by the
AI bot callback (webhook) mode. The bot's receive-message callback delivers
AES-256-CBC encrypted JSON payloads, and passive replies must be encrypted the
same way before being returned in the HTTP response.
For an enterprise-internal smart bot, ``receive_id`` is always an empty string.
"""
import base64
import hashlib
import random
import socket
import struct
import time
from Crypto.Cipher import AES
from common.log import logger
# Error codes (mirrors the official ierror.py)
WXBizMsgCrypt_OK = 0
WXBizMsgCrypt_ValidateSignature_Error = -40001
WXBizMsgCrypt_ParseJson_Error = -40002
WXBizMsgCrypt_ComputeSignature_Error = -40003
WXBizMsgCrypt_IllegalAesKey = -40004
WXBizMsgCrypt_ValidateCorpid_Error = -40005
WXBizMsgCrypt_EncryptAES_Error = -40006
WXBizMsgCrypt_DecryptAES_Error = -40007
WXBizMsgCrypt_IllegalBuffer = -40008
WXBizMsgCrypt_EncodeBase64_Error = -40009
WXBizMsgCrypt_DecodeBase64_Error = -40010
WXBizMsgCrypt_GenReturnJson_Error = -40011
class FormatException(Exception):
pass
def _gen_sha1(token, timestamp, nonce, encrypt):
"""Compute the WeCom message signature with SHA1 over the sorted parts."""
try:
if isinstance(encrypt, bytes):
encrypt = encrypt.decode("utf-8")
sortlist = [str(token), str(timestamp), str(nonce), str(encrypt)]
sortlist.sort()
sha = hashlib.sha1()
sha.update("".join(sortlist).encode("utf-8"))
return WXBizMsgCrypt_OK, sha.hexdigest()
except Exception as e:
logger.error(f"[WecomBot] compute signature error: {e}")
return WXBizMsgCrypt_ComputeSignature_Error, None
class _PKCS7Encoder:
"""PKCS#7 padding with a 32-byte block size (AES-256)."""
block_size = 32
def encode(self, text: bytes) -> bytes:
text_length = len(text)
amount_to_pad = self.block_size - (text_length % self.block_size)
if amount_to_pad == 0:
amount_to_pad = self.block_size
pad = bytes([amount_to_pad])
return text + pad * amount_to_pad
def decode(self, decrypted: bytes) -> bytes:
pad = decrypted[-1]
if pad < 1 or pad > 32:
pad = 0
return decrypted[:-pad] if pad else decrypted
class _Prpcrypt:
"""AES-256-CBC encrypt/decrypt for WeCom callback messages."""
def __init__(self, key: bytes):
self.key = key
self.mode = AES.MODE_CBC
def encrypt(self, text: str, receive_id: str):
text_bytes = text.encode()
# 16-byte random prefix + network-order length + body + receive_id
text_bytes = (
self._get_random_str()
+ struct.pack("I", socket.htonl(len(text_bytes)))
+ text_bytes
+ receive_id.encode()
)
text_bytes = _PKCS7Encoder().encode(text_bytes)
try:
cryptor = AES.new(self.key, self.mode, self.key[:16])
ciphertext = cryptor.encrypt(text_bytes)
return WXBizMsgCrypt_OK, base64.b64encode(ciphertext)
except Exception as e:
logger.error(f"[WecomBot] AES encrypt error: {e}")
return WXBizMsgCrypt_EncryptAES_Error, None
def decrypt(self, text, receive_id: str):
try:
cryptor = AES.new(self.key, self.mode, self.key[:16])
plain_text = cryptor.decrypt(base64.b64decode(text))
except Exception as e:
logger.error(f"[WecomBot] AES decrypt error: {e}")
return WXBizMsgCrypt_DecryptAES_Error, None
try:
pad = plain_text[-1]
content = plain_text[16:-pad]
json_len = socket.ntohl(struct.unpack("I", content[:4])[0])
json_content = content[4 : json_len + 4].decode("utf-8")
from_receive_id = content[json_len + 4 :].decode("utf-8")
except Exception as e:
logger.error(f"[WecomBot] illegal buffer when decrypting: {e}")
return WXBizMsgCrypt_IllegalBuffer, None
if from_receive_id != receive_id:
logger.error(
f"[WecomBot] receive_id not match: expect={receive_id}, got={from_receive_id}"
)
return WXBizMsgCrypt_ValidateCorpid_Error, None
return WXBizMsgCrypt_OK, json_content
@staticmethod
def _get_random_str() -> bytes:
return str(random.randint(1000000000000000, 9999999999999999)).encode()
class WecomBotCrypt:
"""High-level helper for verifying URLs and (de)crypting callback messages."""
def __init__(self, token: str, encoding_aes_key: str, receive_id: str = ""):
try:
self.key = base64.b64decode(encoding_aes_key + "=")
assert len(self.key) == 32
except Exception:
raise FormatException("[WecomBot] invalid EncodingAESKey")
self.token = token
self.receive_id = receive_id
def verify_url(self, msg_signature, timestamp, nonce, echostr):
ret, signature = _gen_sha1(self.token, timestamp, nonce, echostr)
if ret != 0:
return ret, None
if signature != msg_signature:
return WXBizMsgCrypt_ValidateSignature_Error, None
pc = _Prpcrypt(self.key)
return pc.decrypt(echostr, self.receive_id)
def encrypt_msg(self, reply_msg: str, nonce: str, timestamp: str = None):
"""Encrypt a passive-reply JSON string and return the full response JSON.
Returns (ret, response_dict). On success ret==0 and response_dict is a
dict with encrypt/msgsignature/timestamp/nonce fields.
"""
pc = _Prpcrypt(self.key)
ret, encrypt = pc.encrypt(reply_msg, self.receive_id)
if ret != 0:
return ret, None
encrypt = encrypt.decode("utf-8")
if timestamp is None:
timestamp = str(int(time.time()))
ret, signature = _gen_sha1(self.token, timestamp, nonce, encrypt)
if ret != 0:
return ret, None
return WXBizMsgCrypt_OK, {
"encrypt": encrypt,
"msgsignature": signature,
"timestamp": timestamp,
"nonce": nonce,
}
def decrypt_msg(self, post_data, msg_signature, timestamp, nonce):
"""Verify signature and decrypt the encrypted callback payload.
``post_data`` may be the raw request body (bytes/str) containing
``{"encrypt": "..."}`` or the already-extracted encrypt string.
Returns (ret, plaintext_json_str).
"""
import json
encrypt = None
if isinstance(post_data, (bytes, bytearray)):
post_data = post_data.decode("utf-8")
if isinstance(post_data, str):
try:
encrypt = json.loads(post_data).get("encrypt")
except Exception:
encrypt = post_data
elif isinstance(post_data, dict):
encrypt = post_data.get("encrypt")
if not encrypt:
return WXBizMsgCrypt_ParseJson_Error, None
ret, signature = _gen_sha1(self.token, timestamp, nonce, encrypt)
if ret != 0:
return ret, None
if signature != msg_signature:
logger.error("[WecomBot] callback signature not match")
return WXBizMsgCrypt_ValidateSignature_Error, None
pc = _Prpcrypt(self.key)
return pc.decrypt(encrypt, self.receive_id)

View File

@@ -87,11 +87,14 @@ def _get_tmp_dir() -> str:
class WecomBotMessage(ChatMessage):
"""Message wrapper for wecom bot (websocket long-connection mode)."""
def __init__(self, msg_body: dict, is_group: bool = False):
def __init__(self, msg_body: dict, is_group: bool = False, default_aeskey: str = ""):
super().__init__(msg_body)
self.msg_id = msg_body.get("msgid")
self.create_time = msg_body.get("create_time")
self.is_group = is_group
# In callback (webhook) mode the media bodies carry no per-message aeskey;
# the download url is encrypted with the bot's EncodingAESKey instead.
self._default_aeskey = default_aeskey
msg_type = msg_body.get("msgtype")
from_userid = msg_body.get("from", {}).get("userid", "")
@@ -113,7 +116,7 @@ class WecomBotMessage(ChatMessage):
self.ctype = ContextType.IMAGE
image_info = msg_body.get("image", {})
image_url = image_info.get("url", "")
aeskey = image_info.get("aeskey", "")
aeskey = image_info.get("aeskey", "") or self._default_aeskey
tmp_dir = _get_tmp_dir()
image_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}.png")
@@ -147,7 +150,7 @@ class WecomBotMessage(ChatMessage):
elif item_type == "image":
img_info = item.get("image", {})
img_url = img_info.get("url", "")
img_aeskey = img_info.get("aeskey", "")
img_aeskey = img_info.get("aeskey", "") or self._default_aeskey
img_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}_{idx}.png")
try:
img_data = _decrypt_media(img_url, img_aeskey)
@@ -166,7 +169,7 @@ class WecomBotMessage(ChatMessage):
self.ctype = ContextType.FILE
file_info = msg_body.get("file", {})
file_url = file_info.get("url", "")
aeskey = file_info.get("aeskey", "")
aeskey = file_info.get("aeskey", "") or self._default_aeskey
tmp_dir = _get_tmp_dir()
base_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}")
self.content = base_path
@@ -188,7 +191,7 @@ class WecomBotMessage(ChatMessage):
self.ctype = ContextType.FILE
video_info = msg_body.get("video", {})
video_url = video_info.get("url", "")
aeskey = video_info.get("aeskey", "")
aeskey = video_info.get("aeskey", "") or self._default_aeskey
tmp_dir = _get_tmp_dir()
self.content = os.path.join(tmp_dir, f"wecom_{self.msg_id}.mp4")

View File

@@ -24,7 +24,7 @@ from channel.weixin.weixin_message import WeixinMessage
from common.expired_dict import ExpiredDict
from common.log import logger
from common.singleton import singleton
from config import conf
from config import conf, get_weixin_credentials_path
MAX_CONSECUTIVE_FAILURES = 3
BACKOFF_DELAY = 30
@@ -47,14 +47,16 @@ def _load_credentials(cred_path: str) -> dict:
def _save_credentials(cred_path: str, data: dict):
"""Save credentials to JSON file."""
"""Atomically save credentials to JSON file (tmp + rename)."""
os.makedirs(os.path.dirname(cred_path), exist_ok=True)
with open(cred_path, "w") as f:
tmp_path = f"{cred_path}.tmp"
with open(tmp_path, "w") as f:
json.dump(data, f, indent=2)
try:
os.chmod(cred_path, 0o600)
os.chmod(tmp_path, 0o600)
except Exception:
pass
os.replace(tmp_path, cred_path)
@singleton
@@ -73,7 +75,10 @@ class WeixinChannel(ChatChannel):
self.api = None
self._stop_event = threading.Event()
self._poll_thread = None
self._context_tokens = {} # user_id -> context_token
# user_id -> context_token. Guarded by _context_tokens_lock for any
# mutation that races with disk persistence.
self._context_tokens = {}
self._context_tokens_lock = threading.Lock()
self._received_msgs = ExpiredDict(60 * 60 * 7.1)
self._get_updates_buf = ""
self._credentials_path = ""
@@ -91,16 +96,21 @@ class WeixinChannel(ChatChannel):
cdn_base_url = conf().get("weixin_cdn_base_url", CDN_BASE_URL)
token = conf().get("weixin_token", "")
self._credentials_path = os.path.expanduser(
conf().get("weixin_credentials_path", "~/.weixin_cow_credentials.json")
)
self._credentials_path = get_weixin_credentials_path()
# Always load credentials so we can restore context_tokens even when
# the bot token itself comes from config.
creds = _load_credentials(self._credentials_path)
if not token:
creds = _load_credentials(self._credentials_path)
token = creds.get("token", "")
if creds.get("base_url"):
base_url = creds["base_url"]
# Restore persisted context_tokens so scheduler can deliver pushes
# immediately after restart, without waiting for the user to ping
# the bot first.
self._restore_context_tokens_from_creds(creds)
if not token:
token, base_url = self._login_with_retry(base_url)
if not token:
@@ -140,11 +150,16 @@ class WeixinChannel(ChatChannel):
def _relogin(self) -> bool:
"""Re-login after session expiry. Returns True on success."""
base_url = self.api.base_url if self.api else DEFAULT_BASE_URL
if os.path.exists(self._credentials_path):
try:
os.remove(self._credentials_path)
except Exception:
pass
# Clearing the whole credentials file is intentional: the new login
# will issue a fresh `token` and persisted context_tokens belong to
# the previous bot identity, so they must not survive.
with self._context_tokens_lock:
self._context_tokens.clear()
if os.path.exists(self._credentials_path):
try:
os.remove(self._credentials_path)
except Exception:
pass
self.login_status = self.LOGIN_STATUS_WAITING
result = self._qr_login(base_url)
if not result:
@@ -156,9 +171,62 @@ class WeixinChannel(ChatChannel):
cdn_base_url=self.api.cdn_base_url if self.api else CDN_BASE_URL,
)
self.login_status = self.LOGIN_STATUS_OK
self._context_tokens.clear()
return True
# ── Context token persistence ──────────────────────────────────────
# ilink requires every outbound send to echo the context_token from the
# user's latest inbound message. We mirror the in-memory map into the
# credentials JSON so scheduled pushes survive process restarts.
# All mutation + disk IO is serialized via _context_tokens_lock so that
# concurrent updates can never lose each other's writes.
def _restore_context_tokens_from_creds(self, creds: dict) -> None:
if not isinstance(creds, dict):
return
tokens = creds.get("context_tokens")
if not isinstance(tokens, dict):
return
restored = 0
with self._context_tokens_lock:
for user_id, token in tokens.items():
if isinstance(user_id, str) and isinstance(token, str) and token:
self._context_tokens[user_id] = token
restored += 1
if restored:
logger.info(f"[Weixin] Restored {restored} context_tokens from credentials")
def _persist_context_tokens_locked(self) -> None:
"""Flush the token map to disk. Caller must hold _context_tokens_lock."""
if not self._credentials_path:
return
try:
creds = _load_credentials(self._credentials_path) or {}
creds["context_tokens"] = dict(self._context_tokens)
_save_credentials(self._credentials_path, creds)
except Exception as e:
logger.warning(f"[Weixin] Failed to persist context_tokens: {e}")
def _update_context_token(self, user_id: str, token: str) -> None:
"""Update the in-memory token for a user; flush to disk only on change."""
if not user_id or not token:
return
with self._context_tokens_lock:
if self._context_tokens.get(user_id) == token:
return
self._context_tokens[user_id] = token
self._persist_context_tokens_locked()
def _invalidate_context_token(self, user_id: str) -> None:
"""Drop the cached token for a user (used after -14 / send rejection)."""
if not user_id:
return
with self._context_tokens_lock:
if user_id not in self._context_tokens:
return
del self._context_tokens[user_id]
logger.info(f"[Weixin] Invalidated stale context_token for {user_id}")
self._persist_context_tokens_locked()
# ── QR Login ───────────────────────────────────────────────────────
@staticmethod
@@ -391,7 +459,7 @@ class WeixinChannel(ChatChannel):
context_token = raw_msg.get("context_token", "")
if context_token and from_user:
self._context_tokens[from_user] = context_token
self._update_context_token(from_user, context_token)
cdn_base_url = self.api.cdn_base_url if self.api else CDN_BASE_URL
try:
@@ -510,10 +578,30 @@ class WeixinChannel(ChatChannel):
return msg.context_token
return self._context_tokens.get(receiver, "")
def _check_send_response(self, resp, receiver: str) -> None:
"""Inspect a send-API response; drop stale context_token on -14.
ilink uses ret/errcode = -14 to signal that the session (and any
cached context_token) is no longer valid. The plugin keeps running
because the bot itself can re-login; we just need to forget the
per-user token so the next push won't retry forever.
"""
if not isinstance(resp, dict):
return
ret = resp.get("ret")
errcode = resp.get("errcode")
if ret == -14 or errcode == -14:
logger.warning(
f"[Weixin] Send returned -14 (session expired) for "
f"receiver={receiver}; dropping cached context_token"
)
self._invalidate_context_token(receiver)
def _send_text(self, text: str, receiver: str, context_token: str):
if len(text) <= TEXT_CHUNK_LIMIT:
try:
self.api.send_text(receiver, text, context_token)
resp = self.api.send_text(receiver, text, context_token)
self._check_send_response(resp, receiver)
logger.debug(f"[Weixin] Text sent to {receiver}, len={len(text)}")
except Exception as e:
logger.error(f"[Weixin] Failed to send text: {e}")
@@ -522,7 +610,8 @@ class WeixinChannel(ChatChannel):
chunks = self._split_text(text, TEXT_CHUNK_LIMIT)
for i, chunk in enumerate(chunks):
try:
self.api.send_text(receiver, chunk, context_token)
resp = self.api.send_text(receiver, chunk, context_token)
self._check_send_response(resp, receiver)
logger.debug(f"[Weixin] Text chunk {i+1}/{len(chunks)} sent to {receiver}, len={len(chunk)}")
except Exception as e:
logger.error(f"[Weixin] Failed to send text chunk {i+1}/{len(chunks)}: {e}")
@@ -556,13 +645,14 @@ class WeixinChannel(ChatChannel):
return
try:
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=1)
self.api.send_image_item(
resp = self.api.send_image_item(
to=receiver,
context_token=context_token,
encrypt_query_param=result["encrypt_query_param"],
aes_key_b64=result["aes_key_b64"],
ciphertext_size=result["ciphertext_size"],
)
self._check_send_response(resp, receiver)
logger.info(f"[Weixin] Image sent to {receiver}")
except Exception as e:
logger.error(f"[Weixin] Image send failed: {e}")
@@ -575,7 +665,7 @@ class WeixinChannel(ChatChannel):
return
try:
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=3)
self.api.send_file_item(
resp = self.api.send_file_item(
to=receiver,
context_token=context_token,
encrypt_query_param=result["encrypt_query_param"],
@@ -583,6 +673,7 @@ class WeixinChannel(ChatChannel):
file_name=os.path.basename(local_path),
file_size=result["raw_size"],
)
self._check_send_response(resp, receiver)
logger.info(f"[Weixin] File sent to {receiver}")
except Exception as e:
logger.error(f"[Weixin] File send failed: {e}")
@@ -595,13 +686,14 @@ class WeixinChannel(ChatChannel):
return
try:
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=2)
self.api.send_video_item(
resp = self.api.send_video_item(
to=receiver,
context_token=context_token,
encrypt_query_param=result["encrypt_query_param"],
aes_key_b64=result["aes_key_b64"],
ciphertext_size=result["ciphertext_size"],
)
self._check_send_response(resp, receiver)
logger.info(f"[Weixin] Video sent to {receiver}")
except Exception as e:
logger.error(f"[Weixin] Video send failed: {e}")

View File

@@ -1 +1 @@
2.0.9
2.1.3

View File

@@ -3,7 +3,7 @@
import click
from cli import __version__
from cli.commands.skill import skill
from cli.commands.process import start, stop, restart, update, status, logs
from cli.commands.process import start, stop, restart, self_restart, update, status, logs
from cli.commands.context import context
from cli.commands.install import install_browser
from cli.commands.knowledge import knowledge
@@ -68,6 +68,7 @@ main.add_command(skill)
main.add_command(start)
main.add_command(stop)
main.add_command(restart)
main.add_command(self_restart)
main.add_command(update)
main.add_command(status)
main.add_command(logs)

View File

@@ -14,7 +14,7 @@ CHINA_MIRROR = "https://registry.npmmirror.com/-/binary/playwright"
# stream(msg, fg=None) — fg is "yellow" | "green" | "red" | None
StreamFn = Callable[[str, Optional[str]], None]
# on_phase(msg) — coarse-grained progress for chat channels (Chinese)
# on_phase(msg) — coarse-grained progress for chat channels (localized via i18n)
PhaseFn = Callable[[str], None]
@@ -78,15 +78,79 @@ def _is_china_network() -> bool:
def _pip_install(package_spec: str, stream: StreamFn) -> int:
"""Install a package, retrying with --user on permission failure."""
"""Install a package, preferring prebuilt wheels; retry with --user on perm error."""
python = sys.executable
ret = subprocess.call([python, "-m", "pip", "install", package_spec])
base = [python, "-m", "pip", "install", "--prefer-binary"]
ret = subprocess.call(base + [package_spec])
if ret != 0:
stream(" Retrying with --user flag...", "yellow")
ret = subprocess.call([python, "-m", "pip", "install", "--user", package_spec])
ret = subprocess.call(base + ["--user", package_spec])
return ret
def _is_frozen() -> bool:
"""True when running inside a PyInstaller-frozen bundle (desktop backend).
In this mode ``sys.executable`` is the frozen exe (no pip / no ``-m``), so
playwright is already bundled and we only need to download the browser
binary in-process rather than pip-installing anything.
"""
return bool(getattr(sys, "frozen", False))
def _playwright_cli(args: list, env: Optional[dict] = None) -> int:
"""Invoke the Playwright CLI, working in both source and frozen builds.
Source builds shell out to ``python -m playwright <args>``. Frozen builds
can't use ``-m`` (the exe isn't a Python interpreter), so we call
Playwright's driver entrypoint in-process instead. ``env`` overrides are
applied to os.environ for the duration of the call (frozen path) or passed
through to the subprocess (source path).
"""
if not _is_frozen():
cmd = [sys.executable, "-m", "playwright"] + args
return subprocess.call(cmd, env=env)
# Frozen: run the bundled Playwright driver in-process. compute_driver_executable
# returns the Node driver shipped inside the bundle; we spawn it directly.
prev_env = {}
if env:
for k, v in env.items():
prev_env[k] = os.environ.get(k)
os.environ[k] = v
try:
from playwright._impl._driver import compute_driver_executable, get_driver_env
driver = compute_driver_executable()
# compute_driver_executable may return a tuple (node, cli.js) on newer
# Playwright, or a single path on older ones.
if isinstance(driver, (list, tuple)):
cmd = list(driver) + args
else:
cmd = [str(driver)] + args
# get_driver_env() snapshots os.environ, which we've already patched with
# the caller's overrides (PLAYWRIGHT_BROWSERS_PATH / DOWNLOAD_HOST) above,
# so mirror + pinned browsers dir are honored here too.
return subprocess.call(cmd, env=get_driver_env())
except Exception as e:
# Last resort: try the module main via runpy (works if the frozen build
# kept playwright.__main__ importable).
try:
import runpy
sys.argv = ["playwright"] + args
runpy.run_module("playwright", run_name="__main__")
return 0
except SystemExit as se:
return int(se.code or 0)
except Exception:
return 1
finally:
for k, v in prev_env.items():
if v is None:
os.environ.pop(k, None)
else:
os.environ[k] = v
def _default_stream(msg: str, fg: Optional[str] = None) -> None:
"""CLI: colored click output."""
if fg == "yellow":
@@ -112,16 +176,28 @@ def run_install_browser(
stream: Optional callback ``(message, fg)`` for each line. ``fg`` is
``yellow`` / ``green`` / ``red`` or None. Defaults to colored click output.
on_phase: Optional callback for coarse progress (e.g. push to chat);
messages are short Chinese status lines.
messages are short status lines localized via i18n.
Returns:
0 on success, 1 on fatal failure (pip or chromium install failed).
"""
from cli.utils import get_cli_language
# Import `common` only after get_cli_language() runs ensure_sys_path(),
# so it works when `cow` is invoked from outside the project directory.
get_cli_language() # resolve cow_lang so i18n.t reflects config
from common import i18n
_t = i18n.t
stream = stream or _default_stream
python = sys.executable
legacy_mode = False
frozen = _is_frozen()
_phase(on_phase, "🔧 开始安装浏览器工具依赖(约几分钟,请耐心等待)…")
_phase(on_phase, _t(
"🔧 开始安装浏览器工具依赖(约几分钟,请耐心等待)…",
"🔧 Installing browser tool dependencies (a few minutes, please wait)…",
))
glibc = _get_glibc_version()
if glibc and glibc < GLIBC_THRESHOLD:
@@ -136,29 +212,84 @@ def run_install_browser(
stream("")
_phase(
on_phase,
f" 检测到 glibc {glibc_str}(较旧),将安装兼容版 Playwright {PLAYWRIGHT_LEGACY_VERSION}",
_t(
f" 检测到 glibc {glibc_str}(较旧),将安装兼容版 Playwright {PLAYWRIGHT_LEGACY_VERSION}",
f" Detected glibc {glibc_str} (older); installing compatible Playwright {PLAYWRIGHT_LEGACY_VERSION}.",
),
)
target_version = PLAYWRIGHT_LEGACY_VERSION if legacy_mode else PLAYWRIGHT_VERSION
_phase(on_phase, "📦 [1/3] 正在安装 Playwright Python 包…")
stream("[1/3] Installing playwright Python package...", "yellow")
ret = _pip_install(f"playwright=={target_version}", stream)
if ret != 0:
stream("Failed to install playwright package.", "red")
_phase(on_phase, "❌ [1/3] Playwright Python 包安装失败。")
return 1
# Windows-only: greenlet 3.2.x ships no Windows wheel, so pip would build it
# from source (needs MSVC) and fail. Pre-install 3.1.x (has win wheels for
# py3.7-3.13) which still satisfies playwright's greenlet>=3.1.1,<4.
if sys.platform == "win32" and not frozen:
stream("[1/3] Pre-installing greenlet (prebuilt wheel) for Windows...", "yellow")
ret = subprocess.call(
[python, "-m", "pip", "install", "--only-binary=:all:", "greenlet>=3.1.1,<3.2"]
)
if ret != 0:
stream(
" Could not pre-install a prebuilt greenlet wheel.\n"
" playwright may try to build greenlet from source, which needs\n"
" Microsoft C++ Build Tools: https://visualstudio.microsoft.com/visual-cpp-build-tools/",
"yellow",
)
installed = _get_installed_version()
if installed:
stream(f" playwright {installed} installed.", "green")
stream("")
_phase(on_phase, f"[1/3] Playwright 包已安装({installed or target_version})。")
if frozen:
# Desktop bundle: playwright is already shipped inside the app; there is
# no pip and nothing to install. Skip straight to downloading Chromium.
installed = _get_installed_version()
stream(f"[1/3] Playwright is bundled ({installed or 'ok'}), skipping pip install.", "green")
_phase(on_phase, _t(
"✅ [1/3] Playwright 已内置于客户端,跳过安装。",
"✅ [1/3] Playwright is bundled in the app; skipping install.",
))
else:
_phase(on_phase, _t("📦 [1/3] 正在安装 Playwright Python 包…", "📦 [1/3] Installing Playwright Python package…"))
stream("[1/3] Installing playwright Python package...", "yellow")
ret = _pip_install(f"playwright=={target_version}", stream)
if ret != 0:
stream("Failed to install playwright package.", "red")
_phase(on_phase, _t("❌ [1/3] Playwright Python 包安装失败。", "❌ [1/3] Failed to install Playwright Python package."))
return 1
installed = _get_installed_version()
if installed:
stream(f" playwright {installed} installed.", "green")
stream("")
_phase(on_phase, _t(
f"✅ [1/3] Playwright 包已安装({installed or target_version})。",
f"✅ [1/3] Playwright package installed ({installed or target_version}).",
))
# With playwright available, prefer the user's system Chrome/Edge: the browser
# tool drives it directly (channel="chrome"/"msedge"), so we can skip the heavy
# ~150MB Chromium download entirely. Applies to every runtime (desktop, web,
# source) — only headless Linux servers, which usually lack a system browser,
# fall through to the download below. Honors prefer_system_browser via
# resolve_engine, so users who force downloaded Chromium still get it.
try:
from agent.tools.browser import browser_env
summary = browser_env.capability_summary()
if summary.get("ready") and summary.get("engine", {}).get("mode") == "system-chrome":
sc = summary.get("system_chrome") or {}
stream(f"System browser detected ({sc.get('channel')}), skipping Chromium download.", "green")
_phase(on_phase, _t(
f"✅ 检测到系统浏览器({sc.get('channel')}),无需下载 Chromium浏览器工具已就绪。",
f"✅ Detected system browser ({sc.get('channel')}); no Chromium download needed, browser tool is ready.",
))
return 0
except Exception as e:
stream(f" (system browser probe skipped: {e})", None)
if sys.platform == "linux":
_phase(on_phase, "🔧 [2/3] 正在安装 Linux 系统依赖与轻量中文字体(文泉驿正黑,部分步骤可能需要 sudo")
_phase(on_phase, _t(
"🔧 [2/3] 正在安装 Linux 系统依赖与轻量中文字体(文泉驿正黑,部分步骤可能需要 sudo",
"🔧 [2/3] Installing Linux system deps and a lightweight CJK font (WenQuanYi Zen Hei; some steps may need sudo)…",
))
stream("[2/3] Installing system dependencies (Linux)...", "yellow")
ret = subprocess.call([python, "-m", "playwright", "install-deps", "chromium"])
ret = _playwright_cli(["install-deps", "chromium"])
if ret != 0:
stream(
" Could not auto-install system deps (may need sudo).\n"
@@ -183,21 +314,30 @@ def run_install_browser(
stream(" CJK font (wqy-zenhei) installed.", "green")
_phase(
on_phase,
"✅ [2/3] Linux 依赖与字体步骤已执行(若有权限问题请查看服务器日志或手动执行提示命令)。",
_t(
"✅ [2/3] Linux 依赖与字体步骤已执行(若有权限问题请查看服务器日志或手动执行提示命令)。",
"✅ [2/3] Linux deps and font steps executed (on permission issues, check the server log or run the suggested commands manually).",
),
)
else:
stream(f"[2/3] Skipping system deps (not needed on {sys.platform}).", "yellow")
_phase(on_phase, f" [2/3] 当前系统({sys.platform})跳过 Linux 专用依赖。")
_phase(on_phase, _t(
f" [2/3] 当前系统({sys.platform})跳过 Linux 专用依赖。",
f" [2/3] Skipping Linux-specific deps on this platform ({sys.platform}).",
))
stream("")
_phase(on_phase, "🌐 [3/3] 正在下载并安装 Chromium体积较大请耐心等待")
_phase(on_phase, _t(
"🌐 [3/3] 正在下载并安装 Chromium体积较大请耐心等待",
"🌐 [3/3] Downloading and installing Chromium (large download, please wait)…",
))
stream("[3/3] Installing Chromium browser...", "yellow")
cmd = [python, "-m", "playwright", "install", "chromium"]
pw_args = ["install", "chromium"]
if _is_headless_linux() and not legacy_mode:
ver = _version_tuple(installed or "")
if ver >= (1, 57, 0):
cmd.append("--only-shell")
pw_args.append("--only-shell")
stream(" (headless shell for Linux server)", None)
else:
stream(" (full Chromium)", None)
@@ -205,49 +345,78 @@ def run_install_browser(
stream(" (full browser for Linux desktop)", None)
env = os.environ.copy()
# Pin the download location so it survives desktop app updates and matches
# what the runtime looks up (see browser_env.browsers_download_dir()).
try:
from agent.tools.browser.browser_env import browsers_download_dir
env["PLAYWRIGHT_BROWSERS_PATH"] = browsers_download_dir()
stream(f" (browsers dir: {env['PLAYWRIGHT_BROWSERS_PATH']})", None)
except Exception:
pass
use_mirror = _is_china_network()
if use_mirror:
env["PLAYWRIGHT_DOWNLOAD_HOST"] = CHINA_MIRROR
stream(f" (using China mirror: {CHINA_MIRROR})", None)
_phase(on_phase, "📡 检测到国内 pip 源配置Chromium 将优先走国内镜像下载。")
_phase(on_phase, _t(
"📡 检测到国内 pip 源配置Chromium 将优先走国内镜像下载。",
"📡 Detected a China pip mirror; Chromium will be downloaded from the China mirror first.",
))
ret = subprocess.call(cmd, env=env)
ret = _playwright_cli(pw_args, env=env)
if ret != 0 and use_mirror:
stream(" Mirror download failed, retrying with official CDN...", "yellow")
_phase(on_phase, "⚠️ 镜像下载失败,正在改用官方源重试…")
env_no_mirror = os.environ.copy()
_phase(on_phase, _t(
"⚠️ 镜像下载失败,正在改用官方源重试…",
"⚠️ Mirror download failed; retrying with the official CDN…",
))
env_no_mirror = dict(env)
env_no_mirror.pop("PLAYWRIGHT_DOWNLOAD_HOST", None)
ret = subprocess.call(cmd, env=env_no_mirror)
ret = _playwright_cli(pw_args, env=env_no_mirror)
if ret != 0:
stream("Failed to install Chromium.", "red")
_phase(on_phase, "❌ [3/3] Chromium 安装失败。")
_phase(on_phase, _t("❌ [3/3] Chromium 安装失败。", "❌ [3/3] Failed to install Chromium."))
return 1
stream("")
_phase(on_phase, "✅ [3/3] Chromium 已安装。")
_phase(on_phase, _t("✅ [3/3] Chromium 已安装。", "✅ [3/3] Chromium installed."))
stream("Verifying browser installation...", None)
_phase(on_phase, "🔍 正在验证 Playwright 能否正常加载…")
ret = subprocess.call(
[python, "-c", "from playwright.sync_api import sync_playwright; print('OK')"],
stderr=subprocess.DEVNULL,
)
_phase(on_phase, _t("🔍 正在验证 Playwright 能否正常加载…", "🔍 Verifying that Playwright loads correctly…"))
if frozen:
# Frozen: no child interpreter to spawn; import in-process instead.
try:
from playwright.sync_api import sync_playwright # noqa: F401
ret = 0
except Exception:
ret = 1
else:
ret = subprocess.call(
[python, "-c", "from playwright.sync_api import sync_playwright; print('OK')"],
stderr=subprocess.DEVNULL,
)
if ret != 0:
stream(
" Warning: playwright import failed. Browser tool may not work on this system.\n"
" Consider upgrading your OS or using Docker.",
"yellow",
)
_phase(on_phase, "⚠️ 验证未完全通过:本机可能仍无法使用浏览器工具,请查看日志或升级系统。")
_phase(on_phase, _t(
"⚠️ 验证未完全通过:本机可能仍无法使用浏览器工具,请查看日志或升级系统。",
"⚠️ Verification did not fully pass: the browser tool may still not work here; check the log or upgrade your system.",
))
else:
stream(" Verification passed.", "green")
_phase(on_phase, "✅ 验证通过。")
_phase(on_phase, _t("✅ 验证通过。", "✅ Verification passed."))
stream("")
stream("Browser tool ready! Restart CowAgent to enable it.", "green")
_phase(on_phase, "🎉 全部步骤结束。请重启 CowAgent 后使用 browser 工具。")
_phase(on_phase, _t(
"🎉 全部步骤结束。请重启 CowAgent 后使用 browser 工具。",
"🎉 All steps finished. Restart CowAgent to use the browser tool.",
))
return 0

View File

@@ -8,11 +8,28 @@ from typing import Optional
import click
from cli.utils import get_project_root
from cli.utils import get_project_root, load_config_json
_IS_WIN = sys.platform == "win32"
def _is_terminal_only() -> bool:
"""Whether terminal is the only configured channel.
Terminal needs an interactive stdin/tty, which is incompatible with the
background daemon mode (stdout/stdin detached). When terminal is the only
channel, `start` must run in the foreground so it can own the tty.
"""
channel = load_config_json().get("channel_type", "")
if isinstance(channel, str):
names = [c.strip() for c in channel.split(",") if c.strip()]
elif isinstance(channel, (list, tuple)):
names = [str(c).strip() for c in channel if str(c).strip()]
else:
names = []
return names == ["terminal"]
def _get_pid_file():
return os.path.join(get_project_root(), ".cow.pid")
@@ -103,6 +120,12 @@ def start(foreground, no_logs):
python = sys.executable
# Terminal-only setups need an interactive tty; force foreground so the
# terminal channel can read stdin instead of fighting the shell over the tty.
if not foreground and _is_terminal_only():
foreground = True
click.echo("Detected terminal-only channel, starting in foreground...")
if foreground:
click.echo("Starting CowAgent in foreground...")
if _IS_WIN:
@@ -172,6 +195,120 @@ def restart(ctx, no_logs):
ctx.invoke(start, no_logs=no_logs)
# Detached relay that survives the caller's process tree. Run via `python -c`
# with start_new_session=True so it keeps going after the agent's bash child
# (and the main app it kills) both die. Flow: self-check the new code FIRST
# (import app); abort without touching the old process if it fails. Only when
# the new code is loadable does it SIGTERM the old PID, wait for exit (SIGKILL
# fallback), then start a fresh app.py and write the pid.
_RELAY_SCRIPT = r"""
import os, sys, time, signal, subprocess
root, python, app_py, pid_file, log_file = sys.argv[1:6]
old_pid = int(sys.argv[6]) if len(sys.argv) > 6 and sys.argv[6] else 0
def alive(pid):
if not pid:
return False
try:
os.kill(pid, 0)
return True
except OSError:
return False
def log(msg):
try:
with open(log_file, "a") as f:
f.write("[self-restart] " + msg + "\n")
except OSError:
pass
# 0) Self-check: make sure the new code actually loads BEFORE killing anything.
# `import app` exercises top-level imports / syntax of the entry module. If it
# fails, abort and leave the running service untouched — never end up with the
# old process killed and the new one unable to start.
check = subprocess.run(
[python, "-c", "import app"], cwd=root,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
)
if check.returncode != 0:
detail = (check.stdout or b"").decode("utf-8", "replace").strip()
log("self-check FAILED, aborting restart; service left running:\n" + detail)
sys.exit(1)
log("self-check passed")
# 1) Ask the old process to exit gracefully (its SIGTERM handler persists state).
if alive(old_pid):
try:
os.kill(old_pid, signal.SIGTERM)
except OSError:
pass
# 2) Wait up to ~15s for it to go, then force-kill as a backstop.
for _ in range(150):
if not alive(old_pid):
break
time.sleep(0.1)
else:
try:
os.kill(old_pid, signal.SIGKILL)
except OSError:
pass
time.sleep(0.5)
# 3) Start a fresh instance, detached, logging to the same file.
with open(log_file, "a") as f:
proc = subprocess.Popen(
[python, app_py], cwd=root,
stdout=f, stderr=f, start_new_session=True,
)
with open(pid_file, "w") as f:
f.write(str(proc.pid))
log("restarted, new pid=" + str(proc.pid))
"""
@click.command(name="self-restart", hidden=True)
def self_restart():
"""Restart from inside the running agent (detached; survives parent death).
Intended to be invoked by the agent itself (e.g. via bash after editing its
own code), not by users — so it is hidden from `cow help`. Unlike `restart`,
the actual stop+start runs in a detached relay process that outlives the
agent's bash child, which would otherwise die together with the main app it
kills.
"""
if _IS_WIN:
click.echo("self-restart is not supported on Windows; use `cow restart`.", err=True)
sys.exit(1)
root = get_project_root()
app_py = os.path.join(root, "app.py")
if not os.path.exists(app_py):
click.echo("Error: app.py not found in project root.", err=True)
sys.exit(1)
python = sys.executable
pid = _read_pid() or 0
subprocess.Popen(
[
python, "-c", _RELAY_SCRIPT,
root, python, app_py, _get_pid_file(), _get_log_file(), str(pid),
],
cwd=root,
start_new_session=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
click.echo(click.style(
"✓ Restart scheduled. The service will stop and come back in a few seconds.",
fg="green",
))
@click.command()
@click.pass_context
def update(ctx):
@@ -252,7 +389,14 @@ def update(ctx):
def status():
"""Show CowAgent running status."""
from cli import __version__
from cli.utils import load_config_json
from cli.utils import load_config_json, get_cli_language, get_project_root
# get_cli_language() calls ensure_sys_path(), which adds the project root
# to sys.path. Import `common` only AFTER that, otherwise it fails with
# ModuleNotFoundError when `cow` runs from outside the project dir.
get_cli_language() # resolve cow_lang so i18n.t reflects config
from common import i18n
_t = i18n.t
pid = _read_pid()
if pid:
@@ -260,17 +404,24 @@ def status():
else:
click.echo(click.style("● CowAgent is not running", fg="red"))
click.echo(f" 版本: v{__version__}")
click.echo(_t(f" 版本: v{__version__}", f" Version: v{__version__}"))
# Project path bound to this `cow` CLI — disambiguates which checkout the
# command actually controls when the user has multiple clones.
project_root = get_project_root()
click.echo(_t(f" 路径: {project_root}", f" Path: {project_root}"))
cfg = load_config_json()
if cfg:
channel = cfg.get("channel_type", "unknown")
if isinstance(channel, list):
channel = ", ".join(channel)
click.echo(f" 通道: {channel}")
click.echo(f" 模型: {cfg.get('model', 'unknown')}")
click.echo(_t(f" 通道: {channel}", f" Channel: {channel}"))
click.echo(_t(f" 模型: {cfg.get('model', 'unknown')}", f" Model: {cfg.get('model', 'unknown')}"))
mode = "Chat" if cfg.get("agent") is False else "Agent"
click.echo(f" 模式: {mode}")
click.echo(_t(f" 模式: {mode}", f" Mode: {mode}"))
lang_label = "中文" if i18n.get_language() == "zh" else "English"
click.echo(_t(f" 语言: {lang_label}", f" Language: {lang_label}"))
@click.command()

View File

@@ -517,18 +517,26 @@ def _install_targz_bytes(content: bytes, name: str, skills_dir: str, result: Ins
def _print_install_success(name: str, source: str):
"""Print a unified install success message with description and source."""
from cli.utils import get_cli_language
# Import `common` only after get_cli_language() runs ensure_sys_path(),
# so it works when `cow` is invoked from outside the project directory.
get_cli_language() # resolve cow_lang so i18n.t reflects config
from common import i18n
_t = i18n.t
skills_dir = get_skills_dir()
config = load_skills_config()
display = config.get(name, {}).get("display_name", "")
desc = _read_skill_description(os.path.join(skills_dir, name))
click.echo(click.style(f"{name}", fg="green"))
if display and display != name:
click.echo(f" 名称: {display}")
click.echo(_t(f" 名称: {display}", f" Name: {display}"))
if desc:
if len(desc) > 60:
desc = desc[:57] + ""
click.echo(f" 描述: {desc}")
click.echo(f" 来源: {source}")
click.echo(_t(f" 描述: {desc}", f" Description: {desc}"))
click.echo(_t(f" 来源: {source}", f" Source: {source}"))
def _validate_skill_name(name: str):

Some files were not shown because too many files have changed in this diff Show More