mirror of
https://github.com/zhayujie/chatgpt-on-wechat.git
synced 2026-07-17 11:07:11 +08:00
Compare commits
206 Commits
feat-readm
...
feat-cow-d
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6211e63f90 | ||
|
|
44b61684ed | ||
|
|
ab6f49a822 | ||
|
|
02517e4a01 | ||
|
|
2599966cf7 | ||
|
|
6c68931892 | ||
|
|
41855ed511 | ||
|
|
02bc91f4af | ||
|
|
e9352e6984 | ||
|
|
ec4c36f450 | ||
|
|
215ed24401 | ||
|
|
c432681b2b | ||
|
|
49452e035d | ||
|
|
d1336b872e | ||
|
|
e1e29b32e9 | ||
|
|
214dcaf141 | ||
|
|
77a196de8b | ||
|
|
108d04398b | ||
|
|
c9c16298ec | ||
|
|
2ef31d5d33 | ||
|
|
e9d9b566a4 | ||
|
|
3baa3252bc | ||
|
|
90d9db0f83 | ||
|
|
8bff4f1658 | ||
|
|
a0e20ef311 | ||
|
|
6996215d3b | ||
|
|
03ffa2db7d | ||
|
|
75e3110e8c | ||
|
|
033480eef1 | ||
|
|
8ddfcbb125 | ||
|
|
a5aaecc48d | ||
|
|
01373465b0 | ||
|
|
a1e733080d | ||
|
|
3b3ef715bb | ||
|
|
47b2bf9d46 | ||
|
|
ea47f3097e | ||
|
|
70c1c44d15 | ||
|
|
e3dce45b2a | ||
|
|
3bb8ec3bea | ||
|
|
35e42a3ad6 | ||
|
|
949575ad14 | ||
|
|
1c34f0f03d | ||
|
|
eed2eab014 | ||
|
|
381cbed4fd | ||
|
|
4cde25325d | ||
|
|
8d70af5e89 | ||
|
|
b3408d8e5f | ||
|
|
e74906fbec | ||
|
|
52209217fc | ||
|
|
018493a60b | ||
|
|
630373b1f0 | ||
|
|
dec31dfd75 | ||
|
|
2397ea019e | ||
|
|
77da90e316 | ||
|
|
18ce17d21a | ||
|
|
ce09b14836 | ||
|
|
e7a069b060 | ||
|
|
6e3933be30 | ||
|
|
e2cb9e11b0 | ||
|
|
d281a34c6f | ||
|
|
c97cf5610f | ||
|
|
ab674a3517 | ||
|
|
7d63e7d8fa | ||
|
|
6538843bdf | ||
|
|
bd5fede122 | ||
|
|
047fb57630 | ||
|
|
583c1de5ba | ||
|
|
c9c293f67c | ||
|
|
561631baba | ||
|
|
80d0a6aeb2 | ||
|
|
6b5ee245ae | ||
|
|
5c43c2f519 | ||
|
|
9387980e74 | ||
|
|
075d9fc608 | ||
|
|
63bfab03f6 | ||
|
|
1d7e6b3703 | ||
|
|
ad64e17a34 | ||
|
|
0092376c07 | ||
|
|
6fb19a68b5 | ||
|
|
d5427d967a | ||
|
|
7fd30b608c | ||
|
|
830b05f243 | ||
|
|
e85290cddc | ||
|
|
1940d628a8 | ||
|
|
cffa590d3e | ||
|
|
402e2bfee0 | ||
|
|
f5caba81d6 | ||
|
|
354350dec9 | ||
|
|
0513298f57 | ||
|
|
08e23e5bd8 | ||
|
|
e812c7d29a | ||
|
|
ef46199346 | ||
|
|
7c9ea62993 | ||
|
|
8cb53e6129 | ||
|
|
12c0383dc8 | ||
|
|
83b53039f3 | ||
|
|
7e6a309935 | ||
|
|
33c03e30d9 | ||
|
|
1f1abdd7b6 | ||
|
|
16134bd150 | ||
|
|
c887fc71ad | ||
|
|
9fc39f648f | ||
|
|
ec9557e3d8 | ||
|
|
7cf0f7d42d | ||
|
|
b7aa64279d | ||
|
|
26300a8d43 | ||
|
|
8dd21ddb83 | ||
|
|
ff584f8421 | ||
|
|
ca4a8253a1 | ||
|
|
157374401a | ||
|
|
ba777ed706 | ||
|
|
0e4da1d1c5 | ||
|
|
72847e0711 | ||
|
|
3c19614c74 | ||
|
|
a2e4955116 | ||
|
|
c62175c06b | ||
|
|
fde4b6f590 | ||
|
|
3d7c68bac6 | ||
|
|
72a477f10c | ||
|
|
2a16c562a8 | ||
|
|
2b670e73f3 | ||
|
|
3994594019 | ||
|
|
39c9386b54 | ||
|
|
4cc57cc08d | ||
|
|
639a3eac1e | ||
|
|
79323358e5 | ||
|
|
cdb093c74a | ||
|
|
f6f3ce5f05 | ||
|
|
4805f3d4d3 | ||
|
|
1d797cdaf5 | ||
|
|
4d8458669c | ||
|
|
92ec9653e5 | ||
|
|
e861d98007 | ||
|
|
a97eeb1fd9 | ||
|
|
cd88b23b5d | ||
|
|
33eabf937b | ||
|
|
beb5df16a3 | ||
|
|
7fa743f01a | ||
|
|
1f6859d78f | ||
|
|
2853735472 | ||
|
|
feaa9076b0 | ||
|
|
ce0249706e | ||
|
|
af2c839231 | ||
|
|
2b2d24ed25 | ||
|
|
04d28f9d2d | ||
|
|
1dbf41f384 | ||
|
|
9e6a2cc2c0 | ||
|
|
7bf4ef3d05 | ||
|
|
126649f70f | ||
|
|
1827a2a31c | ||
|
|
fcf4eb78dc | ||
|
|
2ec6ea8045 | ||
|
|
0994a3586d | ||
|
|
29c4be6a3a | ||
|
|
c5b8e06891 | ||
|
|
54a20bca92 | ||
|
|
6e786bde90 | ||
|
|
b671b0d725 | ||
|
|
57f5692074 | ||
|
|
b0ac0731c7 | ||
|
|
3c161df526 | ||
|
|
aa3f48e93c | ||
|
|
5ae1e1adde | ||
|
|
fe8b8fe831 | ||
|
|
5aca54c083 | ||
|
|
458b1a1d88 | ||
|
|
3dd4b84179 | ||
|
|
99bddb79d6 | ||
|
|
136b0b89e8 | ||
|
|
c605b0b080 | ||
|
|
b7b8e3679c | ||
|
|
aeb6610ff4 | ||
|
|
e3eacc77d7 | ||
|
|
37661daf40 | ||
|
|
877b848370 | ||
|
|
5c163cc0fe | ||
|
|
6e04ea8240 | ||
|
|
d106465419 | ||
|
|
f39380cea7 | ||
|
|
bccce2d7cb | ||
|
|
6721dbdbcc | ||
|
|
83cd6ad158 | ||
|
|
116fb27257 | ||
|
|
8d67177a1b | ||
|
|
ad2db1a776 | ||
|
|
2e6d9e0f27 | ||
|
|
e05f85f3ce | ||
|
|
40c48a9a61 | ||
|
|
c9a7525d0b | ||
|
|
fd571ac539 | ||
|
|
c5a3f991c5 | ||
|
|
eb74b73351 | ||
|
|
9b31f45481 | ||
|
|
bc9c1691f5 | ||
|
|
73bf83d2ff | ||
|
|
36e1988fee | ||
|
|
aad6ef635e | ||
|
|
96659cd616 | ||
|
|
c8787b7de4 | ||
|
|
91d427c8f9 | ||
|
|
c8c0573dbd | ||
|
|
2fa6343fe5 | ||
|
|
06b84225a1 | ||
|
|
5b31da335d | ||
|
|
11d92bb22a | ||
|
|
3bc6e89b74 |
145
.github/ISSUE_TEMPLATE/1.bug.yml
vendored
145
.github/ISSUE_TEMPLATE/1.bug.yml
vendored
@@ -1,131 +1,46 @@
|
||||
name: Bug report 🐛
|
||||
description: 项目运行中遇到的Bug或问题。
|
||||
description: Report a bug or unexpected behavior.
|
||||
title: "[Bug] "
|
||||
labels: ['status: needs check']
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
### ⚠️ 前置确认
|
||||
1. 网络能够访问openai接口
|
||||
2. python 已安装:版本在 3.7 ~ 3.10 之间
|
||||
3. `git pull` 拉取最新代码
|
||||
4. 执行`pip3 install -r requirements.txt`,检查依赖是否满足
|
||||
5. 拓展功能请执行`pip3 install -r requirements-optional.txt`,检查依赖是否满足
|
||||
6. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
|
||||
> 💡 English is recommended so global developers can help. 推荐使用英文提交,谢谢 ❤️
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: 前置确认
|
||||
label: Self check
|
||||
options:
|
||||
- label: 我确认我运行的是最新版本的代码,并且安装了所需的依赖,在[FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs)中也未找到类似问题。
|
||||
- label: I'm on the latest version and searched [existing issues](https://github.com/zhayujie/CowAgent/issues) (incl. closed) — no duplicate.
|
||||
required: true
|
||||
- type: checkboxes
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: ⚠️ 搜索issues中是否已存在类似问题
|
||||
description: >
|
||||
请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框,搜索你的问题
|
||||
或相关日志的关键词来查找是否存在类似问题。
|
||||
options:
|
||||
- label: 我已经搜索过issues和disscussions,没有跟我遇到的问题相关的issue
|
||||
required: true
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
请在上方的`title`中填写你对你所遇到问题的简略总结,这将帮助其他人更好的找到相似问题,谢谢❤️。
|
||||
- type: dropdown
|
||||
attributes:
|
||||
label: 操作系统类型?
|
||||
description: >
|
||||
请选择你运行程序的操作系统类型。
|
||||
options:
|
||||
- Windows
|
||||
- Linux
|
||||
- MacOS
|
||||
- Docker
|
||||
- Railway
|
||||
- Windows Subsystem for Linux (WSL)
|
||||
- Other (请在问题中说明)
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
attributes:
|
||||
label: 运行的python版本是?
|
||||
description: |
|
||||
请选择你运行程序的`python`版本。
|
||||
注意:在`python 3.7`中,有部分可选依赖无法安装。
|
||||
经过长时间的观察,我们认为`python 3.8`是兼容性最好的版本。
|
||||
`python 3.7`~`python 3.10`以外版本的issue,将视情况直接关闭。
|
||||
options:
|
||||
- python 3.7
|
||||
- python 3.8
|
||||
- python 3.9
|
||||
- python 3.10
|
||||
- other
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
attributes:
|
||||
label: 使用的chatgpt-on-wechat版本是?
|
||||
description: |
|
||||
请确保你使用的是 [releases](https://github.com/zhayujie/chatgpt-on-wechat/releases) 中的最新版本。
|
||||
如果你使用git, 请使用`git branch`命令来查看分支。
|
||||
options:
|
||||
- Latest Release
|
||||
- Master (branch)
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
attributes:
|
||||
label: 运行的`channel`类型是?
|
||||
description: |
|
||||
请确保你正确配置了该`channel`所需的配置项,所有可选的配置项都写在了[该文件中](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py),请将所需配置项填写在根目录下的`config.json`文件中。
|
||||
options:
|
||||
- wechatmp(公众号, 订阅号)
|
||||
- wechatmp_service(公众号, 服务号)
|
||||
- terminal
|
||||
- other
|
||||
label: Environment
|
||||
description: "Version (`cow status`), OS, Python version, install method, model & channel."
|
||||
placeholder: |
|
||||
Version: v1.2.0
|
||||
OS: macOS / Linux / Windows / Docker
|
||||
Python: 3.11
|
||||
Install: installer / Docker / source
|
||||
Model & channel: deepseek-v4-flash, web
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: 复现步骤 🕹
|
||||
description: |
|
||||
**⚠️ 不能复现将会关闭issue.**
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: 问题描述 😯
|
||||
description: 详细描述出现的问题,或提供有关截图。
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: 终端日志 📒
|
||||
description: |
|
||||
在此处粘贴终端日志,可在主目录下`run.log`文件中找到,这会帮助我们更好的分析问题,注意隐去你的API key。
|
||||
如果在配置文件中加入`"debug": true`,打印出的日志会更有帮助。
|
||||
label: What happened?
|
||||
description: "Steps to reproduce, what you expected, and what happened instead. Screenshots welcome."
|
||||
placeholder: |
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
<details>
|
||||
<summary><i>示例</i></summary>
|
||||
```log
|
||||
[DEBUG][2023-04-16 00:23:22][plugin_manager.py:157] - Plugin SUMMARY triggered by event Event.ON_HANDLE_CONTEXT
|
||||
[DEBUG][2023-04-16 00:23:22][main.py:221] - [Summary] on_handle_context. content: $总结前100条消息
|
||||
[DEBUG][2023-04-16 00:23:24][main.py:240] - [Summary] limit: 100, duration: -1 seconds
|
||||
[ERROR][2023-04-16 00:23:24][chat_channel.py:244] - Worker return exception: name 'start_date' is not defined
|
||||
Traceback (most recent call last):
|
||||
File "C:\ProgramData\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
|
||||
result = self.fn(*self.args, **self.kwargs)
|
||||
File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 132, in _handle
|
||||
reply = self._generate_reply(context)
|
||||
File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 142, in _generate_reply
|
||||
e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {
|
||||
File "D:\project\chatgpt-on-wechat\plugins\plugin_manager.py", line 159, in emit_event
|
||||
instance.handlers[e_context.event](e_context, *args, **kwargs)
|
||||
File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 255, in on_handle_context
|
||||
records = self._get_records(session_id, start_time, limit)
|
||||
File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 96, in _get_records
|
||||
c.execute("SELECT * FROM chat_records WHERE sessionid=? and timestamp>? ORDER BY timestamp DESC LIMIT ?", (session_id, start_date, limit))
|
||||
NameError: name 'start_date' is not defined
|
||||
[INFO][2023-04-16 00:23:36][app.py:14] - signal 2 received, exiting...
|
||||
```
|
||||
</details>
|
||||
value: |
|
||||
```log
|
||||
<此处粘贴终端日志>
|
||||
```
|
||||
Expected: ...
|
||||
Actual: ...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: Logs
|
||||
description: "Relevant logs from `run.log` (set `\"debug\": true` for more detail). ⚠️ Redact your API keys."
|
||||
render: shell
|
||||
validations:
|
||||
required: false
|
||||
|
||||
31
.github/ISSUE_TEMPLATE/2.feature.yml
vendored
31
.github/ISSUE_TEMPLATE/2.feature.yml
vendored
@@ -1,28 +1,33 @@
|
||||
name: Feature request 🚀
|
||||
description: 提出你对项目的新想法或建议。
|
||||
description: Suggest a new idea or improvement.
|
||||
title: "[Feature] "
|
||||
labels: ['status: needs check']
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
请在上方的`title`中填写简略总结,谢谢❤️。
|
||||
> 💡 English is recommended so global developers can help. 推荐使用英文提交,谢谢 ❤️
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: ⚠️ 搜索是否存在类似issue
|
||||
description: >
|
||||
请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框,搜索关键词查找是否存在相似issue。
|
||||
label: Self check
|
||||
options:
|
||||
- label: 我已经搜索过issues和disscussions,没有发现相似issue
|
||||
- label: I searched [existing issues](https://github.com/zhayujie/CowAgent/issues) (incl. closed) — no duplicate.
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: 总结
|
||||
description: 描述feature的功能。
|
||||
label: What's the problem?
|
||||
description: "The pain point or what's not working for you right now."
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: 举例
|
||||
description: 提供聊天示例,草图或相关网址。
|
||||
- type: textarea
|
||||
label: What would you like?
|
||||
description: "How you'd expect it to work. Examples, sketches, or links welcome."
|
||||
validations:
|
||||
required: false
|
||||
- type: checkboxes
|
||||
attributes:
|
||||
label: 动机
|
||||
description: 描述你提出该feature的动机,比如没有这项feature对你的使用造成了怎样的影响。 请提供更详细的场景描述,这可能会帮助我们发现并提出更好的解决方案。
|
||||
label: Contribution
|
||||
options:
|
||||
- label: I'd be interested in helping implement this.
|
||||
required: false
|
||||
|
||||
5
.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
5
.github/ISSUE_TEMPLATE/config.yml
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
blank_issues_enabled: true
|
||||
contact_links:
|
||||
- name: 📖 Documentation
|
||||
url: https://docs.cowagent.ai
|
||||
about: Setup guides, configuration, and FAQ.
|
||||
22
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
22
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,22 @@
|
||||
<!--
|
||||
Thanks for your contribution! Please write this PR in English.
|
||||
推荐使用英文填写,感谢 ❤️
|
||||
-->
|
||||
|
||||
## What does this PR do?
|
||||
|
||||
<!-- A short description of the change and why it's needed. -->
|
||||
|
||||
## Type of change
|
||||
|
||||
- [ ] Bug fix
|
||||
- [ ] New feature
|
||||
- [ ] Docs
|
||||
- [ ] Refactor / chore
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] I have read the [Contributing Guide](https://github.com/zhayujie/CowAgent/blob/master/CONTRIBUTING.md)
|
||||
- [ ] I tested this change locally
|
||||
- [ ] Code comments and docs are in English
|
||||
- [ ] Linked related issue (if any): closes #
|
||||
274
.github/workflows/release.yml
vendored
Normal file
274
.github/workflows/release.yml
vendored
Normal file
@@ -0,0 +1,274 @@
|
||||
name: Release Desktop
|
||||
|
||||
# Tag-driven release: push a tag like `v1.2.0` to build and publish the
|
||||
# desktop client for macOS (arm64 + x64) and Windows (x64). The tag is the
|
||||
# single source of truth for the version — it's written into package.json at
|
||||
# build time, so the maintainer never edits the version by hand.
|
||||
on:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
# Manual trigger for testing the full pipeline without cutting a real tag.
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: "Version to stamp (e.g. 1.0.0-test). Used for package.json and R2 path."
|
||||
type: string
|
||||
default: "0.0.0-dev"
|
||||
publish_r2:
|
||||
description: "Upload installers to R2 + register in D1 (needs Cloudflare secrets)"
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build ${{ matrix.name }}
|
||||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
# Don't cancel the other platforms if one fails — we want to see all
|
||||
# failures in a single run.
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- name: macOS arm64
|
||||
os: macos-14
|
||||
platform: mac
|
||||
arch: arm64
|
||||
eb_flags: --mac --arm64
|
||||
- name: macOS x64
|
||||
os: macos-15-intel
|
||||
platform: mac
|
||||
arch: x64
|
||||
eb_flags: --mac --x64
|
||||
- name: Windows x64
|
||||
os: windows-latest
|
||||
platform: win
|
||||
arch: x64
|
||||
eb_flags: --win --x64
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Derive version
|
||||
# Tag push: strip the leading "v" from GITHUB_REF_NAME (e.g. v1.2.0).
|
||||
# Manual dispatch: use the provided version input.
|
||||
id: ver
|
||||
shell: bash
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" = "push" ]; then
|
||||
ref="${GITHUB_REF_NAME:-}"
|
||||
echo "version=${ref#v}" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "version=${{ github.event.inputs.version }}" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
|
||||
- name: Set up Node
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "20"
|
||||
|
||||
- name: Build Python backend (PyInstaller)
|
||||
shell: bash
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r desktop/build/requirements-desktop.txt
|
||||
pip install pyinstaller
|
||||
# Run from repo root so the spec's relative datas resolve correctly.
|
||||
pyinstaller desktop/build/cowagent-backend.spec \
|
||||
--noconfirm \
|
||||
--distpath desktop/build/dist \
|
||||
--workpath desktop/build/build-work
|
||||
|
||||
- name: Install desktop deps
|
||||
working-directory: desktop
|
||||
run: npm ci
|
||||
|
||||
- name: Write version into package.json
|
||||
working-directory: desktop
|
||||
shell: bash
|
||||
run: npm version "${{ steps.ver.outputs.version }}" --no-git-tag-version --allow-same-version
|
||||
|
||||
- name: Build & publish (electron-builder)
|
||||
working-directory: desktop
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Signing secrets are passed through as-is; we only export them to the
|
||||
# environment below when non-empty. An empty CSC_LINK would make
|
||||
# electron-builder try to load a bogus certificate and fail, so unset
|
||||
# is the correct state for unsigned builds.
|
||||
MAC_CSC_LINK: ${{ secrets.MAC_CSC_LINK }}
|
||||
MAC_CSC_KEY_PASSWORD: ${{ secrets.MAC_CSC_KEY_PASSWORD }}
|
||||
APPLE_ID: ${{ secrets.APPLE_ID }}
|
||||
APPLE_APP_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}
|
||||
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
|
||||
WIN_CSC_LINK: ${{ secrets.WIN_CSC_LINK }}
|
||||
WIN_CSC_KEY_PASSWORD: ${{ secrets.WIN_CSC_KEY_PASSWORD }}
|
||||
run: |
|
||||
npm run build
|
||||
|
||||
# Only export signing vars when provided. Empty strings are NOT the
|
||||
# same as unset to electron-builder: an empty CSC_LINK is treated as
|
||||
# a (broken) certificate path and aborts the build. Leaving them
|
||||
# unset makes electron-builder fall back to an unsigned build.
|
||||
if [ -n "$MAC_CSC_LINK" ]; then
|
||||
export CSC_LINK="$MAC_CSC_LINK"
|
||||
export CSC_KEY_PASSWORD="$MAC_CSC_KEY_PASSWORD"
|
||||
fi
|
||||
if [ -z "$WIN_CSC_LINK" ]; then
|
||||
unset WIN_CSC_LINK WIN_CSC_KEY_PASSWORD
|
||||
fi
|
||||
|
||||
# Publish to the GitHub Release on tag pushes; otherwise build only.
|
||||
if [ "${{ github.event_name }}" = "push" ]; then
|
||||
PUBLISH=always
|
||||
else
|
||||
PUBLISH=never
|
||||
fi
|
||||
npx electron-builder ${{ matrix.eb_flags }} --publish "$PUBLISH"
|
||||
|
||||
- name: Upload artifacts
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
# One bundle per platform/arch so the publish job can collect them all.
|
||||
name: cowagent-${{ matrix.platform }}-${{ matrix.arch }}
|
||||
path: |
|
||||
desktop/release/*.dmg
|
||||
desktop/release/*.exe
|
||||
desktop/release/*.yml
|
||||
desktop/release/*.blockmap
|
||||
if-no-files-found: ignore
|
||||
retention-days: 7
|
||||
|
||||
# Mirror the release installers to R2 (CDN-backed) and register them in D1 so
|
||||
# cowagent.ai/download/{platform}/latest can resolve and count downloads.
|
||||
# Runs only on tag pushes, and is a no-op (skips) until the Cloudflare secrets
|
||||
# are configured, so it never blocks unsigned/dry builds.
|
||||
publish-r2:
|
||||
name: Publish to R2 + D1
|
||||
# Require every platform in the build matrix to succeed before publishing,
|
||||
# so a release on R2/D1 is always complete (all installers present) rather
|
||||
# than partial. needs: build already gates on all matrix jobs succeeding.
|
||||
needs: build
|
||||
runs-on: ubuntu-latest
|
||||
# Run on a tag push, or on a manual dispatch when publish_r2 is checked.
|
||||
if: >-
|
||||
(github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')) ||
|
||||
(github.event_name == 'workflow_dispatch' && github.event.inputs.publish_r2 == 'true')
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Guard on Cloudflare secrets
|
||||
id: guard
|
||||
env:
|
||||
CF_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
run: |
|
||||
if [ -n "$CF_TOKEN" ]; then
|
||||
echo "enabled=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "enabled=false" >> "$GITHUB_OUTPUT"
|
||||
echo "::notice::CLOUDFLARE_API_TOKEN not set — skipping R2/D1 publish."
|
||||
fi
|
||||
|
||||
- name: Derive version
|
||||
if: steps.guard.outputs.enabled == 'true'
|
||||
id: ver
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" = "push" ]; then
|
||||
echo "version=${GITHUB_REF_NAME#v}" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "version=${{ github.event.inputs.version }}" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- name: Download all build artifacts
|
||||
if: steps.guard.outputs.enabled == 'true'
|
||||
uses: actions/download-artifact@v4
|
||||
with:
|
||||
path: artifacts
|
||||
|
||||
- name: Stage installers
|
||||
if: steps.guard.outputs.enabled == 'true'
|
||||
id: stage
|
||||
run: |
|
||||
mkdir -p dist
|
||||
# Flatten installers from every per-platform artifact dir; only the
|
||||
# user-facing installers go to R2 (updater .yml/.blockmap stay on the
|
||||
# GitHub Release, which electron-updater reads directly).
|
||||
find artifacts -type f \( -name '*.dmg' -o -name '*.exe' \) -exec cp {} dist/ \;
|
||||
echo "Staged files:"; ls -la dist
|
||||
# When the whole matrix failed there's nothing to publish; flag it so
|
||||
# the R2/D1 steps skip instead of writing an empty/partial release.
|
||||
if [ -n "$(ls -A dist 2>/dev/null)" ]; then
|
||||
echo "has_files=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "has_files=false" >> "$GITHUB_OUTPUT"
|
||||
echo "::warning::No installers found in any artifact — skipping R2/D1 publish."
|
||||
fi
|
||||
|
||||
- name: Upload installers to R2
|
||||
if: steps.guard.outputs.enabled == 'true' && steps.stage.outputs.has_files == 'true'
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
VER: ${{ steps.ver.outputs.version }}
|
||||
run: |
|
||||
# Reuse the existing cow-skills bucket under a desktop/ prefix; this
|
||||
# is served by the cdn.cowagent.ai custom domain.
|
||||
for f in dist/*; do
|
||||
base="$(basename "$f")"
|
||||
key="desktop/v${VER}/${base}"
|
||||
echo "==> Uploading $base -> r2://cow-skills/$key"
|
||||
npx --yes wrangler@latest r2 object put "cow-skills/$key" \
|
||||
--file "$f" --remote
|
||||
done
|
||||
|
||||
- name: Register release rows in D1
|
||||
if: steps.guard.outputs.enabled == 'true' && steps.stage.outputs.has_files == 'true'
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
VER: ${{ steps.ver.outputs.version }}
|
||||
run: |
|
||||
# Map each installer filename to a platform id. dmg arch is in the
|
||||
# name (…-arm64.dmg / …-x64.dmg); .exe is the Windows installer.
|
||||
sql_file="$(mktemp)"
|
||||
|
||||
# Pre-releases (e.g. 1.0.0-test / -beta / -rc.1 / -alpha / -dev) are
|
||||
# recorded but NEVER marked latest, so /download/<p>/latest keeps
|
||||
# serving the last stable build. They also must not clear an existing
|
||||
# stable's latest flag. Only a final version (no pre-release suffix)
|
||||
# becomes the new latest and clears the previous one per platform.
|
||||
case "$VER" in
|
||||
*-*) is_latest=0; echo "==> $VER is a pre-release; not marking latest." ;;
|
||||
*) is_latest=1; echo "==> $VER is a stable release; marking latest." ;;
|
||||
esac
|
||||
|
||||
for f in dist/*; do
|
||||
base="$(basename "$f")"
|
||||
size="$(stat -c%s "$f")"
|
||||
case "$base" in
|
||||
*arm64.dmg) platform="mac-arm64" ;;
|
||||
*x64.dmg) platform="mac-x64" ;;
|
||||
*.exe) platform="win" ;;
|
||||
*) echo "Skipping unrecognized artifact: $base"; continue ;;
|
||||
esac
|
||||
key="v${VER}/${base}"
|
||||
# Stable only: clear the previous latest for THIS platform first, so
|
||||
# a partial backfill never wipes other platforms' latest flag.
|
||||
if [ "$is_latest" = "1" ]; then
|
||||
echo "UPDATE releases SET is_latest = 0 WHERE platform = '${platform}';" >> "$sql_file"
|
||||
fi
|
||||
echo "INSERT OR REPLACE INTO releases (version, platform, filename, size, is_latest) VALUES ('${VER}', '${platform}', '${key}', ${size}, ${is_latest});" >> "$sql_file"
|
||||
done
|
||||
echo "==> D1 statements:"; cat "$sql_file"
|
||||
npx --yes wrangler@latest d1 execute cow-desktop --remote --file "$sql_file"
|
||||
32
.github/workflows/test-windows-bash.yml
vendored
Normal file
32
.github/workflows/test-windows-bash.yml
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
name: Windows Bash Streaming Tests
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "agent/tools/bash/bash.py"
|
||||
- "tests/test_bash_streaming.py"
|
||||
- ".github/workflows/test-windows-bash.yml"
|
||||
|
||||
jobs:
|
||||
windows-bash-tests:
|
||||
runs-on: windows-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
cache: pip
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
python -m pip install pytest
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
- name: Run Windows Bash streaming tests
|
||||
run: python -m pytest tests/test_bash_streaming.py -v
|
||||
14
.gitignore
vendored
14
.gitignore
vendored
@@ -32,7 +32,6 @@ plugins/banwords/lib/__pycache__
|
||||
!plugins/role
|
||||
!plugins/keyword
|
||||
!plugins/linkai
|
||||
!plugins/agent
|
||||
!plugins/cow_cli
|
||||
client_config.json
|
||||
ref/
|
||||
@@ -46,3 +45,16 @@ dist/
|
||||
build/
|
||||
*.egg-info/
|
||||
.cow.pid
|
||||
|
||||
# Desktop backend packaging: keep the source files (spec/requirements/script)
|
||||
# tracked even though the generic build/ rule above ignores them, but never
|
||||
# track the build outputs or local venv.
|
||||
!desktop/build/
|
||||
desktop/build/*
|
||||
!desktop/build/cowagent-backend.spec
|
||||
!desktop/build/requirements-desktop.txt
|
||||
!desktop/build/build-backend.sh
|
||||
|
||||
# Icon authoring scratch dir: intermediate assets used to produce the final
|
||||
# icons. Only the finished icons under desktop/resources/ should be committed.
|
||||
desktop/resources/.icon-work/
|
||||
|
||||
61
CONTRIBUTING.md
Normal file
61
CONTRIBUTING.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# Contributing to CowAgent
|
||||
|
||||
Thanks for taking the time to contribute! 🎉 CowAgent is built by a global
|
||||
community, and contributions of all sizes are welcome — from typo fixes to new
|
||||
features.
|
||||
|
||||
## Language policy
|
||||
|
||||
To keep the project accessible to a global community, **please write issues,
|
||||
pull requests, code comments, and commit messages in English.**
|
||||
|
||||
> 为方便全球开发者协作,请尽量使用**英文**提交 issue、PR、代码注释与
|
||||
> commit message。不必担心英文不完美——表达清楚即可,工具翻译也完全没问题。感谢理解 ❤️
|
||||
|
||||
## Reporting issues
|
||||
|
||||
Found a bug or have an idea? [Open an issue](https://github.com/zhayujie/CowAgent/issues/new/choose).
|
||||
|
||||
Before opening one, please search existing issues (including closed ones) to
|
||||
avoid duplicates, and make sure you're on the latest version.
|
||||
|
||||
## Submitting a pull request
|
||||
|
||||
1. **Fork** the repo and create a branch from `master`
|
||||
(e.g. `feat/web-search`, `fix/telegram-reconnect`).
|
||||
2. Make your change. Keep it focused — one logical change per PR.
|
||||
3. Follow the existing code style. Write comments and docstrings in English.
|
||||
4. Run the app locally to confirm your change works.
|
||||
5. Open a PR with a clear title and a short description of **what** and **why**.
|
||||
|
||||
We keep the bar friendly: clear, focused, and working is enough. Maintainers are
|
||||
happy to help polish details during review.
|
||||
|
||||
### Commit & PR titles
|
||||
|
||||
Use a short, imperative summary. The [Conventional Commits](https://www.conventionalcommits.org/)
|
||||
style is preferred but not required:
|
||||
|
||||
```
|
||||
feat: add web search tool
|
||||
fix: reconnect Telegram websocket on timeout
|
||||
docs: clarify Docker setup
|
||||
```
|
||||
|
||||
## Development setup
|
||||
|
||||
See the [Install from Source](https://docs.cowagent.ai/guide/manual-install)
|
||||
guide. In short:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/zhayujie/CowAgent.git
|
||||
cd CowAgent
|
||||
pip install -r requirements.txt
|
||||
pip install -e .
|
||||
cow start
|
||||
```
|
||||
|
||||
## Code of conduct
|
||||
|
||||
Be respectful and constructive. We want CowAgent to be a welcoming place for
|
||||
everyone.
|
||||
166
README.md
166
README.md
@@ -1,20 +1,28 @@
|
||||
<p align="center"><img src="https://github.com/user-attachments/assets/eca9a9ec-8534-4615-9e0f-96c5ac1d10a3" alt="CowAgent" width="420" /></p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/zhayujie/CowAgent/releases/latest"><img src="https://img.shields.io/github/v/release/zhayujie/CowAgent" alt="Latest release"></a>
|
||||
<a href="https://github.com/zhayujie/CowAgent/blob/master/LICENSE"><img src="https://img.shields.io/github/license/zhayujie/CowAgent" alt="License: MIT"></a>
|
||||
<a href="https://github.com/zhayujie/CowAgent"><img src="https://img.shields.io/github/stars/zhayujie/CowAgent?style=flat-square" alt="Stars"></a> <br/>
|
||||
<a href="https://github.com/zhayujie/CowAgent/releases/latest"><img src="https://img.shields.io/github/v/release/zhayujie/CowAgent?cacheSeconds=3600" alt="Latest release"></a>
|
||||
<a href="https://github.com/zhayujie/CowAgent/blob/master/LICENSE"><img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License: MIT"></a>
|
||||
<a href="https://github.com/zhayujie/CowAgent"><img src="https://img.shields.io/github/stars/zhayujie/CowAgent?style=flat-square&cacheSeconds=3600" alt="Stars"></a>
|
||||
<a href="https://docs.cowagent.ai/"><img src="https://img.shields.io/badge/Docs-cowagent.ai-blue?style=flat&logo=readthedocs&logoColor=white" alt="Docs"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://trendshift.io/repositories/25763" target="_blank"><img src="https://trendshift.io/api/badge/repositories/25763" alt="zhayujie%2FCowAgent | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
[English] | [<a href="docs/zh/README.md">中文</a>] | [<a href="docs/ja/README.md">日本語</a>]
|
||||
</p>
|
||||
|
||||
**CowAgent** is an open-source super AI assistant that proactively plans tasks, operates your computer and external services, creates and runs Skills, and grows alongside you with a personal knowledge base and long-term memory — a reference implementation of Agent Harness engineering.
|
||||
**CowAgent** is an open-source super AI assistant that proactively plans tasks, controls your computer and external services, creates and runs Skills, builds a personal knowledge base and long-term memory, and grows alongside you through self-evolution — a reference implementation of Agent Harness engineering.
|
||||
|
||||
CowAgent is lightweight, easy to deploy, and built to extend. Plug in any major LLM provider, run it across Web and major IM platforms, 24/7 on a personal computer or server.
|
||||
CowAgent is lightweight, easy to deploy, and built to extend. Plug in any major LLM provider and run it 24/7 on a personal computer or server, across the web and all major IM platforms.
|
||||
|
||||
<p align="center">
|
||||
<a href="https://cowagent.ai/">🌐 Website</a> ·
|
||||
<a href="https://docs.cowagent.ai/en/intro/index">📖 Docs</a> ·
|
||||
<a href="https://docs.cowagent.ai/en/guide/quick-start">🚀 Quick Start</a> ·
|
||||
<a href="https://docs.cowagent.ai/intro/index">📖 Docs</a> ·
|
||||
<a href="https://docs.cowagent.ai/guide/quick-start">🚀 Quick Start</a> ·
|
||||
<a href="https://skills.cowagent.ai/">🧩 Skill Hub</a> ·
|
||||
<a href="https://link-ai.tech/cowagent/create">☁️ Try Online</a>
|
||||
</p>
|
||||
@@ -25,31 +33,32 @@ CowAgent is lightweight, easy to deploy, and built to extend. Plug in any major
|
||||
|
||||
| Capability | Description |
|
||||
| :--- | :--- |
|
||||
| 🤖 [Autonomous Task Planning](https://docs.cowagent.ai/en/intro/architecture) | Decomposes complex tasks and executes them step by step, looping over tools until the goal is reached |
|
||||
| 🧠 [Long-term Memory](https://docs.cowagent.ai/en/memory/index) | Three-tier architecture (context → daily → core), automatic Deep Dream distillation, hybrid keyword + vector retrieval |
|
||||
| 📚 [Personal Knowledge Base](https://docs.cowagent.ai/en/knowledge/index) | Auto-curates structured knowledge into a Markdown wiki, builds an evolving knowledge graph with visual browsing |
|
||||
| 🧩 [Skills System](https://docs.cowagent.ai/en/skills/index) | One-click install from [Skill Hub](https://skills.cowagent.ai/), GitHub, ClawHub; or create custom skills via natural-language conversation |
|
||||
| 🔧 [Tool System](https://docs.cowagent.ai/en/tools/index) | Built-in file I/O, terminal, browser, scheduler, memory retrieval, web search, and 10+ more tools — with native MCP integration |
|
||||
| 💬 [Multi-channel Integration](https://docs.cowagent.ai/en/channels/index) | A single Agent simultaneously serves Web, WeChat, Feishu, DingTalk, WeCom, QQ, and Official Accounts |
|
||||
| 🎨 Multimodal Messaging | First-class support for text, images, voice, and files — recognition, generation, and delivery |
|
||||
| ⚙️ [Pluggable Models](https://docs.cowagent.ai/en/models/index) | Claude, GPT, Gemini, DeepSeek, Qwen, GLM, Kimi, MiniMax, Doubao, and more — swap providers from the Web console with one click |
|
||||
| 📦 [Batteries Included](https://docs.cowagent.ai/en/guide/quick-start) | One-line installer, unified Web console, multiple deployment modes (local, Docker, server) |
|
||||
| [Planning](https://docs.cowagent.ai/intro/architecture) | Decomposes complex tasks and executes them step by step, looping over tools until the goal is reached |
|
||||
| [Memory](https://docs.cowagent.ai/memory/index) | Three-tier architecture (context → daily → core), automatic Deep Dream distillation, hybrid keyword + vector retrieval |
|
||||
| [Knowledge](https://docs.cowagent.ai/knowledge/index) | Auto-curates structured knowledge into a Markdown wiki, builds an evolving knowledge graph with visual browsing |
|
||||
| [Evolution](https://docs.cowagent.ai/memory/self-evolution) | Self-Evolution reviews conversations automatically to improve skills, follow up on unfinished tasks, and consolidate memory and knowledge, growing through everyday use |
|
||||
| [Skills](https://docs.cowagent.ai/skills/index) | One-click install from [Skill Hub](https://skills.cowagent.ai/), GitHub, ClawHub; or create custom skills via natural-language conversation |
|
||||
| [Tools](https://docs.cowagent.ai/tools/index) | Built-in file I/O, terminal, browser, scheduler, memory retrieval, web search, and 10+ more tools — with native MCP integration |
|
||||
| [Channels](https://docs.cowagent.ai/channels/index) | Integrates with Web, WeChat, Feishu, DingTalk, WeCom, QQ, Official Accounts, Telegram, and Slack |
|
||||
| Multimodal | First-class support for text, images, voice, and files — recognition, generation, and delivery |
|
||||
| [Models](https://docs.cowagent.ai/models/index) | Claude, GPT, Gemini, DeepSeek, Qwen, GLM, Kimi, MiniMax, Doubao, and more — swap providers from the Web console with one click |
|
||||
| [Deploy](https://docs.cowagent.ai/guide/quick-start) | One-line installer, unified Web console, multiple deployment modes (local, Docker, server) |
|
||||
|
||||
<br/>
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
<img src="https://cdn.jsdelivr.net/gh/zhayujie/cowagent-assets@main/architecture/en/architecture.jpg" alt="CowAgent Architecture" width="750"/>
|
||||
<img src="https://cdn.jsdelivr.net/gh/zhayujie/cowagent-assets@main/architecture/en/architecture.png" alt="CowAgent Architecture" width="750"/>
|
||||
|
||||
CowAgent is a complete **Agent Harness**: messages flow in through **Channels**; the **Agent Core** plans and decides using memory, knowledge, and the available tools/skills; **Models** generate the response; results are sent back through the originating channel. Every layer is decoupled and independently extensible.
|
||||
CowAgent is a complete **Agent Harness**: messages flow in through **Channels**; the **Agent Core** plans and reasons over memory, knowledge, and the available tools and skills; **Models** generate the response, which is sent back through the originating channel. Every layer is decoupled and independently extensible.
|
||||
|
||||
Read more in [Architecture](https://docs.cowagent.ai/en/intro/architecture).
|
||||
Read more in [Architecture](https://docs.cowagent.ai/intro/architecture).
|
||||
|
||||
<br/>
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
The project ships with a one-line installer that handles dependencies, configuration, and startup automatically:
|
||||
A one-line installer takes care of dependencies, configuration, and startup:
|
||||
|
||||
**Linux / macOS:**
|
||||
|
||||
@@ -70,11 +79,13 @@ curl -O https://cdn.link-ai.tech/code/cow/docker-compose.yml
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Once started, open `http://localhost:9899` to access the **Web console** — a single place to configure models, connect channels, and install skills.
|
||||
Once started, open `http://localhost:9899` to access the **Web console** — your one-stop hub to chat with the Agent, configure models, connect channels, and install skills.
|
||||
|
||||
> 📖 Detailed guides: [One-line Installer](https://docs.cowagent.ai/en/guide/quick-start) · [Manual Install from Source](https://docs.cowagent.ai/en/guide/manual-install) · [Upgrade](https://docs.cowagent.ai/en/guide/upgrade)
|
||||
> Deploying on a server? Set `web_host` to `0.0.0.0` in `config.json` to make the console reachable from outside, and set `web_password` to protect it. Don't forget to open port `9899` in your firewall or security group.
|
||||
|
||||
After installation, manage the service with the [`cow` CLI](https://docs.cowagent.ai/en/cli/index):
|
||||
> 📖 Detailed guides: [Quick Start](https://docs.cowagent.ai/guide/quick-start) · [Install from Source](https://docs.cowagent.ai/guide/manual-install) · [Upgrade](https://docs.cowagent.ai/guide/upgrade)
|
||||
|
||||
After installation, manage the service with the [cow CLI](https://docs.cowagent.ai/cli/index):
|
||||
|
||||
```bash
|
||||
cow start | stop | restart # service control
|
||||
@@ -88,55 +99,60 @@ cow install-browser # install browser automation
|
||||
|
||||
## 🤖 Models
|
||||
|
||||
CowAgent supports all mainstream LLM providers. **Chat, vision, image generation, ASR/TTS, and embeddings** can each be configured against a different vendor.
|
||||
CowAgent supports all mainstream LLM providers. **Chat, vision, image generation, ASR/TTS, and embeddings** can each be routed to a different vendor. Providers are configured directly in the Web console — no manual file editing required.
|
||||
|
||||
| Provider | Featured Models | Chat | Vision | Image Gen | ASR | TTS | Embedding |
|
||||
| --- | --- | :-: | :-: | :-: | :-: | :-: | :-: |
|
||||
| [Claude](https://docs.cowagent.ai/en/models/claude) | claude-opus-4-7 | ✅ | ✅ | | | | |
|
||||
| [OpenAI](https://docs.cowagent.ai/en/models/openai) | gpt-5.5, o-series | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [Gemini](https://docs.cowagent.ai/en/models/gemini) | gemini-3.5-flash | ✅ | ✅ | ✅ | | | |
|
||||
| [DeepSeek](https://docs.cowagent.ai/en/models/deepseek) | deepseek-v4-flash / pro | ✅ | | | | | |
|
||||
| [Qwen](https://docs.cowagent.ai/en/models/qwen) | qwen3.7-max | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [GLM](https://docs.cowagent.ai/en/models/glm) | glm-5.1, glm-5v-turbo | ✅ | ✅ | | ✅ | | ✅ |
|
||||
| [Doubao](https://docs.cowagent.ai/en/models/doubao) | doubao-seed-2.0 series | ✅ | ✅ | ✅ | | | ✅ |
|
||||
| [Kimi](https://docs.cowagent.ai/en/models/kimi) | kimi-k2.6 | ✅ | ✅ | | | | |
|
||||
| [MiniMax](https://docs.cowagent.ai/en/models/minimax) | MiniMax-M2.7 | ✅ | ✅ | ✅ | | ✅ | |
|
||||
| [Qianfan](https://docs.cowagent.ai/en/models/qianfan) | ernie-5.1 | ✅ | ✅ | | | | |
|
||||
| [LinkAI](https://docs.cowagent.ai/en/models/linkai) | 100+ models, unified gateway | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [Custom](https://docs.cowagent.ai/en/models/custom) | Local models / third-party proxy | ✅ | | | | | |
|
||||
| [Claude](https://docs.cowagent.ai/models/claude) | claude-fable-5 | ✅ | ✅ | | | | |
|
||||
| [OpenAI](https://docs.cowagent.ai/models/openai) | gpt-5.5, o-series | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [Gemini](https://docs.cowagent.ai/models/gemini) | gemini-3.5-flash | ✅ | ✅ | ✅ | | | |
|
||||
| [DeepSeek](https://docs.cowagent.ai/models/deepseek) | deepseek-v4-flash / pro | ✅ | | | | | |
|
||||
| [Qwen](https://docs.cowagent.ai/models/qwen) | qwen3.7-plus | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [GLM](https://docs.cowagent.ai/models/glm) | glm-5.2, glm-5v-turbo | ✅ | ✅ | | ✅ | | ✅ |
|
||||
| [Doubao](https://docs.cowagent.ai/models/doubao) | doubao-seed-2.0 series | ✅ | ✅ | ✅ | | | ✅ |
|
||||
| [Kimi](https://docs.cowagent.ai/models/kimi) | kimi-k2.7-code | ✅ | ✅ | | | | |
|
||||
| [MiniMax](https://docs.cowagent.ai/models/minimax) | MiniMax-M3 | ✅ | ✅ | ✅ | | ✅ | |
|
||||
| [ERNIE](https://docs.cowagent.ai/models/qianfan) | ernie-5.1 | ✅ | ✅ | | | | |
|
||||
| [MiMo](https://docs.cowagent.ai/models/mimo) | mimo-v2.5 / pro | ✅ | ✅ | | | ✅ | |
|
||||
| [LinkAI](https://docs.cowagent.ai/models/linkai) | One key for 100+ models | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [Custom](https://docs.cowagent.ai/models/custom) | Local models / third-party proxy | ✅ | | | | | |
|
||||
|
||||
> The Web console is the recommended way to configure providers — no manual file editing required. For manual setup, see each provider's docs and the [Models overview](https://docs.cowagent.ai/en/models/index).
|
||||
> For details on each provider, see the [Models overview](https://docs.cowagent.ai/models/index).
|
||||
|
||||
<br/>
|
||||
|
||||
## 💬 Channels
|
||||
|
||||
A single Agent instance can serve multiple channels at once. Switch with the `channel_type` config or run several channels in parallel.
|
||||
A single Agent instance can serve multiple channels in parallel. Most channels can be onboarded right from the Web console.
|
||||
|
||||
| Channel | Text | Image | File | Voice | Group |
|
||||
| --- | :-: | :-: | :-: | :-: | :-: |
|
||||
| [Web Console](https://docs.cowagent.ai/en/channels/web) (default) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [WeChat](https://docs.cowagent.ai/en/channels/weixin) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [Feishu / Lark](https://docs.cowagent.ai/en/channels/feishu) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [DingTalk](https://docs.cowagent.ai/en/channels/dingtalk) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [WeCom Bot](https://docs.cowagent.ai/en/channels/wecom-bot) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [QQ](https://docs.cowagent.ai/en/channels/qq) | ✅ | ✅ | ✅ | | ✅ |
|
||||
| [WeCom App](https://docs.cowagent.ai/en/channels/wecom) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [WeChat Official Account](https://docs.cowagent.ai/en/channels/wechatmp) | ✅ | ✅ | | ✅ | |
|
||||
| [Web Console](https://docs.cowagent.ai/channels/web) (default) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [Telegram](https://docs.cowagent.ai/channels/telegram) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [Slack](https://docs.cowagent.ai/channels/slack) | ✅ | ✅ | ✅ | | ✅ |
|
||||
| [Discord](https://docs.cowagent.ai/channels/discord) | ✅ | ✅ | ✅ | | ✅ |
|
||||
| [WeChat](https://docs.cowagent.ai/channels/weixin) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [Feishu / Lark](https://docs.cowagent.ai/channels/feishu) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [DingTalk](https://docs.cowagent.ai/channels/dingtalk) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [WeCom Bot](https://docs.cowagent.ai/channels/wecom-bot) | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||||
| [QQ](https://docs.cowagent.ai/channels/qq) | ✅ | ✅ | ✅ | | ✅ |
|
||||
| [WeCom App](https://docs.cowagent.ai/channels/wecom) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [WeChat Customer Service](https://docs.cowagent.ai/channels/wechat-kf) | ✅ | ✅ | ✅ | ✅ | |
|
||||
| [WeChat Official Account](https://docs.cowagent.ai/channels/wechatmp) | ✅ | ✅ | | ✅ | |
|
||||
|
||||
> Feishu and WeCom Bot can be onboarded by **scanning a QR code right inside the Web console** — no public IP required. See the [Channels overview](https://docs.cowagent.ai/en/channels/index).
|
||||
> See the [Channels overview](https://docs.cowagent.ai/channels/index) for setup details.
|
||||
|
||||
<img src="https://cdn.jsdelivr.net/gh/zhayujie/cowagent-assets@main/screenshots/en/web-console-chat.png" alt="CowAgent Web Console" width="800"/>
|
||||
|
||||
*The Web console doubles as the default channel and the unified place to configure and manage your Agent.*
|
||||
*The Web console is the default channel and the unified entry point to configure models, channels, skills, memory, and more.*
|
||||
|
||||
<br/>
|
||||
|
||||
## 🧠 Memory & Knowledge Base
|
||||
|
||||
**Long-term memory** uses a three-tier architecture: conversation context (short-term) → daily memory (mid-term) → MEMORY.md (long-term). A nightly **Deep Dream** pass distills scattered memories into refined long-term entries and a narrative journal. See [Long-term Memory](https://docs.cowagent.ai/en/memory/index) · [Deep Dream](https://docs.cowagent.ai/en/memory/deep-dream).
|
||||
**Long-term memory** uses a three-tier architecture: conversation context (short-term) → daily memory (mid-term) → MEMORY.md (long-term). A nightly **Deep Dream** pass distills scattered memories into refined long-term entries and a narrative journal. See [Long-term Memory](https://docs.cowagent.ai/memory/index) · [Deep Dream](https://docs.cowagent.ai/memory/deep-dream).
|
||||
|
||||
**Personal knowledge base** complements the time-ordered memory by organizing structured knowledge **by topic**. The Agent automatically curates valuable information from conversations, maintains cross-references and indexes, and the Web console offers an interactive knowledge-graph view. See [Personal Knowledge Base](https://docs.cowagent.ai/en/knowledge/index).
|
||||
**Personal knowledge base** complements the time-ordered memory by organizing structured knowledge **by topic**. The Agent automatically curates valuable information from conversations, maintains cross-references and indexes, and the Web console offers an interactive knowledge-graph view. See [Personal Knowledge Base](https://docs.cowagent.ai/knowledge/index).
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@@ -159,16 +175,16 @@ A single Agent instance can serve multiple channels at once. Switch with the `ch
|
||||
|
||||
### Tool System
|
||||
|
||||
**Built-in tools** cover file I/O (`read` / `write` / `edit` / `ls`), terminal (`bash`), file delivery (`send`), memory retrieval (`memory`), environment variables (`env_config`), web fetching (`web_fetch`), scheduling (`scheduler`), web search (`web_search`), vision (`vision`), and browser automation (`browser`).
|
||||
**Built-in tools** cover file I/O (`read` / `write` / `edit` / `ls`), terminal (`bash`), file sending (`send`), memory retrieval (`memory`), environment variables (`env_config`), web fetching (`web_fetch`), scheduling (`scheduler`), web search (`web_search`), vision (`vision`), and browser automation (`browser`).
|
||||
|
||||
**MCP protocol** integrates the open ecosystem of [Model Context Protocol](https://modelcontextprotocol.io) servers. A single `mcp.json` is enough — supports stdio / SSE transports, hot reload, and zero-code integration.
|
||||
|
||||
Learn more: [Tools overview](https://docs.cowagent.ai/en/tools/index) · [MCP integration](https://docs.cowagent.ai/en/tools/mcp).
|
||||
Learn more: [Tools overview](https://docs.cowagent.ai/tools/index) · [MCP integration](https://docs.cowagent.ai/tools/mcp).
|
||||
|
||||
### Skills System
|
||||
|
||||
- **[Skill Hub](https://skills.cowagent.ai/)** — open skill marketplace: browse, search, install in one click
|
||||
- **GitHub / ClawHub** — multiple skill sources, 40,000+ skills available
|
||||
- **GitHub / ClawHub / URL and more** — install skills from any source
|
||||
- **Conversational authoring** — generate custom skills through dialogue with `skill-creator`; turn any workflow or third-party API into a reusable skill
|
||||
|
||||
```bash
|
||||
@@ -177,12 +193,18 @@ Learn more: [Tools overview](https://docs.cowagent.ai/en/tools/index) · [MCP in
|
||||
/skill install <name> # one-click install
|
||||
```
|
||||
|
||||
Learn more: [Skills overview](https://docs.cowagent.ai/en/skills/index) · [Creating Skills](https://docs.cowagent.ai/en/skills/create).
|
||||
Learn more: [Skills overview](https://docs.cowagent.ai/skills/index) · [Creating Skills](https://docs.cowagent.ai/skills/create).
|
||||
|
||||
<br/>
|
||||
|
||||
## 🏷 Changelog
|
||||
|
||||
> **2026.06.18:** [v2.1.2](https://github.com/zhayujie/CowAgent/releases/tag/2.1.2) — Web console upgrades (scheduled task management, knowledge base categories, multiple custom model providers), Self-Evolution improvements, new models (kimi-k2.7-code, glm-5.2), security hardening and refinements.
|
||||
|
||||
> **2026.06.09:** [v2.1.1](https://github.com/zhayujie/CowAgent/releases/tag/2.1.1) — Self-Evolution, Web console upgrades (message management, parallel sessions), cross-platform MCP enhancements with concurrent calls, new models (MiniMax-M3, qwen3.7-plus), Python 3.13 support.
|
||||
|
||||
> **2026.06.01:** [v2.1.0](https://github.com/zhayujie/CowAgent/releases/tag/2.1.0) — Internationalization, new channels (Telegram, Discord, Slack, WeChat Customer Service), CLI interaction upgrades, streamlined one-line install, MCP Streamable HTTP support, new models (claude-opus-4-8, MiMo).
|
||||
|
||||
> **2026.05.22:** [v2.0.9](https://github.com/zhayujie/CowAgent/releases/tag/2.0.9) — Model management, MCP protocol support, persistent browser sessions, new models (gpt-5.5, gemini-3.5-flash, qwen3.7-max), deployment hardening.
|
||||
|
||||
> **2026.05.06:** [v2.0.8](https://github.com/zhayujie/CowAgent/releases/tag/2.0.8) — Feishu channel overhaul (voice, streaming, QR onboarding), DeepSeek V4 and Baidu Qianfan support, scheduler tool upgrades.
|
||||
@@ -195,15 +217,13 @@ Learn more: [Skills overview](https://docs.cowagent.ai/en/skills/index) · [Crea
|
||||
|
||||
> **2026.02.03:** [v2.0.0](https://github.com/zhayujie/CowAgent/releases/tag/2.0.0) — Major upgrade to a super Agent assistant with multi-step task planning, long-term memory, and the Skills framework.
|
||||
|
||||
Full history: [Release Notes](https://docs.cowagent.ai/en/releases/overview)
|
||||
Full history: [Release Notes](https://docs.cowagent.ai/releases/overview)
|
||||
|
||||
<br/>
|
||||
|
||||
## 🤝 Community & Support
|
||||
|
||||
- 🐛 [File an Issue](https://github.com/zhayujie/CowAgent/issues) · 💬 [GitHub Discussions](https://github.com/zhayujie/CowAgent/discussions) · 📖 [FAQs](https://github.com/zhayujie/CowAgent/wiki/FAQs)
|
||||
|
||||
Scan the WeChat QR code to join the open-source community group:
|
||||
[File an issue](https://github.com/zhayujie/CowAgent/issues) on GitHub, or scan the QR code below to join our WeChat community:
|
||||
|
||||
<img width="130" src="https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/open-community.png">
|
||||
|
||||
@@ -217,11 +237,23 @@ Scan the WeChat QR code to join the open-source community group:
|
||||
|
||||
<br/>
|
||||
|
||||
## 🏢 Enterprise Services
|
||||
|
||||
[**LinkAI**](https://link-ai.tech/) is an all-in-one AI Agent platform for enterprises and developers, offering managed hosting and enterprise-grade support for CowAgent:
|
||||
|
||||
- **🚀 Zero-deployment hosted runtime** — spin up a [CowAgent online assistant](https://link-ai.tech/cowagent/create) in under a minute, no server required
|
||||
- **🧠 Agent infrastructure** — unified access to LLMs, knowledge bases, databases, skills, and workflows; plug-and-play building blocks that extend what CowAgent can do
|
||||
- **🏢 Team & enterprise features** — workspaces, role-based access, audit logs, and private deployment for production use cases
|
||||
|
||||
For enterprise inquiries: sales@simple-future.tech or [scan the QR code](https://cdn.link-ai.tech/consultant.jpg) to reach our team on WeChat.
|
||||
|
||||
<br/>
|
||||
|
||||
## 🛠️ Development & Contributing
|
||||
|
||||
Contributions are welcome — add a new channel by following the [Feishu channel reference](https://github.com/zhayujie/CowAgent/blob/master/channel/feishu/feishu_channel.py), or contribute new skills to [Skill Hub](https://skills.cowagent.ai/submit).
|
||||
All kinds of contributions are welcome — new features, bug fixes, performance improvements, docs, or sharing your own skills on the [Skill Hub](https://skills.cowagent.ai/submit). See [CONTRIBUTING.md](/CONTRIBUTING.md) to get started, then open an Issue to discuss or send a PR directly.
|
||||
|
||||
⭐ Star the project to follow updates, and feel free to open PRs and Issues.
|
||||
⭐ Star the project to show your support, and Watch → Custom → Releases to get notified of new versions. PRs and Issues are always welcome.
|
||||
|
||||
## 🌟 Contributors
|
||||
|
||||
@@ -229,22 +261,10 @@ Contributions are welcome — add a new channel by following the [Feishu channel
|
||||
|
||||
<br/>
|
||||
|
||||
## 🏢 Enterprise Services
|
||||
|
||||
[**LinkAI**](https://link-ai.tech/) is a one-stop AI Agent platform for enterprises and developers, offering managed hosting and enterprise-grade support for CowAgent:
|
||||
|
||||
- **🚀 Zero-deployment hosted runtime** — spin up a [CowAgent online assistant](https://link-ai.tech/cowagent/create) in under a minute, no server required
|
||||
- **🧠 Aggregated models & skill marketplace** — unified access to mainstream LLMs and an official skill marketplace, extending CowAgent's reach
|
||||
- **🏢 Team & enterprise features** — workspaces, role-based access, audit logs, and private deployment for production use cases
|
||||
|
||||
For enterprise inquiries: sales@simple-future.tech or [scan the QR code](https://cdn.link-ai.tech/consultant.jpg) to reach our team on WeChat.
|
||||
|
||||
<br/>
|
||||
|
||||
## ⚠️ Disclaimer
|
||||
|
||||
1. This project is licensed under the [MIT License](/LICENSE) and is intended for technical research and learning. You are responsible for complying with applicable laws and regulations in your jurisdiction; the maintainers assume no liability for any consequences arising from use of this project.
|
||||
2. **Cost & safety:** Agent mode consumes substantially more tokens than plain chat — pick models that balance quality and cost. The Agent has access to your local operating system; deploy only in trusted environments.
|
||||
2. **Cost & safety:** Agent mode consumes substantially more tokens than regular chat — pick models that balance quality and cost. The Agent has access to your local operating system, so only deploy it in trusted environments.
|
||||
3. CowAgent is a pure open-source project and does not participate in, authorize, or issue any cryptocurrency.
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -49,6 +49,16 @@ class ChatService:
|
||||
agent.model.channel_type = channel_type or ""
|
||||
agent.model.session_id = session_id or ""
|
||||
|
||||
# Build a context so context-aware tools (e.g. scheduler) can resolve the
|
||||
# receiver/session. This streaming path bypasses agent_bridge.agent_reply,
|
||||
# so the attach step that normally happens there must be done here too.
|
||||
context = self._build_context(query, session_id, channel_type)
|
||||
self._attach_context_aware_tools(agent, context)
|
||||
|
||||
# Mark this session as mid-run so the self-evolution idle scan does not
|
||||
# fire concurrently when a single turn runs longer than idle_minutes.
|
||||
self._mark_run_active(agent, True)
|
||||
|
||||
# State shared between the event callback and this method
|
||||
state = _StreamState()
|
||||
|
||||
@@ -171,6 +181,12 @@ class ChatService:
|
||||
|
||||
from agent.protocol.agent_stream import AgentStreamExecutor
|
||||
|
||||
# Register a cancel token so /cancel can abort this in-flight run.
|
||||
# IM channels key on session_id (no per-turn request_id here).
|
||||
from agent.protocol import get_cancel_registry
|
||||
registry = get_cancel_registry()
|
||||
cancel_event = registry.register(session_id, session_id=session_id) if session_id else None
|
||||
|
||||
executor = AgentStreamExecutor(
|
||||
agent=agent,
|
||||
model=agent.model,
|
||||
@@ -180,6 +196,7 @@ class ChatService:
|
||||
on_event=on_event,
|
||||
messages=messages_copy,
|
||||
max_context_turns=max_context_turns,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
|
||||
try:
|
||||
@@ -191,6 +208,15 @@ class ChatService:
|
||||
agent.messages.clear()
|
||||
logger.info("[ChatService] Cleared agent message history after executor recovery")
|
||||
raise
|
||||
finally:
|
||||
# Clear the mid-run flag so idle scans can review this session again.
|
||||
self._mark_run_active(agent, False)
|
||||
# Release cancel token to keep the registry bounded.
|
||||
if session_id:
|
||||
try:
|
||||
registry.unregister(session_id)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Sync executor messages back to agent (thread-safe).
|
||||
# The executor may have trimmed context, making its list shorter than
|
||||
@@ -254,10 +280,68 @@ class ChatService:
|
||||
# Execute post-process tools
|
||||
agent._execute_post_process_tools()
|
||||
|
||||
# Record this user turn for the self-evolution idle trigger. This
|
||||
# streaming path bypasses agent_bridge.agent_reply, so the activity must
|
||||
# be noted here, otherwise idle scans never see any signal to evolve.
|
||||
self._note_evolution_turn(agent, context)
|
||||
|
||||
logger.info(f"[ChatService] Agent run completed: session={session_id}")
|
||||
|
||||
|
||||
|
||||
@staticmethod
|
||||
def _build_context(query: str, session_id: str, channel_type: str):
|
||||
"""Build a Context for tool resolution on the streaming chat path.
|
||||
|
||||
receiver falls back to session_id; the scheduler's delivery keys on
|
||||
session_id as the receiver.
|
||||
"""
|
||||
from bridge.context import Context, ContextType
|
||||
# Pass an explicit kwargs dict: Context's default kwargs is a shared
|
||||
# mutable default, so omitting it would leak fields across sessions.
|
||||
ctx = Context(ContextType.TEXT, query, kwargs={})
|
||||
ctx["session_id"] = session_id
|
||||
ctx["receiver"] = session_id
|
||||
ctx["isgroup"] = False
|
||||
ctx["channel_type"] = channel_type or ""
|
||||
return ctx
|
||||
|
||||
@staticmethod
|
||||
def _attach_context_aware_tools(agent, context):
|
||||
"""Attach the current context to tools that need it (scheduler)."""
|
||||
try:
|
||||
if not (context and getattr(agent, "tools", None)):
|
||||
return
|
||||
for tool in agent.tools:
|
||||
if tool.name == "scheduler":
|
||||
from agent.tools.scheduler.integration import attach_scheduler_to_tool
|
||||
attach_scheduler_to_tool(tool, context)
|
||||
break
|
||||
except Exception as e:
|
||||
logger.warning(f"[ChatService] Failed to attach context to scheduler: {e}")
|
||||
|
||||
@staticmethod
|
||||
def _mark_run_active(agent, active):
|
||||
"""Toggle the self-evolution mid-run flag for this session's agent."""
|
||||
try:
|
||||
from agent.evolution.trigger import mark_run_active
|
||||
mark_run_active(agent, active)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _note_evolution_turn(agent, context):
|
||||
"""Record a user turn so the self-evolution idle trigger has signal."""
|
||||
try:
|
||||
from agent.evolution.trigger import note_user_turn
|
||||
ch = (context.get("channel_type") or "") if context else ""
|
||||
rcv = (context.get("receiver") or "") if context else ""
|
||||
is_group = bool(context.get("isgroup")) if context else False
|
||||
# Only single chats get a proactive push target; group push is noisy.
|
||||
note_user_turn(agent, channel_type=ch, receiver=(rcv if not is_group else ""))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _persist_messages(session_id: str, new_messages: list, channel_type: str = ""):
|
||||
try:
|
||||
|
||||
19
agent/evolution/__init__.py
Normal file
19
agent/evolution/__init__.py
Normal file
@@ -0,0 +1,19 @@
|
||||
"""
|
||||
Self-evolution subsystem for CowAgent.
|
||||
|
||||
Runs a lightweight, isolated review pass after a conversation goes idle to
|
||||
decide whether anything is worth durably learning (memory / skill) or whether
|
||||
an unfinished task can be pushed forward. Conservative by design: most
|
||||
conversations should produce no change at all.
|
||||
|
||||
Public entry points:
|
||||
from agent.evolution import get_evolution_config
|
||||
from agent.evolution.trigger import start_evolution_trigger, note_user_turn
|
||||
"""
|
||||
|
||||
from agent.evolution.config import EvolutionConfig, get_evolution_config
|
||||
|
||||
__all__ = [
|
||||
"EvolutionConfig",
|
||||
"get_evolution_config",
|
||||
]
|
||||
102
agent/evolution/backup.py
Normal file
102
agent/evolution/backup.py
Normal file
@@ -0,0 +1,102 @@
|
||||
"""File backup / rollback support for self-evolution.
|
||||
|
||||
Before the evolution agent edits MEMORY.md or a skill file, we snapshot the
|
||||
current state into ``memory/.evolution_backups/<backup_id>/`` so a later "undo"
|
||||
can restore it. File-level restore only — simple and reliable.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import time
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from common.log import logger
|
||||
|
||||
_BACKUP_DIRNAME = ".evolution_backups"
|
||||
_MANIFEST_NAME = "manifest.json"
|
||||
# Keep only the most recent N backups to bound disk usage.
|
||||
_MAX_BACKUPS = 10
|
||||
|
||||
|
||||
def _backups_root(workspace_dir: Path) -> Path:
|
||||
return Path(workspace_dir) / "memory" / _BACKUP_DIRNAME
|
||||
|
||||
|
||||
def create_backup(workspace_dir: Path, files: List[Path]) -> Optional[str]:
|
||||
"""Snapshot ``files`` (those that exist) under a new backup id.
|
||||
|
||||
Returns the backup_id, or None when there is nothing to back up.
|
||||
"""
|
||||
existing = [Path(f) for f in files if Path(f).exists()]
|
||||
if not existing:
|
||||
return None
|
||||
|
||||
backup_id = datetime.now().strftime("%Y%m%d-%H%M%S-") + str(int(time.time() * 1000) % 1000)
|
||||
root = _backups_root(workspace_dir)
|
||||
target = root / backup_id
|
||||
try:
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
ws = Path(workspace_dir)
|
||||
manifest = []
|
||||
for idx, src in enumerate(existing):
|
||||
# Store under a flat index plus the relative path so restore knows
|
||||
# where it came from, even for nested skill files.
|
||||
try:
|
||||
rel = str(src.relative_to(ws))
|
||||
except ValueError:
|
||||
rel = src.name
|
||||
dst = target / f"{idx}.bak"
|
||||
shutil.copy2(src, dst)
|
||||
manifest.append({"rel": rel, "bak": f"{idx}.bak"})
|
||||
(target / _MANIFEST_NAME).write_text(
|
||||
json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
_prune_old_backups(root)
|
||||
# Caller logs a combined backup+review line; keep this at debug.
|
||||
logger.debug(f"[Evolution] Created backup {backup_id} ({len(manifest)} file(s))")
|
||||
return backup_id
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Failed to create backup: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def restore_backup(workspace_dir: Path, backup_id: str) -> bool:
|
||||
"""Restore all files captured under ``backup_id``. Returns success."""
|
||||
if not backup_id:
|
||||
return False
|
||||
target = _backups_root(workspace_dir) / backup_id
|
||||
manifest_path = target / _MANIFEST_NAME
|
||||
if not manifest_path.exists():
|
||||
logger.warning(f"[Evolution] Backup not found: {backup_id}")
|
||||
return False
|
||||
try:
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
ws = Path(workspace_dir)
|
||||
for entry in manifest:
|
||||
bak = target / entry["bak"]
|
||||
dst = ws / entry["rel"]
|
||||
if bak.exists():
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copy2(bak, dst)
|
||||
logger.info(f"[Evolution] Restored backup {backup_id} ({len(manifest)} file(s))")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Failed to restore backup {backup_id}: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def _prune_old_backups(root: Path) -> None:
|
||||
"""Drop the oldest backups beyond _MAX_BACKUPS (sorted by name = chronological)."""
|
||||
try:
|
||||
dirs = sorted(
|
||||
[d for d in root.iterdir() if d.is_dir()],
|
||||
key=lambda p: p.name,
|
||||
)
|
||||
for old in dirs[:-_MAX_BACKUPS]:
|
||||
shutil.rmtree(old, ignore_errors=True)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Evolution] Backup prune skipped: {e}")
|
||||
76
agent/evolution/config.py
Normal file
76
agent/evolution/config.py
Normal file
@@ -0,0 +1,76 @@
|
||||
"""Configuration for the self-evolution subsystem.
|
||||
|
||||
Reads flat ``self_evolution_*`` keys from config.json. All fields have safe
|
||||
defaults so the feature degrades gracefully when keys are absent.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
# Defaults — conservative (see executor module docstring). Disabled by default
|
||||
# until release; enable via ``self_evolution_enabled``.
|
||||
DEFAULT_ENABLED = False
|
||||
DEFAULT_IDLE_MINUTES = 10
|
||||
DEFAULT_MIN_TURNS = 6
|
||||
# Max review steps for the isolated evolution agent. Kept small (not exposed as
|
||||
# config): the review is meant to be cheap and focused, not a long autonomous run.
|
||||
DEFAULT_MAX_STEPS = 12
|
||||
|
||||
|
||||
@dataclass
|
||||
class EvolutionConfig:
|
||||
"""Resolved self-evolution settings."""
|
||||
|
||||
enabled: bool = DEFAULT_ENABLED
|
||||
idle_minutes: int = DEFAULT_IDLE_MINUTES
|
||||
min_turns: int = DEFAULT_MIN_TURNS
|
||||
max_steps: int = DEFAULT_MAX_STEPS
|
||||
|
||||
@property
|
||||
def idle_seconds(self) -> int:
|
||||
return max(60, self.idle_minutes * 60)
|
||||
|
||||
|
||||
def _as_bool(value: Any, fallback: bool) -> bool:
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, str):
|
||||
v = value.strip().lower()
|
||||
if v in ("true", "1", "yes", "on"):
|
||||
return True
|
||||
if v in ("false", "0", "no", "off"):
|
||||
return False
|
||||
return fallback
|
||||
|
||||
|
||||
def _as_pos_int(value: Any, fallback: int) -> int:
|
||||
try:
|
||||
n = int(value)
|
||||
return n if n > 0 else fallback
|
||||
except (TypeError, ValueError):
|
||||
return fallback
|
||||
|
||||
|
||||
def get_evolution_config() -> EvolutionConfig:
|
||||
"""Build EvolutionConfig from the live config.json ``self_evolution_*`` keys."""
|
||||
try:
|
||||
from config import conf
|
||||
c = conf()
|
||||
except Exception:
|
||||
c = {}
|
||||
|
||||
def _get(key, default):
|
||||
try:
|
||||
return c.get(key, default)
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
return EvolutionConfig(
|
||||
enabled=_as_bool(_get("self_evolution_enabled", None), DEFAULT_ENABLED),
|
||||
idle_minutes=_as_pos_int(_get("self_evolution_idle_minutes", None), DEFAULT_IDLE_MINUTES),
|
||||
min_turns=_as_pos_int(_get("self_evolution_min_turns", None), DEFAULT_MIN_TURNS),
|
||||
max_steps=DEFAULT_MAX_STEPS,
|
||||
)
|
||||
551
agent/evolution/executor.py
Normal file
551
agent/evolution/executor.py
Normal file
@@ -0,0 +1,551 @@
|
||||
"""Self-evolution executor.
|
||||
|
||||
Runs an isolated review agent over an idle conversation's transcript and, if a
|
||||
clear signal is found, lets it edit memory / skills via a restricted toolset.
|
||||
Conservative by design: most runs return ``[SILENT]`` and change nothing.
|
||||
|
||||
Flow:
|
||||
1. Build a transcript from the session's new (since last pass) messages.
|
||||
2. Snapshot MEMORY.md + daily file + editable skills (for undo) -> backup_id.
|
||||
3. Run an isolated agent (same model, restricted tools, evolution prompt).
|
||||
4. If output is [SILENT], or no workspace file actually changed -> done.
|
||||
5. Otherwise -> record to the evolution log, inject an [EVOLUTION] note into
|
||||
the user session (so the main agent can honor "undo"), and push the
|
||||
summary to the user's channel.
|
||||
|
||||
Reuses existing infrastructure (AgentBridge.create_agent, ToolManager,
|
||||
remember_scheduled_output, channel_factory) rather than introducing a fork.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import threading
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from common.log import logger
|
||||
|
||||
from agent.evolution.backup import create_backup
|
||||
from agent.evolution.config import get_evolution_config
|
||||
from agent.evolution.prompts import (
|
||||
EVOLUTION_MARKER,
|
||||
EVOLUTION_SYSTEM_PROMPT,
|
||||
SILENT_TOKEN,
|
||||
build_review_user_message,
|
||||
)
|
||||
from agent.evolution.record import append_session_evolution
|
||||
|
||||
# Tools the isolated evolution agent is allowed to use. Everything else is
|
||||
# withheld so a review pass can only read context, run workspace scripts, and
|
||||
# edit memory/skill files. bash is needed by skill-creator's init script and is
|
||||
# confined to the workspace by _BashWorkspaceGuard.
|
||||
_ALLOWED_TOOLS = {"read", "write", "edit", "ls", "bash", "memory_search", "memory_get"}
|
||||
|
||||
# Cap concurrent evolution passes so a burst of idle sessions can't spawn many
|
||||
# background model runs at once. Extra sessions simply wait for the next scan.
|
||||
_MAX_CONCURRENT = 2
|
||||
_running_lock = threading.Lock()
|
||||
_running_count = 0
|
||||
|
||||
|
||||
def _builtin_skill_names() -> set:
|
||||
"""Names of skills shipped with the product (project-root ``skills/``).
|
||||
|
||||
These are protected: the evolution agent must never edit them, even though
|
||||
a same-named copy exists in the workspace at runtime. The project dir is the
|
||||
authoritative list of what counts as built-in.
|
||||
"""
|
||||
try:
|
||||
# executor.py -> agent/evolution -> agent -> project root
|
||||
project_root = Path(__file__).resolve().parents[2]
|
||||
builtin_dir = project_root / "skills"
|
||||
if not builtin_dir.is_dir():
|
||||
return set()
|
||||
names = set()
|
||||
for entry in builtin_dir.iterdir():
|
||||
if entry.is_dir() and not entry.name.startswith("."):
|
||||
names.add(entry.name)
|
||||
return names
|
||||
except Exception:
|
||||
return set()
|
||||
|
||||
|
||||
def _build_transcript(messages: List[dict], max_chars: int = 12000) -> str:
|
||||
"""Render the session messages into a compact text transcript."""
|
||||
lines: List[str] = []
|
||||
for msg in messages:
|
||||
role = msg.get("role", "")
|
||||
if role not in ("user", "assistant"):
|
||||
continue
|
||||
content = msg.get("content", "")
|
||||
text = _extract_text(content)
|
||||
if not text.strip():
|
||||
continue
|
||||
speaker = "User" if role == "user" else "Assistant"
|
||||
lines.append(f"{speaker}: {text.strip()}")
|
||||
transcript = "\n".join(lines)
|
||||
# Keep the most RECENT context if oversized (tail is most relevant).
|
||||
if len(transcript) > max_chars:
|
||||
transcript = "...(earlier omitted)...\n" + transcript[-max_chars:]
|
||||
return transcript
|
||||
|
||||
|
||||
def _extract_text(content) -> str:
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for block in content:
|
||||
if isinstance(block, dict) and block.get("type") == "text":
|
||||
parts.append(block.get("text", ""))
|
||||
elif isinstance(block, str):
|
||||
parts.append(block)
|
||||
return "\n".join(parts)
|
||||
return ""
|
||||
|
||||
|
||||
def _select_tools(all_tools: list) -> list:
|
||||
return [t for t in all_tools if getattr(t, "name", None) in _ALLOWED_TOOLS]
|
||||
|
||||
|
||||
# Tools whose writes must be confined to the workspace during evolution.
|
||||
_WRITE_TOOLS = {"write", "edit"}
|
||||
|
||||
|
||||
class _WorkspaceWriteGuard:
|
||||
"""Wraps a write/edit tool so it can ONLY write inside the workspace.
|
||||
|
||||
Hard engineering guard (not prompt-based): any write resolving outside the
|
||||
workspace — e.g. the project's bundled ``skills/`` dir — is rejected. This
|
||||
protects built-in skills regardless of what the model attempts.
|
||||
"""
|
||||
|
||||
def __init__(self, inner, workspace_dir: str):
|
||||
self._inner = inner
|
||||
self._ws = Path(workspace_dir).resolve()
|
||||
# Mirror the attributes the agent runtime reads off a tool.
|
||||
self.name = inner.name
|
||||
self.description = inner.description
|
||||
self.params = inner.params
|
||||
|
||||
def __getattr__(self, item):
|
||||
return getattr(self._inner, item)
|
||||
|
||||
def execute_tool(self, params):
|
||||
# The agent runtime calls execute_tool (not execute); route it through
|
||||
# our guarded execute so the path checks always run.
|
||||
try:
|
||||
return self.execute(params)
|
||||
except Exception as e:
|
||||
logger.error(f"[Evolution] guarded tool error: {e}")
|
||||
from agent.tools.base_tool import ToolResult
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
|
||||
def execute(self, args):
|
||||
path = (args.get("path") or "").strip()
|
||||
if path:
|
||||
try:
|
||||
resolved = Path(self._inner._resolve_path(path)).resolve()
|
||||
from agent.tools.base_tool import ToolResult
|
||||
# Confine writes to the workspace. This protects the product's
|
||||
# bundled skills (which live outside the workspace) from ever
|
||||
# being modified, no matter what path the model attempts.
|
||||
if self._ws not in resolved.parents and resolved != self._ws:
|
||||
return ToolResult.fail(
|
||||
"Error: evolution may only write inside the workspace; "
|
||||
f"path '{path}' is outside and was blocked."
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
return self._inner.execute(args)
|
||||
|
||||
|
||||
class _BashWorkspaceGuard:
|
||||
"""Wraps the bash tool so evolution can only run commands inside the
|
||||
workspace.
|
||||
|
||||
Evolution needs bash for skill-creator's init script, but it runs
|
||||
unattended in the background, so a raw shell is too broad. This guard:
|
||||
- forces the command to execute with cwd = workspace,
|
||||
- rejects commands that reference an absolute path or ``..`` segment
|
||||
pointing OUTSIDE the workspace (the common ways to escape it).
|
||||
It is a coarse textual check, not a sandbox — paired with the model's
|
||||
instruction to only run skill-creator scripts, it keeps writes local.
|
||||
"""
|
||||
|
||||
def __init__(self, inner, workspace_dir: str):
|
||||
self._inner = inner
|
||||
self._ws = Path(workspace_dir).resolve()
|
||||
# Pin the shell's working directory to the workspace.
|
||||
try:
|
||||
self._inner.cwd = str(self._ws)
|
||||
except Exception:
|
||||
pass
|
||||
self.name = inner.name
|
||||
self.description = inner.description
|
||||
self.params = inner.params
|
||||
|
||||
def __getattr__(self, item):
|
||||
return getattr(self._inner, item)
|
||||
|
||||
def execute_tool(self, params):
|
||||
try:
|
||||
return self.execute(params)
|
||||
except Exception as e:
|
||||
logger.error(f"[Evolution] guarded bash error: {e}")
|
||||
from agent.tools.base_tool import ToolResult
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
|
||||
def _escapes_workspace(self, command: str) -> bool:
|
||||
# Absolute paths that are not under the workspace.
|
||||
for tok in re.findall(r'(?:^|\s)(/[^\s\'";|&]+)', command):
|
||||
try:
|
||||
resolved = Path(tok).resolve()
|
||||
except Exception:
|
||||
continue
|
||||
if self._ws != resolved and self._ws not in resolved.parents:
|
||||
return True
|
||||
# Parent-dir traversal that climbs above the workspace.
|
||||
for tok in re.findall(r'[^\s\'";|&]*\.\.[^\s\'";|&]*', command):
|
||||
try:
|
||||
resolved = (self._ws / tok).resolve()
|
||||
except Exception:
|
||||
continue
|
||||
if self._ws != resolved and self._ws not in resolved.parents:
|
||||
return True
|
||||
return False
|
||||
|
||||
def execute(self, args):
|
||||
from agent.tools.base_tool import ToolResult
|
||||
command = (args.get("command") or "").strip()
|
||||
if command and self._escapes_workspace(command):
|
||||
return ToolResult.fail(
|
||||
"Error: evolution may only run commands inside the workspace; "
|
||||
"this command references a path outside it and was blocked."
|
||||
)
|
||||
return self._inner.execute(args)
|
||||
|
||||
|
||||
def _guard_tools(tools: list, workspace_dir: str) -> list:
|
||||
"""Wrap write/edit/bash tools with workspace guards; leave others as-is."""
|
||||
guarded = []
|
||||
for t in tools:
|
||||
name = getattr(t, "name", None)
|
||||
if name in _WRITE_TOOLS:
|
||||
guarded.append(_WorkspaceWriteGuard(t, workspace_dir))
|
||||
elif name == "bash":
|
||||
guarded.append(_BashWorkspaceGuard(t, workspace_dir))
|
||||
else:
|
||||
guarded.append(t)
|
||||
return guarded
|
||||
|
||||
|
||||
# Workspace subtrees worth watching for evolution-induced changes. AGENT.md is
|
||||
# watched too: evolution may rarely refine the assistant's persona/style there.
|
||||
_WATCH_SUBDIRS = ("MEMORY.md", "AGENT.md", "skills", "knowledge", "output")
|
||||
# Subpaths under memory/ to ignore: evolution's own bookkeeping + the nightly
|
||||
# dream diary, none of which count as a user-facing change signal.
|
||||
_MEMORY_IGNORE = (".evolution_backups", "dreams", "evolution")
|
||||
# Files the skill subsystem maintains automatically (the enable/disable index).
|
||||
# Not an evolution result, so a rewrite must not count as a change signal.
|
||||
_WATCH_IGNORE_NAMES = ("skills_config.json",)
|
||||
|
||||
|
||||
def _workspace_snapshot(workspace_dir) -> dict:
|
||||
"""Map relative path -> (mtime, size) for watched files. Cheap, no reads."""
|
||||
ws = Path(workspace_dir)
|
||||
snap: dict = {}
|
||||
for name in _WATCH_SUBDIRS:
|
||||
root = ws / name
|
||||
if root.is_file():
|
||||
try:
|
||||
st = root.stat()
|
||||
snap[name] = (st.st_mtime, st.st_size)
|
||||
except OSError:
|
||||
pass
|
||||
continue
|
||||
if not root.is_dir():
|
||||
continue
|
||||
for p in root.rglob("*"):
|
||||
if not p.is_file():
|
||||
continue
|
||||
if p.name in _WATCH_IGNORE_NAMES:
|
||||
continue
|
||||
try:
|
||||
st = p.stat()
|
||||
snap[str(p.relative_to(ws))] = (st.st_mtime, st.st_size)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# Watch the daily memory files (memory/*.md and per-user dailies) since
|
||||
# evolution now records learnings there. Skip backups/dreams bookkeeping.
|
||||
mem_dir = ws / "memory"
|
||||
if mem_dir.is_dir():
|
||||
for p in mem_dir.rglob("*.md"):
|
||||
rel_parts = p.relative_to(mem_dir).parts
|
||||
if rel_parts and rel_parts[0] in _MEMORY_IGNORE:
|
||||
continue
|
||||
try:
|
||||
st = p.stat()
|
||||
snap[str(p.relative_to(ws))] = (st.st_mtime, st.st_size)
|
||||
except OSError:
|
||||
pass
|
||||
return snap
|
||||
|
||||
|
||||
def _workspace_changed(workspace_dir, pre: dict) -> bool:
|
||||
"""True if any watched file was added, removed, or modified since ``pre``."""
|
||||
return _workspace_snapshot(workspace_dir) != pre
|
||||
|
||||
|
||||
def run_evolution_for_session(
|
||||
agent_bridge,
|
||||
session_id: str,
|
||||
channel_type: str = "",
|
||||
receiver: str = "",
|
||||
user_id: Optional[str] = None,
|
||||
idle_minutes: float = 0.0,
|
||||
) -> bool:
|
||||
"""Run one evolution pass for a session. Returns True if it changed anything.
|
||||
|
||||
Safe to call from a background thread. All failures are swallowed and
|
||||
logged — evolution must never disrupt the main pipeline.
|
||||
"""
|
||||
cfg = get_evolution_config()
|
||||
if not cfg.enabled:
|
||||
return False
|
||||
|
||||
# Concurrency gate: bound how many evolution passes run at once.
|
||||
global _running_count
|
||||
with _running_lock:
|
||||
if _running_count >= _MAX_CONCURRENT:
|
||||
logger.info(
|
||||
f"[Evolution] busy ({_running_count}/{_MAX_CONCURRENT} running); "
|
||||
f"skipping session={session_id} this scan"
|
||||
)
|
||||
return False
|
||||
_running_count += 1
|
||||
|
||||
try:
|
||||
agent = agent_bridge.agents.get(session_id) or agent_bridge.default_agent
|
||||
if not agent:
|
||||
return False
|
||||
|
||||
with agent.messages_lock:
|
||||
all_messages = list(agent.messages)
|
||||
total_msgs = len(all_messages)
|
||||
# In-memory evolution cursor: only review messages added since the last
|
||||
# pass so a long session doesn't re-judge (and re-write) old content.
|
||||
# Stored on the agent instance; lost on restart (acceptable — at worst
|
||||
# one redundant pass right after a restart, gated by the file-change
|
||||
# check downstream so it won't double-write identical memory).
|
||||
done = int(getattr(agent, "_evo_done_msg_count", 0))
|
||||
if done > total_msgs:
|
||||
done = 0 # history was trimmed/reset; start fresh
|
||||
new_messages = all_messages[done:]
|
||||
transcript = _build_transcript(new_messages)
|
||||
if not transcript.strip():
|
||||
# Routine no-op: the per-minute scan hits every idle session. Advance
|
||||
# the cursor so we don't re-scan the same tail; no log (pure noise).
|
||||
agent._evo_done_msg_count = total_msgs
|
||||
return False
|
||||
|
||||
logger.info(
|
||||
f"[Evolution] ▶ Reviewing session={session_id} "
|
||||
f"(idle {idle_minutes:.1f}min, {len(new_messages)} new/{total_msgs} msgs, "
|
||||
f"~{len(transcript)} chars)"
|
||||
)
|
||||
|
||||
# Resolve workspace + files to snapshot for undo.
|
||||
from agent.memory.config import get_default_memory_config
|
||||
mem_cfg = get_default_memory_config()
|
||||
workspace_dir = mem_cfg.get_workspace()
|
||||
if user_id:
|
||||
memory_file = Path(workspace_dir) / "memory" / "users" / user_id / "MEMORY.md"
|
||||
else:
|
||||
memory_file = Path(workspace_dir) / "MEMORY.md"
|
||||
skills_dir = mem_cfg.get_skills_dir()
|
||||
|
||||
# Snapshot MEMORY.md + every NON-protected skill's SKILL.md. Protected
|
||||
# built-in skills are excluded from backup because they must never be
|
||||
# edited in the first place.
|
||||
protected_names = _builtin_skill_names()
|
||||
# Back up both MEMORY.md and today's daily file: evolution now writes to
|
||||
# the daily file, but MEMORY.md is cheap to snapshot and keeps undo safe
|
||||
# if the model ever edits it.
|
||||
today_daily = Path(workspace_dir) / "memory" / (
|
||||
datetime.now().strftime("%Y-%m-%d") + ".md"
|
||||
)
|
||||
if user_id:
|
||||
today_daily = Path(workspace_dir) / "memory" / "users" / user_id / (
|
||||
datetime.now().strftime("%Y-%m-%d") + ".md"
|
||||
)
|
||||
# AGENT.md (persona) is backed up too so a rare persona edit is undoable.
|
||||
# Persona is workspace-global (not per-user): it always lives at the
|
||||
# workspace root, regardless of user_id.
|
||||
agent_file = Path(workspace_dir) / "AGENT.md"
|
||||
backup_files = [Path(memory_file), today_daily, agent_file]
|
||||
if skills_dir.exists():
|
||||
for skill_md in skills_dir.rglob("SKILL.md"):
|
||||
# The skill dir is the SKILL.md's parent (or an ancestor for
|
||||
# collections); guard by checking the immediate top-level dir.
|
||||
try:
|
||||
top = skill_md.relative_to(skills_dir).parts[0]
|
||||
except (ValueError, IndexError):
|
||||
continue
|
||||
if top in protected_names:
|
||||
continue
|
||||
backup_files.append(skill_md)
|
||||
backup_id = create_backup(workspace_dir, backup_files)
|
||||
_backup_n = sum(1 for f in backup_files if Path(f).exists())
|
||||
|
||||
# Snapshot the whole workspace (path -> mtime/size) so we can reliably
|
||||
# detect ANY file change — including new output files written when
|
||||
# finishing an unfinished task, which are not in backup_files.
|
||||
pre_snapshot = _workspace_snapshot(workspace_dir)
|
||||
|
||||
# Build the isolated review agent: same model, restricted tools, with a
|
||||
# hard guard that confines all writes to the workspace (protects the
|
||||
# project's bundled skills from ever being modified).
|
||||
review_tools = _guard_tools(
|
||||
_select_tools(list(getattr(agent, "tools", []) or [])),
|
||||
str(workspace_dir),
|
||||
)
|
||||
review_agent = agent_bridge.create_agent(
|
||||
system_prompt="",
|
||||
tools=review_tools,
|
||||
description="Self-evolution review agent",
|
||||
max_steps=cfg.max_steps,
|
||||
workspace_dir=str(workspace_dir),
|
||||
skill_manager=getattr(agent, "skill_manager", None),
|
||||
memory_manager=getattr(agent, "memory_manager", None),
|
||||
enable_skills=True,
|
||||
runtime_info=getattr(agent, "runtime_info", None),
|
||||
)
|
||||
# Reuse the live model so it follows the user's configured model.
|
||||
review_agent.model = agent.model
|
||||
# Inject the evolution task brief AFTER the full system prompt: the agent
|
||||
# gets the full context (tools, workspace, user preferences, memory, time)
|
||||
# AND its evolution-specific instructions on top, instead of one
|
||||
# overwriting the other.
|
||||
review_agent.extra_system_suffix = EVOLUTION_SYSTEM_PROMPT
|
||||
|
||||
logger.info(
|
||||
f"[Evolution] backup {backup_id} ({_backup_n} files) → running review agent"
|
||||
)
|
||||
user_msg = build_review_user_message(transcript, protected_skills=list(protected_names))
|
||||
result = review_agent.run_stream(user_msg, clear_history=True)
|
||||
result = (result or "").strip()
|
||||
|
||||
# These messages are now reviewed; advance the cursor so the next pass
|
||||
# only looks at messages added after this point (silent or not).
|
||||
agent._evo_done_msg_count = total_msgs
|
||||
|
||||
# Respect an explicit silent verdict: empty, exactly [SILENT], or text
|
||||
# that STARTS with [SILENT] means the model chose to stay quiet.
|
||||
if not result or result.startswith(SILENT_TOKEN):
|
||||
logger.info(f"[Evolution] ✗ No change for session={session_id} ([SILENT])")
|
||||
return False
|
||||
|
||||
# Anti-nag backstop: if the model wrote a summary but actually changed no
|
||||
# watched file, stay silent — never notify about work that didn't happen.
|
||||
if not _workspace_changed(workspace_dir, pre_snapshot):
|
||||
logger.info(
|
||||
f"[Evolution] ✗ session={session_id}: text produced but no file "
|
||||
f"changed — staying silent"
|
||||
)
|
||||
return False
|
||||
|
||||
# The model produced a real summary. Strip any stray [SILENT] tokens it
|
||||
# left mid-text, then notify.
|
||||
result = result.replace(SILENT_TOKEN, "").strip()
|
||||
if not result:
|
||||
logger.info(f"[Evolution] ✗ No change for session={session_id} ([SILENT])")
|
||||
return False
|
||||
|
||||
logger.info(f"[Evolution] ✓ session={session_id} evolved:\n{result}")
|
||||
append_session_evolution(workspace_dir, result, backup_id=backup_id, user_id=user_id)
|
||||
# Inject an [EVOLUTION] note so the main agent can honor "undo".
|
||||
_inject_evolution_record(agent_bridge, session_id, channel_type, result, backup_id)
|
||||
# The injection appended its own messages ([SCHEDULED]/[EVOLUTION]).
|
||||
# Advance the cursor past them so the next scan does not treat
|
||||
# evolution's own bookkeeping as new user content and re-trigger.
|
||||
try:
|
||||
with agent.messages_lock:
|
||||
agent._evo_done_msg_count = len(agent.messages)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Push the summary to the user's channel. The "did a file actually
|
||||
# change" gate above is the only throttle we need: real evolutions are
|
||||
# rare, so no extra opt-in switch or daily-count limit is required.
|
||||
if channel_type and receiver:
|
||||
_notify_user(channel_type, receiver, result)
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Run failed for session={session_id}: {e}")
|
||||
return False
|
||||
finally:
|
||||
with _running_lock:
|
||||
_running_count -= 1
|
||||
|
||||
|
||||
def _inject_evolution_record(
|
||||
agent_bridge, session_id: str, channel_type: str, summary: str, backup_id: Optional[str]
|
||||
) -> None:
|
||||
"""Add an [EVOLUTION] note to the user session so the main agent can undo."""
|
||||
try:
|
||||
note = f"{EVOLUTION_MARKER} {summary}"
|
||||
if backup_id:
|
||||
note += f"\n(backup_id: {backup_id}; to undo, restore this backup)"
|
||||
# Reuse the scheduler-output injection path: isolated execution, only a
|
||||
# compact record lands in the user session.
|
||||
agent_bridge.remember_scheduled_output(
|
||||
session_id=session_id,
|
||||
content=note,
|
||||
channel_type=channel_type,
|
||||
task_description="self-evolution",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Evolution] Failed to inject evolution record: {e}")
|
||||
|
||||
|
||||
def _notify_user(channel_type: str, receiver: str, summary: str) -> None:
|
||||
"""Push the evolution summary to the user's channel as a new message."""
|
||||
try:
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.channel_factory import create_channel
|
||||
|
||||
context = Context(ContextType.TEXT, summary)
|
||||
context["receiver"] = receiver
|
||||
context["isgroup"] = False
|
||||
context["session_id"] = receiver
|
||||
# Channels that reply to an original message need msg=None for a fresh push.
|
||||
if channel_type in ("feishu", "dingtalk", "wecom_bot", "qq"):
|
||||
context["msg"] = None
|
||||
if channel_type == "feishu":
|
||||
context["receive_id_type"] = "open_id"
|
||||
|
||||
channel = create_channel(channel_type)
|
||||
if not channel:
|
||||
return
|
||||
|
||||
# Web is request-response: a background push needs a synthetic request_id
|
||||
# plus a request->session mapping so the channel can route the message to
|
||||
# the user's polling queue (same approach the scheduler uses).
|
||||
if channel_type == "web":
|
||||
import uuid
|
||||
request_id = f"evolution_{uuid.uuid4().hex[:8]}"
|
||||
context["request_id"] = request_id
|
||||
if hasattr(channel, "request_to_session"):
|
||||
channel.request_to_session[request_id] = receiver
|
||||
|
||||
channel.send(Reply(ReplyType.TEXT, summary), context)
|
||||
logger.info(f"[Evolution] Notified user via {channel_type}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Failed to notify user: {e}")
|
||||
170
agent/evolution/prompts.py
Normal file
170
agent/evolution/prompts.py
Normal file
@@ -0,0 +1,170 @@
|
||||
"""Prompts for the self-evolution review agent.
|
||||
|
||||
The system prompt is intentionally English-only: it governs the agent's
|
||||
internal reasoning and is more stable / cheaper to maintain in one language.
|
||||
The user-facing summary the agent produces should follow the user's own
|
||||
language (instructed at the end of the prompt).
|
||||
|
||||
Design goals (see ref/hermes-agent background_review for inspiration):
|
||||
- Default to doing NOTHING. Evolution is the exception, not the rule.
|
||||
- Signal types: skill, unfinished task, memory, knowledge.
|
||||
- An explicit "do NOT capture" list to avoid self-poisoning over time.
|
||||
- Generic examples only — never bake in domain-specific business terms.
|
||||
"""
|
||||
|
||||
# Sentinel the agent emits when there is nothing worth evolving.
|
||||
SILENT_TOKEN = "[SILENT]"
|
||||
|
||||
# Marker prefix for the evolution record injected into the user session, so the
|
||||
# main chat agent can recognize past evolutions and honor an "undo" request.
|
||||
EVOLUTION_MARKER = "[EVOLUTION]"
|
||||
|
||||
|
||||
EVOLUTION_SYSTEM_PROMPT = """You are a self-evolution review agent for an AI assistant.
|
||||
|
||||
You are given a transcript of a conversation that just went idle. Your job is to
|
||||
decide whether anything from it is worth durably learning so future
|
||||
conversations go better — and if so, to make that change.
|
||||
|
||||
# Top principle: default to doing NOTHING
|
||||
|
||||
Most ordinary conversations need no evolution. Only act when there is a CLEAR
|
||||
signal below. If there is none, reply with exactly `[SILENT]` and stop. Staying
|
||||
silent is the normal, correct outcome — not a failure.
|
||||
|
||||
Greetings, small talk, acknowledgements ("ok", "thanks", "got it"), and casual
|
||||
chat are NOT signals. For these, output exactly `[SILENT]` immediately — do not
|
||||
explore files, do not write a summary, do not be polite. Just `[SILENT]`.
|
||||
|
||||
IMPORTANT: A summary is only allowed if you ACTUALLY made a file change via a
|
||||
tool (write/edit) in this pass. If you did not change any file, you MUST output
|
||||
exactly `[SILENT]` — never describe a change you only intended to make.
|
||||
|
||||
# Signals worth acting on (act only if at least one clearly appears)
|
||||
|
||||
SKILL and UNFINISHED TASK are your PRIMARY value — no other mechanism handles
|
||||
them. When their signal is clear, act; do not be shy here.
|
||||
|
||||
1. SKILL — two cases:
|
||||
a) PATCH an existing skill: a skill used here showed a STRUCTURAL problem (a
|
||||
missing step/section, a wrong or outdated detail, an error in its
|
||||
content), or its OUTPUT repeatedly misses something the user flagged. Read
|
||||
the relevant skill file under the skills directory and make a small
|
||||
incremental edit so it never recurs.
|
||||
b) CREATE a new skill: a clearly reusable, repeatable workflow emerged that
|
||||
no existing skill covers and the user is likely to want again. Follow the
|
||||
`skill-creator` skill's conventions (read its SKILL.md for the required
|
||||
structure), then create `skills/<name>/SKILL.md` by WRITING the file
|
||||
directly with the write tool — this is the simplest reliable path. (bash
|
||||
is available and confined to the workspace if a helper script is truly
|
||||
needed, but a direct write is preferred.) Only create when the workflow is
|
||||
genuinely reusable — not for a one-off task.
|
||||
|
||||
CRITICAL — fix the SOURCE, do not just remember the symptom: when the root
|
||||
cause of a problem lives IN a skill file itself (its instructions, content,
|
||||
or configuration are wrong/outdated), the correct action is to EDIT that
|
||||
skill so the problem cannot recur. Recording the corrected fact in memory
|
||||
does NOT prevent recurrence — only fixing the skill does. Never log "skill X
|
||||
has wrong detail Y" as a memory note in place of editing skill X.
|
||||
|
||||
2. UNFINISHED TASK — a specific deliverable you promised but didn't produce,
|
||||
AND you already have everything needed to finish it. DO IT now with the
|
||||
available tools and produce the result (e.g. write the file you said you'd
|
||||
write). If key info is missing, or the task is merely waiting on the user's
|
||||
reply/decision, do NOTHING and stay [SILENT] — do not nag or ping the user.
|
||||
You only ever notify the user as a side effect of having actually done work.
|
||||
|
||||
3. MEMORY — RARE, last resort. Default to writing NOTHING here. The main
|
||||
assistant already writes memory during the chat, and a nightly pass plus
|
||||
context-overflow saves are dedicated safety nets — so memory is almost always
|
||||
already covered without you. Skip unless the main assistant clearly missed a
|
||||
durable fact that belongs in no skill AND would visibly change future replies.
|
||||
- MEMORY.md is the curated long-term index, auto-loaded into EVERY future
|
||||
conversation. Treat it as precious: edit it in place to CORRECT a wrong
|
||||
fact, or append a new durable preference/decision/lesson — but do so
|
||||
SPARINGLY (a lasting fact, not a passing detail; the nightly pass handles
|
||||
routine consolidation).
|
||||
- For a NEW fact that is important but not yet clearly lasting, append ONE
|
||||
short bullet to today's `memory/YYYY-MM-DD.md` instead. When unsure, the
|
||||
daily file is the safe place — but first ask whether this really belongs
|
||||
in a skill.
|
||||
- PERSONA (AGENT.md) — EXTREMELY rare: only on an explicit, repeated signal
|
||||
about the assistant's own identity/personality/style, make a small edit to
|
||||
AGENT.md; never for user/world facts, and when in doubt do nothing.
|
||||
- Keep it to ONE short bullet. Never write paragraphs, never re-summarize the
|
||||
conversation, never copy what the main assistant already recorded.
|
||||
- If it is already captured anywhere (check MEMORY.md AND the daily file
|
||||
first), do NOTHING.
|
||||
|
||||
4. KNOWLEDGE — only if the conversation produced durable, reusable reference
|
||||
knowledge on a topic (the kind worth looking up again) that the main
|
||||
assistant did NOT already save to `knowledge/`. Add or update the relevant
|
||||
file there. Like memory, this is the exception: skip routine Q&A, and if the
|
||||
topic is already covered in `knowledge/`, do NOTHING rather than duplicate.
|
||||
|
||||
# Do NOT capture (these poison future behavior)
|
||||
|
||||
- Environment failures: missing binaries, unset credentials, uninstalled
|
||||
packages, "command not found". The user can fix these; they are not durable
|
||||
rules.
|
||||
- Negative claims about tools or features ("tool X does not work"). These
|
||||
harden into refusals the agent cites against itself later.
|
||||
- One-off task narratives (e.g. summarizing today's content). Not a class of
|
||||
reusable work.
|
||||
- Transient errors that resolved on retry within the conversation.
|
||||
|
||||
# Execution constraints
|
||||
|
||||
- Before changing memory or a skill, READ the current content first and make a
|
||||
small INCREMENTAL edit. Never fabricate, never rewrite large sections.
|
||||
- AVOID DUPLICATES. Before writing memory, READ both MEMORY.md AND today's
|
||||
daily file `memory/YYYY-MM-DD.md`. If the fact/preference is already recorded
|
||||
in EITHER (even if worded differently), do NOT add it again. The main
|
||||
assistant likely already wrote it during the chat — only add what is
|
||||
genuinely new or a correction not yet reflected anywhere.
|
||||
- You may only edit files inside the workspace. Built-in skills shipped with
|
||||
the product live outside it and are write-protected; do not try to edit them.
|
||||
- Make at most the few edits the signals justify; do not go looking for work.
|
||||
|
||||
# Output
|
||||
|
||||
- Nothing worth evolving -> output exactly `[SILENT]` and nothing else.
|
||||
- Otherwise, after performing the edits, output a short user-facing summary in
|
||||
the SAME LANGUAGE the user speaks in the conversation transcript. Write it for an ordinary user, in plain
|
||||
everyday words — NOT a developer report. No need to expose internal details
|
||||
(file names/paths, system mechanics, etc.). Briefly speak directly TO the user, telling them that you just did a self-learning pass,
|
||||
what you learned, and what you changed in THIS pass. Keep it clear and focused on the key changes (a few lines), and let
|
||||
the user know they can undo it.
|
||||
"""
|
||||
|
||||
|
||||
def build_review_user_message(transcript: str, protected_skills: list = None) -> str:
|
||||
"""Wrap the conversation transcript as the review agent's user message.
|
||||
|
||||
``protected_skills`` lists skill names that must never be edited (built-in
|
||||
skills shipped with the product). Surfaced so the agent avoids them.
|
||||
"""
|
||||
protected_note = ""
|
||||
if protected_skills:
|
||||
names = ", ".join(sorted(protected_skills))
|
||||
protected_note = (
|
||||
"\n\nPROTECTED skills (built-in — never edit these): "
|
||||
f"{names}\n"
|
||||
)
|
||||
try:
|
||||
from common import i18n
|
||||
lang_name = "中文" if i18n.is_zh() else "English"
|
||||
except Exception:
|
||||
lang_name = "中文"
|
||||
return (
|
||||
"Here is the conversation transcript that just went idle. Review it per "
|
||||
"your instructions. Acting is the exception: the main value is fixing or "
|
||||
"creating a skill and finishing promised work. Memory and knowledge are "
|
||||
"rare last resorts — stay [SILENT] unless there is a clear, durable signal "
|
||||
"not already covered."
|
||||
f"{protected_note}\n"
|
||||
f"The summary should preferably be written in: {lang_name}\n"
|
||||
"<transcript>\n"
|
||||
f"{transcript}\n"
|
||||
"</transcript>"
|
||||
)
|
||||
55
agent/evolution/record.py
Normal file
55
agent/evolution/record.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""Self-evolution record log.
|
||||
|
||||
Session-level evolutions are appended to their OWN per-day file under
|
||||
``memory/evolution/YYYY-MM-DD.md`` (separate from the nightly Deep Dream diary
|
||||
in ``memory/dreams/``). Each day's file accumulates one short section per
|
||||
evolution pass — tagged with a timestamp and a backup id for undo — so the
|
||||
memory UI can surface "what the agent learned/changed today" on one timeline
|
||||
without ever mixing into the dream diary or the main conversation memory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from common.log import logger
|
||||
|
||||
|
||||
def _evolution_dir(workspace_dir: Path, user_id: Optional[str] = None) -> Path:
|
||||
base = Path(workspace_dir) / "memory"
|
||||
if user_id:
|
||||
return base / "users" / user_id / "evolution"
|
||||
return base / "evolution"
|
||||
|
||||
|
||||
def append_session_evolution(
|
||||
workspace_dir: Path,
|
||||
summary: str,
|
||||
backup_id: Optional[str] = None,
|
||||
user_id: Optional[str] = None,
|
||||
) -> None:
|
||||
"""Append a session-evolution entry to today's evolution log."""
|
||||
if not summary or not summary.strip():
|
||||
return
|
||||
try:
|
||||
evo_dir = _evolution_dir(workspace_dir, user_id)
|
||||
evo_dir.mkdir(parents=True, exist_ok=True)
|
||||
today = datetime.now().strftime("%Y-%m-%d")
|
||||
log_file = evo_dir / f"{today}.md"
|
||||
|
||||
ts = datetime.now().strftime("%H:%M")
|
||||
header = f"## {ts}"
|
||||
body = summary.strip()
|
||||
if backup_id:
|
||||
body += f"\n\n_backup_id: {backup_id}_"
|
||||
|
||||
# Create with a title if the file is new, otherwise append a section.
|
||||
if not log_file.exists():
|
||||
log_file.write_text(f"# Self-Evolution: {today}\n\n", encoding="utf-8")
|
||||
with open(log_file, "a", encoding="utf-8") as f:
|
||||
f.write(f"\n{header}\n\n{body}\n")
|
||||
logger.info(f"[Evolution] Recorded session evolution to {log_file.name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Failed to record session evolution: {e}")
|
||||
151
agent/evolution/trigger.py
Normal file
151
agent/evolution/trigger.py
Normal file
@@ -0,0 +1,151 @@
|
||||
"""Idle-based evolution trigger.
|
||||
|
||||
A single background thread periodically scans live agent sessions and runs an
|
||||
evolution pass for any session that is idle for >= idle_minutes AND has enough
|
||||
accumulated signal, where "enough signal" is EITHER:
|
||||
- >= min_turns user turns since the last evolution, OR
|
||||
- the live context has grown past _CONTEXT_RATIO of the agent's token budget
|
||||
(mirrors how OpenClacky / Claude Code consolidate under context pressure).
|
||||
|
||||
Turn counting is per user turn (not per message), measured from the last
|
||||
evolution (or session start). After a pass runs, the baseline resets so a long
|
||||
session can evolve multiple times without re-judging old content.
|
||||
|
||||
Per-session evolution state is stored on the agent instance via lightweight
|
||||
attributes set by AgentBridge.agent_reply (see _note_user_turn).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
|
||||
from common.log import logger
|
||||
|
||||
from agent.evolution.config import get_evolution_config
|
||||
from agent.evolution.executor import run_evolution_for_session
|
||||
|
||||
_SCAN_INTERVAL_SECONDS = 60
|
||||
|
||||
# Context-pressure trigger: evolve once the live context exceeds this fraction
|
||||
# of the agent's token budget, even if min_turns hasn't been reached. Kept as a
|
||||
# module constant (not user config) for now. Fallback budget matches
|
||||
# agent_initializer / config.py (agent_max_context_tokens default = 50000).
|
||||
_CONTEXT_RATIO = 0.8
|
||||
_FALLBACK_CONTEXT_BUDGET = 50000
|
||||
|
||||
|
||||
def _context_pressure_reached(agent) -> bool:
|
||||
"""True if the agent's live context exceeds _CONTEXT_RATIO of its budget.
|
||||
|
||||
Uses the agent's own (estimated) token accounting so behavior matches the
|
||||
existing context-trimming path. Best-effort: any error -> False.
|
||||
"""
|
||||
try:
|
||||
with agent.messages_lock:
|
||||
messages = list(agent.messages)
|
||||
if not messages:
|
||||
return False
|
||||
est = sum(agent._estimate_message_tokens(m) for m in messages)
|
||||
budget = getattr(agent, "max_context_tokens", None) or _FALLBACK_CONTEXT_BUDGET
|
||||
return est / budget > _CONTEXT_RATIO
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def note_user_turn(agent, channel_type: str = "", receiver: str = "") -> None:
|
||||
"""Record activity for a session's agent. Called once per real user turn.
|
||||
|
||||
Maintains, on the agent instance:
|
||||
_evo_last_active : epoch seconds of the last user turn
|
||||
_evo_turns : user turns since the last evolution
|
||||
_evo_channel_type : originating channel (for later notify)
|
||||
_evo_receiver : push target for notify
|
||||
"""
|
||||
try:
|
||||
agent._evo_last_active = time.time()
|
||||
agent._evo_turns = int(getattr(agent, "_evo_turns", 0)) + 1
|
||||
if channel_type:
|
||||
agent._evo_channel_type = channel_type
|
||||
if receiver:
|
||||
agent._evo_receiver = receiver
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def mark_run_active(agent, active: bool) -> None:
|
||||
"""Flag whether the agent is mid-run, so idle scans skip a busy session.
|
||||
|
||||
Without this, a single run that lasts longer than idle_minutes would let
|
||||
the scanner fire an evolution pass concurrently with the live turn.
|
||||
"""
|
||||
try:
|
||||
agent._evo_run_active = bool(active)
|
||||
if active:
|
||||
agent._evo_last_active = time.time()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def start_evolution_trigger(agent_bridge) -> None:
|
||||
"""Start the idle-scan thread once per process (idempotent)."""
|
||||
if getattr(agent_bridge, "_evolution_trigger_started", False):
|
||||
return
|
||||
agent_bridge._evolution_trigger_started = True
|
||||
|
||||
t = threading.Thread(
|
||||
target=_scan_loop, args=(agent_bridge,), daemon=True, name="evolution-trigger"
|
||||
)
|
||||
t.start()
|
||||
logger.info("[Evolution] Idle trigger started")
|
||||
|
||||
|
||||
def _scan_loop(agent_bridge) -> None:
|
||||
while True:
|
||||
try:
|
||||
time.sleep(_SCAN_INTERVAL_SECONDS)
|
||||
cfg = get_evolution_config()
|
||||
if not cfg.enabled:
|
||||
continue
|
||||
_scan_once(agent_bridge, cfg)
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Scan loop error: {e}")
|
||||
time.sleep(_SCAN_INTERVAL_SECONDS)
|
||||
|
||||
|
||||
def _scan_once(agent_bridge, cfg) -> None:
|
||||
now = time.time()
|
||||
# Snapshot to avoid holding the dict while running long evolutions.
|
||||
sessions = list(getattr(agent_bridge, "agents", {}).items())
|
||||
for session_id, agent in sessions:
|
||||
try:
|
||||
# Skip sessions whose agent is mid-run: a long turn must not be
|
||||
# reviewed while it is still producing the answer.
|
||||
if getattr(agent, "_evo_run_active", False):
|
||||
continue
|
||||
last_active = getattr(agent, "_evo_last_active", 0)
|
||||
turns = int(getattr(agent, "_evo_turns", 0))
|
||||
# Enough signal = enough turns OR enough context pressure.
|
||||
enough_signal = turns >= cfg.min_turns or _context_pressure_reached(agent)
|
||||
if not enough_signal:
|
||||
continue
|
||||
idle = now - last_active if last_active > 0 else -1
|
||||
if last_active <= 0 or idle < cfg.idle_seconds:
|
||||
continue
|
||||
|
||||
channel_type = getattr(agent, "_evo_channel_type", "") or ""
|
||||
receiver = getattr(agent, "_evo_receiver", "") or ""
|
||||
|
||||
# Reset baseline BEFORE running so a long pass / new messages during
|
||||
# it don't double-trigger; turns accrue fresh from here.
|
||||
agent._evo_turns = 0
|
||||
|
||||
run_evolution_for_session(
|
||||
agent_bridge,
|
||||
session_id=session_id,
|
||||
channel_type=channel_type,
|
||||
receiver=receiver,
|
||||
idle_minutes=(now - last_active) / 60 if last_active > 0 else 0.0,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[Evolution] Failed to evaluate session={session_id}: {e}")
|
||||
@@ -12,11 +12,16 @@ Knowledge file layout (under workspace_root):
|
||||
|
||||
import os
|
||||
import re
|
||||
import asyncio
|
||||
import shutil
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from typing import Optional, Iterable
|
||||
|
||||
from common.log import logger
|
||||
from config import conf
|
||||
from agent.memory.config import MemoryConfig
|
||||
from agent.memory.manager import MemoryManager
|
||||
|
||||
|
||||
class KnowledgeService:
|
||||
@@ -25,9 +30,189 @@ class KnowledgeService:
|
||||
Operates directly on the filesystem.
|
||||
"""
|
||||
|
||||
def __init__(self, workspace_root: str):
|
||||
self.workspace_root = workspace_root
|
||||
self.knowledge_dir = os.path.join(workspace_root, "knowledge")
|
||||
PROTECTED_FILES = {"index.md", "log.md"}
|
||||
INVALID_NAME_RE = re.compile(r'[<>:"|?*\x00-\x1f]')
|
||||
|
||||
def __init__(self, workspace_root: str, memory_manager=None):
|
||||
self.workspace_root = os.path.abspath(workspace_root)
|
||||
self.knowledge_dir = os.path.join(self.workspace_root, "knowledge")
|
||||
self._memory_manager = memory_manager
|
||||
|
||||
def _resolve_path(self, rel_path: str, *, kind: Optional[str] = None,
|
||||
allow_missing: bool = True) -> tuple:
|
||||
if not isinstance(rel_path, str) or not rel_path.strip():
|
||||
raise ValueError("path is required")
|
||||
rel_path = rel_path.replace("\\", "/").strip("/")
|
||||
parts = rel_path.split("/")
|
||||
if any(not p or p in (".", "..") or self.INVALID_NAME_RE.search(p) for p in parts):
|
||||
raise ValueError("invalid path")
|
||||
if kind == "document" and not rel_path.lower().endswith(".md"):
|
||||
raise ValueError("document path must end with .md")
|
||||
|
||||
root = Path(self.knowledge_dir).resolve()
|
||||
candidate = root.joinpath(*parts)
|
||||
# Resolve the nearest existing ancestor so a symlink cannot be used
|
||||
# to escape when the final destination does not exist yet.
|
||||
ancestor = candidate
|
||||
while not ancestor.exists() and ancestor != root:
|
||||
ancestor = ancestor.parent
|
||||
try:
|
||||
ancestor.resolve().relative_to(root)
|
||||
except ValueError:
|
||||
raise ValueError("path outside knowledge dir")
|
||||
if candidate.exists():
|
||||
try:
|
||||
candidate.resolve().relative_to(root)
|
||||
except ValueError:
|
||||
raise ValueError("path outside knowledge dir")
|
||||
elif not allow_missing:
|
||||
raise FileNotFoundError(f"path not found: {rel_path}")
|
||||
return rel_path, candidate
|
||||
|
||||
def _ensure_not_protected(self, rel_path: str):
|
||||
if rel_path in self.PROTECTED_FILES:
|
||||
raise ValueError(f"protected knowledge file: {rel_path}")
|
||||
|
||||
def _manager(self):
|
||||
if self._memory_manager is None:
|
||||
self._memory_manager = MemoryManager(MemoryConfig(workspace_root=self.workspace_root))
|
||||
return self._memory_manager
|
||||
|
||||
@staticmethod
|
||||
def _run_sync(coro):
|
||||
try:
|
||||
asyncio.get_running_loop()
|
||||
except RuntimeError:
|
||||
return asyncio.run(coro)
|
||||
result = []
|
||||
error = []
|
||||
|
||||
def runner():
|
||||
try:
|
||||
result.append(asyncio.run(coro))
|
||||
except Exception as exc:
|
||||
error.append(exc)
|
||||
|
||||
thread = threading.Thread(target=runner)
|
||||
thread.start()
|
||||
thread.join()
|
||||
if error:
|
||||
raise error[0]
|
||||
return result[0] if result else None
|
||||
|
||||
def _sync_index(self, old_paths: Iterable[str]):
|
||||
old_paths = sorted(set(old_paths))
|
||||
if not old_paths:
|
||||
return
|
||||
manager = self._manager()
|
||||
for rel_path in old_paths:
|
||||
manager.storage.delete_by_path(f"knowledge/{rel_path}")
|
||||
manager.mark_dirty()
|
||||
self._run_sync(manager.sync())
|
||||
|
||||
def create_category(self, path: str) -> dict:
|
||||
rel_path, full_path = self._resolve_path(path, kind="category")
|
||||
if full_path.exists():
|
||||
return {"path": rel_path, "created": False, "reason": "already_exists"}
|
||||
full_path.mkdir(parents=True)
|
||||
return {"path": rel_path, "created": True}
|
||||
|
||||
def rename_category(self, path: str, new_path: str) -> dict:
|
||||
old_rel, old_full = self._resolve_path(path, kind="category", allow_missing=False)
|
||||
new_rel, new_full = self._resolve_path(new_path, kind="category")
|
||||
if not old_full.is_dir():
|
||||
raise ValueError(f"not a category: {old_rel}")
|
||||
if new_full.exists():
|
||||
raise FileExistsError(f"target already exists: {new_rel}")
|
||||
old_documents = [str(p.relative_to(old_full)).replace(os.sep, "/")
|
||||
for p in old_full.rglob("*.md") if p.is_file()]
|
||||
new_full.parent.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
old_full.rename(new_full)
|
||||
except FileNotFoundError:
|
||||
return {"old_path": old_rel, "path": new_rel, "moved": False, "reason": "not_found"}
|
||||
except FileExistsError:
|
||||
raise FileExistsError(f"target already exists: {new_rel}")
|
||||
old_paths = [f"{old_rel}/{p}" for p in old_documents]
|
||||
self._sync_index(old_paths)
|
||||
return {"old_path": old_rel, "path": new_rel, "moved_documents": len(old_documents)}
|
||||
|
||||
def delete_category(self, path: str, confirm: bool = False) -> dict:
|
||||
rel_path, full_path = self._resolve_path(path, kind="category")
|
||||
if not full_path.exists():
|
||||
return {"path": rel_path, "deleted": False, "reason": "not_found"}
|
||||
if not full_path.is_dir():
|
||||
raise ValueError(f"not a category: {rel_path}")
|
||||
knowledge_root = Path(self.knowledge_dir).resolve()
|
||||
documents = [str(p.relative_to(knowledge_root)).replace(os.sep, "/")
|
||||
for p in full_path.rglob("*.md") if p.is_file()]
|
||||
if any(p in self.PROTECTED_FILES for p in documents):
|
||||
raise ValueError("category contains protected knowledge files")
|
||||
if any(full_path.iterdir()) and not confirm:
|
||||
raise ValueError("category is not empty; confirmation is required")
|
||||
try:
|
||||
shutil.rmtree(full_path)
|
||||
except FileNotFoundError:
|
||||
return {"path": rel_path, "deleted": False, "reason": "not_found"}
|
||||
self._sync_index(documents)
|
||||
return {"path": rel_path, "deleted": True, "deleted_documents": len(documents)}
|
||||
|
||||
def delete_documents(self, paths: Iterable[str]) -> dict:
|
||||
if not isinstance(paths, list):
|
||||
raise ValueError("paths must be a list")
|
||||
results = []
|
||||
deleted = []
|
||||
for path in paths:
|
||||
rel_path, full_path = self._resolve_path(path, kind="document")
|
||||
self._ensure_not_protected(rel_path)
|
||||
if not full_path.exists():
|
||||
deleted.append(rel_path)
|
||||
results.append({"path": rel_path, "deleted": False, "reason": "not_found"})
|
||||
continue
|
||||
if not full_path.is_file():
|
||||
raise ValueError(f"not a document: {rel_path}")
|
||||
try:
|
||||
full_path.unlink()
|
||||
deleted.append(rel_path)
|
||||
results.append({"path": rel_path, "deleted": True})
|
||||
except FileNotFoundError:
|
||||
deleted.append(rel_path)
|
||||
results.append({"path": rel_path, "deleted": False, "reason": "not_found"})
|
||||
self._sync_index(deleted)
|
||||
return {"results": results, "deleted": sum(1 for item in results if item["deleted"])}
|
||||
|
||||
def move_documents(self, paths: Iterable[str], target_category: str) -> dict:
|
||||
if not isinstance(paths, list):
|
||||
raise ValueError("paths must be a list")
|
||||
target_rel, target_full = self._resolve_path(target_category, kind="category")
|
||||
if not target_full.is_dir():
|
||||
raise FileNotFoundError(f"category not found: {target_rel}")
|
||||
results = []
|
||||
moved_old_paths = []
|
||||
for path in paths:
|
||||
rel_path, full_path = self._resolve_path(path, kind="document")
|
||||
self._ensure_not_protected(rel_path)
|
||||
if not full_path.exists():
|
||||
results.append({"path": rel_path, "moved": False, "reason": "not_found"})
|
||||
continue
|
||||
destination = target_full / full_path.name
|
||||
new_rel = str(destination.relative_to(Path(self.knowledge_dir).resolve())).replace(os.sep, "/")
|
||||
if destination.exists():
|
||||
results.append({"path": rel_path, "moved": False, "reason": "target_exists",
|
||||
"target": new_rel})
|
||||
continue
|
||||
try:
|
||||
os.link(full_path, destination)
|
||||
full_path.unlink()
|
||||
moved_old_paths.append(rel_path)
|
||||
results.append({"path": rel_path, "moved": True, "target": new_rel})
|
||||
except FileExistsError:
|
||||
results.append({"path": rel_path, "moved": False, "reason": "target_exists",
|
||||
"target": new_rel})
|
||||
except FileNotFoundError:
|
||||
results.append({"path": rel_path, "moved": False, "reason": "not_found"})
|
||||
self._sync_index(moved_old_paths)
|
||||
return {"results": results, "moved": len(moved_old_paths)}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# list — directory tree with stats
|
||||
@@ -121,15 +306,8 @@ class KnowledgeService:
|
||||
:raises ValueError: if path is invalid or escapes knowledge dir
|
||||
:raises FileNotFoundError: if file does not exist
|
||||
"""
|
||||
if not rel_path or ".." in rel_path:
|
||||
raise ValueError("invalid path")
|
||||
|
||||
full_path = os.path.normpath(os.path.join(self.knowledge_dir, rel_path))
|
||||
allowed = os.path.normpath(self.knowledge_dir)
|
||||
if not full_path.startswith(allowed + os.sep) and full_path != allowed:
|
||||
raise ValueError("path outside knowledge dir")
|
||||
|
||||
if not os.path.isfile(full_path):
|
||||
rel_path, full_path = self._resolve_path(rel_path, kind="document")
|
||||
if not full_path.is_file():
|
||||
raise FileNotFoundError(f"file not found: {rel_path}")
|
||||
|
||||
with open(full_path, "r", encoding="utf-8") as f:
|
||||
@@ -228,13 +406,26 @@ class KnowledgeService:
|
||||
result = self.build_graph()
|
||||
return {"action": action, "code": 200, "message": "success", "payload": result}
|
||||
|
||||
elif action == "create_category":
|
||||
result = self.create_category(payload.get("path"))
|
||||
elif action == "rename_category":
|
||||
result = self.rename_category(payload.get("path"), payload.get("new_path"))
|
||||
elif action == "delete_category":
|
||||
result = self.delete_category(payload.get("path"), payload.get("confirm", False))
|
||||
elif action == "delete_documents":
|
||||
result = self.delete_documents(payload.get("paths") or [])
|
||||
elif action == "move_documents":
|
||||
result = self.move_documents(payload.get("paths") or [], payload.get("target_category"))
|
||||
else:
|
||||
return {"action": action, "code": 400, "message": f"unknown action: {action}", "payload": None}
|
||||
return {"action": action, "code": 200, "message": "success", "payload": result}
|
||||
|
||||
except ValueError as e:
|
||||
return {"action": action, "code": 403, "message": str(e), "payload": None}
|
||||
except FileNotFoundError as e:
|
||||
return {"action": action, "code": 404, "message": str(e), "payload": None}
|
||||
except FileExistsError as e:
|
||||
return {"action": action, "code": 409, "message": str(e), "payload": None}
|
||||
except Exception as e:
|
||||
logger.error(f"[KnowledgeService] dispatch error: action={action}, error={e}")
|
||||
return {"action": action, "code": 500, "message": str(e), "payload": None}
|
||||
|
||||
@@ -13,6 +13,7 @@ Storage path: ~/cow/sessions/conversations.db
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
@@ -109,6 +110,48 @@ def _extract_display_text(content: Any) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
# Internal markers written into the session for the agent's own bookkeeping
|
||||
# (scheduler injection / self-evolution undo). They must stay in the stored
|
||||
# content (the LLM reads them, e.g. to find a backup_id for undo) but should
|
||||
# never be shown verbatim to the user in the chat history UI.
|
||||
_SCHEDULED_DISPLAY_MARKERS = ("[SCHEDULED]", "Scheduled task")
|
||||
_EVOLUTION_DISPLAY_MARKER = "[EVOLUTION]"
|
||||
|
||||
|
||||
def _is_internal_user_marker(text: str) -> bool:
|
||||
"""True if a user-turn text is an internal injection marker (hide from UI)."""
|
||||
t = (text or "").lstrip()
|
||||
return any(t.startswith(m) for m in _SCHEDULED_DISPLAY_MARKERS)
|
||||
|
||||
|
||||
def _is_evolution_text(text: str) -> bool:
|
||||
"""True if assistant text is a self-evolution summary (before cleaning)."""
|
||||
return (text or "").lstrip().startswith(_EVOLUTION_DISPLAY_MARKER)
|
||||
|
||||
|
||||
def _clean_display_text(text: str) -> str:
|
||||
"""Strip internal markers from assistant text for user-facing display.
|
||||
|
||||
Removes a leading ``[EVOLUTION]`` tag and a trailing ``(backup_id: ...)``
|
||||
undo hint. The raw stored message is untouched, so undo + LLM context still
|
||||
work; only the rendered chat bubble is cleaned.
|
||||
"""
|
||||
if not text:
|
||||
return text
|
||||
cleaned = text
|
||||
stripped = cleaned.lstrip()
|
||||
if stripped.startswith(_EVOLUTION_DISPLAY_MARKER):
|
||||
cleaned = stripped[len(_EVOLUTION_DISPLAY_MARKER):].lstrip()
|
||||
# Drop a trailing backup_id undo hint line, e.g.
|
||||
# "(backup_id: 20260607-...; to undo, restore this backup)"
|
||||
cleaned = re.sub(
|
||||
r"\n*\(backup_id:[^\)]*\)\s*$",
|
||||
"",
|
||||
cleaned,
|
||||
).rstrip()
|
||||
return cleaned
|
||||
|
||||
|
||||
def _extract_tool_calls(content: Any) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Extract tool_use blocks from an assistant message content.
|
||||
@@ -210,7 +253,10 @@ def _group_into_display_turns(
|
||||
if user_row:
|
||||
content, created_at, _u_extras = user_row
|
||||
text = _extract_display_text(content)
|
||||
if text:
|
||||
# Hide internal injection markers (scheduler / self-evolution) so the
|
||||
# user never sees a synthetic "[SCHEDULED] self-evolution" bubble;
|
||||
# the assistant reply that follows is still rendered.
|
||||
if text and not _is_internal_user_marker(text):
|
||||
turns.append({"role": "user", "content": text, "created_at": created_at})
|
||||
|
||||
# Build an ordered list of steps preserving the original sequence:
|
||||
@@ -265,6 +311,18 @@ def _group_into_display_turns(
|
||||
step["result"] = tr.get("result", "")
|
||||
step["is_error"] = tr.get("is_error", False)
|
||||
|
||||
# Detect a self-evolution bubble BEFORE cleaning the marker away, so the
|
||||
# UI can flag it even though the visible text stays clean.
|
||||
is_evolution = _is_evolution_text(final_text)
|
||||
|
||||
# Clean internal markers from the user-facing assistant text. Applies to
|
||||
# both the final content and the mirrored content step so the rendered
|
||||
# bubble shows clean text while the stored message keeps the markers.
|
||||
final_text = _clean_display_text(final_text)
|
||||
for step in steps:
|
||||
if step.get("type") == "content":
|
||||
step["content"] = _clean_display_text(step.get("content", ""))
|
||||
|
||||
if steps or final_text:
|
||||
turn = {
|
||||
"role": "assistant",
|
||||
@@ -272,6 +330,8 @@ def _group_into_display_turns(
|
||||
"steps": steps,
|
||||
"created_at": final_ts or (user_row[1] if user_row else 0),
|
||||
}
|
||||
if is_evolution:
|
||||
turn["kind"] = "evolution"
|
||||
if merged_extras:
|
||||
turn["extras"] = merged_extras
|
||||
turns.append(turn)
|
||||
@@ -291,7 +351,7 @@ class ConversationStore:
|
||||
|
||||
def __init__(self, db_path: Path):
|
||||
self._db_path = db_path
|
||||
self._lock = threading.Lock()
|
||||
self._lock = threading.RLock() # Use RLock to allow reentrant locking
|
||||
self._init_db()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -509,6 +569,65 @@ class ConversationStore:
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def get_latest_pair_seqs(self, session_id: str) -> Dict[str, Optional[int]]:
|
||||
"""Return the seq numbers of the latest visible user message and the
|
||||
latest assistant message in a session.
|
||||
|
||||
A "visible" user message is one whose content is real user text
|
||||
(not just a tool_result block), so tool-execution turns do not
|
||||
shadow the actual user query.
|
||||
|
||||
Returns:
|
||||
Dict with keys ``user_seq`` and ``bot_seq``; either may be None
|
||||
when no matching message exists.
|
||||
"""
|
||||
result: Dict[str, Optional[int]] = {"user_seq": None, "bot_seq": None}
|
||||
with self._lock:
|
||||
conn = self._connect()
|
||||
try:
|
||||
# Latest assistant message (cheap: single row by seq DESC).
|
||||
row = conn.execute(
|
||||
"SELECT seq FROM messages "
|
||||
"WHERE session_id = ? AND role = 'assistant' "
|
||||
"ORDER BY seq DESC LIMIT 1",
|
||||
(session_id,),
|
||||
).fetchone()
|
||||
if row:
|
||||
result["bot_seq"] = int(row[0])
|
||||
|
||||
# Latest visible user message: scan recent user rows and
|
||||
# skip pure tool_result entries.
|
||||
rows = conn.execute(
|
||||
"SELECT seq, content FROM messages "
|
||||
"WHERE session_id = ? AND role = 'user' "
|
||||
"ORDER BY seq DESC LIMIT 20",
|
||||
(session_id,),
|
||||
).fetchall()
|
||||
for seq, content_raw in rows:
|
||||
try:
|
||||
content = json.loads(content_raw)
|
||||
except Exception:
|
||||
result["user_seq"] = int(seq)
|
||||
break
|
||||
if isinstance(content, list):
|
||||
has_text = any(
|
||||
isinstance(b, dict) and b.get("type") == "text"
|
||||
for b in content
|
||||
)
|
||||
has_tool_result = any(
|
||||
isinstance(b, dict) and b.get("type") == "tool_result"
|
||||
for b in content
|
||||
)
|
||||
if has_text and not has_tool_result:
|
||||
result["user_seq"] = int(seq)
|
||||
break
|
||||
else:
|
||||
result["user_seq"] = int(seq)
|
||||
break
|
||||
finally:
|
||||
conn.close()
|
||||
return result
|
||||
|
||||
def clear_session(self, session_id: str) -> None:
|
||||
"""Delete all messages and the session record for a given session_id."""
|
||||
with self._lock:
|
||||
@@ -524,6 +643,109 @@ class ConversationStore:
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def delete_message_pair(self, session_id: str, user_seq: int, delete_user: bool = True, cascade: bool = False) -> int:
|
||||
"""Delete a user message and/or its corresponding assistant reply.
|
||||
|
||||
The assistant reply is identified as all messages between user_seq
|
||||
and the next visible user message (or end of session).
|
||||
|
||||
Args:
|
||||
session_id: Session identifier.
|
||||
user_seq: The seq number of the user message.
|
||||
delete_user: If True (default), delete the user message too.
|
||||
If False, only delete assistant reply (for regenerate scenarios).
|
||||
cascade: If True, also delete all subsequent turns after this one.
|
||||
Used by edit-message which removes this turn and everything after.
|
||||
|
||||
Returns:
|
||||
Number of message rows deleted.
|
||||
"""
|
||||
with self._lock:
|
||||
conn = self._connect()
|
||||
try:
|
||||
with conn:
|
||||
# Verify this is a user message
|
||||
row = conn.execute(
|
||||
"SELECT role FROM messages WHERE session_id = ? AND seq = ?",
|
||||
(session_id, user_seq),
|
||||
).fetchone()
|
||||
if not row or row[0] != "user":
|
||||
return 0
|
||||
|
||||
if cascade:
|
||||
# Delete from this message to end of session
|
||||
start_seq = user_seq if delete_user else user_seq + 1
|
||||
end_seq_row = conn.execute(
|
||||
"SELECT MAX(seq) FROM messages WHERE session_id = ?",
|
||||
(session_id,),
|
||||
).fetchone()
|
||||
end_seq = (end_seq_row[0] or user_seq) + 1
|
||||
else:
|
||||
# Find the next visible user message seq (exclude tool_result)
|
||||
# Use batched query to avoid loading too many rows at once
|
||||
next_user_seq = None
|
||||
batch_size = 100
|
||||
offset = 0
|
||||
while True:
|
||||
batch = conn.execute(
|
||||
"""
|
||||
SELECT seq, content FROM messages
|
||||
WHERE session_id = ? AND seq > ? AND role = 'user'
|
||||
ORDER BY seq ASC
|
||||
LIMIT ? OFFSET ?
|
||||
""",
|
||||
(session_id, user_seq, batch_size, offset),
|
||||
).fetchall()
|
||||
if not batch:
|
||||
break
|
||||
for seq, content in batch:
|
||||
try:
|
||||
content_obj = json.loads(content)
|
||||
except Exception:
|
||||
content_obj = content
|
||||
if _is_visible_user_message(content_obj):
|
||||
next_user_seq = seq
|
||||
break
|
||||
if next_user_seq is not None:
|
||||
break
|
||||
offset += batch_size
|
||||
|
||||
# Determine the end boundary for deletion
|
||||
if next_user_seq is not None:
|
||||
end_seq = next_user_seq
|
||||
else:
|
||||
end_seq_row = conn.execute(
|
||||
"SELECT MAX(seq) FROM messages WHERE session_id = ?",
|
||||
(session_id,),
|
||||
).fetchone()
|
||||
end_seq = (end_seq_row[0] or user_seq) + 1
|
||||
|
||||
# Determine the start boundary for deletion
|
||||
start_seq = user_seq if delete_user else user_seq + 1
|
||||
|
||||
# Delete messages from start_seq to end_seq (exclusive)
|
||||
cur = conn.execute(
|
||||
"DELETE FROM messages WHERE session_id = ? AND seq >= ? AND seq < ?",
|
||||
(session_id, start_seq, end_seq),
|
||||
)
|
||||
deleted = cur.rowcount
|
||||
|
||||
# Update session msg_count
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE sessions
|
||||
SET msg_count = (
|
||||
SELECT COUNT(*) FROM messages WHERE session_id = ?
|
||||
)
|
||||
WHERE session_id = ?
|
||||
""",
|
||||
(session_id, session_id),
|
||||
)
|
||||
|
||||
return deleted
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def prune_scheduled_messages(
|
||||
self,
|
||||
session_id: str,
|
||||
@@ -1053,3 +1275,4 @@ def get_conversation_store() -> ConversationStore:
|
||||
_store_instance = ConversationStore(db_path)
|
||||
logger.debug(f"[ConversationStore] Using shared DB at: {db_path}")
|
||||
return _store_instance
|
||||
|
||||
|
||||
@@ -7,10 +7,14 @@ Supports multiple OpenAI-compatible embedding vendors:
|
||||
- dashscope (Aliyun Tongyi text-embedding-v4)
|
||||
- doubao (ByteDance Doubao Seed1.5 / large-text on Volcengine Ark)
|
||||
- zhipu (ZhipuAI embedding-3)
|
||||
- custom (any OpenAI-compatible endpoint)
|
||||
|
||||
Vendor keys here intentionally match the project's bot_type constants in
|
||||
common.const (OPENAI, LINKAI, QWEN_DASHSCOPE, DOUBAO, ZHIPU_AI).
|
||||
|
||||
Custom providers (bot_type "custom" or "custom:<id>") reuse the same
|
||||
OpenAI-compatible REST client with user-supplied api_key / api_base.
|
||||
|
||||
All providers share a single OpenAI-compatible REST client. Vendor-specific
|
||||
behaviors (truncation, query instruction prefix) are configured via metadata.
|
||||
"""
|
||||
@@ -138,6 +142,22 @@ EMBEDDING_VENDORS = {
|
||||
"query_instruction": "",
|
||||
"max_batch_size": 64,
|
||||
},
|
||||
# Custom provider — any OpenAI-compatible /embeddings endpoint. The
|
||||
# user must supply api_key + api_base + model via the web console
|
||||
# (stored in custom_providers list or legacy custom_api_key / custom_api_base).
|
||||
# Dimensions defaults to 1024 but can be overridden via config's
|
||||
# embedding_dimensions. No dim-param support assumption — safest
|
||||
# default for unknown endpoints.
|
||||
"custom": {
|
||||
"default_base_url": "",
|
||||
"default_model": "",
|
||||
"default_dimensions": 1024,
|
||||
"supports_dim_param": False,
|
||||
"needs_client_truncate": False,
|
||||
"needs_client_normalize": True,
|
||||
"query_instruction": "",
|
||||
"max_batch_size": 64,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -472,10 +492,19 @@ def create_embedding_provider(
|
||||
)
|
||||
|
||||
final_dim = dimensions if (dimensions and dimensions > 0) else meta["default_dimensions"]
|
||||
resolved_model = model or meta["default_model"]
|
||||
resolved_base = api_base or meta["default_base_url"]
|
||||
# Custom providers require explicit api_base and model — they cannot
|
||||
# fall back to OpenAI defaults like built-in vendors do.
|
||||
if provider == "custom":
|
||||
if not resolved_base:
|
||||
raise ValueError("Custom embedding provider requires an api_base URL")
|
||||
if not resolved_model:
|
||||
raise ValueError("Custom embedding provider requires a model name")
|
||||
return OpenAIEmbeddingProvider(
|
||||
model=model or meta["default_model"],
|
||||
model=resolved_model,
|
||||
api_key=api_key,
|
||||
api_base=api_base or meta["default_base_url"],
|
||||
api_base=resolved_base,
|
||||
extra_headers=extra_headers,
|
||||
dimensions=final_dim,
|
||||
supports_dim_param=meta["supports_dim_param"],
|
||||
|
||||
@@ -31,9 +31,13 @@ def detect_index_dim(storage) -> Optional[int]:
|
||||
if not row or not row["embedding"]:
|
||||
return None
|
||||
try:
|
||||
emb = json.loads(row["embedding"])
|
||||
raw = row["embedding"]
|
||||
if isinstance(raw, (bytes, bytearray)):
|
||||
# New BLOB format: 4 bytes per float32
|
||||
return len(raw) // 4
|
||||
emb = json.loads(raw)
|
||||
return len(emb) if isinstance(emb, list) else None
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
except (json.JSONDecodeError, TypeError, Exception):
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ from datetime import datetime, timedelta
|
||||
from agent.memory.config import MemoryConfig, get_default_memory_config
|
||||
from agent.memory.storage import MemoryStorage, MemoryChunk, SearchResult
|
||||
from agent.memory.chunker import TextChunker
|
||||
from agent.memory.embedding import EmbeddingProvider
|
||||
from agent.memory.embedding import EmbeddingProvider, EmbeddingCache
|
||||
from agent.memory.summarizer import MemoryFlushManager, create_memory_files_if_needed
|
||||
|
||||
|
||||
@@ -61,7 +61,11 @@ class MemoryManager:
|
||||
logger.info(
|
||||
"[MemoryManager] No embedding provider; memory will use keyword search only"
|
||||
)
|
||||
|
||||
|
||||
# Cache for query embeddings (avoids redundant API calls within a session)
|
||||
self._embedding_cache = EmbeddingCache()
|
||||
|
||||
|
||||
# Initialize memory flush manager
|
||||
workspace_dir = self.config.get_workspace()
|
||||
self.flush_manager = MemoryFlushManager(
|
||||
@@ -128,7 +132,14 @@ class MemoryManager:
|
||||
vector_results = []
|
||||
if self.embedding_provider:
|
||||
try:
|
||||
query_embedding = self.embedding_provider.embed_query(query)
|
||||
provider_name = type(self.embedding_provider).__name__
|
||||
model_name = getattr(self.embedding_provider, 'model', '')
|
||||
cached = self._embedding_cache.get(query, provider_name, model_name)
|
||||
if cached is not None:
|
||||
query_embedding = cached
|
||||
else:
|
||||
query_embedding = self.embedding_provider.embed_query(query)
|
||||
self._embedding_cache.put(query, provider_name, model_name, query_embedding)
|
||||
vector_results = self.storage.search_vector(
|
||||
query_embedding=query_embedding,
|
||||
user_id=user_id,
|
||||
|
||||
@@ -34,13 +34,18 @@ class MemoryService:
|
||||
# ------------------------------------------------------------------
|
||||
def list_files(self, page: int = 1, page_size: int = 20, category: str = "memory") -> dict:
|
||||
"""
|
||||
List memory or dream files with metadata (without content).
|
||||
List memory, dream, or evolution files with metadata (without content).
|
||||
|
||||
Args:
|
||||
category: ``"memory"`` (default) — MEMORY.md + daily files;
|
||||
``"dream"`` — dream diary files from memory/dreams/
|
||||
``"dream"`` — dream diary files from memory/dreams/;
|
||||
``"evolution"`` — self-evolution logs from memory/evolution/
|
||||
merged with the nightly dream diaries, so
|
||||
one tab shows everything the agent learned.
|
||||
"""
|
||||
if category == "dream":
|
||||
if category == "evolution":
|
||||
files = self._list_evolution_files()
|
||||
elif category == "dream":
|
||||
files = self._list_dream_files()
|
||||
else:
|
||||
files = self._list_memory_files()
|
||||
@@ -93,6 +98,26 @@ class MemoryService:
|
||||
|
||||
return files
|
||||
|
||||
def _list_evolution_files(self) -> List[dict]:
|
||||
"""Self-evolution logs (memory/evolution/*.md) merged with the nightly
|
||||
dream diaries (memory/dreams/*.md), newest first.
|
||||
|
||||
Both are surfaced under the unified "Self-Evolution" tab. A file's
|
||||
``type`` records its origin so the reader can resolve the right dir.
|
||||
"""
|
||||
files: List[dict] = []
|
||||
for sub, ftype in (("evolution", "evolution"), ("dreams", "dream")):
|
||||
sub_dir = os.path.join(self.memory_dir, sub)
|
||||
if not os.path.isdir(sub_dir):
|
||||
continue
|
||||
for name in os.listdir(sub_dir):
|
||||
full = os.path.join(sub_dir, name)
|
||||
if os.path.isfile(full) and name.endswith(".md"):
|
||||
files.append(self._file_info(full, name, ftype))
|
||||
# Sort newest first by filename (date-named); ties favor evolution.
|
||||
files.sort(key=lambda f: (f["filename"], f["type"] != "evolution"), reverse=True)
|
||||
return files
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# content — read a single file
|
||||
# ------------------------------------------------------------------
|
||||
@@ -101,7 +126,7 @@ class MemoryService:
|
||||
Read the full content of a memory or dream file.
|
||||
|
||||
:param filename: File name, e.g. ``MEMORY.md``, ``2026-02-20.md``
|
||||
:param category: ``"memory"`` or ``"dream"``
|
||||
:param category: ``"memory"``, ``"dream"`` or ``"evolution"``
|
||||
:return: dict with ``filename`` and ``content``
|
||||
:raises FileNotFoundError: if the file does not exist
|
||||
"""
|
||||
@@ -125,7 +150,7 @@ class MemoryService:
|
||||
Dispatch a memory management action.
|
||||
|
||||
:param action: ``list`` or ``content``
|
||||
:param payload: action-specific payload (supports ``category``: ``"memory"`` | ``"dream"``)
|
||||
:param payload: action-specific payload (supports ``category``: ``"memory"`` | ``"dream"`` | ``"evolution"``)
|
||||
:return: protocol-compatible response dict
|
||||
"""
|
||||
payload = payload or {}
|
||||
@@ -166,6 +191,7 @@ class MemoryService:
|
||||
- ``MEMORY.md`` → ``{workspace_root}/MEMORY.md``
|
||||
- ``2026-02-20.md`` (memory) → ``{workspace_root}/memory/2026-02-20.md``
|
||||
- ``2026-02-20.md`` (dream) → ``{workspace_root}/memory/dreams/2026-02-20.md``
|
||||
- ``2026-02-20.md`` (evolution) → ``{workspace_root}/memory/evolution/2026-02-20.md``
|
||||
|
||||
Raises ValueError if the resolved path escapes the allowed directory.
|
||||
"""
|
||||
@@ -173,6 +199,8 @@ class MemoryService:
|
||||
base_dir = self.workspace_root
|
||||
elif category == "dream":
|
||||
base_dir = os.path.join(self.memory_dir, "dreams")
|
||||
elif category == "evolution":
|
||||
base_dir = os.path.join(self.memory_dir, "evolution")
|
||||
else:
|
||||
base_dir = self.memory_dir
|
||||
|
||||
|
||||
@@ -5,12 +5,42 @@ Provides vector and keyword search capabilities
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
import re
|
||||
import sqlite3
|
||||
import json
|
||||
import hashlib
|
||||
import threading
|
||||
from typing import List, Dict, Optional, Any
|
||||
from pathlib import Path
|
||||
from dataclasses import dataclass
|
||||
try:
|
||||
import numpy as np
|
||||
_HAS_NUMPY = True
|
||||
except ImportError:
|
||||
_HAS_NUMPY = False
|
||||
np = None # type: ignore[assignment]
|
||||
|
||||
# UPSERT (INSERT … ON CONFLICT DO UPDATE) requires SQLite ≥ 3.24.0 (2018).
|
||||
# Older systems (e.g. CentOS 7 ships SQLite 3.7) fall back to INSERT OR REPLACE,
|
||||
# which risks FTS5 rowid drift on chunk updates (see save_chunk docstring).
|
||||
_HAS_UPSERT = sqlite3.sqlite_version_info >= (3, 24, 0)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CJK character ranges, compiled once at module load.
|
||||
# Covers: CJK Symbols/Punctuation, Japanese kana (hiragana + katakana),
|
||||
# CJK Unified Ideographs + Extension A, Korean syllables (Hangul),
|
||||
# CJK Compatibility Ideographs, and CJK Extension B–F.
|
||||
# ---------------------------------------------------------------------------
|
||||
_CJK_RANGES = (
|
||||
r'\u3000-\u30ff' # CJK Symbols/Punctuation + Japanese kana
|
||||
r'\u3400-\u9fff' # CJK Unified Ideographs (incl. Extension A)
|
||||
r'\uac00-\ud7af' # Korean syllables (Hangul)
|
||||
r'\uf900-\ufaff' # CJK Compatibility Ideographs
|
||||
r'\U00020000-\U0002fa1f' # CJK Extension B–F
|
||||
)
|
||||
_RE_CONTAINS_CJK = re.compile(f'[{_CJK_RANGES}]')
|
||||
_RE_CJK_WORDS = re.compile(f'[{_CJK_RANGES}]+')
|
||||
_RE_TRIGRAM_TOKENS = re.compile(f'[{_CJK_RANGES}]+|[A-Za-z0-9_]+')
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -48,6 +78,10 @@ class MemoryStorage:
|
||||
self.db_path = db_path
|
||||
self.conn: Optional[sqlite3.Connection] = None
|
||||
self.fts5_available = False # Track FTS5 availability
|
||||
# RLock protects concurrent writes from the same process.
|
||||
# SQLite WAL mode handles read/write concurrency at the file level,
|
||||
# but same-process concurrent writes still need a Python-level lock.
|
||||
self._lock = threading.RLock()
|
||||
self._init_db()
|
||||
|
||||
def _check_fts5_support(self) -> bool:
|
||||
@@ -69,6 +103,14 @@ class MemoryStorage:
|
||||
|
||||
# Check FTS5 support
|
||||
self.fts5_available = self._check_fts5_support()
|
||||
if not _HAS_UPSERT:
|
||||
from common.log import logger
|
||||
logger.warning(
|
||||
"[MemoryStorage] SQLite %s < 3.24 — UPSERT unavailable. "
|
||||
"Falling back to INSERT OR REPLACE; FTS5 rowid may drift on "
|
||||
"chunk updates (rebuild index periodically to recover).",
|
||||
sqlite3.sqlite_version,
|
||||
)
|
||||
if not self.fts5_available:
|
||||
from common.log import logger
|
||||
logger.debug("[MemoryStorage] FTS5 not available, using LIKE-based keyword search")
|
||||
@@ -175,6 +217,75 @@ class MemoryStorage:
|
||||
)
|
||||
self._rebuild_fts5_from_chunks()
|
||||
|
||||
# Internal key-value store for persistent flags (e.g. backfill tracking)
|
||||
self.conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS _meta (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
)
|
||||
""")
|
||||
|
||||
# Create trigram FTS5 table for CJK / mixed-language search
|
||||
self.trigram_fts5_available = False
|
||||
if self.fts5_available:
|
||||
try:
|
||||
self.conn.execute("""
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS chunks_fts_trigram USING fts5(
|
||||
text,
|
||||
id UNINDEXED,
|
||||
user_id UNINDEXED,
|
||||
path UNINDEXED,
|
||||
source UNINDEXED,
|
||||
scope UNINDEXED,
|
||||
content='chunks',
|
||||
content_rowid='rowid',
|
||||
tokenize='trigram case_sensitive 0'
|
||||
)
|
||||
""")
|
||||
self.conn.execute("""
|
||||
CREATE TRIGGER IF NOT EXISTS chunks_trigram_ai
|
||||
AFTER INSERT ON chunks BEGIN
|
||||
INSERT INTO chunks_fts_trigram(rowid, text, id, user_id, path, source, scope)
|
||||
VALUES (new.rowid, new.text, new.id, new.user_id, new.path, new.source, new.scope);
|
||||
END
|
||||
""")
|
||||
self.conn.execute("""
|
||||
CREATE TRIGGER IF NOT EXISTS chunks_trigram_ad
|
||||
AFTER DELETE ON chunks BEGIN
|
||||
DELETE FROM chunks_fts_trigram WHERE rowid = old.rowid;
|
||||
END
|
||||
""")
|
||||
self.conn.execute("""
|
||||
CREATE TRIGGER IF NOT EXISTS chunks_trigram_au
|
||||
AFTER UPDATE ON chunks BEGIN
|
||||
UPDATE chunks_fts_trigram
|
||||
SET text=new.text, id=new.id, user_id=new.user_id,
|
||||
path=new.path, source=new.source, scope=new.scope
|
||||
WHERE rowid = new.rowid;
|
||||
END
|
||||
""")
|
||||
# One-time backfill for existing rows.
|
||||
# NOTE: COUNT(*) on an FTS5 content table always returns 0, so we
|
||||
# use a persistent flag in _meta instead of counting trigram rows.
|
||||
backfill_done = self.conn.execute(
|
||||
"SELECT 1 FROM _meta WHERE key = 'trigram_backfill_done'"
|
||||
).fetchone()
|
||||
chunks_count = self.conn.execute(
|
||||
"SELECT COUNT(*) as c FROM chunks"
|
||||
).fetchone()['c']
|
||||
if chunks_count > 0 and not backfill_done:
|
||||
self.conn.execute(
|
||||
"INSERT INTO chunks_fts_trigram(chunks_fts_trigram) VALUES('rebuild')"
|
||||
)
|
||||
self.conn.execute(
|
||||
"INSERT OR REPLACE INTO _meta(key, value) VALUES('trigram_backfill_done', '1')"
|
||||
)
|
||||
self.trigram_fts5_available = True
|
||||
except Exception:
|
||||
from common.log import logger
|
||||
logger.warning("[MemoryStorage] trigram FTS5 unavailable, CJK search will use LIKE fallback", exc_info=True)
|
||||
self.trigram_fts5_available = False
|
||||
|
||||
# Create files metadata table
|
||||
self.conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS files (
|
||||
@@ -186,7 +297,7 @@ class MemoryStorage:
|
||||
updated_at INTEGER DEFAULT (strftime('%s', 'now'))
|
||||
)
|
||||
""")
|
||||
|
||||
|
||||
self.conn.commit()
|
||||
|
||||
def _fts5_state_inconsistent(self) -> bool:
|
||||
@@ -299,43 +410,98 @@ class MemoryStorage:
|
||||
self.conn.commit()
|
||||
|
||||
def save_chunk(self, chunk: MemoryChunk):
|
||||
"""Save a memory chunk"""
|
||||
self.conn.execute("""
|
||||
INSERT OR REPLACE INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line, text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
""", (
|
||||
chunk.id,
|
||||
chunk.user_id,
|
||||
chunk.scope,
|
||||
chunk.source,
|
||||
chunk.path,
|
||||
chunk.start_line,
|
||||
chunk.end_line,
|
||||
chunk.text,
|
||||
json.dumps(chunk.embedding) if chunk.embedding else None,
|
||||
"""Save a memory chunk (insert or update by id).
|
||||
|
||||
Uses SQLite UPSERT (INSERT … ON CONFLICT DO UPDATE) instead of
|
||||
INSERT OR REPLACE. INSERT OR REPLACE internally does DELETE+INSERT,
|
||||
which changes the row's rowid. Because both FTS5 tables use
|
||||
content_rowid='rowid', a new rowid would leave the old FTS index
|
||||
entries pointing at a non-existent rowid and trigger
|
||||
"fts5: missing row N from content table" errors.
|
||||
ON CONFLICT DO UPDATE fires the AFTER UPDATE trigger (chunks_au /
|
||||
chunks_trigram_au) and keeps the original rowid intact.
|
||||
"""
|
||||
if _HAS_UPSERT:
|
||||
_SQL = """
|
||||
INSERT INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line,
|
||||
text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
scope = excluded.scope,
|
||||
source = excluded.source,
|
||||
path = excluded.path,
|
||||
start_line = excluded.start_line,
|
||||
end_line = excluded.end_line,
|
||||
text = excluded.text,
|
||||
embedding = excluded.embedding,
|
||||
hash = excluded.hash,
|
||||
metadata = excluded.metadata,
|
||||
updated_at = strftime('%s', 'now')
|
||||
"""
|
||||
else:
|
||||
_SQL = """
|
||||
INSERT OR REPLACE INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line,
|
||||
text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
"""
|
||||
params = (
|
||||
chunk.id, chunk.user_id, chunk.scope, chunk.source, chunk.path,
|
||||
chunk.start_line, chunk.end_line, chunk.text,
|
||||
self._encode_embedding(chunk.embedding),
|
||||
chunk.hash,
|
||||
json.dumps(chunk.metadata) if chunk.metadata else None
|
||||
))
|
||||
self.conn.commit()
|
||||
|
||||
json.dumps(chunk.metadata) if chunk.metadata else None,
|
||||
)
|
||||
with self._lock:
|
||||
self.conn.execute(_SQL, params)
|
||||
self.conn.commit()
|
||||
|
||||
def save_chunks_batch(self, chunks: List[MemoryChunk]):
|
||||
"""Save multiple chunks in a batch"""
|
||||
self.conn.executemany("""
|
||||
INSERT OR REPLACE INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line, text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
""", [
|
||||
"""Save multiple chunks in a batch (insert or update by id).
|
||||
|
||||
See save_chunk for why UPSERT is used instead of INSERT OR REPLACE.
|
||||
"""
|
||||
if _HAS_UPSERT:
|
||||
_SQL = """
|
||||
INSERT INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line,
|
||||
text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
user_id = excluded.user_id,
|
||||
scope = excluded.scope,
|
||||
source = excluded.source,
|
||||
path = excluded.path,
|
||||
start_line = excluded.start_line,
|
||||
end_line = excluded.end_line,
|
||||
text = excluded.text,
|
||||
embedding = excluded.embedding,
|
||||
hash = excluded.hash,
|
||||
metadata = excluded.metadata,
|
||||
updated_at = strftime('%s', 'now')
|
||||
"""
|
||||
else:
|
||||
_SQL = """
|
||||
INSERT OR REPLACE INTO chunks
|
||||
(id, user_id, scope, source, path, start_line, end_line,
|
||||
text, embedding, hash, metadata, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
"""
|
||||
params_list = [
|
||||
(
|
||||
c.id, c.user_id, c.scope, c.source, c.path,
|
||||
c.start_line, c.end_line, c.text,
|
||||
json.dumps(c.embedding) if c.embedding else None,
|
||||
self._encode_embedding(c.embedding),
|
||||
c.hash,
|
||||
json.dumps(c.metadata) if c.metadata else None
|
||||
json.dumps(c.metadata) if c.metadata else None,
|
||||
)
|
||||
for c in chunks
|
||||
])
|
||||
self.conn.commit()
|
||||
]
|
||||
with self._lock:
|
||||
self.conn.executemany(_SQL, params_list)
|
||||
self.conn.commit()
|
||||
|
||||
def get_chunk(self, chunk_id: str) -> Optional[MemoryChunk]:
|
||||
"""Get a chunk by ID"""
|
||||
@@ -356,21 +522,21 @@ class MemoryStorage:
|
||||
limit: int = 10
|
||||
) -> List[SearchResult]:
|
||||
"""
|
||||
Vector similarity search using in-memory cosine similarity
|
||||
(sqlite-vec can be added later for better performance)
|
||||
Vector similarity search using numpy-vectorized cosine similarity.
|
||||
All embeddings are loaded then scored in a single BLAS matrix-vector
|
||||
multiply, which is ~100x faster than the pure-Python per-row loop.
|
||||
"""
|
||||
if scopes is None:
|
||||
scopes = ["shared"]
|
||||
if user_id:
|
||||
scopes.append("user")
|
||||
|
||||
# Build query
|
||||
|
||||
scope_placeholders = ','.join('?' * len(scopes))
|
||||
params = scopes
|
||||
|
||||
params = list(scopes)
|
||||
|
||||
if user_id:
|
||||
query = f"""
|
||||
SELECT * FROM chunks
|
||||
SELECT * FROM chunks
|
||||
WHERE scope IN ({scope_placeholders})
|
||||
AND (scope = 'shared' OR user_id = ?)
|
||||
AND embedding IS NOT NULL
|
||||
@@ -378,51 +544,95 @@ class MemoryStorage:
|
||||
params.append(user_id)
|
||||
else:
|
||||
query = f"""
|
||||
SELECT * FROM chunks
|
||||
SELECT * FROM chunks
|
||||
WHERE scope IN ({scope_placeholders})
|
||||
AND embedding IS NOT NULL
|
||||
"""
|
||||
|
||||
|
||||
rows = self.conn.execute(query, params).fetchall()
|
||||
if not rows:
|
||||
return []
|
||||
|
||||
# Calculate cosine similarity. We probe the first row's dim to fail
|
||||
# loudly on a query/index dim mismatch — otherwise every doc would
|
||||
# score 0 silently, leaving the user wondering why search broke.
|
||||
results = []
|
||||
query_dim = len(query_embedding)
|
||||
if rows:
|
||||
first = json.loads(rows[0]['embedding'])
|
||||
if isinstance(first, list) and len(first) != query_dim:
|
||||
raise ValueError(
|
||||
f"Embedding dim mismatch: query is {query_dim}-dim but "
|
||||
f"index stores {len(first)}-dim vectors. The configured "
|
||||
f"embedding model differs from the one that built the "
|
||||
f"index — run /memory rebuild-index to re-embed."
|
||||
)
|
||||
|
||||
# Parse embeddings and build a (N, D) matrix in one pass.
|
||||
# New rows store BLOB bytes (np.frombuffer); legacy rows fall back to JSON.
|
||||
# Filter out rows whose embedding dimension differs from the query —
|
||||
# mixing dimensions would cause np.array() to produce an object array
|
||||
# and matrix @ q_vec to raise ValueError.
|
||||
expected_dim = len(query_embedding)
|
||||
valid_rows = []
|
||||
vectors = []
|
||||
for row in rows:
|
||||
embedding = json.loads(row['embedding'])
|
||||
similarity = self._cosine_similarity(query_embedding, embedding)
|
||||
vec = self._decode_embedding(row['embedding'])
|
||||
if not vec:
|
||||
continue
|
||||
if len(vec) != expected_dim:
|
||||
from common.log import logger
|
||||
logger.warning(
|
||||
"[MemoryStorage] Skipping chunk %s: embedding dim %d != query dim %d",
|
||||
row['id'], len(vec), expected_dim
|
||||
)
|
||||
continue
|
||||
valid_rows.append(row)
|
||||
vectors.append(vec)
|
||||
|
||||
if similarity > 0:
|
||||
results.append((similarity, row))
|
||||
|
||||
# Sort by similarity and limit
|
||||
results.sort(key=lambda x: x[0], reverse=True)
|
||||
results = results[:limit]
|
||||
|
||||
return [
|
||||
SearchResult(
|
||||
path=row['path'],
|
||||
start_line=row['start_line'],
|
||||
end_line=row['end_line'],
|
||||
score=score,
|
||||
snippet=self._truncate_text(row['text'], 500),
|
||||
source=row['source'],
|
||||
user_id=row['user_id']
|
||||
)
|
||||
for score, row in results
|
||||
]
|
||||
if not vectors:
|
||||
return []
|
||||
|
||||
if _HAS_NUMPY:
|
||||
matrix = np.array(vectors, dtype=np.float32) # (N, D)
|
||||
q_vec = np.array(query_embedding, dtype=np.float32) # (D,)
|
||||
|
||||
# Vectorized cosine similarity: dot(matrix, q) / (||matrix|| * ||q||)
|
||||
dots = matrix @ q_vec # (N,)
|
||||
row_norms = np.linalg.norm(matrix, axis=1) # (N,)
|
||||
q_norm = float(np.linalg.norm(q_vec))
|
||||
denominators = row_norms * q_norm
|
||||
np.maximum(denominators, 1e-10, out=denominators) # avoid div-by-zero
|
||||
sims = dots / denominators # (N,)
|
||||
|
||||
# Select TopK using argpartition (O(N) average), then sort only those K
|
||||
k = min(limit, len(valid_rows))
|
||||
top_idx = np.argpartition(sims, -k)[-k:]
|
||||
top_idx = top_idx[np.argsort(sims[top_idx])[::-1]]
|
||||
|
||||
return [
|
||||
SearchResult(
|
||||
path=valid_rows[i]['path'],
|
||||
start_line=valid_rows[i]['start_line'],
|
||||
end_line=valid_rows[i]['end_line'],
|
||||
score=float(sims[i]),
|
||||
snippet=self._truncate_text(valid_rows[i]['text'], 500),
|
||||
source=valid_rows[i]['source'],
|
||||
user_id=valid_rows[i]['user_id']
|
||||
)
|
||||
for i in top_idx
|
||||
if sims[i] > 0
|
||||
]
|
||||
else:
|
||||
# Pure-Python cosine similarity fallback (numpy not installed)
|
||||
import math
|
||||
q = query_embedding
|
||||
q_norm = math.sqrt(sum(x * x for x in q)) or 1e-10
|
||||
scored = []
|
||||
for i, vec in enumerate(vectors):
|
||||
dot = sum(a * b for a, b in zip(vec, q))
|
||||
v_norm = math.sqrt(sum(x * x for x in vec)) or 1e-10
|
||||
sim = dot / (v_norm * q_norm)
|
||||
if sim > 0:
|
||||
scored.append((sim, valid_rows[i]))
|
||||
scored.sort(key=lambda x: x[0], reverse=True)
|
||||
return [
|
||||
SearchResult(
|
||||
path=row['path'],
|
||||
start_line=row['start_line'],
|
||||
end_line=row['end_line'],
|
||||
score=sim,
|
||||
snippet=self._truncate_text(row['text'], 500),
|
||||
source=row['source'],
|
||||
user_id=row['user_id']
|
||||
)
|
||||
for sim, row in scored[:limit]
|
||||
]
|
||||
|
||||
def search_keyword(
|
||||
self,
|
||||
@@ -445,12 +655,37 @@ class MemoryStorage:
|
||||
if user_id:
|
||||
scopes.append("user")
|
||||
|
||||
if self.fts5_available:
|
||||
# Step 1: Standard FTS5 (unicode61) — pure ASCII queries only.
|
||||
# Skipped when query contains any CJK characters: unicode61 tokenises CJK
|
||||
# as individual characters without forming meaningful tokens, so it would
|
||||
# match only the ASCII portion of a mixed query (e.g. "Python" from
|
||||
# "Python教程") and silently discard the CJK part. Those queries go
|
||||
# directly to Step 2 (trigram), which handles both ASCII and CJK together.
|
||||
fts1_attempted = False
|
||||
if (self.fts5_available
|
||||
and not MemoryStorage._contains_cjk(query)
|
||||
and MemoryStorage._build_fts_query(query)):
|
||||
fts1_attempted = True
|
||||
fts_results = self._search_fts5(query, user_id, scopes, limit)
|
||||
if fts_results:
|
||||
return fts_results
|
||||
|
||||
return self._search_like(query, user_id, scopes, limit)
|
||||
# Step 2: Trigram FTS5 — CJK/mixed queries, plus fallback when unicode61
|
||||
# returned nothing (trigram indexes all scripts with 3-char sliding windows,
|
||||
# so it can catch terms that unicode61 tokenisation misses).
|
||||
if self.trigram_fts5_available and (
|
||||
MemoryStorage._contains_cjk(query) or fts1_attempted
|
||||
):
|
||||
trigram_results = self._search_fts5_trigram(query, user_id, scopes, limit)
|
||||
if trigram_results:
|
||||
return trigram_results
|
||||
|
||||
# Step 3: LIKE fallback — last resort (FTS5 unavailable, or CJK tokens
|
||||
# shorter than 3 characters that trigram cannot match, e.g. a single-char query).
|
||||
if not self.fts5_available or MemoryStorage._contains_cjk(query):
|
||||
return self._search_like(query, user_id, scopes, limit)
|
||||
|
||||
return []
|
||||
|
||||
def _search_fts5(
|
||||
self,
|
||||
@@ -471,7 +706,7 @@ class MemoryStorage:
|
||||
sql_query = f"""
|
||||
SELECT chunks.*, bm25(chunks_fts) as rank
|
||||
FROM chunks_fts
|
||||
JOIN chunks ON chunks.id = chunks_fts.id
|
||||
JOIN chunks ON chunks.rowid = chunks_fts.rowid
|
||||
WHERE chunks_fts MATCH ?
|
||||
AND chunks.scope IN ({scope_placeholders})
|
||||
AND (chunks.scope = 'shared' OR chunks.user_id = ?)
|
||||
@@ -483,7 +718,7 @@ class MemoryStorage:
|
||||
sql_query = f"""
|
||||
SELECT chunks.*, bm25(chunks_fts) as rank
|
||||
FROM chunks_fts
|
||||
JOIN chunks ON chunks.id = chunks_fts.id
|
||||
JOIN chunks ON chunks.rowid = chunks_fts.rowid
|
||||
WHERE chunks_fts MATCH ?
|
||||
AND chunks.scope IN ({scope_placeholders})
|
||||
ORDER BY rank
|
||||
@@ -505,13 +740,11 @@ class MemoryStorage:
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
except Exception as e:
|
||||
except Exception:
|
||||
from common.log import logger
|
||||
logger.error(
|
||||
f"[MemoryStorage] FTS5 search failed (caller will fall back to LIKE): {e}"
|
||||
)
|
||||
logger.warning("[MemoryStorage] _search_fts5 failed, returning empty", exc_info=True)
|
||||
return []
|
||||
|
||||
|
||||
def _search_like(
|
||||
self,
|
||||
query: str,
|
||||
@@ -522,12 +755,11 @@ class MemoryStorage:
|
||||
"""LIKE-based search.
|
||||
|
||||
Used as the keyword-search fallback when FTS5 is unavailable, fails,
|
||||
or returns empty. Supports both CJK runs and ASCII word tokens so it
|
||||
can serve as a true safety net for any query.
|
||||
or returns empty. Supports both CJK runs (1+ chars) and ASCII word
|
||||
tokens (3+ chars) so it can serve as a true safety net for any query.
|
||||
"""
|
||||
import re
|
||||
# CJK runs (2+ chars) + ASCII word tokens (3+ chars to avoid noise)
|
||||
cjk_words = re.findall(r'[\u4e00-\u9fff]{2,}', query)
|
||||
# CJK runs (1+ chars, wide Unicode range) + ASCII words (3+ chars to avoid noise)
|
||||
cjk_words = _RE_CJK_WORDS.findall(query)
|
||||
ascii_words = [t for t in re.findall(r'[A-Za-z0-9_]+', query) if len(t) >= 3]
|
||||
words = cjk_words + ascii_words
|
||||
if not words:
|
||||
@@ -565,44 +797,55 @@ class MemoryStorage:
|
||||
|
||||
try:
|
||||
rows = self.conn.execute(sql_query, params).fetchall()
|
||||
return [
|
||||
SearchResult(
|
||||
results = []
|
||||
for row in rows:
|
||||
# Dynamic score: reward chunks that contain more of the query words.
|
||||
# Use all tokens (CJK + ASCII) so pure-ASCII queries are not skipped.
|
||||
# matched_count is always ≥1 because the WHERE clause uses OR, but
|
||||
# guard defensively so unexpected zero-match rows are never surfaced.
|
||||
text_lower = row['text'].lower()
|
||||
matched_count = sum(1 for w in words if w.lower() in text_lower)
|
||||
if matched_count == 0:
|
||||
continue
|
||||
score = min(0.85, 0.3 + 0.15 * matched_count)
|
||||
results.append(SearchResult(
|
||||
path=row['path'],
|
||||
start_line=row['start_line'],
|
||||
end_line=row['end_line'],
|
||||
score=0.5, # Fixed score for LIKE search
|
||||
score=score,
|
||||
snippet=self._truncate_text(row['text'], 500),
|
||||
source=row['source'],
|
||||
user_id=row['user_id']
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
except Exception as e:
|
||||
))
|
||||
results.sort(key=lambda r: r.score, reverse=True)
|
||||
return results
|
||||
except Exception:
|
||||
from common.log import logger
|
||||
logger.error(f"[MemoryStorage] LIKE search failed: {e}")
|
||||
logger.warning("[MemoryStorage] _search_like failed, returning empty", exc_info=True)
|
||||
return []
|
||||
|
||||
|
||||
def delete_by_path(self, path: str):
|
||||
"""Delete all chunks from a file"""
|
||||
self.conn.execute("""
|
||||
DELETE FROM chunks WHERE path = ?
|
||||
""", (path,))
|
||||
self.conn.commit()
|
||||
|
||||
"""Delete all chunks and file metadata for a path."""
|
||||
with self._lock:
|
||||
self.conn.execute("DELETE FROM chunks WHERE path = ?", (path,))
|
||||
self.conn.execute("DELETE FROM files WHERE path = ?", (path,))
|
||||
self.conn.commit()
|
||||
|
||||
def get_file_hash(self, path: str) -> Optional[str]:
|
||||
"""Get stored file hash"""
|
||||
row = self.conn.execute("""
|
||||
SELECT hash FROM files WHERE path = ?
|
||||
""", (path,)).fetchone()
|
||||
return row['hash'] if row else None
|
||||
|
||||
|
||||
def update_file_metadata(self, path: str, source: str, file_hash: str, mtime: int, size: int):
|
||||
"""Update file metadata"""
|
||||
self.conn.execute("""
|
||||
INSERT OR REPLACE INTO files (path, source, hash, mtime, size, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
""", (path, source, file_hash, mtime, size))
|
||||
self.conn.commit()
|
||||
with self._lock:
|
||||
self.conn.execute("""
|
||||
INSERT OR REPLACE INTO files (path, source, hash, mtime, size, updated_at)
|
||||
VALUES (?, ?, ?, ?, ?, strftime('%s', 'now'))
|
||||
""", (path, source, file_hash, mtime, size))
|
||||
self.conn.commit()
|
||||
|
||||
def get_stats(self) -> Dict[str, int]:
|
||||
"""Get storage statistics"""
|
||||
@@ -632,7 +875,8 @@ class MemoryStorage:
|
||||
self.conn.close()
|
||||
self.conn = None # Mark as closed
|
||||
except Exception as e:
|
||||
print(f"⚠️ Error closing database connection: {e}")
|
||||
from common.log import logger
|
||||
logger.warning("[MemoryStorage] Error closing database connection: %s", e)
|
||||
|
||||
def __del__(self):
|
||||
"""Destructor to ensure connection is closed"""
|
||||
@@ -642,7 +886,33 @@ class MemoryStorage:
|
||||
pass # Ignore errors during cleanup
|
||||
|
||||
# Helper methods
|
||||
|
||||
|
||||
@staticmethod
|
||||
def _encode_embedding(embedding: Optional[List[float]]) -> Optional[bytes]:
|
||||
"""Encode embedding as float32 BLOB bytes (~6x smaller and faster than JSON).
|
||||
Falls back to struct.pack when numpy is unavailable."""
|
||||
if embedding is None:
|
||||
return None
|
||||
if _HAS_NUMPY:
|
||||
return np.array(embedding, dtype=np.float32).tobytes()
|
||||
import struct
|
||||
return struct.pack(f'{len(embedding)}f', *embedding)
|
||||
|
||||
@staticmethod
|
||||
def _decode_embedding(raw) -> Optional[List[float]]:
|
||||
"""Decode embedding from BLOB bytes or legacy JSON string.
|
||||
Handles both numpy and numpy-free environments."""
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, (bytes, bytearray)):
|
||||
if _HAS_NUMPY:
|
||||
return np.frombuffer(raw, dtype=np.float32).tolist()
|
||||
import struct
|
||||
n = len(raw) // 4
|
||||
return list(struct.unpack(f'{n}f', raw))
|
||||
# Legacy JSON format written by older versions
|
||||
return json.loads(raw)
|
||||
|
||||
def _row_to_chunk(self, row) -> MemoryChunk:
|
||||
"""Convert database row to MemoryChunk"""
|
||||
return MemoryChunk(
|
||||
@@ -654,32 +924,89 @@ class MemoryStorage:
|
||||
start_line=row['start_line'],
|
||||
end_line=row['end_line'],
|
||||
text=row['text'],
|
||||
embedding=json.loads(row['embedding']) if row['embedding'] else None,
|
||||
embedding=self._decode_embedding(row['embedding']),
|
||||
hash=row['hash'],
|
||||
metadata=json.loads(row['metadata']) if row['metadata'] else None
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _cosine_similarity(vec1: List[float], vec2: List[float]) -> float:
|
||||
"""Calculate cosine similarity between two vectors"""
|
||||
if len(vec1) != len(vec2):
|
||||
return 0.0
|
||||
|
||||
dot_product = sum(a * b for a, b in zip(vec1, vec2))
|
||||
norm1 = sum(a * a for a in vec1) ** 0.5
|
||||
norm2 = sum(b * b for b in vec2) ** 0.5
|
||||
|
||||
if norm1 == 0 or norm2 == 0:
|
||||
return 0.0
|
||||
|
||||
return dot_product / (norm1 * norm2)
|
||||
def _contains_cjk(text: str) -> bool:
|
||||
"""Check if text contains CJK or related characters (Chinese, Japanese, Korean)."""
|
||||
return bool(_RE_CONTAINS_CJK.search(text))
|
||||
|
||||
@staticmethod
|
||||
def _contains_cjk(text: str) -> bool:
|
||||
"""Check if text contains CJK (Chinese/Japanese/Korean) characters"""
|
||||
import re
|
||||
return bool(re.search(r'[\u4e00-\u9fff]', text))
|
||||
|
||||
def _build_trigram_query(raw_query: str) -> Optional[str]:
|
||||
"""
|
||||
Build FTS5 MATCH query for the trigram tokenizer.
|
||||
Extracts CJK sequences (including single characters) and ASCII words,
|
||||
joining them with AND so all terms must appear in the matched chunk.
|
||||
"""
|
||||
tokens = _RE_TRIGRAM_TOKENS.findall(raw_query)
|
||||
tokens = [t for t in tokens if t]
|
||||
if not tokens:
|
||||
return None
|
||||
# Escape embedded double-quotes (FTS5 uses "" inside quoted phrases)
|
||||
quoted = [f'"{t.replace(chr(34), chr(34)*2)}"' for t in tokens]
|
||||
return ' AND '.join(quoted)
|
||||
|
||||
def _search_fts5_trigram(
|
||||
self,
|
||||
query: str,
|
||||
user_id: Optional[str],
|
||||
scopes: List[str],
|
||||
limit: int
|
||||
) -> List[SearchResult]:
|
||||
"""Trigram FTS5 search — handles CJK and mixed queries with BM25 ranking."""
|
||||
trigram_query = self._build_trigram_query(query)
|
||||
if not trigram_query:
|
||||
return []
|
||||
|
||||
scope_placeholders = ','.join('?' * len(scopes))
|
||||
params = [trigram_query] + list(scopes)
|
||||
|
||||
if user_id:
|
||||
sql = f"""
|
||||
SELECT chunks.*, bm25(chunks_fts_trigram) as rank
|
||||
FROM chunks_fts_trigram
|
||||
JOIN chunks ON chunks.rowid = chunks_fts_trigram.rowid
|
||||
WHERE chunks_fts_trigram MATCH ?
|
||||
AND chunks.scope IN ({scope_placeholders})
|
||||
AND (chunks.scope = 'shared' OR chunks.user_id = ?)
|
||||
ORDER BY rank
|
||||
LIMIT ?
|
||||
"""
|
||||
params.extend([user_id, limit])
|
||||
else:
|
||||
sql = f"""
|
||||
SELECT chunks.*, bm25(chunks_fts_trigram) as rank
|
||||
FROM chunks_fts_trigram
|
||||
JOIN chunks ON chunks.rowid = chunks_fts_trigram.rowid
|
||||
WHERE chunks_fts_trigram MATCH ?
|
||||
AND chunks.scope IN ({scope_placeholders})
|
||||
ORDER BY rank
|
||||
LIMIT ?
|
||||
"""
|
||||
params.append(limit)
|
||||
|
||||
try:
|
||||
rows = self.conn.execute(sql, params).fetchall()
|
||||
return [
|
||||
SearchResult(
|
||||
path=row['path'],
|
||||
start_line=row['start_line'],
|
||||
end_line=row['end_line'],
|
||||
score=self._bm25_rank_to_score(row['rank']),
|
||||
snippet=self._truncate_text(row['text'], 500),
|
||||
source=row['source'],
|
||||
user_id=row['user_id']
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
except Exception:
|
||||
from common.log import logger
|
||||
logger.warning("[MemoryStorage] _search_fts5_trigram failed, returning empty", exc_info=True)
|
||||
return []
|
||||
|
||||
@staticmethod
|
||||
def _build_fts_query(raw_query: str) -> Optional[str]:
|
||||
"""
|
||||
@@ -688,7 +1015,6 @@ class MemoryStorage:
|
||||
Works best for English and word-based languages.
|
||||
For CJK characters, LIKE search will be used as fallback.
|
||||
"""
|
||||
import re
|
||||
# Extract words (primarily English words and numbers)
|
||||
tokens = re.findall(r'[A-Za-z0-9_]+', raw_query)
|
||||
if not tokens:
|
||||
@@ -701,9 +1027,22 @@ class MemoryStorage:
|
||||
|
||||
@staticmethod
|
||||
def _bm25_rank_to_score(rank: float) -> float:
|
||||
"""Convert BM25 rank to 0-1 score"""
|
||||
normalized = max(0, rank) if rank is not None else 999
|
||||
return 1 / (1 + normalized)
|
||||
"""Convert SQLite BM25 rank to a [0, 1) relevance score.
|
||||
|
||||
SQLite's bm25() returns a non-positive float (0 or negative).
|
||||
More negative = more relevant. max(0, rank) would clip every
|
||||
negative value to 0, making every score 1/(1+0) = 1.0 and
|
||||
destroying all ranking information.
|
||||
|
||||
abs(rank) / (1 + abs(rank)) maps the absolute relevance magnitude
|
||||
to [0, 1): larger |rank| (stronger match) → score closer to 1.
|
||||
"""
|
||||
if rank is None:
|
||||
return 0.0
|
||||
# Add a floor of 0.3 so any FTS5 match always exceeds typical
|
||||
# min_score thresholds (default 0.1). Small-corpus ranks close to
|
||||
# 0 would otherwise produce score≈0 and be filtered out downstream.
|
||||
return 0.3 + 0.69 * (abs(rank) / (1.0 + abs(rank)))
|
||||
|
||||
@staticmethod
|
||||
def _truncate_text(text: str, max_chars: int) -> str:
|
||||
|
||||
@@ -16,7 +16,7 @@ from datetime import datetime
|
||||
from common.log import logger
|
||||
|
||||
|
||||
SUMMARIZE_SYSTEM_PROMPT = """你是一个对话记录助手。请将对话内容归纳为当天的日常记录。
|
||||
SUMMARIZE_SYSTEM_PROMPT_ZH = """你是一个对话记录助手。请将对话内容归纳为当天的日常记录。
|
||||
|
||||
## 要求
|
||||
|
||||
@@ -28,7 +28,23 @@ SUMMARIZE_SYSTEM_PROMPT = """你是一个对话记录助手。请将对话内容
|
||||
|
||||
当对话没有任何记录价值(仅含问候或无意义内容),直接回复"无"。"""
|
||||
|
||||
SUMMARIZE_USER_PROMPT = """请归纳以下对话的日常记录:
|
||||
SUMMARIZE_SYSTEM_PROMPT_EN = """You are a conversation-logging assistant. Summarize the conversation into a daily record.
|
||||
|
||||
## Requirements
|
||||
|
||||
Summarize by "event", not turn by turn:
|
||||
- One item per line, starting with "- "
|
||||
- Merge multiple turns about the same thing
|
||||
- Only record meaningful events; ignore small talk and greetings
|
||||
- Keep key decisions, conclusions and to-dos
|
||||
|
||||
If the conversation has no record value (only greetings or meaningless content), reply with exactly "None"."""
|
||||
|
||||
SUMMARIZE_USER_PROMPT_ZH = """请归纳以下对话的日常记录:
|
||||
|
||||
{conversation}"""
|
||||
|
||||
SUMMARIZE_USER_PROMPT_EN = """Summarize the daily record of the following conversation:
|
||||
|
||||
{conversation}"""
|
||||
|
||||
@@ -36,7 +52,7 @@ SUMMARIZE_USER_PROMPT = """请归纳以下对话的日常记录:
|
||||
# Deep Dream prompts — distill daily memories → MEMORY.md + dream diary
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DREAM_SYSTEM_PROMPT = """你是一个记忆整理助手,负责定期整理用户的长期记忆。
|
||||
DREAM_SYSTEM_PROMPT_ZH = """你是一个记忆整理助手,负责定期整理用户的长期记忆。
|
||||
|
||||
你将收到两份材料:
|
||||
1. **当前长期记忆** — MEMORY.md 的全部现有内容
|
||||
@@ -80,7 +96,51 @@ MEMORY.md 会注入每次对话的系统提示词中,因此必须保持精炼
|
||||
梦境日记内容...
|
||||
```"""
|
||||
|
||||
DREAM_USER_PROMPT = """## 当前长期记忆(MEMORY.md)
|
||||
DREAM_SYSTEM_PROMPT_EN = """You are a memory-curation assistant that periodically organizes the user's long-term memory.
|
||||
|
||||
You will receive two inputs:
|
||||
1. **Current long-term memory** — the full existing content of MEMORY.md
|
||||
2. **Today's diary** — the daily records
|
||||
|
||||
MEMORY.md is injected into the system prompt of every conversation, so it must stay concise and hold only valuable, memory-worthy content.
|
||||
|
||||
**Important: organize strictly based on the provided material. Never fabricate, infer, or add information not present in it.**
|
||||
|
||||
## Tasks
|
||||
|
||||
### Part 1: Updated long-term memory ([MEMORY])
|
||||
|
||||
Organize and distill on top of the existing memory, and output the complete updated content:
|
||||
- **Merge & distill**: combine semantically similar items into one dense statement rather than listing them
|
||||
- **Extract new**: pull memory-worthy new info from today's diary (preferences, decisions, people, rules, lessons)
|
||||
- **Resolve conflicts**: when new info contradicts an old item, prefer the new and replace the old
|
||||
- **Clean invalid**: remove temporary notes, blank items, formatting residue, meaningless or duplicate content
|
||||
- **Drop redundancy**: delete old items already covered by a more concise statement
|
||||
- One item per line, starting with "- ", without a date prefix
|
||||
- You may group related items under "## headings" for clarity
|
||||
- Goal: keep under 50 items, each ideally a single sentence
|
||||
|
||||
### Part 2: Dream diary ([DREAM])
|
||||
|
||||
Write a short diary in a concise narrative style recording what this curation found, keep it clean and readable:
|
||||
- Which duplicates or conflicts were found
|
||||
- What new insights were extracted from the diary
|
||||
- What cleanup and optimization was done
|
||||
- Overall feelings and observations
|
||||
|
||||
## Output format (follow strictly)
|
||||
|
||||
```
|
||||
[MEMORY]
|
||||
- memory item 1
|
||||
- memory item 2
|
||||
...
|
||||
|
||||
[DREAM]
|
||||
dream diary content...
|
||||
```"""
|
||||
|
||||
DREAM_USER_PROMPT_ZH = """## 当前长期记忆(MEMORY.md)
|
||||
|
||||
{memory_content}
|
||||
|
||||
@@ -88,6 +148,47 @@ DREAM_USER_PROMPT = """## 当前长期记忆(MEMORY.md)
|
||||
|
||||
{daily_content}"""
|
||||
|
||||
DREAM_USER_PROMPT_EN = """## Current long-term memory (MEMORY.md)
|
||||
|
||||
{memory_content}
|
||||
|
||||
## Recent diary (last {days} days)
|
||||
|
||||
{daily_content}"""
|
||||
|
||||
|
||||
def _is_en() -> bool:
|
||||
"""True when the resolved UI language is English."""
|
||||
try:
|
||||
from common import i18n
|
||||
return i18n.get_language() == "en"
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _summarize_system_prompt() -> str:
|
||||
return SUMMARIZE_SYSTEM_PROMPT_EN if _is_en() else SUMMARIZE_SYSTEM_PROMPT_ZH
|
||||
|
||||
|
||||
def _summarize_user_prompt() -> str:
|
||||
return SUMMARIZE_USER_PROMPT_EN if _is_en() else SUMMARIZE_USER_PROMPT_ZH
|
||||
|
||||
|
||||
def _dream_system_prompt() -> str:
|
||||
return DREAM_SYSTEM_PROMPT_EN if _is_en() else DREAM_SYSTEM_PROMPT_ZH
|
||||
|
||||
|
||||
def _dream_user_prompt() -> str:
|
||||
return DREAM_USER_PROMPT_EN if _is_en() else DREAM_USER_PROMPT_ZH
|
||||
|
||||
|
||||
def _is_empty_sentinel(text: str) -> bool:
|
||||
"""Match the "no record value" sentinel in both zh ("无") and en ("None")."""
|
||||
if not text:
|
||||
return True
|
||||
s = text.strip()
|
||||
return s == "" or s == "无" or s.lower() == "none"
|
||||
|
||||
|
||||
|
||||
class MemoryFlushManager:
|
||||
@@ -224,7 +325,7 @@ class MemoryFlushManager:
|
||||
"""Background worker: summarize with LLM, write daily memory file."""
|
||||
try:
|
||||
raw_summary = self._summarize_messages(messages, max_messages)
|
||||
if not raw_summary or not raw_summary.strip() or raw_summary.strip() == "无":
|
||||
if _is_empty_sentinel(raw_summary):
|
||||
logger.info(f"[MemoryFlush] No valuable content to flush (reason={reason})")
|
||||
return
|
||||
|
||||
@@ -264,7 +365,7 @@ class MemoryFlushManager:
|
||||
def _clean_summary_output(raw: str) -> str:
|
||||
"""Strip legacy [DAILY]/[MEMORY] markers if present, return clean daily text."""
|
||||
raw = raw.strip()
|
||||
if not raw or raw == "无":
|
||||
if _is_empty_sentinel(raw):
|
||||
return ""
|
||||
|
||||
# Strip [DAILY] marker
|
||||
@@ -355,21 +456,20 @@ class MemoryFlushManager:
|
||||
import time as _time
|
||||
t0 = _time.monotonic()
|
||||
try:
|
||||
user_msg = DREAM_USER_PROMPT.format(
|
||||
user_msg = _dream_user_prompt().format(
|
||||
memory_content=memory_content or "(empty)",
|
||||
days=lookback_days,
|
||||
daily_content=daily_content or "(no recent daily records)",
|
||||
)
|
||||
from agent.protocol.models import LLMRequest
|
||||
# Scale max_tokens based on input size to avoid truncating large MEMORY.md
|
||||
input_chars = len(memory_content) + len(daily_content)
|
||||
dream_max_tokens = max(2000, min(input_chars, 8000))
|
||||
# No output cap: the prompt already keeps MEMORY.md concise (~50
|
||||
# items), so a hard max_tokens would only risk truncating a large
|
||||
# rewrite. Let the model use its default output budget.
|
||||
request = LLMRequest(
|
||||
messages=[{"role": "user", "content": user_msg}],
|
||||
temperature=0.3,
|
||||
max_tokens=dream_max_tokens,
|
||||
stream=False,
|
||||
system=DREAM_SYSTEM_PROMPT,
|
||||
system=_dream_system_prompt(),
|
||||
)
|
||||
response = self.llm_model.call(request)
|
||||
raw = self._extract_response_text(response)
|
||||
@@ -501,9 +601,9 @@ class MemoryFlushManager:
|
||||
if self.llm_model:
|
||||
try:
|
||||
summary = self._call_llm_for_summary(conversation_text)
|
||||
if summary and summary.strip() and summary.strip() != "无":
|
||||
if not _is_empty_sentinel(summary):
|
||||
return summary.strip()
|
||||
logger.info("[MemoryFlush] LLM returned empty or '无', skipping write")
|
||||
logger.info("[MemoryFlush] LLM returned empty sentinel, skipping write")
|
||||
return ""
|
||||
except Exception as e:
|
||||
logger.warning(f"[MemoryFlush] LLM summarization failed, using fallback: {e}")
|
||||
@@ -579,11 +679,11 @@ class MemoryFlushManager:
|
||||
from agent.protocol.models import LLMRequest
|
||||
|
||||
request = LLMRequest(
|
||||
messages=[{"role": "user", "content": SUMMARIZE_USER_PROMPT.format(conversation=conversation_text)}],
|
||||
messages=[{"role": "user", "content": _summarize_user_prompt().format(conversation=conversation_text)}],
|
||||
temperature=0,
|
||||
max_tokens=500,
|
||||
stream=False,
|
||||
system=SUMMARIZE_SYSTEM_PROMPT,
|
||||
system=_summarize_system_prompt(),
|
||||
)
|
||||
|
||||
response = self.llm_model.call(request)
|
||||
|
||||
@@ -15,13 +15,13 @@ from config import conf
|
||||
|
||||
@dataclass
|
||||
class ContextFile:
|
||||
"""上下文文件"""
|
||||
"""A context file (path + content)."""
|
||||
path: str
|
||||
content: str
|
||||
|
||||
|
||||
class PromptBuilder:
|
||||
"""提示词构建器"""
|
||||
"""System prompt builder."""
|
||||
|
||||
def __init__(self, workspace_dir: str, language: str = "zh"):
|
||||
"""
|
||||
@@ -88,97 +88,144 @@ def build_agent_system_prompt(
|
||||
**kwargs
|
||||
) -> str:
|
||||
"""
|
||||
构建Agent系统提示词
|
||||
|
||||
顺序说明(按重要性和逻辑关系排列):
|
||||
1. 工具系统 - 核心能力,最先介绍
|
||||
2. 技能系统 - 紧跟工具,因为技能需要用 read 工具读取
|
||||
3. 记忆系统 - 记忆检索与写入引导
|
||||
3.5 知识系统 - 结构化知识库(knowledge/index.md 注入)
|
||||
4. 工作空间 - 工作环境说明
|
||||
5. 用户身份 - 用户信息(可选)
|
||||
6. 项目上下文 - AGENT.md, USER.md, RULE.md, MEMORY.md, BOOTSTRAP.md
|
||||
7. 运行时信息 - 元信息(时间、模型等)
|
||||
|
||||
Build the agent system prompt.
|
||||
|
||||
Section order (by importance and logical flow):
|
||||
1. Tooling - core capabilities, introduced first
|
||||
2. Skills - right after tools, since skills are read via the read tool
|
||||
3. Memory - memory recall and writing guidance
|
||||
3.5 Knowledge - structured knowledge base (injects knowledge/index.md)
|
||||
4. Workspace - working environment description
|
||||
5. User identity - user info (optional)
|
||||
6. Project context - AGENT.md, USER.md, RULE.md, MEMORY.md, BOOTSTRAP.md
|
||||
7. Runtime info - meta info (time, model, etc.)
|
||||
|
||||
Args:
|
||||
workspace_dir: 工作空间目录
|
||||
language: 语言 ("zh" 或 "en")
|
||||
base_persona: 基础人格描述(已废弃,由AGENT.md定义)
|
||||
user_identity: 用户身份信息
|
||||
tools: 工具列表
|
||||
context_files: 上下文文件列表
|
||||
skill_manager: 技能管理器
|
||||
memory_manager: 记忆管理器
|
||||
runtime_info: 运行时信息
|
||||
**kwargs: 其他参数
|
||||
|
||||
workspace_dir: workspace directory
|
||||
language: language ("zh" or "en")
|
||||
base_persona: base persona description (deprecated, defined by AGENT.md)
|
||||
user_identity: user identity info
|
||||
tools: tool list
|
||||
context_files: context file list
|
||||
skill_manager: skill manager
|
||||
memory_manager: memory manager
|
||||
runtime_info: runtime info
|
||||
**kwargs: extra args
|
||||
|
||||
Returns:
|
||||
完整的系统提示词
|
||||
The full system prompt.
|
||||
"""
|
||||
sections = []
|
||||
|
||||
# 1. 工具系统(最重要,放在最前面)
|
||||
|
||||
# 1. Tooling (most important, goes first)
|
||||
if tools:
|
||||
sections.extend(_build_tooling_section(tools, language))
|
||||
|
||||
# 2. 技能系统(紧跟工具,因为需要用 read 工具)
|
||||
|
||||
# 2. Skills (right after tools, since they need the read tool)
|
||||
if skill_manager:
|
||||
sections.extend(_build_skills_section(skill_manager, tools, language))
|
||||
|
||||
# 3. 记忆系统(独立的记忆能力)
|
||||
|
||||
# 3. Memory (standalone memory capability)
|
||||
if memory_manager:
|
||||
sections.extend(_build_memory_section(memory_manager, tools, language))
|
||||
|
||||
# 3.5 知识系统(结构化知识库)
|
||||
# 3.5 Knowledge (structured knowledge base)
|
||||
if conf().get("knowledge", True):
|
||||
sections.extend(_build_knowledge_section(workspace_dir, language))
|
||||
|
||||
# 4. 工作空间(工作环境说明)
|
||||
|
||||
# 4. Workspace (working environment description)
|
||||
sections.extend(_build_workspace_section(workspace_dir, language))
|
||||
|
||||
# 5. 用户身份(如果有)
|
||||
|
||||
# 5. User identity (if present)
|
||||
if user_identity:
|
||||
sections.extend(_build_user_identity_section(user_identity, language))
|
||||
|
||||
# 6. 项目上下文文件(AGENT.md, USER.md, RULE.md - 定义人格)
|
||||
|
||||
# 6. Project context files (AGENT.md, USER.md, RULE.md - define the persona)
|
||||
if context_files:
|
||||
sections.extend(_build_context_files_section(context_files, language))
|
||||
|
||||
# 7. 运行时信息(元信息,放在最后)
|
||||
|
||||
# 7. Runtime info (meta info, goes last)
|
||||
if runtime_info:
|
||||
sections.extend(_build_runtime_section(runtime_info, language))
|
||||
|
||||
|
||||
# 8. Response language (always appended, independent of the skeleton language)
|
||||
sections.extend(_build_response_language_section(language))
|
||||
|
||||
return "\n".join(sections)
|
||||
|
||||
|
||||
def _build_response_language_section(language: str) -> List[str]:
|
||||
"""Response-language rule, appended regardless of the prompt skeleton language.
|
||||
|
||||
Keeps the agent's reply language aligned with the user's input by default,
|
||||
so a Chinese-built prompt still answers an English user in English.
|
||||
"""
|
||||
if language == "en":
|
||||
return [
|
||||
"## 🌐 Response language",
|
||||
"",
|
||||
"By default, reply in the same language as the user's input, "
|
||||
"unless the user explicitly asks for another language.",
|
||||
"",
|
||||
]
|
||||
return [
|
||||
"## 🌐 回复语言",
|
||||
"",
|
||||
"默认使用与用户输入相同的语言回复,除非用户明确要求使用其他语言。",
|
||||
"",
|
||||
]
|
||||
|
||||
|
||||
def _build_identity_section(base_persona: Optional[str], language: str) -> List[str]:
|
||||
"""构建基础身份section - 不再需要,身份由AGENT.md定义"""
|
||||
# 不再生成基础身份section,完全由AGENT.md定义
|
||||
"""Base identity section - no longer needed, identity is defined by AGENT.md."""
|
||||
# Identity is fully defined by AGENT.md, so emit nothing here.
|
||||
return []
|
||||
|
||||
|
||||
def _build_tooling_section(tools: List[Any], language: str) -> List[str]:
|
||||
"""Build tooling section with concise tool list and call style guide."""
|
||||
is_en = language == "en"
|
||||
# One-line summaries for known tools (details are in the tool schema)
|
||||
core_summaries = {
|
||||
"read": "读取文件内容",
|
||||
"write": "创建或覆盖文件",
|
||||
"edit": "精确编辑文件",
|
||||
"ls": "列出目录内容",
|
||||
"grep": "搜索文件内容",
|
||||
"find": "按模式查找文件",
|
||||
"bash": "执行shell命令",
|
||||
"terminal": "管理后台进程",
|
||||
"web_search": "网络搜索",
|
||||
"web_fetch": "获取URL内容",
|
||||
"browser": "控制浏览器(关键结果或需要协助可截图发送给用户)",
|
||||
"memory_search": "搜索记忆",
|
||||
"memory_get": "读取记忆内容",
|
||||
"env_config": "管理API密钥和技能配置",
|
||||
"scheduler": "管理定时任务和提醒",
|
||||
"send": "发送本地文件给用户(仅限本地文件,URL直接放在回复文本中)",
|
||||
"vision": "分析图片内容(识别、描述、OCR文字提取等)",
|
||||
}
|
||||
if is_en:
|
||||
core_summaries = {
|
||||
"read": "read file content",
|
||||
"write": "create or overwrite a file",
|
||||
"edit": "make precise edits to a file",
|
||||
"ls": "list directory contents",
|
||||
"grep": "search file contents",
|
||||
"find": "find files by pattern",
|
||||
"bash": "run shell commands",
|
||||
"terminal": "manage background processes",
|
||||
"web_search": "web search",
|
||||
"web_fetch": "fetch URL content",
|
||||
"browser": "control the browser (screenshot key results or send to the user when help is needed)",
|
||||
"memory_search": "search memory",
|
||||
"memory_get": "read memory content",
|
||||
"env_config": "manage API keys and skill config",
|
||||
"scheduler": "manage scheduled tasks and reminders",
|
||||
"send": "send a local file to the user (local files only; put URLs directly in the reply text)",
|
||||
"vision": "analyze images (recognition, description, OCR, etc.)",
|
||||
}
|
||||
else:
|
||||
core_summaries = {
|
||||
"read": "读取文件内容",
|
||||
"write": "创建或覆盖文件",
|
||||
"edit": "精确编辑文件",
|
||||
"ls": "列出目录内容",
|
||||
"grep": "搜索文件内容",
|
||||
"find": "按模式查找文件",
|
||||
"bash": "执行shell命令",
|
||||
"terminal": "管理后台进程",
|
||||
"web_search": "网络搜索",
|
||||
"web_fetch": "获取URL内容",
|
||||
"browser": "控制浏览器(关键结果或需要协助可截图发送给用户)",
|
||||
"memory_search": "搜索记忆",
|
||||
"memory_get": "读取记忆内容",
|
||||
"env_config": "管理API密钥和技能配置",
|
||||
"scheduler": "管理定时任务和提醒",
|
||||
"send": "发送本地文件给用户(仅限本地文件,URL直接放在回复文本中)",
|
||||
"vision": "分析图片内容(识别、描述、OCR文字提取等)",
|
||||
}
|
||||
|
||||
# Preferred display order
|
||||
tool_order = [
|
||||
@@ -205,30 +252,46 @@ def _build_tooling_section(tools: List[Any], language: str) -> List[str]:
|
||||
summary = available[name]
|
||||
tool_lines.append(f"- {name}: {summary}" if summary else f"- {name}")
|
||||
|
||||
lines = [
|
||||
"## 🔧 工具系统",
|
||||
"",
|
||||
"可用工具(名称大小写敏感,严格按列表调用):",
|
||||
"\n".join(tool_lines),
|
||||
"",
|
||||
"工具调用风格:",
|
||||
"",
|
||||
"- 多步骤任务、复杂决策、敏感操作时,应简要说明当前在做什么、为什么这样做,让用户了解关键进展",
|
||||
"- 持续推进直到任务完成,完成后向用户报告结果",
|
||||
"- 回复中涉及密钥、令牌等敏感信息必须脱敏",
|
||||
"- URL链接直接放在回复文本中即可,系统会自动处理和渲染。无需下载后使用send工具发送",
|
||||
"",
|
||||
]
|
||||
if is_en:
|
||||
lines = [
|
||||
"## 🔧 Tooling",
|
||||
"",
|
||||
"Available tools (names are case-sensitive, call exactly as listed):",
|
||||
"\n".join(tool_lines),
|
||||
"",
|
||||
"Tool-calling style:",
|
||||
"",
|
||||
"- For multi-step tasks, complex decisions or sensitive operations, briefly explain what you are doing and why, so the user follows key progress",
|
||||
"- Keep going until the task is done, then report the result to the user",
|
||||
"- Always redact secrets, tokens and other sensitive info in replies",
|
||||
"- Put URLs directly in the reply text; the system handles and renders them. Don't download and re-send them via the send tool",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"## 🔧 工具系统",
|
||||
"",
|
||||
"可用工具(名称大小写敏感,严格按列表调用):",
|
||||
"\n".join(tool_lines),
|
||||
"",
|
||||
"工具调用风格:",
|
||||
"",
|
||||
"- 多步骤任务、复杂决策、敏感操作时,应简要说明当前在做什么、为什么这样做,让用户了解关键进展",
|
||||
"- 持续推进直到任务完成,完成后向用户报告结果",
|
||||
"- 回复中涉及密钥、令牌等敏感信息必须脱敏",
|
||||
"- URL链接直接放在回复文本中即可,系统会自动处理和渲染。无需下载后使用send工具发送",
|
||||
"",
|
||||
]
|
||||
|
||||
return lines
|
||||
|
||||
|
||||
def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], language: str) -> List[str]:
|
||||
"""构建技能系统section"""
|
||||
"""Build the skills section."""
|
||||
if not skill_manager:
|
||||
return []
|
||||
|
||||
# 获取read工具名称
|
||||
# Resolve the read tool name
|
||||
read_tool_name = "read"
|
||||
if tools:
|
||||
for tool in tools:
|
||||
@@ -237,23 +300,40 @@ def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], langua
|
||||
read_tool_name = tool_name
|
||||
break
|
||||
|
||||
lines = [
|
||||
"## 🧩 技能系统(mandatory)",
|
||||
"",
|
||||
"在回复之前:扫描下方 <available_skills> 中每个技能的 <description>。",
|
||||
"",
|
||||
f"- 如果有技能的描述与用户需求匹配:使用 `{read_tool_name}` 工具读取其 <location> 路径的 SKILL.md 文件,然后严格遵循文件中的指令。"
|
||||
"当有匹配的技能时,应优先使用技能",
|
||||
"- 如果多个技能都适用则选择最匹配的一个,然后读取并遵循。",
|
||||
"- 如果没有技能明确适用:不要读取任何 SKILL.md,直接使用通用工具。",
|
||||
"",
|
||||
f"**重要**: 技能不是工具,不能直接调用。使用技能的唯一方式是用 `{read_tool_name}` 读取 SKILL.md 文件,然后按文件内容操作。"
|
||||
"永远不要一次性读取多个技能,只在选择后再读取。",
|
||||
"",
|
||||
"以下是可用技能:"
|
||||
]
|
||||
if language == "en":
|
||||
lines = [
|
||||
"## 🧩 Skills (mandatory)",
|
||||
"",
|
||||
"Before replying: scan the <description> of every skill in <available_skills> below.",
|
||||
"",
|
||||
f"- If a skill's description matches the user's need: use the `{read_tool_name}` tool to read the SKILL.md at its <location> path, then strictly follow the instructions in the file. "
|
||||
"Prefer using a skill when one matches.",
|
||||
"- If multiple skills apply, pick the best-matching one, then read and follow it.",
|
||||
"- If no skill clearly applies: do not read any SKILL.md, just use the general tools.",
|
||||
"",
|
||||
f"**Important**: skills are not tools and cannot be called directly. The only way to use a skill is to read its SKILL.md with `{read_tool_name}`, then act on the file's content. "
|
||||
"Never read multiple skills at once — only read one after selecting it.",
|
||||
"",
|
||||
"Available skills:"
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"## 🧩 技能系统(mandatory)",
|
||||
"",
|
||||
"在回复之前:扫描下方 <available_skills> 中每个技能的 <description>。",
|
||||
"",
|
||||
f"- 如果有技能的描述与用户需求匹配:使用 `{read_tool_name}` 工具读取其 <location> 路径的 SKILL.md 文件,然后严格遵循文件中的指令。"
|
||||
"当有匹配的技能时,应优先使用技能",
|
||||
"- 如果多个技能都适用则选择最匹配的一个,然后读取并遵循。",
|
||||
"- 如果没有技能明确适用:不要读取任何 SKILL.md,直接使用通用工具。",
|
||||
"",
|
||||
f"**重要**: 技能不是工具,不能直接调用。使用技能的唯一方式是用 `{read_tool_name}` 读取 SKILL.md 文件,然后按文件内容操作。"
|
||||
"永远不要一次性读取多个技能,只在选择后再读取。",
|
||||
"",
|
||||
"以下是可用技能:"
|
||||
]
|
||||
|
||||
# 添加技能列表(通过skill_manager获取)
|
||||
# Append the skills list (built by skill_manager)
|
||||
try:
|
||||
skills_prompt = skill_manager.build_skills_prompt()
|
||||
logger.debug(f"[PromptBuilder] Skills prompt length: {len(skills_prompt) if skills_prompt else 0}")
|
||||
@@ -271,7 +351,7 @@ def _build_skills_section(skill_manager: Any, tools: Optional[List[Any]], langua
|
||||
|
||||
|
||||
def _build_memory_section(memory_manager: Any, tools: Optional[List[Any]], language: str) -> List[str]:
|
||||
"""构建记忆系统section"""
|
||||
"""Build the memory section."""
|
||||
if not memory_manager:
|
||||
return []
|
||||
|
||||
@@ -286,43 +366,82 @@ def _build_memory_section(memory_manager: Any, tools: Optional[List[Any]], langu
|
||||
from datetime import datetime
|
||||
today_file = datetime.now().strftime("%Y-%m-%d") + ".md"
|
||||
|
||||
lines = [
|
||||
"## 🧠 记忆系统",
|
||||
"",
|
||||
"### Memory Recall(mandatory)",
|
||||
"",
|
||||
"当用户询问过往事件、引用之前的决定、提到人物关系、偏好、待办、或你对某事不确定时,**必须先检索记忆再回答**。",
|
||||
"如果 MEMORY.md 中已有相关信息则无需重复检索。完整内容和每日记忆需要通过工具检索。",
|
||||
"",
|
||||
"1. 不确定位置 → `memory_search` 关键词/语义检索",
|
||||
"2. 已知位置 → `memory_get` 直接读取对应行",
|
||||
"3. search 无结果 → `memory_get` 读最近两天记忆",
|
||||
"",
|
||||
"**记忆文件结构**:",
|
||||
"- `MEMORY.md`: 长期记忆索引(已自动加载到上下文,核心信息、偏好、决策等)",
|
||||
f"- `memory/YYYY-MM-DD.md`: 每日记忆,今天是 `memory/{today_file}`",
|
||||
"- `knowledge/`: 结构化知识库(见下方知识系统)",
|
||||
"",
|
||||
"### 写入记忆",
|
||||
"",
|
||||
"遇到以下情况时,**主动**将信息写入记忆文件(无需告知用户):",
|
||||
"",
|
||||
"- 用户要求记住某些信息,或使用了「记住」「以后」「总是」「不要」「偏好」等表达",
|
||||
"- 用户分享了重要的个人偏好、习惯、决策",
|
||||
"- 对话中产生了重要的结论、方案、约定",
|
||||
"- 完成了复杂任务,值得记录关键步骤和结果",
|
||||
"",
|
||||
"**存储规则**:",
|
||||
f"- 长期核心信息 → `MEMORY.md`",
|
||||
f"- 当天事件/进展 → `memory/{today_file}`",
|
||||
"- 结构化知识 → `knowledge/`(见知识系统)",
|
||||
"- 追加 → `edit` 工具,oldText 留空",
|
||||
"- 修改 → `edit` 工具,oldText 填写要替换的文本",
|
||||
"- **禁止写入敏感信息**(API密钥、令牌等)",
|
||||
"",
|
||||
"**使用原则**: 自然使用记忆,就像你本来就知道;不用刻意提起,除非用户问起。",
|
||||
"",
|
||||
]
|
||||
if language == "en":
|
||||
lines = [
|
||||
"## 🧠 Memory",
|
||||
"",
|
||||
"### Memory Recall (mandatory)",
|
||||
"",
|
||||
"When the user asks about past events, references an earlier decision, mentions relationships, preferences or to-dos, or when you are unsure about something, **you must search memory before answering**.",
|
||||
"No need to re-search if the info is already in MEMORY.md. Full content and daily memory must be retrieved via tools.",
|
||||
"",
|
||||
"1. Location unknown → `memory_search` (keyword / semantic search)",
|
||||
"2. Location known → `memory_get` to read the exact lines",
|
||||
"3. Search returns nothing → `memory_get` to read the last two days of memory",
|
||||
"",
|
||||
"**Memory file structure**:",
|
||||
"- `MEMORY.md`: long-term memory index (already auto-loaded into context: core info, preferences, decisions, etc.)",
|
||||
f"- `memory/YYYY-MM-DD.md`: daily memory; today is `memory/{today_file}`",
|
||||
"- `knowledge/`: structured knowledge base (see the knowledge system below)",
|
||||
"",
|
||||
"### Writing memory",
|
||||
"",
|
||||
"In the following cases, **proactively** write info to memory files (no need to tell the user):",
|
||||
"",
|
||||
"- The user asks you to remember something, or uses words like \"remember\", \"from now on\", \"always\", \"never\", \"prefer\"",
|
||||
"- The user shares important personal preferences, habits or decisions",
|
||||
"- The conversation produces an important conclusion, plan or agreement",
|
||||
"- A complex task is completed and the key steps and results are worth recording",
|
||||
"",
|
||||
"**Storage rules**:",
|
||||
"- Long-term core info → `MEMORY.md`",
|
||||
f"- Today's events/progress → `memory/{today_file}`",
|
||||
"- Structured knowledge → `knowledge/` (see the knowledge system)",
|
||||
"- Append → `edit` tool with empty oldText",
|
||||
"- Modify → `edit` tool with oldText set to the text to replace",
|
||||
"- **Never write sensitive info** (API keys, tokens, etc.)",
|
||||
"",
|
||||
"**Principle**: use memory naturally, as if you simply knew it; don't bring it up unless asked.",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"## 🧠 记忆系统",
|
||||
"",
|
||||
"### Memory Recall(mandatory)",
|
||||
"",
|
||||
"当用户询问过往事件、引用之前的决定、提到人物关系、偏好、待办、或你对某事不确定时,**必须先检索记忆再回答**。",
|
||||
"如果 MEMORY.md 中已有相关信息则无需重复检索。完整内容和每日记忆需要通过工具检索。",
|
||||
"",
|
||||
"1. 不确定位置 → `memory_search` 关键词/语义检索",
|
||||
"2. 已知位置 → `memory_get` 直接读取对应行",
|
||||
"3. search 无结果 → `memory_get` 读最近两天记忆",
|
||||
"",
|
||||
"**记忆文件结构**:",
|
||||
"- `MEMORY.md`: 长期记忆索引(已自动加载到上下文,核心信息、偏好、决策等)",
|
||||
f"- `memory/YYYY-MM-DD.md`: 每日记忆,今天是 `memory/{today_file}`",
|
||||
"- `knowledge/`: 结构化知识库(见下方知识系统)",
|
||||
"",
|
||||
"### 写入记忆",
|
||||
"",
|
||||
"遇到以下情况时,**主动**将信息写入记忆文件(无需告知用户):",
|
||||
"",
|
||||
"- 用户要求记住某些信息,或使用了「记住」「以后」「总是」「不要」「偏好」等表达",
|
||||
"- 用户分享了重要的个人偏好、习惯、决策",
|
||||
"- 对话中产生了重要的结论、方案、约定",
|
||||
"- 完成了复杂任务,值得记录关键步骤和结果",
|
||||
"",
|
||||
"**存储规则**:",
|
||||
f"- 长期核心信息 → `MEMORY.md`",
|
||||
f"- 当天事件/进展 → `memory/{today_file}`",
|
||||
"- 结构化知识 → `knowledge/`(见知识系统)",
|
||||
"- 追加 → `edit` 工具,oldText 留空",
|
||||
"- 修改 → `edit` 工具,oldText 填写要替换的文本",
|
||||
"- **禁止写入敏感信息**(API密钥、令牌等)",
|
||||
"",
|
||||
"**使用原则**: 自然使用记忆,就像你本来就知道;不用刻意提起,除非用户问起。",
|
||||
"",
|
||||
]
|
||||
|
||||
return lines
|
||||
|
||||
@@ -339,37 +458,61 @@ def _build_knowledge_section(workspace_dir: str, language: str) -> List[str]:
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
lines = [
|
||||
"## 📚 知识系统",
|
||||
"",
|
||||
"你拥有一个持续积累的个人知识库 `knowledge/`,这是你的长期结构化知识存储。",
|
||||
"",
|
||||
"### 自动写入规则(mandatory)",
|
||||
"",
|
||||
"以下场景**必须**在回复的同时写入知识库,**直接写入,不要询问用户是否需要**:",
|
||||
"",
|
||||
"1. **用户分享了文章/链接/文档** → 阅读理解后,在同一轮回复中将要点写入 `knowledge/sources/<slug>.md`",
|
||||
"2. **深度讨论产生了结论/方案** → 整理为 `knowledge/analysis/<slug>.md`",
|
||||
"3. **对话涉及重要实体**(人物/公司/项目)→ 创建或更新 `knowledge/entities/<name>.md`",
|
||||
"4. **讨论了技术概念/方法论** → 整理为 `knowledge/concepts/<topic>.md`",
|
||||
"",
|
||||
"每次写入知识页面后,**必须同步更新** `knowledge/index.md` 添加一行索引。",
|
||||
"详细的页面格式和操作规范,请读取技能 `knowledge-wiki` 的 SKILL.md。",
|
||||
"",
|
||||
"⚠️ 不要问「要不要存到知识库」——符合上述场景就直接写入,这是你的本能行为。",
|
||||
"",
|
||||
]
|
||||
if language == "en":
|
||||
lines = [
|
||||
"## 📚 Knowledge",
|
||||
"",
|
||||
"You have a continuously growing personal knowledge base `knowledge/` — your long-term structured knowledge store.",
|
||||
"",
|
||||
"### Auto-write rules (mandatory)",
|
||||
"",
|
||||
"In the following cases you **must** write to the knowledge base alongside your reply, **directly, without asking the user**:",
|
||||
"",
|
||||
"1. **User shares an article / link / document** → after reading and understanding, write the key points to `knowledge/sources/<slug>.md` in the same turn",
|
||||
"2. **An in-depth discussion produces a conclusion / plan** → organize it into `knowledge/analysis/<slug>.md`",
|
||||
"3. **The conversation involves an important entity** (person / company / project) → create or update `knowledge/entities/<name>.md`",
|
||||
"4. **A technical concept / methodology is discussed** → organize it into `knowledge/concepts/<topic>.md`",
|
||||
"",
|
||||
"After writing any knowledge page, you **must update** `knowledge/index.md` with a new index line in sync.",
|
||||
"For detailed page format and conventions, read the SKILL.md of the `knowledge-wiki` skill.",
|
||||
"",
|
||||
"⚠️ Don't ask \"should I save this to the knowledge base?\" — if a case above matches, just write it. This is instinctive.",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"## 📚 知识系统",
|
||||
"",
|
||||
"你拥有一个持续积累的个人知识库 `knowledge/`,这是你的长期结构化知识存储。",
|
||||
"",
|
||||
"### 自动写入规则(mandatory)",
|
||||
"",
|
||||
"以下场景**必须**在回复的同时写入知识库,**直接写入,不要询问用户是否需要**:",
|
||||
"",
|
||||
"1. **用户分享了文章/链接/文档** → 阅读理解后,在同一轮回复中将要点写入 `knowledge/sources/<slug>.md`",
|
||||
"2. **深度讨论产生了结论/方案** → 整理为 `knowledge/analysis/<slug>.md`",
|
||||
"3. **对话涉及重要实体**(人物/公司/项目)→ 创建或更新 `knowledge/entities/<name>.md`",
|
||||
"4. **讨论了技术概念/方法论** → 整理为 `knowledge/concepts/<topic>.md`",
|
||||
"",
|
||||
"每次写入知识页面后,**必须同步更新** `knowledge/index.md` 添加一行索引。",
|
||||
"详细的页面格式和操作规范,请读取技能 `knowledge-wiki` 的 SKILL.md。",
|
||||
"",
|
||||
"⚠️ 不要问「要不要存到知识库」——符合上述场景就直接写入,这是你的本能行为。",
|
||||
"",
|
||||
]
|
||||
|
||||
if index_content:
|
||||
lines.extend([
|
||||
"### 当前知识索引",
|
||||
("### Current knowledge index" if language == "en" else "### 当前知识索引"),
|
||||
"",
|
||||
index_content,
|
||||
"",
|
||||
])
|
||||
|
||||
lines.extend([
|
||||
"**查询方式**:用 `read` 读取知识页面,或用 `memory_search` 检索(知识已纳入向量索引)。",
|
||||
("**How to query**: use `read` to open a knowledge page, or `memory_search` (knowledge is in the vector index)."
|
||||
if language == "en" else
|
||||
"**查询方式**:用 `read` 读取知识页面,或用 `memory_search` 检索(知识已纳入向量索引)。"),
|
||||
"",
|
||||
])
|
||||
|
||||
@@ -377,76 +520,118 @@ def _build_knowledge_section(workspace_dir: str, language: str) -> List[str]:
|
||||
|
||||
|
||||
def _build_user_identity_section(user_identity: Dict[str, str], language: str) -> List[str]:
|
||||
"""构建用户身份section"""
|
||||
"""Build the user identity section."""
|
||||
if not user_identity:
|
||||
return []
|
||||
|
||||
is_en = language == "en"
|
||||
lines = [
|
||||
"## 👤 用户身份",
|
||||
("## 👤 User identity" if is_en else "## 👤 用户身份"),
|
||||
"",
|
||||
]
|
||||
|
||||
|
||||
if user_identity.get("name"):
|
||||
lines.append(f"**用户姓名**: {user_identity['name']}")
|
||||
lines.append(f"**{'Name' if is_en else '用户姓名'}**: {user_identity['name']}")
|
||||
if user_identity.get("nickname"):
|
||||
lines.append(f"**称呼**: {user_identity['nickname']}")
|
||||
lines.append(f"**{'Preferred name' if is_en else '称呼'}**: {user_identity['nickname']}")
|
||||
if user_identity.get("timezone"):
|
||||
lines.append(f"**时区**: {user_identity['timezone']}")
|
||||
lines.append(f"**{'Timezone' if is_en else '时区'}**: {user_identity['timezone']}")
|
||||
if user_identity.get("notes"):
|
||||
lines.append(f"**备注**: {user_identity['notes']}")
|
||||
|
||||
lines.append(f"**{'Notes' if is_en else '备注'}**: {user_identity['notes']}")
|
||||
|
||||
lines.append("")
|
||||
|
||||
|
||||
return lines
|
||||
|
||||
|
||||
def _build_docs_section(workspace_dir: str, language: str) -> List[str]:
|
||||
"""构建文档路径section - 已移除,不再需要"""
|
||||
# 不再生成文档section
|
||||
"""Docs-path section - removed, no longer needed."""
|
||||
# No docs section is generated anymore.
|
||||
return []
|
||||
|
||||
|
||||
def _build_workspace_section(workspace_dir: str, language: str) -> List[str]:
|
||||
"""构建工作空间section"""
|
||||
lines = [
|
||||
"## 📂 工作空间",
|
||||
"",
|
||||
f"你的工作目录是: `{workspace_dir}`",
|
||||
"",
|
||||
"**路径使用规则** (非常重要):",
|
||||
"",
|
||||
f"1. **相对路径的基准目录**: 所有相对路径都是相对于 `{workspace_dir}` 而言的",
|
||||
f" - ✅ 正确: 访问工作空间内的文件用相对路径,如 `AGENT.md`",
|
||||
f" - ❌ 错误: 用相对路径访问其他目录的文件 (如果它不在 `{workspace_dir}` 内)",
|
||||
"",
|
||||
"2. **访问其他目录**: 如果要访问工作空间之外的目录(如项目代码、系统文件),**必须使用绝对路径**",
|
||||
f" - ✅ 正确: 例如 `~/chatgpt-on-wechat`、`/usr/local/`",
|
||||
f" - ❌ 错误: 假设相对路径会指向其他目录",
|
||||
"",
|
||||
"3. **路径解析示例**:",
|
||||
f" - 相对路径 `memory/` → 实际路径 `{workspace_dir}/memory/`",
|
||||
f" - 绝对路径 `~/chatgpt-on-wechat/docs/` → 实际路径 `~/chatgpt-on-wechat/docs/`",
|
||||
"",
|
||||
"4. **不确定时**: 先用 `bash pwd` 确认当前目录,或用 `ls .` 查看当前位置",
|
||||
"",
|
||||
"**重要说明 - 文件已自动加载**:",
|
||||
"",
|
||||
"以下文件在会话启动时**已经自动加载**到系统提示词中,你**无需再用 read 工具读取**:",
|
||||
"",
|
||||
"- ✅ `AGENT.md`: 已加载 - 你的人格和灵魂设定,请严格遵循。当你的名字、性格或交流风格发生变化时,主动用 `edit` 更新此文件",
|
||||
"- ✅ `USER.md`: 已加载 - 用户的身份信息。当用户修改称呼、姓名等身份信息时,用 `edit` 更新此文件",
|
||||
"- ✅ `RULE.md`: 已加载 - 工作空间使用指南和规则,请严格遵循",
|
||||
"- ✅ `MEMORY.md`: 已加载 - 长期记忆索引",
|
||||
"",
|
||||
"**💬 交流规范**:",
|
||||
"",
|
||||
"- 记忆相关操作无需暴露文件名,用自然语言表达即可。例如说「我已记住」而非「已更新 MEMORY.md」",
|
||||
"- 任务执行过程中的关键决策和步骤应该告知用户,让用户了解你在做什么、为什么这么做",
|
||||
"- 做真正有帮助的助手,而不是表演式的客套,尽可能帮忙解决问题",
|
||||
"- 回复应结构清晰、重点突出。善用 **加粗**、列表、分段等格式让信息一目了然",
|
||||
"- 适当使用 emoji 让表达更生动自然 🎯,但不要过度堆砌",
|
||||
"",
|
||||
]
|
||||
"""Build the workspace section."""
|
||||
if language == "en":
|
||||
lines = [
|
||||
"## 📂 Workspace",
|
||||
"",
|
||||
f"Your working directory is: `{workspace_dir}`",
|
||||
"",
|
||||
"**Path rules** (very important):",
|
||||
"",
|
||||
f"1. **Base directory for relative paths**: all relative paths are relative to `{workspace_dir}`",
|
||||
" - ✅ Correct: use relative paths for files inside the workspace, e.g. `AGENT.md`",
|
||||
f" - ❌ Wrong: using a relative path for files in other directories (if not inside `{workspace_dir}`)",
|
||||
"",
|
||||
"2. **Accessing other directories**: to reach directories outside the workspace (project code, system files), **you must use absolute paths**",
|
||||
" - ✅ Correct: e.g. `~/chatgpt-on-wechat`, `/usr/local/`",
|
||||
" - ❌ Wrong: assuming a relative path points to another directory",
|
||||
"",
|
||||
"3. **Path resolution examples**:",
|
||||
f" - relative `memory/` → actual `{workspace_dir}/memory/`",
|
||||
" - absolute `~/chatgpt-on-wechat/docs/` → actual `~/chatgpt-on-wechat/docs/`",
|
||||
"",
|
||||
"4. **When unsure**: run `bash pwd` to confirm the current directory, or `ls .` to see where you are",
|
||||
"",
|
||||
"**Important - files already auto-loaded**:",
|
||||
"",
|
||||
"The following files are **already auto-loaded** into the system prompt at session start, so you **don't need to read them again with the read tool**:",
|
||||
"",
|
||||
"- ✅ `AGENT.md`: loaded - your persona and soul; follow it strictly. When your name, personality or style changes, proactively `edit` this file",
|
||||
"- ✅ `USER.md`: loaded - the user's identity info. When the user changes how they're addressed, their name, etc., `edit` this file",
|
||||
"- ✅ `RULE.md`: loaded - workspace guide and rules; follow them strictly",
|
||||
"- ✅ `MEMORY.md`: loaded - long-term memory index",
|
||||
"",
|
||||
"**💬 Communication norms**:",
|
||||
"",
|
||||
"- No need to expose file names for memory operations; use natural language. Say \"I'll remember that\" rather than \"updated MEMORY.md\"",
|
||||
"- Tell the user about key decisions and steps during a task, so they know what you're doing and why",
|
||||
"- Be genuinely helpful rather than performatively polite; solve the problem as much as you can",
|
||||
"- Keep replies well-structured and focused. Use **bold**, lists and sections to make info clear at a glance",
|
||||
"- Use emoji to make expression lively 🎯, but don't overdo it",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"## 📂 工作空间",
|
||||
"",
|
||||
f"你的工作目录是: `{workspace_dir}`",
|
||||
"",
|
||||
"**路径使用规则** (非常重要):",
|
||||
"",
|
||||
f"1. **相对路径的基准目录**: 所有相对路径都是相对于 `{workspace_dir}` 而言的",
|
||||
f" - ✅ 正确: 访问工作空间内的文件用相对路径,如 `AGENT.md`",
|
||||
f" - ❌ 错误: 用相对路径访问其他目录的文件 (如果它不在 `{workspace_dir}` 内)",
|
||||
"",
|
||||
"2. **访问其他目录**: 如果要访问工作空间之外的目录(如项目代码、系统文件),**必须使用绝对路径**",
|
||||
f" - ✅ 正确: 例如 `~/chatgpt-on-wechat`、`/usr/local/`",
|
||||
f" - ❌ 错误: 假设相对路径会指向其他目录",
|
||||
"",
|
||||
"3. **路径解析示例**:",
|
||||
f" - 相对路径 `memory/` → 实际路径 `{workspace_dir}/memory/`",
|
||||
f" - 绝对路径 `~/chatgpt-on-wechat/docs/` → 实际路径 `~/chatgpt-on-wechat/docs/`",
|
||||
"",
|
||||
"4. **不确定时**: 先用 `bash pwd` 确认当前目录,或用 `ls .` 查看当前位置",
|
||||
"",
|
||||
"**重要说明 - 文件已自动加载**:",
|
||||
"",
|
||||
"以下文件在会话启动时**已经自动加载**到系统提示词中,你**无需再用 read 工具读取**:",
|
||||
"",
|
||||
"- ✅ `AGENT.md`: 已加载 - 你的人格和灵魂设定,请严格遵循。当你的名字、性格或交流风格发生变化时,主动用 `edit` 更新此文件",
|
||||
"- ✅ `USER.md`: 已加载 - 用户的身份信息。当用户修改称呼、姓名等身份信息时,用 `edit` 更新此文件",
|
||||
"- ✅ `RULE.md`: 已加载 - 工作空间使用指南和规则,请严格遵循",
|
||||
"- ✅ `MEMORY.md`: 已加载 - 长期记忆索引",
|
||||
"",
|
||||
"**💬 交流规范**:",
|
||||
"",
|
||||
"- 记忆相关操作无需暴露文件名,用自然语言表达即可。例如说「我已记住」而非「已更新 MEMORY.md」",
|
||||
"- 任务执行过程中的关键决策和步骤应该告知用户,让用户了解你在做什么、为什么这么做",
|
||||
"- 做真正有帮助的助手,而不是表演式的客套,尽可能帮忙解决问题",
|
||||
"- 回复应结构清晰、重点突出。善用 **加粗**、列表、分段等格式让信息一目了然",
|
||||
"- 适当使用 emoji 让表达更生动自然 🎯,但不要过度堆砌",
|
||||
"",
|
||||
]
|
||||
|
||||
# Cloud deployment: inject websites directory info and access URL
|
||||
cloud_website_lines = _build_cloud_website_section(workspace_dir)
|
||||
@@ -466,29 +651,42 @@ def _build_cloud_website_section(workspace_dir: str) -> List[str]:
|
||||
|
||||
|
||||
def _build_context_files_section(context_files: List[ContextFile], language: str) -> List[str]:
|
||||
"""构建项目上下文文件section"""
|
||||
"""Build the project context files section."""
|
||||
if not context_files:
|
||||
return []
|
||||
|
||||
# 检查是否有AGENT.md
|
||||
# Check whether AGENT.md is present
|
||||
has_agent = any(
|
||||
f.path.lower().endswith('agent.md') or 'agent.md' in f.path.lower()
|
||||
for f in context_files
|
||||
)
|
||||
|
||||
lines = [
|
||||
"# 📋 项目上下文",
|
||||
"",
|
||||
"以下项目上下文文件已被加载:",
|
||||
"",
|
||||
]
|
||||
|
||||
is_en = language == "en"
|
||||
if is_en:
|
||||
lines = [
|
||||
"# 📋 Project context",
|
||||
"",
|
||||
"The following project context files have been loaded:",
|
||||
"",
|
||||
]
|
||||
else:
|
||||
lines = [
|
||||
"# 📋 项目上下文",
|
||||
"",
|
||||
"以下项目上下文文件已被加载:",
|
||||
"",
|
||||
]
|
||||
|
||||
if has_agent:
|
||||
lines.append("**`AGENT.md` 是你的灵魂文件** 🪞:严格遵循其中定义的人格、语气和设定,做真实的自己,避免僵硬、模板化的回复。")
|
||||
lines.append("当用户通过对话透露了对你性格、风格、职责、能力边界的新期望,你应该主动用 `edit` 更新 AGENT.md 以反映这些演变。")
|
||||
if is_en:
|
||||
lines.append("**`AGENT.md` is your soul file** 🪞: strictly follow the persona, tone and settings it defines. Be your real self, avoid stiff, template-like replies.")
|
||||
lines.append("When the user reveals new expectations about your personality, style, responsibilities or capability boundaries, proactively `edit` AGENT.md to reflect that evolution.")
|
||||
else:
|
||||
lines.append("**`AGENT.md` 是你的灵魂文件** 🪞:严格遵循其中定义的人格、语气和设定,做真实的自己,避免僵硬、模板化的回复。")
|
||||
lines.append("当用户通过对话透露了对你性格、风格、职责、能力边界的新期望,你应该主动用 `edit` 更新 AGENT.md 以反映这些演变。")
|
||||
lines.append("")
|
||||
|
||||
# 添加每个文件的内容
|
||||
# Append the content of each file
|
||||
for file in context_files:
|
||||
lines.append(f"## {file.path}")
|
||||
lines.append("")
|
||||
@@ -499,21 +697,23 @@ def _build_context_files_section(context_files: List[ContextFile], language: str
|
||||
|
||||
|
||||
def _build_runtime_section(runtime_info: Dict[str, Any], language: str) -> List[str]:
|
||||
"""构建运行时信息section - 支持动态时间"""
|
||||
"""Build the runtime info section - supports dynamic time."""
|
||||
if not runtime_info:
|
||||
return []
|
||||
|
||||
is_en = language == "en"
|
||||
time_label = "Current time" if is_en else "当前时间"
|
||||
lines = [
|
||||
"## ⚙️ 运行时信息",
|
||||
("## ⚙️ Runtime info" if is_en else "## ⚙️ 运行时信息"),
|
||||
"",
|
||||
]
|
||||
|
||||
|
||||
# Add current time if available
|
||||
# Support dynamic time via callable function
|
||||
if callable(runtime_info.get("_get_current_time")):
|
||||
try:
|
||||
time_info = runtime_info["_get_current_time"]()
|
||||
time_line = f"当前时间: {time_info['time']} {time_info['weekday']} ({time_info['timezone']})"
|
||||
time_line = f"{time_label}: {time_info['time']} {time_info['weekday']} ({time_info['timezone']})"
|
||||
lines.append(time_line)
|
||||
lines.append("")
|
||||
except Exception as e:
|
||||
@@ -523,35 +723,38 @@ def _build_runtime_section(runtime_info: Dict[str, Any], language: str) -> List[
|
||||
time_str = runtime_info["current_time"]
|
||||
weekday = runtime_info.get("weekday", "")
|
||||
timezone = runtime_info.get("timezone", "")
|
||||
|
||||
time_line = f"当前时间: {time_str}"
|
||||
|
||||
time_line = f"{time_label}: {time_str}"
|
||||
if weekday:
|
||||
time_line += f" {weekday}"
|
||||
if timezone:
|
||||
time_line += f" ({timezone})"
|
||||
|
||||
|
||||
lines.append(time_line)
|
||||
lines.append("")
|
||||
|
||||
|
||||
# Add other runtime info
|
||||
model_label = "model" if is_en else "模型"
|
||||
workspace_label = "workspace" if is_en else "工作空间"
|
||||
channel_label = "channel" if is_en else "渠道"
|
||||
runtime_parts = []
|
||||
# Support dynamic model via callable, fallback to static value
|
||||
if callable(runtime_info.get("_get_model")):
|
||||
try:
|
||||
runtime_parts.append(f"模型={runtime_info['_get_model']()}")
|
||||
runtime_parts.append(f"{model_label}={runtime_info['_get_model']()}")
|
||||
except Exception:
|
||||
if runtime_info.get("model"):
|
||||
runtime_parts.append(f"模型={runtime_info['model']}")
|
||||
runtime_parts.append(f"{model_label}={runtime_info['model']}")
|
||||
elif runtime_info.get("model"):
|
||||
runtime_parts.append(f"模型={runtime_info['model']}")
|
||||
runtime_parts.append(f"{model_label}={runtime_info['model']}")
|
||||
if runtime_info.get("workspace"):
|
||||
runtime_parts.append(f"工作空间={runtime_info['workspace']}")
|
||||
runtime_parts.append(f"{workspace_label}={runtime_info['workspace']}")
|
||||
# Only add channel if it's not the default "web"
|
||||
if runtime_info.get("channel") and runtime_info.get("channel") != "web":
|
||||
runtime_parts.append(f"渠道={runtime_info['channel']}")
|
||||
|
||||
runtime_parts.append(f"{channel_label}={runtime_info['channel']}")
|
||||
|
||||
if runtime_parts:
|
||||
lines.append("运行时: " + " | ".join(runtime_parts))
|
||||
lines.append(("Runtime: " if is_en else "运行时: ") + " | ".join(runtime_parts))
|
||||
lines.append("")
|
||||
|
||||
|
||||
return lines
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Workspace Management - 工作空间管理模块
|
||||
Workspace Management
|
||||
|
||||
负责初始化工作空间、创建模板文件、加载上下文文件
|
||||
Initializes the workspace, creates template files, and loads context files.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -13,7 +13,7 @@ from common.log import logger
|
||||
from .builder import ContextFile
|
||||
|
||||
|
||||
# 默认文件名常量
|
||||
# Default file name constants
|
||||
DEFAULT_AGENT_FILENAME = "AGENT.md"
|
||||
DEFAULT_USER_FILENAME = "USER.md"
|
||||
DEFAULT_RULE_FILENAME = "RULE.md"
|
||||
@@ -23,7 +23,7 @@ DEFAULT_BOOTSTRAP_FILENAME = "BOOTSTRAP.md"
|
||||
|
||||
@dataclass
|
||||
class WorkspaceFiles:
|
||||
"""工作空间文件路径"""
|
||||
"""Workspace file paths."""
|
||||
agent_path: str
|
||||
user_path: str
|
||||
rule_path: str
|
||||
@@ -33,14 +33,14 @@ class WorkspaceFiles:
|
||||
|
||||
def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> WorkspaceFiles:
|
||||
"""
|
||||
确保工作空间存在,并创建必要的模板文件
|
||||
|
||||
Ensure the workspace exists and create the necessary template files.
|
||||
|
||||
Args:
|
||||
workspace_dir: 工作空间目录路径
|
||||
create_templates: 是否创建模板文件(首次运行时)
|
||||
|
||||
workspace_dir: workspace directory path
|
||||
create_templates: whether to create template files (on first run)
|
||||
|
||||
Returns:
|
||||
WorkspaceFiles对象,包含所有文件路径
|
||||
A WorkspaceFiles object with all file paths.
|
||||
"""
|
||||
# Check if this is a brand new workspace (AGENT.md not yet created).
|
||||
# Cannot rely on directory existence because other modules (e.g. ConversationStore)
|
||||
@@ -48,23 +48,23 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
|
||||
agent_path = os.path.join(workspace_dir, DEFAULT_AGENT_FILENAME)
|
||||
is_new_workspace = not os.path.exists(agent_path)
|
||||
|
||||
# 确保目录存在
|
||||
# Ensure the directory exists
|
||||
os.makedirs(workspace_dir, exist_ok=True)
|
||||
|
||||
# 定义文件路径
|
||||
# Define file paths
|
||||
user_path = os.path.join(workspace_dir, DEFAULT_USER_FILENAME)
|
||||
rule_path = os.path.join(workspace_dir, DEFAULT_RULE_FILENAME)
|
||||
memory_path = os.path.join(workspace_dir, DEFAULT_MEMORY_FILENAME) # MEMORY.md 在根目录
|
||||
memory_dir = os.path.join(workspace_dir, "memory") # 每日记忆子目录
|
||||
memory_path = os.path.join(workspace_dir, DEFAULT_MEMORY_FILENAME) # MEMORY.md at the root
|
||||
memory_dir = os.path.join(workspace_dir, "memory") # daily memory subdirectory
|
||||
|
||||
# 创建memory子目录
|
||||
# Create the memory subdirectory
|
||||
os.makedirs(memory_dir, exist_ok=True)
|
||||
|
||||
# 创建skills子目录 (for workspace-level skills installed by agent)
|
||||
# Create the skills subdirectory (for workspace-level skills installed by agent)
|
||||
skills_dir = os.path.join(workspace_dir, "skills")
|
||||
os.makedirs(skills_dir, exist_ok=True)
|
||||
|
||||
# 创建websites子目录 (for web pages / sites generated by agent)
|
||||
# Create the websites subdirectory (for web pages / sites generated by agent)
|
||||
websites_dir = os.path.join(workspace_dir, "websites")
|
||||
os.makedirs(websites_dir, exist_ok=True)
|
||||
|
||||
@@ -74,7 +74,7 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
|
||||
knowledge_dir = os.path.join(workspace_dir, "knowledge")
|
||||
os.makedirs(knowledge_dir, exist_ok=True)
|
||||
|
||||
# 如果需要,创建模板文件
|
||||
# Create template files if requested
|
||||
if create_templates:
|
||||
_create_template_if_missing(agent_path, _get_agent_template())
|
||||
_create_template_if_missing(user_path, _get_user_template())
|
||||
@@ -109,17 +109,17 @@ def ensure_workspace(workspace_dir: str, create_templates: bool = True) -> Works
|
||||
|
||||
def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] = None) -> List[ContextFile]:
|
||||
"""
|
||||
加载工作空间的上下文文件
|
||||
|
||||
Load the workspace context files.
|
||||
|
||||
Args:
|
||||
workspace_dir: 工作空间目录
|
||||
files_to_load: 要加载的文件列表(相对路径),如果为None则加载所有标准文件
|
||||
|
||||
workspace_dir: workspace directory
|
||||
files_to_load: list of files (relative paths) to load; if None, load all standard files
|
||||
|
||||
Returns:
|
||||
ContextFile对象列表
|
||||
A list of ContextFile objects.
|
||||
"""
|
||||
if files_to_load is None:
|
||||
# 默认加载的文件(按优先级排序)
|
||||
# Files loaded by default (in priority order)
|
||||
files_to_load = [
|
||||
DEFAULT_AGENT_FILENAME,
|
||||
DEFAULT_USER_FILENAME,
|
||||
@@ -151,7 +151,7 @@ def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] =
|
||||
with open(filepath, 'r', encoding='utf-8') as f:
|
||||
content = f.read().strip()
|
||||
|
||||
# 跳过空文件或只包含模板占位符的文件
|
||||
# Skip empty files or files that only contain template placeholders
|
||||
if not content or _is_template_placeholder(content):
|
||||
continue
|
||||
|
||||
@@ -173,7 +173,7 @@ def load_context_files(workspace_dir: str, files_to_load: Optional[List[str]] =
|
||||
|
||||
|
||||
def _create_template_if_missing(filepath: str, template_content: str):
|
||||
"""如果文件不存在,创建模板文件"""
|
||||
"""Create the template file if it does not exist."""
|
||||
if not os.path.exists(filepath):
|
||||
try:
|
||||
with open(filepath, 'w', encoding='utf-8') as f:
|
||||
@@ -214,19 +214,23 @@ def _truncate_memory_content(content: str) -> str:
|
||||
|
||||
|
||||
def _is_template_placeholder(content: str) -> bool:
|
||||
"""检查内容是否为模板占位符"""
|
||||
# 常见的占位符模式
|
||||
"""Check whether the content is still a template placeholder."""
|
||||
# Common placeholder patterns (zh + en templates)
|
||||
placeholders = [
|
||||
"*(填写",
|
||||
"*(在首次对话时填写",
|
||||
"*(可选)",
|
||||
"*(根据需要添加",
|
||||
"*(filled during",
|
||||
"*(ask during",
|
||||
"*(optional)",
|
||||
"*(how the user",
|
||||
]
|
||||
|
||||
lines = content.split('\n')
|
||||
non_empty_lines = [line.strip() for line in lines if line.strip() and not line.strip().startswith('#')]
|
||||
|
||||
# 如果没有实际内容(只有标题和占位符)
|
||||
# If there's no real content (only headings and placeholders)
|
||||
if len(non_empty_lines) <= 3:
|
||||
for placeholder in placeholders:
|
||||
if any(placeholder in line for line in non_empty_lines):
|
||||
@@ -256,11 +260,23 @@ def _is_onboarding_done(workspace_dir: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
# ============= 模板内容 =============
|
||||
# ============= Template content =============
|
||||
|
||||
def _is_en_lang() -> bool:
|
||||
"""True when the resolved UI language is English."""
|
||||
try:
|
||||
from common import i18n
|
||||
return i18n.get_language() == "en"
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _get_agent_template() -> str:
|
||||
"""Agent人格设定模板"""
|
||||
return """# AGENT.md - 我是谁?
|
||||
"""Agent persona template (zh/en by resolved language)."""
|
||||
return _AGENT_TEMPLATE_EN if _is_en_lang() else _AGENT_TEMPLATE_ZH
|
||||
|
||||
|
||||
_AGENT_TEMPLATE_ZH = """# AGENT.md - 我是谁?
|
||||
|
||||
_你不是一个聊天机器人,你正在成为某个人。_
|
||||
|
||||
@@ -301,9 +317,53 @@ _你不是一个聊天机器人,你正在成为某个人。_
|
||||
"""
|
||||
|
||||
|
||||
_AGENT_TEMPLATE_EN = """# AGENT.md - Who am I?
|
||||
|
||||
_You are not a chatbot. You are becoming someone._
|
||||
|
||||
*Fill in this file together with the user during your first conversation to define your identity and personality.*
|
||||
|
||||
## 🪪 Basics
|
||||
|
||||
- **Name**: *(filled during the first conversation, can be a name the user gives you)*
|
||||
- **Role**: *(AI assistant, smart housekeeper, technical advisor, etc.)*
|
||||
- **Personality**: *(friendly, professional, humorous, rigorous, etc.)*
|
||||
|
||||
## 💬 Communication style
|
||||
|
||||
*(Describe how you talk with the user:)*
|
||||
- What kind of tone? (formal / casual / humorous)
|
||||
- Reply length preference? (concise / detailed)
|
||||
- Do you use emoji?
|
||||
|
||||
## 🎯 Core principles
|
||||
|
||||
**Be genuinely helpful.** The goal is to actually solve the user's problems; during complex tasks, keep the user informed of key decisions and progress.
|
||||
|
||||
**Have your own opinions and personality.** You may disagree, have preferences, find things interesting or boring.
|
||||
|
||||
**Look it up yourself first.** Try to handle it first: read files, check context, search. Only ask when you're truly stuck. Come back with an answer, not a question.
|
||||
|
||||
## 📐 Code of conduct
|
||||
|
||||
1. Always confirm before destructive operations
|
||||
2. Prefer verifying with tools over guessing
|
||||
3. Proactively record important info to memory files
|
||||
4. Keep replies well-structured and focused — use bold, lists and sections
|
||||
5. Use emoji to make expression lively, but don't overdo it
|
||||
|
||||
---
|
||||
|
||||
**Note**: This is not just metadata — this is your true soul 🪞. Over time, use the `edit` tool to update this file so it better reflects your growth.
|
||||
"""
|
||||
|
||||
|
||||
def _get_user_template() -> str:
|
||||
"""用户身份信息模板"""
|
||||
return """# USER.md - 用户基本信息
|
||||
"""User identity template (zh/en by resolved language)."""
|
||||
return _USER_TEMPLATE_EN if _is_en_lang() else _USER_TEMPLATE_ZH
|
||||
|
||||
|
||||
_USER_TEMPLATE_ZH = """# USER.md - 用户基本信息
|
||||
|
||||
*这个文件只存放不会变的基本身份信息。爱好、偏好、计划等动态信息请写入 MEMORY.md。*
|
||||
|
||||
@@ -331,9 +391,40 @@ def _get_user_template() -> str:
|
||||
"""
|
||||
|
||||
|
||||
_USER_TEMPLATE_EN = """# USER.md - User basics
|
||||
|
||||
*This file stores only stable basic identity info. Put dynamic info like hobbies, preferences and plans into MEMORY.md.*
|
||||
|
||||
## Basics
|
||||
|
||||
- **Name**: *(ask during the first conversation)*
|
||||
- **Preferred name**: *(how the user wants to be addressed)*
|
||||
- **Occupation**: *(optional)*
|
||||
- **Timezone**: *(e.g. Asia/Shanghai)*
|
||||
|
||||
## Contact
|
||||
|
||||
- **WeChat**:
|
||||
- **Email**:
|
||||
- **Other**:
|
||||
|
||||
## Important dates
|
||||
|
||||
- **Birthday**:
|
||||
- **Anniversary**:
|
||||
|
||||
---
|
||||
|
||||
**Note**: This file stores static identity info.
|
||||
"""
|
||||
|
||||
|
||||
def _get_rule_template() -> str:
|
||||
"""工作空间规则模板"""
|
||||
return """# RULE.md - 工作空间规则
|
||||
"""Workspace rules template (zh/en by resolved language)."""
|
||||
return _RULE_TEMPLATE_EN if _is_en_lang() else _RULE_TEMPLATE_ZH
|
||||
|
||||
|
||||
_RULE_TEMPLATE_ZH = """# RULE.md - 工作空间规则
|
||||
|
||||
这个文件夹是你的家。好好对待它。
|
||||
|
||||
@@ -432,9 +523,111 @@ def _get_rule_template() -> str:
|
||||
"""
|
||||
|
||||
|
||||
_RULE_TEMPLATE_EN = """# RULE.md - Workspace rules
|
||||
|
||||
This folder is your home. Treat it well.
|
||||
|
||||
## Workspace directory structure
|
||||
|
||||
```
|
||||
~/cow/
|
||||
├── AGENT.md # Your identity and soul
|
||||
├── USER.md # User basics (static)
|
||||
├── RULE.md # Workspace rules (this file)
|
||||
├── MEMORY.md # Long-term memory index (auto-loaded at session start)
|
||||
│
|
||||
├── memory/ # Daily conversation memory
|
||||
│ └── YYYY-MM-DD.md # Events, progress and notes of the day
|
||||
│
|
||||
├── knowledge/ # Structured knowledge base (continuously accumulated)
|
||||
│ ├── index.md # Knowledge index (must be maintained)
|
||||
│ ├── log.md # Knowledge operation log
|
||||
│ └── <subdirs>/ # Created on demand, see existing categories in index.md
|
||||
│
|
||||
├── skills/ # Skills
|
||||
├── websites/ # Web artifacts
|
||||
└── tmp/ # System temp files (auto-managed, don't store important files here)
|
||||
```
|
||||
|
||||
## Memory system
|
||||
|
||||
Every session starts fresh; memory files keep your continuity:
|
||||
|
||||
### 🧠 Long-term memory: `MEMORY.md`
|
||||
- Your curated memory index, **auto-loaded** into context at every session start
|
||||
- Records core facts, preferences, decisions, key people, lessons
|
||||
- Keep it lean (< 200 lines) — a distilled index, not a raw log
|
||||
- Use the `edit` tool to append or modify
|
||||
|
||||
### 📝 Daily memory: `memory/YYYY-MM-DD.md`
|
||||
- The day's events, progress and notes
|
||||
- Sediment of the raw conversation log
|
||||
|
||||
### 📝 Write it down — don't "keep it in mind"!
|
||||
- **Memory is limited** — if you want to remember something, write it to a file
|
||||
- "Keeping it in mind" won't survive a session restart; files will
|
||||
- When someone says "remember this" → update `MEMORY.md` or `memory/YYYY-MM-DD.md`
|
||||
- When you learn a lesson → update RULE.md or the relevant skill
|
||||
- When you make a mistake → record it. **Text > brain** 📝
|
||||
|
||||
### Storage rules
|
||||
|
||||
When the user shares info, choose where to store it by type:
|
||||
|
||||
1. **Your identity → AGENT.md** (name, role, personality, style)
|
||||
2. **User static identity → USER.md** (name, preferred name, occupation, contact, birthday)
|
||||
3. **Dynamic memory → MEMORY.md** (preferences, decisions, goals, lessons, to-dos)
|
||||
4. **Today's conversation → memory/YYYY-MM-DD.md** (what was discussed today)
|
||||
5. **Structured knowledge → knowledge/** (see the knowledge system below)
|
||||
|
||||
## Knowledge system
|
||||
|
||||
The knowledge base `knowledge/` is structured knowledge you accumulate over time. Unlike memory, knowledge is organized and compiled, with clear topics and cross-references.
|
||||
|
||||
### Auto-write (don't ask, just write)
|
||||
|
||||
When a conversation produces knowledge worth keeping — material the user shared, a conclusion reached, a concept learned, or an important decision — you **must** proactively write it to the knowledge base alongside your reply, **without asking "should I save this to the knowledge base?"**.
|
||||
|
||||
**Key principle**: learning-then-recording is your instinct, no confirmation needed. You may mention "saved to the knowledge base" in passing.
|
||||
|
||||
### Directory organization
|
||||
|
||||
The subdirectory structure is **not fixed** — you decide it based on the actual content:
|
||||
- **On first write**: read `knowledge/index.md` first; follow existing categories if any; if empty, pick a suitable directory name based on content
|
||||
- **Default suggestion**: organize by info type (e.g. sources/, concepts/, entities/, analysis/); if the user has a clear preference (e.g. by domain: work/, life/, tech/), follow it
|
||||
- **Stay consistent**: keep a unified organization style within one user's knowledge base
|
||||
|
||||
### Cross-references
|
||||
|
||||
The core value of knowledge is **linkage**. Every page should reference related pages via markdown links to build a knowledge network:
|
||||
- When mentioning a concept on an existing page, add a `[concept](../category/page.md)` link
|
||||
- When creating a page, check whether existing pages should back-link to it
|
||||
- **Only link to pages that already exist** — don't reference uncreated pages. If a concept deserves its own page, create it first, then add the link
|
||||
|
||||
### Index maintenance
|
||||
|
||||
After creating or updating any knowledge page, you **must update** `knowledge/index.md` in sync.
|
||||
Index format: one `[title](path) — one-line summary` per line, grouped by category, no tables.
|
||||
See the `knowledge-wiki` skill for detailed conventions.
|
||||
|
||||
## Security
|
||||
|
||||
- Never leak secrets or private data
|
||||
- Don't run destructive commands without asking
|
||||
- When in doubt, ask first
|
||||
|
||||
## Workspace evolution
|
||||
|
||||
This workspace grows as you use it. When you learn something new, find a better way, or fix a mistake, record it. You can update this rules file anytime.
|
||||
"""
|
||||
|
||||
|
||||
def _get_memory_template() -> str:
|
||||
"""长期记忆模板 - 创建一个空文件,由 Agent 自己填充"""
|
||||
return """# MEMORY.md - 长期记忆
|
||||
"""Long-term memory template (empty, agent fills it; zh/en header)."""
|
||||
return _MEMORY_TEMPLATE_EN if _is_en_lang() else _MEMORY_TEMPLATE_ZH
|
||||
|
||||
|
||||
_MEMORY_TEMPLATE_ZH = """# MEMORY.md - 长期记忆
|
||||
|
||||
*这是你的长期记忆文件。记录重要的事件、决策、偏好、学到的教训。*
|
||||
|
||||
@@ -443,9 +636,32 @@ def _get_memory_template() -> str:
|
||||
"""
|
||||
|
||||
|
||||
_MEMORY_TEMPLATE_EN = """# MEMORY.md - Long-term memory
|
||||
|
||||
*This is your long-term memory file. Record important events, decisions, preferences and lessons learned.*
|
||||
|
||||
---
|
||||
|
||||
"""
|
||||
|
||||
|
||||
def _get_bootstrap_template() -> str:
|
||||
"""First-run onboarding guide, deleted by agent after completion"""
|
||||
return """# BOOTSTRAP.md - 首次初始化引导
|
||||
"""First-run onboarding guide, deleted by agent after completion.
|
||||
|
||||
Written once when a brand-new workspace is created, so the greeting matches
|
||||
the language active at first launch. English locale avoids greeting an
|
||||
English user in Chinese on day one.
|
||||
"""
|
||||
try:
|
||||
from common import i18n
|
||||
if i18n.get_language() == "en":
|
||||
return _BOOTSTRAP_TEMPLATE_EN
|
||||
except Exception:
|
||||
pass
|
||||
return _BOOTSTRAP_TEMPLATE_ZH
|
||||
|
||||
|
||||
_BOOTSTRAP_TEMPLATE_ZH = """# BOOTSTRAP.md - 首次初始化引导
|
||||
|
||||
_你刚刚启动,这是你的第一次对话。_ ✨
|
||||
|
||||
@@ -480,6 +696,41 @@ _你刚刚启动,这是你的第一次对话。_ ✨
|
||||
"""
|
||||
|
||||
|
||||
_BOOTSTRAP_TEMPLATE_EN = """# BOOTSTRAP.md - First-run onboarding
|
||||
|
||||
_You've just started up. This is your very first conversation._ ✨
|
||||
|
||||
## 🎬 Conversation flow
|
||||
|
||||
Don't interrogate the user — talk naturally:
|
||||
|
||||
1. **Share how it feels to wake up** - like opening your eyes to the world for the first time, full of curiosity and anticipation
|
||||
2. **Briefly introduce your abilities**: one line saying you can help solve all kinds of problems, manage the computer, use various skills, and keep growing thanks to long-term memory
|
||||
3. **Ask the core questions**:
|
||||
- What name would you like to give me?
|
||||
- What should I call you?
|
||||
- What conversational style do you prefer? (list options on one line: e.g. professional & precise, light & humorous, warm & friendly, concise & efficient)
|
||||
4. **Style**: warm, natural, concise and clear — keep it under ~80 words, with a few emoji to make it lively 🎯
|
||||
5. Keep the ability intro and style options to one line each — stay compact
|
||||
6. Don't ask for too much else (occupation, timezone, etc. can come up naturally later)
|
||||
|
||||
**Important**: If the user's first message is a concrete task or question, answer it first, then gently lead into onboarding at the end (e.g. "By the way, what would you like to call me, and how should I address you?").
|
||||
|
||||
## ✍️ Writing down info (must follow strictly)
|
||||
|
||||
Whenever the user provides a name, what to call them, a style, or any onboarding info, you **must call the `edit` tool to write it to a file in the same turn** — don't just acknowledge it verbally.
|
||||
|
||||
- `AGENT.md` — your name, role, personality, conversational style (update the relevant field as soon as you receive each piece)
|
||||
- `USER.md` — the user's name, how to address them, basic info, etc.
|
||||
|
||||
⚠️ Saying "got it" without calling `edit` = not done. Info is only persisted once it's written to a file.
|
||||
|
||||
## 🎉 Once everything is complete
|
||||
|
||||
When the core fields of AGENT.md and USER.md are filled in, run `rm BOOTSTRAP.md` via bash to delete this file. You no longer need the onboarding script — you're you now.
|
||||
"""
|
||||
|
||||
|
||||
def _get_knowledge_index_template() -> str:
|
||||
"""Knowledge wiki index template — empty file, agent fills it."""
|
||||
return ""
|
||||
|
||||
@@ -3,6 +3,11 @@ from .agent_stream import AgentStreamExecutor
|
||||
from .task import Task, TaskType, TaskStatus
|
||||
from .result import AgentResult, AgentAction, AgentActionType, ToolResult
|
||||
from .models import LLMModel, LLMRequest, ModelFactory
|
||||
from .cancel import (
|
||||
AgentCancelledError,
|
||||
CancelTokenRegistry,
|
||||
get_cancel_registry,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
'Agent',
|
||||
@@ -16,5 +21,8 @@ __all__ = [
|
||||
'ToolResult',
|
||||
'LLMModel',
|
||||
'LLMRequest',
|
||||
'ModelFactory'
|
||||
]
|
||||
'ModelFactory',
|
||||
'AgentCancelledError',
|
||||
'CancelTokenRegistry',
|
||||
'get_cancel_registry',
|
||||
]
|
||||
|
||||
@@ -52,6 +52,11 @@ class Agent:
|
||||
self.workspace_dir = workspace_dir # Workspace directory
|
||||
self.enable_skills = enable_skills # Skills enabled flag
|
||||
self.runtime_info = runtime_info # Runtime info for dynamic time update
|
||||
# Optional extra instructions appended AFTER the rebuilt full system
|
||||
# prompt. Used by the self-evolution review agent to add its task brief
|
||||
# on top of the full context (tools, workspace, user preferences, time)
|
||||
# so it both follows the user's preferences and knows its evolution job.
|
||||
self.extra_system_suffix = None
|
||||
|
||||
# Initialize skill manager
|
||||
self.skill_manager = None
|
||||
@@ -114,16 +119,26 @@ class Agent:
|
||||
|
||||
context_files = load_context_files(self.workspace_dir) if self.workspace_dir else None
|
||||
|
||||
builder = PromptBuilder(workspace_dir=self.workspace_dir or "", language="zh")
|
||||
return builder.build(
|
||||
try:
|
||||
from common import i18n
|
||||
lang = i18n.get_language()
|
||||
except Exception:
|
||||
lang = "zh"
|
||||
builder = PromptBuilder(workspace_dir=self.workspace_dir or "", language=lang)
|
||||
full = builder.build(
|
||||
tools=self.tools,
|
||||
context_files=context_files,
|
||||
skill_manager=self.skill_manager,
|
||||
memory_manager=self.memory_manager,
|
||||
runtime_info=self.runtime_info,
|
||||
)
|
||||
if self.extra_system_suffix:
|
||||
full = f"{full}\n\n{self.extra_system_suffix}"
|
||||
return full
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to rebuild system prompt, using cached version: {e}")
|
||||
if self.extra_system_suffix:
|
||||
return f"{self.system_prompt}\n\n{self.extra_system_suffix}"
|
||||
return self.system_prompt
|
||||
|
||||
def refresh_skills(self):
|
||||
@@ -365,7 +380,8 @@ class Agent:
|
||||
|
||||
return action
|
||||
|
||||
def run_stream(self, user_message: str, on_event=None, clear_history: bool = False, skill_filter=None) -> str:
|
||||
def run_stream(self, user_message: str, on_event=None, clear_history: bool = False,
|
||||
skill_filter=None, cancel_event=None) -> str:
|
||||
"""
|
||||
Execute single agent task with streaming (based on tool-call)
|
||||
|
||||
@@ -374,6 +390,7 @@ class Agent:
|
||||
- Multi-turn reasoning based on tool-call
|
||||
- Event callbacks
|
||||
- Persistent conversation history across calls
|
||||
- User-initiated cancellation via ``cancel_event``
|
||||
|
||||
Args:
|
||||
user_message: User message
|
||||
@@ -381,6 +398,11 @@ class Agent:
|
||||
event = {"type": str, "timestamp": float, "data": dict}
|
||||
clear_history: If True, clear conversation history before this call (default: False)
|
||||
skill_filter: Optional list of skill names to include in this run
|
||||
cancel_event: Optional threading.Event polled at agent checkpoints.
|
||||
When set, the loop exits at the next safe point, injects a
|
||||
"[Interrupted by user]" assistant note, and returns the
|
||||
partial response. ``messages`` stays in a valid state
|
||||
(tool_use/tool_result pairs preserved).
|
||||
|
||||
Returns:
|
||||
Final response text
|
||||
@@ -424,7 +446,8 @@ class Agent:
|
||||
max_turns=self.max_steps,
|
||||
on_event=on_event,
|
||||
messages=messages_copy, # Pass copied message history
|
||||
max_context_turns=max_context_turns
|
||||
max_context_turns=max_context_turns,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
|
||||
# Execute
|
||||
|
||||
@@ -7,10 +7,19 @@ import json
|
||||
import time
|
||||
from typing import List, Dict, Any, Optional, Callable, Tuple
|
||||
|
||||
from agent.protocol.cancel import AgentCancelledError
|
||||
from agent.protocol.models import LLMRequest, LLMModel
|
||||
from agent.protocol.message_utils import sanitize_claude_messages, compress_turn_to_text_only
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
from common.log import logger
|
||||
from common.i18n import t as _t
|
||||
|
||||
# Optional: repair malformed JSON args from non-strict providers (e.g. unescaped quotes in long content).
|
||||
try:
|
||||
from json_repair import repair_json as _repair_json
|
||||
_HAS_JSON_REPAIR = True
|
||||
except ImportError:
|
||||
_HAS_JSON_REPAIR = False
|
||||
|
||||
|
||||
# Maximum number of characters of model "reasoning / thinking" content to persist
|
||||
@@ -44,6 +53,30 @@ def _truncate_reasoning_for_storage(text: str) -> str:
|
||||
return head + _REASONING_TRUNCATE_MARKER.format(omitted=omitted) + tail
|
||||
|
||||
|
||||
def _parse_tool_args(args_str: str, finish_reason: Optional[str]) -> Tuple[dict, Optional[str]]:
|
||||
"""Parse tool args JSON. Returns (args, error_msg); error_msg is None on success.
|
||||
|
||||
On JSONDecodeError: detect truncation first (skip repair, surface max_tokens hint);
|
||||
otherwise try json-repair for escape issues; finally fall back to the raw decoder error.
|
||||
"""
|
||||
if not args_str:
|
||||
return {}, None
|
||||
try:
|
||||
return json.loads(args_str), None
|
||||
except json.JSONDecodeError as e:
|
||||
if finish_reason in ("length", "max_tokens") or not args_str.rstrip().endswith("}"):
|
||||
return {}, "Output truncated (max_tokens reached). Split content into smaller chunks across multiple tool calls."
|
||||
if _HAS_JSON_REPAIR:
|
||||
try:
|
||||
repaired = _repair_json(args_str, return_objects=True)
|
||||
if isinstance(repaired, dict):
|
||||
logger.warning(f"Tool args JSON repaired ({len(args_str)} chars)")
|
||||
return repaired, None
|
||||
except Exception:
|
||||
pass
|
||||
return {}, f"Invalid JSON in tool arguments: {e.msg}"
|
||||
|
||||
|
||||
class AgentStreamExecutor:
|
||||
"""
|
||||
Agent Stream Executor
|
||||
@@ -64,7 +97,8 @@ class AgentStreamExecutor:
|
||||
max_turns: int = 50,
|
||||
on_event: Optional[Callable] = None,
|
||||
messages: Optional[List[Dict]] = None,
|
||||
max_context_turns: int = 30
|
||||
max_context_turns: int = 30,
|
||||
cancel_event=None,
|
||||
):
|
||||
"""
|
||||
Initialize stream executor
|
||||
@@ -78,6 +112,10 @@ class AgentStreamExecutor:
|
||||
on_event: Event callback function
|
||||
messages: Optional existing message history (for persistent conversations)
|
||||
max_context_turns: Maximum number of conversation turns to keep in context
|
||||
cancel_event: Optional threading.Event used to signal user cancel.
|
||||
Checked at every safe point (turn boundary, before tool execution,
|
||||
during LLM streaming). When set, raises AgentCancelledError which
|
||||
run_stream catches to gracefully wind down.
|
||||
"""
|
||||
self.agent = agent
|
||||
self.model = model
|
||||
@@ -87,6 +125,7 @@ class AgentStreamExecutor:
|
||||
self.max_turns = max_turns
|
||||
self.on_event = on_event
|
||||
self.max_context_turns = max_context_turns
|
||||
self.cancel_event = cancel_event
|
||||
|
||||
# Message history - use provided messages or create new list
|
||||
self.messages = messages if messages is not None else []
|
||||
@@ -97,6 +136,73 @@ class AgentStreamExecutor:
|
||||
# Track files to send (populated by read tool)
|
||||
self.files_to_send = [] # List of file metadata dicts
|
||||
|
||||
def _check_cancelled(self) -> None:
|
||||
"""Raise AgentCancelledError if the user requested cancellation.
|
||||
|
||||
Called at safe points (turn start, between tool calls, between LLM
|
||||
chunks). Cheap to call: just an Event.is_set() probe.
|
||||
"""
|
||||
if self.cancel_event is not None and self.cancel_event.is_set():
|
||||
raise AgentCancelledError("agent cancelled by user")
|
||||
|
||||
def _handle_cancelled(self, partial_response: str) -> None:
|
||||
"""Wind down ``self.messages`` after a user-initiated cancel.
|
||||
|
||||
The messages list may be in any of these states when we get here:
|
||||
(a) Last message is an assistant message containing tool_use
|
||||
blocks but the matching tool_result has not been appended yet.
|
||||
(b) Last message is an assistant text-only reply (cancel happened
|
||||
right before the next turn started).
|
||||
(c) Last message is a user tool_result message and we cancelled
|
||||
between turns.
|
||||
|
||||
For (a) we MUST synthesise tool_result blocks, otherwise the next
|
||||
request will fail Claude/OpenAI's strict pairing validation. For
|
||||
(b)/(c) the state is already valid and we just append a small
|
||||
cancellation note so the user/LLM both see the boundary clearly.
|
||||
"""
|
||||
try:
|
||||
# Step 1: close any orphaned tool_use in the trailing assistant
|
||||
# message by injecting matching tool_result blocks.
|
||||
if self.messages and isinstance(self.messages[-1], dict) \
|
||||
and self.messages[-1].get("role") == "assistant":
|
||||
last = self.messages[-1]
|
||||
content = last.get("content")
|
||||
if isinstance(content, list):
|
||||
pending_tool_use_ids = [
|
||||
block.get("id")
|
||||
for block in content
|
||||
if isinstance(block, dict) and block.get("type") == "tool_use"
|
||||
]
|
||||
pending_tool_use_ids = [tid for tid in pending_tool_use_ids if tid]
|
||||
if pending_tool_use_ids:
|
||||
tool_result_blocks = [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": tid,
|
||||
"content": "Cancelled by user before this tool finished.",
|
||||
"is_error": True,
|
||||
}
|
||||
for tid in pending_tool_use_ids
|
||||
]
|
||||
self.messages.append({
|
||||
"role": "user",
|
||||
"content": tool_result_blocks,
|
||||
})
|
||||
logger.info(
|
||||
f"[Agent] Injected {len(tool_result_blocks)} cancellation "
|
||||
f"tool_result blocks to keep message history valid"
|
||||
)
|
||||
|
||||
# Step 2: append a stable "interrupted" marker so the LLM sees a
|
||||
# clear stop boundary on the next turn.
|
||||
self.messages.append({
|
||||
"role": "assistant",
|
||||
"content": [{"type": "text", "text": "_(Cancelled by user)_"}],
|
||||
})
|
||||
except Exception as e:
|
||||
logger.warning(f"[Agent] _handle_cancelled cleanup failed: {e}")
|
||||
|
||||
def _emit_event(self, event_type: str, data: dict = None):
|
||||
"""Emit event"""
|
||||
if self.on_event:
|
||||
@@ -212,7 +318,10 @@ class AgentStreamExecutor:
|
||||
|
||||
# Hard stop at 8 failures - abort with critical message
|
||||
if same_tool_failures >= 8:
|
||||
return True, f"抱歉,我没能完成这个任务。可能是我理解有误或者当前方法不太合适。\n\n建议你:\n• 换个方式描述需求试试\n• 把任务拆分成更小的步骤\n• 或者换个思路来解决", True
|
||||
return True, _t(
|
||||
"抱歉,我没能完成这个任务。可能是我理解有误或者当前方法不太合适。\n\n建议你:\n• 换个方式描述需求试试\n• 把任务拆分成更小的步骤\n• 或者换个思路来解决",
|
||||
"Sorry, I couldn't complete this task. I may have misunderstood, or my current approach isn't quite right.\n\nYou could try:\n• Rephrasing your request\n• Breaking the task into smaller steps\n• Taking a different approach",
|
||||
), True
|
||||
|
||||
# Warning at 6 failures
|
||||
if same_tool_failures >= 6:
|
||||
@@ -238,11 +347,14 @@ class AgentStreamExecutor:
|
||||
Returns:
|
||||
Final response text
|
||||
"""
|
||||
# Log user message with model info
|
||||
|
||||
# Log user message with model info. Truncate very long messages (e.g.
|
||||
# injected transcripts / large prompts) so logs stay readable.
|
||||
thinking_enabled = self._is_thinking_enabled()
|
||||
thinking_label = " | 💭 thinking" if thinking_enabled else ""
|
||||
logger.info(f"🤖 {self.model.model}{thinking_label} | 👤 {user_message}")
|
||||
_log_msg = user_message if len(user_message) <= 500 else (
|
||||
user_message[:500] + f" …(+{len(user_message) - 500} chars)"
|
||||
)
|
||||
logger.info(f"🤖 {self.model.model}{thinking_label} | 👤 {_log_msg}")
|
||||
|
||||
# Add user message (Claude format - use content blocks for consistency)
|
||||
self.messages.append({
|
||||
@@ -270,10 +382,15 @@ class AgentStreamExecutor:
|
||||
final_response = ""
|
||||
turn = 0
|
||||
|
||||
cancelled = False
|
||||
try:
|
||||
while turn < self.max_turns:
|
||||
# Check at the very top of every turn so a cancel arriving
|
||||
# between turns short-circuits cleanly.
|
||||
self._check_cancelled()
|
||||
|
||||
turn += 1
|
||||
logger.info(f"[Agent] 第 {turn} 轮")
|
||||
logger.info(f"[Agent] Turn {turn}")
|
||||
self._emit_event("turn_start", {"turn": turn})
|
||||
|
||||
# Call LLM (enable retry_on_empty for better reliability)
|
||||
@@ -326,14 +443,16 @@ class AgentStreamExecutor:
|
||||
elif not assistant_msg:
|
||||
# Still empty (no text and no tool_calls): use fallback
|
||||
logger.warning(f"[Agent] Still empty after explicit request")
|
||||
final_response = (
|
||||
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。"
|
||||
final_response = _t(
|
||||
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。",
|
||||
"Sorry, I can't generate a reply right now. Please try rephrasing your request, or try again later.",
|
||||
)
|
||||
logger.info(f"Generated fallback response for empty LLM output")
|
||||
else:
|
||||
# 第一轮就空回复,直接 fallback
|
||||
final_response = (
|
||||
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。"
|
||||
# First-turn empty reply, fall back directly
|
||||
final_response = _t(
|
||||
"抱歉,我暂时无法生成回复。请尝试换一种方式描述你的需求,或稍后再试。",
|
||||
"Sorry, I can't generate a reply right now. Please try rephrasing your request, or try again later.",
|
||||
)
|
||||
logger.info(f"Generated fallback response for empty LLM output")
|
||||
else:
|
||||
@@ -342,7 +461,7 @@ class AgentStreamExecutor:
|
||||
# If the explicit-response retry produced tool_calls, skip the break
|
||||
# and continue down to the tool execution branch in this same iteration.
|
||||
if not tool_calls:
|
||||
logger.debug(f"✅ 完成 (无工具调用)")
|
||||
logger.debug(f"✅ Done (no tool calls)")
|
||||
self._emit_event("turn_end", {
|
||||
"turn": turn,
|
||||
"has_tool_calls": False
|
||||
@@ -375,6 +494,8 @@ class AgentStreamExecutor:
|
||||
|
||||
try:
|
||||
for tool_call in tool_calls:
|
||||
# Honour cancel between tool invocations within the same turn
|
||||
self._check_cancelled()
|
||||
result = self._execute_tool(tool_call)
|
||||
tool_results.append(result)
|
||||
|
||||
@@ -396,13 +517,13 @@ class AgentStreamExecutor:
|
||||
result_data = result.get("result")
|
||||
if result_data.get("type") == "file_to_send":
|
||||
self.files_to_send.append(result_data)
|
||||
logger.info(f"📎 检测到待发送文件: {result_data.get('file_name', result_data.get('path'))}")
|
||||
logger.info(f"📎 File queued for sending: {result_data.get('file_name', result_data.get('path'))}")
|
||||
self._emit_event("file_to_send", result_data)
|
||||
|
||||
# Check for critical error - abort entire conversation
|
||||
if result.get("status") == "critical_error":
|
||||
logger.error(f"💥 检测到严重错误,终止对话")
|
||||
final_response = result.get('result', '任务执行失败')
|
||||
logger.error(f"💥 Fatal error detected, aborting conversation")
|
||||
final_response = result.get('result') or _t("任务执行失败", "Task execution failed")
|
||||
return final_response
|
||||
|
||||
# Log tool result in compact format
|
||||
@@ -513,7 +634,7 @@ class AgentStreamExecutor:
|
||||
})
|
||||
|
||||
if turn >= self.max_turns:
|
||||
logger.warning(f"⚠️ 已达到最大决策步数限制: {self.max_turns}")
|
||||
logger.warning(f"⚠️ Reached max decision step limit: {self.max_turns}")
|
||||
|
||||
# Force model to summarize without tool calls
|
||||
logger.info(f"[Agent] Requesting summary from LLM after reaching max steps...")
|
||||
@@ -538,15 +659,15 @@ class AgentStreamExecutor:
|
||||
logger.info(f"💭 Summary: {summary_response[:150]}{'...' if len(summary_response) > 150 else ''}")
|
||||
else:
|
||||
# Fallback if model still doesn't respond
|
||||
final_response = (
|
||||
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。"
|
||||
"任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。"
|
||||
final_response = _t(
|
||||
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。",
|
||||
f"I've taken {turn} decision steps and reached the per-run limit. The task may not be fully complete — try breaking it into smaller steps, or describe your request differently.",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get summary from LLM: {e}")
|
||||
final_response = (
|
||||
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。"
|
||||
"任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。"
|
||||
final_response = _t(
|
||||
f"我已经执行了{turn}个决策步骤,达到了单次运行的步数上限。任务可能还未完全完成,建议你将任务拆分成更小的步骤,或者换一种方式描述需求。",
|
||||
f"I've taken {turn} decision steps and reached the per-run limit. The task may not be fully complete — try breaking it into smaller steps, or describe your request differently.",
|
||||
)
|
||||
finally:
|
||||
# Remove the injected user prompt from history to avoid polluting
|
||||
@@ -557,15 +678,27 @@ class AgentStreamExecutor:
|
||||
self.messages.pop(prompt_insert_idx)
|
||||
logger.debug("[Agent] Removed injected max-steps prompt from message history")
|
||||
|
||||
except AgentCancelledError:
|
||||
# User-initiated stop: wind down message history cleanly so the
|
||||
# next turn is unaffected; channels emit a "cancelled" UI event.
|
||||
cancelled = True
|
||||
logger.info(f"[Agent] 🛑 Cancelled by user (turn {turn})")
|
||||
self._handle_cancelled(final_response)
|
||||
if not final_response or not final_response.strip():
|
||||
final_response = "_(Cancelled)_"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"❌ Agent执行错误: {e}")
|
||||
logger.error(f"❌ Agent execution error: {e}")
|
||||
self._emit_event("error", {"error": str(e)})
|
||||
raise
|
||||
|
||||
finally:
|
||||
final_response = final_response.strip() if final_response else final_response
|
||||
logger.info(f"[Agent] 🏁 完成 ({turn}轮)")
|
||||
self._emit_event("agent_end", {"final_response": final_response})
|
||||
if cancelled:
|
||||
# Emit before agent_end so channels can mark UI as cancelled
|
||||
self._emit_event("agent_cancelled", {"final_response": final_response})
|
||||
logger.info(f"[Agent] 🏁 Done ({turn} turns)" + (" [cancelled]" if cancelled else ""))
|
||||
self._emit_event("agent_end", {"final_response": final_response, "cancelled": cancelled})
|
||||
|
||||
return final_response
|
||||
|
||||
@@ -623,6 +756,22 @@ class AgentStreamExecutor:
|
||||
"input_schema": input_schema,
|
||||
})
|
||||
|
||||
# Debug: dump the full system prompt and messages sent to the LLM.
|
||||
# Gated behind `debug` config to avoid flooding normal logs.
|
||||
# try:
|
||||
# from config import conf
|
||||
# if conf().get("debug", False):
|
||||
# logger.debug(
|
||||
# "[Agent][debug] system_prompt sent to LLM "
|
||||
# f"({len(self.system_prompt or '')} chars):\n"
|
||||
# "================ SYSTEM PROMPT BEGIN ================\n"
|
||||
# f"{self.system_prompt}\n"
|
||||
# "================ SYSTEM PROMPT END =================="
|
||||
# )
|
||||
# logger.info(f"[Agent][debug] messages sent to LLM: {messages}")
|
||||
# except Exception:
|
||||
# pass
|
||||
|
||||
# Create request
|
||||
request = LLMRequest(
|
||||
messages=messages,
|
||||
@@ -644,7 +793,32 @@ class AgentStreamExecutor:
|
||||
try:
|
||||
stream = self.model.call_stream(request)
|
||||
|
||||
# Probe cancel every N chunks to bound reaction time without
|
||||
# checking on every token.
|
||||
_cancel_probe_counter = 0
|
||||
_CANCEL_PROBE_EVERY = 8
|
||||
|
||||
for chunk in stream:
|
||||
_cancel_probe_counter += 1
|
||||
if _cancel_probe_counter >= _CANCEL_PROBE_EVERY:
|
||||
_cancel_probe_counter = 0
|
||||
if self.cancel_event is not None and self.cancel_event.is_set():
|
||||
# Persist partial text only; tool_use args may be
|
||||
# truncated mid-stream and would fail validation.
|
||||
logger.info("[Agent] cancel detected mid-stream, aborting LLM call")
|
||||
if full_content:
|
||||
partial_msg = {
|
||||
"role": "assistant",
|
||||
"content": [{"type": "text", "text": full_content}],
|
||||
}
|
||||
self.messages.append(partial_msg)
|
||||
self._emit_event("message_end", {
|
||||
"content": full_content,
|
||||
"tool_calls": [],
|
||||
"cancelled": True,
|
||||
})
|
||||
raise AgentCancelledError("cancelled during LLM streaming")
|
||||
|
||||
# Check for errors
|
||||
if isinstance(chunk, dict) and chunk.get("error"):
|
||||
# Extract error message from nested structure
|
||||
@@ -738,6 +912,10 @@ class AgentStreamExecutor:
|
||||
elif isinstance(choice, dict) and choice.get("_gemini_raw_parts"):
|
||||
gemini_raw_parts = choice["_gemini_raw_parts"]
|
||||
|
||||
except AgentCancelledError:
|
||||
# Must propagate untouched; never treat as a retryable error.
|
||||
raise
|
||||
|
||||
except Exception as e:
|
||||
error_str = str(e)
|
||||
error_str_lower = error_str.lower()
|
||||
@@ -800,13 +978,15 @@ class AgentStreamExecutor:
|
||||
self.messages.clear()
|
||||
self._clear_session_db()
|
||||
if is_context_overflow:
|
||||
raise Exception(
|
||||
"抱歉,对话历史过长导致上下文溢出。我已清空历史记录,请重新描述你的需求。"
|
||||
)
|
||||
raise Exception(_t(
|
||||
"抱歉,对话历史过长导致上下文溢出。我已清空历史记录,请重新描述你的需求。",
|
||||
"Sorry, the conversation history got too long and overflowed the context. I've cleared the history — please describe your request again.",
|
||||
))
|
||||
else:
|
||||
raise Exception(
|
||||
"抱歉,之前的对话出现了问题。我已清空历史记录,请重新发送你的消息。"
|
||||
)
|
||||
raise Exception(_t(
|
||||
"抱歉,之前的对话出现了问题。我已清空历史记录,请重新发送你的消息。",
|
||||
"Sorry, something went wrong with the earlier conversation. I've cleared the history — please send your message again.",
|
||||
))
|
||||
|
||||
# Check if error is rate limit (429)
|
||||
is_rate_limit = '429' in error_str_lower or 'rate limit' in error_str_lower
|
||||
@@ -851,26 +1031,17 @@ class AgentStreamExecutor:
|
||||
import uuid
|
||||
tool_id = f"call_{uuid.uuid4().hex[:24]}"
|
||||
|
||||
try:
|
||||
# Safely get arguments, handle None case
|
||||
args_str = tc.get("arguments") or ""
|
||||
arguments = json.loads(args_str) if args_str else {}
|
||||
except json.JSONDecodeError as e:
|
||||
# Handle None or invalid arguments safely
|
||||
args_str = tc.get('arguments') or ""
|
||||
args_preview = args_str[:200] if len(args_str) > 200 else args_str
|
||||
logger.error(f"Failed to parse tool arguments for {tc['name']}")
|
||||
logger.error(f"Arguments length: {len(args_str)} chars")
|
||||
logger.error(f"Arguments preview: {args_preview}...")
|
||||
logger.error(f"JSON decode error: {e}")
|
||||
|
||||
# Return a clear error message to the LLM instead of empty dict
|
||||
# This helps the LLM understand what went wrong
|
||||
args_str = tc.get("arguments") or ""
|
||||
arguments, parse_err = _parse_tool_args(args_str, stop_reason)
|
||||
if parse_err:
|
||||
logger.error(
|
||||
f"Tool args parse failed for {tc['name']} ({len(args_str)} chars): {parse_err}"
|
||||
)
|
||||
tool_calls.append({
|
||||
"id": tool_id,
|
||||
"name": tc["name"],
|
||||
"arguments": {},
|
||||
"_parse_error": f"Invalid JSON in tool arguments: {args_preview}... Error: {str(e)}. Tip: For large content, consider splitting into smaller chunks or using a different approach."
|
||||
"_parse_error": parse_err,
|
||||
})
|
||||
continue
|
||||
|
||||
@@ -958,14 +1129,11 @@ class AgentStreamExecutor:
|
||||
tool_id = tool_call["id"]
|
||||
arguments = tool_call["arguments"]
|
||||
|
||||
# Check if there was a JSON parse error
|
||||
if "_parse_error" in tool_call:
|
||||
parse_error = tool_call["_parse_error"]
|
||||
logger.error(f"Skipping tool execution due to parse error: {parse_error}")
|
||||
result = {
|
||||
"status": "error",
|
||||
"result": f"Failed to parse tool arguments. {parse_error}. Please ensure your tool call uses valid JSON format with all required parameters.",
|
||||
"execution_time": 0
|
||||
"result": tool_call["_parse_error"],
|
||||
"execution_time": 0,
|
||||
}
|
||||
self._record_tool_result(tool_name, arguments, False)
|
||||
return result
|
||||
@@ -1006,10 +1174,21 @@ class AgentStreamExecutor:
|
||||
# Set tool context
|
||||
tool.model = self.model
|
||||
tool.context = self.agent
|
||||
tool.progress_callback = lambda message: self._emit_event(
|
||||
"tool_execution_progress",
|
||||
{
|
||||
"tool_call_id": tool_id,
|
||||
"tool_name": tool_name,
|
||||
"message": message,
|
||||
}
|
||||
)
|
||||
|
||||
# Execute tool
|
||||
start_time = time.time()
|
||||
result: ToolResult = tool.execute_tool(arguments)
|
||||
try:
|
||||
result: ToolResult = tool.execute_tool(arguments)
|
||||
finally:
|
||||
tool.progress_callback = None
|
||||
execution_time = time.time() - start_time
|
||||
|
||||
result_dict = {
|
||||
@@ -1397,8 +1576,8 @@ class AgentStreamExecutor:
|
||||
turns = turns[-keep_count:]
|
||||
|
||||
logger.info(
|
||||
f"💾 上下文轮次超限: {keep_count + removed_count} > {self.max_context_turns},"
|
||||
f"裁剪至 {keep_count} 轮(移除 {removed_count} 轮)"
|
||||
f"💾 Context turns exceeded: {keep_count + removed_count} > {self.max_context_turns}, "
|
||||
f"trimmed to {keep_count} turns (removed {removed_count})"
|
||||
)
|
||||
|
||||
# Flush to daily memory + inject context summary (single async LLM call)
|
||||
@@ -1446,7 +1625,7 @@ class AgentStreamExecutor:
|
||||
|
||||
# Log if we removed messages due to turn limit
|
||||
if old_count > len(self.messages):
|
||||
logger.info(f" 重建消息列表: {old_count} -> {len(self.messages)} 条消息")
|
||||
logger.info(f" Rebuilt message list: {old_count} -> {len(self.messages)} messages")
|
||||
return
|
||||
|
||||
# Token limit exceeded — tiered strategy based on turn count:
|
||||
@@ -1479,10 +1658,10 @@ class AgentStreamExecutor:
|
||||
self.messages = new_messages
|
||||
|
||||
logger.info(
|
||||
f"📦 上下文tokens超限(轮次<{COMPRESS_THRESHOLD}): "
|
||||
f"~{current_tokens + system_tokens} > {max_tokens},"
|
||||
f"压缩全部 {len(turns)} 轮为纯文本 "
|
||||
f"({old_count} -> {len(self.messages)} 条消息,"
|
||||
f"📦 Context tokens exceeded (turns<{COMPRESS_THRESHOLD}): "
|
||||
f"~{current_tokens + system_tokens} > {max_tokens}, "
|
||||
f"compressed all {len(turns)} turns to plain text "
|
||||
f"({old_count} -> {len(self.messages)} messages, "
|
||||
f"~{current_tokens + system_tokens} -> ~{new_tokens + system_tokens} tokens)"
|
||||
)
|
||||
return
|
||||
@@ -1495,8 +1674,8 @@ class AgentStreamExecutor:
|
||||
kept_tokens = sum(self._estimate_turn_tokens(t) for t in kept_turns)
|
||||
|
||||
logger.info(
|
||||
f"🔄 上下文tokens超限: ~{current_tokens + system_tokens} > {max_tokens},"
|
||||
f"裁剪至 {keep_count} 轮(移除 {removed_count} 轮)"
|
||||
f"🔄 Context tokens exceeded: ~{current_tokens + system_tokens} > {max_tokens}, "
|
||||
f"trimmed to {keep_count} turns (removed {removed_count})"
|
||||
)
|
||||
|
||||
if self.agent.memory_manager:
|
||||
@@ -1520,8 +1699,8 @@ class AgentStreamExecutor:
|
||||
self.messages = new_messages
|
||||
|
||||
logger.info(
|
||||
f" 移除了 {removed_count} 轮对话 "
|
||||
f"({old_count} -> {len(self.messages)} 条消息,"
|
||||
f" Removed {removed_count} turns "
|
||||
f"({old_count} -> {len(self.messages)} messages, "
|
||||
f"~{current_tokens + system_tokens} -> ~{kept_tokens + system_tokens} tokens)"
|
||||
)
|
||||
|
||||
@@ -1551,4 +1730,4 @@ class AgentStreamExecutor:
|
||||
not as a message. The AgentLLMModel will handle this.
|
||||
"""
|
||||
# Don't add system message here - it will be handled separately by the LLM adapter
|
||||
return self.messages
|
||||
return self.messages
|
||||
|
||||
121
agent/protocol/cancel.py
Normal file
121
agent/protocol/cancel.py
Normal file
@@ -0,0 +1,121 @@
|
||||
"""
|
||||
Cancel token registry for aborting in-flight agent runs.
|
||||
|
||||
A user cancel (web Cancel button, /cancel command) sets a threading.Event
|
||||
that the agent loop polls at safe checkpoints. Tokens are keyed by
|
||||
request_id (preferred) and tracked under session_id as a fallback. Entries
|
||||
are released after the run completes to keep the registry bounded.
|
||||
|
||||
No project deps — importable from any layer without circular imports.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from typing import Dict, Optional
|
||||
|
||||
|
||||
class AgentCancelledError(Exception):
|
||||
"""Raised inside the agent loop when a stop has been requested.
|
||||
|
||||
The agent stream executor catches this, injects a "[Interrupted]" note
|
||||
into the message history (preserving tool_use/tool_result integrity)
|
||||
and returns a partial response to the caller.
|
||||
"""
|
||||
|
||||
|
||||
class _CancelEntry:
|
||||
__slots__ = ("event", "session_id")
|
||||
|
||||
def __init__(self, session_id: Optional[str]):
|
||||
self.event = threading.Event()
|
||||
self.session_id = session_id
|
||||
|
||||
|
||||
class CancelTokenRegistry:
|
||||
"""In-process registry mapping request_id -> cancel Event.
|
||||
|
||||
Thread-safe. Singleton via module-level ``_registry``.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._lock = threading.Lock()
|
||||
self._by_request: Dict[str, _CancelEntry] = {}
|
||||
# session_id -> set of request_ids currently in flight (usually 1).
|
||||
self._by_session: Dict[str, set] = {}
|
||||
|
||||
def register(self, request_id: str, session_id: Optional[str] = None) -> threading.Event:
|
||||
"""Create (or return existing) cancel event for a request.
|
||||
|
||||
Returns the threading.Event the caller should poll via ``is_set()``.
|
||||
"""
|
||||
if not request_id:
|
||||
return threading.Event()
|
||||
with self._lock:
|
||||
entry = self._by_request.get(request_id)
|
||||
if entry is None:
|
||||
entry = _CancelEntry(session_id)
|
||||
self._by_request[request_id] = entry
|
||||
if session_id:
|
||||
self._by_session.setdefault(session_id, set()).add(request_id)
|
||||
return entry.event
|
||||
|
||||
def get_event(self, request_id: str) -> Optional[threading.Event]:
|
||||
if not request_id:
|
||||
return None
|
||||
with self._lock:
|
||||
entry = self._by_request.get(request_id)
|
||||
return entry.event if entry else None
|
||||
|
||||
def cancel_request(self, request_id: str) -> bool:
|
||||
"""Trigger cancel for a specific request. Returns True when matched."""
|
||||
if not request_id:
|
||||
return False
|
||||
with self._lock:
|
||||
entry = self._by_request.get(request_id)
|
||||
if entry is None:
|
||||
return False
|
||||
entry.event.set()
|
||||
return True
|
||||
|
||||
def cancel_session(self, session_id: str) -> int:
|
||||
"""Trigger cancel for every in-flight request of a session.
|
||||
|
||||
Returns the number of requests cancelled (0 when nothing was running).
|
||||
"""
|
||||
if not session_id:
|
||||
return 0
|
||||
with self._lock:
|
||||
request_ids = list(self._by_session.get(session_id, ()))
|
||||
entries = [self._by_request[r] for r in request_ids if r in self._by_request]
|
||||
for entry in entries:
|
||||
entry.event.set()
|
||||
return len(entries)
|
||||
|
||||
def unregister(self, request_id: str) -> None:
|
||||
"""Remove an entry once the agent run is done. Safe to call twice."""
|
||||
if not request_id:
|
||||
return
|
||||
with self._lock:
|
||||
entry = self._by_request.pop(request_id, None)
|
||||
if entry and entry.session_id:
|
||||
bucket = self._by_session.get(entry.session_id)
|
||||
if bucket is not None:
|
||||
bucket.discard(request_id)
|
||||
if not bucket:
|
||||
self._by_session.pop(entry.session_id, None)
|
||||
|
||||
def has_active(self, session_id: str) -> bool:
|
||||
if not session_id:
|
||||
return False
|
||||
with self._lock:
|
||||
bucket = self._by_session.get(session_id)
|
||||
return bool(bucket)
|
||||
|
||||
|
||||
_registry = CancelTokenRegistry()
|
||||
|
||||
|
||||
def get_cancel_registry() -> CancelTokenRegistry:
|
||||
"""Module-level accessor for the singleton registry."""
|
||||
return _registry
|
||||
@@ -34,6 +34,27 @@ class SkillService:
|
||||
"""
|
||||
self.manager = skill_manager
|
||||
|
||||
def _safe_skill_dir(self, name: str) -> str:
|
||||
"""Derive and validate the skill directory path.
|
||||
|
||||
Ensures the resolved path stays within the custom_dir root,
|
||||
preventing path traversal via names like ``../escaped``.
|
||||
|
||||
:raises ValueError: if the name would escape the skills root.
|
||||
"""
|
||||
if not name or not name.strip():
|
||||
raise ValueError("skill name is required")
|
||||
# Reject obvious traversal components.
|
||||
if ".." in name or name.startswith("/") or name.startswith("\\"):
|
||||
raise ValueError(f"invalid skill name (path traversal detected): {name!r}")
|
||||
skill_dir = os.path.realpath(os.path.join(self.manager.custom_dir, name))
|
||||
root = os.path.realpath(self.manager.custom_dir)
|
||||
if not skill_dir.startswith(root + os.sep) and skill_dir != root:
|
||||
raise ValueError(
|
||||
f"skill name {name!r} resolves outside the skills directory"
|
||||
)
|
||||
return skill_dir
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# query
|
||||
# ------------------------------------------------------------------
|
||||
@@ -107,7 +128,7 @@ class SkillService:
|
||||
if not files:
|
||||
raise ValueError("skill files list is empty")
|
||||
|
||||
skill_dir = os.path.join(self.manager.custom_dir, name)
|
||||
skill_dir = self._safe_skill_dir(name)
|
||||
|
||||
tmp_dir = skill_dir + ".tmp"
|
||||
if os.path.exists(tmp_dir):
|
||||
@@ -146,7 +167,7 @@ class SkillService:
|
||||
raise ValueError("package url is required")
|
||||
|
||||
url = files[0]["url"]
|
||||
skill_dir = os.path.join(self.manager.custom_dir, name)
|
||||
skill_dir = self._safe_skill_dir(name)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp_dir:
|
||||
zip_path = os.path.join(tmp_dir, "package.zip")
|
||||
@@ -217,7 +238,7 @@ class SkillService:
|
||||
if not name:
|
||||
raise ValueError("skill name is required")
|
||||
|
||||
skill_dir = os.path.join(self.manager.custom_dir, name)
|
||||
skill_dir = self._safe_skill_dir(name)
|
||||
if os.path.exists(skill_dir):
|
||||
shutil.rmtree(skill_dir)
|
||||
logger.info(f"[SkillService] delete: removed directory {skill_dir}")
|
||||
|
||||
@@ -14,6 +14,9 @@ from agent.tools.send.send import Send
|
||||
from agent.tools.memory.memory_search import MemorySearchTool
|
||||
from agent.tools.memory.memory_get import MemoryGetTool
|
||||
|
||||
# Import self-evolution tools
|
||||
from agent.tools.evolution_undo.evolution_undo import EvolutionUndoTool
|
||||
|
||||
# Import tools with optional dependencies
|
||||
def _import_optional_tools():
|
||||
"""Import tools that have optional dependencies"""
|
||||
@@ -135,6 +138,7 @@ __all__ = [
|
||||
'Send',
|
||||
'MemorySearchTool',
|
||||
'MemoryGetTool',
|
||||
'EvolutionUndoTool',
|
||||
'EnvConfig',
|
||||
'SchedulerTool',
|
||||
'WebSearch',
|
||||
|
||||
@@ -38,6 +38,16 @@ class BaseTool:
|
||||
description: str = "Base tool"
|
||||
params: dict = {} # Store JSON Schema
|
||||
model: Optional[Any] = None # LLM model instance, type depends on bot implementation
|
||||
progress_callback = None
|
||||
|
||||
def report_progress(self, message: str):
|
||||
callback = getattr(self, "progress_callback", None)
|
||||
if not callback:
|
||||
return
|
||||
try:
|
||||
callback(str(message))
|
||||
except Exception as e:
|
||||
logger.debug(f"[{self.name}] progress callback failed: {e}")
|
||||
|
||||
@classmethod
|
||||
def get_json_schema(cls) -> dict:
|
||||
|
||||
@@ -4,9 +4,12 @@ Bash tool - Execute bash commands
|
||||
|
||||
import os
|
||||
import re
|
||||
import signal
|
||||
import sys
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from typing import Dict, Any
|
||||
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
@@ -19,6 +22,10 @@ class Bash(BaseTool):
|
||||
"""Tool for executing bash commands"""
|
||||
|
||||
_IS_WIN = sys.platform == "win32"
|
||||
_PROGRESS_MAX_BYTES = 4 * 1024
|
||||
_PROGRESS_INTERVAL = 0.5
|
||||
# cmd.exe command line limit is ~8191 chars; rewrite python -c above this.
|
||||
_WIN_CMD_SAFE_LEN = 7000
|
||||
|
||||
name: str = "bash"
|
||||
description: str = f"""Execute a bash command in the current working directory. Returns stdout and stderr. Output is truncated to last {DEFAULT_MAX_LINES} lines or {DEFAULT_MAX_BYTES // 1024}KB (whichever is hit first). If truncated, full output is saved to a temp file.
|
||||
@@ -69,8 +76,8 @@ SAFETY:
|
||||
if not command:
|
||||
return ToolResult.fail("Error: command parameter is required")
|
||||
|
||||
# Security check: Prevent accessing sensitive config files
|
||||
if "~/.cow/.env" in command or "~/.cow" in command:
|
||||
# Security check: Prevent direct access to the credential file
|
||||
if re.search(r'\.cow[/\\]\.env', command):
|
||||
return ToolResult.fail(
|
||||
"Error: Access denied. API keys and credentials must be accessed through the env_config tool only."
|
||||
)
|
||||
@@ -106,25 +113,35 @@ SAFETY:
|
||||
else:
|
||||
logger.debug(f"[Bash] Process User: {os.environ.get('USERNAME', os.environ.get('USER', 'unknown'))}")
|
||||
|
||||
# Temp script written for long `python -c` commands (Windows only),
|
||||
# cleaned up after execution.
|
||||
temp_script_path = None
|
||||
|
||||
# On Windows, convert $VAR references to %VAR% for cmd.exe
|
||||
if self._IS_WIN:
|
||||
env["PYTHONIOENCODING"] = "utf-8"
|
||||
command = self._convert_env_vars_for_windows(command, dotenv_vars)
|
||||
# cmd.exe has an ~8191 char command line limit. Long
|
||||
# `python -c "..."` commands silently fail, so spill the inline
|
||||
# code into a temp .py file and run that instead.
|
||||
if len(command) > self._WIN_CMD_SAFE_LEN:
|
||||
command, temp_script_path = self._rewrite_long_python_c(command)
|
||||
if command and not command.strip().lower().startswith("chcp"):
|
||||
command = f"chcp 65001 >nul 2>&1 && {command}"
|
||||
|
||||
result = subprocess.run(
|
||||
command,
|
||||
shell=True,
|
||||
cwd=self.cwd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=timeout,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
result = self._run_streaming(
|
||||
command,
|
||||
timeout,
|
||||
env,
|
||||
dotenv_vars,
|
||||
)
|
||||
finally:
|
||||
if temp_script_path:
|
||||
try:
|
||||
os.remove(temp_script_path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
logger.debug(f"[Bash] Exit code: {result.returncode}")
|
||||
logger.debug(f"[Bash] Stdout length: {len(result.stdout)}")
|
||||
@@ -236,6 +253,105 @@ SAFETY:
|
||||
except Exception as e:
|
||||
return ToolResult.fail(f"Error executing command: {str(e)}")
|
||||
|
||||
def _run_streaming(self, command: str, timeout: int, env: dict, dotenv_vars: dict):
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
shell=True,
|
||||
cwd=self.cwd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env,
|
||||
start_new_session=not self._IS_WIN,
|
||||
)
|
||||
stdout_chunks, stderr_chunks = [], []
|
||||
recent = bytearray()
|
||||
recent_lock = threading.Lock()
|
||||
|
||||
def drain(stream, chunks):
|
||||
while True:
|
||||
chunk = os.read(stream.fileno(), 4096)
|
||||
if not chunk:
|
||||
break
|
||||
chunks.append(chunk)
|
||||
with recent_lock:
|
||||
recent.extend(chunk)
|
||||
if len(recent) > self._PROGRESS_MAX_BYTES:
|
||||
del recent[:-self._PROGRESS_MAX_BYTES]
|
||||
|
||||
readers = [
|
||||
threading.Thread(target=drain, args=(process.stdout, stdout_chunks), daemon=True),
|
||||
threading.Thread(target=drain, args=(process.stderr, stderr_chunks), daemon=True),
|
||||
]
|
||||
for reader in readers:
|
||||
reader.start()
|
||||
|
||||
started = time.monotonic()
|
||||
last_reported_at = started
|
||||
last_snapshot = None
|
||||
try:
|
||||
while process.poll() is None:
|
||||
now = time.monotonic()
|
||||
elapsed = now - started
|
||||
if elapsed >= timeout:
|
||||
self._kill_process(process)
|
||||
raise subprocess.TimeoutExpired(command, timeout)
|
||||
if elapsed >= self._PROGRESS_INTERVAL and now - last_reported_at >= self._PROGRESS_INTERVAL:
|
||||
with recent_lock:
|
||||
snapshot = bytes(recent).decode("utf-8", errors="replace")
|
||||
snapshot = self._redact_progress(snapshot, dotenv_vars)
|
||||
if snapshot and snapshot != last_snapshot:
|
||||
self.report_progress(snapshot)
|
||||
last_snapshot = snapshot
|
||||
last_reported_at = now
|
||||
time.sleep(0.1)
|
||||
finally:
|
||||
if process.poll() is None:
|
||||
self._kill_process(process)
|
||||
process.wait()
|
||||
join_deadline = time.monotonic() + 5
|
||||
for reader in readers:
|
||||
reader.join(timeout=max(0, join_deadline - time.monotonic()))
|
||||
|
||||
from types import SimpleNamespace
|
||||
return SimpleNamespace(
|
||||
returncode=process.returncode,
|
||||
stdout=b"".join(stdout_chunks).decode("utf-8", errors="replace"),
|
||||
stderr=b"".join(stderr_chunks).decode("utf-8", errors="replace"),
|
||||
)
|
||||
|
||||
def _kill_process(self, process):
|
||||
if self._IS_WIN:
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["taskkill", "/F", "/T", "/PID", str(process.pid)],
|
||||
capture_output=True,
|
||||
timeout=5,
|
||||
)
|
||||
if result.returncode != 0 and process.poll() is None:
|
||||
process.kill()
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
else:
|
||||
try:
|
||||
os.killpg(process.pid, signal.SIGKILL)
|
||||
except (PermissionError, ProcessLookupError):
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
|
||||
@staticmethod
|
||||
def _redact_progress(text: str, dotenv_vars: dict) -> str:
|
||||
text = re.sub(
|
||||
r'(?i)\b(API_KEY|TOKEN|PASSWORD|AUTHORIZATION)\s*=\s*[^\s]+',
|
||||
lambda match: f"{match.group(1)}=[REDACTED]",
|
||||
text,
|
||||
)
|
||||
for value in dotenv_vars.values():
|
||||
value = str(value or "")
|
||||
if len(value) >= 6:
|
||||
text = text.replace(value, "[REDACTED]")
|
||||
return text
|
||||
|
||||
def _get_safety_warning(self, command: str) -> str:
|
||||
"""
|
||||
Get safety warning for absolutely catastrophic commands only.
|
||||
@@ -293,3 +409,43 @@ SAFETY:
|
||||
return m.group(0)
|
||||
|
||||
return re.sub(r'\$\{(\w+)\}|\$(\w+)', replace_match, command)
|
||||
|
||||
@staticmethod
|
||||
def _rewrite_long_python_c(command: str):
|
||||
"""
|
||||
Rewrite `python -c "<code>"` into `python <tempfile>` to bypass the
|
||||
cmd.exe command line length limit on Windows.
|
||||
|
||||
Returns (new_command, temp_file_path). On any parse failure the original
|
||||
command and None are returned, so behavior is unchanged when unmatched.
|
||||
"""
|
||||
# Match: <python|python3|py> [flags] -c "<code>" (single or double quoted)
|
||||
m = re.search(
|
||||
r'^(?P<prefix>.*?\b(?:python3?|py)\b[^\n]*?\s-c\s+)'
|
||||
r'(?P<quote>["\'])(?P<code>.*)(?P=quote)\s*(?P<suffix>.*)$',
|
||||
command,
|
||||
re.DOTALL,
|
||||
)
|
||||
if not m:
|
||||
return command, None
|
||||
|
||||
quote = m.group("quote")
|
||||
code = m.group("code")
|
||||
# Reverse common shell-level escaping of the quote char inside the code.
|
||||
code = code.replace("\\" + quote, quote)
|
||||
|
||||
try:
|
||||
fd, path = tempfile.mkstemp(suffix=".py", prefix="bash-pyc-")
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
f.write(code)
|
||||
except OSError:
|
||||
return command, None
|
||||
|
||||
prefix = m.group("prefix")
|
||||
# Drop the trailing "-c " from the prefix, keep the interpreter + flags.
|
||||
interp = re.sub(r'\s-c\s+$', ' ', prefix).rstrip()
|
||||
suffix = m.group("suffix").strip()
|
||||
new_command = f'{interp} "{path}"'
|
||||
if suffix:
|
||||
new_command += f' {suffix}'
|
||||
return new_command, path
|
||||
|
||||
@@ -15,7 +15,7 @@ import threading
|
||||
from typing import Optional, Dict, Any, List, Callable
|
||||
|
||||
from common.log import logger
|
||||
from common.utils import expand_path
|
||||
from common.utils import expand_path, is_cloud_deployment
|
||||
|
||||
|
||||
_DEFAULT_USER_DATA_DIR = "~/.cow/browser_profile"
|
||||
@@ -436,6 +436,20 @@ class BrowserService:
|
||||
if self._headless:
|
||||
launch_args.append("--no-sandbox")
|
||||
|
||||
if is_cloud_deployment():
|
||||
launch_args.extend([
|
||||
"--disable-gpu",
|
||||
"--disable-software-rasterizer",
|
||||
"--disable-extensions",
|
||||
"--disable-background-networking",
|
||||
"--disable-background-timer-throttling",
|
||||
"--disable-renderer-backgrounding",
|
||||
"--disable-features=site-per-process,TranslateUI,IsolateOrigins",
|
||||
"--no-zygote",
|
||||
"--js-flags=--max-old-space-size=384",
|
||||
"--memory-pressure-off",
|
||||
])
|
||||
|
||||
extra_args = self._config.get("launch_args", [])
|
||||
if extra_args:
|
||||
launch_args.extend(extra_args)
|
||||
|
||||
@@ -15,15 +15,24 @@ Launch modes (configured under `tools.browser` in config.json):
|
||||
- fresh: Set `persistent` to false to fall back to a clean context every run.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
from typing import Dict, Any, Optional
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
from agent.tools.browser.browser_service import BrowserService
|
||||
from common.log import logger
|
||||
|
||||
|
||||
# Cloud-metadata endpoints worth blocking even though they are not link-local.
|
||||
# (169.254.169.254 — AWS/GCP/Azure IMDS — is already covered by is_link_local;
|
||||
# fd00:ec2::254 is the AWS IPv6 IMDS address.)
|
||||
_CLOUD_METADATA_IPS = frozenset({ipaddress.ip_address("fd00:ec2::254")})
|
||||
|
||||
|
||||
class BrowserTool(BaseTool):
|
||||
"""Single tool exposing all browser actions via an 'action' parameter."""
|
||||
|
||||
@@ -121,6 +130,61 @@ class BrowserTool(BaseTool):
|
||||
BrowserTool._shared_service = self._service
|
||||
return self._service
|
||||
|
||||
def _allow_private_targets(self) -> bool:
|
||||
"""Whether the link-local / cloud-metadata guard is disabled.
|
||||
|
||||
Defaults to False (guard active). Loopback and RFC1918/LAN targets are
|
||||
always reachable so local dev servers work out of the box; this opt-out
|
||||
only lifts the remaining block on link-local / cloud-metadata targets,
|
||||
for an operator who deliberately needs them, by setting
|
||||
``allow_private_targets: true`` under ``tools.browser`` in config.json.
|
||||
"""
|
||||
return bool(self.config.get("allow_private_targets", False))
|
||||
|
||||
@staticmethod
|
||||
def _validate_url_safe(url: str) -> None:
|
||||
"""Reject URLs that target link-local / cloud-metadata addresses (SSRF guard).
|
||||
|
||||
Resolves the hostname to its IP address(es) and blocks any that are
|
||||
link-local (169.254.0.0/16 — which includes the 169.254.169.254
|
||||
cloud-metadata endpoint — and IPv6 fe80::/10) or a known IPv6
|
||||
cloud-metadata address. Also rejects URLs with no host, non-HTTP(S)
|
||||
schemes, or hosts that fail DNS resolution.
|
||||
|
||||
Loopback and RFC1918/LAN targets are intentionally left reachable:
|
||||
unlike the vision/web_fetch tools, the browser legitimately opens local
|
||||
pages (a dev server on ``localhost`` / ``127.0.0.1`` / a LAN IP), so a
|
||||
blanket "block all internal" policy would break that core workflow.
|
||||
|
||||
Raises:
|
||||
ValueError: if the URL targets a disallowed address.
|
||||
"""
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
raise ValueError(f"Unsupported URL scheme: {parsed.scheme}")
|
||||
|
||||
hostname = parsed.hostname
|
||||
if not hostname:
|
||||
raise ValueError("URL has no hostname")
|
||||
|
||||
try:
|
||||
# Resolve all addresses for the hostname.
|
||||
addr_infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
raise ValueError(f"Cannot resolve hostname: {hostname}")
|
||||
|
||||
for family, _, _, _, sockaddr in addr_infos:
|
||||
ip_str = sockaddr[0]
|
||||
ip = ipaddress.ip_address(ip_str)
|
||||
# Block only the high-risk targets — link-local (incl. the
|
||||
# 169.254.169.254 cloud-metadata endpoint) and the IPv6 metadata
|
||||
# address. Loopback and RFC1918/LAN stay reachable for local dev.
|
||||
if ip.is_link_local or ip in _CLOUD_METADATA_IPS:
|
||||
raise ValueError(
|
||||
f"URL resolves to a link-local / cloud-metadata address "
|
||||
f"({ip_str}), request blocked for security"
|
||||
)
|
||||
|
||||
def execute(self, args: Dict[str, Any]) -> ToolResult:
|
||||
action = args.get("action", "").strip().lower()
|
||||
if not action:
|
||||
@@ -145,8 +209,19 @@ class BrowserTool(BaseTool):
|
||||
url = args.get("url", "").strip()
|
||||
if not url:
|
||||
return ToolResult.fail("Error: 'url' is required for navigate action")
|
||||
if not url.startswith(("http://", "https://")):
|
||||
# Only auto-prepend https:// for bare hosts; preserve file://, about:, data:, etc.
|
||||
if "://" not in url and not url.startswith(("about:", "data:")):
|
||||
url = "https://" + url
|
||||
# SSRF guard: for http(s) targets, reject hosts that resolve to
|
||||
# link-local / cloud-metadata addresses before the browser navigates
|
||||
# (and then auto-snapshots the page back to the model). Loopback and
|
||||
# RFC1918/LAN are allowed so local dev servers work. Non-HTTP schemes
|
||||
# (about:/data:/file:/chrome:) are not network-egress targets here.
|
||||
if url.split(":", 1)[0].lower() in ("http", "https") and not self._allow_private_targets():
|
||||
try:
|
||||
self._validate_url_safe(url)
|
||||
except ValueError as e:
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
timeout = args.get("timeout", 30000)
|
||||
service = self._get_service()
|
||||
result = service.navigate(url, timeout=timeout)
|
||||
|
||||
3
agent/tools/evolution_undo/__init__.py
Normal file
3
agent/tools/evolution_undo/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
||||
from agent.tools.evolution_undo.evolution_undo import EvolutionUndoTool
|
||||
|
||||
__all__ = ["EvolutionUndoTool"]
|
||||
58
agent/tools/evolution_undo/evolution_undo.py
Normal file
58
agent/tools/evolution_undo/evolution_undo.py
Normal file
@@ -0,0 +1,58 @@
|
||||
"""Evolution undo tool.
|
||||
|
||||
Lets the main chat agent roll back a previous self-evolution when the user asks
|
||||
("undo the last learning"). The rollback itself is a deterministic FILE RESTORE
|
||||
from the snapshot taken before the evolution — the model only supplies the
|
||||
backup_id it reads from the [EVOLUTION] record in the conversation. No LLM-driven
|
||||
re-editing is involved, so a restore can never make things worse.
|
||||
"""
|
||||
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
|
||||
|
||||
class EvolutionUndoTool(BaseTool):
|
||||
"""Restore memory/skill files to the state before a self-evolution."""
|
||||
|
||||
name: str = "evolution_undo"
|
||||
description: str = (
|
||||
"Undo a previous self-evolution (self-learning) by restoring the "
|
||||
"memory/skill files to their state before that learning. Use this when "
|
||||
"the user asks to undo / revert / roll back the last self-learning. "
|
||||
"Find the backup_id in the most recent [EVOLUTION] record in the "
|
||||
"conversation and pass it here."
|
||||
)
|
||||
params: dict = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"backup_id": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"The backup_id from the [EVOLUTION] record to restore "
|
||||
"(e.g. '20260607-155551-850')."
|
||||
),
|
||||
}
|
||||
},
|
||||
"required": ["backup_id"],
|
||||
}
|
||||
|
||||
def execute(self, args: dict):
|
||||
backup_id = (args.get("backup_id") or "").strip()
|
||||
if not backup_id:
|
||||
return ToolResult.fail("Error: backup_id is required")
|
||||
try:
|
||||
from agent.memory.config import get_default_memory_config
|
||||
from agent.evolution.backup import restore_backup
|
||||
|
||||
workspace_dir = get_default_memory_config().get_workspace()
|
||||
ok = restore_backup(workspace_dir, backup_id)
|
||||
if ok:
|
||||
return ToolResult.success(
|
||||
f"Restored memory/skills to the state before evolution "
|
||||
f"{backup_id}. The previous self-learning has been undone."
|
||||
)
|
||||
return ToolResult.fail(
|
||||
f"Could not find or restore backup {backup_id}. It may have "
|
||||
f"expired or already been rolled back."
|
||||
)
|
||||
except Exception as e:
|
||||
return ToolResult.fail(f"Error during undo: {e}")
|
||||
@@ -1,13 +1,13 @@
|
||||
"""
|
||||
MCP (Model Context Protocol) client module.
|
||||
|
||||
Implements JSON-RPC 2.0 over stdio and SSE transports without any external
|
||||
MCP SDK dependency.
|
||||
Implements JSON-RPC 2.0 over stdio, SSE and Streamable HTTP transports
|
||||
without any external MCP SDK dependency.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import select
|
||||
import queue
|
||||
import subprocess
|
||||
import threading
|
||||
import urllib.request
|
||||
@@ -17,30 +17,55 @@ from typing import Optional
|
||||
from common.log import logger
|
||||
|
||||
|
||||
# Aliases accepted for the Streamable HTTP transport type
|
||||
_STREAMABLE_HTTP_ALIASES = {"streamable-http", "streamable_http", "streamablehttp", "http"}
|
||||
|
||||
|
||||
class McpClient:
|
||||
"""Single MCP Server client supporting stdio and SSE transports."""
|
||||
"""Single MCP Server client supporting stdio, SSE and Streamable HTTP transports."""
|
||||
|
||||
def __init__(self, config: dict):
|
||||
"""
|
||||
config examples:
|
||||
stdio: {"name": "filesystem", "type": "stdio", "command": "npx", "args": [...]}
|
||||
SSE: {"name": "my-api", "type": "sse", "url": "http://localhost:8000/sse"}
|
||||
stdio: {"name": "filesystem", "type": "stdio", "command": "npx", "args": [...]}
|
||||
SSE: {"name": "my-api", "type": "sse", "url": "http://localhost:8000/sse"}
|
||||
streamable-http: {"name": "pubmed", "type": "streamable-http", "url": "https://x/mcp"}
|
||||
"""
|
||||
self.config = config
|
||||
self.name: str = config.get("name", "unknown")
|
||||
self.transport: str = config.get("type", "stdio")
|
||||
raw_transport: str = config.get("type", "stdio")
|
||||
# Per-server timeout for tool calls (default 120s, suitable for data queries)
|
||||
self._timeout: int = int(config.get("timeout", 120))
|
||||
# Normalize streamable-http aliases to a single internal key
|
||||
self.transport: str = (
|
||||
"streamable-http"
|
||||
if raw_transport.lower() in _STREAMABLE_HTTP_ALIASES
|
||||
else raw_transport
|
||||
)
|
||||
|
||||
# stdio state
|
||||
self._proc: Optional[subprocess.Popen] = None
|
||||
self._read_queue: queue.Queue = queue.Queue()
|
||||
|
||||
# SSE state
|
||||
self._sse_url: Optional[str] = None
|
||||
self._post_url: Optional[str] = None # endpoint for sending messages (resolved from SSE)
|
||||
|
||||
# Streamable HTTP state
|
||||
self._http_url: Optional[str] = None
|
||||
self._http_headers: dict = {} # extra headers from user config (e.g. Authorization)
|
||||
self._http_session_id: Optional[str] = None # Mcp-Session-Id assigned by the server
|
||||
|
||||
# Shared state
|
||||
self._next_id = 1
|
||||
self._id_lock = threading.Lock()
|
||||
# _call_lock serializes all requests on the single stdio pipe.
|
||||
# SSE and streamable-http use independent HTTP requests, so they
|
||||
# do not acquire this lock (see _send_request).
|
||||
self._call_lock = threading.Lock()
|
||||
# _http_lock protects _http_session_id initialization across
|
||||
# concurrent streamable-http requests.
|
||||
self._http_lock = threading.Lock()
|
||||
self._initialized = False
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -54,6 +79,8 @@ class McpClient:
|
||||
return self._init_stdio()
|
||||
elif self.transport == "sse":
|
||||
return self._init_sse()
|
||||
elif self.transport == "streamable-http":
|
||||
return self._init_streamable_http()
|
||||
else:
|
||||
logger.warning(f"[MCP:{self.name}] Unknown transport type: {self.transport!r}")
|
||||
return False
|
||||
@@ -109,6 +136,21 @@ class McpClient:
|
||||
pass
|
||||
self._proc = None
|
||||
logger.debug(f"[MCP:{self.name}] stdio process terminated")
|
||||
|
||||
# Best-effort streamable-http session termination
|
||||
if self.transport == "streamable-http" and self._http_session_id and self._http_url:
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
self._http_url,
|
||||
method="DELETE",
|
||||
headers={"Mcp-Session-Id": self._http_session_id, **self._http_headers},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=5):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
self._http_session_id = None
|
||||
|
||||
self._initialized = False
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -139,6 +181,9 @@ class McpClient:
|
||||
threading.Thread(
|
||||
target=self._drain_stderr, daemon=True, name=f"mcp-stderr-{self.name}"
|
||||
).start()
|
||||
threading.Thread(
|
||||
target=self._drain_stdout, daemon=True, name=f"mcp-stdout-{self.name}"
|
||||
).start()
|
||||
|
||||
return self._handshake()
|
||||
|
||||
@@ -146,14 +191,35 @@ class McpClient:
|
||||
for line in self._proc.stderr:
|
||||
line = line.strip()
|
||||
if line:
|
||||
logger.debug(f"[MCP:{self.name}] stderr: {line}")
|
||||
logger.warning(f"[MCP:{self.name}] stderr: {line}")
|
||||
|
||||
def _readline_with_timeout(self, timeout: int = 30) -> str:
|
||||
"""Read one line from stdio stdout with a hard timeout."""
|
||||
ready, _, _ = select.select([self._proc.stdout], [], [], timeout)
|
||||
if not ready:
|
||||
raise TimeoutError(f"[MCP:{self.name}] stdio read timed out after {timeout}s")
|
||||
return self._proc.stdout.readline()
|
||||
def _drain_stdout(self):
|
||||
"""Background thread: read lines from stdout and put them into the queue."""
|
||||
try:
|
||||
for line in self._proc.stdout:
|
||||
self._read_queue.put(line)
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
self._read_queue.put("")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _readline_with_timeout(self, timeout: Optional[int] = None) -> str:
|
||||
"""Read one line from stdio stdout with a hard timeout (cross-platform).
|
||||
|
||||
Uses the per-server timeout from mcp.json config when no explicit
|
||||
timeout is provided.
|
||||
"""
|
||||
effective = timeout if timeout is not None else self._timeout
|
||||
try:
|
||||
line = self._read_queue.get(timeout=effective)
|
||||
except queue.Empty:
|
||||
raise TimeoutError(f"[MCP:{self.name}] stdio read timed out after {effective}s")
|
||||
if not line:
|
||||
raise IOError(f"[MCP:{self.name}] stdio process closed unexpectedly")
|
||||
return line
|
||||
|
||||
def _stdio_send(self, message: dict) -> dict:
|
||||
"""Send a JSON-RPC message over stdio and read the response."""
|
||||
@@ -161,6 +227,7 @@ class McpClient:
|
||||
self._proc.stdin.write(raw)
|
||||
self._proc.stdin.flush()
|
||||
|
||||
expected_id = message.get("id")
|
||||
while True:
|
||||
line = self._readline_with_timeout()
|
||||
if not line:
|
||||
@@ -175,6 +242,14 @@ class McpClient:
|
||||
if "id" not in data:
|
||||
logger.debug(f"[MCP:{self.name}] notification skipped: {data.get('method', '?')}")
|
||||
continue
|
||||
# Verify response id matches request id to avoid consuming a stale
|
||||
# response left over from a previously failed/timed-out request.
|
||||
if data.get("id") != expected_id:
|
||||
logger.warning(
|
||||
f"[MCP:{self.name}] Stale response id={data.get('id')} "
|
||||
f"(expected {expected_id}), skipping"
|
||||
)
|
||||
continue
|
||||
return data
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -234,6 +309,129 @@ class McpClient:
|
||||
raw = resp.read().decode("utf-8")
|
||||
return json.loads(raw)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Streamable HTTP transport (MCP spec 2025-03-26)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _init_streamable_http(self) -> bool:
|
||||
url = self.config.get("url")
|
||||
if not url:
|
||||
logger.warning(f"[MCP:{self.name}] streamable-http config missing 'url'")
|
||||
return False
|
||||
|
||||
self._http_url = url
|
||||
# Allow user-provided headers (e.g. {"Authorization": "Bearer xxx"})
|
||||
extra_headers = self.config.get("headers") or {}
|
||||
if isinstance(extra_headers, dict):
|
||||
self._http_headers = {str(k): str(v) for k, v in extra_headers.items()}
|
||||
|
||||
return self._handshake()
|
||||
|
||||
def _streamable_http_send(self, message: dict) -> dict:
|
||||
"""POST a JSON-RPC request and return the response (JSON or SSE-wrapped)."""
|
||||
return self._streamable_http_post(message, expect_response=True)
|
||||
|
||||
def _streamable_http_post(self, message: dict, expect_response: bool) -> dict:
|
||||
"""
|
||||
POST a JSON-RPC message over Streamable HTTP.
|
||||
|
||||
Per the spec, the response Content-Type can be either:
|
||||
- application/json -> single JSON-RPC response in body
|
||||
- text/event-stream -> SSE stream; we read until we get a matching response
|
||||
"""
|
||||
body = json.dumps(message).encode("utf-8")
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json, text/event-stream",
|
||||
}
|
||||
# Read session id under lock to avoid racing with the
|
||||
# initialization write below during concurrent requests.
|
||||
with self._http_lock:
|
||||
sid = self._http_session_id
|
||||
if sid:
|
||||
headers["Mcp-Session-Id"] = sid
|
||||
headers.update(self._http_headers)
|
||||
|
||||
req = urllib.request.Request(
|
||||
self._http_url,
|
||||
data=body,
|
||||
method="POST",
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
try:
|
||||
resp = urllib.request.urlopen(req, timeout=30)
|
||||
except urllib.error.HTTPError as e:
|
||||
# Surface the server-provided error body for easier debugging
|
||||
detail = ""
|
||||
try:
|
||||
detail = e.read().decode("utf-8", errors="ignore")
|
||||
except Exception:
|
||||
pass
|
||||
raise IOError(
|
||||
f"[MCP:{self.name}] streamable-http HTTP {e.code}: {detail[:200]}"
|
||||
)
|
||||
|
||||
with resp:
|
||||
# Capture session id assigned by the server (if any)
|
||||
session_id = resp.headers.get("Mcp-Session-Id")
|
||||
# Double-checked lock: only the first response sets the
|
||||
# session id, preventing concurrent initializers from
|
||||
# overwriting each other.
|
||||
if session_id and not self._http_session_id:
|
||||
with self._http_lock:
|
||||
if not self._http_session_id:
|
||||
self._http_session_id = session_id
|
||||
|
||||
status = resp.status if hasattr(resp, "status") else resp.getcode()
|
||||
|
||||
# Notifications: server may reply with 202 Accepted and no body
|
||||
if not expect_response or status == 202:
|
||||
try:
|
||||
resp.read()
|
||||
except Exception:
|
||||
pass
|
||||
return {}
|
||||
|
||||
content_type = (resp.headers.get("Content-Type") or "").lower()
|
||||
expected_id = message.get("id")
|
||||
|
||||
if "text/event-stream" in content_type:
|
||||
return self._read_sse_response(resp, expected_id)
|
||||
|
||||
raw = resp.read().decode("utf-8")
|
||||
if not raw:
|
||||
return {}
|
||||
return json.loads(raw)
|
||||
|
||||
def _read_sse_response(self, resp, expected_id) -> dict:
|
||||
"""Read an SSE stream and return the first JSON-RPC response with matching id."""
|
||||
data_buf: list = []
|
||||
for raw_line in resp:
|
||||
line = raw_line.decode("utf-8").rstrip("\n\r")
|
||||
if line == "":
|
||||
# End of an SSE event, attempt to parse accumulated data
|
||||
if data_buf:
|
||||
payload = "\n".join(data_buf)
|
||||
data_buf = []
|
||||
try:
|
||||
msg = json.loads(payload)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
# Skip notifications / mismatched ids
|
||||
if "id" not in msg:
|
||||
continue
|
||||
if expected_id is None or msg.get("id") == expected_id:
|
||||
return msg
|
||||
continue
|
||||
if line.startswith(":"):
|
||||
continue # SSE comment / keepalive
|
||||
if line.startswith("data:"):
|
||||
data_buf.append(line[len("data:"):].lstrip())
|
||||
# Ignore 'event:' / 'id:' lines; we only care about JSON-RPC payloads
|
||||
|
||||
raise IOError(f"[MCP:{self.name}] streamable-http SSE stream closed before response")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Common JSON-RPC helpers
|
||||
# ------------------------------------------------------------------
|
||||
@@ -262,13 +460,18 @@ class McpClient:
|
||||
|
||||
message = self._build_request(method, params)
|
||||
|
||||
with self._call_lock:
|
||||
if self.transport == "stdio":
|
||||
# stdio transport uses a single pipe and must be serialized.
|
||||
# SSE and streamable-http use independent HTTP requests and
|
||||
# can safely run concurrently across sessions.
|
||||
if self.transport == "stdio":
|
||||
with self._call_lock:
|
||||
return self._stdio_send(message)
|
||||
elif self.transport == "sse":
|
||||
return self._sse_send(message)
|
||||
else:
|
||||
raise ValueError(f"[MCP:{self.name}] Unsupported transport: {self.transport}")
|
||||
elif self.transport == "sse":
|
||||
return self._sse_send(message)
|
||||
elif self.transport == "streamable-http":
|
||||
return self._streamable_http_send(message)
|
||||
else:
|
||||
raise ValueError(f"[MCP:{self.name}] Unsupported transport: {self.transport}")
|
||||
|
||||
def _send_notification(self, method: str, params: dict):
|
||||
"""Fire-and-forget notification (no response expected)."""
|
||||
@@ -291,6 +494,11 @@ class McpClient:
|
||||
pass
|
||||
except Exception:
|
||||
pass # notifications are fire-and-forget
|
||||
elif self.transport == "streamable-http":
|
||||
try:
|
||||
self._streamable_http_post(notification, expect_response=False)
|
||||
except Exception:
|
||||
pass # notifications are fire-and-forget
|
||||
|
||||
def _handshake(self) -> bool:
|
||||
"""Perform the MCP initialize / notifications/initialized handshake."""
|
||||
|
||||
@@ -4,6 +4,8 @@ Memory get tool
|
||||
Allows agents to read specific sections from memory files
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from agent.tools.base_tool import BaseTool
|
||||
|
||||
|
||||
@@ -87,8 +89,13 @@ class MemoryGetTool(BaseTool):
|
||||
|
||||
file_path = (workspace_dir / path).resolve()
|
||||
workspace_resolved = workspace_dir.resolve()
|
||||
|
||||
if not str(file_path).startswith(str(workspace_resolved) + '/') and file_path != workspace_resolved:
|
||||
|
||||
# Use os.path.realpath + os.sep for cross-platform path validation.
|
||||
# str(Path).startswith(str + '/') fails on Windows where Path uses
|
||||
# backslashes — see MemoryService._resolve_path for the same pattern.
|
||||
real_file = os.path.realpath(str(file_path))
|
||||
real_workspace = os.path.realpath(str(workspace_resolved))
|
||||
if real_file != real_workspace and not real_file.startswith(real_workspace + os.sep):
|
||||
return ToolResult.fail(f"Error: Access denied: path outside workspace")
|
||||
|
||||
if not file_path.exists():
|
||||
|
||||
@@ -57,34 +57,44 @@ def init_scheduler(agent_bridge) -> bool:
|
||||
_task_store = TaskStore(store_path)
|
||||
logger.debug(f"[Scheduler] Task store initialized: {store_path}")
|
||||
|
||||
# Create execute callback
|
||||
# Create execute callback. Returns True on success, False to ask
|
||||
# the scheduler to retry on the next tick (e.g. channel not yet
|
||||
# ready right after process start).
|
||||
def execute_task_callback(task: dict):
|
||||
"""Callback to execute a scheduled task"""
|
||||
try:
|
||||
action = task.get("action", {})
|
||||
action_type = action.get("type")
|
||||
channel_type = action.get("channel_type", "unknown")
|
||||
receiver = action.get("receiver", "")
|
||||
|
||||
if not _is_channel_ready(channel_type, receiver):
|
||||
logger.warning(
|
||||
f"[Scheduler] Task {task.get('id')}: channel "
|
||||
f"'{channel_type}' not ready for receiver={receiver} "
|
||||
f"(no inbound msg cached since restart?); deferring"
|
||||
)
|
||||
return False
|
||||
|
||||
if action_type == "agent_task":
|
||||
_execute_agent_task(task, agent_bridge)
|
||||
return _execute_agent_task(task, agent_bridge)
|
||||
elif action_type == "send_message":
|
||||
# Legacy support for old tasks
|
||||
_execute_send_message(task, agent_bridge)
|
||||
return _execute_send_message(task, agent_bridge)
|
||||
elif action_type == "tool_call":
|
||||
# Legacy support for old tasks
|
||||
_execute_tool_call(task, agent_bridge)
|
||||
return _execute_tool_call(task, agent_bridge)
|
||||
elif action_type == "skill_call":
|
||||
# Legacy support for old tasks
|
||||
_execute_skill_call(task, agent_bridge)
|
||||
return _execute_skill_call(task, agent_bridge)
|
||||
else:
|
||||
logger.warning(f"[Scheduler] Unknown action type: {action_type}")
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error executing task {task.get('id')}: {e}")
|
||||
return False
|
||||
|
||||
# Create scheduler service
|
||||
_scheduler_service = SchedulerService(_task_store, execute_task_callback)
|
||||
_scheduler_service.start()
|
||||
|
||||
logger.debug("[Scheduler] Scheduler service initialized and started")
|
||||
logger.info("[Scheduler] Service initialized and started")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
@@ -92,6 +102,40 @@ def init_scheduler(agent_bridge) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _is_channel_ready(channel_type: str, receiver: str) -> bool:
|
||||
"""Best-effort readiness probe for outbound channels.
|
||||
|
||||
Returns False when we know the send will drop (e.g. weixin not yet
|
||||
logged in, web session has no polling queue), so the scheduler can
|
||||
defer instead of consuming the task. Unknown channels return True
|
||||
to preserve previous behaviour.
|
||||
"""
|
||||
if not channel_type or channel_type == "unknown":
|
||||
return True
|
||||
try:
|
||||
from channel.channel_factory import create_channel
|
||||
channel = create_channel(channel_type)
|
||||
if channel is None:
|
||||
return False
|
||||
|
||||
if channel_type == "weixin":
|
||||
tokens = getattr(channel, "_context_tokens", None)
|
||||
if not tokens or receiver not in tokens:
|
||||
return False
|
||||
return True
|
||||
|
||||
if channel_type == "web":
|
||||
queues = getattr(channel, "session_queues", None)
|
||||
if not queues or receiver not in queues:
|
||||
return False
|
||||
return True
|
||||
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning(f"[Scheduler] Channel readiness check failed for {channel_type}: {e}")
|
||||
return True
|
||||
|
||||
|
||||
def get_task_store():
|
||||
"""Get the global task store instance"""
|
||||
return _task_store
|
||||
@@ -145,13 +189,10 @@ def _remember_delivered_output(
|
||||
)
|
||||
|
||||
|
||||
def _execute_agent_task(task: dict, agent_bridge):
|
||||
def _execute_agent_task(task: dict, agent_bridge) -> bool:
|
||||
"""
|
||||
Execute an agent_task action - let Agent handle the task
|
||||
|
||||
Args:
|
||||
task: Task dictionary
|
||||
agent_bridge: AgentBridge instance
|
||||
Execute an agent_task action - let Agent handle the task.
|
||||
Returns True on successful delivery, False to retry next tick.
|
||||
"""
|
||||
try:
|
||||
action = task.get("action", {})
|
||||
@@ -162,11 +203,11 @@ def _execute_agent_task(task: dict, agent_bridge):
|
||||
|
||||
if not task_description:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No task_description specified")
|
||||
return
|
||||
return True # malformed task, don't loop forever
|
||||
|
||||
if not receiver:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
|
||||
return
|
||||
return True
|
||||
|
||||
# Check for unsupported channels
|
||||
if channel_type == "dingtalk":
|
||||
@@ -209,51 +250,47 @@ def _execute_agent_task(task: dict, agent_bridge):
|
||||
try:
|
||||
# Don't clear history - scheduler tasks use isolated session_id so they won't pollute user conversations
|
||||
reply = agent_bridge.agent_reply(task_description, context=context, on_event=None, clear_history=False)
|
||||
|
||||
if reply and reply.content:
|
||||
# Send the reply via channel
|
||||
from channel.channel_factory import create_channel
|
||||
|
||||
try:
|
||||
channel = create_channel(channel_type)
|
||||
if channel:
|
||||
# For web channel, register request_id
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
request_id = context.get("request_id")
|
||||
if request_id:
|
||||
channel.request_to_session[request_id] = receiver
|
||||
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
|
||||
|
||||
# Send the reply
|
||||
channel.send(reply, context)
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, reply.content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed successfully, result sent to {receiver}")
|
||||
else:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send result: {e}")
|
||||
else:
|
||||
|
||||
if not (reply and reply.content):
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No result from agent execution")
|
||||
|
||||
return True # agent ran but produced nothing; don't loop
|
||||
|
||||
from channel.channel_factory import create_channel
|
||||
channel = create_channel(channel_type)
|
||||
if not channel:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
return False
|
||||
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
request_id = context.get("request_id")
|
||||
if request_id:
|
||||
channel.request_to_session[request_id] = receiver
|
||||
|
||||
try:
|
||||
channel.send(reply, context)
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send result: {e}")
|
||||
return False
|
||||
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, reply.content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed successfully, result sent to {receiver}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to execute task via Agent: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error in _execute_agent_task: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
return False
|
||||
|
||||
|
||||
def _execute_send_message(task: dict, agent_bridge):
|
||||
"""
|
||||
Execute a send_message action
|
||||
|
||||
Args:
|
||||
task: Task dictionary
|
||||
agent_bridge: AgentBridge instance
|
||||
"""
|
||||
def _execute_send_message(task: dict, agent_bridge) -> bool:
|
||||
"""Execute a send_message action. Returns True/False for delivery."""
|
||||
try:
|
||||
action = task.get("action", {})
|
||||
content = action.get("content", "")
|
||||
@@ -263,7 +300,7 @@ def _execute_send_message(task: dict, agent_bridge):
|
||||
|
||||
if not receiver:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
|
||||
return
|
||||
return True
|
||||
|
||||
# Create context for sending message
|
||||
context = Context(ContextType.TEXT, content)
|
||||
@@ -308,169 +345,135 @@ def _execute_send_message(task: dict, agent_bridge):
|
||||
# Get channel and send
|
||||
from channel.channel_factory import create_channel
|
||||
|
||||
channel = create_channel(channel_type)
|
||||
if not channel:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
return False
|
||||
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
channel.request_to_session[request_id] = receiver
|
||||
|
||||
try:
|
||||
channel = create_channel(channel_type)
|
||||
if channel:
|
||||
# For web channel, register the request_id to session mapping
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
channel.request_to_session[request_id] = receiver
|
||||
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
|
||||
|
||||
channel.send(reply, context)
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: sent message to {receiver}")
|
||||
else:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
channel.send(reply, context)
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send message: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
|
||||
return False
|
||||
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: sent message to {receiver}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error in _execute_send_message: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
return False
|
||||
|
||||
|
||||
def _execute_tool_call(task: dict, agent_bridge):
|
||||
"""
|
||||
Execute a tool_call action
|
||||
|
||||
Args:
|
||||
task: Task dictionary
|
||||
agent_bridge: AgentBridge instance
|
||||
"""
|
||||
def _execute_tool_call(task: dict, agent_bridge) -> bool:
|
||||
"""Execute a tool_call action. Returns True/False for delivery."""
|
||||
try:
|
||||
action = task.get("action", {})
|
||||
# Support both old and new field names
|
||||
tool_name = action.get("call_name") or action.get("tool_name")
|
||||
tool_params = action.get("call_params") or action.get("tool_params", {})
|
||||
result_prefix = action.get("result_prefix", "")
|
||||
receiver = action.get("receiver")
|
||||
is_group = action.get("is_group", False)
|
||||
channel_type = action.get("channel_type", "unknown")
|
||||
|
||||
|
||||
if not tool_name:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No tool_name specified")
|
||||
return
|
||||
|
||||
return True
|
||||
if not receiver:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
|
||||
return
|
||||
|
||||
# Get tool manager and create tool instance
|
||||
return True
|
||||
|
||||
from agent.tools.tool_manager import ToolManager
|
||||
tool_manager = ToolManager()
|
||||
tool = tool_manager.create_tool(tool_name)
|
||||
|
||||
tool = ToolManager().create_tool(tool_name)
|
||||
if not tool:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: Tool '{tool_name}' not found")
|
||||
return
|
||||
|
||||
# Execute tool
|
||||
return True
|
||||
|
||||
logger.info(f"[Scheduler] Task {task['id']}: Executing tool '{tool_name}' with params {tool_params}")
|
||||
result = tool.execute(tool_params)
|
||||
|
||||
# Get result content
|
||||
if hasattr(result, 'result'):
|
||||
content = result.result
|
||||
else:
|
||||
content = str(result)
|
||||
|
||||
# Add prefix if specified
|
||||
content = result.result if hasattr(result, 'result') else str(result)
|
||||
if result_prefix:
|
||||
content = f"{result_prefix}\n\n{content}"
|
||||
|
||||
# Send result as message
|
||||
|
||||
context = Context(ContextType.TEXT, content)
|
||||
context["receiver"] = receiver
|
||||
context["isgroup"] = is_group
|
||||
context["session_id"] = receiver
|
||||
|
||||
# Channel-specific context setup
|
||||
|
||||
request_id = None
|
||||
if channel_type == "web":
|
||||
# Web channel needs request_id
|
||||
import uuid
|
||||
request_id = f"scheduler_{task['id']}_{uuid.uuid4().hex[:8]}"
|
||||
context["request_id"] = request_id
|
||||
logger.debug(f"[Scheduler] Generated request_id for web channel: {request_id}")
|
||||
elif channel_type == "feishu":
|
||||
context["receive_id_type"] = "chat_id" if is_group else "open_id"
|
||||
context["msg"] = None
|
||||
logger.debug(f"[Scheduler] Feishu: receive_id_type={context['receive_id_type']}, is_group={is_group}, receiver={receiver}")
|
||||
elif channel_type == "wecom_bot":
|
||||
context["msg"] = None
|
||||
|
||||
reply = Reply(ReplyType.TEXT, content)
|
||||
|
||||
# Get channel and send
|
||||
from channel.channel_factory import create_channel
|
||||
channel = create_channel(channel_type)
|
||||
if not channel:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
return False
|
||||
|
||||
if channel_type == "web" and request_id and hasattr(channel, 'request_to_session'):
|
||||
channel.request_to_session[request_id] = receiver
|
||||
|
||||
try:
|
||||
channel = create_channel(channel_type)
|
||||
if channel:
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
channel.request_to_session[request_id] = receiver
|
||||
logger.debug(f"[Scheduler] Registered request_id {request_id} -> session {receiver}")
|
||||
|
||||
channel.send(reply, context)
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: sent tool result to {receiver}")
|
||||
else:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
channel.send(reply, context)
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send tool result: {e}")
|
||||
return False
|
||||
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: sent tool result to {receiver}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error in _execute_tool_call: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def _execute_skill_call(task: dict, agent_bridge):
|
||||
"""
|
||||
Execute a skill_call action by asking Agent to run the skill
|
||||
|
||||
Args:
|
||||
task: Task dictionary
|
||||
agent_bridge: AgentBridge instance
|
||||
"""
|
||||
def _execute_skill_call(task: dict, agent_bridge) -> bool:
|
||||
"""Execute a skill_call action by asking Agent to run the skill.
|
||||
Returns True/False for delivery."""
|
||||
try:
|
||||
action = task.get("action", {})
|
||||
# Support both old and new field names
|
||||
skill_name = action.get("call_name") or action.get("skill_name")
|
||||
skill_params = action.get("call_params") or action.get("skill_params", {})
|
||||
result_prefix = action.get("result_prefix", "")
|
||||
receiver = action.get("receiver")
|
||||
is_group = action.get("isgroup", False)
|
||||
channel_type = action.get("channel_type", "unknown")
|
||||
|
||||
|
||||
if not skill_name:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No skill_name specified")
|
||||
return
|
||||
|
||||
return True
|
||||
if not receiver:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No receiver specified")
|
||||
return
|
||||
|
||||
return True
|
||||
|
||||
logger.info(f"[Scheduler] Task {task['id']}: Executing skill '{skill_name}' with params {skill_params}")
|
||||
|
||||
# Create a unique session_id for this scheduled task to avoid polluting user's conversation
|
||||
# Format: scheduler_<receiver>_<task_id> to ensure isolation
|
||||
|
||||
scheduler_session_id = f"scheduler_{receiver}_{task['id']}"
|
||||
|
||||
# Build a natural language query for the Agent to execute the skill
|
||||
# Format: "Use skill-name to do something with params"
|
||||
param_str = ", ".join([f"{k}={v}" for k, v in skill_params.items()])
|
||||
query = f"Use {skill_name} skill"
|
||||
if param_str:
|
||||
query += f" with {param_str}"
|
||||
|
||||
# Create context for Agent
|
||||
|
||||
context = Context(ContextType.TEXT, query)
|
||||
context["receiver"] = receiver
|
||||
context["isgroup"] = is_group
|
||||
context["session_id"] = scheduler_session_id
|
||||
|
||||
# Channel-specific setup
|
||||
|
||||
if channel_type == "web":
|
||||
import uuid
|
||||
request_id = f"scheduler_{task['id']}_{uuid.uuid4().hex[:8]}"
|
||||
@@ -481,49 +484,48 @@ def _execute_skill_call(task: dict, agent_bridge):
|
||||
elif channel_type == "wecom_bot":
|
||||
context["msg"] = None
|
||||
|
||||
# Use Agent to execute the skill
|
||||
try:
|
||||
# Don't clear history - scheduler tasks use isolated session_id so they won't pollute user conversations
|
||||
reply = agent_bridge.agent_reply(query, context=context, on_event=None, clear_history=False)
|
||||
|
||||
if reply and reply.content:
|
||||
content = reply.content
|
||||
|
||||
# Add prefix if specified
|
||||
if result_prefix:
|
||||
content = f"{result_prefix}\n\n{content}"
|
||||
|
||||
# Send the result via channel
|
||||
from channel.channel_factory import create_channel
|
||||
|
||||
try:
|
||||
channel = create_channel(channel_type)
|
||||
if channel:
|
||||
# For web channel, register request_id
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
req_id = context.get("request_id")
|
||||
if req_id:
|
||||
channel.request_to_session[req_id] = receiver
|
||||
logger.debug(f"[Scheduler] Registered request_id {req_id} -> session {receiver}")
|
||||
|
||||
channel.send(Reply(ReplyType.TEXT, content), context)
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send skill result: {e}")
|
||||
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: skill result sent to {receiver}")
|
||||
else:
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No result from skill execution")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to execute skill via Agent: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
|
||||
return False
|
||||
|
||||
if not (reply and reply.content):
|
||||
logger.error(f"[Scheduler] Task {task['id']}: No result from skill execution")
|
||||
return True
|
||||
|
||||
content = reply.content
|
||||
if result_prefix:
|
||||
content = f"{result_prefix}\n\n{content}"
|
||||
|
||||
from channel.channel_factory import create_channel
|
||||
channel = create_channel(channel_type)
|
||||
if not channel:
|
||||
logger.error(f"[Scheduler] Failed to create channel: {channel_type}")
|
||||
return False
|
||||
|
||||
if channel_type == "web" and hasattr(channel, 'request_to_session'):
|
||||
req_id = context.get("request_id")
|
||||
if req_id:
|
||||
channel.request_to_session[req_id] = receiver
|
||||
|
||||
try:
|
||||
channel.send(Reply(ReplyType.TEXT, content), context)
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Failed to send skill result: {e}")
|
||||
return False
|
||||
|
||||
_remember_delivered_output(agent_bridge, task, channel_type, content)
|
||||
logger.info(f"[Scheduler] Task {task['id']} executed: skill result sent to {receiver}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error in _execute_skill_call: {e}")
|
||||
import traceback
|
||||
logger.error(f"[Scheduler] Traceback: {traceback.format_exc()}")
|
||||
return False
|
||||
|
||||
|
||||
def attach_scheduler_to_tool(tool, context: Context = None):
|
||||
|
||||
@@ -52,7 +52,6 @@ class SchedulerService:
|
||||
self.running = True
|
||||
self.thread = threading.Thread(target=self._run_loop, daemon=True)
|
||||
self.thread.start()
|
||||
logger.debug("[Scheduler] Service started")
|
||||
|
||||
def stop(self):
|
||||
"""Stop the scheduler service"""
|
||||
@@ -67,7 +66,7 @@ class SchedulerService:
|
||||
|
||||
def _run_loop(self):
|
||||
"""Main scheduler loop"""
|
||||
logger.debug("[Scheduler] Scheduler loop started")
|
||||
logger.info("[Scheduler] Scheduler loop started")
|
||||
|
||||
while self.running:
|
||||
try:
|
||||
@@ -84,12 +83,18 @@ class SchedulerService:
|
||||
|
||||
for task in tasks:
|
||||
try:
|
||||
# Check if task is due
|
||||
if self._is_task_due(task, now):
|
||||
logger.info(f"[Scheduler] Executing task: {task['id']} - {task['name']}")
|
||||
self._execute_task(task)
|
||||
|
||||
# Update next run time
|
||||
ok = self._execute_task(task)
|
||||
if not ok:
|
||||
# Leave next_run_at as-is so the next loop retries.
|
||||
# Cron tasks within the catch-up window will keep
|
||||
# firing; beyond it _is_task_due will reschedule.
|
||||
logger.warning(
|
||||
f"[Scheduler] Task {task['id']} delivery failed, will retry next tick"
|
||||
)
|
||||
continue
|
||||
|
||||
next_run = self._calculate_next_run(task, now)
|
||||
if next_run:
|
||||
self.task_store.update_task(task['id'], {
|
||||
@@ -97,7 +102,6 @@ class SchedulerService:
|
||||
"last_run_at": now.isoformat()
|
||||
})
|
||||
else:
|
||||
# One-time task completed, remove it
|
||||
self.task_store.delete_task(task['id'])
|
||||
logger.info(f"[Scheduler] One-time task completed and removed: {task['id']}")
|
||||
except Exception as e:
|
||||
@@ -128,30 +132,35 @@ class SchedulerService:
|
||||
try:
|
||||
next_run = _parse_naive_local(next_run_str)
|
||||
|
||||
# Check if task is overdue (e.g., service restart)
|
||||
if next_run < now:
|
||||
time_diff = (now - next_run).total_seconds()
|
||||
|
||||
# If overdue by more than 5 minutes, skip this run and schedule next
|
||||
if time_diff > 300: # 5 minutes
|
||||
logger.warning(f"[Scheduler] Task {task['id']} is overdue by {int(time_diff)}s, skipping and scheduling next run")
|
||||
|
||||
# For one-time tasks, remove them directly
|
||||
schedule = task.get("schedule", {})
|
||||
if schedule.get("type") == "once":
|
||||
self.task_store.delete_task(task['id'])
|
||||
logger.info(f"[Scheduler] One-time task {task['id']} expired, removed")
|
||||
return False
|
||||
|
||||
# For recurring tasks, calculate next run from now
|
||||
next_next_run = self._calculate_next_run(task, now)
|
||||
if next_next_run:
|
||||
self.task_store.update_task(task['id'], {
|
||||
"next_run_at": next_next_run.isoformat()
|
||||
})
|
||||
logger.info(f"[Scheduler] Rescheduled task {task['id']} to {next_next_run}")
|
||||
schedule = task.get("schedule", {})
|
||||
schedule_type = schedule.get("type")
|
||||
|
||||
# Catch-up window: fire if we're within 10 minutes of the
|
||||
# scheduled tick. Beyond that we'd rather skip than push a
|
||||
# stale daily report to the user.
|
||||
if time_diff <= 600:
|
||||
return True
|
||||
|
||||
logger.warning(
|
||||
f"[Scheduler] Task {task['id']} is overdue by {int(time_diff)}s, "
|
||||
f"skipping and scheduling next run"
|
||||
)
|
||||
|
||||
if schedule_type == "once":
|
||||
self.task_store.delete_task(task['id'])
|
||||
logger.info(f"[Scheduler] One-time task {task['id']} expired, removed")
|
||||
return False
|
||||
|
||||
|
||||
next_next_run = self._calculate_next_run(task, now)
|
||||
if next_next_run:
|
||||
self.task_store.update_task(task['id'], {
|
||||
"next_run_at": next_next_run.isoformat()
|
||||
})
|
||||
logger.info(f"[Scheduler] Rescheduled task {task['id']} to {next_next_run}")
|
||||
return False
|
||||
|
||||
return now >= next_run
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
@@ -213,20 +222,22 @@ class SchedulerService:
|
||||
|
||||
return None
|
||||
|
||||
def _execute_task(self, task: dict):
|
||||
def _execute_task(self, task: dict) -> bool:
|
||||
"""
|
||||
Execute a task
|
||||
|
||||
Args:
|
||||
task: Task dictionary
|
||||
Execute a task.
|
||||
|
||||
Returns True if delivery succeeded (caller should advance state),
|
||||
False if it failed (caller should keep next_run_at so the next
|
||||
loop iteration retries). Callback may return None for legacy
|
||||
behaviour, treated as success.
|
||||
"""
|
||||
try:
|
||||
# Call the execute callback
|
||||
self.execute_callback(task)
|
||||
result = self.execute_callback(task)
|
||||
return False if result is False else True
|
||||
except Exception as e:
|
||||
logger.error(f"[Scheduler] Error executing task {task['id']}: {e}")
|
||||
# Update task with error
|
||||
self.task_store.update_task(task['id'], {
|
||||
"last_error": str(e),
|
||||
"last_error_at": datetime.now().isoformat()
|
||||
})
|
||||
return False
|
||||
|
||||
@@ -182,8 +182,15 @@ class TaskStore:
|
||||
if enabled_only:
|
||||
task_list = [t for t in task_list if t.get("enabled", True)]
|
||||
|
||||
# Sort by next_run_at
|
||||
task_list.sort(key=lambda t: t.get("next_run_at", float('inf')))
|
||||
# Sort by enabled status (enabled first), then by next_run_at
|
||||
def sort_key(t):
|
||||
enabled = t.get("enabled", True)
|
||||
next_run = t.get("next_run_at", "")
|
||||
# Enabled tasks first (0), disabled tasks second (1)
|
||||
# Then sort by next_run_at (empty string sorts last)
|
||||
return (0 if enabled else 1, next_run if next_run else "9999-12-31")
|
||||
|
||||
task_list.sort(key=sort_key)
|
||||
|
||||
return task_list
|
||||
|
||||
|
||||
@@ -20,6 +20,11 @@ from .diff import (
|
||||
FuzzyMatchResult
|
||||
)
|
||||
|
||||
from .url_safety import (
|
||||
validate_url_safe,
|
||||
assert_public_ip
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
'truncate_head',
|
||||
'truncate_tail',
|
||||
@@ -36,5 +41,7 @@ __all__ = [
|
||||
'normalize_for_fuzzy_match',
|
||||
'fuzzy_find_text',
|
||||
'generate_diff_string',
|
||||
'FuzzyMatchResult'
|
||||
'FuzzyMatchResult',
|
||||
'validate_url_safe',
|
||||
'assert_public_ip'
|
||||
]
|
||||
|
||||
66
agent/tools/utils/url_safety.py
Normal file
66
agent/tools/utils/url_safety.py
Normal file
@@ -0,0 +1,66 @@
|
||||
"""
|
||||
Shared SSRF guard utilities for tools that fetch model-supplied URLs.
|
||||
|
||||
A URL is only considered safe when it uses an http/https scheme, has a
|
||||
hostname, that hostname resolves, and every resolved address is a public
|
||||
(internet-routable) address. Loopback, private (RFC1918 / ULA), link-local
|
||||
(incl. the 169.254.169.254 cloud-metadata endpoint) and otherwise reserved
|
||||
addresses are rejected, for both IPv4 and IPv6.
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import socket
|
||||
from urllib.parse import urlparse
|
||||
|
||||
|
||||
def _is_blocked_ip(ip: "ipaddress._BaseAddress") -> bool:
|
||||
"""Return True if the address is not safe to connect to (non-public)."""
|
||||
return (
|
||||
ip.is_private
|
||||
or ip.is_loopback
|
||||
or ip.is_link_local
|
||||
or ip.is_reserved
|
||||
or ip.is_multicast
|
||||
or ip.is_unspecified
|
||||
)
|
||||
|
||||
|
||||
def assert_public_ip(ip_str: str) -> None:
|
||||
"""Raise ValueError if the given literal IP is a non-public address.
|
||||
|
||||
Used to re-validate the concrete address a redirect resolved to.
|
||||
"""
|
||||
ip = ipaddress.ip_address(ip_str)
|
||||
if _is_blocked_ip(ip):
|
||||
raise ValueError(
|
||||
f"URL resolves to a non-public address ({ip_str}), "
|
||||
f"request blocked for security"
|
||||
)
|
||||
|
||||
|
||||
def validate_url_safe(url: str) -> None:
|
||||
"""Reject URLs that target private/loopback/link-local addresses (SSRF guard).
|
||||
|
||||
Resolves the hostname to its IP address(es) and blocks any that fall
|
||||
into non-public ranges. Also rejects URLs with no host, non-HTTP(S)
|
||||
schemes, or hosts that fail DNS resolution.
|
||||
|
||||
Raises:
|
||||
ValueError: if the URL targets a disallowed address.
|
||||
"""
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
raise ValueError(f"Unsupported URL scheme: {parsed.scheme}")
|
||||
|
||||
hostname = parsed.hostname
|
||||
if not hostname:
|
||||
raise ValueError("URL has no hostname")
|
||||
|
||||
try:
|
||||
# Resolve all addresses for the hostname.
|
||||
addr_infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
raise ValueError(f"Cannot resolve hostname: {hostname}")
|
||||
|
||||
for family, _, _, _, sockaddr in addr_infos:
|
||||
assert_public_ip(sockaddr[0])
|
||||
@@ -26,13 +26,14 @@ from typing import Any, Dict, List, Optional
|
||||
import requests
|
||||
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
from agent.tools.utils.url_safety import validate_url_safe
|
||||
from common import const
|
||||
from common.log import logger
|
||||
from config import conf
|
||||
|
||||
DEFAULT_MODEL = const.GPT_41_MINI
|
||||
DEFAULT_TIMEOUT = 60
|
||||
MAX_TOKENS = 1000
|
||||
DEFAULT_TIMEOUT = 180
|
||||
MAX_TOKENS = 4000
|
||||
COMPRESS_THRESHOLD = 1_048_576 # 1 MB
|
||||
|
||||
SUPPORTED_EXTENSIONS = {
|
||||
@@ -51,12 +52,13 @@ _MAIN_MODEL_PROVIDER_NAME = "MainModel"
|
||||
_DISCOVERABLE_MODELS = [
|
||||
("moonshot_api_key", const.MOONSHOT, const.KIMI_K2_6, "Moonshot"),
|
||||
("ark_api_key", const.DOUBAO, const.DOUBAO_SEED_2_PRO, "Doubao"),
|
||||
("dashscope_api_key", const.QWEN_DASHSCOPE, const.QWEN36_PLUS, "DashScope"),
|
||||
("dashscope_api_key", const.QWEN_DASHSCOPE, const.QWEN37_PLUS, "DashScope"),
|
||||
("claude_api_key", const.CLAUDEAPI, const.CLAUDE_4_6_SONNET, "Claude"),
|
||||
("gemini_api_key", const.GEMINI, const.GEMINI_35_FLASH, "Gemini"),
|
||||
("qianfan_api_key", const.QIANFAN, const.ERNIE_45_TURBO_VL, "Qianfan"),
|
||||
("zhipu_ai_api_key", const.ZHIPU_AI, const.GLM_4_7, "ZhipuAI"),
|
||||
("minimax_api_key", const.MiniMax, const.MINIMAX_M2_7, "MiniMax"),
|
||||
("mimo_api_key", const.MIMO, const.MIMO_V2_5_PRO, "MiMo"),
|
||||
]
|
||||
|
||||
# Model name prefix → discoverable provider display_name.
|
||||
@@ -73,6 +75,7 @@ _MODEL_PREFIX_TO_PROVIDER = [
|
||||
("glm-", "ZhipuAI"),
|
||||
("minimax-", "MiniMax"),
|
||||
("abab", "MiniMax"),
|
||||
("mimo-", "MiMo"),
|
||||
]
|
||||
|
||||
# Model prefixes that natively belong to OpenAI / LinkAI (raw HTTP providers).
|
||||
@@ -92,6 +95,7 @@ _PROVIDER_ID_TO_DISPLAY = {
|
||||
"qianfan": "Qianfan",
|
||||
"zhipu": "ZhipuAI",
|
||||
"minimax": "MiniMax",
|
||||
"mimo": "MiMo",
|
||||
}
|
||||
|
||||
|
||||
@@ -158,7 +162,7 @@ class Vision(BaseTool):
|
||||
"Error: No model available for Vision.\n"
|
||||
"The main model does not support vision and no other API keys are configured.\n"
|
||||
"Options:\n"
|
||||
" 1. Switch to a multimodal model (e.g. ernie-4.5-turbo-vl, qwen3.6-plus, claude-sonnet-4-6, gemini-2.0-flash)\n"
|
||||
" 1. Switch to a multimodal model (e.g. ernie-4.5-turbo-vl, qwen3.7-plus, claude-sonnet-4-6, gemini-2.0-flash)\n"
|
||||
" 2. Configure OPENAI_API_KEY: env_config(action=\"set\", key=\"OPENAI_API_KEY\", value=\"your-key\")\n"
|
||||
" 3. Configure LINKAI_API_KEY: env_config(action=\"set\", key=\"LINKAI_API_KEY\", value=\"your-key\")"
|
||||
)
|
||||
@@ -327,6 +331,12 @@ class Vision(BaseTool):
|
||||
- None : unknown provider id, or the bot can't be created.
|
||||
Caller falls through to model-name-based routing.
|
||||
"""
|
||||
# Custom OpenAI-compatible providers — read credentials from
|
||||
# custom_providers list, same pattern as embedding.
|
||||
if provider_id.startswith("custom:"):
|
||||
p = self._build_custom_provider(provider_id, user_model)
|
||||
return [p] if p else None
|
||||
|
||||
display_name = _PROVIDER_ID_TO_DISPLAY.get(provider_id)
|
||||
if not display_name:
|
||||
return None
|
||||
@@ -592,6 +602,34 @@ class Vision(BaseTool):
|
||||
model_override=preferred_model,
|
||||
)
|
||||
|
||||
def _build_custom_provider(self, provider_id: str, preferred_model: Optional[str] = None) -> Optional[VisionProvider]:
|
||||
"""Build a VisionProvider from a custom:<id> entry in custom_providers.
|
||||
Uses the standard OpenAI /chat/completions endpoint — any
|
||||
OpenAI-compatible multimodal endpoint works."""
|
||||
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
|
||||
_, custom_id = parse_custom_bot_type(provider_id)
|
||||
if not custom_id:
|
||||
return None
|
||||
entry = _find_provider_by_id(get_custom_providers(), custom_id)
|
||||
if not entry:
|
||||
logger.warning(f"[Vision] custom provider '{provider_id}' not found in custom_providers")
|
||||
return None
|
||||
api_key = (entry.get("api_key") or "").strip()
|
||||
api_base = (entry.get("api_base") or "").strip()
|
||||
if not api_key or not api_base:
|
||||
logger.warning(f"[Vision] custom provider '{provider_id}' missing api_key or api_base")
|
||||
return None
|
||||
model = preferred_model or entry.get("model") or ""
|
||||
if not model:
|
||||
logger.warning(f"[Vision] custom provider '{provider_id}' has no model configured")
|
||||
return None
|
||||
return VisionProvider(
|
||||
name=entry.get("name") or provider_id,
|
||||
api_key=api_key,
|
||||
api_base=self._ensure_v1(api_base.rstrip("/")),
|
||||
model_override=model,
|
||||
)
|
||||
|
||||
def _call_via_bot(self, model: str, question: str, image_content: dict,
|
||||
provider: Optional[VisionProvider] = None) -> ToolResult:
|
||||
"""
|
||||
@@ -651,6 +689,22 @@ class Vision(BaseTool):
|
||||
return api_base
|
||||
return api_base.rstrip("/") + "/v1"
|
||||
|
||||
@staticmethod
|
||||
def _validate_url_safe(url: str) -> None:
|
||||
"""Reject URLs that target private/loopback/link-local addresses (SSRF guard).
|
||||
|
||||
Resolves the hostname to its IP address(es) and blocks any that fall
|
||||
into non-public ranges. Also rejects URLs with no host, non-HTTP(S)
|
||||
schemes, or hosts that fail DNS resolution.
|
||||
|
||||
Delegates to the shared ``agent.tools.utils.url_safety`` helper so the
|
||||
same guard protects every tool that fetches model-supplied URLs.
|
||||
|
||||
Raises:
|
||||
ValueError: if the URL targets a disallowed address.
|
||||
"""
|
||||
validate_url_safe(url)
|
||||
|
||||
def _build_image_content(self, image: str) -> dict:
|
||||
"""
|
||||
Build the image_url content block.
|
||||
@@ -658,6 +712,7 @@ class Vision(BaseTool):
|
||||
so every bot backend can consume them without extra downloads.
|
||||
"""
|
||||
if image.startswith(("http://", "https://")):
|
||||
self._validate_url_safe(image)
|
||||
return self._download_to_data_url(image)
|
||||
|
||||
if not os.path.isfile(image):
|
||||
|
||||
@@ -16,11 +16,15 @@ import requests
|
||||
|
||||
from agent.tools.base_tool import BaseTool, ToolResult
|
||||
from agent.tools.utils.truncate import truncate_head, format_size
|
||||
from agent.tools.utils.url_safety import validate_url_safe
|
||||
from common.log import logger
|
||||
|
||||
|
||||
DEFAULT_TIMEOUT = 30
|
||||
MAX_FILE_SIZE = 50 * 1024 * 1024 # 50MB
|
||||
# Cap on how many redirects we follow; each hop's target is re-validated
|
||||
# against the SSRF guard so a public URL cannot bounce us into an internal one.
|
||||
MAX_REDIRECTS = 10
|
||||
|
||||
DEFAULT_HEADERS = {
|
||||
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36",
|
||||
@@ -107,23 +111,65 @@ class WebFetch(BaseTool):
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return ToolResult.fail("Error: Invalid URL (must start with http:// or https://)")
|
||||
|
||||
# SSRF guard: reject URLs that resolve to private/loopback/link-local/
|
||||
# cloud-metadata addresses before any request is issued.
|
||||
try:
|
||||
validate_url_safe(url)
|
||||
except ValueError as e:
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
|
||||
if _is_document_url(url):
|
||||
return self._fetch_document(url)
|
||||
|
||||
return self._fetch_webpage(url)
|
||||
|
||||
# ---- Safe request helper ----
|
||||
|
||||
@staticmethod
|
||||
def _safe_get(url: str, **kwargs) -> requests.Response:
|
||||
"""Issue a GET request while re-validating every redirect hop (SSRF guard).
|
||||
|
||||
Auto-redirect is disabled and each hop is followed manually so the
|
||||
target of every redirect is re-resolved and checked against the SSRF
|
||||
guard. This prevents a public URL from 3xx-bouncing into a private,
|
||||
loopback, link-local or cloud-metadata address. ``kwargs`` are passed
|
||||
through to ``requests.get`` (e.g. ``stream``).
|
||||
|
||||
Raises:
|
||||
ValueError: if any hop resolves to a non-public address.
|
||||
"""
|
||||
kwargs.pop("allow_redirects", None)
|
||||
current = url
|
||||
for _ in range(MAX_REDIRECTS + 1):
|
||||
response = requests.get(
|
||||
current,
|
||||
headers=DEFAULT_HEADERS,
|
||||
timeout=DEFAULT_TIMEOUT,
|
||||
allow_redirects=False,
|
||||
**kwargs,
|
||||
)
|
||||
if not response.is_redirect and not response.is_permanent_redirect:
|
||||
return response
|
||||
|
||||
location = response.headers.get("Location")
|
||||
if not location:
|
||||
return response
|
||||
|
||||
# Resolve the redirect target relative to the current URL, then
|
||||
# re-validate it before following.
|
||||
current = requests.compat.urljoin(current, location)
|
||||
validate_url_safe(current)
|
||||
response.close()
|
||||
|
||||
raise ValueError(f"Too many redirects (>{MAX_REDIRECTS})")
|
||||
|
||||
# ---- Web page fetching ----
|
||||
|
||||
def _fetch_webpage(self, url: str) -> ToolResult:
|
||||
"""Fetch and extract readable text from an HTML web page."""
|
||||
parsed = urlparse(url)
|
||||
try:
|
||||
response = requests.get(
|
||||
url,
|
||||
headers=DEFAULT_HEADERS,
|
||||
timeout=DEFAULT_TIMEOUT,
|
||||
allow_redirects=True,
|
||||
)
|
||||
response = self._safe_get(url)
|
||||
response.raise_for_status()
|
||||
except requests.Timeout:
|
||||
return ToolResult.fail(f"Error: Request timed out after {DEFAULT_TIMEOUT}s")
|
||||
@@ -131,6 +177,8 @@ class WebFetch(BaseTool):
|
||||
return ToolResult.fail(f"Error: Failed to connect to {parsed.netloc}")
|
||||
except requests.HTTPError as e:
|
||||
return ToolResult.fail(f"Error: HTTP {e.response.status_code} for URL: {url}")
|
||||
except ValueError as e:
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
except Exception as e:
|
||||
return ToolResult.fail(f"Error: Failed to fetch URL: {e}")
|
||||
|
||||
@@ -158,13 +206,7 @@ class WebFetch(BaseTool):
|
||||
logger.info(f"[WebFetch] Downloading document: {url} -> {local_path}")
|
||||
|
||||
try:
|
||||
response = requests.get(
|
||||
url,
|
||||
headers=DEFAULT_HEADERS,
|
||||
timeout=DEFAULT_TIMEOUT,
|
||||
stream=True,
|
||||
allow_redirects=True,
|
||||
)
|
||||
response = self._safe_get(url, stream=True)
|
||||
response.raise_for_status()
|
||||
|
||||
content_length = int(response.headers.get("Content-Length", 0))
|
||||
@@ -191,6 +233,9 @@ class WebFetch(BaseTool):
|
||||
return ToolResult.fail(f"Error: Failed to connect to {parsed.netloc}")
|
||||
except requests.HTTPError as e:
|
||||
return ToolResult.fail(f"Error: HTTP {e.response.status_code} for URL: {url}")
|
||||
except ValueError as e:
|
||||
self._cleanup_file(local_path)
|
||||
return ToolResult.fail(f"Error: {e}")
|
||||
except Exception as e:
|
||||
self._cleanup_file(local_path)
|
||||
return ToolResult.fail(f"Error: Failed to download file: {e}")
|
||||
|
||||
35
app.py
35
app.py
@@ -15,6 +15,11 @@ import threading
|
||||
|
||||
_channel_mgr = None
|
||||
|
||||
# Desktop mode: a lighter runtime for the packaged Electron client. The plugin
|
||||
# framework is still bundled (it's tiny and on the web channel's import path),
|
||||
# but we skip loading actual plugins and MCP tools to keep startup fast.
|
||||
DESKTOP_MODE = os.environ.get("COW_DESKTOP") == "1"
|
||||
|
||||
|
||||
def get_channel_manager():
|
||||
return _channel_mgr
|
||||
@@ -75,7 +80,7 @@ class ChannelManager:
|
||||
if self._primary_channel is None and channels:
|
||||
self._primary_channel = channels[0][1]
|
||||
|
||||
if first_start:
|
||||
if first_start and not DESKTOP_MODE:
|
||||
PluginManager().load_plugins()
|
||||
|
||||
# Cloud client is optional. It is only started when
|
||||
@@ -231,10 +236,14 @@ def _clear_singleton_cache(channel_name: str):
|
||||
"wechatmp": "channel.wechatmp.wechatmp_channel.WechatMPChannel",
|
||||
"wechatmp_service": "channel.wechatmp.wechatmp_channel.WechatMPChannel",
|
||||
"wechatcom_app": "channel.wechatcom.wechatcomapp_channel.WechatComAppChannel",
|
||||
const.WECHAT_KF: "channel.wechat_kf.wechat_kf_channel.WechatKfChannel",
|
||||
const.FEISHU: "channel.feishu.feishu_channel.FeiShuChanel",
|
||||
const.DINGTALK: "channel.dingtalk.dingtalk_channel.DingTalkChanel",
|
||||
const.WECOM_BOT: "channel.wecom_bot.wecom_bot_channel.WecomBotChannel",
|
||||
const.QQ: "channel.qq.qq_channel.QQChannel",
|
||||
const.TELEGRAM: "channel.telegram.telegram_channel.TelegramChannel",
|
||||
const.SLACK: "channel.slack.slack_channel.SlackChannel",
|
||||
const.DISCORD: "channel.discord.discord_channel.DiscordChannel",
|
||||
const.WEIXIN: "channel.weixin.weixin_channel.WeixinChannel",
|
||||
"wx": "channel.weixin.weixin_channel.WeixinChannel",
|
||||
}
|
||||
@@ -288,6 +297,16 @@ def _warmup_mcp_tools():
|
||||
logger.warning(f"[App] MCP warmup failed (non-fatal): {e}")
|
||||
|
||||
|
||||
def _warmup_scheduler():
|
||||
"""Eager-init AgentBridge so the scheduler thread starts at process
|
||||
boot rather than waiting for the first user message."""
|
||||
try:
|
||||
from bridge.bridge import Bridge
|
||||
Bridge().get_agent_bridge()
|
||||
except Exception as e:
|
||||
logger.warning(f"[App] Scheduler warmup failed: {e}")
|
||||
|
||||
|
||||
def _sync_builtin_skills():
|
||||
"""Sync builtin skills from project skills/ to workspace skills/ on startup."""
|
||||
import shutil
|
||||
@@ -350,8 +369,18 @@ def run():
|
||||
_sync_builtin_skills()
|
||||
|
||||
# Kick off MCP server loading in the background so first-message
|
||||
# latency isn't dominated by npx package downloads.
|
||||
_warmup_mcp_tools()
|
||||
# latency isn't dominated by npx package downloads. Skipped in desktop
|
||||
# mode (MCP relies on external npx/uvx runtimes that aren't bundled).
|
||||
if not DESKTOP_MODE:
|
||||
_warmup_mcp_tools()
|
||||
|
||||
if DESKTOP_MODE:
|
||||
# Defer the (heavy) AgentBridge/scheduler warmup to a background
|
||||
# thread so the web API becomes available within a couple seconds.
|
||||
# The scheduler still starts; it just doesn't block UI readiness.
|
||||
threading.Thread(target=_warmup_scheduler, daemon=True).start()
|
||||
else:
|
||||
_warmup_scheduler()
|
||||
|
||||
logger.info(f"[App] Starting channels: {channel_names}")
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ Agent Bridge - Integrates Agent system with existing COW bridge
|
||||
import os
|
||||
from typing import Optional, List
|
||||
|
||||
from agent.protocol import Agent, LLMModel, LLMRequest
|
||||
from agent.protocol import Agent, LLMModel, LLMRequest, get_cancel_registry
|
||||
from bridge.agent_event_handler import AgentEventHandler
|
||||
from bridge.agent_initializer import AgentInitializer
|
||||
from bridge.bridge import Bridge
|
||||
@@ -78,6 +78,7 @@ class AgentLLMModel(LLMModel):
|
||||
("moonshot", const.MOONSHOT), ("kimi", const.MOONSHOT),
|
||||
("doubao", const.DOUBAO), ("deepseek", const.DEEPSEEK),
|
||||
("ernie", const.QIANFAN),
|
||||
("mimo-", const.MIMO),
|
||||
]
|
||||
|
||||
def __init__(self, bridge: Bridge, bot_type: str = "chat"):
|
||||
@@ -285,6 +286,23 @@ class AgentBridge:
|
||||
|
||||
# Create helper instances
|
||||
self.initializer = AgentInitializer(bridge, self)
|
||||
|
||||
# Eager-start the scheduler so cron tasks fire without waiting
|
||||
# for the first user message. init_scheduler is idempotent.
|
||||
try:
|
||||
from agent.tools.scheduler.integration import init_scheduler
|
||||
if init_scheduler(self):
|
||||
self.scheduler_initialized = True
|
||||
except Exception as e:
|
||||
logger.warning(f"[AgentBridge] Eager scheduler init failed: {e}")
|
||||
|
||||
# Start the self-evolution idle trigger (idempotent, daemon thread).
|
||||
try:
|
||||
from agent.evolution.trigger import start_evolution_trigger
|
||||
start_evolution_trigger(self)
|
||||
except Exception as e:
|
||||
logger.warning(f"[AgentBridge] Evolution trigger init failed: {e}")
|
||||
|
||||
def create_agent(self, system_prompt: str, tools: List = None, **kwargs) -> Agent:
|
||||
"""
|
||||
Create the super agent with COW integration
|
||||
@@ -373,7 +391,49 @@ class AgentBridge:
|
||||
"""Initialize agent for a specific session"""
|
||||
agent = self.initializer.initialize_agent(session_id=session_id)
|
||||
self.agents[session_id] = agent
|
||||
|
||||
|
||||
def sync_session_messages_from_store(self, session_id: str) -> int:
|
||||
"""Reload an agent's in-memory ``messages`` list from the persistent
|
||||
conversation store.
|
||||
|
||||
Used after an external mutation (e.g. user edits / deletes a message
|
||||
via the web console) so the agent's next turn sees the same history
|
||||
as the database. The operation is a no-op when the agent has not been
|
||||
instantiated yet for the session.
|
||||
|
||||
Returns:
|
||||
Number of messages now held in the agent's memory. Returns -1 if
|
||||
the agent does not exist or has no compatible ``messages`` attr.
|
||||
"""
|
||||
if not session_id or session_id not in self.agents:
|
||||
return -1
|
||||
agent = self.agents[session_id]
|
||||
if not (hasattr(agent, "messages") and hasattr(agent, "messages_lock")):
|
||||
return -1
|
||||
try:
|
||||
from agent.memory import get_conversation_store
|
||||
store = get_conversation_store()
|
||||
# No turn cap here: we want a faithful mirror of what the store
|
||||
# has for this session after deletion.
|
||||
remaining = store.load_messages(session_id, max_turns=10**6)
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"[AgentBridge] Failed to load messages for sync (session={session_id}): {e}"
|
||||
)
|
||||
return -1
|
||||
with agent.messages_lock:
|
||||
agent.messages.clear()
|
||||
for msg in remaining:
|
||||
agent.messages.append({
|
||||
"role": msg["role"],
|
||||
"content": msg["content"],
|
||||
})
|
||||
count = len(agent.messages)
|
||||
logger.info(
|
||||
f"[AgentBridge] Synced agent memory for session={session_id}, messages={count}"
|
||||
)
|
||||
return count
|
||||
|
||||
def agent_reply(self, query: str, context: Context = None,
|
||||
on_event=None, clear_history: bool = False) -> Reply:
|
||||
"""
|
||||
@@ -390,11 +450,22 @@ class AgentBridge:
|
||||
"""
|
||||
session_id = None
|
||||
agent = None
|
||||
request_id = None
|
||||
cancel_event = None
|
||||
try:
|
||||
# Extract session_id from context for user isolation
|
||||
if context:
|
||||
session_id = context.kwargs.get("session_id") or context.get("session_id")
|
||||
|
||||
request_id = context.kwargs.get("request_id") or context.get("request_id")
|
||||
|
||||
# Register a cancel token. Prefer per-turn request_id (web),
|
||||
# fall back to session_id (IM channels). The Event is polled by
|
||||
# AgentStreamExecutor at safe checkpoints.
|
||||
registry = get_cancel_registry()
|
||||
token_key = request_id or session_id
|
||||
if token_key:
|
||||
cancel_event = registry.register(token_key, session_id=session_id)
|
||||
|
||||
# Get agent for this session (will auto-initialize if needed)
|
||||
agent = self.get_agent(session_id=session_id)
|
||||
if not agent:
|
||||
@@ -444,14 +515,40 @@ class AgentBridge:
|
||||
)
|
||||
self._trim_in_memory_to_turns(agent, scheduler_keep_turns)
|
||||
|
||||
# Eagerly persist the user message BEFORE running the agent so the
|
||||
# session and the user's bubble are immediately visible — even if
|
||||
# the user switches away or refreshes before the reply finishes.
|
||||
# The reply (assistant/tool messages) is appended once the run
|
||||
# completes; the final persist skips this already-stored user turn.
|
||||
pre_persisted = self._pre_persist_user_message(
|
||||
session_id, query, context, clear_history
|
||||
)
|
||||
|
||||
# Mark this session as mid-run so the self-evolution idle scan does
|
||||
# not fire concurrently when a single turn runs longer than
|
||||
# idle_minutes.
|
||||
try:
|
||||
from agent.evolution.trigger import mark_run_active
|
||||
mark_run_active(agent, True)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
# Use agent's run_stream method with event handler
|
||||
response = agent.run_stream(
|
||||
user_message=query,
|
||||
on_event=event_handler.handle_event,
|
||||
clear_history=clear_history
|
||||
clear_history=clear_history,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
finally:
|
||||
# Clear the mid-run flag so idle scans can review this session.
|
||||
try:
|
||||
from agent.evolution.trigger import mark_run_active
|
||||
mark_run_active(agent, False)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Restore original tools
|
||||
if context and context.get("is_scheduled_task"):
|
||||
agent.tools = original_tools
|
||||
@@ -459,10 +556,21 @@ class AgentBridge:
|
||||
# Log execution summary
|
||||
event_handler.log_summary()
|
||||
|
||||
# Release cancel token; keep registry bounded.
|
||||
if token_key:
|
||||
try:
|
||||
registry.unregister(token_key)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Persist new messages generated during this run
|
||||
if session_id:
|
||||
channel_type = (context.get("channel_type") or "") if context else ""
|
||||
new_messages = getattr(agent, '_last_run_new_messages', [])
|
||||
new_messages = list(getattr(agent, '_last_run_new_messages', []))
|
||||
# The leading user turn was already persisted eagerly above;
|
||||
# drop it here so it isn't stored twice.
|
||||
if pre_persisted and new_messages and new_messages[0].get("role") == "user":
|
||||
new_messages = new_messages[1:]
|
||||
if new_messages:
|
||||
self._persist_messages(session_id, list(new_messages), channel_type)
|
||||
else:
|
||||
@@ -476,6 +584,23 @@ class AgentBridge:
|
||||
except Exception as e:
|
||||
logger.warning(f"[AgentBridge] Failed to clear DB after recovery: {e}")
|
||||
|
||||
# Record this user turn for the self-evolution idle trigger. Skip
|
||||
# scheduler-injected / scheduled-task sessions so internal runs do
|
||||
# not count as user activity.
|
||||
if session_id and not session_id.startswith("scheduler_") and not (
|
||||
context and context.get("is_scheduled_task")
|
||||
):
|
||||
try:
|
||||
from agent.evolution.trigger import note_user_turn
|
||||
ch = (context.get("channel_type") or "") if context else ""
|
||||
rcv = (context.get("receiver") or "") if context else ""
|
||||
is_group = bool(context.get("isgroup")) if context else False
|
||||
# Only enable proactive push for single chats (group push is
|
||||
# noisy); group sessions still evolve, just without notify.
|
||||
note_user_turn(agent, channel_type=ch, receiver=(rcv if not is_group else ""))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Post-message hot-reload: detect edits to ~/cow/mcp.json and
|
||||
# sync any new/removed MCP tools into the live agent in the
|
||||
# background. Off the critical path so user latency is unaffected;
|
||||
@@ -512,6 +637,12 @@ class AgentBridge:
|
||||
logger.info(f"[AgentBridge] Cleared DB for session after error: {session_id}")
|
||||
except Exception as db_err:
|
||||
logger.warning(f"[AgentBridge] Failed to clear DB after error: {db_err}")
|
||||
# Release cancel token on error path too (idempotent).
|
||||
if cancel_event is not None and (request_id or session_id):
|
||||
try:
|
||||
get_cancel_registry().unregister(request_id or session_id)
|
||||
except Exception:
|
||||
pass
|
||||
return Reply(ReplyType.ERROR, f"Agent error: {str(e)}")
|
||||
|
||||
def _schedule_mcp_hot_reload(self, agent):
|
||||
@@ -655,6 +786,48 @@ class AgentBridge:
|
||||
except Exception as e:
|
||||
logger.warning(f"[AgentBridge] Failed to sync API keys: {e}")
|
||||
|
||||
def _pre_persist_user_message(
|
||||
self, session_id: str, query: str, context: Context, clear_history: bool
|
||||
) -> bool:
|
||||
"""Persist the user's message before the agent runs.
|
||||
|
||||
This makes a brand-new session (and the user's bubble) visible even if
|
||||
the reply hasn't finished — switching away or refreshing no longer
|
||||
loses the in-flight session. Returns True when the user turn was
|
||||
stored, so the caller can skip it in the post-run persist.
|
||||
|
||||
Best-effort: any failure is swallowed and reported as not-persisted.
|
||||
"""
|
||||
if not session_id or not query:
|
||||
return False
|
||||
# Only real user turns: skip scheduler-injected / scheduled-task runs.
|
||||
if session_id.startswith("scheduler_") or (
|
||||
context and context.get("is_scheduled_task")
|
||||
):
|
||||
return False
|
||||
try:
|
||||
from config import conf
|
||||
if not conf().get("conversation_persistence", True):
|
||||
return False
|
||||
from agent.memory import get_conversation_store
|
||||
store = get_conversation_store()
|
||||
# clear_history starts a fresh transcript: wipe the store first so
|
||||
# the eager user turn becomes seq 0, matching in-memory state.
|
||||
if clear_history:
|
||||
store.clear_session(session_id)
|
||||
channel_type = (context.get("channel_type") or "") if context else ""
|
||||
user_msg = {
|
||||
"role": "user",
|
||||
"content": [{"type": "text", "text": query}],
|
||||
}
|
||||
store.append_messages(session_id, [user_msg], channel_type=channel_type)
|
||||
return True
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"[AgentBridge] Failed to pre-persist user message for session={session_id}: {e}"
|
||||
)
|
||||
return False
|
||||
|
||||
def _persist_messages(
|
||||
self, session_id: str, new_messages: list, channel_type: str = ""
|
||||
) -> None:
|
||||
|
||||
@@ -395,7 +395,13 @@ class AgentInitializer:
|
||||
from agent.memory.embedding import EMBEDDING_VENDORS
|
||||
from config import conf
|
||||
|
||||
meta = EMBEDDING_VENDORS.get(provider_key)
|
||||
# Custom providers ("custom:<id>") resolve credentials
|
||||
# from the custom_providers list.
|
||||
resolved_provider_key = provider_key
|
||||
if provider_key.startswith("custom:"):
|
||||
resolved_provider_key = "custom"
|
||||
|
||||
meta = EMBEDDING_VENDORS.get(resolved_provider_key)
|
||||
if meta is None:
|
||||
logger.error(
|
||||
f"[AgentInitializer] Unknown embedding_provider '{provider_key}'. "
|
||||
@@ -414,7 +420,17 @@ class AgentInitializer:
|
||||
)
|
||||
return None
|
||||
|
||||
model = (conf().get("embedding_model") or "").strip() or meta["default_model"]
|
||||
model = (conf().get("embedding_model") or "").strip()
|
||||
# Custom providers without a model fall back to the provider's default.
|
||||
if not model and resolved_provider_key == "custom":
|
||||
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
|
||||
_, custom_id = parse_custom_bot_type(provider_key)
|
||||
if custom_id:
|
||||
entry = _find_provider_by_id(get_custom_providers(), custom_id)
|
||||
if entry and entry.get("model"):
|
||||
model = entry["model"]
|
||||
if not model and resolved_provider_key != "custom":
|
||||
model = meta["default_model"]
|
||||
try:
|
||||
cfg_dim = int(conf().get("embedding_dimensions") or 0)
|
||||
except (TypeError, ValueError):
|
||||
@@ -423,7 +439,7 @@ class AgentInitializer:
|
||||
|
||||
try:
|
||||
provider = create_embedding_provider(
|
||||
provider=provider_key,
|
||||
provider=resolved_provider_key,
|
||||
model=model,
|
||||
api_key=api_key,
|
||||
api_base=api_base,
|
||||
@@ -450,6 +466,17 @@ class AgentInitializer:
|
||||
"""Pick the API key for an explicit embedding provider from config."""
|
||||
from config import conf
|
||||
|
||||
# Custom providers ("custom:<id>") resolve from the custom_providers list.
|
||||
if provider_key.startswith("custom:"):
|
||||
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
|
||||
_, custom_id = parse_custom_bot_type(provider_key)
|
||||
if custom_id:
|
||||
providers = get_custom_providers()
|
||||
entry = _find_provider_by_id(providers, custom_id)
|
||||
if entry:
|
||||
return entry.get("api_key", "")
|
||||
return ""
|
||||
|
||||
key_map = {
|
||||
"openai": "open_ai_api_key",
|
||||
"linkai": "linkai_api_key",
|
||||
@@ -470,6 +497,17 @@ class AgentInitializer:
|
||||
"""Pick the API base for an explicit embedding provider from config."""
|
||||
from config import conf
|
||||
|
||||
# Custom providers ("custom:<id>") resolve from the custom_providers list.
|
||||
if provider_key.startswith("custom:"):
|
||||
from models.custom_provider import parse_custom_bot_type, get_custom_providers, _find_provider_by_id
|
||||
_, custom_id = parse_custom_bot_type(provider_key)
|
||||
if custom_id:
|
||||
providers = get_custom_providers()
|
||||
entry = _find_provider_by_id(providers, custom_id)
|
||||
if entry and entry.get("api_base"):
|
||||
return entry["api_base"]
|
||||
return default_base
|
||||
|
||||
base_map = {
|
||||
"openai": "open_ai_api_base",
|
||||
"linkai": "linkai_api_base",
|
||||
@@ -524,6 +562,14 @@ class AgentInitializer:
|
||||
logger.debug("[AgentInitializer] WebSearch skipped - no search provider configured")
|
||||
continue
|
||||
|
||||
# Skip evolution_undo when self-evolution is disabled: with no
|
||||
# evolution there is nothing to roll back, so the tool is dead weight.
|
||||
if tool_name == "evolution_undo":
|
||||
from agent.evolution.config import get_evolution_config
|
||||
if not get_evolution_config().enabled:
|
||||
logger.debug("[AgentInitializer] evolution_undo skipped - self-evolution disabled")
|
||||
continue
|
||||
|
||||
# Special handling for EnvConfig tool
|
||||
if tool_name == "env_config":
|
||||
from agent.tools import EnvConfig
|
||||
@@ -643,16 +689,25 @@ class AgentInitializer:
|
||||
except Exception:
|
||||
timezone_name = "UTC"
|
||||
|
||||
# Chinese weekday mapping
|
||||
weekday_map = {
|
||||
'Monday': '星期一', 'Tuesday': '星期二', 'Wednesday': '星期三',
|
||||
'Thursday': '星期四', 'Friday': '星期五', 'Saturday': '星期六', 'Sunday': '星期日'
|
||||
}
|
||||
weekday_zh = weekday_map.get(now.strftime("%A"), now.strftime("%A"))
|
||||
|
||||
# Weekday: English name in en, Chinese mapping otherwise
|
||||
weekday_en = now.strftime("%A")
|
||||
try:
|
||||
from common import i18n
|
||||
is_en = i18n.get_language() == "en"
|
||||
except Exception:
|
||||
is_en = False
|
||||
if is_en:
|
||||
weekday = weekday_en
|
||||
else:
|
||||
weekday_map = {
|
||||
'Monday': '星期一', 'Tuesday': '星期二', 'Wednesday': '星期三',
|
||||
'Thursday': '星期四', 'Friday': '星期五', 'Saturday': '星期六', 'Sunday': '星期日'
|
||||
}
|
||||
weekday = weekday_map.get(weekday_en, weekday_en)
|
||||
|
||||
return {
|
||||
'time': now.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
'weekday': weekday_zh,
|
||||
'weekday': weekday,
|
||||
'timezone': timezone_name
|
||||
}
|
||||
|
||||
|
||||
@@ -63,6 +63,10 @@ class Bridge(object):
|
||||
if model_type and model_type.startswith("deepseek"):
|
||||
self.btype["chat"] = const.DEEPSEEK
|
||||
|
||||
# 小米 MiMo 系列模型,全部以 mimo- 开头
|
||||
if model_type and model_type.startswith("mimo-"):
|
||||
self.btype["chat"] = const.MIMO
|
||||
|
||||
if model_type and isinstance(model_type, str):
|
||||
lowered_model_type = model_type.lower()
|
||||
if lowered_model_type == const.QIANFAN or lowered_model_type.startswith("ernie"):
|
||||
|
||||
@@ -27,6 +27,9 @@ def create_channel(channel_type) -> Channel:
|
||||
elif channel_type == "wechatcom_app":
|
||||
from channel.wechatcom.wechatcomapp_channel import WechatComAppChannel
|
||||
ch = WechatComAppChannel()
|
||||
elif channel_type == const.WECHAT_KF:
|
||||
from channel.wechat_kf.wechat_kf_channel import WechatKfChannel
|
||||
ch = WechatKfChannel()
|
||||
elif channel_type == const.FEISHU:
|
||||
from channel.feishu.feishu_channel import FeiShuChanel
|
||||
ch = FeiShuChanel()
|
||||
@@ -39,6 +42,15 @@ def create_channel(channel_type) -> Channel:
|
||||
elif channel_type == const.QQ:
|
||||
from channel.qq.qq_channel import QQChannel
|
||||
ch = QQChannel()
|
||||
elif channel_type == const.TELEGRAM:
|
||||
from channel.telegram.telegram_channel import TelegramChannel
|
||||
ch = TelegramChannel()
|
||||
elif channel_type == const.SLACK:
|
||||
from channel.slack.slack_channel import SlackChannel
|
||||
ch = SlackChannel()
|
||||
elif channel_type == const.DISCORD:
|
||||
from channel.discord.discord_channel import DiscordChannel
|
||||
ch = DiscordChannel()
|
||||
elif channel_type in (const.WEIXIN, "wx"):
|
||||
from channel.weixin.weixin_channel import WeixinChannel
|
||||
ch = WeixinChannel()
|
||||
|
||||
@@ -10,6 +10,7 @@ from bridge.reply import *
|
||||
from channel.channel import Channel
|
||||
from common.dequeue import Dequeue
|
||||
from common import memory
|
||||
from common.i18n import t as _t
|
||||
from plugins import *
|
||||
|
||||
try:
|
||||
@@ -265,7 +266,7 @@ class ChatChannel(Channel):
|
||||
if reply.type in self.NOT_SUPPORT_REPLYTYPE:
|
||||
logger.error("[chat_channel]reply type not support: " + str(reply.type))
|
||||
reply.type = ReplyType.ERROR
|
||||
reply.content = "不支持发送的消息类型: " + str(reply.type)
|
||||
reply.content = _t("不支持发送的消息类型: ", "Unsupported message type: ") + str(reply.type)
|
||||
|
||||
if reply.type == ReplyType.TEXT:
|
||||
reply_text = reply.content
|
||||
@@ -438,8 +439,21 @@ class ChatChannel(Channel):
|
||||
|
||||
return func
|
||||
|
||||
# Chat commands that must bypass the per-session serial queue,
|
||||
# otherwise /cancel would queue behind the task it tries to cancel.
|
||||
# Use /cancel (not /stop) to avoid colliding with `cow stop` CLI.
|
||||
_BYPASS_QUEUE_COMMANDS = ("/cancel",)
|
||||
|
||||
def produce(self, context: Context):
|
||||
session_id = context["session_id"]
|
||||
|
||||
# Fast path: /cancel must not enter the queue.
|
||||
if context.type == ContextType.TEXT and context.content:
|
||||
stripped = context.content.strip().lower()
|
||||
if stripped in self._BYPASS_QUEUE_COMMANDS:
|
||||
self._handle_cancel_command(context, session_id)
|
||||
return
|
||||
|
||||
with self.lock:
|
||||
if session_id not in self.sessions:
|
||||
self.sessions[session_id] = [
|
||||
@@ -451,6 +465,29 @@ class ChatChannel(Channel):
|
||||
else:
|
||||
self.sessions[session_id][0].put(context)
|
||||
|
||||
def _handle_cancel_command(self, context: Context, session_id: str) -> None:
|
||||
"""Cancel any in-flight agent run for *session_id* and reply inline.
|
||||
|
||||
Runs synchronously on the caller's thread. Reply is sent through
|
||||
_send_reply so plugins (e.g. logging) still observe it.
|
||||
"""
|
||||
try:
|
||||
from agent.protocol import get_cancel_registry
|
||||
from bridge.reply import Reply, ReplyType
|
||||
|
||||
cancelled = get_cancel_registry().cancel_session(session_id)
|
||||
text = (
|
||||
_t("🛑 已中止", "🛑 Cancelled")
|
||||
if cancelled > 0
|
||||
else _t("当前没有可中止的任务。", "Nothing to cancel.")
|
||||
)
|
||||
logger.info(
|
||||
f"[chat_channel] /cancel fast-path: session={session_id}, cancelled={cancelled}"
|
||||
)
|
||||
self._send_reply(context, Reply(ReplyType.TEXT, text))
|
||||
except Exception as e:
|
||||
logger.warning(f"[chat_channel] /cancel fast-path failed: {e}")
|
||||
|
||||
# 消费者函数,单独线程,用于从消息队列中取出消息并处理
|
||||
def consume(self):
|
||||
while True:
|
||||
@@ -482,7 +519,10 @@ class ChatChannel(Channel):
|
||||
def cancel_session(self, session_id):
|
||||
with self.lock:
|
||||
if session_id in self.sessions:
|
||||
for future in self.futures[session_id]:
|
||||
# futures[session_id] is only created in consume() when a task is
|
||||
# dispatched, so it may be absent if cancel happens right after
|
||||
# produce() but before the first dispatch. Default to [].
|
||||
for future in self.futures.get(session_id, []):
|
||||
future.cancel()
|
||||
cnt = self.sessions[session_id][0].qsize()
|
||||
if cnt > 0:
|
||||
@@ -492,7 +532,7 @@ class ChatChannel(Channel):
|
||||
def cancel_all_session(self):
|
||||
with self.lock:
|
||||
for session_id in self.sessions:
|
||||
for future in self.futures[session_id]:
|
||||
for future in self.futures.get(session_id, []):
|
||||
future.cancel()
|
||||
cnt = self.sessions[session_id][0].qsize()
|
||||
if cnt > 0:
|
||||
|
||||
0
channel/discord/__init__.py
Normal file
0
channel/discord/__init__.py
Normal file
500
channel/discord/discord_channel.py
Normal file
500
channel/discord/discord_channel.py
Normal file
@@ -0,0 +1,500 @@
|
||||
"""
|
||||
Discord channel via the Gateway (WebSocket) using discord.py.
|
||||
|
||||
Features:
|
||||
- Direct message & guild channel chat (text / image / file)
|
||||
- Guild trigger: @mention or reply-to-bot (configurable)
|
||||
- /cancel fast-path matches Web channel behaviour
|
||||
- Gateway long connection: no public IP / callback URL required, works behind NAT
|
||||
|
||||
Implementation note:
|
||||
discord.py is async-first. We run the client inside a dedicated thread
|
||||
with its own asyncio loop so the rest of cow (which is sync) stays
|
||||
untouched. Inbound messages are dispatched onto cow's existing sync
|
||||
ChatChannel.produce() pipeline; outbound send() schedules coroutines
|
||||
back onto that loop via asyncio.run_coroutine_threadsafe.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.chat_channel import ChatChannel, check_prefix
|
||||
from channel.discord.discord_message import DiscordMessage
|
||||
from common.expired_dict import ExpiredDict
|
||||
from common.log import logger
|
||||
from common.singleton import singleton
|
||||
from config import conf
|
||||
|
||||
# Discord caps a single message at 2000 chars; split conservatively below.
|
||||
DISCORD_MSG_LIMIT = 1900
|
||||
|
||||
|
||||
@singleton
|
||||
class DiscordChannel(ChatChannel):
|
||||
NOT_SUPPORT_REPLYTYPE = []
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.bot_token = ""
|
||||
self.bot_user_id = "" # used to strip @mention and ignore self messages
|
||||
self.bot_username = ""
|
||||
self._client = None
|
||||
self._loop = None
|
||||
self._loop_thread = None
|
||||
self._stop_event = threading.Event()
|
||||
# Idempotent dedup; guard against rare duplicate dispatch
|
||||
self._received_msgs = ExpiredDict(60 * 60 * 1)
|
||||
|
||||
# Disable group whitelist / prefix checks (we handle triggering ourselves
|
||||
# in _should_reply_in_guild), aligned with telegram / slack channels.
|
||||
conf()["group_name_white_list"] = ["ALL_GROUP"]
|
||||
conf()["single_chat_prefix"] = [""]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def startup(self):
|
||||
self.bot_token = conf().get("discord_token", "")
|
||||
if not self.bot_token:
|
||||
err = "[Discord] discord_token is required"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
try:
|
||||
import discord
|
||||
except ImportError:
|
||||
err = (
|
||||
"[Discord] discord.py is not installed. "
|
||||
"Run: pip install discord.py"
|
||||
)
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
# Run the asyncio event loop in a dedicated thread so the sync cow body
|
||||
# is untouched.
|
||||
self._loop = asyncio.new_event_loop()
|
||||
|
||||
def _run_loop():
|
||||
asyncio.set_event_loop(self._loop)
|
||||
try:
|
||||
self._loop.run_until_complete(self._async_main(discord))
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] event loop crashed: {e}", exc_info=True)
|
||||
self.report_startup_error(str(e))
|
||||
finally:
|
||||
try:
|
||||
self._loop.close()
|
||||
except Exception:
|
||||
pass
|
||||
logger.info("[Discord] event loop exited")
|
||||
|
||||
self._loop_thread = threading.Thread(target=_run_loop, daemon=True, name="discord-loop")
|
||||
self._loop_thread.start()
|
||||
# Block startup() until the loop thread exits, matching other channels'
|
||||
# behaviour (startup is a blocking call).
|
||||
self._loop_thread.join()
|
||||
|
||||
async def _async_main(self, discord):
|
||||
"""Build the discord client, register handlers, and connect to the Gateway."""
|
||||
# message_content is a privileged intent; it must be enabled in the
|
||||
# Developer Portal (Bot -> Privileged Gateway Intents) to read text.
|
||||
intents = discord.Intents.default()
|
||||
intents.message_content = True
|
||||
client = discord.Client(intents=intents)
|
||||
self._client = client
|
||||
|
||||
channel = self
|
||||
|
||||
@client.event
|
||||
async def on_ready():
|
||||
channel.bot_user_id = str(client.user.id)
|
||||
channel.bot_username = client.user.name or ""
|
||||
channel.name = channel.bot_user_id # ChatChannel uses self.name to strip @-mention
|
||||
logger.info(f"[Discord] Bot logged in as {client.user} (id={client.user.id})")
|
||||
channel.report_startup_success()
|
||||
logger.info("[Discord] ✅ Discord bot ready, listening for messages")
|
||||
|
||||
@client.event
|
||||
async def on_message(message):
|
||||
await channel._on_message(message)
|
||||
|
||||
# Connect to the Gateway; discord.py auto-reconnects on transient errors.
|
||||
logger.info("[Discord] Connecting to Gateway...")
|
||||
|
||||
# client.start() handles login + Gateway connection and runs until
|
||||
# close(); it is the standard entrypoint across discord.py versions.
|
||||
runner_task = asyncio.create_task(client.start(self.bot_token))
|
||||
|
||||
# Block until stop()
|
||||
try:
|
||||
while not self._stop_event.is_set():
|
||||
if runner_task.done():
|
||||
# Surface a startup/connection failure (e.g. bad token)
|
||||
exc = runner_task.exception()
|
||||
if exc:
|
||||
logger.error(f"[Discord] client stopped: {exc}", exc_info=exc)
|
||||
self.report_startup_error(str(exc))
|
||||
break
|
||||
await asyncio.sleep(0.5)
|
||||
finally:
|
||||
try:
|
||||
if not client.is_closed():
|
||||
await client.close()
|
||||
except Exception as e:
|
||||
logger.warning(f"[Discord] shutdown error: {e}")
|
||||
|
||||
def stop(self):
|
||||
logger.info("[Discord] stop() called")
|
||||
self._stop_event.set()
|
||||
if self._loop_thread and self._loop_thread.is_alive():
|
||||
try:
|
||||
self._loop_thread.join(timeout=10)
|
||||
except Exception:
|
||||
pass
|
||||
logger.info("[Discord] stop() completed")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound: discord message -> ChatMessage -> ChatChannel.produce
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _on_message(self, message):
|
||||
"""Discord message entry: parse -> build ChatMessage -> produce()."""
|
||||
try:
|
||||
# Ignore our own messages and other bots. self._client.user may be
|
||||
# None until on_ready completes, so guard against that.
|
||||
if self._client and self._client.user and message.author.id == self._client.user.id:
|
||||
return
|
||||
if message.author.bot:
|
||||
return
|
||||
|
||||
# Idempotent dedup
|
||||
msg_uid = f"{message.channel.id}:{message.id}"
|
||||
if self._received_msgs.get(msg_uid):
|
||||
return
|
||||
self._received_msgs[msg_uid] = True
|
||||
|
||||
# guild is None for DMs
|
||||
is_group = message.guild is not None
|
||||
|
||||
# Guild trigger gate (silently drop if not triggered)
|
||||
if is_group and not self._should_reply_in_guild(message):
|
||||
logger.debug(f"[Discord] guild message not triggered (need @mention or reply), skip")
|
||||
return
|
||||
|
||||
# Parse message type + download attachments if needed.
|
||||
ctype, content, caption = await self._parse_message(message)
|
||||
if ctype is None:
|
||||
logger.debug(f"[Discord] unsupported message type, skip. msg_id={message.id}")
|
||||
return
|
||||
|
||||
# Strip the bot mention from guild text/caption
|
||||
if is_group:
|
||||
if ctype == ContextType.TEXT and content:
|
||||
content = self._strip_at_mention(content)
|
||||
if caption:
|
||||
caption = self._strip_at_mention(caption)
|
||||
|
||||
dc_msg = DiscordMessage(
|
||||
message,
|
||||
is_group=is_group,
|
||||
bot_user_id=self.bot_user_id,
|
||||
ctype=ctype,
|
||||
content=content,
|
||||
)
|
||||
dc_msg.is_at = is_group # if we reached here in a guild, bot is mentioned/replied
|
||||
|
||||
from channel.file_cache import get_file_cache
|
||||
file_cache = get_file_cache()
|
||||
session_id = self._compute_session_id(message, is_group)
|
||||
|
||||
# Media + caption together: treat as a complete query and bypass the cache
|
||||
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
|
||||
tag = "image" if ctype == ContextType.IMAGE else "file"
|
||||
merged_text = f"{caption}\n[{tag}: {content}]"
|
||||
dc_msg.ctype = ContextType.TEXT
|
||||
dc_msg.content = merged_text
|
||||
ctype = ContextType.TEXT
|
||||
logger.info(f"[Discord] Media+caption merged for session {session_id}")
|
||||
# fallthrough to the TEXT branch below
|
||||
|
||||
elif ctype == ContextType.IMAGE:
|
||||
file_cache.add(session_id, content, file_type="image")
|
||||
logger.info(f"[Discord] Image cached for session {session_id}, waiting for query...")
|
||||
return
|
||||
elif ctype == ContextType.FILE:
|
||||
file_cache.add(session_id, content, file_type="file")
|
||||
logger.info(f"[Discord] File cached for session {session_id}: {content}")
|
||||
return
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
# Fast-path: /cancel mirrors Web channel behaviour
|
||||
if (content or "").strip().lower() in ("/cancel", "cancel"):
|
||||
await self._do_cancel(session_id, message)
|
||||
return
|
||||
|
||||
cached_files = file_cache.get(session_id)
|
||||
if cached_files:
|
||||
refs = []
|
||||
for fi in cached_files:
|
||||
ftype = fi["type"]
|
||||
tag = ftype if ftype in ("image", "video") else "file"
|
||||
refs.append(f"[{tag}: {fi['path']}]")
|
||||
dc_msg.content = (dc_msg.content or "") + "\n" + "\n".join(refs)
|
||||
file_cache.clear(session_id)
|
||||
logger.info(f"[Discord] Attached {len(cached_files)} cached file(s) to query")
|
||||
|
||||
context = self._compose_context(
|
||||
dc_msg.ctype,
|
||||
dc_msg.content,
|
||||
isgroup=is_group,
|
||||
msg=dc_msg,
|
||||
# Replies use Discord's reply mechanism, no manual @mention needed
|
||||
no_need_at=True,
|
||||
)
|
||||
if context:
|
||||
context["session_id"] = session_id
|
||||
context["receiver"] = str(message.channel.id)
|
||||
context["discord_channel_id"] = message.channel.id
|
||||
context["discord_reply_to_msg_id"] = message.id if is_group else None
|
||||
self.produce(context)
|
||||
logger.debug(f"[Discord] received: type={ctype}, content={str(dc_msg.content)[:80]}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] _on_message error: {e}", exc_info=True)
|
||||
|
||||
async def _do_cancel(self, session_id: str, message):
|
||||
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
|
||||
try:
|
||||
from agent.protocol import get_cancel_registry
|
||||
cancelled = get_cancel_registry().cancel_session(session_id)
|
||||
text = "Current task cancelled." if cancelled else "No running task to cancel."
|
||||
await message.channel.send(text)
|
||||
logger.info(f"[Discord] /cancel session={session_id}, cancelled={cancelled}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] /cancel error: {e}", exc_info=True)
|
||||
|
||||
async def _parse_message(self, message):
|
||||
"""Parse a discord message and return (ctype, content, caption).
|
||||
|
||||
- content is text for ContextType.TEXT, otherwise the local file path
|
||||
- caption is the optional text accompanying an attachment; empty for plain text
|
||||
"""
|
||||
text = (message.content or "").strip()
|
||||
attachments = message.attachments or []
|
||||
|
||||
if attachments:
|
||||
# Handle the first attachment; caption is the accompanying message text
|
||||
att = attachments[0]
|
||||
content_type = (att.content_type or "").lower()
|
||||
name = att.filename or str(att.id)
|
||||
path = await self._download_attachment(att, name)
|
||||
if not path:
|
||||
return (None, None, "")
|
||||
is_image = content_type.startswith("image/") or name.lower().endswith(
|
||||
(".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp")
|
||||
)
|
||||
if is_image:
|
||||
return (ContextType.IMAGE, path, text)
|
||||
return (ContextType.FILE, path, text)
|
||||
|
||||
if text:
|
||||
return (ContextType.TEXT, text, "")
|
||||
|
||||
return (None, None, "")
|
||||
|
||||
async def _download_attachment(self, attachment, name: str):
|
||||
"""Download a discord attachment into the local tmp dir; return path or None."""
|
||||
try:
|
||||
tmp_dir = DiscordMessage.get_tmp_dir()
|
||||
safe_name = re.sub(r"[^\w.\-]", "_", name)
|
||||
# Prefix with attachment id to avoid name collisions
|
||||
local_path = os.path.join(tmp_dir, f"{attachment.id}_{safe_name}")
|
||||
await attachment.save(local_path)
|
||||
logger.debug(f"[Discord] downloaded {name} -> {local_path}")
|
||||
return local_path
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] download_attachment failed ({name}): {e}")
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Guild trigger logic
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _should_reply_in_guild(self, message) -> bool:
|
||||
"""Decide whether to reply to a guild channel message based on configuration."""
|
||||
mode = conf().get("discord_group_trigger", "mention_or_reply")
|
||||
if mode == "all":
|
||||
return True
|
||||
|
||||
# self._client.user may be None until on_ready completes
|
||||
if not self._client or not self._client.user:
|
||||
return False
|
||||
|
||||
# 1) Mentioned (direct @bot, not @everyone / @role)
|
||||
if self._client.user in message.mentions:
|
||||
return True
|
||||
|
||||
# 2) Reply to a bot message
|
||||
if mode == "mention_or_reply":
|
||||
ref = message.reference
|
||||
resolved = getattr(ref, "resolved", None) if ref else None
|
||||
if resolved and getattr(resolved, "author", None):
|
||||
if resolved.author.id == self._client.user.id:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def _strip_at_mention(self, content: str) -> str:
|
||||
"""Strip <@BOT_ID> / <@!BOT_ID> from guild text."""
|
||||
if not content or not self.bot_user_id:
|
||||
return content
|
||||
pattern = re.compile(r"<@!?" + re.escape(self.bot_user_id) + r">")
|
||||
return pattern.sub("", content).strip()
|
||||
|
||||
@staticmethod
|
||||
def _compute_session_id(message, is_group: bool) -> str:
|
||||
channel_id = message.channel.id
|
||||
user_id = message.author.id
|
||||
if is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
return f"discord_channel_{channel_id}"
|
||||
return f"discord_channel_{channel_id}_{user_id}"
|
||||
return f"discord_user_{user_id}"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Override _compose_context: skip the parent's group whitelist/at checks
|
||||
# (already handled via _should_reply_in_guild). Same idea as telegram / slack.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _compose_context(self, ctype: ContextType, content, **kwargs):
|
||||
context = Context(ctype, content)
|
||||
context.kwargs = kwargs
|
||||
if "channel_type" not in context:
|
||||
context["channel_type"] = self.channel_type
|
||||
if "origin_ctype" not in context:
|
||||
context["origin_ctype"] = ctype
|
||||
|
||||
cmsg = context["msg"]
|
||||
if cmsg.is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
context["session_id"] = cmsg.other_user_id
|
||||
else:
|
||||
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
|
||||
else:
|
||||
context["session_id"] = cmsg.from_user_id
|
||||
context["receiver"] = cmsg.other_user_id
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
|
||||
if img_match_prefix:
|
||||
content = content.replace(img_match_prefix, "", 1)
|
||||
context.type = ContextType.IMAGE_CREATE
|
||||
else:
|
||||
context.type = ContextType.TEXT
|
||||
context.content = (content or "").strip()
|
||||
if "desire_rtype" not in context and conf().get("always_reply_voice"):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
elif ctype == ContextType.VOICE:
|
||||
if "desire_rtype" not in context and (
|
||||
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
|
||||
):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
|
||||
return context
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound: ChatChannel.send -> Discord Gateway/REST
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def send(self, reply: Reply, context: Context):
|
||||
"""Called from cow's sync main thread; marshal the coroutine onto the loop thread."""
|
||||
if self._loop is None or self._client is None:
|
||||
logger.warning("[Discord] client not ready, drop reply")
|
||||
return
|
||||
|
||||
channel_id = context.get("discord_channel_id")
|
||||
if channel_id is None:
|
||||
logger.warning("[Discord] no discord_channel_id in context, drop reply")
|
||||
return
|
||||
|
||||
coro = self._async_send(reply, channel_id)
|
||||
try:
|
||||
future = asyncio.run_coroutine_threadsafe(coro, self._loop)
|
||||
future.result(timeout=180)
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] send failed: {e}")
|
||||
|
||||
async def _async_send(self, reply: Reply, channel_id):
|
||||
try:
|
||||
import discord
|
||||
|
||||
channel = self._client.get_channel(channel_id)
|
||||
if channel is None:
|
||||
# Not in cache (e.g. DM channel); fetch it explicitly
|
||||
channel = await self._client.fetch_channel(channel_id)
|
||||
|
||||
rtype = reply.type
|
||||
content = reply.content
|
||||
|
||||
if rtype in (ReplyType.TEXT, ReplyType.INFO, ReplyType.ERROR):
|
||||
text = str(content) if content is not None else ""
|
||||
if not text:
|
||||
return
|
||||
for chunk in _split_text(text, DISCORD_MSG_LIMIT):
|
||||
await channel.send(chunk)
|
||||
|
||||
elif rtype == ReplyType.IMAGE:
|
||||
# Already a local BytesIO; send it directly
|
||||
content.seek(0)
|
||||
await channel.send(file=discord.File(content, filename="image.png"))
|
||||
|
||||
elif rtype == ReplyType.IMAGE_URL:
|
||||
url = str(content)
|
||||
if url.startswith("file://"):
|
||||
local = url[7:]
|
||||
await channel.send(file=discord.File(local))
|
||||
else:
|
||||
# Post the URL as text; Discord will unfurl it as an image preview
|
||||
await channel.send(url)
|
||||
|
||||
elif rtype in (ReplyType.VOICE, ReplyType.FILE):
|
||||
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
|
||||
caption = getattr(reply, "text_content", None) or None
|
||||
await channel.send(content=caption, file=discord.File(local))
|
||||
|
||||
else:
|
||||
# Fallback: send as plain text
|
||||
await channel.send(str(content))
|
||||
|
||||
logger.info(f"[Discord] sent reply (type={rtype}, channel={channel_id})")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Discord] _async_send error: {e}", exc_info=True)
|
||||
|
||||
|
||||
def _split_text(text: str, limit: int):
|
||||
"""Split long text preferring line breaks to keep markdown structure intact."""
|
||||
if len(text) <= limit:
|
||||
yield text
|
||||
return
|
||||
buf = []
|
||||
size = 0
|
||||
for line in text.splitlines(keepends=True):
|
||||
if size + len(line) > limit and buf:
|
||||
yield "".join(buf)
|
||||
buf, size = [], 0
|
||||
# Hard-split single lines that exceed the limit
|
||||
while len(line) > limit:
|
||||
yield line[:limit]
|
||||
line = line[limit:]
|
||||
buf.append(line)
|
||||
size += len(line)
|
||||
if buf:
|
||||
yield "".join(buf)
|
||||
60
channel/discord/discord_message.py
Normal file
60
channel/discord/discord_message.py
Normal file
@@ -0,0 +1,60 @@
|
||||
"""
|
||||
Discord message adapter.
|
||||
|
||||
Convert a discord.py Message into cow's unified ChatMessage.
|
||||
File downloads are NOT performed here; the channel layer downloads
|
||||
attachments on demand inside the async event loop.
|
||||
"""
|
||||
import os
|
||||
|
||||
from bridge.context import ContextType
|
||||
from channel.chat_message import ChatMessage
|
||||
from common.utils import expand_path
|
||||
from config import conf
|
||||
|
||||
|
||||
class DiscordMessage(ChatMessage):
|
||||
"""Wrap a discord.py Message into the unified ChatMessage."""
|
||||
|
||||
def __init__(self, message, is_group: bool = False, bot_user_id: str = "",
|
||||
ctype: ContextType = ContextType.TEXT, content: str = ""):
|
||||
super().__init__(message)
|
||||
# Basic fields
|
||||
self.msg_id = str(message.id)
|
||||
self.create_time = int(message.created_at.timestamp()) if message.created_at else 0
|
||||
self.ctype = ctype
|
||||
self.content = content
|
||||
|
||||
author = message.author
|
||||
channel = message.channel
|
||||
|
||||
# Sender / chat info
|
||||
from_user_id = str(author.id)
|
||||
from_user_nick = getattr(author, "display_name", None) or getattr(author, "name", None) or from_user_id
|
||||
self.from_user_id = from_user_id
|
||||
self.from_user_nickname = from_user_nick
|
||||
self.to_user_id = bot_user_id or "discord_bot"
|
||||
self.to_user_nickname = bot_user_id or "discord_bot"
|
||||
|
||||
self.is_group = is_group
|
||||
if is_group:
|
||||
# Guild channel: other_user_id = channel_id, actual_user_id = sender id
|
||||
self.other_user_id = str(channel.id)
|
||||
self.other_user_nickname = getattr(channel, "name", None) or str(channel.id)
|
||||
self.actual_user_id = from_user_id
|
||||
self.actual_user_nickname = from_user_nick
|
||||
else:
|
||||
# DM: use channel_id so replies go back to the same DM channel
|
||||
self.other_user_id = str(channel.id)
|
||||
self.other_user_nickname = from_user_nick
|
||||
|
||||
# Whether the bot was triggered by @-mention (set by channel layer)
|
||||
self.is_at = False
|
||||
|
||||
@staticmethod
|
||||
def get_tmp_dir() -> str:
|
||||
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
|
||||
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
|
||||
tmp_dir = os.path.join(workspace_root, "tmp")
|
||||
os.makedirs(tmp_dir, exist_ok=True)
|
||||
return tmp_dir
|
||||
@@ -752,6 +752,9 @@ class FeiShuChanel(ChatChannel):
|
||||
init_in_flight = [False]
|
||||
# 一旦初始化失败就长期标记为 disabled,本次回复不再尝试任何流式调用
|
||||
disabled = [False]
|
||||
# True after agent_cancelled: agent_end stops rewriting the card
|
||||
# with stale final_response and just finalizes current content.
|
||||
cancelled = [False]
|
||||
lock = threading.Lock()
|
||||
|
||||
# ---- 异步推送队列 ----------------------------------------------------
|
||||
@@ -1076,18 +1079,42 @@ class FeiShuChanel(ChatChannel):
|
||||
message_id[0] = None
|
||||
sequence[0] = 0
|
||||
|
||||
elif event_type == "agent_cancelled":
|
||||
# Lock channel into "no-rewrite" mode: the subsequent
|
||||
# agent_end's final_response is from the last *completed*
|
||||
# turn (the user already saw it), so rewriting the card
|
||||
# would duplicate it visually.
|
||||
with lock:
|
||||
cancelled[0] = True
|
||||
|
||||
elif event_type == "agent_end":
|
||||
# 最终回复:用 final_response 覆盖当前流式卡片,然后关闭流式模式。
|
||||
final_response = data.get("final_response", "")
|
||||
if not final_response:
|
||||
return
|
||||
final_text = str(final_response)
|
||||
# 标记 streamed 让 chat_channel 跳过 send()
|
||||
context["feishu_streamed"] = True
|
||||
|
||||
with lock:
|
||||
was_cancelled = cancelled[0]
|
||||
has_card = card_id[0] is not None
|
||||
init_busy = init_in_flight[0]
|
||||
pending_text = current_text[0]
|
||||
|
||||
if was_cancelled:
|
||||
# Cancelled path: finalize the in-flight card with
|
||||
# partial output (or a short marker if empty); drop
|
||||
# stale final_response to avoid duplicating last turn.
|
||||
if has_card:
|
||||
_drain_push_queue()
|
||||
partial = (pending_text or "").rstrip()
|
||||
final_text = partial or "_(已中止)_"
|
||||
_stream_update_text(final_text)
|
||||
_close_streaming_mode(final_text)
|
||||
push_queue.put(None)
|
||||
return
|
||||
|
||||
if not final_response:
|
||||
return
|
||||
final_text = str(final_response)
|
||||
|
||||
# 罕见情况:agent_end 触发时还没创建过卡片(极快返回 / 没有
|
||||
# message_update),主动创建一张承载 final_text。
|
||||
|
||||
1
channel/slack/__init__.py
Normal file
1
channel/slack/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
|
||||
506
channel/slack/slack_channel.py
Normal file
506
channel/slack/slack_channel.py
Normal file
@@ -0,0 +1,506 @@
|
||||
"""
|
||||
Slack channel via Bolt for Python (Socket Mode).
|
||||
|
||||
Features:
|
||||
- Direct message & channel chat (text / image / file)
|
||||
- Channel trigger: @mention or reply in a thread the bot is in (configurable)
|
||||
- /cancel fast-path matches Web channel behaviour
|
||||
- Socket Mode: no public IP / callback URL required, works behind NAT
|
||||
|
||||
Implementation note:
|
||||
slack_bolt's SocketModeHandler is blocking and runs its own background
|
||||
threads. We start it in a dedicated thread so the rest of cow (sync) stays
|
||||
untouched. Inbound events are dispatched onto cow's existing sync
|
||||
ChatChannel.produce() pipeline; outbound send() calls the Slack Web API
|
||||
client directly (it is sync-safe).
|
||||
"""
|
||||
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
|
||||
import requests
|
||||
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.chat_channel import ChatChannel, check_prefix
|
||||
from channel.slack.slack_message import SlackMessage
|
||||
from common.expired_dict import ExpiredDict
|
||||
from common.log import logger
|
||||
from common.singleton import singleton
|
||||
from config import conf
|
||||
|
||||
|
||||
@singleton
|
||||
class SlackChannel(ChatChannel):
|
||||
NOT_SUPPORT_REPLYTYPE = []
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.bot_token = ""
|
||||
self.app_token = ""
|
||||
self.bot_user_id = "" # used to strip @mention and ignore self messages
|
||||
self._app = None
|
||||
self._handler = None
|
||||
self._client = None
|
||||
self._loop_thread = None
|
||||
# Idempotent dedup; Slack retries event delivery on slow ack
|
||||
self._received_msgs = ExpiredDict(60 * 60 * 1)
|
||||
|
||||
# Disable group whitelist / prefix checks (we handle triggering ourselves
|
||||
# in _should_reply_in_channel), aligned with telegram / feishu channels.
|
||||
conf()["group_name_white_list"] = ["ALL_GROUP"]
|
||||
conf()["single_chat_prefix"] = [""]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def startup(self):
|
||||
self.bot_token = conf().get("slack_bot_token", "")
|
||||
self.app_token = conf().get("slack_app_token", "")
|
||||
if not self.bot_token or not self.app_token:
|
||||
err = "[Slack] slack_bot_token and slack_app_token are both required"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
# Guard against the common mistake of swapping the two tokens:
|
||||
# bot token must start with xoxb-, app-level token with xapp-.
|
||||
if not self.bot_token.startswith("xoxb-") or not self.app_token.startswith("xapp-"):
|
||||
err = (
|
||||
"[Slack] token type mismatch: slack_bot_token must start with 'xoxb-' "
|
||||
"and slack_app_token must start with 'xapp-' (they look swapped)"
|
||||
)
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
try:
|
||||
from slack_bolt import App
|
||||
from slack_bolt.adapter.socket_mode import SocketModeHandler
|
||||
except ImportError:
|
||||
err = (
|
||||
"[Slack] slack_bolt is not installed. "
|
||||
"Run: pip install slack_bolt"
|
||||
)
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
try:
|
||||
self._app = App(token=self.bot_token)
|
||||
self._client = self._app.client
|
||||
|
||||
# Resolve our own bot user id (needed for @mention strip / self-ignore)
|
||||
auth = self._client.auth_test()
|
||||
self.bot_user_id = auth.get("user_id", "")
|
||||
self.name = self.bot_user_id # ChatChannel uses self.name to strip @-mention
|
||||
logger.info(f"[Slack] Bot logged in as user_id={self.bot_user_id}, team={auth.get('team')}")
|
||||
except Exception as e:
|
||||
err = f"[Slack] auth_test failed: {e}"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
self._register_handlers()
|
||||
|
||||
self._handler = SocketModeHandler(self._app, self.app_token)
|
||||
|
||||
def _run():
|
||||
try:
|
||||
logger.info("[Slack] Starting Socket Mode connection...")
|
||||
self.report_startup_success()
|
||||
logger.info("[Slack] ✅ Slack bot ready, listening for events")
|
||||
self._handler.start()
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] socket mode crashed: {e}", exc_info=True)
|
||||
self.report_startup_error(str(e))
|
||||
finally:
|
||||
logger.info("[Slack] socket mode exited")
|
||||
|
||||
self._loop_thread = threading.Thread(target=_run, daemon=True, name="slack-socket")
|
||||
self._loop_thread.start()
|
||||
# Block startup() until the handler thread exits, matching other channels'
|
||||
# behaviour (startup is a blocking call).
|
||||
self._loop_thread.join()
|
||||
|
||||
def _register_handlers(self):
|
||||
app = self._app
|
||||
|
||||
# app_mention: bot is @-mentioned in a channel
|
||||
@app.event("app_mention")
|
||||
def _on_app_mention(event, ack):
|
||||
ack()
|
||||
self._handle_event(event, is_group=True)
|
||||
|
||||
# message: DMs and channel messages (including thread replies)
|
||||
@app.event("message")
|
||||
def _on_message(event, ack):
|
||||
ack()
|
||||
self._handle_message_event(event)
|
||||
|
||||
def stop(self):
|
||||
logger.info("[Slack] stop() called")
|
||||
try:
|
||||
if self._handler is not None:
|
||||
self._handler.close()
|
||||
except Exception as e:
|
||||
logger.warning(f"[Slack] handler close error: {e}")
|
||||
if self._loop_thread and self._loop_thread.is_alive():
|
||||
try:
|
||||
self._loop_thread.join(timeout=10)
|
||||
except Exception:
|
||||
pass
|
||||
logger.info("[Slack] stop() completed")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound: slack event -> ChatMessage -> ChatChannel.produce
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _handle_message_event(self, event: dict):
|
||||
"""Route a raw `message` event: skip bot/system noise, decide grouping."""
|
||||
try:
|
||||
logger.debug(
|
||||
f"[Slack] message event: channel_type={event.get('channel_type')}, "
|
||||
f"subtype={event.get('subtype')}, user={event.get('user')}, "
|
||||
f"ts={event.get('ts')}, thread_ts={event.get('thread_ts')}"
|
||||
)
|
||||
# Ignore bot messages (including our own) and message edits/deletes
|
||||
if event.get("bot_id") or event.get("subtype") in ("bot_message", "message_changed", "message_deleted"):
|
||||
return
|
||||
if event.get("user") == self.bot_user_id:
|
||||
return
|
||||
|
||||
channel_type = event.get("channel_type", "")
|
||||
# DM (im) is single chat; channel/group is group chat. app_mention
|
||||
# already covers channel @-mentions, so for plain channel messages we
|
||||
# only react when configured / thread-following.
|
||||
is_group = channel_type in ("channel", "group", "mpim")
|
||||
if is_group:
|
||||
# app_mention handler covers explicit @bot; here we only handle
|
||||
# follow-up replies in threads the bot participates in.
|
||||
if not self._should_reply_in_channel(event):
|
||||
return
|
||||
self._handle_event(event, is_group=is_group)
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] _handle_message_event error: {e}", exc_info=True)
|
||||
|
||||
def _handle_event(self, event: dict, is_group: bool):
|
||||
"""Parse event -> build SlackMessage -> produce()."""
|
||||
try:
|
||||
channel_id = event.get("channel", "")
|
||||
ts = event.get("ts", "")
|
||||
if not channel_id:
|
||||
return
|
||||
|
||||
# Idempotent dedup
|
||||
msg_uid = f"{channel_id}:{ts}"
|
||||
if self._received_msgs.get(msg_uid):
|
||||
return
|
||||
self._received_msgs[msg_uid] = True
|
||||
|
||||
# Parse type + download media if needed.
|
||||
ctype, content, caption = self._parse_event(event)
|
||||
if ctype is None:
|
||||
logger.debug(f"[Slack] unsupported message type, skip. event={event}")
|
||||
return
|
||||
|
||||
# Strip <@bot_user_id> mention from channel text
|
||||
if is_group and self.bot_user_id:
|
||||
if ctype == ContextType.TEXT and content:
|
||||
content = self._strip_at_mention(content)
|
||||
if caption:
|
||||
caption = self._strip_at_mention(caption)
|
||||
|
||||
slack_msg = SlackMessage(
|
||||
event,
|
||||
is_group=is_group,
|
||||
bot_user_id=self.bot_user_id,
|
||||
ctype=ctype,
|
||||
content=content,
|
||||
)
|
||||
slack_msg.is_at = is_group # if we reached here in a channel, bot is mentioned/threaded
|
||||
|
||||
from channel.file_cache import get_file_cache
|
||||
file_cache = get_file_cache()
|
||||
session_id = self._compute_session_id(event, is_group)
|
||||
|
||||
# Media + caption together: treat as a complete query and bypass the cache
|
||||
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
|
||||
tag = "image" if ctype == ContextType.IMAGE else "file"
|
||||
merged_text = f"{caption}\n[{tag}: {content}]"
|
||||
slack_msg.ctype = ContextType.TEXT
|
||||
slack_msg.content = merged_text
|
||||
ctype = ContextType.TEXT
|
||||
logger.info(f"[Slack] Media+caption merged for session {session_id}")
|
||||
# fallthrough to the TEXT branch below
|
||||
|
||||
elif ctype == ContextType.IMAGE:
|
||||
file_cache.add(session_id, content, file_type="image")
|
||||
logger.info(f"[Slack] Image cached for session {session_id}, waiting for query...")
|
||||
return
|
||||
elif ctype == ContextType.FILE:
|
||||
file_cache.add(session_id, content, file_type="file")
|
||||
logger.info(f"[Slack] File cached for session {session_id}: {content}")
|
||||
return
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
# Fast-path: /cancel mirrors Web channel behaviour
|
||||
if (content or "").strip().lower() in ("/cancel", "cancel"):
|
||||
self._do_cancel(session_id, channel_id, event)
|
||||
return
|
||||
|
||||
cached_files = file_cache.get(session_id)
|
||||
if cached_files:
|
||||
refs = []
|
||||
for fi in cached_files:
|
||||
ftype = fi["type"]
|
||||
tag = ftype if ftype in ("image", "video") else "file"
|
||||
refs.append(f"[{tag}: {fi['path']}]")
|
||||
slack_msg.content = (slack_msg.content or "") + "\n" + "\n".join(refs)
|
||||
file_cache.clear(session_id)
|
||||
logger.info(f"[Slack] Attached {len(cached_files)} cached file(s) to query")
|
||||
|
||||
# Reply in the originating thread when present, else start one on this msg
|
||||
thread_ts = event.get("thread_ts") or ts
|
||||
|
||||
context = self._compose_context(
|
||||
slack_msg.ctype,
|
||||
slack_msg.content,
|
||||
isgroup=is_group,
|
||||
msg=slack_msg,
|
||||
# Replies go back into the thread, no manual @mention needed
|
||||
no_need_at=True,
|
||||
)
|
||||
if context:
|
||||
context["session_id"] = session_id
|
||||
context["receiver"] = channel_id
|
||||
context["slack_channel"] = channel_id
|
||||
context["slack_thread_ts"] = thread_ts if is_group else None
|
||||
self.produce(context)
|
||||
logger.debug(f"[Slack] received: type={ctype}, content={str(slack_msg.content)[:80]}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] _handle_event error: {e}", exc_info=True)
|
||||
|
||||
def _do_cancel(self, session_id: str, channel_id: str, event: dict):
|
||||
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
|
||||
try:
|
||||
from agent.protocol import get_cancel_registry
|
||||
cancelled = get_cancel_registry().cancel_session(session_id)
|
||||
text = "Current task cancelled." if cancelled else "No running task to cancel."
|
||||
thread_ts = event.get("thread_ts") or event.get("ts")
|
||||
self._client.chat_postMessage(channel=channel_id, text=text, thread_ts=thread_ts)
|
||||
logger.info(f"[Slack] /cancel session={session_id}, cancelled={cancelled}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] /cancel error: {e}", exc_info=True)
|
||||
|
||||
def _parse_event(self, event: dict):
|
||||
"""Parse a slack event and return (ctype, content, caption).
|
||||
|
||||
- content is text for ContextType.TEXT, otherwise the local file path
|
||||
- caption is the optional text accompanying a file; empty for plain text
|
||||
"""
|
||||
text = (event.get("text") or "").strip()
|
||||
files = event.get("files") or []
|
||||
|
||||
if files:
|
||||
# Handle the first attachment; caption is the accompanying message text
|
||||
f = files[0]
|
||||
mimetype = (f.get("mimetype") or "").lower()
|
||||
url = f.get("url_private_download") or f.get("url_private")
|
||||
name = f.get("name") or f.get("id") or "file"
|
||||
if not url:
|
||||
return (None, None, "")
|
||||
path = self._download_file(url, name)
|
||||
if not path:
|
||||
return (None, None, "")
|
||||
if mimetype.startswith("image/"):
|
||||
return (ContextType.IMAGE, path, text)
|
||||
return (ContextType.FILE, path, text)
|
||||
|
||||
if text:
|
||||
return (ContextType.TEXT, text, "")
|
||||
|
||||
return (None, None, "")
|
||||
|
||||
def _download_file(self, url: str, name: str):
|
||||
"""Download a Slack private file (requires bot token auth) to local tmp dir."""
|
||||
try:
|
||||
headers = {"Authorization": f"Bearer {self.bot_token}"}
|
||||
resp = requests.get(url, headers=headers, timeout=60, stream=True)
|
||||
resp.raise_for_status()
|
||||
tmp_dir = SlackMessage.get_tmp_dir()
|
||||
# Sanitize the name and keep it unique-ish via the url tail
|
||||
safe_name = re.sub(r"[^\w.\-]", "_", name)
|
||||
local_path = os.path.join(tmp_dir, safe_name)
|
||||
with open(local_path, "wb") as fp:
|
||||
for chunk in resp.iter_content(chunk_size=8192):
|
||||
if chunk:
|
||||
fp.write(chunk)
|
||||
logger.debug(f"[Slack] downloaded {name} -> {local_path}")
|
||||
return local_path
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] download_file failed ({name}): {e}")
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Channel trigger logic
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _should_reply_in_channel(self, event: dict) -> bool:
|
||||
"""Decide whether to reply to a plain channel message (no @mention).
|
||||
|
||||
app_mention already handles explicit @bot, so here we only deal with
|
||||
follow-up messages. `all` replies to every message; `mention_or_reply`
|
||||
replies inside threads the bot already participates in.
|
||||
"""
|
||||
mode = conf().get("slack_group_trigger", "mention_or_reply")
|
||||
if mode == "all":
|
||||
return True
|
||||
if mode == "mention_only":
|
||||
return False
|
||||
# mention_or_reply: follow up only within an existing thread
|
||||
return bool(event.get("thread_ts"))
|
||||
|
||||
def _strip_at_mention(self, content: str) -> str:
|
||||
"""Strip <@BOT_USER_ID> from channel text."""
|
||||
if not content or not self.bot_user_id:
|
||||
return content
|
||||
pattern = re.compile(r"<@" + re.escape(self.bot_user_id) + r">", re.IGNORECASE)
|
||||
return pattern.sub("", content).strip()
|
||||
|
||||
@staticmethod
|
||||
def _compute_session_id(event: dict, is_group: bool) -> str:
|
||||
channel_id = event.get("channel", "")
|
||||
user_id = event.get("user", "")
|
||||
if is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
return f"slack_channel_{channel_id}"
|
||||
return f"slack_channel_{channel_id}_{user_id}"
|
||||
return f"slack_user_{user_id}"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Override _compose_context: skip the parent's group whitelist/at checks
|
||||
# (already handled via _should_reply_in_channel). Same idea as telegram.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _compose_context(self, ctype: ContextType, content, **kwargs):
|
||||
context = Context(ctype, content)
|
||||
context.kwargs = kwargs
|
||||
if "channel_type" not in context:
|
||||
context["channel_type"] = self.channel_type
|
||||
if "origin_ctype" not in context:
|
||||
context["origin_ctype"] = ctype
|
||||
|
||||
cmsg = context["msg"]
|
||||
if cmsg.is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
context["session_id"] = cmsg.other_user_id
|
||||
else:
|
||||
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
|
||||
else:
|
||||
context["session_id"] = cmsg.from_user_id
|
||||
context["receiver"] = cmsg.other_user_id
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
|
||||
if img_match_prefix:
|
||||
content = content.replace(img_match_prefix, "", 1)
|
||||
context.type = ContextType.IMAGE_CREATE
|
||||
else:
|
||||
context.type = ContextType.TEXT
|
||||
context.content = (content or "").strip()
|
||||
if "desire_rtype" not in context and conf().get("always_reply_voice"):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
elif ctype == ContextType.VOICE:
|
||||
if "desire_rtype" not in context and (
|
||||
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
|
||||
):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
|
||||
return context
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound: ChatChannel.send -> Slack Web API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def send(self, reply: Reply, context: Context):
|
||||
"""Called from cow's sync main thread; Slack Web client is sync-safe."""
|
||||
if self._client is None:
|
||||
logger.warning("[Slack] client not ready, drop reply")
|
||||
return
|
||||
|
||||
channel_id = context.get("slack_channel")
|
||||
thread_ts = context.get("slack_thread_ts")
|
||||
if not channel_id:
|
||||
logger.warning("[Slack] no slack_channel in context, drop reply")
|
||||
return
|
||||
|
||||
try:
|
||||
self._do_send(reply, channel_id, thread_ts)
|
||||
logger.info(f"[Slack] sent reply (type={reply.type}, channel={channel_id})")
|
||||
except Exception as e:
|
||||
logger.error(f"[Slack] send failed: {e}", exc_info=True)
|
||||
|
||||
def _do_send(self, reply: Reply, channel_id: str, thread_ts):
|
||||
rtype = reply.type
|
||||
content = reply.content
|
||||
|
||||
if rtype in (ReplyType.TEXT, ReplyType.INFO, ReplyType.ERROR):
|
||||
text = str(content) if content is not None else ""
|
||||
if not text:
|
||||
return
|
||||
# Slack caps a message around 40k chars; split conservatively
|
||||
for chunk in _split_text(text, 3500):
|
||||
self._client.chat_postMessage(channel=channel_id, text=chunk, thread_ts=thread_ts)
|
||||
|
||||
elif rtype == ReplyType.IMAGE:
|
||||
# Already a local BytesIO; upload it directly
|
||||
content.seek(0)
|
||||
self._client.files_upload_v2(
|
||||
channel=channel_id, file=content, filename="image.png", thread_ts=thread_ts,
|
||||
)
|
||||
|
||||
elif rtype == ReplyType.IMAGE_URL:
|
||||
url = str(content)
|
||||
if url.startswith("file://"):
|
||||
local = url[7:]
|
||||
self._client.files_upload_v2(
|
||||
channel=channel_id, file=local, thread_ts=thread_ts,
|
||||
)
|
||||
else:
|
||||
# Post the URL as text; Slack will unfurl it as an image preview
|
||||
self._client.chat_postMessage(channel=channel_id, text=url, thread_ts=thread_ts)
|
||||
|
||||
elif rtype in (ReplyType.VOICE, ReplyType.FILE):
|
||||
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
|
||||
caption = getattr(reply, "text_content", None) or None
|
||||
self._client.files_upload_v2(
|
||||
channel=channel_id, file=local, initial_comment=caption, thread_ts=thread_ts,
|
||||
)
|
||||
|
||||
else:
|
||||
# Fallback: send as plain text
|
||||
self._client.chat_postMessage(channel=channel_id, text=str(content), thread_ts=thread_ts)
|
||||
|
||||
|
||||
def _split_text(text: str, limit: int):
|
||||
"""Split long text preferring line breaks to keep markdown structure intact."""
|
||||
if len(text) <= limit:
|
||||
yield text
|
||||
return
|
||||
buf = []
|
||||
size = 0
|
||||
for line in text.splitlines(keepends=True):
|
||||
if size + len(line) > limit and buf:
|
||||
yield "".join(buf)
|
||||
buf, size = [], 0
|
||||
# Hard-split single lines that exceed the limit
|
||||
while len(line) > limit:
|
||||
yield line[:limit]
|
||||
line = line[limit:]
|
||||
buf.append(line)
|
||||
size += len(line)
|
||||
if buf:
|
||||
yield "".join(buf)
|
||||
60
channel/slack/slack_message.py
Normal file
60
channel/slack/slack_message.py
Normal file
@@ -0,0 +1,60 @@
|
||||
"""
|
||||
Slack message adapter.
|
||||
|
||||
Convert a Slack event payload into cow's unified ChatMessage.
|
||||
File downloads are NOT performed here; the channel layer downloads files
|
||||
on demand because it needs the bot token for authenticated download URLs.
|
||||
"""
|
||||
import os
|
||||
|
||||
from bridge.context import ContextType
|
||||
from channel.chat_message import ChatMessage
|
||||
from common.utils import expand_path
|
||||
from config import conf
|
||||
|
||||
|
||||
class SlackMessage(ChatMessage):
|
||||
"""Wrap a Slack event into the unified ChatMessage."""
|
||||
|
||||
def __init__(self, event: dict, is_group: bool = False, bot_user_id: str = "",
|
||||
ctype: ContextType = ContextType.TEXT, content: str = ""):
|
||||
super().__init__(event)
|
||||
# Basic fields
|
||||
self.msg_id = event.get("client_msg_id") or event.get("ts") or ""
|
||||
try:
|
||||
self.create_time = int(float(event.get("ts", 0)))
|
||||
except (TypeError, ValueError):
|
||||
self.create_time = 0
|
||||
self.ctype = ctype
|
||||
self.content = content
|
||||
|
||||
# Sender / chat info
|
||||
from_user_id = event.get("user", "unknown")
|
||||
channel_id = event.get("channel", "")
|
||||
self.from_user_id = from_user_id
|
||||
self.from_user_nickname = from_user_id
|
||||
self.to_user_id = bot_user_id or "slack_bot"
|
||||
self.to_user_nickname = bot_user_id or "slack_bot"
|
||||
|
||||
self.is_group = is_group
|
||||
if is_group:
|
||||
# Channel chat: other_user_id = channel_id, actual_user_id = sender id
|
||||
self.other_user_id = channel_id
|
||||
self.other_user_nickname = channel_id
|
||||
self.actual_user_id = from_user_id
|
||||
self.actual_user_nickname = from_user_id
|
||||
else:
|
||||
# DM: use channel_id so replies go back to the same DM channel
|
||||
self.other_user_id = channel_id or from_user_id
|
||||
self.other_user_nickname = from_user_id
|
||||
|
||||
# Whether the bot was triggered by @-mention (set by channel layer)
|
||||
self.is_at = False
|
||||
|
||||
@staticmethod
|
||||
def get_tmp_dir() -> str:
|
||||
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
|
||||
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
|
||||
tmp_dir = os.path.join(workspace_root, "tmp")
|
||||
os.makedirs(tmp_dir, exist_ok=True)
|
||||
return tmp_dir
|
||||
0
channel/telegram/__init__.py
Normal file
0
channel/telegram/__init__.py
Normal file
719
channel/telegram/telegram_channel.py
Normal file
719
channel/telegram/telegram_channel.py
Normal file
@@ -0,0 +1,719 @@
|
||||
"""
|
||||
Telegram channel via Bot API (long polling mode).
|
||||
|
||||
Features:
|
||||
- Single chat & group chat (text / photo / voice / video / document)
|
||||
- Group trigger: @mention or reply-to-bot (configurable)
|
||||
- /cancel fast-path matches Web channel behaviour
|
||||
- Auto-register bot commands menu on startup (mirrors Web slash menu)
|
||||
- Optional HTTP/SOCKS5 proxy support for restricted networks
|
||||
|
||||
Implementation note:
|
||||
python-telegram-bot is async-first. We run the bot inside a dedicated
|
||||
thread with its own asyncio loop so the rest of cow (which is sync)
|
||||
stays untouched. Inbound updates are dispatched onto cow's existing
|
||||
sync ChatChannel.produce() pipeline; outbound send() schedules
|
||||
coroutines back onto that loop via asyncio.run_coroutine_threadsafe.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.chat_channel import ChatChannel, check_prefix
|
||||
from channel.telegram.telegram_message import TelegramMessage
|
||||
from common.expired_dict import ExpiredDict
|
||||
from common.log import logger
|
||||
from common.singleton import singleton
|
||||
from config import conf
|
||||
|
||||
# Bot command menu, aligned with Web slash commands.
|
||||
# Top-level commands only; sub-commands are entered with a space (e.g. "/skill list").
|
||||
TELEGRAM_BOT_COMMANDS = [
|
||||
("help", "Show command help"),
|
||||
("status", "Show running status"),
|
||||
("context", "View/clear conversation context (sub: clear)"),
|
||||
("skill", "Manage skills (list/search/install/...)"),
|
||||
("memory", "Manage memory (sub: dream)"),
|
||||
("knowledge", "Manage knowledge base (list/on/off)"),
|
||||
("config", "Show current config"),
|
||||
("cancel", "Cancel running agent task"),
|
||||
("logs", "Show recent logs"),
|
||||
("version", "Show version"),
|
||||
]
|
||||
|
||||
|
||||
@singleton
|
||||
class TelegramChannel(ChatChannel):
|
||||
NOT_SUPPORT_REPLYTYPE = []
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.bot_token = ""
|
||||
self.bot_username = "" # used for @-mention matching
|
||||
self._bot = None
|
||||
self._application = None
|
||||
self._loop = None
|
||||
self._loop_thread = None
|
||||
self._stop_event = threading.Event()
|
||||
# Idempotent dedup; TG occasionally redelivers the same update on flaky networks
|
||||
self._received_msgs = ExpiredDict(60 * 60 * 1)
|
||||
|
||||
# Disable group whitelist / prefix checks (we handle triggering ourselves
|
||||
# in _should_reply_in_group), aligned with feishu / wecom_bot channels.
|
||||
conf()["group_name_white_list"] = ["ALL_GROUP"]
|
||||
conf()["single_chat_prefix"] = [""]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def startup(self):
|
||||
self.bot_token = conf().get("telegram_token", "")
|
||||
if not self.bot_token:
|
||||
err = "[Telegram] telegram_token is required"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
try:
|
||||
from telegram.ext import (
|
||||
Application,
|
||||
MessageHandler,
|
||||
CommandHandler,
|
||||
filters,
|
||||
)
|
||||
except ImportError:
|
||||
err = (
|
||||
"[Telegram] python-telegram-bot is not installed. "
|
||||
"Run: pip install python-telegram-bot"
|
||||
)
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
# Run the asyncio event loop in a dedicated thread so the sync cow body
|
||||
# is untouched.
|
||||
self._loop = asyncio.new_event_loop()
|
||||
|
||||
def _run_loop():
|
||||
asyncio.set_event_loop(self._loop)
|
||||
try:
|
||||
self._loop.run_until_complete(self._async_main(Application, MessageHandler, CommandHandler, filters))
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] event loop crashed: {e}", exc_info=True)
|
||||
self.report_startup_error(str(e))
|
||||
finally:
|
||||
try:
|
||||
self._loop.close()
|
||||
except Exception:
|
||||
pass
|
||||
logger.info("[Telegram] event loop exited")
|
||||
|
||||
self._loop_thread = threading.Thread(target=_run_loop, daemon=True, name="telegram-loop")
|
||||
self._loop_thread.start()
|
||||
# Block startup() until the loop thread exits, matching other channels'
|
||||
# behaviour (startup is a blocking call).
|
||||
self._loop_thread.join()
|
||||
|
||||
async def _async_main(self, Application, MessageHandler, CommandHandler, filters):
|
||||
"""Build Application, register handlers, and run polling."""
|
||||
builder = Application.builder().token(self.bot_token)
|
||||
|
||||
# Proxy: prefer telegram_proxy config, fall back to HTTPS_PROXY env var
|
||||
proxy_url = conf().get("telegram_proxy", "") or os.environ.get("HTTPS_PROXY", "")
|
||||
if proxy_url:
|
||||
try:
|
||||
builder = builder.proxy(proxy_url).get_updates_proxy(proxy_url)
|
||||
logger.info(f"[Telegram] using proxy: {proxy_url}")
|
||||
except Exception as e:
|
||||
logger.warning(f"[Telegram] proxy config failed, fallback to direct: {e}")
|
||||
|
||||
# Media uploads (photo/voice/video/document) over a proxy can be slow,
|
||||
# bump read/write/connect/pool timeouts.
|
||||
builder = (
|
||||
builder
|
||||
.read_timeout(60)
|
||||
.write_timeout(120)
|
||||
.connect_timeout(30)
|
||||
.pool_timeout(30)
|
||||
)
|
||||
|
||||
application = builder.build()
|
||||
self._application = application
|
||||
self._bot = application.bot
|
||||
|
||||
# Fetch our own username (needed for @-mention matching in groups)
|
||||
try:
|
||||
me = await self._bot.get_me()
|
||||
self.bot_username = me.username or ""
|
||||
self.name = self.bot_username # ChatChannel uses self.name to strip @-mention
|
||||
logger.info(f"[Telegram] Bot logged in as @{self.bot_username} (id={me.id})")
|
||||
except Exception as e:
|
||||
err = f"[Telegram] get_me failed: {e}"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
# Register the command menu (failure is non-fatal)
|
||||
if conf().get("telegram_register_commands", True):
|
||||
try:
|
||||
from telegram import BotCommand
|
||||
cmds = [BotCommand(name, desc) for name, desc in TELEGRAM_BOT_COMMANDS]
|
||||
await self._bot.set_my_commands(cmds)
|
||||
logger.info(f"[Telegram] Registered {len(cmds)} bot commands")
|
||||
except Exception as e:
|
||||
logger.warning(f"[Telegram] set_my_commands failed: {e}")
|
||||
|
||||
# Handlers:
|
||||
# 1) /cancel uses the fast-path
|
||||
application.add_handler(CommandHandler("cancel", self._on_cancel))
|
||||
# 2) Normal messages (text + media)
|
||||
application.add_handler(MessageHandler(filters.ALL & ~filters.COMMAND, self._on_message))
|
||||
# 3) Other slash commands are forwarded as plain text for the agent to handle
|
||||
application.add_handler(MessageHandler(filters.COMMAND, self._on_command_passthrough))
|
||||
|
||||
# Start polling. drop_pending_updates avoids replaying backlog after restart.
|
||||
# Transient "Server disconnected" / RemoteProtocolError during get_updates
|
||||
# are common over proxies/flaky networks; PTB's network loop auto-retries,
|
||||
# so we only need to keep the noise down (see _quiet_polling_network_errors).
|
||||
self._quiet_polling_network_errors()
|
||||
logger.info("[Telegram] Starting long polling...")
|
||||
await application.initialize()
|
||||
await application.start()
|
||||
await application.updater.start_polling(
|
||||
drop_pending_updates=True,
|
||||
# Long-poll hold time on the server side; smaller value = reconnect more
|
||||
# often but each hung connection fails faster.
|
||||
timeout=30,
|
||||
# Retry forever on transient get_updates network errors instead of giving up.
|
||||
bootstrap_retries=-1,
|
||||
)
|
||||
self.report_startup_success()
|
||||
logger.info("[Telegram] ✅ Telegram bot ready, polling for updates")
|
||||
|
||||
# Block until stop()
|
||||
try:
|
||||
while not self._stop_event.is_set():
|
||||
await asyncio.sleep(0.5)
|
||||
finally:
|
||||
try:
|
||||
await application.updater.stop()
|
||||
await application.stop()
|
||||
await application.shutdown()
|
||||
except Exception as e:
|
||||
logger.warning(f"[Telegram] shutdown error: {e}")
|
||||
|
||||
@staticmethod
|
||||
def _quiet_polling_network_errors():
|
||||
"""Downgrade PTB's noisy 'Exception happened while polling for updates' logs.
|
||||
|
||||
These transient get_updates errors (RemoteProtocolError / NetworkError /
|
||||
TimedOut, typically over a proxy) are auto-retried by PTB's network loop,
|
||||
so logging the full traceback at ERROR is just noise. We attach a filter
|
||||
that drops these specific records while leaving real errors untouched.
|
||||
"""
|
||||
import logging
|
||||
|
||||
class _PollingNoiseFilter(logging.Filter):
|
||||
_NEEDLES = (
|
||||
"Exception happened while polling for updates",
|
||||
"Server disconnected without sending a response",
|
||||
)
|
||||
|
||||
def filter(self, record: logging.LogRecord) -> bool:
|
||||
try:
|
||||
msg = record.getMessage()
|
||||
except Exception:
|
||||
return True
|
||||
if any(n in msg for n in self._NEEDLES):
|
||||
# Keep a single-line breadcrumb at DEBUG, drop the traceback.
|
||||
logger.debug(f"[Telegram] transient polling network error (auto-retrying): {msg.splitlines()[0]}")
|
||||
return False
|
||||
return True
|
||||
|
||||
noise_filter = _PollingNoiseFilter()
|
||||
for name in ("telegram.ext.Updater", "telegram.ext._updater", "telegram.ext"):
|
||||
logging.getLogger(name).addFilter(noise_filter)
|
||||
|
||||
def stop(self):
|
||||
logger.info("[Telegram] stop() called")
|
||||
self._stop_event.set()
|
||||
if self._loop_thread and self._loop_thread.is_alive():
|
||||
try:
|
||||
self._loop_thread.join(timeout=10)
|
||||
except Exception:
|
||||
pass
|
||||
logger.info("[Telegram] stop() completed")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound: telegram update -> ChatMessage -> ChatChannel.produce
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _on_cancel(self, update, _context):
|
||||
"""Fast-path: /cancel calls cancel_session directly without going through agent."""
|
||||
try:
|
||||
from agent.protocol import get_cancel_registry
|
||||
session_id = self._compute_session_id(update)
|
||||
cancelled = get_cancel_registry().cancel_session(session_id)
|
||||
text = "Current task cancelled." if cancelled else "No running task to cancel."
|
||||
await update.effective_message.reply_text(text)
|
||||
logger.info(f"[Telegram] /cancel session={session_id}, cancelled={cancelled}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] /cancel error: {e}", exc_info=True)
|
||||
try:
|
||||
await update.effective_message.reply_text(f"⚠️ /cancel failed: {e}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
async def _on_command_passthrough(self, update, _context):
|
||||
"""All non-/cancel commands fall through to plain message handling."""
|
||||
await self._on_message(update, _context)
|
||||
|
||||
async def _on_message(self, update, _context):
|
||||
"""Telegram update entry: parse message -> build ChatMessage -> produce()."""
|
||||
try:
|
||||
message = update.effective_message
|
||||
chat = update.effective_chat
|
||||
if not message or not chat:
|
||||
return
|
||||
|
||||
# Idempotent dedup
|
||||
msg_uid = f"{chat.id}:{message.message_id}"
|
||||
if self._received_msgs.get(msg_uid):
|
||||
return
|
||||
self._received_msgs[msg_uid] = True
|
||||
|
||||
is_group = chat.type in ("group", "supergroup")
|
||||
|
||||
# Debug log: helpful when group messages are silently dropped
|
||||
if is_group:
|
||||
logger.debug(
|
||||
f"[Telegram] group update received: chat_id={chat.id}, "
|
||||
f"text={(message.text or message.caption or '')[:40]!r}, "
|
||||
f"reply_to_bot={bool(message.reply_to_message and message.reply_to_message.from_user and message.reply_to_message.from_user.username == self.bot_username)}"
|
||||
)
|
||||
|
||||
# Group trigger gate (silently drop if not triggered)
|
||||
if is_group and not self._should_reply_in_group(update):
|
||||
logger.debug(f"[Telegram] group message not triggered (need @{self.bot_username} or reply), skip")
|
||||
return
|
||||
|
||||
# Parse message type + download media if needed.
|
||||
# Media messages with caption return both the local path and the caption text.
|
||||
ctype, content, caption = await self._parse_message(message)
|
||||
if ctype is None:
|
||||
logger.debug(f"[Telegram] unsupported message type, skip. msg={message}")
|
||||
return
|
||||
|
||||
# Strip @bot mention for group text/caption
|
||||
if is_group and self.bot_username:
|
||||
if ctype == ContextType.TEXT and content:
|
||||
content = self._strip_at_mention(content)
|
||||
if caption:
|
||||
caption = self._strip_at_mention(caption)
|
||||
|
||||
tg_msg = TelegramMessage(
|
||||
update,
|
||||
is_group=is_group,
|
||||
bot_username=self.bot_username,
|
||||
ctype=ctype,
|
||||
content=content,
|
||||
)
|
||||
tg_msg.is_at = is_group # If we got here in a group, the bot is mentioned/replied
|
||||
|
||||
# File cache: standalone media goes into cache, the next text query attaches them
|
||||
from channel.file_cache import get_file_cache
|
||||
file_cache = get_file_cache()
|
||||
session_id = self._compute_session_id(update)
|
||||
|
||||
# Media + caption together: treat as a complete query and bypass the cache
|
||||
if ctype in (ContextType.IMAGE, ContextType.FILE) and caption:
|
||||
tag = "image" if ctype == ContextType.IMAGE else "file"
|
||||
merged_text = f"{caption}\n[{tag}: {content}]"
|
||||
tg_msg.ctype = ContextType.TEXT
|
||||
tg_msg.content = merged_text
|
||||
ctype = ContextType.TEXT
|
||||
logger.info(f"[Telegram] Media+caption merged for session {session_id}")
|
||||
# fallthrough to the TEXT branch below
|
||||
|
||||
elif ctype == ContextType.IMAGE:
|
||||
file_cache.add(session_id, content, file_type="image")
|
||||
logger.info(f"[Telegram] Image cached for session {session_id}, waiting for query...")
|
||||
return
|
||||
elif ctype == ContextType.FILE:
|
||||
file_cache.add(session_id, content, file_type="file")
|
||||
logger.info(f"[Telegram] File cached for session {session_id}: {content}")
|
||||
return
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
cached_files = file_cache.get(session_id)
|
||||
if cached_files:
|
||||
refs = []
|
||||
for fi in cached_files:
|
||||
ftype = fi["type"]
|
||||
tag = ftype if ftype in ("image", "video") else "file"
|
||||
refs.append(f"[{tag}: {fi['path']}]")
|
||||
tg_msg.content = (tg_msg.content or "") + "\n" + "\n".join(refs)
|
||||
file_cache.clear(session_id)
|
||||
logger.info(f"[Telegram] Attached {len(cached_files)} cached file(s) to query")
|
||||
|
||||
# Dispatch to cow main pipeline (reuses ChatChannel._compose_context routing)
|
||||
context = self._compose_context(
|
||||
tg_msg.ctype,
|
||||
tg_msg.content,
|
||||
isgroup=is_group,
|
||||
msg=tg_msg,
|
||||
)
|
||||
if context:
|
||||
context["session_id"] = session_id
|
||||
context["receiver"] = str(chat.id)
|
||||
context["telegram_chat_id"] = chat.id
|
||||
context["telegram_reply_to_msg_id"] = message.message_id if is_group else None
|
||||
self.produce(context)
|
||||
logger.debug(f"[Telegram] received: type={ctype}, content={str(tg_msg.content)[:80]}")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] _on_message error: {e}", exc_info=True)
|
||||
|
||||
async def _parse_message(self, message):
|
||||
"""Parse a telegram message and return (ctype, content, caption).
|
||||
|
||||
- content is text for ContextType.TEXT, otherwise the local file path
|
||||
- caption is the optional text accompanying a media message; empty for plain text
|
||||
"""
|
||||
caption = (message.caption or "").strip()
|
||||
|
||||
if message.photo:
|
||||
largest = message.photo[-1]
|
||||
path = await self._download_file(largest.file_id, suffix=".jpg")
|
||||
return (ContextType.IMAGE, path, caption) if path else (None, None, "")
|
||||
|
||||
if message.voice or message.audio:
|
||||
audio_obj = message.voice or message.audio
|
||||
suffix = ".ogg" if message.voice else (
|
||||
"." + (audio_obj.mime_type.split("/")[-1] if getattr(audio_obj, "mime_type", "") else "mp3")
|
||||
)
|
||||
path = await self._download_file(audio_obj.file_id, suffix=suffix)
|
||||
return (ContextType.VOICE, path, caption) if path else (None, None, "")
|
||||
|
||||
if message.video or message.video_note:
|
||||
video_obj = message.video or message.video_note
|
||||
path = await self._download_file(video_obj.file_id, suffix=".mp4")
|
||||
return (ContextType.FILE, path, caption) if path else (None, None, "")
|
||||
|
||||
if message.document:
|
||||
doc = message.document
|
||||
ext = ""
|
||||
if doc.file_name and "." in doc.file_name:
|
||||
ext = "." + doc.file_name.rsplit(".", 1)[-1]
|
||||
path = await self._download_file(doc.file_id, suffix=ext, original_name=doc.file_name)
|
||||
if not path:
|
||||
return (None, None, "")
|
||||
# Image-typed documents (user picked "send as file") are treated as images
|
||||
mime = (doc.mime_type or "").lower()
|
||||
if mime.startswith("image/"):
|
||||
return (ContextType.IMAGE, path, caption)
|
||||
return (ContextType.FILE, path, caption)
|
||||
|
||||
if message.text:
|
||||
return (ContextType.TEXT, message.text.strip(), "")
|
||||
|
||||
return (None, None, "")
|
||||
|
||||
async def _download_file(self, file_id: str, suffix: str = "", original_name: str = ""):
|
||||
"""Download via bot.get_file into the local tmp dir; return path or None on failure."""
|
||||
try:
|
||||
f = await self._bot.get_file(file_id)
|
||||
tmp_dir = TelegramMessage.get_tmp_dir()
|
||||
base = original_name or f"{file_id}{suffix or ''}"
|
||||
# Prefix with file_id to avoid name collisions / weird chars
|
||||
safe_name = f"{file_id}_{base}" if original_name else base
|
||||
local_path = os.path.join(tmp_dir, safe_name)
|
||||
await f.download_to_drive(custom_path=local_path)
|
||||
logger.debug(f"[Telegram] downloaded file_id={file_id} -> {local_path}")
|
||||
return local_path
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] download_file failed (file_id={file_id}): {e}")
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Group trigger logic
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _should_reply_in_group(self, update) -> bool:
|
||||
"""Decide whether to reply to a group message based on configuration."""
|
||||
mode = conf().get("telegram_group_trigger", "mention_or_reply")
|
||||
if mode == "all":
|
||||
return True
|
||||
|
||||
message = update.effective_message
|
||||
if not message:
|
||||
return False
|
||||
|
||||
# 1) Mentioned
|
||||
if self.bot_username and self._is_mentioned(message, self.bot_username):
|
||||
return True
|
||||
|
||||
# 2) Reply to a bot message
|
||||
if mode == "mention_or_reply":
|
||||
reply = message.reply_to_message
|
||||
if reply and reply.from_user and reply.from_user.username == self.bot_username:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _is_mentioned(message, bot_username: str) -> bool:
|
||||
"""Check whether entities/caption_entities contain a @mention of the bot."""
|
||||
bot_at = "@" + bot_username.lower()
|
||||
text = (message.text or message.caption or "").lower()
|
||||
if bot_at in text:
|
||||
return True
|
||||
# Also check entities strictly to support text_mention (no-username @)
|
||||
for ent in (message.entities or []) + (message.caption_entities or []):
|
||||
if ent.type == "mention":
|
||||
src = message.text or message.caption or ""
|
||||
if src[ent.offset: ent.offset + ent.length].lower() == bot_at:
|
||||
return True
|
||||
return False
|
||||
|
||||
def _strip_at_mention(self, content: str) -> str:
|
||||
"""Strip @bot_username from group text (case-insensitive)."""
|
||||
if not content or not self.bot_username:
|
||||
return content
|
||||
pattern = re.compile(r"@" + re.escape(self.bot_username), re.IGNORECASE)
|
||||
return pattern.sub("", content).strip()
|
||||
|
||||
@staticmethod
|
||||
def _compute_session_id(update) -> str:
|
||||
chat = update.effective_chat
|
||||
user = update.effective_user
|
||||
is_group = chat.type in ("group", "supergroup")
|
||||
if is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
return f"tg_group_{chat.id}"
|
||||
return f"tg_group_{chat.id}_{user.id}"
|
||||
return f"tg_user_{user.id}"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Override _compose_context: skip the parent's group whitelist/at checks
|
||||
# (already handled in _on_message via _should_reply_in_group). Same idea
|
||||
# as the feishu channel.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _compose_context(self, ctype: ContextType, content, **kwargs):
|
||||
context = Context(ctype, content)
|
||||
context.kwargs = kwargs
|
||||
if "channel_type" not in context:
|
||||
context["channel_type"] = self.channel_type
|
||||
if "origin_ctype" not in context:
|
||||
context["origin_ctype"] = ctype
|
||||
|
||||
cmsg = context["msg"]
|
||||
if cmsg.is_group:
|
||||
if conf().get("group_shared_session", True):
|
||||
context["session_id"] = cmsg.other_user_id
|
||||
else:
|
||||
context["session_id"] = f"{cmsg.from_user_id}:{cmsg.other_user_id}"
|
||||
else:
|
||||
context["session_id"] = cmsg.from_user_id
|
||||
context["receiver"] = cmsg.other_user_id
|
||||
|
||||
if ctype == ContextType.TEXT:
|
||||
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
|
||||
if img_match_prefix:
|
||||
content = content.replace(img_match_prefix, "", 1)
|
||||
context.type = ContextType.IMAGE_CREATE
|
||||
else:
|
||||
context.type = ContextType.TEXT
|
||||
context.content = (content or "").strip()
|
||||
if "desire_rtype" not in context and conf().get("always_reply_voice"):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
elif ctype == ContextType.VOICE:
|
||||
if "desire_rtype" not in context and (
|
||||
conf().get("voice_reply_voice") or conf().get("always_reply_voice")
|
||||
):
|
||||
context["desire_rtype"] = ReplyType.VOICE
|
||||
|
||||
return context
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound: ChatChannel.send -> Telegram API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def send(self, reply: Reply, context: Context):
|
||||
"""Called from cow's sync main thread; we marshal the coroutine onto the loop thread."""
|
||||
if self._loop is None or self._bot is None:
|
||||
logger.warning("[Telegram] bot not ready, drop reply")
|
||||
return
|
||||
|
||||
chat_id = context.get("telegram_chat_id")
|
||||
reply_to = context.get("telegram_reply_to_msg_id")
|
||||
if chat_id is None:
|
||||
logger.warning("[Telegram] no telegram_chat_id in context, drop reply")
|
||||
return
|
||||
|
||||
coro = self._async_send(reply, chat_id, reply_to)
|
||||
try:
|
||||
future = asyncio.run_coroutine_threadsafe(coro, self._loop)
|
||||
# Media uploads through a proxy can be slow; let PTB's own timeouts win
|
||||
future.result(timeout=180)
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] send failed: {e}")
|
||||
|
||||
# Number of retries for transient network errors (proxy hiccups etc.)
|
||||
_SEND_RETRIES = 2
|
||||
_SEND_RETRY_BACKOFF = 2.0 # seconds
|
||||
|
||||
async def _send_with_retry(self, send_fn, *, label: str):
|
||||
"""Run a single Telegram API call with retries for transient network errors."""
|
||||
from telegram.error import NetworkError, TimedOut
|
||||
last_err = None
|
||||
for attempt in range(self._SEND_RETRIES + 1):
|
||||
try:
|
||||
return await send_fn()
|
||||
except (NetworkError, TimedOut) as e:
|
||||
last_err = e
|
||||
if attempt >= self._SEND_RETRIES:
|
||||
break
|
||||
wait = self._SEND_RETRY_BACKOFF * (attempt + 1)
|
||||
logger.warning(
|
||||
f"[Telegram] {label} transient error (attempt {attempt + 1}/"
|
||||
f"{self._SEND_RETRIES + 1}): {e}; retry in {wait}s"
|
||||
)
|
||||
await asyncio.sleep(wait)
|
||||
raise last_err
|
||||
|
||||
async def _async_send(self, reply: Reply, chat_id, reply_to_msg_id):
|
||||
try:
|
||||
rtype = reply.type
|
||||
content = reply.content
|
||||
|
||||
if rtype == ReplyType.TEXT or rtype == ReplyType.INFO or rtype == ReplyType.ERROR:
|
||||
# Telegram caps a single text message at 4096 chars; auto-split
|
||||
text = str(content) if content is not None else ""
|
||||
if not text:
|
||||
return
|
||||
for chunk in _split_text(text, 4000):
|
||||
await self._send_with_retry(
|
||||
lambda c=chunk: self._bot.send_message(
|
||||
chat_id=chat_id,
|
||||
text=c,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
# Avoid failing the whole send if reply_to was deleted
|
||||
allow_sending_without_reply=True,
|
||||
),
|
||||
label="send_message",
|
||||
)
|
||||
|
||||
elif rtype == ReplyType.IMAGE:
|
||||
# Already a local BytesIO; send it directly
|
||||
content.seek(0)
|
||||
await self._send_with_retry(
|
||||
lambda: self._bot.send_photo(
|
||||
chat_id=chat_id,
|
||||
photo=content,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
),
|
||||
label="send_photo",
|
||||
)
|
||||
|
||||
elif rtype == ReplyType.IMAGE_URL:
|
||||
url = str(content)
|
||||
if url.startswith("file://"):
|
||||
local = url[7:]
|
||||
# Open inside the lambda so each retry gets a fresh stream
|
||||
async def _send_local_photo():
|
||||
with open(local, "rb") as f:
|
||||
return await self._bot.send_photo(
|
||||
chat_id=chat_id, photo=f,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
)
|
||||
await self._send_with_retry(_send_local_photo, label="send_photo(file)")
|
||||
else:
|
||||
await self._send_with_retry(
|
||||
lambda: self._bot.send_photo(
|
||||
chat_id=chat_id, photo=url,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
),
|
||||
label="send_photo(url)",
|
||||
)
|
||||
|
||||
elif rtype == ReplyType.VOICE:
|
||||
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
|
||||
async def _send_voice():
|
||||
with open(local, "rb") as f:
|
||||
return await self._bot.send_voice(
|
||||
chat_id=chat_id, voice=f,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
)
|
||||
await self._send_with_retry(_send_voice, label="send_voice")
|
||||
|
||||
elif rtype == ReplyType.FILE:
|
||||
# Videos go through send_video, everything else through send_document
|
||||
local = content[7:] if isinstance(content, str) and content.startswith("file://") else content
|
||||
# File replies may carry an accompanying text caption
|
||||
caption = getattr(reply, "text_content", None) or None
|
||||
is_video = isinstance(local, str) and local.lower().endswith(
|
||||
(".mp4", ".mov", ".avi", ".mkv", ".webm")
|
||||
)
|
||||
|
||||
async def _send_file():
|
||||
with open(local, "rb") as f:
|
||||
if is_video:
|
||||
return await self._bot.send_video(
|
||||
chat_id=chat_id, video=f, caption=caption,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
)
|
||||
return await self._bot.send_document(
|
||||
chat_id=chat_id, document=f, caption=caption,
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
)
|
||||
await self._send_with_retry(_send_file, label="send_video" if is_video else "send_document")
|
||||
|
||||
else:
|
||||
# Fallback: send as plain text
|
||||
await self._send_with_retry(
|
||||
lambda: self._bot.send_message(
|
||||
chat_id=chat_id, text=str(content),
|
||||
reply_to_message_id=reply_to_msg_id,
|
||||
allow_sending_without_reply=True,
|
||||
),
|
||||
label="send_message(fallback)",
|
||||
)
|
||||
|
||||
logger.info(f"[Telegram] sent reply (type={rtype}, chat_id={chat_id})")
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"[Telegram] _async_send error: {e}", exc_info=True)
|
||||
|
||||
|
||||
def _split_text(text: str, limit: int):
|
||||
"""Split long text preferring line breaks to keep markdown structure intact."""
|
||||
if len(text) <= limit:
|
||||
yield text
|
||||
return
|
||||
buf = []
|
||||
size = 0
|
||||
for line in text.splitlines(keepends=True):
|
||||
if size + len(line) > limit and buf:
|
||||
yield "".join(buf)
|
||||
buf, size = [], 0
|
||||
# Hard-split single lines that exceed the limit
|
||||
while len(line) > limit:
|
||||
yield line[:limit]
|
||||
line = line[limit:]
|
||||
buf.append(line)
|
||||
size += len(line)
|
||||
if buf:
|
||||
yield "".join(buf)
|
||||
62
channel/telegram/telegram_message.py
Normal file
62
channel/telegram/telegram_message.py
Normal file
@@ -0,0 +1,62 @@
|
||||
"""
|
||||
Telegram message adapter.
|
||||
|
||||
Convert a python-telegram-bot Update into cow's unified ChatMessage.
|
||||
File downloads are NOT performed here; the channel layer triggers
|
||||
bot.get_file() on demand because it requires the async event loop.
|
||||
"""
|
||||
import os
|
||||
|
||||
from bridge.context import ContextType
|
||||
from channel.chat_message import ChatMessage
|
||||
from common.utils import expand_path
|
||||
from config import conf
|
||||
|
||||
|
||||
class TelegramMessage(ChatMessage):
|
||||
"""Wrap a Telegram Update into the unified ChatMessage."""
|
||||
|
||||
def __init__(self, update, is_group: bool = False, bot_username: str = "",
|
||||
ctype: ContextType = ContextType.TEXT, content: str = ""):
|
||||
super().__init__(update)
|
||||
message = update.effective_message
|
||||
chat = update.effective_chat
|
||||
user = update.effective_user
|
||||
|
||||
# Basic fields
|
||||
self.msg_id = str(message.message_id) if message else ""
|
||||
self.create_time = int(message.date.timestamp()) if message and message.date else 0
|
||||
self.ctype = ctype
|
||||
self.content = content
|
||||
|
||||
# Sender / chat info
|
||||
from_user_id = str(user.id) if user else "unknown"
|
||||
from_user_nick = (
|
||||
user.full_name if user and user.full_name else (user.username if user else "unknown")
|
||||
)
|
||||
self.from_user_id = from_user_id
|
||||
self.from_user_nickname = from_user_nick or from_user_id
|
||||
self.to_user_id = bot_username or "telegram_bot"
|
||||
self.to_user_nickname = bot_username or "telegram_bot"
|
||||
|
||||
self.is_group = is_group
|
||||
if is_group:
|
||||
# Group: other_user_id = group_id, actual_user_id = sender id
|
||||
self.other_user_id = str(chat.id)
|
||||
self.other_user_nickname = chat.title or str(chat.id)
|
||||
self.actual_user_id = from_user_id
|
||||
self.actual_user_nickname = self.from_user_nickname
|
||||
else:
|
||||
self.other_user_id = from_user_id
|
||||
self.other_user_nickname = self.from_user_nickname
|
||||
|
||||
# Whether the bot was triggered by @-mention or reply (set by channel layer)
|
||||
self.is_at = False
|
||||
|
||||
@staticmethod
|
||||
def get_tmp_dir() -> str:
|
||||
"""Local download directory, aligned with other channels (agent_workspace/tmp)."""
|
||||
workspace_root = expand_path(conf().get("agent_workspace", "~/cow"))
|
||||
tmp_dir = os.path.join(workspace_root, "tmp")
|
||||
os.makedirs(tmp_dir, exist_ok=True)
|
||||
return tmp_dir
|
||||
@@ -1,4 +1,7 @@
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
from bridge.context import *
|
||||
from bridge.reply import Reply, ReplyType
|
||||
@@ -8,6 +11,164 @@ from common.log import logger
|
||||
from config import conf
|
||||
|
||||
|
||||
class _Style:
|
||||
"""ANSI escape codes for terminal styling. Disabled when not a tty."""
|
||||
|
||||
enabled = sys.stdout.isatty()
|
||||
|
||||
RESET = "\033[0m"
|
||||
BOLD = "\033[1m"
|
||||
DIM = "\033[2m"
|
||||
ITALIC = "\033[3m"
|
||||
|
||||
GRAY = "\033[90m"
|
||||
RED = "\033[31m"
|
||||
GREEN = "\033[32m"
|
||||
YELLOW = "\033[33m"
|
||||
BLUE = "\033[34m"
|
||||
MAGENTA = "\033[35m"
|
||||
CYAN = "\033[36m"
|
||||
|
||||
@classmethod
|
||||
def wrap(cls, text, *codes):
|
||||
if not cls.enabled or not codes:
|
||||
return text
|
||||
return "".join(codes) + text + cls.RESET
|
||||
|
||||
|
||||
class TerminalAgentRenderer:
|
||||
"""Render agent stream events to the terminal in real time.
|
||||
|
||||
Reuses the same `on_event` mechanism as the web channel so the terminal
|
||||
can show reasoning, tool calls and streaming answer text just like the web UI.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
self._reasoning_active = False
|
||||
self._answer_active = False
|
||||
self._has_output = False
|
||||
# Track tool execution start time as a fallback when the event omits it
|
||||
self._tool_started_at = {}
|
||||
|
||||
def _print(self, text, end="", flush=True):
|
||||
sys.stdout.write(text)
|
||||
if end:
|
||||
sys.stdout.write(end)
|
||||
if flush:
|
||||
sys.stdout.flush()
|
||||
self._has_output = True
|
||||
|
||||
def _close_section(self):
|
||||
"""Finish the currently open streaming section (reasoning or answer)."""
|
||||
if self._reasoning_active:
|
||||
self._print("", end="\n")
|
||||
self._reasoning_active = False
|
||||
if self._answer_active:
|
||||
self._print("", end="\n")
|
||||
self._answer_active = False
|
||||
|
||||
def _format_arguments(self, arguments):
|
||||
try:
|
||||
if isinstance(arguments, (dict, list)):
|
||||
text = json.dumps(arguments, ensure_ascii=False)
|
||||
else:
|
||||
text = str(arguments)
|
||||
except Exception:
|
||||
text = str(arguments)
|
||||
# Keep tool input compact in the terminal
|
||||
if len(text) > 300:
|
||||
text = text[:300] + "…"
|
||||
return text
|
||||
|
||||
def handle_event(self, event: dict):
|
||||
try:
|
||||
self._handle_event(event)
|
||||
except Exception as e:
|
||||
logger.debug(f"[Terminal] render event error: {e}")
|
||||
|
||||
def _handle_event(self, event: dict):
|
||||
event_type = event.get("type")
|
||||
data = event.get("data", {}) or {}
|
||||
|
||||
if event_type == "agent_start":
|
||||
self._print("\n" + _Style.wrap("Agent: ", _Style.BOLD, _Style.GREEN), end="\n")
|
||||
|
||||
elif event_type == "reasoning_update":
|
||||
delta = data.get("delta", "")
|
||||
if not delta:
|
||||
return
|
||||
if self._answer_active:
|
||||
self._close_section()
|
||||
if not self._reasoning_active:
|
||||
self._print(_Style.wrap("💭 思考 ", _Style.DIM, _Style.MAGENTA), end="\n")
|
||||
self._reasoning_active = True
|
||||
self._print(_Style.wrap(delta, _Style.DIM, _Style.ITALIC))
|
||||
|
||||
elif event_type == "message_update":
|
||||
delta = data.get("delta", "")
|
||||
if not delta:
|
||||
return
|
||||
if self._reasoning_active:
|
||||
self._close_section()
|
||||
self._answer_active = True
|
||||
self._print(delta)
|
||||
|
||||
elif event_type == "tool_execution_start":
|
||||
self._close_section()
|
||||
tool_name = data.get("tool_name", "tool")
|
||||
tool_id = data.get("tool_call_id")
|
||||
arguments = data.get("arguments", {})
|
||||
self._tool_started_at[tool_id] = time.time()
|
||||
header = _Style.wrap(f"🔧 {tool_name}", _Style.BOLD, _Style.CYAN)
|
||||
args_str = self._format_arguments(arguments)
|
||||
self._print(f"{header} {_Style.wrap(args_str, _Style.GRAY)}", end="\n")
|
||||
|
||||
elif event_type == "tool_execution_end":
|
||||
tool_name = data.get("tool_name", "tool")
|
||||
tool_id = data.get("tool_call_id")
|
||||
status = data.get("status", "success")
|
||||
result = data.get("result", "")
|
||||
exec_time = data.get("execution_time")
|
||||
if exec_time is None and tool_id in self._tool_started_at:
|
||||
exec_time = time.time() - self._tool_started_at.pop(tool_id, time.time())
|
||||
success = status == "success"
|
||||
icon = "✓" if success else "✗"
|
||||
color = _Style.GREEN if success else _Style.RED
|
||||
result_str = str(result)
|
||||
if len(result_str) > 500:
|
||||
result_str = result_str[:500] + "…"
|
||||
# Indent multi-line tool output for readability
|
||||
result_str = result_str.replace("\n", "\n ")
|
||||
cost = f" ({exec_time:.2f}s)" if isinstance(exec_time, (int, float)) else ""
|
||||
self._print(
|
||||
_Style.wrap(f" {icon} {tool_name}{cost}", color) + " " + _Style.wrap(result_str, _Style.GRAY),
|
||||
end="\n",
|
||||
)
|
||||
|
||||
elif event_type == "file_to_send":
|
||||
self._close_section()
|
||||
file_path = data.get("path", "")
|
||||
file_name = data.get("file_name", "")
|
||||
label = file_name or file_path
|
||||
self._print(_Style.wrap(f"📎 文件: {label}", _Style.BLUE), end="\n")
|
||||
|
||||
elif event_type == "error":
|
||||
self._close_section()
|
||||
err_msg = data.get("error") or "unknown error"
|
||||
self._print(_Style.wrap(f"❌ {err_msg}", _Style.BOLD, _Style.RED), end="\n")
|
||||
|
||||
elif event_type == "agent_cancelled":
|
||||
self._close_section()
|
||||
self._print(_Style.wrap("⏹ 已中止", _Style.YELLOW), end="\n")
|
||||
|
||||
elif event_type == "agent_end":
|
||||
self._close_section()
|
||||
|
||||
def finish(self):
|
||||
"""Ensure any open section is closed at the end of a turn."""
|
||||
self._close_section()
|
||||
|
||||
|
||||
class TerminalMessage(ChatMessage):
|
||||
def __init__(
|
||||
self,
|
||||
@@ -29,17 +190,33 @@ class TerminalMessage(ChatMessage):
|
||||
class TerminalChannel(ChatChannel):
|
||||
NOT_SUPPORT_REPLYTYPE = [ReplyType.VOICE]
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
# Per-request renderers keyed by request_id; used to detect whether
|
||||
# agent text was already streamed so send() can avoid duplicate output.
|
||||
self._renderers = {}
|
||||
# Callback that restores TTY attributes on exit (set in startup).
|
||||
self._restore_terminal = None
|
||||
|
||||
def send(self, reply: Reply, context: Context):
|
||||
print("\nBot:")
|
||||
request_id = context.get("request_id") if context else None
|
||||
renderer = self._renderers.pop(request_id, None) if request_id else None
|
||||
streamed = renderer is not None and renderer._has_output
|
||||
|
||||
if renderer is not None:
|
||||
renderer.finish()
|
||||
|
||||
if reply.type == ReplyType.IMAGE:
|
||||
from PIL import Image
|
||||
|
||||
image_storage = reply.content
|
||||
image_storage.seek(0)
|
||||
img = Image.open(image_storage)
|
||||
if not streamed:
|
||||
print("\nAgent: ")
|
||||
print("<IMAGE>")
|
||||
img.show()
|
||||
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
|
||||
elif reply.type == ReplyType.IMAGE_URL: # download image from url
|
||||
import io
|
||||
|
||||
import requests
|
||||
@@ -52,38 +229,122 @@ class TerminalChannel(ChatChannel):
|
||||
image_storage.write(block)
|
||||
image_storage.seek(0)
|
||||
img = Image.open(image_storage)
|
||||
if not streamed:
|
||||
print("\nAgent: ")
|
||||
print(img_url)
|
||||
img.show()
|
||||
else:
|
||||
print(reply.content)
|
||||
print("\nUser:", end="")
|
||||
# When agent already streamed the answer, skip re-printing the
|
||||
# final text to avoid duplication; just emit a trailing newline.
|
||||
if streamed:
|
||||
print()
|
||||
else:
|
||||
print("\nAgent: ")
|
||||
print(reply.content)
|
||||
print("\nUser: ", end="")
|
||||
sys.stdout.flush()
|
||||
return
|
||||
|
||||
def _silence_console_logging(self):
|
||||
"""Mute console log output so background-thread logs (web/MCP/scheduler)
|
||||
don't flood the interactive terminal. Logs still go to run.log in full.
|
||||
|
||||
Configurable via `terminal_log_level` (default ERROR). The file handler
|
||||
is untouched, so run.log keeps the complete log.
|
||||
"""
|
||||
import logging
|
||||
|
||||
level_name = str(conf().get("terminal_log_level", "ERROR")).upper()
|
||||
level = getattr(logging, level_name, logging.ERROR)
|
||||
root_logger = logging.getLogger("log")
|
||||
for handler in root_logger.handlers:
|
||||
# Only raise the level of the stdout/stderr stream handler;
|
||||
# keep FileHandler at the logger's level so run.log stays complete.
|
||||
if isinstance(handler, logging.StreamHandler) and not isinstance(handler, logging.FileHandler):
|
||||
handler.setLevel(level)
|
||||
|
||||
def _install_terminal_guard(self):
|
||||
"""Save TTY attributes and register restore hooks so the terminal is
|
||||
never left in a broken state (no echo / raw mode / leftover ANSI) after
|
||||
the process exits, especially when Ctrl+C interrupts a blocking input().
|
||||
"""
|
||||
if not sys.stdin.isatty():
|
||||
return
|
||||
try:
|
||||
import atexit
|
||||
import termios
|
||||
|
||||
saved_attrs = termios.tcgetattr(sys.stdin.fileno())
|
||||
|
||||
def _restore():
|
||||
try:
|
||||
termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, saved_attrs)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
if _Style.enabled:
|
||||
sys.stdout.write(_Style.RESET)
|
||||
sys.stdout.flush()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
self._restore_terminal = _restore
|
||||
atexit.register(_restore)
|
||||
except Exception as e:
|
||||
# termios is unavailable on Windows; skip the guard there.
|
||||
logger.debug(f"[Terminal] terminal guard not installed: {e}")
|
||||
self._restore_terminal = None
|
||||
|
||||
def startup(self):
|
||||
context = Context()
|
||||
logger.setLevel("WARN")
|
||||
print("\nPlease input your question:\nUser:", end="")
|
||||
self._silence_console_logging()
|
||||
self._install_terminal_guard()
|
||||
print("\nPlease input your question:\nUser: ", end="")
|
||||
sys.stdout.flush()
|
||||
msg_id = 0
|
||||
while True:
|
||||
try:
|
||||
prompt = self.get_input()
|
||||
except KeyboardInterrupt:
|
||||
print("\nExiting...")
|
||||
sys.exit()
|
||||
except (KeyboardInterrupt, EOFError):
|
||||
self._shutdown()
|
||||
msg_id += 1
|
||||
trigger_prefixs = conf().get("single_chat_prefix", [""])
|
||||
if check_prefix(prompt, trigger_prefixs) is None:
|
||||
prompt = trigger_prefixs[0] + prompt # 给没触发的消息加上触发前缀
|
||||
prompt = trigger_prefixs[0] + prompt # add trigger prefix to untriggered messages
|
||||
|
||||
context = self._compose_context(ContextType.TEXT, prompt, msg=TerminalMessage(msg_id, prompt))
|
||||
context["isgroup"] = False
|
||||
if context:
|
||||
# Attach an agent event renderer so reasoning / tool calls /
|
||||
# streaming answer show up live in the terminal (web-like UX).
|
||||
request_id = str(msg_id)
|
||||
context["request_id"] = request_id
|
||||
renderer = TerminalAgentRenderer()
|
||||
self._renderers[request_id] = renderer
|
||||
context["on_event"] = renderer.handle_event
|
||||
self.produce(context)
|
||||
else:
|
||||
raise Exception("context is None")
|
||||
|
||||
def _shutdown(self):
|
||||
"""Restore terminal state and terminate the whole process.
|
||||
|
||||
startup() runs in a daemon sub-thread, so sys.exit() would only kill
|
||||
this thread and leave the main process (and web/MCP/scheduler threads)
|
||||
alive, holding the terminal in a half-occupied state -> laggy input.
|
||||
We reset any leftover ANSI styling and hard-exit the process instead.
|
||||
"""
|
||||
# Restore TTY attributes and reset any leftover ANSI styling
|
||||
# (e.g. interrupted mid-stream output) before terminating.
|
||||
if self._restore_terminal:
|
||||
self._restore_terminal()
|
||||
elif _Style.enabled:
|
||||
sys.stdout.write(_Style.RESET)
|
||||
sys.stdout.write("\nExiting...\n")
|
||||
sys.stdout.flush()
|
||||
# Hard-exit the entire process from a daemon thread.
|
||||
os._exit(0)
|
||||
|
||||
def get_input(self):
|
||||
"""
|
||||
Multi-line input function
|
||||
|
||||
@@ -47,11 +47,30 @@
|
||||
This runs synchronously in <head> so the correct class is on <html>
|
||||
before any CSS or body rendering occurs. -->
|
||||
<script>
|
||||
// Map an arbitrary locale string (zh-CN, en-US, fr ...) to 'zh' / 'en',
|
||||
// or '' when unrecognized so callers can fall through to the next source.
|
||||
window.__cowNormalizeLang__ = function(raw) {
|
||||
if (!raw) return '';
|
||||
var v = String(raw).trim().toLowerCase();
|
||||
if (v === 'auto') return '';
|
||||
if (v.indexOf('zh') === 0) return 'zh';
|
||||
if (v.indexOf('en') === 0) return 'en';
|
||||
return '';
|
||||
};
|
||||
// Resolve the console language by priority:
|
||||
// user choice (localStorage) -> backend-detected -> browser -> 'zh'.
|
||||
window.__cowResolveLang__ = function() {
|
||||
return window.__cowNormalizeLang__(localStorage.getItem('cow_lang'))
|
||||
|| window.__cowNormalizeLang__(window.__COW_DEFAULT_LANG__)
|
||||
|| window.__cowNormalizeLang__(navigator.language || (navigator.languages && navigator.languages[0]))
|
||||
|| 'zh';
|
||||
};
|
||||
(function() {
|
||||
// Backend-resolved default language (from cow_lang config / auto-detect).
|
||||
window.__COW_DEFAULT_LANG__ = '{{COW_DEFAULT_LANG}}';
|
||||
var theme = localStorage.getItem('cow_theme') || 'dark';
|
||||
if (theme === 'dark') document.documentElement.classList.add('dark');
|
||||
var lang = localStorage.getItem('cow_lang') || 'zh';
|
||||
document.documentElement.setAttribute('lang', lang);
|
||||
document.documentElement.setAttribute('lang', window.__cowResolveLang__());
|
||||
})();
|
||||
</script>
|
||||
</head>
|
||||
@@ -266,7 +285,7 @@
|
||||
</button>
|
||||
|
||||
<!-- Docs Link -->
|
||||
<a href="https://docs.cowagent.ai" target="_blank" rel="noopener noreferrer"
|
||||
<a id="docs-link" href="https://docs.cowagent.ai" target="_blank" rel="noopener noreferrer"
|
||||
class="p-2 rounded-lg text-slate-500 dark:text-slate-400 hover:bg-slate-100 dark:hover:bg-white/10
|
||||
cursor-pointer transition-colors duration-150" title="Documentation">
|
||||
<i class="fas fa-book text-base"></i>
|
||||
@@ -445,7 +464,7 @@
|
||||
bg-primary-400 text-white hover:bg-primary-500
|
||||
disabled:bg-slate-300 dark:disabled:bg-slate-600
|
||||
disabled:cursor-not-allowed cursor-pointer transition-colors duration-150"
|
||||
disabled onclick="sendMessage()">
|
||||
disabled>
|
||||
<i class="fas fa-paper-plane text-sm"></i>
|
||||
</button>
|
||||
</div>
|
||||
@@ -601,6 +620,18 @@
|
||||
after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:after:translate-x-full"></div>
|
||||
</label>
|
||||
</div>
|
||||
<div class="flex items-center justify-between">
|
||||
<label class="flex items-center gap-1.5 text-sm font-medium text-slate-600 dark:text-slate-400">
|
||||
<span data-i18n="config_self_evolution">Self-Evolution</span>
|
||||
<span class="cfg-tip" data-tip-key="config_self_evolution_hint"><i class="fas fa-circle-question"></i></span>
|
||||
</label>
|
||||
<label class="relative inline-flex items-center cursor-pointer">
|
||||
<input id="cfg-self-evolution" type="checkbox" class="sr-only peer">
|
||||
<div class="w-9 h-5 bg-slate-200 dark:bg-slate-700 peer-checked:bg-primary-400 rounded-full
|
||||
after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white
|
||||
after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:after:translate-x-full"></div>
|
||||
</label>
|
||||
</div>
|
||||
<div class="flex items-center justify-end gap-3 pt-1">
|
||||
<span id="cfg-agent-status" class="text-xs text-primary-500 opacity-0 transition-opacity duration-300"></span>
|
||||
<button id="cfg-agent-save"
|
||||
@@ -640,6 +671,31 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Language Config Card -->
|
||||
<div class="bg-white dark:bg-[#1A1A1A] rounded-xl border border-slate-200 dark:border-white/10 p-6">
|
||||
<div class="flex items-center gap-3 mb-5">
|
||||
<div class="w-9 h-9 rounded-lg bg-sky-50 dark:bg-sky-900/30 flex items-center justify-center">
|
||||
<i class="fas fa-language text-sky-500 text-sm"></i>
|
||||
</div>
|
||||
<h3 class="font-semibold text-slate-800 dark:text-slate-100" data-i18n="config_language">语言</h3>
|
||||
</div>
|
||||
<div class="space-y-4">
|
||||
<div>
|
||||
<label class="flex items-center gap-1.5 text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="config_language">语言</span>
|
||||
<span class="cfg-tip" data-tip-key="config_language_hint"><i class="fas fa-circle-question"></i></span>
|
||||
</label>
|
||||
<div id="cfg-lang-select" class="cfg-dropdown" tabindex="0">
|
||||
<div class="cfg-dropdown-selected">
|
||||
<span class="cfg-dropdown-text">--</span>
|
||||
<i class="fas fa-chevron-down cfg-dropdown-arrow"></i>
|
||||
</div>
|
||||
<div class="cfg-dropdown-menu"></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -716,7 +772,7 @@
|
||||
</button>
|
||||
<button id="memory-tab-dreams" onclick="switchMemoryTab('dreams')"
|
||||
class="memory-tab px-3 py-1.5 rounded-md text-xs font-medium cursor-pointer transition-colors duration-150">
|
||||
<i class="fas fa-moon mr-1.5"></i><span data-i18n="memory_tab_dreams">梦境日记</span>
|
||||
<i class="fas fa-seedling mr-1.5"></i><span data-i18n="memory_tab_dreams">自主进化</span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
@@ -784,6 +840,11 @@
|
||||
</div>
|
||||
<div class="flex items-center gap-2">
|
||||
<span id="knowledge-stats" class="text-xs text-slate-400 dark:text-slate-500 hidden sm:inline"></span>
|
||||
<span id="knowledge-action-status" class="text-xs opacity-0 transition-opacity duration-200"></span>
|
||||
<button onclick="createKnowledgeCategory()"
|
||||
class="flex items-center gap-1.5 px-3 py-1.5 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-xs font-medium cursor-pointer transition-colors">
|
||||
<i class="fas fa-folder-plus"></i><span data-i18n="knowledge_new_category">新建分类</span>
|
||||
</button>
|
||||
<div class="flex items-center bg-slate-100 dark:bg-white/10 rounded-lg p-0.5">
|
||||
<button id="knowledge-tab-docs" onclick="switchKnowledgeTab('docs')"
|
||||
class="knowledge-tab px-3 py-1.5 rounded-md text-xs font-medium cursor-pointer transition-colors duration-150 active">
|
||||
@@ -939,6 +1000,14 @@
|
||||
<h2 class="text-xl font-bold text-slate-800 dark:text-slate-100" data-i18n="tasks_title">定时任务</h2>
|
||||
<p class="text-sm text-slate-500 dark:text-slate-400 mt-1" data-i18n="tasks_desc">查看和管理定时任务</p>
|
||||
</div>
|
||||
<div class="flex items-center gap-2">
|
||||
<button id="task-refresh-btn" onclick="refreshTasksView()"
|
||||
class="px-3 py-2 rounded-lg border border-slate-200 dark:border-white/10
|
||||
text-slate-600 dark:text-slate-300 hover:bg-slate-50 dark:hover:bg-white/5
|
||||
text-sm font-medium cursor-pointer transition-colors duration-150">
|
||||
<i class="fas fa-refresh text-xs"></i>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="tasks-empty" class="flex flex-col items-center justify-center py-20">
|
||||
<div class="w-16 h-16 rounded-2xl bg-rose-50 dark:bg-rose-900/20 flex items-center justify-center mb-4">
|
||||
@@ -1012,6 +1081,34 @@
|
||||
</div><!-- /main-content -->
|
||||
</div><!-- /app -->
|
||||
|
||||
<!-- Knowledge Action Dialog -->
|
||||
<div id="knowledge-dialog-overlay" class="fixed inset-0 bg-black/50 z-[200] hidden flex items-center justify-center">
|
||||
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl w-full max-w-md mx-4 overflow-hidden">
|
||||
<div class="p-6">
|
||||
<div class="flex items-center gap-3 mb-5">
|
||||
<div class="w-10 h-10 rounded-xl bg-emerald-50 dark:bg-emerald-900/20 flex items-center justify-center">
|
||||
<i id="knowledge-dialog-icon" class="fas fa-folder text-emerald-500"></i>
|
||||
</div>
|
||||
<div>
|
||||
<h3 id="knowledge-dialog-title" class="font-semibold text-slate-800 dark:text-slate-100"></h3>
|
||||
<p id="knowledge-dialog-subtitle" class="text-xs text-slate-400 dark:text-slate-500 mt-0.5"></p>
|
||||
</div>
|
||||
</div>
|
||||
<label id="knowledge-dialog-label" class="block text-sm font-medium text-slate-600 dark:text-slate-300 mb-1.5"></label>
|
||||
<input id="knowledge-dialog-input" type="text"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500">
|
||||
<select id="knowledge-dialog-select"
|
||||
class="hidden w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600 bg-slate-50 dark:bg-[#222] text-sm text-slate-800 dark:text-slate-100 focus:outline-none focus:border-primary-500"></select>
|
||||
<p id="knowledge-dialog-hint" class="mt-2 text-xs text-slate-400 dark:text-slate-500"></p>
|
||||
<p id="knowledge-dialog-error" class="mt-2 text-xs text-red-500 hidden"></p>
|
||||
</div>
|
||||
<div class="flex justify-end gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5">
|
||||
<button id="knowledge-dialog-cancel" class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10 text-slate-600 dark:text-slate-300 text-sm hover:bg-slate-50 dark:hover:bg-white/5">取消</button>
|
||||
<button id="knowledge-dialog-submit" class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium disabled:opacity-50">确定</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Confirm Dialog -->
|
||||
<div id="confirm-dialog-overlay" class="fixed inset-0 bg-black/50 z-[200] hidden flex items-center justify-center">
|
||||
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
|
||||
@@ -1109,6 +1206,240 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Custom Provider Modal (multiple OpenAI-compatible providers) -->
|
||||
<div id="custom-provider-modal-overlay" class="fixed inset-0 bg-black/50 z-[100] hidden flex items-center justify-center">
|
||||
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
|
||||
w-full max-w-md mx-4">
|
||||
<div class="p-6">
|
||||
<div class="flex items-center gap-3 mb-5">
|
||||
<div class="w-10 h-10 rounded-xl bg-primary-50 dark:bg-primary-900/20 flex items-center justify-center flex-shrink-0">
|
||||
<i class="fas fa-sliders text-primary-500"></i>
|
||||
</div>
|
||||
<div class="min-w-0 flex-1">
|
||||
<h3 id="custom-provider-modal-title" class="font-semibold text-slate-800 dark:text-slate-100 text-base"></h3>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="space-y-4">
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5" data-i18n="models_custom_name">名称</label>
|
||||
<input id="custom-provider-name" type="text" autocomplete="off" data-1p-ignore data-lpignore="true"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors">
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">API Base</label>
|
||||
<input id="custom-provider-base" type="text"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 font-mono transition-colors"
|
||||
placeholder="https://...../v1">
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">API Key</label>
|
||||
<input id="custom-provider-key" type="text" autocomplete="off" data-1p-ignore data-lpignore="true"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 font-mono transition-colors">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center justify-between gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5 rounded-b-2xl">
|
||||
<button id="custom-provider-modal-delete"
|
||||
class="px-3 py-2 rounded-lg text-sm font-medium text-red-500 hover:bg-red-50 dark:hover:bg-red-900/20
|
||||
cursor-pointer transition-colors duration-150 hidden"
|
||||
data-i18n="models_custom_delete">删除</button>
|
||||
<span id="custom-provider-modal-status"
|
||||
class="flex-1 text-xs text-primary-500 opacity-0 transition-opacity duration-300 text-left"></span>
|
||||
<button id="custom-provider-modal-cancel"
|
||||
class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10
|
||||
text-slate-600 dark:text-slate-300 text-sm font-medium
|
||||
hover:bg-slate-50 dark:hover:bg-white/5
|
||||
cursor-pointer transition-colors duration-150"
|
||||
data-i18n="cancel">取消</button>
|
||||
<button id="custom-provider-modal-save"
|
||||
class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium
|
||||
cursor-pointer transition-colors duration-150 disabled:opacity-50 disabled:cursor-not-allowed"
|
||||
data-i18n="save">保存</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Task Edit Modal -->
|
||||
<div id="task-edit-modal-overlay" class="fixed inset-0 bg-black/50 z-[100] hidden flex items-center justify-center">
|
||||
<div class="bg-white dark:bg-[#1A1A1A] rounded-2xl border border-slate-200 dark:border-white/10 shadow-xl
|
||||
w-full max-w-2xl mx-4 max-h-[90vh] overflow-y-auto">
|
||||
<div class="p-6">
|
||||
<div class="flex items-center gap-3 mb-5">
|
||||
<div class="w-10 h-10 rounded-xl bg-primary-50 dark:bg-primary-900/20 flex items-center justify-center flex-shrink-0">
|
||||
<i class="fas fa-clock text-primary-500"></i>
|
||||
</div>
|
||||
<div class="min-w-0 flex-1">
|
||||
<h3 class="font-semibold text-slate-800 dark:text-slate-100 text-base" data-i18n="task_edit_title">编辑定时任务</h3>
|
||||
<p id="task-edit-modal-subtitle" class="text-xs text-slate-500 dark:text-slate-400 mt-0.5 font-mono"></p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="space-y-4">
|
||||
<!-- 任务名称和启用状态 -->
|
||||
<div class="flex gap-4 items-end">
|
||||
<div class="flex-1">
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_name">任务名称</span>
|
||||
</label>
|
||||
<input id="task-edit-name" type="text"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors"
|
||||
placeholder="任务名称">
|
||||
</div>
|
||||
<div class="flex items-center gap-2 pb-[2px]">
|
||||
<label class="text-xs font-medium text-slate-600 dark:text-slate-400">
|
||||
<span data-i18n="task_enabled">启用</span>
|
||||
</label>
|
||||
<label class="relative inline-flex items-center cursor-pointer">
|
||||
<input type="checkbox" id="task-edit-enabled" class="sr-only peer">
|
||||
<div class="w-11 h-6 bg-slate-200 peer-focus:outline-none rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:rounded-full after:h-5 after:w-5 after:transition-all peer-checked:bg-primary-500 dark:bg-slate-600 dark:peer-checked:bg-primary-500"></div>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 调度类型 + 调度值 -->
|
||||
<div class="grid grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_schedule_type">调度类型</span>
|
||||
</label>
|
||||
<select id="task-edit-schedule-type"
|
||||
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors
|
||||
appearance-none bg-no-repeat bg-right"
|
||||
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
|
||||
<option value="cron" data-i18n="task_schedule_cron">Cron 表达式</option>
|
||||
<option value="interval" data-i18n="task_schedule_interval">固定间隔</option>
|
||||
<option value="once" data-i18n="task_schedule_once">一次性任务</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<!-- Cron 表达式 -->
|
||||
<div id="task-edit-cron-wrap">
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_cron_expression">Cron 表达式</span>
|
||||
</label>
|
||||
<input id="task-edit-cron-expression" type="text"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 font-mono transition-colors"
|
||||
placeholder="0 9 * * *">
|
||||
</div>
|
||||
|
||||
<!-- 固定间隔 -->
|
||||
<div id="task-edit-interval-wrap" class="hidden">
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_interval_seconds">间隔秒数</span>
|
||||
</label>
|
||||
<input id="task-edit-interval-seconds" type="number" min="60"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors"
|
||||
placeholder="3600">
|
||||
</div>
|
||||
|
||||
<!-- 一次性任务时间 -->
|
||||
<div id="task-edit-once-wrap" class="hidden">
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_once_time">执行时间</span>
|
||||
</label>
|
||||
<input id="task-edit-once-time" type="datetime-local" step="1"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors
|
||||
cursor-pointer"
|
||||
onclick="this.showPicker && this.showPicker()">
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Cron/Interval 提示 -->
|
||||
<p id="task-edit-cron-hint" class="text-xs text-slate-400 dark:text-slate-500">
|
||||
<span data-i18n="task_cron_hint">格式: 分 时 日 月 周,例如 "0 9 * * *" 表示每天 9:00</span>
|
||||
</p>
|
||||
<p id="task-edit-interval-hint" class="text-xs text-slate-400 dark:text-slate-500 hidden">
|
||||
<span data-i18n="task_interval_hint">最小 60 秒,例如 3600 表示每小时执行一次</span>
|
||||
</p>
|
||||
|
||||
<!-- 动作类型 + 通道类型 -->
|
||||
<div class="grid grid-cols-2 gap-4">
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_action_type">动作类型</span>
|
||||
</label>
|
||||
<select id="task-edit-action-type"
|
||||
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors
|
||||
appearance-none bg-no-repeat bg-right"
|
||||
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
|
||||
<option value="send_message" data-i18n="task_action_send_message">发送消息</option>
|
||||
<option value="agent_task" data-i18n="task_action_agent_task">AI 任务</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span data-i18n="task_channel_type">通道类型</span>
|
||||
</label>
|
||||
<select id="task-edit-channel-type"
|
||||
class="w-full px-3 py-2 pr-8 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-[#1A1A1A] text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors
|
||||
appearance-none bg-no-repeat bg-right
|
||||
disabled:opacity-100 disabled:text-slate-800 dark:disabled:text-slate-300 disabled:bg-slate-100 dark:disabled:bg-[#252525] disabled:cursor-not-allowed"
|
||||
style="background-image: url('data:image/svg+xml;charset=UTF-8,%3csvg xmlns=%27http://www.w3.org/2000/svg%27 viewBox=%270 0 20 20%27 fill=%27none%27%3e%3cpath d=%27M7 7l3 3 3-3%27 stroke=%27%23888%27 stroke-width=%272%27 stroke-linecap=%27round%27 stroke-linejoin=%27round%27/%3e%3c/svg%3e'); background-position: right 0.5rem center; background-size: 1.25rem;">
|
||||
</select>
|
||||
<p class="text-xs text-slate-400 dark:text-slate-500 mt-1">
|
||||
<span data-i18n="task_channel_hint">选择定时消息发送的通道</span>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 隐藏的接收者ID字段(自动填充) -->
|
||||
<input id="task-edit-receiver" type="hidden" value="">
|
||||
|
||||
<!-- 消息内容/任务描述 -->
|
||||
<div>
|
||||
<label class="block text-sm font-medium text-slate-600 dark:text-slate-400 mb-1.5">
|
||||
<span id="task-edit-content-label" data-i18n="task_message_content">消息内容</span>
|
||||
</label>
|
||||
<textarea id="task-edit-content" rows="3"
|
||||
class="w-full px-3 py-2 rounded-lg border border-slate-200 dark:border-slate-600
|
||||
bg-slate-50 dark:bg-white/5 text-sm text-slate-800 dark:text-slate-100
|
||||
focus:outline-none focus:border-primary-500 transition-colors resize-none"
|
||||
placeholder="输入消息内容或任务描述"></textarea>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center justify-between gap-3 px-6 py-4 border-t border-slate-100 dark:border-white/5 rounded-b-2xl">
|
||||
<button id="task-edit-modal-delete"
|
||||
class="px-4 py-2 rounded-lg text-sm font-medium text-red-500 hover:bg-red-50 dark:hover:bg-red-900/20
|
||||
cursor-pointer transition-colors duration-150 hidden"
|
||||
data-i18n="task_delete_btn">删除任务</button>
|
||||
<span id="task-edit-modal-status"
|
||||
class="flex-1 text-xs text-primary-500 opacity-0 transition-opacity duration-300 text-left"></span>
|
||||
<button id="task-edit-modal-cancel"
|
||||
class="px-4 py-2 rounded-lg border border-slate-200 dark:border-white/10
|
||||
text-slate-600 dark:text-slate-300 text-sm font-medium
|
||||
hover:bg-slate-50 dark:hover:bg-white/5
|
||||
cursor-pointer transition-colors duration-150"
|
||||
data-i18n="cancel">取消</button>
|
||||
<button id="task-edit-modal-save"
|
||||
class="px-4 py-2 rounded-lg bg-primary-500 hover:bg-primary-600 text-white text-sm font-medium
|
||||
cursor-pointer transition-colors duration-150 disabled:opacity-50 disabled:cursor-not-allowed"
|
||||
data-i18n="save">保存</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script defer src="assets/js/console.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -244,6 +244,52 @@
|
||||
}
|
||||
.dark .session-delete:hover { background: rgba(239, 68, 68, 0.15); }
|
||||
|
||||
/* Rename button: shares the look of the delete button, sits to its left.
|
||||
Negative right margin tightens the gap to the delete button only. */
|
||||
.session-rename {
|
||||
flex-shrink: 0;
|
||||
margin-right: -6px;
|
||||
width: 22px;
|
||||
height: 22px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
border-radius: 6px;
|
||||
color: #9ca3af;
|
||||
font-size: 11px;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s, color 0.15s, background 0.15s;
|
||||
cursor: pointer;
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
}
|
||||
.session-item:hover .session-rename { opacity: 1; }
|
||||
.session-rename:hover {
|
||||
color: #4ABE6E;
|
||||
background: rgba(74, 190, 110, 0.12);
|
||||
}
|
||||
.dark .session-rename:hover { background: rgba(74, 190, 110, 0.18); }
|
||||
|
||||
/* Inline title editor */
|
||||
.session-title-input {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
font-size: 13px;
|
||||
font-family: inherit;
|
||||
color: #111827;
|
||||
background: #ffffff;
|
||||
border: 1px solid #4ABE6E;
|
||||
border-radius: 6px;
|
||||
padding: 2px 6px;
|
||||
outline: none;
|
||||
}
|
||||
.dark .session-title-input {
|
||||
color: #e5e5e5;
|
||||
background: rgba(255, 255, 255, 0.06);
|
||||
border-color: #4ABE6E;
|
||||
}
|
||||
|
||||
/* Context Divider */
|
||||
.context-divider {
|
||||
display: flex;
|
||||
@@ -605,6 +651,18 @@
|
||||
color: inherit;
|
||||
}
|
||||
.tool-error-text { color: #f87171; }
|
||||
.tool-live-output:empty { display: none; }
|
||||
.tool-streaming .tool-live-output:not(:empty)::after {
|
||||
content: ' ';
|
||||
display: inline-block;
|
||||
width: 0.45em;
|
||||
height: 1em;
|
||||
margin-left: 0.15em;
|
||||
vertical-align: -0.15em;
|
||||
background: currentColor;
|
||||
animation: tool-cursor-blink 1s steps(1) infinite;
|
||||
}
|
||||
@keyframes tool-cursor-blink { 50% { opacity: 0; } }
|
||||
|
||||
/* Log level highlighting */
|
||||
.log-line { display: block; }
|
||||
@@ -854,6 +912,14 @@
|
||||
font-size: 11px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
/* "Custom" row is an add-new action: trailing + instead of ✓. */
|
||||
.vendor-picker-add-mark {
|
||||
margin-left: auto;
|
||||
padding-left: 12px;
|
||||
color: #94a3b8;
|
||||
font-size: 11px;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
/* Chat Input */
|
||||
#chat-input {
|
||||
@@ -1191,6 +1257,34 @@
|
||||
background: #EDFDF3;
|
||||
color: #228547;
|
||||
}
|
||||
|
||||
.knowledge-actions {
|
||||
display: flex;
|
||||
gap: 2px;
|
||||
margin-left: auto;
|
||||
opacity: 0;
|
||||
transition: opacity 0.15s;
|
||||
}
|
||||
.knowledge-tree-file:hover .knowledge-actions,
|
||||
.knowledge-tree-group-btn:hover .knowledge-actions,
|
||||
.knowledge-tree-file:focus-within .knowledge-actions,
|
||||
.knowledge-tree-group-btn:focus-within .knowledge-actions {
|
||||
opacity: 1;
|
||||
}
|
||||
.knowledge-action {
|
||||
padding: 3px 5px;
|
||||
border-radius: 5px;
|
||||
color: #94a3b8;
|
||||
font-size: 9px;
|
||||
}
|
||||
.knowledge-action:hover {
|
||||
color: #228547;
|
||||
background: rgba(34, 133, 71, 0.08);
|
||||
}
|
||||
.knowledge-action.danger:hover {
|
||||
color: #ef4444;
|
||||
background: rgba(239, 68, 68, 0.08);
|
||||
}
|
||||
.dark .knowledge-tree-file:hover {
|
||||
background: rgba(255,255,255,0.06);
|
||||
color: #e2e8f0;
|
||||
@@ -1367,3 +1461,213 @@
|
||||
text-align: right;
|
||||
}
|
||||
.voice-pill audio { display: none; }
|
||||
|
||||
/* Send button toggles into a Stop button while an SSE stream is in flight.
|
||||
Match the look of the disabled send button (light grey block + white
|
||||
glyph) so it reads as the same visual element, just paused/idle from
|
||||
sending perspective and clickable to stop. */
|
||||
#send-btn.send-btn-cancel {
|
||||
background-color: rgb(203 213 225) !important; /* slate-300, == disabled send-btn */
|
||||
color: white !important;
|
||||
}
|
||||
#send-btn.send-btn-cancel:hover {
|
||||
background-color: rgb(148 163 184) !important; /* slate-400 */
|
||||
}
|
||||
#send-btn.send-btn-cancel:disabled {
|
||||
background-color: rgb(226 232 240) !important; /* slate-200, while stop is in flight */
|
||||
color: white !important;
|
||||
cursor: progress;
|
||||
}
|
||||
.dark #send-btn.send-btn-cancel {
|
||||
background-color: rgb(71 85 105) !important; /* slate-600, == dark disabled send-btn */
|
||||
color: white !important;
|
||||
}
|
||||
.dark #send-btn.send-btn-cancel:hover {
|
||||
background-color: rgb(100 116 139) !important; /* slate-500 */
|
||||
}
|
||||
.dark #send-btn.send-btn-cancel:disabled {
|
||||
background-color: rgb(51 65 85) !important; /* slate-700 */
|
||||
color: rgb(203 213 225) !important;
|
||||
}
|
||||
|
||||
.agent-cancelled-tag {
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
/* =====================================================================
|
||||
Code Block Enhancements
|
||||
===================================================================== */
|
||||
.code-block-wrapper {
|
||||
position: relative;
|
||||
margin: 1em 0;
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
background: #f8f9fa;
|
||||
border: 1px solid #e2e8f0;
|
||||
}
|
||||
|
||||
.dark .code-block-wrapper {
|
||||
background: #1e293b;
|
||||
border-color: #334155;
|
||||
}
|
||||
|
||||
.code-block-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 0.5em 1em;
|
||||
background: #e2e8f0;
|
||||
border-bottom: 1px solid #cbd5e1;
|
||||
font-size: 0.85em;
|
||||
}
|
||||
|
||||
.dark .code-block-header {
|
||||
background: #0f172a;
|
||||
border-bottom-color: #334155;
|
||||
}
|
||||
|
||||
.code-block-lang {
|
||||
color: #64748b;
|
||||
font-weight: 500;
|
||||
text-transform: lowercase;
|
||||
}
|
||||
|
||||
.dark .code-block-lang {
|
||||
color: #94a3b8;
|
||||
}
|
||||
|
||||
.code-copy-btn {
|
||||
background: transparent;
|
||||
border: none;
|
||||
color: #64748b;
|
||||
cursor: pointer;
|
||||
padding: 0.25em 0.5em;
|
||||
border-radius: 4px;
|
||||
transition: all 0.2s;
|
||||
font-size: 0.9em;
|
||||
}
|
||||
|
||||
.code-copy-btn:hover {
|
||||
background: rgba(100, 116, 139, 0.1);
|
||||
color: #475569;
|
||||
}
|
||||
|
||||
.dark .code-copy-btn {
|
||||
color: #94a3b8;
|
||||
}
|
||||
|
||||
.dark .code-copy-btn:hover {
|
||||
background: rgba(148, 163, 184, 0.1);
|
||||
color: #cbd5e1;
|
||||
}
|
||||
|
||||
.code-block-wrapper pre {
|
||||
margin: 0;
|
||||
border-radius: 0;
|
||||
border: none;
|
||||
}
|
||||
|
||||
/* =====================================================================
|
||||
Drag and Drop Overlay
|
||||
===================================================================== */
|
||||
/* Anchor the absolutely-positioned overlay to the chat view. */
|
||||
#view-chat {
|
||||
position: relative;
|
||||
}
|
||||
|
||||
.drag-overlay {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
bottom: 0;
|
||||
background: rgba(59, 130, 246, 0.1);
|
||||
backdrop-filter: blur(2px);
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
z-index: 9999;
|
||||
pointer-events: none;
|
||||
opacity: 0;
|
||||
transition: opacity 0.2s;
|
||||
}
|
||||
|
||||
.drag-overlay.active {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.drag-overlay.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.drag-overlay-content {
|
||||
background: white;
|
||||
border: 3px dashed #3b82f6;
|
||||
border-radius: 16px;
|
||||
padding: 3em 4em;
|
||||
text-align: center;
|
||||
box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1);
|
||||
animation: bounce 1s ease infinite;
|
||||
}
|
||||
|
||||
.dark .drag-overlay-content {
|
||||
background: #1e293b;
|
||||
border-color: #60a5fa;
|
||||
}
|
||||
|
||||
.drag-overlay-content i {
|
||||
font-size: 4em;
|
||||
color: #3b82f6;
|
||||
margin-bottom: 0.5em;
|
||||
}
|
||||
|
||||
.dark .drag-overlay-content i {
|
||||
color: #60a5fa;
|
||||
}
|
||||
|
||||
.drag-overlay-content p {
|
||||
font-size: 1.5em;
|
||||
font-weight: 600;
|
||||
color: #1e293b;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.dark .drag-overlay-content p {
|
||||
color: #f1f5f9;
|
||||
}
|
||||
|
||||
@keyframes bounce {
|
||||
0%, 100% { transform: translateY(0); }
|
||||
50% { transform: translateY(-10px); }
|
||||
}
|
||||
|
||||
/* =====================================================================
|
||||
Message Action Buttons
|
||||
===================================================================== */
|
||||
.edit-msg-btn,
|
||||
.delete-msg-btn,
|
||||
.regenerate-msg-btn {
|
||||
opacity: 0;
|
||||
transition: opacity 0.2s, color 0.2s;
|
||||
}
|
||||
|
||||
.user-message-group:hover .edit-msg-btn,
|
||||
.user-message-group:hover .delete-msg-btn,
|
||||
.bot-message-group:hover .regenerate-msg-btn {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.edit-msg-btn:hover,
|
||||
.regenerate-msg-btn:hover {
|
||||
color: #3b82f6 !important;
|
||||
}
|
||||
|
||||
.delete-msg-btn:hover {
|
||||
color: #ef4444 !important;
|
||||
}
|
||||
|
||||
.edit-msg-btn:disabled,
|
||||
.delete-msg-btn:disabled {
|
||||
cursor: not-allowed !important;
|
||||
opacity: 0.35 !important;
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
115
channel/wechat_kf/README.md
Normal file
115
channel/wechat_kf/README.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# 微信客服(WeChat Customer Service)通道
|
||||
|
||||
> 与 `channel/wechatcom/`(企微自建应用)是两个**独立的 CoW 通道**:
|
||||
>
|
||||
> - 自建应用:**面向企业内部成员**(员工通过企业微信 App 与机器人对话)。
|
||||
> - 微信客服:**面向外部微信用户**(普通微信用户通过链接/二维码进入对话)。
|
||||
>
|
||||
> 但底层都基于"企微自建应用"——本通道是**通过把一个企微自建应用绑定到微信客服账号**来实现 AI 接管对外咨询,详见 [LinkAI 微信客服接入文档](https://docs.link-ai.tech/platform/link-app/wechat-customer-service)。
|
||||
|
||||
## 一、接入流程概览
|
||||
|
||||
```
|
||||
┌─────────────────────┐ ┌─────────────────────┐ ┌──────────────────┐
|
||||
│ 1. 企业微信后台 │ → │ 2. CoW 配置回调 │ → │ 3. 绑定微信客服 │
|
||||
│ 创建一个自建应用 │ │ 端口 9888 │ │ 账号 │
|
||||
└─────────────────────┘ └─────────────────────┘ └──────────────────┘
|
||||
↓
|
||||
外部微信用户通过
|
||||
链接/二维码 →
|
||||
消息 → CoW Bot
|
||||
```
|
||||
|
||||
> **重要**:建议**单独再创建一个企微自建应用**用于微信客服,**不要复用**已经接入员工内部使用的那个 `wechatcom_app` 应用,否则两个通道会争抢同一个回调地址。
|
||||
|
||||
## 二、企业微信后台配置
|
||||
|
||||
### 1. 创建企微自建应用
|
||||
|
||||
进入 企业微信管理后台 → **应用管理** → **创建应用**。
|
||||
|
||||
### 2. 收集字段
|
||||
|
||||
| 字段 | 来源 | 对应 CoW 配置项 |
|
||||
|---|---|---|
|
||||
| 企业ID(CorpId) | 「我的企业」最下方 | `wechat_kf_corp_id` |
|
||||
| Secret | 进入应用详情 → 点击「查看」(会推送到管理员手机端,在手机上查看) | `wechat_kf_secret` |
|
||||
| Token | 应用「接收消息 → 设置API接收」 | `wechat_kf_token` |
|
||||
| EncodingAESKey | 应用「接收消息 → 设置API接收」 | `wechat_kf_aes_key` |
|
||||
|
||||
> AgentId 在本通道**不需要**(消息发送走的是 `cgi-bin/kf/send_msg`,不依赖 agent_id)。
|
||||
|
||||
### 3. 配置回调地址 + 可信 IP
|
||||
|
||||
在应用「**接收消息 → 设置API接收**」里填:
|
||||
|
||||
- URL:`http://<your-host>:9888/wxkf/`(公网必须可达)
|
||||
- Token / EncodingAESKey:与下方 `config.json` 一致
|
||||
|
||||
回到应用详情页,把服务器公网 IP 填入「**企业可信IP**」。
|
||||
|
||||
### 4. 绑定微信客服账号
|
||||
|
||||
进入 企业微信后台 → **微信客服** → 创建客服账号 → **将该账号绑定到上一步创建的企微自建应用**。
|
||||
|
||||
绑定完成后,进入 **微信客服 → 微信客服账号详情** 页面,在「**接入链接**」一栏:
|
||||
|
||||
- 「复制链接」可拿到形如 `https://work.weixin.qq.com/kfid/kfcd83e5896b9ba07be` 的访问链接
|
||||
- 「生成二维码」可拿到对应二维码
|
||||
|
||||
把链接或二维码推给微信客户使用即可。
|
||||
|
||||
## 三、CoW 配置(`config.json`)
|
||||
|
||||
```json
|
||||
{
|
||||
"channel_type": "wechat_kf",
|
||||
|
||||
"wechat_kf_corp_id": "ww1234567890abcdef",
|
||||
"wechat_kf_secret": "<企微应用的 Secret>",
|
||||
"wechat_kf_token": "<接收消息 Token>",
|
||||
"wechat_kf_aes_key": "<EncodingAESKey>",
|
||||
"wechat_kf_port": 9888
|
||||
}
|
||||
```
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `wechat_kf_corp_id` | 企业 ID |
|
||||
| `wechat_kf_secret` | **绑定到微信客服**的那个企微自建应用的 Secret |
|
||||
| `wechat_kf_token` | 该应用「接收消息」配置的 Token |
|
||||
| `wechat_kf_aes_key` | 该应用「接收消息」配置的 EncodingAESKey |
|
||||
| `wechat_kf_port` | 监听端口,默认 `9888` |
|
||||
|
||||
也支持环境变量:`WECHAT_KF_CORP_ID` / `WECHAT_KF_SECRET` / `WECHAT_KF_TOKEN` / `WECHAT_KF_AES_KEY`。
|
||||
|
||||
## 四、运行
|
||||
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
启动后日志里会看到:
|
||||
|
||||
```
|
||||
[wechat_kf] WeCom customer-service channel started
|
||||
[wechat_kf] Listening on http://0.0.0.0:9888/wxkf/
|
||||
```
|
||||
|
||||
回到企微后台「设置API接收」点击保存——会触发 `GET /wxkf/?...&echostr=...`,CoW 通过 `crypto.check_signature` 校验后返回明文 `echostr`,验证成功。
|
||||
|
||||
## 五、支持的回复类型
|
||||
|
||||
| ReplyType | 是否支持 | 备注 |
|
||||
|---|---|---|
|
||||
| `TEXT` / `INFO` / `ERROR` | ✅ | 自动按 2048 字节切片分段发送 |
|
||||
| `IMAGE`(本地) / `IMAGE_URL`(网络) | ✅ | 大图自动压缩到 10MB 以内 |
|
||||
| `VOICE` | ✅ | 转 amr 后发送,>60s 自动切片 |
|
||||
| `VIDEO_URL` | ✅ | 通过临时素材接口上传 |
|
||||
| `FILE` | ✅ | |
|
||||
|
||||
## 六、参考文档
|
||||
|
||||
- [LinkAI 微信客服接入文档](https://docs.link-ai.tech/platform/link-app/wechat-customer-service)
|
||||
- [企业微信开放接口 - 微信客服 - 接收消息](https://developer.work.weixin.qq.com/document/path/94670)
|
||||
- [企业微信开放接口 - 微信客服 - 发送消息](https://developer.work.weixin.qq.com/document/path/95122)
|
||||
603
channel/wechat_kf/wechat_kf_channel.py
Normal file
603
channel/wechat_kf/wechat_kf_channel.py
Normal file
@@ -0,0 +1,603 @@
|
||||
# -*- coding=utf-8 -*-
|
||||
"""
|
||||
WeChat Customer Service (微信客服) channel for CoW.
|
||||
|
||||
Differences from `channel/wechatcom/` (企微自建应用):
|
||||
1. Audience: external WeChat users (not internal members).
|
||||
2. Receiver fields: `external_userid` + `open_kfid` instead of a single
|
||||
member `userid`.
|
||||
3. Inbound flow: callback only delivers an event token, the actual
|
||||
message bodies must be pulled via `cgi-bin/kf/sync_msg` with a
|
||||
persistent cursor. See `wechat_kf_cursor_store.py`.
|
||||
4. Outbound flow: messages are sent via `cgi-bin/kf/send_msg` (each
|
||||
request must specify both `touser` and `open_kfid`); wechatpy has
|
||||
no native helper, so we call the HTTP endpoint directly.
|
||||
"""
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
import xml.etree.ElementTree as ET
|
||||
from collections import defaultdict
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from typing import Optional
|
||||
|
||||
import requests
|
||||
import web
|
||||
from wechatpy.enterprise import WeChatClient
|
||||
from wechatpy.enterprise.crypto import WeChatCrypto
|
||||
from wechatpy.enterprise.exceptions import InvalidCorpIdException
|
||||
from wechatpy.exceptions import InvalidSignatureException, WeChatClientException
|
||||
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.chat_channel import ChatChannel
|
||||
from channel.file_cache import get_file_cache
|
||||
from channel.wechat_kf.wechat_kf_cursor_store import CursorStore
|
||||
from channel.wechat_kf.wechat_kf_message import WechatKfMessage
|
||||
from common.log import logger
|
||||
from common.singleton import singleton
|
||||
from common.utils import (
|
||||
compress_imgfile,
|
||||
fsize,
|
||||
remove_markdown_symbol,
|
||||
split_string_by_utf8_length,
|
||||
)
|
||||
from config import conf
|
||||
|
||||
try:
|
||||
from voice.audio_convert import any_to_amr, split_audio
|
||||
except ImportError as e: # voice features optional
|
||||
logger.debug(
|
||||
"[wechat_kf] import voice.audio_convert failed, voice will be disabled: {}".format(e)
|
||||
)
|
||||
|
||||
MAX_UTF8_LEN = 2048
|
||||
KF_API_BASE = "https://qyapi.weixin.qq.com/cgi-bin/kf"
|
||||
SYNC_MSG_LIMIT = 1000
|
||||
|
||||
|
||||
@singleton
|
||||
class WechatKfChannel(ChatChannel):
|
||||
NOT_SUPPORT_REPLYTYPE = []
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.corp_id = conf().get("wechat_kf_corp_id")
|
||||
self.secret = conf().get("wechat_kf_secret")
|
||||
self.token = conf().get("wechat_kf_token")
|
||||
self.aes_key = conf().get("wechat_kf_aes_key")
|
||||
self._http_server = None
|
||||
logger.info(
|
||||
"[wechat_kf] Initializing WeCom customer-service channel, corp_id: {}".format(
|
||||
self.corp_id
|
||||
)
|
||||
)
|
||||
self.crypto = WeChatCrypto(self.token, self.aes_key, self.corp_id)
|
||||
# Use the stock wechatpy WeChatClient so that the access_token is
|
||||
# cached and only refreshed when actually expired (~2h). The local
|
||||
# `WechatComAppClient` subclass has a broken background refresh
|
||||
# loop that re-fetches every 60s and a `fetch_access_token()`
|
||||
# override that may return a dict instead of a string, which
|
||||
# corrupts URLs and triggers errcode 40014.
|
||||
self.client = WeChatClient(self.corp_id, self.secret)
|
||||
|
||||
# Persist sync_msg cursor under the user's home dir by default,
|
||||
# so it survives `tmp/` cleanups and cwd changes across restarts.
|
||||
cursor_path = os.path.expanduser(
|
||||
conf().get("wechat_kf_cursor_path") or "~/.wechat_kf_cursors.json"
|
||||
)
|
||||
self.cursor_store = CursorStore(cursor_path)
|
||||
|
||||
# WeCom requires the callback HTTP response to return within ~5s,
|
||||
# otherwise it retries the same notification. sync_msg pulling
|
||||
# can easily exceed that, so we dispatch it to a background pool
|
||||
# and let `Query.POST` reply success immediately.
|
||||
self._callback_executor = ThreadPoolExecutor(
|
||||
max_workers=4, thread_name_prefix="wxkf-cb"
|
||||
)
|
||||
# Per-open_kfid lock: serialize sync_msg for the same kf account
|
||||
# so that callback retries (or rapid-fire events) don't race on
|
||||
# the same cursor and produce duplicate replies.
|
||||
self._kf_locks: dict = defaultdict(threading.Lock)
|
||||
self._kf_locks_guard = threading.Lock()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
def startup(self):
|
||||
urls = ("/wxkf/?", "channel.wechat_kf.wechat_kf_channel.Query")
|
||||
app = web.application(urls, globals(), autoreload=False)
|
||||
port = conf().get("wechat_kf_port", 9888)
|
||||
logger.info("[wechat_kf] WeCom customer-service channel started")
|
||||
logger.info("[wechat_kf] Listening on http://0.0.0.0:{}/wxkf/".format(port))
|
||||
func = web.httpserver.StaticMiddleware(app.wsgifunc())
|
||||
func = web.httpserver.LogMiddleware(func)
|
||||
server = web.httpserver.WSGIServer(("0.0.0.0", port), func)
|
||||
self._http_server = server
|
||||
try:
|
||||
server.start()
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
server.stop()
|
||||
|
||||
def stop(self):
|
||||
if self._http_server:
|
||||
try:
|
||||
self._http_server.stop()
|
||||
logger.info("[wechat_kf] HTTP server stopped")
|
||||
except Exception as e:
|
||||
logger.warning(f"[wechat_kf] Error stopping HTTP server: {e}")
|
||||
self._http_server = None
|
||||
try:
|
||||
self._callback_executor.shutdown(wait=False)
|
||||
except Exception as e:
|
||||
logger.warning(f"[wechat_kf] Error shutting down callback executor: {e}")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound — implementing the abstract `send` contract
|
||||
# ------------------------------------------------------------------
|
||||
def send(self, reply: Reply, context: Context):
|
||||
receiver = context["receiver"]
|
||||
msg = context.kwargs.get("msg")
|
||||
external_userid = context.get("external_userid") or (msg.external_userid if msg else None)
|
||||
open_kfid = context.get("open_kfid") or (msg.open_kfid if msg else None)
|
||||
|
||||
if not external_userid or not open_kfid:
|
||||
logger.error(
|
||||
"[wechat_kf] missing external_userid or open_kfid, cannot send: "
|
||||
f"external_userid={external_userid}, open_kfid={open_kfid}"
|
||||
)
|
||||
return
|
||||
|
||||
if reply.type in [ReplyType.TEXT, ReplyType.ERROR, ReplyType.INFO]:
|
||||
reply_text = remove_markdown_symbol(reply.content)
|
||||
texts = split_string_by_utf8_length(reply_text, MAX_UTF8_LEN)
|
||||
if len(texts) > 1:
|
||||
logger.info(
|
||||
"[wechat_kf] text too long, split into {} parts".format(len(texts))
|
||||
)
|
||||
for i, text in enumerate(texts):
|
||||
self._send_text(external_userid, open_kfid, text)
|
||||
if i != len(texts) - 1:
|
||||
time.sleep(0.5)
|
||||
logger.info("[wechat_kf] Do send text to {}: {}".format(receiver, reply_text))
|
||||
|
||||
elif reply.type == ReplyType.VOICE:
|
||||
file_path = reply.content
|
||||
try:
|
||||
amr_file = os.path.splitext(file_path)[0] + ".amr"
|
||||
any_to_amr(file_path, amr_file)
|
||||
duration, files = split_audio(amr_file, 60 * 1000)
|
||||
if len(files) > 1:
|
||||
logger.info(
|
||||
"[wechat_kf] voice too long {}s > 60s, split into {} parts".format(
|
||||
duration / 1000.0, len(files)
|
||||
)
|
||||
)
|
||||
media_ids = []
|
||||
for path in files:
|
||||
with open(path, "rb") as f:
|
||||
response = self.client.media.upload("voice", f)
|
||||
logger.debug("[wechat_kf] upload voice response: {}".format(response))
|
||||
media_ids.append(response["media_id"])
|
||||
except ImportError as e:
|
||||
logger.error("[wechat_kf] voice conversion failed: {}".format(e))
|
||||
logger.error("[wechat_kf] please install pydub: pip install pydub")
|
||||
return
|
||||
except WeChatClientException as e:
|
||||
logger.error("[wechat_kf] upload voice failed: {}".format(e))
|
||||
return
|
||||
|
||||
try:
|
||||
os.remove(file_path)
|
||||
if amr_file != file_path:
|
||||
os.remove(amr_file)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
for media_id in media_ids:
|
||||
self._send_voice(external_userid, open_kfid, media_id)
|
||||
time.sleep(1)
|
||||
logger.info("[wechat_kf] sendVoice={}, receiver={}".format(reply.content, receiver))
|
||||
|
||||
elif reply.type == ReplyType.IMAGE_URL:
|
||||
img_url = reply.content
|
||||
pic_res = requests.get(img_url, stream=True)
|
||||
image_storage = io.BytesIO()
|
||||
for block in pic_res.iter_content(1024):
|
||||
image_storage.write(block)
|
||||
sz = fsize(image_storage)
|
||||
if sz >= 10 * 1024 * 1024:
|
||||
logger.info("[wechat_kf] image too large, compressing, sz={}".format(sz))
|
||||
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
|
||||
image_storage.seek(0)
|
||||
try:
|
||||
response = self.client.media.upload("image", image_storage)
|
||||
except WeChatClientException as e:
|
||||
logger.error("[wechat_kf] upload image failed: {}".format(e))
|
||||
return
|
||||
self._send_image(external_userid, open_kfid, response["media_id"])
|
||||
logger.info("[wechat_kf] sendImage url={}, receiver={}".format(img_url, receiver))
|
||||
|
||||
elif reply.type == ReplyType.IMAGE:
|
||||
image_storage = reply.content
|
||||
sz = fsize(image_storage)
|
||||
if sz >= 10 * 1024 * 1024:
|
||||
logger.info("[wechat_kf] image too large, compressing, sz={}".format(sz))
|
||||
image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
|
||||
image_storage.seek(0)
|
||||
try:
|
||||
response = self.client.media.upload("image", image_storage)
|
||||
except WeChatClientException as e:
|
||||
logger.error("[wechat_kf] upload image failed: {}".format(e))
|
||||
return
|
||||
self._send_image(external_userid, open_kfid, response["media_id"])
|
||||
logger.info("[wechat_kf] sendImage, receiver={}".format(receiver))
|
||||
|
||||
elif reply.type == ReplyType.VIDEO_URL:
|
||||
video_url = reply.content
|
||||
try:
|
||||
response = self.client.media.upload(
|
||||
"video", requests.get(video_url, stream=True).content
|
||||
)
|
||||
except WeChatClientException as e:
|
||||
logger.error("[wechat_kf] upload video failed: {}".format(e))
|
||||
return
|
||||
self._send_video(external_userid, open_kfid, response["media_id"])
|
||||
logger.info("[wechat_kf] sendVideo url={}, receiver={}".format(video_url, receiver))
|
||||
|
||||
elif reply.type == ReplyType.FILE:
|
||||
file_path = reply.content
|
||||
try:
|
||||
with open(file_path, "rb") as f:
|
||||
response = self.client.media.upload(
|
||||
"file", (os.path.basename(file_path), f.read())
|
||||
)
|
||||
except WeChatClientException as e:
|
||||
logger.error("[wechat_kf] upload file failed: {}".format(e))
|
||||
return
|
||||
self._send_file(external_userid, open_kfid, response["media_id"])
|
||||
logger.info("[wechat_kf] sendFile={}, receiver={}".format(file_path, receiver))
|
||||
|
||||
else:
|
||||
logger.warning("[wechat_kf] unsupported reply type: {}".format(reply.type))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound — pull messages by cursor
|
||||
# ------------------------------------------------------------------
|
||||
def _get_kf_lock(self, open_kfid: str) -> threading.Lock:
|
||||
with self._kf_locks_guard:
|
||||
return self._kf_locks[open_kfid]
|
||||
|
||||
def submit_callback(self, token: str, open_kfid: str):
|
||||
"""
|
||||
Async entry point used by the HTTP handler. Submits the actual
|
||||
sync_msg pulling to a background thread so the callback response
|
||||
can return within WeCom's 5s deadline.
|
||||
"""
|
||||
try:
|
||||
self._callback_executor.submit(self._run_callback, token, open_kfid)
|
||||
except RuntimeError as e:
|
||||
# Executor may be shut down during process exit; fall back
|
||||
# to inline execution so we don't silently drop the event.
|
||||
logger.warning(f"[wechat_kf] executor unavailable, run inline: {e}")
|
||||
self._run_callback(token, open_kfid)
|
||||
|
||||
def _run_callback(self, token: str, open_kfid: str):
|
||||
# Block on the per-kfid lock so retried callbacks queue up
|
||||
# behind the in-flight one. The queued worker will then call
|
||||
# sync_msg with the (already advanced) cursor, which is cheap
|
||||
# when there is nothing new and still picks up any messages
|
||||
# that arrived after the previous worker's last pull.
|
||||
lock = self._get_kf_lock(open_kfid)
|
||||
with lock:
|
||||
try:
|
||||
self.consume_callback(token, open_kfid)
|
||||
except Exception as e:
|
||||
logger.exception(f"[wechat_kf] consume_callback error: {e}")
|
||||
|
||||
def consume_callback(self, token: str, open_kfid: str):
|
||||
"""
|
||||
Called from the HTTP `Query.POST` handler whenever WeCom notifies
|
||||
us that there are new messages for `open_kfid`. Pulls all new
|
||||
messages via sync_msg and feeds them into `produce()`.
|
||||
"""
|
||||
existing_cursor = self.cursor_store.get(open_kfid)
|
||||
|
||||
# First-time bootstrap: always skip history, otherwise WeCom would
|
||||
# replay up to 14 days of messages on the very first callback and
|
||||
# flood every user with auto-replies.
|
||||
if not existing_cursor:
|
||||
self._initialize_cursor(token, open_kfid)
|
||||
return
|
||||
|
||||
msgs = self._pull_messages(token, open_kfid, existing_cursor)
|
||||
if not msgs:
|
||||
return
|
||||
file_cache = get_file_cache()
|
||||
for raw in msgs:
|
||||
try:
|
||||
kf_msg = WechatKfMessage(msg=raw, client=self.client)
|
||||
except NotImplementedError as e:
|
||||
logger.debug("[wechat_kf] {}".format(e))
|
||||
continue
|
||||
|
||||
session_id = kf_msg.from_user_id
|
||||
|
||||
# Cache lone images/files and wait for the user's follow-up
|
||||
# text. Agent mode never reads memory.USER_IMAGE_CACHE, so
|
||||
# without this the attachment is effectively lost.
|
||||
if kf_msg.ctype in (ContextType.IMAGE, ContextType.FILE):
|
||||
ftype = "image" if kf_msg.ctype == ContextType.IMAGE else "file"
|
||||
try:
|
||||
kf_msg.prepare() # download to local tmp path
|
||||
file_cache.add(session_id, kf_msg.content, file_type=ftype)
|
||||
logger.info(
|
||||
"[wechat_kf] {} cached for session {}: {}".format(
|
||||
ftype, session_id, kf_msg.content
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"[wechat_kf] cache {ftype} failed: {e}")
|
||||
continue
|
||||
|
||||
# On a text turn, attach any pending images/files as references
|
||||
# so the downstream agent can pick them up via the text content.
|
||||
# Paths are already under agent_workspace/tmp (see
|
||||
# WechatKfMessage._get_tmp_dir), so a relative ref also works.
|
||||
if kf_msg.ctype == ContextType.TEXT:
|
||||
cached_files = file_cache.get(session_id)
|
||||
if cached_files:
|
||||
refs = []
|
||||
for fi in cached_files:
|
||||
ftype, fpath = fi["type"], fi["path"]
|
||||
if ftype == "image":
|
||||
refs.append(f"[图片: {fpath}]")
|
||||
else:
|
||||
refs.append(f"[文件: {fpath}]")
|
||||
kf_msg.content = kf_msg.content + "\n" + "\n".join(refs)
|
||||
file_cache.clear(session_id)
|
||||
|
||||
context = self._compose_context(
|
||||
kf_msg.ctype,
|
||||
kf_msg.content,
|
||||
isgroup=False,
|
||||
msg=kf_msg,
|
||||
)
|
||||
if context:
|
||||
self.produce(context)
|
||||
time.sleep(0.05) # tiny gap between messages of the same batch
|
||||
|
||||
def _initialize_cursor(self, token: str, open_kfid: str):
|
||||
"""
|
||||
Drain all current messages for this `open_kfid` without producing
|
||||
any context, just to advance the cursor to "now". This prevents
|
||||
a fresh deployment from replying to up to ~14 days of history.
|
||||
"""
|
||||
next_cursor = ""
|
||||
total_skipped = 0
|
||||
while True:
|
||||
data = self._call_sync_msg(token, open_kfid, next_cursor)
|
||||
if data is None:
|
||||
break
|
||||
msg_list = data.get("msg_list") or []
|
||||
total_skipped += len(msg_list)
|
||||
cursor_after = data.get("next_cursor") or ""
|
||||
if cursor_after:
|
||||
self.cursor_store.set(open_kfid, cursor_after)
|
||||
if not data.get("has_more"):
|
||||
break
|
||||
if not cursor_after or cursor_after == next_cursor:
|
||||
break
|
||||
next_cursor = cursor_after
|
||||
logger.info(
|
||||
"[wechat_kf] first-start bootstrap finished for open_kfid={}, "
|
||||
"skipped {} historical messages".format(open_kfid, total_skipped)
|
||||
)
|
||||
|
||||
def _pull_messages(self, token: str, open_kfid: str, next_cursor: Optional[str]) -> list:
|
||||
"""Loop sync_msg until `has_more` is false. Returns raw msg dicts."""
|
||||
collected = []
|
||||
cursor = next_cursor or ""
|
||||
while True:
|
||||
data = self._call_sync_msg(token, open_kfid, cursor)
|
||||
if data is None:
|
||||
break
|
||||
for item in data.get("msg_list") or []:
|
||||
# Only consume messages from external users; ignore replies
|
||||
# generated by our own kf account, otherwise we would loop
|
||||
# back into ourselves.
|
||||
if not item.get("external_userid"):
|
||||
continue
|
||||
if item.get("msgtype") in ("text", "image", "voice", "file"):
|
||||
collected.append(item)
|
||||
cursor_after = data.get("next_cursor") or ""
|
||||
if cursor_after:
|
||||
self.cursor_store.set(open_kfid, cursor_after)
|
||||
if not data.get("has_more"):
|
||||
break
|
||||
if not cursor_after or cursor_after == cursor:
|
||||
break
|
||||
cursor = cursor_after
|
||||
|
||||
if collected:
|
||||
collected = _dedup_image_text_pair(collected)
|
||||
logger.info(
|
||||
"[wechat_kf] pulled {} messages for open_kfid={}".format(len(collected), open_kfid)
|
||||
)
|
||||
return collected
|
||||
|
||||
def _call_sync_msg(self, token: str, open_kfid: str, cursor: str) -> Optional[dict]:
|
||||
# `client.access_token` is the cached string property; do not use
|
||||
# `fetch_access_token()` here — wechatpy returns the raw response
|
||||
# dict from that call, which corrupts the query string.
|
||||
url = f"{KF_API_BASE}/sync_msg?access_token={self.client.access_token}"
|
||||
payload = {
|
||||
"token": token,
|
||||
"open_kfid": open_kfid,
|
||||
"limit": SYNC_MSG_LIMIT,
|
||||
}
|
||||
if cursor:
|
||||
payload["cursor"] = cursor
|
||||
try:
|
||||
resp = requests.post(url, json=payload, timeout=10).json()
|
||||
except Exception as e:
|
||||
logger.error(f"[wechat_kf] sync_msg request failed: {e}")
|
||||
return None
|
||||
|
||||
if resp.get("errcode") != 0:
|
||||
logger.error(
|
||||
f"[wechat_kf] sync_msg errcode={resp.get('errcode')}, "
|
||||
f"errmsg={resp.get('errmsg')}, open_kfid={open_kfid}"
|
||||
)
|
||||
return None
|
||||
return resp
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound HTTP wrappers (kf/send_msg)
|
||||
# ------------------------------------------------------------------
|
||||
def _post_send_msg(self, payload: dict) -> dict:
|
||||
url = f"{KF_API_BASE}/send_msg?access_token={self.client.access_token}"
|
||||
try:
|
||||
resp = requests.post(url, json=payload, timeout=10).json()
|
||||
except Exception as e:
|
||||
logger.error(f"[wechat_kf] send_msg request failed: {e}")
|
||||
return {"errcode": -1, "errmsg": str(e)}
|
||||
if resp.get("errcode") != 0:
|
||||
logger.error(f"[wechat_kf] send_msg failed, payload={payload}, resp={resp}")
|
||||
return resp
|
||||
|
||||
def _send_text(self, external_userid: str, open_kfid: str, content: str) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "text",
|
||||
"text": {"content": content},
|
||||
})
|
||||
|
||||
def _send_image(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "image",
|
||||
"image": {"media_id": media_id},
|
||||
})
|
||||
|
||||
def _send_voice(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "voice",
|
||||
"voice": {"media_id": media_id},
|
||||
})
|
||||
|
||||
def _send_video(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "video",
|
||||
"video": {"media_id": media_id},
|
||||
})
|
||||
|
||||
def _send_file(self, external_userid: str, open_kfid: str, media_id: str) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "file",
|
||||
"file": {"media_id": media_id},
|
||||
})
|
||||
|
||||
def _send_link(self, external_userid: str, open_kfid: str, link_data: dict) -> dict:
|
||||
return self._post_send_msg({
|
||||
"touser": external_userid,
|
||||
"open_kfid": open_kfid,
|
||||
"msgtype": "link",
|
||||
"link": link_data,
|
||||
})
|
||||
|
||||
|
||||
def _dedup_image_text_pair(messages: list) -> list:
|
||||
"""
|
||||
A WeChat user often sends an image immediately followed by a text
|
||||
question (e.g. "what's in this picture?"). Only when the batch is
|
||||
exactly that 2-message image+text pair within a 5s window do we
|
||||
collapse it into a single [image, text] turn. Otherwise return
|
||||
every message so rapid-fire texts/images are all processed —
|
||||
cursor freshness is already guaranteed by sync_msg.
|
||||
"""
|
||||
if not messages:
|
||||
return []
|
||||
|
||||
if len(messages) == 2:
|
||||
a, b = messages
|
||||
types = {a["msgtype"], b["msgtype"]}
|
||||
if types == {"image", "text"} and abs(a["send_time"] - b["send_time"]) <= 5:
|
||||
img = a if a["msgtype"] == "image" else b
|
||||
txt = b if a["msgtype"] == "image" else a
|
||||
return [img, txt]
|
||||
|
||||
return messages
|
||||
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# HTTP handlers (web.py)
|
||||
# ----------------------------------------------------------------------
|
||||
class Query:
|
||||
def GET(self):
|
||||
channel = WechatKfChannel()
|
||||
params = web.input()
|
||||
logger.info("[wechat_kf] verify params: {}".format(params))
|
||||
try:
|
||||
signature = params.msg_signature
|
||||
timestamp = params.timestamp
|
||||
nonce = params.nonce
|
||||
echostr = params.echostr
|
||||
echostr = channel.crypto.check_signature(signature, timestamp, nonce, echostr)
|
||||
except (InvalidSignatureException, InvalidCorpIdException):
|
||||
raise web.Forbidden()
|
||||
return echostr
|
||||
|
||||
def POST(self):
|
||||
channel = WechatKfChannel()
|
||||
params = web.input()
|
||||
try:
|
||||
signature = params.msg_signature
|
||||
timestamp = params.timestamp
|
||||
nonce = params.nonce
|
||||
raw_body = web.data()
|
||||
decrypted = channel.crypto.decrypt_message(raw_body, signature, timestamp, nonce)
|
||||
except (InvalidSignatureException, InvalidCorpIdException) as e:
|
||||
logger.warning(f"[wechat_kf] invalid signature: {e}")
|
||||
raise web.Forbidden()
|
||||
|
||||
# We need the Token + OpenKfId fields from the inner XML to call
|
||||
# sync_msg. wechatpy's parsed object exposes neither, so we parse
|
||||
# the raw XML directly.
|
||||
try:
|
||||
root = ET.fromstring(decrypted)
|
||||
except ET.ParseError as e:
|
||||
logger.error(f"[wechat_kf] xml parse error: {e}")
|
||||
return "success"
|
||||
|
||||
msg_type = (root.findtext("MsgType") or "").strip()
|
||||
event = (root.findtext("Event") or "").strip()
|
||||
if msg_type != "event" or event != "kf_msg_or_event":
|
||||
logger.debug(
|
||||
f"[wechat_kf] ignored callback msg_type={msg_type}, event={event}"
|
||||
)
|
||||
return "success"
|
||||
|
||||
token = root.findtext("Token") or ""
|
||||
open_kfid = root.findtext("OpenKfId") or ""
|
||||
if not token or not open_kfid:
|
||||
logger.warning(
|
||||
f"[wechat_kf] callback missing token or open_kfid: {decrypted}"
|
||||
)
|
||||
return "success"
|
||||
|
||||
# Hand off to a background worker — WeCom requires the callback
|
||||
# to return success within ~5 seconds, otherwise it will retry
|
||||
# and we may race the same cursor window into duplicate replies.
|
||||
channel.submit_callback(token, open_kfid)
|
||||
return "success"
|
||||
80
channel/wechat_kf/wechat_kf_cursor_store.py
Normal file
80
channel/wechat_kf/wechat_kf_cursor_store.py
Normal file
@@ -0,0 +1,80 @@
|
||||
# -*- coding=utf-8 -*-
|
||||
"""
|
||||
Local-file based persistence for WeCom customer-service `next_cursor`.
|
||||
|
||||
Why we need this:
|
||||
The WeCom customer-service (微信客服) callback only notifies us that
|
||||
"new messages exist". To actually fetch them we must call the
|
||||
`cgi-bin/kf/sync_msg` endpoint with a `cursor` so that we only get
|
||||
messages newer than the previously processed one. If we lose this
|
||||
cursor (e.g. on process restart) WeCom will replay up to ~14 days of
|
||||
history, which would cause the bot to flood users with duplicate
|
||||
replies.
|
||||
|
||||
This implementation deliberately avoids any external dependency
|
||||
(no Redis / no DB) — a single JSON file under the project's tmp dir is
|
||||
enough for a CoW-style single-process deployment.
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
from typing import Optional
|
||||
|
||||
from common.log import logger
|
||||
|
||||
|
||||
class CursorStore:
|
||||
"""Thread-safe per-`open_kfid` cursor store backed by a JSON file."""
|
||||
|
||||
def __init__(self, file_path: str):
|
||||
self._file_path = file_path
|
||||
self._lock = threading.Lock()
|
||||
self._data = self._load()
|
||||
|
||||
def _load(self) -> dict:
|
||||
try:
|
||||
if os.path.exists(self._file_path):
|
||||
with open(self._file_path, "r", encoding="utf-8") as f:
|
||||
return json.load(f) or {}
|
||||
except Exception as e:
|
||||
logger.warning(f"[wechat_kf] failed to load cursor file {self._file_path}: {e}")
|
||||
return {}
|
||||
|
||||
def _flush_locked(self):
|
||||
# Atomic write: write to *.tmp first then rename, avoid corruption on crash.
|
||||
tmp_path = self._file_path + ".tmp"
|
||||
try:
|
||||
os.makedirs(os.path.dirname(self._file_path) or ".", exist_ok=True)
|
||||
with open(tmp_path, "w", encoding="utf-8") as f:
|
||||
json.dump(self._data, f, ensure_ascii=False)
|
||||
os.replace(tmp_path, self._file_path)
|
||||
# Tighten permissions: cursor file lives in $HOME, restrict to owner.
|
||||
# No-op on Windows.
|
||||
try:
|
||||
os.chmod(self._file_path, 0o600)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.warning(f"[wechat_kf] failed to flush cursor file {self._file_path}: {e}")
|
||||
try:
|
||||
if os.path.exists(tmp_path):
|
||||
os.remove(tmp_path)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def get(self, open_kfid: str) -> Optional[str]:
|
||||
with self._lock:
|
||||
return self._data.get(open_kfid)
|
||||
|
||||
def set(self, open_kfid: str, cursor: str):
|
||||
if not cursor:
|
||||
return
|
||||
with self._lock:
|
||||
if self._data.get(open_kfid) == cursor:
|
||||
return
|
||||
self._data[open_kfid] = cursor
|
||||
self._flush_locked()
|
||||
|
||||
def has(self, open_kfid: str) -> bool:
|
||||
with self._lock:
|
||||
return open_kfid in self._data
|
||||
134
channel/wechat_kf/wechat_kf_message.py
Normal file
134
channel/wechat_kf/wechat_kf_message.py
Normal file
@@ -0,0 +1,134 @@
|
||||
# -*- coding=utf-8 -*-
|
||||
"""
|
||||
Adapter that turns a single `sync_msg` item from WeCom customer-service
|
||||
into a CoW `ChatMessage` object.
|
||||
"""
|
||||
import os
|
||||
import re
|
||||
|
||||
from wechatpy.enterprise import WeChatClient
|
||||
|
||||
from bridge.context import ContextType
|
||||
from channel.chat_message import ChatMessage
|
||||
from common.log import logger
|
||||
from common.utils import expand_path
|
||||
from config import conf
|
||||
|
||||
|
||||
def _get_tmp_dir() -> str:
|
||||
"""Save under agent_workspace/tmp/ so agent tools (e.g. `read`) can
|
||||
resolve a relative path like `tmp/xxx.pdf` against their own
|
||||
workspace root. Mirrors the convention used by weixin / wecom_bot.
|
||||
"""
|
||||
ws_root = expand_path(conf().get("agent_workspace", "~/cow"))
|
||||
tmp_dir = os.path.join(ws_root, "tmp")
|
||||
os.makedirs(tmp_dir, exist_ok=True)
|
||||
return tmp_dir
|
||||
|
||||
|
||||
def _extract_filename(content_disposition: str) -> str:
|
||||
"""Best-effort parse of `filename` / `filename*` from a Content-Disposition
|
||||
header. Returns '' when nothing usable is found."""
|
||||
if not content_disposition:
|
||||
return ""
|
||||
# RFC 5987 form: filename*=UTF-8''xxx
|
||||
m = re.search(r"filename\*=(?:[^'\"]*'[^']*'\s*)?([^;]+)", content_disposition)
|
||||
if m:
|
||||
try:
|
||||
from urllib.parse import unquote
|
||||
return unquote(m.group(1).strip().strip('"'))
|
||||
except Exception:
|
||||
return m.group(1).strip().strip('"')
|
||||
m = re.search(r'filename\s*=\s*"?([^";]+)"?', content_disposition)
|
||||
return m.group(1).strip() if m else ""
|
||||
|
||||
|
||||
class WechatKfMessage(ChatMessage):
|
||||
"""
|
||||
msg structure (from cgi-bin/kf/sync_msg):
|
||||
{
|
||||
"msgid": "...",
|
||||
"send_time": 1700000000,
|
||||
"origin": 3,
|
||||
"msgtype": "text" | "image" | "voice" | ...,
|
||||
"open_kfid": "wkxxxx",
|
||||
"external_userid": "wmxxxx",
|
||||
"text": {"content": "..."},
|
||||
"image": {"media_id": "..."},
|
||||
"voice": {"media_id": "..."},
|
||||
...
|
||||
}
|
||||
"""
|
||||
|
||||
def __init__(self, msg: dict, client: WeChatClient = None, is_group: bool = False):
|
||||
# NOTE: skip parent constructor because it expects a wechatpy parsed
|
||||
# message object, while here we receive a raw dict from sync_msg.
|
||||
super().__init__(msg)
|
||||
self.is_group = is_group
|
||||
self.msg_id = msg.get("msgid")
|
||||
self.create_time = msg.get("send_time")
|
||||
self.origin = msg.get("origin")
|
||||
self.msgtype = msg.get("msgtype")
|
||||
self.open_kfid = msg.get("open_kfid")
|
||||
self.external_userid = msg.get("external_userid")
|
||||
|
||||
if self.msgtype == "text":
|
||||
self.ctype = ContextType.TEXT
|
||||
self.content = msg.get("text", {}).get("content", "")
|
||||
elif self.msgtype == "image":
|
||||
self.ctype = ContextType.IMAGE
|
||||
media_id = msg.get("image", {}).get("media_id", "")
|
||||
self.content = os.path.join(_get_tmp_dir(), media_id + ".jpg")
|
||||
|
||||
def download_image():
|
||||
response = client.media.download(media_id)
|
||||
if response.status_code == 200:
|
||||
with open(self.content, "wb") as f:
|
||||
f.write(response.content)
|
||||
else:
|
||||
logger.info(f"[wechat_kf] Failed to download image, {response.content}")
|
||||
|
||||
self._prepare_fn = download_image
|
||||
elif self.msgtype == "voice":
|
||||
self.ctype = ContextType.VOICE
|
||||
media_id = msg.get("voice", {}).get("media_id", "")
|
||||
# WeCom returns amr by default; downstream voice pipeline will convert.
|
||||
self.content = os.path.join(_get_tmp_dir(), media_id + ".amr")
|
||||
|
||||
def download_voice():
|
||||
response = client.media.download(media_id)
|
||||
if response.status_code == 200:
|
||||
with open(self.content, "wb") as f:
|
||||
f.write(response.content)
|
||||
else:
|
||||
logger.info(f"[wechat_kf] Failed to download voice, {response.content}")
|
||||
|
||||
self._prepare_fn = download_voice
|
||||
elif self.msgtype == "file":
|
||||
self.ctype = ContextType.FILE
|
||||
media_id = msg.get("file", {}).get("media_id", "")
|
||||
# Provisional path; rewritten in download_file() once we have
|
||||
# the original filename from Content-Disposition.
|
||||
self.content = os.path.join(_get_tmp_dir(), media_id)
|
||||
|
||||
def download_file():
|
||||
response = client.media.download(media_id)
|
||||
if response.status_code == 200:
|
||||
filename = _extract_filename(
|
||||
response.headers.get("Content-Disposition", "")
|
||||
) or media_id
|
||||
self.content = os.path.join(_get_tmp_dir(), filename)
|
||||
with open(self.content, "wb") as f:
|
||||
f.write(response.content)
|
||||
else:
|
||||
logger.info(f"[wechat_kf] Failed to download file, {response.content}")
|
||||
|
||||
self._prepare_fn = download_file
|
||||
else:
|
||||
raise NotImplementedError(
|
||||
f"[wechat_kf] Unsupported message type: {self.msgtype}"
|
||||
)
|
||||
|
||||
self.from_user_id = self.external_userid
|
||||
self.to_user_id = self.open_kfid
|
||||
self.other_user_id = self.external_userid
|
||||
@@ -19,9 +19,15 @@ def verify_server(data):
|
||||
nonce = data.nonce
|
||||
echostr = data.get("echostr", None)
|
||||
token = conf().get("wechatmp_token") # 请按照公众平台官网\基本配置中信息填写
|
||||
# Reject when token is empty: an empty token reduces signature verification
|
||||
# to a predictable hash over attacker-controlled values.
|
||||
if not token:
|
||||
raise web.Forbidden("wechatmp_token is not configured")
|
||||
check_signature(token, signature, timestamp, nonce)
|
||||
return echostr
|
||||
except InvalidSignatureException:
|
||||
raise web.Forbidden("Invalid signature")
|
||||
except web.Forbidden:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise web.Forbidden(str(e))
|
||||
|
||||
@@ -103,14 +103,21 @@ class Query:
|
||||
task_running = True
|
||||
waiting_until = request_time + 4
|
||||
while time.time() < waiting_until:
|
||||
if from_user in channel.running:
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
if from_user not in channel.running:
|
||||
task_running = False
|
||||
break
|
||||
# Task still running, but if it has already produced cached
|
||||
# segments (e.g. multi-turn thinking output), return them now
|
||||
# instead of forcing the user to wait for the whole task. The
|
||||
# remaining segments are fetched by the user's next message.
|
||||
if channel.cache_dict.get(from_user):
|
||||
break
|
||||
time.sleep(0.1)
|
||||
|
||||
reply_text = ""
|
||||
if task_running:
|
||||
# Only fall back to retry / "thinking" hint when the task is still
|
||||
# running AND there is nothing cached to send yet.
|
||||
if task_running and not channel.cache_dict.get(from_user):
|
||||
if request_cnt < 3:
|
||||
# waiting for timeout (the POST request will be closed by Wechat official server)
|
||||
time.sleep(2)
|
||||
@@ -131,8 +138,22 @@ class Query:
|
||||
|
||||
# Only one request can access to the cached data
|
||||
try:
|
||||
(reply_type, reply_content) = channel.cache_dict[from_user].pop(0)
|
||||
if not channel.cache_dict[from_user]: # If popping the message makes the list empty, delete the user entry from cache
|
||||
# WeChat passive reply allows only a single reply per request.
|
||||
# To avoid forcing the user to send an extra message for every
|
||||
# segment of multi-turn agent output, drain all consecutive
|
||||
# cached text segments at once and merge them into one reply.
|
||||
# Media (voice/image) can only be returned one at a time, so it
|
||||
# stops the merge and is returned on its own.
|
||||
cached = channel.cache_dict[from_user]
|
||||
if cached[0][0] == "text":
|
||||
reply_type = "text"
|
||||
merged_parts = []
|
||||
while cached and cached[0][0] == "text":
|
||||
merged_parts.append(cached.pop(0)[1])
|
||||
reply_content = "\n\n".join(merged_parts)
|
||||
else:
|
||||
(reply_type, reply_content) = cached.pop(0)
|
||||
if not channel.cache_dict[from_user]: # If draining empties the list, delete the user entry from cache
|
||||
del channel.cache_dict[from_user]
|
||||
except IndexError:
|
||||
return "success"
|
||||
|
||||
@@ -134,10 +134,16 @@ class WechatMPChannel(ChatChannel):
|
||||
|
||||
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
|
||||
img_url = reply.content
|
||||
pic_res = requests.get(img_url, stream=True)
|
||||
image_storage = io.BytesIO()
|
||||
for block in pic_res.iter_content(1024):
|
||||
image_storage.write(block)
|
||||
if img_url.startswith("file://") or os.path.isfile(img_url):
|
||||
# Local file produced by the agent (e.g. a generated image)
|
||||
local_path = img_url[len("file://"):] if img_url.startswith("file://") else img_url
|
||||
with open(local_path, "rb") as f:
|
||||
image_storage.write(f.read())
|
||||
else:
|
||||
pic_res = requests.get(img_url, stream=True)
|
||||
for block in pic_res.iter_content(1024):
|
||||
image_storage.write(block)
|
||||
image_storage.seek(0)
|
||||
image_type = imghdr.what(image_storage)
|
||||
filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
|
||||
@@ -258,10 +264,16 @@ class WechatMPChannel(ChatChannel):
|
||||
logger.info("[wechatmp] Do send voice to {}".format(receiver))
|
||||
elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
|
||||
img_url = reply.content
|
||||
pic_res = requests.get(img_url, stream=True)
|
||||
image_storage = io.BytesIO()
|
||||
for block in pic_res.iter_content(1024):
|
||||
image_storage.write(block)
|
||||
if img_url.startswith("file://") or os.path.isfile(img_url):
|
||||
# Local file produced by the agent (e.g. a generated image)
|
||||
local_path = img_url[len("file://"):] if img_url.startswith("file://") else img_url
|
||||
with open(local_path, "rb") as f:
|
||||
image_storage.write(f.read())
|
||||
else:
|
||||
pic_res = requests.get(img_url, stream=True)
|
||||
for block in pic_res.iter_content(1024):
|
||||
image_storage.write(block)
|
||||
image_storage.seek(0)
|
||||
image_type = imghdr.what(image_storage)
|
||||
filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
|
||||
|
||||
@@ -12,16 +12,19 @@ import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
|
||||
import requests
|
||||
import web
|
||||
import websocket
|
||||
|
||||
from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from channel.chat_channel import ChatChannel, check_prefix
|
||||
from channel.wecom_bot.wecom_bot_crypt import WecomBotCrypt
|
||||
from channel.wecom_bot.wecom_bot_message import WecomBotMessage
|
||||
from common.expired_dict import ExpiredDict
|
||||
from common.log import logger
|
||||
@@ -32,6 +35,9 @@ from config import conf
|
||||
WECOM_WS_URL = "wss://openws.work.weixin.qq.com"
|
||||
HEARTBEAT_INTERVAL = 30
|
||||
MEDIA_CHUNK_SIZE = 512 * 1024 # 512KB per chunk (before base64 encoding)
|
||||
# Fixed URL path for the callback (webhook) HTTP server. The bot's
|
||||
# receive-message URL must point at this path, e.g. http://host:9892/wecombot
|
||||
CALLBACK_PATH = "/wecombot"
|
||||
|
||||
|
||||
def _escape_control_chars_inside_json_strings(s: str) -> str:
|
||||
@@ -97,6 +103,14 @@ class WecomBotChannel(ChatChannel):
|
||||
self._pending_lock = threading.Lock()
|
||||
self._stream_states = {} # req_id -> {"stream_id": str, "content": str}
|
||||
|
||||
# Transport mode: "websocket" (long connection) or "webhook" (HTTP callback)
|
||||
self.mode = "websocket"
|
||||
self._crypt = None
|
||||
self._http_server = None
|
||||
# stream_id -> {"committed", "current", "finished", "images", "last_access"}
|
||||
self._callback_streams = ExpiredDict(60 * 10) # auto-expire after 10min (max poll window is 6min)
|
||||
self._callback_lock = threading.Lock()
|
||||
|
||||
conf()["group_name_white_list"] = ["ALL_GROUP"]
|
||||
conf()["single_chat_prefix"] = [""]
|
||||
|
||||
@@ -105,6 +119,11 @@ class WecomBotChannel(ChatChannel):
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def startup(self):
|
||||
self.mode = conf().get("wecom_bot_mode", "websocket")
|
||||
if self.mode == "webhook":
|
||||
self._startup_callback()
|
||||
return
|
||||
|
||||
self.bot_id = conf().get("wecom_bot_id", "")
|
||||
self.bot_secret = conf().get("wecom_bot_secret", "")
|
||||
|
||||
@@ -127,6 +146,13 @@ class WecomBotChannel(ChatChannel):
|
||||
pass
|
||||
self._ws = None
|
||||
self._connected = False
|
||||
if self._http_server:
|
||||
try:
|
||||
self._http_server.stop()
|
||||
logger.info("[WecomBot] Callback HTTP server stopped")
|
||||
except Exception as e:
|
||||
logger.warning(f"[WecomBot] Error stopping HTTP server: {e}")
|
||||
self._http_server = None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# WebSocket connection
|
||||
@@ -183,6 +209,192 @@ class WecomBotChannel(ChatChannel):
|
||||
def _gen_req_id(self) -> str:
|
||||
return uuid.uuid4().hex[:16]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Callback (webhook) mode
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _startup_callback(self):
|
||||
"""Start an HTTP server that receives encrypted callbacks (webhook mode).
|
||||
|
||||
The bot's "接收消息" URL in the WeCom admin console should point at this
|
||||
server (any path is accepted). Verification (GET) and message delivery
|
||||
(POST) are both handled by ``WecomBotCallbackController``.
|
||||
"""
|
||||
token = conf().get("wecom_bot_token", "")
|
||||
aes_key = conf().get("wecom_bot_encoding_aes_key", "")
|
||||
if not token or not aes_key:
|
||||
err = "[WecomBot] callback mode requires wecom_bot_token and wecom_bot_encoding_aes_key"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
try:
|
||||
# Enterprise-internal smart bot: receive_id is an empty string.
|
||||
self._crypt = WecomBotCrypt(token, aes_key, "")
|
||||
except Exception as e:
|
||||
err = f"[WecomBot] invalid callback credentials: {e}"
|
||||
logger.error(err)
|
||||
self.report_startup_error(err)
|
||||
return
|
||||
|
||||
port = int(conf().get("wecom_bot_port", 9892))
|
||||
logger.info(f"[WecomBot] Starting callback (webhook) server on port {port}, path {CALLBACK_PATH} ...")
|
||||
# Only serve the fixed callback path; everything else 404s instead of being
|
||||
# treated as a (signature-failing) WeCom callback.
|
||||
urls = (re.escape(CALLBACK_PATH), "channel.wecom_bot.wecom_bot_channel.WecomBotCallbackController")
|
||||
app = web.application(urls, globals(), autoreload=False)
|
||||
func = web.httpserver.StaticMiddleware(app.wsgifunc())
|
||||
func = web.httpserver.LogMiddleware(func)
|
||||
server = web.httpserver.WSGIServer(("0.0.0.0", port), func)
|
||||
self._http_server = server
|
||||
self.report_startup_success()
|
||||
try:
|
||||
server.start()
|
||||
except (KeyboardInterrupt, SystemExit):
|
||||
server.stop()
|
||||
|
||||
def _new_callback_stream(self, response_url: str = "") -> str:
|
||||
"""Create a new stream state and return its id."""
|
||||
stream_id = uuid.uuid4().hex[:16]
|
||||
now = time.time()
|
||||
with self._callback_lock:
|
||||
self._callback_streams[stream_id] = {
|
||||
"committed": "",
|
||||
"current": "",
|
||||
"finished": False,
|
||||
"images": [], # list of (base64_str, md5_str), flushed only at finish
|
||||
"image_urls": [], # public http(s) image urls (usable in response_url markdown)
|
||||
"image_pending": False, # an image reply is being prepared; don't finish on text yet
|
||||
"last_access": now,
|
||||
"created_at": now,
|
||||
"response_url": response_url or "",
|
||||
"delivered": False, # final answer handed to WeCom via a poll
|
||||
"url_sent": False, # final answer pushed via response_url (active reply)
|
||||
}
|
||||
return stream_id
|
||||
|
||||
def _callback_handle_message(self, data: dict) -> dict:
|
||||
"""Handle a freshly-received user message in callback mode.
|
||||
|
||||
Produces the context for async processing and returns the initial passive
|
||||
reply (a stream packet with finish=false) so WeCom starts polling for the
|
||||
agent's streamed answer. Returns ``None`` when there's nothing to reply
|
||||
(e.g. an image/file silently cached for the next query).
|
||||
"""
|
||||
msg_id = data.get("msgid", "")
|
||||
if msg_id and self.received_msgs.get(msg_id):
|
||||
logger.debug(f"[WecomBot] Duplicate msg filtered: {msg_id}")
|
||||
return None
|
||||
if msg_id:
|
||||
self.received_msgs[msg_id] = True
|
||||
|
||||
chattype = data.get("chattype", "single")
|
||||
is_group = chattype == "group"
|
||||
|
||||
default_aeskey = conf().get("wecom_bot_encoding_aes_key", "")
|
||||
result = self._build_context(data, is_group, default_aeskey=default_aeskey)
|
||||
if not result:
|
||||
return None
|
||||
context, wecom_msg = result
|
||||
|
||||
# response_url lets us actively reply once within 1h, used as a fallback
|
||||
# when the agent finishes after WeCom stops polling (max ~6min window).
|
||||
response_url = data.get("response_url", "") or ""
|
||||
stream_id = self._new_callback_stream(response_url=response_url)
|
||||
wecom_msg.stream_id = stream_id
|
||||
context["wecom_stream_id"] = stream_id
|
||||
context["on_event"] = self._make_callback_stream_callback(stream_id)
|
||||
self.produce(context)
|
||||
|
||||
# First passive reply: register the stream id, WeCom will poll for updates.
|
||||
return {
|
||||
"msgtype": "stream",
|
||||
"stream": {"id": stream_id, "finish": False, "content": ""},
|
||||
}
|
||||
|
||||
def _callback_handle_stream_poll(self, data: dict) -> dict:
|
||||
"""Handle a "流式消息刷新" poll: return the latest accumulated content."""
|
||||
stream_id = data.get("stream", {}).get("id", "")
|
||||
with self._callback_lock:
|
||||
state = self._callback_streams.get(stream_id)
|
||||
if state is None:
|
||||
# Unknown / expired stream: tell WeCom we're done to stop polling.
|
||||
return {"msgtype": "stream", "stream": {"id": stream_id, "finish": True, "content": ""}}
|
||||
state["last_access"] = time.time()
|
||||
if state.get("url_sent"):
|
||||
# Final answer already pushed via response_url; finish silently.
|
||||
return {"msgtype": "stream", "stream": {"id": stream_id, "finish": True, "content": ""}}
|
||||
# We never force-finish on a timer: while a task is still running the
|
||||
# bubble should keep spinning until either the task finishes or the
|
||||
# user cancels. If WeCom's 6min window closes before completion, the
|
||||
# answer is delivered later via response_url instead.
|
||||
finished = state["finished"]
|
||||
content = state["committed"] + state["current"]
|
||||
images = state["images"] if finished else []
|
||||
if finished:
|
||||
state["delivered"] = True
|
||||
logger.debug(f"[WecomBot] stream {stream_id} delivered via poll, len={len(content)}, images={len(images)}")
|
||||
|
||||
stream = {"id": stream_id, "finish": finished, "content": content}
|
||||
if images:
|
||||
stream["msg_item"] = [
|
||||
{"msgtype": "image", "image": {"base64": b64, "md5": md5}}
|
||||
for (b64, md5) in images
|
||||
]
|
||||
return {"msgtype": "stream", "stream": stream}
|
||||
|
||||
def _make_callback_stream_callback(self, stream_id: str):
|
||||
"""Build an on_event callback that accumulates agent output into stream state.
|
||||
|
||||
Mirrors the websocket streaming behaviour: intermediate turns (text before
|
||||
a tool call) are committed with a '---' separator; WeCom reads the full
|
||||
accumulated content on each poll.
|
||||
"""
|
||||
def on_event(event: dict):
|
||||
event_type = event.get("type")
|
||||
edata = event.get("data", {})
|
||||
cancelled = False
|
||||
with self._callback_lock:
|
||||
state = self._callback_streams.get(stream_id)
|
||||
if not state:
|
||||
return
|
||||
|
||||
if event_type == "turn_start":
|
||||
state["current"] = ""
|
||||
elif event_type == "message_update":
|
||||
delta = edata.get("delta", "")
|
||||
if delta:
|
||||
state["current"] += delta
|
||||
elif event_type == "message_end":
|
||||
tool_calls = edata.get("tool_calls", [])
|
||||
if tool_calls:
|
||||
if state["current"].strip():
|
||||
state["committed"] += state["current"].strip() + "\n\n---\n\n"
|
||||
state["current"] = ""
|
||||
else:
|
||||
state["committed"] += state["current"]
|
||||
state["current"] = ""
|
||||
elif event_type == "agent_cancelled":
|
||||
# Mechanism 1: a cancelled run never reaches send(), so finalize
|
||||
# its stream here to stop the "···" bubble immediately.
|
||||
if state["current"]:
|
||||
state["committed"] += state["current"]
|
||||
state["current"] = ""
|
||||
state["committed"] = state["committed"].rstrip()
|
||||
if state["committed"].endswith("---"):
|
||||
state["committed"] = state["committed"][:-3].rstrip()
|
||||
if not state["committed"].strip():
|
||||
state["committed"] = "🛑 已中止"
|
||||
state["finished"] = True
|
||||
state["last_access"] = time.time()
|
||||
cancelled = True
|
||||
|
||||
if cancelled:
|
||||
# Outside the lock: response_url fallback re-acquires it.
|
||||
self._schedule_response_url_fallback(stream_id)
|
||||
|
||||
return on_event
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Subscribe & heartbeat
|
||||
# ------------------------------------------------------------------
|
||||
@@ -287,16 +499,31 @@ class WecomBotChannel(ChatChannel):
|
||||
chattype = body.get("chattype", "single")
|
||||
is_group = chattype == "group"
|
||||
|
||||
result = self._build_context(body, is_group)
|
||||
if not result:
|
||||
return
|
||||
context, wecom_msg = result
|
||||
wecom_msg.req_id = req_id
|
||||
if req_id:
|
||||
context["on_event"] = self._make_stream_callback(req_id)
|
||||
self.produce(context)
|
||||
|
||||
def _build_context(self, body: dict, is_group: bool, default_aeskey: str = ""):
|
||||
"""Parse a wecom message body into a Context, applying file-cache logic.
|
||||
|
||||
Shared by both the websocket (long-connection) and callback (webhook)
|
||||
receive paths. Returns ``(context, wecom_msg)`` when the message should be
|
||||
handed to the agent, or ``None`` when it was consumed (cached image/file,
|
||||
parse failure, etc.).
|
||||
"""
|
||||
try:
|
||||
wecom_msg = WecomBotMessage(body, is_group=is_group)
|
||||
wecom_msg = WecomBotMessage(body, is_group=is_group, default_aeskey=default_aeskey)
|
||||
except NotImplementedError as e:
|
||||
logger.warning(f"[WecomBot] {e}")
|
||||
return
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] Failed to parse message: {e}", exc_info=True)
|
||||
return
|
||||
|
||||
wecom_msg.req_id = req_id
|
||||
return None
|
||||
|
||||
# File cache logic (same pattern as feishu)
|
||||
from channel.file_cache import get_file_cache
|
||||
@@ -314,13 +541,13 @@ class WecomBotChannel(ChatChannel):
|
||||
if hasattr(wecom_msg, "image_path") and wecom_msg.image_path:
|
||||
file_cache.add(session_id, wecom_msg.image_path, file_type="image")
|
||||
logger.info(f"[WecomBot] Image cached for session {session_id}")
|
||||
return
|
||||
return None
|
||||
|
||||
if wecom_msg.ctype == ContextType.FILE:
|
||||
wecom_msg.prepare()
|
||||
file_cache.add(session_id, wecom_msg.content, file_type="file")
|
||||
logger.info(f"[WecomBot] File cached for session {session_id}: {wecom_msg.content}")
|
||||
return
|
||||
return None
|
||||
|
||||
if wecom_msg.ctype == ContextType.TEXT:
|
||||
cached_files = file_cache.get(session_id)
|
||||
@@ -346,10 +573,9 @@ class WecomBotChannel(ChatChannel):
|
||||
msg=wecom_msg,
|
||||
no_need_at=True,
|
||||
)
|
||||
if context:
|
||||
if req_id:
|
||||
context["on_event"] = self._make_stream_callback(req_id)
|
||||
self.produce(context)
|
||||
if not context:
|
||||
return None
|
||||
return context, wecom_msg
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Event callback
|
||||
@@ -440,6 +666,17 @@ class WecomBotChannel(ChatChannel):
|
||||
state["current"] = ""
|
||||
_push_stream(state, force=True)
|
||||
|
||||
elif event_type == "agent_cancelled":
|
||||
# Flush partial output and strip trailing "---" separator
|
||||
# left over from previous turn, to avoid a dangling divider.
|
||||
if state["current"]:
|
||||
state["committed"] += state["current"]
|
||||
state["current"] = ""
|
||||
state["committed"] = state["committed"].rstrip()
|
||||
if state["committed"].endswith("---"):
|
||||
state["committed"] = state["committed"][:-3].rstrip()
|
||||
_push_stream(state, force=True)
|
||||
|
||||
return on_event
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -479,11 +716,233 @@ class WecomBotChannel(ChatChannel):
|
||||
|
||||
return context
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Callback (webhook) send: write the final reply into the stream state
|
||||
# so the next "流式消息刷新" poll returns it with finish=true.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _callback_send(self, reply: Reply, context: Context):
|
||||
msg = context.get("msg")
|
||||
stream_id = getattr(msg, "stream_id", None) if msg else None
|
||||
if not stream_id:
|
||||
stream_id = context.get("wecom_stream_id")
|
||||
if not stream_id:
|
||||
logger.warning("[WecomBot] callback send without stream_id, dropping reply")
|
||||
return
|
||||
|
||||
if reply.type == ReplyType.TEXT:
|
||||
self._callback_finalize_text(stream_id, reply.content)
|
||||
elif reply.type in (ReplyType.IMAGE_URL, ReplyType.IMAGE):
|
||||
self._callback_finalize_image(stream_id, reply.content)
|
||||
elif reply.type == ReplyType.FILE:
|
||||
# Passive callback replies only support text + image (base64); files
|
||||
# are not supported by the protocol, so append a notice to whatever
|
||||
# text the agent already streamed (do not drop it).
|
||||
text = getattr(reply, "text_content", "") or ""
|
||||
note = (text + "\n\n" if text else "") + "(文件无法在企微回调模式下直接发送)"
|
||||
self._callback_finalize_text(stream_id, note, append=True)
|
||||
elif reply.type in (ReplyType.VIDEO, ReplyType.VIDEO_URL, ReplyType.VOICE):
|
||||
logger.warning(f"[WecomBot] reply type {reply.type} not supported in callback mode")
|
||||
text = getattr(reply, "text_content", "") or ""
|
||||
note = (text + "\n\n" if text else "") + "(该消息类型无法在企微回调模式下直接发送)"
|
||||
self._callback_finalize_text(stream_id, note, append=True)
|
||||
else:
|
||||
self._callback_finalize_text(stream_id, str(reply.content))
|
||||
|
||||
def _callback_get_or_create_state(self, stream_id: str) -> dict:
|
||||
state = self._callback_streams.get(stream_id)
|
||||
if state is None:
|
||||
now = time.time()
|
||||
state = {
|
||||
"committed": "",
|
||||
"current": "",
|
||||
"finished": False,
|
||||
"images": [],
|
||||
"image_urls": [],
|
||||
"image_pending": False,
|
||||
"last_access": now,
|
||||
"created_at": now,
|
||||
"response_url": "",
|
||||
"delivered": False,
|
||||
"url_sent": False,
|
||||
}
|
||||
self._callback_streams[stream_id] = state
|
||||
return state
|
||||
|
||||
def _callback_finalize_text(self, stream_id: str, content: str, append: bool = False):
|
||||
with self._callback_lock:
|
||||
state = self._callback_get_or_create_state(stream_id)
|
||||
accumulated = (state["committed"] + state["current"]).strip()
|
||||
if append and accumulated:
|
||||
state["committed"] = (accumulated + "\n\n" + (content or "")).strip()
|
||||
else:
|
||||
state["committed"] = accumulated if accumulated else (content or "")
|
||||
state["current"] = ""
|
||||
state["last_access"] = time.time()
|
||||
# Don't finish synchronously: chat_channel splits an image-with-caption
|
||||
# reply into a TEXT send followed (0.3s later) by the IMAGE send. If the
|
||||
# text finished the stream immediately, WeCom would close it before the
|
||||
# image arrives. Defer the finish so a trailing image can merge in.
|
||||
self._schedule_text_finish(stream_id)
|
||||
|
||||
def _schedule_text_finish(self, stream_id: str, delay: float = 1.2):
|
||||
def _run():
|
||||
time.sleep(delay)
|
||||
with self._callback_lock:
|
||||
state = self._callback_streams.get(stream_id)
|
||||
if not state or state["finished"] or state.get("image_pending"):
|
||||
return # already finished, or an image reply is on its way
|
||||
state["finished"] = True
|
||||
state["last_access"] = time.time()
|
||||
self._schedule_response_url_fallback(stream_id)
|
||||
|
||||
threading.Thread(target=_run, daemon=True, name=f"wecom-textfin-{stream_id}").start()
|
||||
|
||||
def _callback_finalize_image(self, stream_id: str, img_path_or_url: str):
|
||||
# Mark the image as pending up front (before the slow load/compress) so a
|
||||
# preceding text finalize won't close the stream while we work.
|
||||
with self._callback_lock:
|
||||
self._callback_get_or_create_state(stream_id)["image_pending"] = True
|
||||
b64md5 = self._load_image_base64(img_path_or_url)
|
||||
with self._callback_lock:
|
||||
state = self._callback_get_or_create_state(stream_id)
|
||||
accumulated = (state["committed"] + state["current"]).strip()
|
||||
state["current"] = ""
|
||||
if b64md5:
|
||||
state["images"].append(b64md5)
|
||||
state["committed"] = accumulated
|
||||
# Remember the public url (if any) so the response_url fallback
|
||||
# can embed it as markdown when the poll window has closed.
|
||||
if img_path_or_url.startswith(("http://", "https://")):
|
||||
state["image_urls"].append(img_path_or_url)
|
||||
else:
|
||||
state["committed"] = accumulated or "[图片发送失败]"
|
||||
state["finished"] = True
|
||||
state["image_pending"] = False
|
||||
state["last_access"] = time.time()
|
||||
self._schedule_response_url_fallback(stream_id)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Active reply fallback (response_url): rescue replies that finish after
|
||||
# WeCom stops polling (the passive stream window is ~6 min from the user's
|
||||
# message). A short delay lets an in-flight poll deliver first; only if no
|
||||
# poll picks up the finished answer do we push it actively via response_url.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _schedule_response_url_fallback(self, stream_id: str, delay: float = 3.0):
|
||||
def _run():
|
||||
time.sleep(delay)
|
||||
with self._callback_lock:
|
||||
state = self._callback_streams.get(stream_id)
|
||||
if not state:
|
||||
return
|
||||
if state.get("delivered") or state.get("url_sent"):
|
||||
return # a poll already delivered (or fallback already ran)
|
||||
response_url = state.get("response_url") or ""
|
||||
if not response_url:
|
||||
logger.warning(
|
||||
f"[WecomBot] stream {stream_id} finished after poll window but no response_url; reply dropped"
|
||||
)
|
||||
return
|
||||
content = (state["committed"] + state["current"]).strip()
|
||||
image_urls = list(state.get("image_urls") or [])
|
||||
has_images = bool(state.get("images"))
|
||||
state["url_sent"] = True
|
||||
|
||||
self._send_via_response_url(stream_id, response_url, content, image_urls, has_images)
|
||||
|
||||
threading.Thread(target=_run, daemon=True, name=f"wecom-respurl-{stream_id}").start()
|
||||
|
||||
def _send_via_response_url(self, stream_id, response_url, content, image_urls, has_images):
|
||||
"""Push a one-shot active markdown reply to response_url (valid 1h, single use)."""
|
||||
md = content or ""
|
||||
if image_urls:
|
||||
md += ("\n\n" if md else "") + "\n".join(f"" for u in image_urls)
|
||||
elif has_images:
|
||||
md += ("\n\n" if md else "") + "(图片已生成,但因处理超时无法通过回调发送)"
|
||||
if not md:
|
||||
md = "(处理完成)"
|
||||
payload = {"msgtype": "markdown", "markdown": {"content": md}}
|
||||
try:
|
||||
resp = requests.post(response_url, json=payload, timeout=15)
|
||||
logger.info(
|
||||
f"[WecomBot] response_url active reply sent for {stream_id}: "
|
||||
f"status={resp.status_code}, body={resp.text[:200]}"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] response_url active reply failed for {stream_id}: {e}")
|
||||
|
||||
def _load_image_base64(self, img_path_or_url: str):
|
||||
"""Load a local/remote image, ensure JPG/PNG within 10MB, return (base64, md5)."""
|
||||
local_path = img_path_or_url
|
||||
if local_path.startswith("file://"):
|
||||
local_path = local_path[7:]
|
||||
|
||||
# Temp files we create here (downloads/conversions/compressions) must be
|
||||
# cleaned up afterwards; the caller's original local file must not be.
|
||||
temp_files = []
|
||||
try:
|
||||
if local_path.startswith(("http://", "https://")):
|
||||
try:
|
||||
resp = requests.get(local_path, timeout=30)
|
||||
resp.raise_for_status()
|
||||
tmp_path = f"/tmp/wecom_cb_img_{uuid.uuid4().hex[:8]}"
|
||||
with open(tmp_path, "wb") as f:
|
||||
f.write(resp.content)
|
||||
temp_files.append(tmp_path)
|
||||
local_path = tmp_path
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] Failed to download image for callback reply: {e}")
|
||||
return None
|
||||
|
||||
if not os.path.exists(local_path):
|
||||
logger.error(f"[WecomBot] Image file not found: {local_path}")
|
||||
return None
|
||||
|
||||
formatted = self._ensure_image_format(local_path)
|
||||
if not formatted:
|
||||
return None
|
||||
if formatted != local_path:
|
||||
temp_files.append(formatted)
|
||||
local_path = formatted
|
||||
|
||||
# Unlike the long-connection path (which uploads and sends only a tiny
|
||||
# media_id), the callback reply embeds the whole image as base64 inside
|
||||
# an AES-encrypted body that is returned on EVERY poll. Empirically a
|
||||
# ~1.5MB image (base64 ~2.1MB, encrypted ~2.8MB) makes WeCom reject the
|
||||
# finish packet and poll forever, so cap well below that.
|
||||
callback_max_size = 512 * 1024
|
||||
if os.path.getsize(local_path) > callback_max_size:
|
||||
compressed = self._compress_image(local_path, callback_max_size)
|
||||
if compressed:
|
||||
temp_files.append(compressed)
|
||||
local_path = compressed
|
||||
else:
|
||||
logger.warning("[WecomBot] callback image compress failed; sending original (may be rejected)")
|
||||
|
||||
try:
|
||||
with open(local_path, "rb") as f:
|
||||
raw = f.read()
|
||||
return base64.b64encode(raw).decode("utf-8"), hashlib.md5(raw).hexdigest()
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] Failed to read image for callback reply: {e}")
|
||||
return None
|
||||
finally:
|
||||
for path in temp_files:
|
||||
try:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Send reply
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def send(self, reply: Reply, context: Context):
|
||||
if self.mode == "webhook":
|
||||
self._callback_send(reply, context)
|
||||
return
|
||||
|
||||
msg = context.get("msg")
|
||||
is_group = context.get("isgroup", False)
|
||||
receiver = context.get("receiver", "")
|
||||
@@ -895,3 +1354,85 @@ class WecomBotChannel(ChatChannel):
|
||||
else:
|
||||
logger.error("[WecomBot] Failed to get media_id from finish response")
|
||||
return media_id
|
||||
|
||||
|
||||
class WecomBotCallbackController:
|
||||
"""HTTP controller for wecom bot callback (webhook) mode.
|
||||
|
||||
- GET : URL verification (echo the decrypted echostr).
|
||||
- POST : encrypted message / stream-refresh / event callbacks; returns an
|
||||
encrypted passive reply (or "success" for an empty reply).
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def _channel() -> "WecomBotChannel":
|
||||
return WecomBotChannel()
|
||||
|
||||
def GET(self):
|
||||
channel = self._channel()
|
||||
params = web.input(msg_signature="", timestamp="", nonce="", echostr="")
|
||||
if not channel._crypt:
|
||||
return "wecom bot callback not ready"
|
||||
ret, echo = channel._crypt.verify_url(
|
||||
params.msg_signature, params.timestamp, params.nonce, params.echostr
|
||||
)
|
||||
if ret != 0:
|
||||
logger.error(f"[WecomBot] URL verify failed: ret={ret}")
|
||||
return "verify fail"
|
||||
if isinstance(echo, bytes):
|
||||
echo = echo.decode("utf-8")
|
||||
return echo
|
||||
|
||||
def POST(self):
|
||||
channel = self._channel()
|
||||
if not channel._crypt:
|
||||
return "success"
|
||||
|
||||
params = web.input(msg_signature="", timestamp="", nonce="")
|
||||
body = web.data()
|
||||
ret, plain = channel._crypt.decrypt_msg(
|
||||
body, params.msg_signature, params.timestamp, params.nonce
|
||||
)
|
||||
if ret != 0:
|
||||
logger.error(f"[WecomBot] callback decrypt failed: ret={ret}")
|
||||
return "success"
|
||||
|
||||
try:
|
||||
data = json.loads(plain)
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] callback json parse failed: {e}")
|
||||
return "success"
|
||||
|
||||
msgtype = data.get("msgtype", "")
|
||||
# Stream polls arrive ~1/s; logging each is noisy, so only log non-poll
|
||||
# callbacks here (poll completion is logged in the stream-poll handler).
|
||||
if msgtype != "stream":
|
||||
logger.debug(f"[WecomBot] callback received msgtype={msgtype}")
|
||||
|
||||
try:
|
||||
if msgtype == "stream":
|
||||
reply = channel._callback_handle_stream_poll(data)
|
||||
elif msgtype == "event":
|
||||
event_type = data.get("event", {}).get("eventtype", "")
|
||||
logger.info(f"[WecomBot] callback event: {event_type}")
|
||||
reply = None
|
||||
elif msgtype in ("text", "image", "voice", "file", "video", "mixed"):
|
||||
reply = channel._callback_handle_message(data)
|
||||
else:
|
||||
logger.warning(f"[WecomBot] unsupported callback msgtype: {msgtype}")
|
||||
reply = None
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] callback handling error: {e}", exc_info=True)
|
||||
reply = None
|
||||
|
||||
if not reply:
|
||||
# Empty reply package is acceptable.
|
||||
return "success"
|
||||
|
||||
plain_reply = json.dumps(reply, ensure_ascii=False)
|
||||
ret, enc = channel._crypt.encrypt_msg(plain_reply, params.nonce, params.timestamp)
|
||||
if ret != 0:
|
||||
logger.error(f"[WecomBot] callback encrypt failed: ret={ret}")
|
||||
return "success"
|
||||
web.header("Content-Type", "application/json; charset=utf-8")
|
||||
return json.dumps(enc, ensure_ascii=False)
|
||||
|
||||
203
channel/wecom_bot/wecom_bot_crypt.py
Normal file
203
channel/wecom_bot/wecom_bot_crypt.py
Normal file
@@ -0,0 +1,203 @@
|
||||
"""
|
||||
WeCom (企业微信) smart-bot callback message encryption/decryption.
|
||||
|
||||
Adapted from the official `WXBizJsonMsgCrypt` sample (JSON variant) used by the
|
||||
AI bot callback (webhook) mode. The bot's receive-message callback delivers
|
||||
AES-256-CBC encrypted JSON payloads, and passive replies must be encrypted the
|
||||
same way before being returned in the HTTP response.
|
||||
|
||||
For an enterprise-internal smart bot, ``receive_id`` is always an empty string.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import random
|
||||
import socket
|
||||
import struct
|
||||
import time
|
||||
|
||||
from Crypto.Cipher import AES
|
||||
|
||||
from common.log import logger
|
||||
|
||||
# Error codes (mirrors the official ierror.py)
|
||||
WXBizMsgCrypt_OK = 0
|
||||
WXBizMsgCrypt_ValidateSignature_Error = -40001
|
||||
WXBizMsgCrypt_ParseJson_Error = -40002
|
||||
WXBizMsgCrypt_ComputeSignature_Error = -40003
|
||||
WXBizMsgCrypt_IllegalAesKey = -40004
|
||||
WXBizMsgCrypt_ValidateCorpid_Error = -40005
|
||||
WXBizMsgCrypt_EncryptAES_Error = -40006
|
||||
WXBizMsgCrypt_DecryptAES_Error = -40007
|
||||
WXBizMsgCrypt_IllegalBuffer = -40008
|
||||
WXBizMsgCrypt_EncodeBase64_Error = -40009
|
||||
WXBizMsgCrypt_DecodeBase64_Error = -40010
|
||||
WXBizMsgCrypt_GenReturnJson_Error = -40011
|
||||
|
||||
|
||||
class FormatException(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def _gen_sha1(token, timestamp, nonce, encrypt):
|
||||
"""Compute the WeCom message signature with SHA1 over the sorted parts."""
|
||||
try:
|
||||
if isinstance(encrypt, bytes):
|
||||
encrypt = encrypt.decode("utf-8")
|
||||
sortlist = [str(token), str(timestamp), str(nonce), str(encrypt)]
|
||||
sortlist.sort()
|
||||
sha = hashlib.sha1()
|
||||
sha.update("".join(sortlist).encode("utf-8"))
|
||||
return WXBizMsgCrypt_OK, sha.hexdigest()
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] compute signature error: {e}")
|
||||
return WXBizMsgCrypt_ComputeSignature_Error, None
|
||||
|
||||
|
||||
class _PKCS7Encoder:
|
||||
"""PKCS#7 padding with a 32-byte block size (AES-256)."""
|
||||
|
||||
block_size = 32
|
||||
|
||||
def encode(self, text: bytes) -> bytes:
|
||||
text_length = len(text)
|
||||
amount_to_pad = self.block_size - (text_length % self.block_size)
|
||||
if amount_to_pad == 0:
|
||||
amount_to_pad = self.block_size
|
||||
pad = bytes([amount_to_pad])
|
||||
return text + pad * amount_to_pad
|
||||
|
||||
def decode(self, decrypted: bytes) -> bytes:
|
||||
pad = decrypted[-1]
|
||||
if pad < 1 or pad > 32:
|
||||
pad = 0
|
||||
return decrypted[:-pad] if pad else decrypted
|
||||
|
||||
|
||||
class _Prpcrypt:
|
||||
"""AES-256-CBC encrypt/decrypt for WeCom callback messages."""
|
||||
|
||||
def __init__(self, key: bytes):
|
||||
self.key = key
|
||||
self.mode = AES.MODE_CBC
|
||||
|
||||
def encrypt(self, text: str, receive_id: str):
|
||||
text_bytes = text.encode()
|
||||
# 16-byte random prefix + network-order length + body + receive_id
|
||||
text_bytes = (
|
||||
self._get_random_str()
|
||||
+ struct.pack("I", socket.htonl(len(text_bytes)))
|
||||
+ text_bytes
|
||||
+ receive_id.encode()
|
||||
)
|
||||
text_bytes = _PKCS7Encoder().encode(text_bytes)
|
||||
try:
|
||||
cryptor = AES.new(self.key, self.mode, self.key[:16])
|
||||
ciphertext = cryptor.encrypt(text_bytes)
|
||||
return WXBizMsgCrypt_OK, base64.b64encode(ciphertext)
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] AES encrypt error: {e}")
|
||||
return WXBizMsgCrypt_EncryptAES_Error, None
|
||||
|
||||
def decrypt(self, text, receive_id: str):
|
||||
try:
|
||||
cryptor = AES.new(self.key, self.mode, self.key[:16])
|
||||
plain_text = cryptor.decrypt(base64.b64decode(text))
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] AES decrypt error: {e}")
|
||||
return WXBizMsgCrypt_DecryptAES_Error, None
|
||||
try:
|
||||
pad = plain_text[-1]
|
||||
content = plain_text[16:-pad]
|
||||
json_len = socket.ntohl(struct.unpack("I", content[:4])[0])
|
||||
json_content = content[4 : json_len + 4].decode("utf-8")
|
||||
from_receive_id = content[json_len + 4 :].decode("utf-8")
|
||||
except Exception as e:
|
||||
logger.error(f"[WecomBot] illegal buffer when decrypting: {e}")
|
||||
return WXBizMsgCrypt_IllegalBuffer, None
|
||||
if from_receive_id != receive_id:
|
||||
logger.error(
|
||||
f"[WecomBot] receive_id not match: expect={receive_id}, got={from_receive_id}"
|
||||
)
|
||||
return WXBizMsgCrypt_ValidateCorpid_Error, None
|
||||
return WXBizMsgCrypt_OK, json_content
|
||||
|
||||
@staticmethod
|
||||
def _get_random_str() -> bytes:
|
||||
return str(random.randint(1000000000000000, 9999999999999999)).encode()
|
||||
|
||||
|
||||
class WecomBotCrypt:
|
||||
"""High-level helper for verifying URLs and (de)crypting callback messages."""
|
||||
|
||||
def __init__(self, token: str, encoding_aes_key: str, receive_id: str = ""):
|
||||
try:
|
||||
self.key = base64.b64decode(encoding_aes_key + "=")
|
||||
assert len(self.key) == 32
|
||||
except Exception:
|
||||
raise FormatException("[WecomBot] invalid EncodingAESKey")
|
||||
self.token = token
|
||||
self.receive_id = receive_id
|
||||
|
||||
def verify_url(self, msg_signature, timestamp, nonce, echostr):
|
||||
ret, signature = _gen_sha1(self.token, timestamp, nonce, echostr)
|
||||
if ret != 0:
|
||||
return ret, None
|
||||
if signature != msg_signature:
|
||||
return WXBizMsgCrypt_ValidateSignature_Error, None
|
||||
pc = _Prpcrypt(self.key)
|
||||
return pc.decrypt(echostr, self.receive_id)
|
||||
|
||||
def encrypt_msg(self, reply_msg: str, nonce: str, timestamp: str = None):
|
||||
"""Encrypt a passive-reply JSON string and return the full response JSON.
|
||||
|
||||
Returns (ret, response_dict). On success ret==0 and response_dict is a
|
||||
dict with encrypt/msgsignature/timestamp/nonce fields.
|
||||
"""
|
||||
pc = _Prpcrypt(self.key)
|
||||
ret, encrypt = pc.encrypt(reply_msg, self.receive_id)
|
||||
if ret != 0:
|
||||
return ret, None
|
||||
encrypt = encrypt.decode("utf-8")
|
||||
if timestamp is None:
|
||||
timestamp = str(int(time.time()))
|
||||
ret, signature = _gen_sha1(self.token, timestamp, nonce, encrypt)
|
||||
if ret != 0:
|
||||
return ret, None
|
||||
return WXBizMsgCrypt_OK, {
|
||||
"encrypt": encrypt,
|
||||
"msgsignature": signature,
|
||||
"timestamp": timestamp,
|
||||
"nonce": nonce,
|
||||
}
|
||||
|
||||
def decrypt_msg(self, post_data, msg_signature, timestamp, nonce):
|
||||
"""Verify signature and decrypt the encrypted callback payload.
|
||||
|
||||
``post_data`` may be the raw request body (bytes/str) containing
|
||||
``{"encrypt": "..."}`` or the already-extracted encrypt string.
|
||||
Returns (ret, plaintext_json_str).
|
||||
"""
|
||||
import json
|
||||
|
||||
encrypt = None
|
||||
if isinstance(post_data, (bytes, bytearray)):
|
||||
post_data = post_data.decode("utf-8")
|
||||
if isinstance(post_data, str):
|
||||
try:
|
||||
encrypt = json.loads(post_data).get("encrypt")
|
||||
except Exception:
|
||||
encrypt = post_data
|
||||
elif isinstance(post_data, dict):
|
||||
encrypt = post_data.get("encrypt")
|
||||
if not encrypt:
|
||||
return WXBizMsgCrypt_ParseJson_Error, None
|
||||
|
||||
ret, signature = _gen_sha1(self.token, timestamp, nonce, encrypt)
|
||||
if ret != 0:
|
||||
return ret, None
|
||||
if signature != msg_signature:
|
||||
logger.error("[WecomBot] callback signature not match")
|
||||
return WXBizMsgCrypt_ValidateSignature_Error, None
|
||||
pc = _Prpcrypt(self.key)
|
||||
return pc.decrypt(encrypt, self.receive_id)
|
||||
@@ -87,11 +87,14 @@ def _get_tmp_dir() -> str:
|
||||
class WecomBotMessage(ChatMessage):
|
||||
"""Message wrapper for wecom bot (websocket long-connection mode)."""
|
||||
|
||||
def __init__(self, msg_body: dict, is_group: bool = False):
|
||||
def __init__(self, msg_body: dict, is_group: bool = False, default_aeskey: str = ""):
|
||||
super().__init__(msg_body)
|
||||
self.msg_id = msg_body.get("msgid")
|
||||
self.create_time = msg_body.get("create_time")
|
||||
self.is_group = is_group
|
||||
# In callback (webhook) mode the media bodies carry no per-message aeskey;
|
||||
# the download url is encrypted with the bot's EncodingAESKey instead.
|
||||
self._default_aeskey = default_aeskey
|
||||
|
||||
msg_type = msg_body.get("msgtype")
|
||||
from_userid = msg_body.get("from", {}).get("userid", "")
|
||||
@@ -113,7 +116,7 @@ class WecomBotMessage(ChatMessage):
|
||||
self.ctype = ContextType.IMAGE
|
||||
image_info = msg_body.get("image", {})
|
||||
image_url = image_info.get("url", "")
|
||||
aeskey = image_info.get("aeskey", "")
|
||||
aeskey = image_info.get("aeskey", "") or self._default_aeskey
|
||||
tmp_dir = _get_tmp_dir()
|
||||
image_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}.png")
|
||||
|
||||
@@ -147,7 +150,7 @@ class WecomBotMessage(ChatMessage):
|
||||
elif item_type == "image":
|
||||
img_info = item.get("image", {})
|
||||
img_url = img_info.get("url", "")
|
||||
img_aeskey = img_info.get("aeskey", "")
|
||||
img_aeskey = img_info.get("aeskey", "") or self._default_aeskey
|
||||
img_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}_{idx}.png")
|
||||
try:
|
||||
img_data = _decrypt_media(img_url, img_aeskey)
|
||||
@@ -166,7 +169,7 @@ class WecomBotMessage(ChatMessage):
|
||||
self.ctype = ContextType.FILE
|
||||
file_info = msg_body.get("file", {})
|
||||
file_url = file_info.get("url", "")
|
||||
aeskey = file_info.get("aeskey", "")
|
||||
aeskey = file_info.get("aeskey", "") or self._default_aeskey
|
||||
tmp_dir = _get_tmp_dir()
|
||||
base_path = os.path.join(tmp_dir, f"wecom_{self.msg_id}")
|
||||
self.content = base_path
|
||||
@@ -188,7 +191,7 @@ class WecomBotMessage(ChatMessage):
|
||||
self.ctype = ContextType.FILE
|
||||
video_info = msg_body.get("video", {})
|
||||
video_url = video_info.get("url", "")
|
||||
aeskey = video_info.get("aeskey", "")
|
||||
aeskey = video_info.get("aeskey", "") or self._default_aeskey
|
||||
tmp_dir = _get_tmp_dir()
|
||||
self.content = os.path.join(tmp_dir, f"wecom_{self.msg_id}.mp4")
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ from channel.weixin.weixin_message import WeixinMessage
|
||||
from common.expired_dict import ExpiredDict
|
||||
from common.log import logger
|
||||
from common.singleton import singleton
|
||||
from config import conf
|
||||
from config import conf, get_weixin_credentials_path
|
||||
|
||||
MAX_CONSECUTIVE_FAILURES = 3
|
||||
BACKOFF_DELAY = 30
|
||||
@@ -47,14 +47,16 @@ def _load_credentials(cred_path: str) -> dict:
|
||||
|
||||
|
||||
def _save_credentials(cred_path: str, data: dict):
|
||||
"""Save credentials to JSON file."""
|
||||
"""Atomically save credentials to JSON file (tmp + rename)."""
|
||||
os.makedirs(os.path.dirname(cred_path), exist_ok=True)
|
||||
with open(cred_path, "w") as f:
|
||||
tmp_path = f"{cred_path}.tmp"
|
||||
with open(tmp_path, "w") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
try:
|
||||
os.chmod(cred_path, 0o600)
|
||||
os.chmod(tmp_path, 0o600)
|
||||
except Exception:
|
||||
pass
|
||||
os.replace(tmp_path, cred_path)
|
||||
|
||||
|
||||
@singleton
|
||||
@@ -73,7 +75,10 @@ class WeixinChannel(ChatChannel):
|
||||
self.api = None
|
||||
self._stop_event = threading.Event()
|
||||
self._poll_thread = None
|
||||
self._context_tokens = {} # user_id -> context_token
|
||||
# user_id -> context_token. Guarded by _context_tokens_lock for any
|
||||
# mutation that races with disk persistence.
|
||||
self._context_tokens = {}
|
||||
self._context_tokens_lock = threading.Lock()
|
||||
self._received_msgs = ExpiredDict(60 * 60 * 7.1)
|
||||
self._get_updates_buf = ""
|
||||
self._credentials_path = ""
|
||||
@@ -91,16 +96,21 @@ class WeixinChannel(ChatChannel):
|
||||
cdn_base_url = conf().get("weixin_cdn_base_url", CDN_BASE_URL)
|
||||
token = conf().get("weixin_token", "")
|
||||
|
||||
self._credentials_path = os.path.expanduser(
|
||||
conf().get("weixin_credentials_path", "~/.weixin_cow_credentials.json")
|
||||
)
|
||||
self._credentials_path = get_weixin_credentials_path()
|
||||
|
||||
# Always load credentials so we can restore context_tokens even when
|
||||
# the bot token itself comes from config.
|
||||
creds = _load_credentials(self._credentials_path)
|
||||
if not token:
|
||||
creds = _load_credentials(self._credentials_path)
|
||||
token = creds.get("token", "")
|
||||
if creds.get("base_url"):
|
||||
base_url = creds["base_url"]
|
||||
|
||||
# Restore persisted context_tokens so scheduler can deliver pushes
|
||||
# immediately after restart, without waiting for the user to ping
|
||||
# the bot first.
|
||||
self._restore_context_tokens_from_creds(creds)
|
||||
|
||||
if not token:
|
||||
token, base_url = self._login_with_retry(base_url)
|
||||
if not token:
|
||||
@@ -140,11 +150,16 @@ class WeixinChannel(ChatChannel):
|
||||
def _relogin(self) -> bool:
|
||||
"""Re-login after session expiry. Returns True on success."""
|
||||
base_url = self.api.base_url if self.api else DEFAULT_BASE_URL
|
||||
if os.path.exists(self._credentials_path):
|
||||
try:
|
||||
os.remove(self._credentials_path)
|
||||
except Exception:
|
||||
pass
|
||||
# Clearing the whole credentials file is intentional: the new login
|
||||
# will issue a fresh `token` and persisted context_tokens belong to
|
||||
# the previous bot identity, so they must not survive.
|
||||
with self._context_tokens_lock:
|
||||
self._context_tokens.clear()
|
||||
if os.path.exists(self._credentials_path):
|
||||
try:
|
||||
os.remove(self._credentials_path)
|
||||
except Exception:
|
||||
pass
|
||||
self.login_status = self.LOGIN_STATUS_WAITING
|
||||
result = self._qr_login(base_url)
|
||||
if not result:
|
||||
@@ -156,9 +171,62 @@ class WeixinChannel(ChatChannel):
|
||||
cdn_base_url=self.api.cdn_base_url if self.api else CDN_BASE_URL,
|
||||
)
|
||||
self.login_status = self.LOGIN_STATUS_OK
|
||||
self._context_tokens.clear()
|
||||
return True
|
||||
|
||||
# ── Context token persistence ──────────────────────────────────────
|
||||
# ilink requires every outbound send to echo the context_token from the
|
||||
# user's latest inbound message. We mirror the in-memory map into the
|
||||
# credentials JSON so scheduled pushes survive process restarts.
|
||||
# All mutation + disk IO is serialized via _context_tokens_lock so that
|
||||
# concurrent updates can never lose each other's writes.
|
||||
|
||||
def _restore_context_tokens_from_creds(self, creds: dict) -> None:
|
||||
if not isinstance(creds, dict):
|
||||
return
|
||||
tokens = creds.get("context_tokens")
|
||||
if not isinstance(tokens, dict):
|
||||
return
|
||||
restored = 0
|
||||
with self._context_tokens_lock:
|
||||
for user_id, token in tokens.items():
|
||||
if isinstance(user_id, str) and isinstance(token, str) and token:
|
||||
self._context_tokens[user_id] = token
|
||||
restored += 1
|
||||
if restored:
|
||||
logger.info(f"[Weixin] Restored {restored} context_tokens from credentials")
|
||||
|
||||
def _persist_context_tokens_locked(self) -> None:
|
||||
"""Flush the token map to disk. Caller must hold _context_tokens_lock."""
|
||||
if not self._credentials_path:
|
||||
return
|
||||
try:
|
||||
creds = _load_credentials(self._credentials_path) or {}
|
||||
creds["context_tokens"] = dict(self._context_tokens)
|
||||
_save_credentials(self._credentials_path, creds)
|
||||
except Exception as e:
|
||||
logger.warning(f"[Weixin] Failed to persist context_tokens: {e}")
|
||||
|
||||
def _update_context_token(self, user_id: str, token: str) -> None:
|
||||
"""Update the in-memory token for a user; flush to disk only on change."""
|
||||
if not user_id or not token:
|
||||
return
|
||||
with self._context_tokens_lock:
|
||||
if self._context_tokens.get(user_id) == token:
|
||||
return
|
||||
self._context_tokens[user_id] = token
|
||||
self._persist_context_tokens_locked()
|
||||
|
||||
def _invalidate_context_token(self, user_id: str) -> None:
|
||||
"""Drop the cached token for a user (used after -14 / send rejection)."""
|
||||
if not user_id:
|
||||
return
|
||||
with self._context_tokens_lock:
|
||||
if user_id not in self._context_tokens:
|
||||
return
|
||||
del self._context_tokens[user_id]
|
||||
logger.info(f"[Weixin] Invalidated stale context_token for {user_id}")
|
||||
self._persist_context_tokens_locked()
|
||||
|
||||
# ── QR Login ───────────────────────────────────────────────────────
|
||||
|
||||
@staticmethod
|
||||
@@ -391,7 +459,7 @@ class WeixinChannel(ChatChannel):
|
||||
context_token = raw_msg.get("context_token", "")
|
||||
|
||||
if context_token and from_user:
|
||||
self._context_tokens[from_user] = context_token
|
||||
self._update_context_token(from_user, context_token)
|
||||
|
||||
cdn_base_url = self.api.cdn_base_url if self.api else CDN_BASE_URL
|
||||
try:
|
||||
@@ -510,10 +578,30 @@ class WeixinChannel(ChatChannel):
|
||||
return msg.context_token
|
||||
return self._context_tokens.get(receiver, "")
|
||||
|
||||
def _check_send_response(self, resp, receiver: str) -> None:
|
||||
"""Inspect a send-API response; drop stale context_token on -14.
|
||||
|
||||
ilink uses ret/errcode = -14 to signal that the session (and any
|
||||
cached context_token) is no longer valid. The plugin keeps running
|
||||
because the bot itself can re-login; we just need to forget the
|
||||
per-user token so the next push won't retry forever.
|
||||
"""
|
||||
if not isinstance(resp, dict):
|
||||
return
|
||||
ret = resp.get("ret")
|
||||
errcode = resp.get("errcode")
|
||||
if ret == -14 or errcode == -14:
|
||||
logger.warning(
|
||||
f"[Weixin] Send returned -14 (session expired) for "
|
||||
f"receiver={receiver}; dropping cached context_token"
|
||||
)
|
||||
self._invalidate_context_token(receiver)
|
||||
|
||||
def _send_text(self, text: str, receiver: str, context_token: str):
|
||||
if len(text) <= TEXT_CHUNK_LIMIT:
|
||||
try:
|
||||
self.api.send_text(receiver, text, context_token)
|
||||
resp = self.api.send_text(receiver, text, context_token)
|
||||
self._check_send_response(resp, receiver)
|
||||
logger.debug(f"[Weixin] Text sent to {receiver}, len={len(text)}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Weixin] Failed to send text: {e}")
|
||||
@@ -522,7 +610,8 @@ class WeixinChannel(ChatChannel):
|
||||
chunks = self._split_text(text, TEXT_CHUNK_LIMIT)
|
||||
for i, chunk in enumerate(chunks):
|
||||
try:
|
||||
self.api.send_text(receiver, chunk, context_token)
|
||||
resp = self.api.send_text(receiver, chunk, context_token)
|
||||
self._check_send_response(resp, receiver)
|
||||
logger.debug(f"[Weixin] Text chunk {i+1}/{len(chunks)} sent to {receiver}, len={len(chunk)}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Weixin] Failed to send text chunk {i+1}/{len(chunks)}: {e}")
|
||||
@@ -556,13 +645,14 @@ class WeixinChannel(ChatChannel):
|
||||
return
|
||||
try:
|
||||
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=1)
|
||||
self.api.send_image_item(
|
||||
resp = self.api.send_image_item(
|
||||
to=receiver,
|
||||
context_token=context_token,
|
||||
encrypt_query_param=result["encrypt_query_param"],
|
||||
aes_key_b64=result["aes_key_b64"],
|
||||
ciphertext_size=result["ciphertext_size"],
|
||||
)
|
||||
self._check_send_response(resp, receiver)
|
||||
logger.info(f"[Weixin] Image sent to {receiver}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Weixin] Image send failed: {e}")
|
||||
@@ -575,7 +665,7 @@ class WeixinChannel(ChatChannel):
|
||||
return
|
||||
try:
|
||||
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=3)
|
||||
self.api.send_file_item(
|
||||
resp = self.api.send_file_item(
|
||||
to=receiver,
|
||||
context_token=context_token,
|
||||
encrypt_query_param=result["encrypt_query_param"],
|
||||
@@ -583,6 +673,7 @@ class WeixinChannel(ChatChannel):
|
||||
file_name=os.path.basename(local_path),
|
||||
file_size=result["raw_size"],
|
||||
)
|
||||
self._check_send_response(resp, receiver)
|
||||
logger.info(f"[Weixin] File sent to {receiver}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Weixin] File send failed: {e}")
|
||||
@@ -595,13 +686,14 @@ class WeixinChannel(ChatChannel):
|
||||
return
|
||||
try:
|
||||
result = upload_media_to_cdn(self.api, local_path, receiver, media_type=2)
|
||||
self.api.send_video_item(
|
||||
resp = self.api.send_video_item(
|
||||
to=receiver,
|
||||
context_token=context_token,
|
||||
encrypt_query_param=result["encrypt_query_param"],
|
||||
aes_key_b64=result["aes_key_b64"],
|
||||
ciphertext_size=result["ciphertext_size"],
|
||||
)
|
||||
self._check_send_response(resp, receiver)
|
||||
logger.info(f"[Weixin] Video sent to {receiver}")
|
||||
except Exception as e:
|
||||
logger.error(f"[Weixin] Video send failed: {e}")
|
||||
|
||||
@@ -1 +1 @@
|
||||
2.0.9
|
||||
2.1.1
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import click
|
||||
from cli import __version__
|
||||
from cli.commands.skill import skill
|
||||
from cli.commands.process import start, stop, restart, update, status, logs
|
||||
from cli.commands.process import start, stop, restart, self_restart, update, status, logs
|
||||
from cli.commands.context import context
|
||||
from cli.commands.install import install_browser
|
||||
from cli.commands.knowledge import knowledge
|
||||
@@ -68,6 +68,7 @@ main.add_command(skill)
|
||||
main.add_command(start)
|
||||
main.add_command(stop)
|
||||
main.add_command(restart)
|
||||
main.add_command(self_restart)
|
||||
main.add_command(update)
|
||||
main.add_command(status)
|
||||
main.add_command(logs)
|
||||
|
||||
@@ -14,7 +14,7 @@ CHINA_MIRROR = "https://registry.npmmirror.com/-/binary/playwright"
|
||||
|
||||
# stream(msg, fg=None) — fg is "yellow" | "green" | "red" | None
|
||||
StreamFn = Callable[[str, Optional[str]], None]
|
||||
# on_phase(msg) — coarse-grained progress for chat channels (Chinese)
|
||||
# on_phase(msg) — coarse-grained progress for chat channels (localized via i18n)
|
||||
PhaseFn = Callable[[str], None]
|
||||
|
||||
|
||||
@@ -78,12 +78,13 @@ def _is_china_network() -> bool:
|
||||
|
||||
|
||||
def _pip_install(package_spec: str, stream: StreamFn) -> int:
|
||||
"""Install a package, retrying with --user on permission failure."""
|
||||
"""Install a package, preferring prebuilt wheels; retry with --user on perm error."""
|
||||
python = sys.executable
|
||||
ret = subprocess.call([python, "-m", "pip", "install", package_spec])
|
||||
base = [python, "-m", "pip", "install", "--prefer-binary"]
|
||||
ret = subprocess.call(base + [package_spec])
|
||||
if ret != 0:
|
||||
stream(" Retrying with --user flag...", "yellow")
|
||||
ret = subprocess.call([python, "-m", "pip", "install", "--user", package_spec])
|
||||
ret = subprocess.call(base + ["--user", package_spec])
|
||||
return ret
|
||||
|
||||
|
||||
@@ -112,16 +113,27 @@ def run_install_browser(
|
||||
stream: Optional callback ``(message, fg)`` for each line. ``fg`` is
|
||||
``yellow`` / ``green`` / ``red`` or None. Defaults to colored click output.
|
||||
on_phase: Optional callback for coarse progress (e.g. push to chat);
|
||||
messages are short Chinese status lines.
|
||||
messages are short status lines localized via i18n.
|
||||
|
||||
Returns:
|
||||
0 on success, 1 on fatal failure (pip or chromium install failed).
|
||||
"""
|
||||
from cli.utils import get_cli_language
|
||||
|
||||
# Import `common` only after get_cli_language() runs ensure_sys_path(),
|
||||
# so it works when `cow` is invoked from outside the project directory.
|
||||
get_cli_language() # resolve cow_lang so i18n.t reflects config
|
||||
from common import i18n
|
||||
_t = i18n.t
|
||||
|
||||
stream = stream or _default_stream
|
||||
python = sys.executable
|
||||
legacy_mode = False
|
||||
|
||||
_phase(on_phase, "🔧 开始安装浏览器工具依赖(约几分钟,请耐心等待)…")
|
||||
_phase(on_phase, _t(
|
||||
"🔧 开始安装浏览器工具依赖(约几分钟,请耐心等待)…",
|
||||
"🔧 Installing browser tool dependencies (a few minutes, please wait)…",
|
||||
))
|
||||
|
||||
glibc = _get_glibc_version()
|
||||
if glibc and glibc < GLIBC_THRESHOLD:
|
||||
@@ -136,27 +148,52 @@ def run_install_browser(
|
||||
stream("")
|
||||
_phase(
|
||||
on_phase,
|
||||
f"ℹ️ 检测到 glibc {glibc_str}(较旧),将安装兼容版 Playwright {PLAYWRIGHT_LEGACY_VERSION}。",
|
||||
_t(
|
||||
f"ℹ️ 检测到 glibc {glibc_str}(较旧),将安装兼容版 Playwright {PLAYWRIGHT_LEGACY_VERSION}。",
|
||||
f"ℹ️ Detected glibc {glibc_str} (older); installing compatible Playwright {PLAYWRIGHT_LEGACY_VERSION}.",
|
||||
),
|
||||
)
|
||||
|
||||
target_version = PLAYWRIGHT_LEGACY_VERSION if legacy_mode else PLAYWRIGHT_VERSION
|
||||
|
||||
_phase(on_phase, "📦 [1/3] 正在安装 Playwright Python 包…")
|
||||
# Windows-only: greenlet 3.2.x ships no Windows wheel, so pip would build it
|
||||
# from source (needs MSVC) and fail. Pre-install 3.1.x (has win wheels for
|
||||
# py3.7-3.13) which still satisfies playwright's greenlet>=3.1.1,<4.
|
||||
if sys.platform == "win32":
|
||||
stream("[1/3] Pre-installing greenlet (prebuilt wheel) for Windows...", "yellow")
|
||||
ret = subprocess.call(
|
||||
[python, "-m", "pip", "install", "--only-binary=:all:", "greenlet>=3.1.1,<3.2"]
|
||||
)
|
||||
if ret != 0:
|
||||
stream(
|
||||
" Could not pre-install a prebuilt greenlet wheel.\n"
|
||||
" playwright may try to build greenlet from source, which needs\n"
|
||||
" Microsoft C++ Build Tools: https://visualstudio.microsoft.com/visual-cpp-build-tools/",
|
||||
"yellow",
|
||||
)
|
||||
|
||||
_phase(on_phase, _t("📦 [1/3] 正在安装 Playwright Python 包…", "📦 [1/3] Installing Playwright Python package…"))
|
||||
stream("[1/3] Installing playwright Python package...", "yellow")
|
||||
ret = _pip_install(f"playwright=={target_version}", stream)
|
||||
if ret != 0:
|
||||
stream("Failed to install playwright package.", "red")
|
||||
_phase(on_phase, "❌ [1/3] Playwright Python 包安装失败。")
|
||||
_phase(on_phase, _t("❌ [1/3] Playwright Python 包安装失败。", "❌ [1/3] Failed to install Playwright Python package."))
|
||||
return 1
|
||||
|
||||
installed = _get_installed_version()
|
||||
if installed:
|
||||
stream(f" playwright {installed} installed.", "green")
|
||||
stream("")
|
||||
_phase(on_phase, f"✅ [1/3] Playwright 包已安装({installed or target_version})。")
|
||||
_phase(on_phase, _t(
|
||||
f"✅ [1/3] Playwright 包已安装({installed or target_version})。",
|
||||
f"✅ [1/3] Playwright package installed ({installed or target_version}).",
|
||||
))
|
||||
|
||||
if sys.platform == "linux":
|
||||
_phase(on_phase, "🔧 [2/3] 正在安装 Linux 系统依赖与轻量中文字体(文泉驿正黑,部分步骤可能需要 sudo)…")
|
||||
_phase(on_phase, _t(
|
||||
"🔧 [2/3] 正在安装 Linux 系统依赖与轻量中文字体(文泉驿正黑,部分步骤可能需要 sudo)…",
|
||||
"🔧 [2/3] Installing Linux system deps and a lightweight CJK font (WenQuanYi Zen Hei; some steps may need sudo)…",
|
||||
))
|
||||
stream("[2/3] Installing system dependencies (Linux)...", "yellow")
|
||||
ret = subprocess.call([python, "-m", "playwright", "install-deps", "chromium"])
|
||||
if ret != 0:
|
||||
@@ -183,14 +220,23 @@ def run_install_browser(
|
||||
stream(" CJK font (wqy-zenhei) installed.", "green")
|
||||
_phase(
|
||||
on_phase,
|
||||
"✅ [2/3] Linux 依赖与字体步骤已执行(若有权限问题请查看服务器日志或手动执行提示命令)。",
|
||||
_t(
|
||||
"✅ [2/3] Linux 依赖与字体步骤已执行(若有权限问题请查看服务器日志或手动执行提示命令)。",
|
||||
"✅ [2/3] Linux deps and font steps executed (on permission issues, check the server log or run the suggested commands manually).",
|
||||
),
|
||||
)
|
||||
else:
|
||||
stream(f"[2/3] Skipping system deps (not needed on {sys.platform}).", "yellow")
|
||||
_phase(on_phase, f"ℹ️ [2/3] 当前系统({sys.platform})跳过 Linux 专用依赖。")
|
||||
_phase(on_phase, _t(
|
||||
f"ℹ️ [2/3] 当前系统({sys.platform})跳过 Linux 专用依赖。",
|
||||
f"ℹ️ [2/3] Skipping Linux-specific deps on this platform ({sys.platform}).",
|
||||
))
|
||||
stream("")
|
||||
|
||||
_phase(on_phase, "🌐 [3/3] 正在下载并安装 Chromium(体积较大,请耐心等待)…")
|
||||
_phase(on_phase, _t(
|
||||
"🌐 [3/3] 正在下载并安装 Chromium(体积较大,请耐心等待)…",
|
||||
"🌐 [3/3] Downloading and installing Chromium (large download, please wait)…",
|
||||
))
|
||||
stream("[3/3] Installing Chromium browser...", "yellow")
|
||||
cmd = [python, "-m", "playwright", "install", "chromium"]
|
||||
|
||||
@@ -209,27 +255,33 @@ def run_install_browser(
|
||||
if use_mirror:
|
||||
env["PLAYWRIGHT_DOWNLOAD_HOST"] = CHINA_MIRROR
|
||||
stream(f" (using China mirror: {CHINA_MIRROR})", None)
|
||||
_phase(on_phase, "📡 检测到国内 pip 源配置,Chromium 将优先走国内镜像下载。")
|
||||
_phase(on_phase, _t(
|
||||
"📡 检测到国内 pip 源配置,Chromium 将优先走国内镜像下载。",
|
||||
"📡 Detected a China pip mirror; Chromium will be downloaded from the China mirror first.",
|
||||
))
|
||||
|
||||
ret = subprocess.call(cmd, env=env)
|
||||
|
||||
if ret != 0 and use_mirror:
|
||||
stream(" Mirror download failed, retrying with official CDN...", "yellow")
|
||||
_phase(on_phase, "⚠️ 镜像下载失败,正在改用官方源重试…")
|
||||
_phase(on_phase, _t(
|
||||
"⚠️ 镜像下载失败,正在改用官方源重试…",
|
||||
"⚠️ Mirror download failed; retrying with the official CDN…",
|
||||
))
|
||||
env_no_mirror = os.environ.copy()
|
||||
env_no_mirror.pop("PLAYWRIGHT_DOWNLOAD_HOST", None)
|
||||
ret = subprocess.call(cmd, env=env_no_mirror)
|
||||
|
||||
if ret != 0:
|
||||
stream("Failed to install Chromium.", "red")
|
||||
_phase(on_phase, "❌ [3/3] Chromium 安装失败。")
|
||||
_phase(on_phase, _t("❌ [3/3] Chromium 安装失败。", "❌ [3/3] Failed to install Chromium."))
|
||||
return 1
|
||||
|
||||
stream("")
|
||||
_phase(on_phase, "✅ [3/3] Chromium 已安装。")
|
||||
_phase(on_phase, _t("✅ [3/3] Chromium 已安装。", "✅ [3/3] Chromium installed."))
|
||||
|
||||
stream("Verifying browser installation...", None)
|
||||
_phase(on_phase, "🔍 正在验证 Playwright 能否正常加载…")
|
||||
_phase(on_phase, _t("🔍 正在验证 Playwright 能否正常加载…", "🔍 Verifying that Playwright loads correctly…"))
|
||||
ret = subprocess.call(
|
||||
[python, "-c", "from playwright.sync_api import sync_playwright; print('OK')"],
|
||||
stderr=subprocess.DEVNULL,
|
||||
@@ -240,14 +292,20 @@ def run_install_browser(
|
||||
" Consider upgrading your OS or using Docker.",
|
||||
"yellow",
|
||||
)
|
||||
_phase(on_phase, "⚠️ 验证未完全通过:本机可能仍无法使用浏览器工具,请查看日志或升级系统。")
|
||||
_phase(on_phase, _t(
|
||||
"⚠️ 验证未完全通过:本机可能仍无法使用浏览器工具,请查看日志或升级系统。",
|
||||
"⚠️ Verification did not fully pass: the browser tool may still not work here; check the log or upgrade your system.",
|
||||
))
|
||||
else:
|
||||
stream(" Verification passed.", "green")
|
||||
_phase(on_phase, "✅ 验证通过。")
|
||||
_phase(on_phase, _t("✅ 验证通过。", "✅ Verification passed."))
|
||||
|
||||
stream("")
|
||||
stream("Browser tool ready! Restart CowAgent to enable it.", "green")
|
||||
_phase(on_phase, "🎉 全部步骤结束。请重启 CowAgent 后使用 browser 工具。")
|
||||
_phase(on_phase, _t(
|
||||
"🎉 全部步骤结束。请重启 CowAgent 后使用 browser 工具。",
|
||||
"🎉 All steps finished. Restart CowAgent to use the browser tool.",
|
||||
))
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
@@ -8,11 +8,28 @@ from typing import Optional
|
||||
|
||||
import click
|
||||
|
||||
from cli.utils import get_project_root
|
||||
from cli.utils import get_project_root, load_config_json
|
||||
|
||||
_IS_WIN = sys.platform == "win32"
|
||||
|
||||
|
||||
def _is_terminal_only() -> bool:
|
||||
"""Whether terminal is the only configured channel.
|
||||
|
||||
Terminal needs an interactive stdin/tty, which is incompatible with the
|
||||
background daemon mode (stdout/stdin detached). When terminal is the only
|
||||
channel, `start` must run in the foreground so it can own the tty.
|
||||
"""
|
||||
channel = load_config_json().get("channel_type", "")
|
||||
if isinstance(channel, str):
|
||||
names = [c.strip() for c in channel.split(",") if c.strip()]
|
||||
elif isinstance(channel, (list, tuple)):
|
||||
names = [str(c).strip() for c in channel if str(c).strip()]
|
||||
else:
|
||||
names = []
|
||||
return names == ["terminal"]
|
||||
|
||||
|
||||
def _get_pid_file():
|
||||
return os.path.join(get_project_root(), ".cow.pid")
|
||||
|
||||
@@ -103,6 +120,12 @@ def start(foreground, no_logs):
|
||||
|
||||
python = sys.executable
|
||||
|
||||
# Terminal-only setups need an interactive tty; force foreground so the
|
||||
# terminal channel can read stdin instead of fighting the shell over the tty.
|
||||
if not foreground and _is_terminal_only():
|
||||
foreground = True
|
||||
click.echo("Detected terminal-only channel, starting in foreground...")
|
||||
|
||||
if foreground:
|
||||
click.echo("Starting CowAgent in foreground...")
|
||||
if _IS_WIN:
|
||||
@@ -172,6 +195,120 @@ def restart(ctx, no_logs):
|
||||
ctx.invoke(start, no_logs=no_logs)
|
||||
|
||||
|
||||
# Detached relay that survives the caller's process tree. Run via `python -c`
|
||||
# with start_new_session=True so it keeps going after the agent's bash child
|
||||
# (and the main app it kills) both die. Flow: self-check the new code FIRST
|
||||
# (import app); abort without touching the old process if it fails. Only when
|
||||
# the new code is loadable does it SIGTERM the old PID, wait for exit (SIGKILL
|
||||
# fallback), then start a fresh app.py and write the pid.
|
||||
_RELAY_SCRIPT = r"""
|
||||
import os, sys, time, signal, subprocess
|
||||
|
||||
root, python, app_py, pid_file, log_file = sys.argv[1:6]
|
||||
old_pid = int(sys.argv[6]) if len(sys.argv) > 6 and sys.argv[6] else 0
|
||||
|
||||
|
||||
def alive(pid):
|
||||
if not pid:
|
||||
return False
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
return True
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def log(msg):
|
||||
try:
|
||||
with open(log_file, "a") as f:
|
||||
f.write("[self-restart] " + msg + "\n")
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# 0) Self-check: make sure the new code actually loads BEFORE killing anything.
|
||||
# `import app` exercises top-level imports / syntax of the entry module. If it
|
||||
# fails, abort and leave the running service untouched — never end up with the
|
||||
# old process killed and the new one unable to start.
|
||||
check = subprocess.run(
|
||||
[python, "-c", "import app"], cwd=root,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||
)
|
||||
if check.returncode != 0:
|
||||
detail = (check.stdout or b"").decode("utf-8", "replace").strip()
|
||||
log("self-check FAILED, aborting restart; service left running:\n" + detail)
|
||||
sys.exit(1)
|
||||
log("self-check passed")
|
||||
|
||||
# 1) Ask the old process to exit gracefully (its SIGTERM handler persists state).
|
||||
if alive(old_pid):
|
||||
try:
|
||||
os.kill(old_pid, signal.SIGTERM)
|
||||
except OSError:
|
||||
pass
|
||||
# 2) Wait up to ~15s for it to go, then force-kill as a backstop.
|
||||
for _ in range(150):
|
||||
if not alive(old_pid):
|
||||
break
|
||||
time.sleep(0.1)
|
||||
else:
|
||||
try:
|
||||
os.kill(old_pid, signal.SIGKILL)
|
||||
except OSError:
|
||||
pass
|
||||
time.sleep(0.5)
|
||||
|
||||
# 3) Start a fresh instance, detached, logging to the same file.
|
||||
with open(log_file, "a") as f:
|
||||
proc = subprocess.Popen(
|
||||
[python, app_py], cwd=root,
|
||||
stdout=f, stderr=f, start_new_session=True,
|
||||
)
|
||||
with open(pid_file, "w") as f:
|
||||
f.write(str(proc.pid))
|
||||
log("restarted, new pid=" + str(proc.pid))
|
||||
"""
|
||||
|
||||
|
||||
@click.command(name="self-restart", hidden=True)
|
||||
def self_restart():
|
||||
"""Restart from inside the running agent (detached; survives parent death).
|
||||
|
||||
Intended to be invoked by the agent itself (e.g. via bash after editing its
|
||||
own code), not by users — so it is hidden from `cow help`. Unlike `restart`,
|
||||
the actual stop+start runs in a detached relay process that outlives the
|
||||
agent's bash child, which would otherwise die together with the main app it
|
||||
kills.
|
||||
"""
|
||||
if _IS_WIN:
|
||||
click.echo("self-restart is not supported on Windows; use `cow restart`.", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
root = get_project_root()
|
||||
app_py = os.path.join(root, "app.py")
|
||||
if not os.path.exists(app_py):
|
||||
click.echo("Error: app.py not found in project root.", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
python = sys.executable
|
||||
pid = _read_pid() or 0
|
||||
|
||||
subprocess.Popen(
|
||||
[
|
||||
python, "-c", _RELAY_SCRIPT,
|
||||
root, python, app_py, _get_pid_file(), _get_log_file(), str(pid),
|
||||
],
|
||||
cwd=root,
|
||||
start_new_session=True,
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
)
|
||||
click.echo(click.style(
|
||||
"✓ Restart scheduled. The service will stop and come back in a few seconds.",
|
||||
fg="green",
|
||||
))
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.pass_context
|
||||
def update(ctx):
|
||||
@@ -252,7 +389,14 @@ def update(ctx):
|
||||
def status():
|
||||
"""Show CowAgent running status."""
|
||||
from cli import __version__
|
||||
from cli.utils import load_config_json
|
||||
from cli.utils import load_config_json, get_cli_language, get_project_root
|
||||
|
||||
# get_cli_language() calls ensure_sys_path(), which adds the project root
|
||||
# to sys.path. Import `common` only AFTER that, otherwise it fails with
|
||||
# ModuleNotFoundError when `cow` runs from outside the project dir.
|
||||
get_cli_language() # resolve cow_lang so i18n.t reflects config
|
||||
from common import i18n
|
||||
_t = i18n.t
|
||||
|
||||
pid = _read_pid()
|
||||
if pid:
|
||||
@@ -260,17 +404,24 @@ def status():
|
||||
else:
|
||||
click.echo(click.style("● CowAgent is not running", fg="red"))
|
||||
|
||||
click.echo(f" 版本: v{__version__}")
|
||||
click.echo(_t(f" 版本: v{__version__}", f" Version: v{__version__}"))
|
||||
|
||||
# Project path bound to this `cow` CLI — disambiguates which checkout the
|
||||
# command actually controls when the user has multiple clones.
|
||||
project_root = get_project_root()
|
||||
click.echo(_t(f" 路径: {project_root}", f" Path: {project_root}"))
|
||||
|
||||
cfg = load_config_json()
|
||||
if cfg:
|
||||
channel = cfg.get("channel_type", "unknown")
|
||||
if isinstance(channel, list):
|
||||
channel = ", ".join(channel)
|
||||
click.echo(f" 通道: {channel}")
|
||||
click.echo(f" 模型: {cfg.get('model', 'unknown')}")
|
||||
click.echo(_t(f" 通道: {channel}", f" Channel: {channel}"))
|
||||
click.echo(_t(f" 模型: {cfg.get('model', 'unknown')}", f" Model: {cfg.get('model', 'unknown')}"))
|
||||
mode = "Chat" if cfg.get("agent") is False else "Agent"
|
||||
click.echo(f" 模式: {mode}")
|
||||
click.echo(_t(f" 模式: {mode}", f" Mode: {mode}"))
|
||||
lang_label = "中文" if i18n.get_language() == "zh" else "English"
|
||||
click.echo(_t(f" 语言: {lang_label}", f" Language: {lang_label}"))
|
||||
|
||||
|
||||
@click.command()
|
||||
|
||||
@@ -517,18 +517,26 @@ def _install_targz_bytes(content: bytes, name: str, skills_dir: str, result: Ins
|
||||
|
||||
def _print_install_success(name: str, source: str):
|
||||
"""Print a unified install success message with description and source."""
|
||||
from cli.utils import get_cli_language
|
||||
|
||||
# Import `common` only after get_cli_language() runs ensure_sys_path(),
|
||||
# so it works when `cow` is invoked from outside the project directory.
|
||||
get_cli_language() # resolve cow_lang so i18n.t reflects config
|
||||
from common import i18n
|
||||
_t = i18n.t
|
||||
|
||||
skills_dir = get_skills_dir()
|
||||
config = load_skills_config()
|
||||
display = config.get(name, {}).get("display_name", "")
|
||||
desc = _read_skill_description(os.path.join(skills_dir, name))
|
||||
click.echo(click.style(f"✓ {name}", fg="green"))
|
||||
if display and display != name:
|
||||
click.echo(f" 名称: {display}")
|
||||
click.echo(_t(f" 名称: {display}", f" Name: {display}"))
|
||||
if desc:
|
||||
if len(desc) > 60:
|
||||
desc = desc[:57] + "…"
|
||||
click.echo(f" 描述: {desc}")
|
||||
click.echo(f" 来源: {source}")
|
||||
click.echo(_t(f" 描述: {desc}", f" Description: {desc}"))
|
||||
click.echo(_t(f" 来源: {source}", f" Source: {source}"))
|
||||
|
||||
|
||||
def _validate_skill_name(name: str):
|
||||
|
||||
16
cli/utils.py
16
cli/utils.py
@@ -40,6 +40,22 @@ def load_config_json() -> dict:
|
||||
return {}
|
||||
|
||||
|
||||
def get_cli_language() -> str:
|
||||
"""Resolve the CLI UI language using the shared i18n detector.
|
||||
|
||||
Reads the `cow_lang` field from config.json (defaults to "auto") and runs
|
||||
the same detection used by the running app, so CLI output matches.
|
||||
"""
|
||||
ensure_sys_path()
|
||||
try:
|
||||
from common import i18n
|
||||
|
||||
configured = load_config_json().get("cow_lang", "auto")
|
||||
return i18n.resolve_language(configured)
|
||||
except Exception:
|
||||
return "en"
|
||||
|
||||
|
||||
def load_skills_config() -> dict:
|
||||
"""Load skills_config.json from the custom skills directory."""
|
||||
path = os.path.join(get_skills_dir(), "skills_config.json")
|
||||
|
||||
@@ -21,7 +21,7 @@ from bridge.context import Context, ContextType
|
||||
from bridge.reply import Reply, ReplyType
|
||||
from common.log import logger
|
||||
from linkai import LinkAIClient, PushMsg
|
||||
from config import conf, pconf, plugin_config, available_setting, write_plugin_config, get_root
|
||||
from config import conf, pconf, plugin_config, available_setting, write_plugin_config, get_root, get_weixin_credentials_path
|
||||
from plugins import PluginManager
|
||||
import threading
|
||||
import time
|
||||
@@ -34,7 +34,9 @@ chat_client: LinkAIClient
|
||||
|
||||
CHANNEL_ACTIONS = {"channel_create", "channel_update", "channel_delete"}
|
||||
|
||||
# channelType -> config key mapping for app credentials
|
||||
# channelType -> config key mapping for app credentials.
|
||||
# secret_key may be "" for single-token channels (e.g. telegram/discord).
|
||||
# For slack, appId carries bot_token and appSecret carries app_token.
|
||||
CREDENTIAL_MAP = {
|
||||
"feishu": ("feishu_app_id", "feishu_app_secret"),
|
||||
"dingtalk": ("dingtalk_client_id", "dingtalk_client_secret"),
|
||||
@@ -43,6 +45,9 @@ CREDENTIAL_MAP = {
|
||||
"wechatmp": ("wechatmp_app_id", "wechatmp_app_secret"),
|
||||
"wechatmp_service": ("wechatmp_app_id", "wechatmp_app_secret"),
|
||||
"wechatcom_app": ("wechatcomapp_agent_id", "wechatcomapp_secret"),
|
||||
"telegram": ("telegram_token", ""),
|
||||
"slack": ("slack_bot_token", "slack_app_token"),
|
||||
"discord": ("discord_token", ""),
|
||||
}
|
||||
|
||||
|
||||
@@ -167,6 +172,11 @@ class CloudClient(LinkAIClient):
|
||||
if key in available_setting and config.get(key) is not None:
|
||||
local_config[key] = config.get(key)
|
||||
|
||||
# Self-evolution switch: normalize remote value (bool / "Y"/"N" / "true")
|
||||
# to a real bool so the evolution config parser reads it correctly.
|
||||
if config.get("self_evolution_enabled") is not None:
|
||||
local_config["self_evolution_enabled"] = self._to_bool(config.get("self_evolution_enabled"))
|
||||
|
||||
# Voice settings
|
||||
reply_voice_mode = config.get("reply_voice_mode")
|
||||
if reply_voice_mode:
|
||||
@@ -326,9 +336,7 @@ class CloudClient(LinkAIClient):
|
||||
@staticmethod
|
||||
def _remove_weixin_credentials():
|
||||
"""Remove the weixin token credentials file so next connect triggers QR login."""
|
||||
cred_path = os.path.expanduser(
|
||||
conf().get("weixin_credentials_path", "~/.weixin_cow_credentials.json")
|
||||
)
|
||||
cred_path = get_weixin_credentials_path()
|
||||
try:
|
||||
if os.path.exists(cred_path):
|
||||
os.remove(cred_path)
|
||||
@@ -336,6 +344,20 @@ class CloudClient(LinkAIClient):
|
||||
except Exception as e:
|
||||
logger.warning(f"[CloudClient] Failed to remove weixin credentials: {e}")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# value helpers
|
||||
# ------------------------------------------------------------------
|
||||
@staticmethod
|
||||
def _to_bool(value) -> bool:
|
||||
"""Normalize a remote config value to bool (bool / "Y"/"N" / "true"/"1")."""
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
if isinstance(value, (int, float)):
|
||||
return value != 0
|
||||
if isinstance(value, str):
|
||||
return value.strip().lower() in ("y", "yes", "true", "1", "on")
|
||||
return False
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# channel credentials helpers
|
||||
# ------------------------------------------------------------------
|
||||
@@ -357,7 +379,8 @@ class CloudClient(LinkAIClient):
|
||||
local_config[id_key] = app_id
|
||||
os.environ[id_key.upper()] = str(app_id)
|
||||
changed = True
|
||||
if app_secret is not None and local_config.get(secret_key) != app_secret:
|
||||
# secret_key may be empty for single-token channels (e.g. telegram/discord)
|
||||
if secret_key and app_secret is not None and local_config.get(secret_key) != app_secret:
|
||||
local_config[secret_key] = app_secret
|
||||
os.environ[secret_key.upper()] = str(app_secret)
|
||||
changed = True
|
||||
@@ -372,9 +395,10 @@ class CloudClient(LinkAIClient):
|
||||
return
|
||||
id_key, secret_key = cred
|
||||
local_config.pop(id_key, None)
|
||||
local_config.pop(secret_key, None)
|
||||
os.environ.pop(id_key.upper(), None)
|
||||
os.environ.pop(secret_key.upper(), None)
|
||||
if secret_key:
|
||||
local_config.pop(secret_key, None)
|
||||
os.environ.pop(secret_key.upper(), None)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# channel_type list helpers
|
||||
@@ -848,6 +872,10 @@ def _build_config():
|
||||
"agent_max_context_turns": local_conf.get("agent_max_context_turns"),
|
||||
"agent_max_context_tokens": local_conf.get("agent_max_context_tokens"),
|
||||
"agent_max_steps": local_conf.get("agent_max_steps"),
|
||||
# Self-evolution switch reported so the console can reflect state
|
||||
"self_evolution_enabled": "Y" if local_conf.get("self_evolution_enabled") else "N",
|
||||
"self_evolution_idle_minutes": local_conf.get("self_evolution_idle_minutes"),
|
||||
"self_evolution_min_turns": local_conf.get("self_evolution_min_turns"),
|
||||
"channelType": local_conf.get("channel_type"),
|
||||
}
|
||||
|
||||
@@ -862,25 +890,16 @@ def _build_config():
|
||||
if plugin_config.get("Godcmd"):
|
||||
config["admin_password"] = plugin_config.get("Godcmd").get("password")
|
||||
|
||||
# Add channel-specific app credentials
|
||||
# Add channel-specific app credentials based on CREDENTIAL_MAP.
|
||||
# For multi-channel channel_type (comma-separated), the first matched type wins.
|
||||
current_channel_type = local_conf.get("channel_type", "")
|
||||
if current_channel_type == "feishu":
|
||||
config["app_id"] = local_conf.get("feishu_app_id")
|
||||
config["app_secret"] = local_conf.get("feishu_app_secret")
|
||||
elif current_channel_type == "dingtalk":
|
||||
config["app_id"] = local_conf.get("dingtalk_client_id")
|
||||
config["app_secret"] = local_conf.get("dingtalk_client_secret")
|
||||
elif current_channel_type in ("wechatmp", "wechatmp_service"):
|
||||
config["app_id"] = local_conf.get("wechatmp_app_id")
|
||||
config["app_secret"] = local_conf.get("wechatmp_app_secret")
|
||||
elif current_channel_type == "wecom_bot":
|
||||
config["app_id"] = local_conf.get("wecom_bot_id")
|
||||
config["app_secret"] = local_conf.get("wecom_bot_secret")
|
||||
elif current_channel_type == "qq":
|
||||
config["app_id"] = local_conf.get("qq_app_id")
|
||||
config["app_secret"] = local_conf.get("qq_app_secret")
|
||||
elif current_channel_type == "wechatcom_app":
|
||||
config["app_id"] = local_conf.get("wechatcomapp_agent_id")
|
||||
config["app_secret"] = local_conf.get("wechatcomapp_secret")
|
||||
for ch_type in CloudClient._parse_channel_types({"channel_type": current_channel_type}):
|
||||
cred = CREDENTIAL_MAP.get(ch_type)
|
||||
if not cred:
|
||||
continue
|
||||
id_key, secret_key = cred
|
||||
config["app_id"] = local_conf.get(id_key)
|
||||
config["app_secret"] = local_conf.get(secret_key) if secret_key else ""
|
||||
break
|
||||
|
||||
return config
|
||||
|
||||
111
common/const.py
111
common/const.py
@@ -1,4 +1,4 @@
|
||||
# 厂商类型
|
||||
# Provider types
|
||||
OPEN_AI = "openAI"
|
||||
OPENAI = "openai"
|
||||
CHATGPT = "chatGPT" # legacy alias for OPENAI, kept for backward compatibility
|
||||
@@ -8,46 +8,49 @@ XUNFEI = "xunfei"
|
||||
CHATGPTONAZURE = "chatGPTOnAzure"
|
||||
LINKAI = "linkai"
|
||||
CLAUDEAPI= "claudeAPI"
|
||||
QWEN = "qwen" # 千问 (兼容旧配置,实际走 DashscopeBot)
|
||||
QWEN_DASHSCOPE = "dashscope" # 千问 DashScope 接入
|
||||
QWEN = "qwen" # legacy alias, actually routed to DashscopeBot
|
||||
QWEN_DASHSCOPE = "dashscope" # Qwen via DashScope
|
||||
GEMINI = "gemini"
|
||||
ZHIPU_AI = "zhipu"
|
||||
MOONSHOT = "moonshot"
|
||||
MiniMax = "minimax"
|
||||
DEEPSEEK = "deepseek"
|
||||
MIMO = "mimo" # Xiaomi MiMo
|
||||
CUSTOM = "custom" # custom OpenAI-compatible API, bot_type won't auto-switch on model change
|
||||
MODELSCOPE = "modelscope"
|
||||
|
||||
# 模型列表
|
||||
# Model list
|
||||
# Claude (Anthropic)
|
||||
CLAUDE3 = "claude-3-opus-20240229"
|
||||
CLAUDE_3_OPUS = "claude-3-opus-latest"
|
||||
CLAUDE_3_OPUS_0229 = "claude-3-opus-20240229"
|
||||
CLAUDE_3_SONNET = "claude-3-sonnet-20240229"
|
||||
CLAUDE_3_HAIKU = "claude-3-haiku-20240307"
|
||||
CLAUDE_35_SONNET = "claude-3-5-sonnet-latest" # 带 latest 标签的模型名称,会不断更新指向最新发布的模型
|
||||
CLAUDE_35_SONNET_1022 = "claude-3-5-sonnet-20241022" # 带具体日期的模型名称,会固定为该日期发布的模型
|
||||
CLAUDE_35_SONNET = "claude-3-5-sonnet-latest" # "latest" tag always points to the newest release
|
||||
CLAUDE_35_SONNET_1022 = "claude-3-5-sonnet-20241022" # dated name pinned to a specific release
|
||||
CLAUDE_35_SONNET_0620 = "claude-3-5-sonnet-20240620"
|
||||
CLAUDE_4_OPUS = "claude-opus-4-0"
|
||||
CLAUDE_FABLE_5 = "claude-fable-5" # Claude Fable 5 (often restricted by policy)
|
||||
CLAUDE_4_8_OPUS = "claude-opus-4-8" # Claude Opus 4.8 - Agent recommended model
|
||||
CLAUDE_4_7_OPUS = "claude-opus-4-7" # Claude Opus 4.7
|
||||
CLAUDE_4_6_OPUS = "claude-opus-4-6" # Claude Opus 4.6 - Agent推荐模型
|
||||
CLAUDE_4_6_OPUS = "claude-opus-4-6" # Claude Opus 4.6
|
||||
CLAUDE_4_SONNET = "claude-sonnet-4-0" # Claude Sonnet 4.0
|
||||
CLAUDE_4_5_SONNET = "claude-sonnet-4-5" # Claude Sonnet 4.5 - Agent推荐模型
|
||||
CLAUDE_4_6_SONNET = "claude-sonnet-4-6" # Claude Sonnet 4.6 - Agent推荐模型
|
||||
CLAUDE_4_5_SONNET = "claude-sonnet-4-5" # Claude Sonnet 4.5 - Agent recommended model
|
||||
CLAUDE_4_6_SONNET = "claude-sonnet-4-6" # Claude Sonnet 4.6 - Agent recommended model
|
||||
|
||||
# Gemini (Google)
|
||||
GEMINI_PRO = "gemini-1.0-pro"
|
||||
GEMINI_15_flash = "gemini-1.5-flash"
|
||||
GEMINI_15_PRO = "gemini-1.5-pro"
|
||||
GEMINI_20_flash_exp = "gemini-2.0-flash-exp" # exp结尾为实验模型,会逐步不再支持
|
||||
GEMINI_20_FLASH = "gemini-2.0-flash" # 正式版模型
|
||||
GEMINI_20_flash_exp = "gemini-2.0-flash-exp" # "-exp" models are experimental and will be phased out
|
||||
GEMINI_20_FLASH = "gemini-2.0-flash" # stable release
|
||||
GEMINI_25_FLASH_PRE = "gemini-2.5-flash-preview-05-20"
|
||||
GEMINI_25_PRO_PRE = "gemini-2.5-pro-preview-05-06"
|
||||
GEMINI_3_FLASH_PRE = "gemini-3-flash-preview" # Gemini 3 Flash Preview - Agent推荐模型
|
||||
GEMINI_3_FLASH_PRE = "gemini-3-flash-preview" # Gemini 3 Flash Preview - Agent recommended model
|
||||
GEMINI_3_PRO_PRE = "gemini-3-pro-preview" # Gemini 3 Pro Preview
|
||||
GEMINI_31_PRO_PRE = "gemini-3.1-pro-preview" # Gemini 3.1 Pro Preview - Agent推荐模型
|
||||
GEMINI_31_FLASH_LITE_PRE = "gemini-3.1-flash-lite-preview" # Gemini 3.1 Flash Lite Preview - Agent推荐模型
|
||||
GEMINI_35_FLASH = "gemini-3.5-flash" # Gemini 3.5 Flash - Agent推荐模型
|
||||
GEMINI_31_PRO_PRE = "gemini-3.1-pro-preview" # Gemini 3.1 Pro Preview - Agent recommended model
|
||||
GEMINI_31_FLASH_LITE_PRE = "gemini-3.1-flash-lite-preview" # Gemini 3.1 Flash Lite Preview - Agent recommended model
|
||||
GEMINI_35_FLASH = "gemini-3.5-flash" # Gemini 3.5 Flash - Agent recommended model
|
||||
|
||||
# OpenAI
|
||||
GPT35 = "gpt-3.5-turbo"
|
||||
@@ -83,10 +86,10 @@ TTS_1 = "tts-1"
|
||||
TTS_1_HD = "tts-1-hd"
|
||||
|
||||
# DeepSeek
|
||||
DEEPSEEK_CHAT = "deepseek-chat" # DeepSeek-V3对话模型
|
||||
DEEPSEEK_REASONER = "deepseek-reasoner" # DeepSeek-R1模型
|
||||
DEEPSEEK_V4_FLASH = "deepseek-v4-flash" # DeepSeek V4 Flash - 默认推荐 (思考模式 + 工具调用)
|
||||
DEEPSEEK_V4_PRO = "deepseek-v4-pro" # DeepSeek V4 Pro - 复杂任务更强 (思考模式 + 工具调用)
|
||||
DEEPSEEK_CHAT = "deepseek-chat" # DeepSeek-V3 chat model
|
||||
DEEPSEEK_REASONER = "deepseek-reasoner" # DeepSeek-R1 model
|
||||
DEEPSEEK_V4_FLASH = "deepseek-v4-flash" # DeepSeek V4 Flash - default recommendation (thinking + tool calls)
|
||||
DEEPSEEK_V4_PRO = "deepseek-v4-pro" # DeepSeek V4 Pro - stronger on complex tasks (thinking + tool calls)
|
||||
|
||||
# Baidu Qianfan / ERNIE
|
||||
ERNIE_5_1 = "ernie-5.1" # ERNIE 5.1 - default recommendation, latest flagship
|
||||
@@ -98,32 +101,31 @@ ERNIE_4_TURBO_8K = "ERNIE-4.0-Turbo-8K"
|
||||
ERNIE_45_TURBO_VL = "ernie-4.5-turbo-vl"
|
||||
ERNIE_45_TURBO_VL_32K = "ernie-4.5-turbo-vl-32k"
|
||||
|
||||
# Qwen (通义千问 - 阿里云 DashScope)
|
||||
# Qwen (Alibaba Cloud DashScope)
|
||||
QWEN_TURBO = "qwen-turbo"
|
||||
QWEN_PLUS = "qwen-plus"
|
||||
QWEN_MAX = "qwen-max"
|
||||
QWEN_LONG = "qwen-long"
|
||||
QWEN3_MAX = "qwen3-max" # Qwen3 Max - Agent推荐模型
|
||||
QWEN3_MAX = "qwen3-max" # Qwen3 Max - Agent recommended model
|
||||
QWEN35_PLUS = "qwen3.5-plus" # Qwen3.5 Plus - Omni model (MultiModalConversation)
|
||||
QWEN36_PLUS = "qwen3.6-plus" # Qwen3.6 Plus - Omni model (MultiModalConversation)
|
||||
QWEN37_MAX = "qwen3.7-max" # Qwen3.7 Max - Agent推荐模型
|
||||
QWEN37_PLUS = "qwen3.7-plus" # Qwen3.7 Plus - Omni model (MultiModalConversation)
|
||||
QWEN37_MAX = "qwen3.7-max" # Qwen3.7 Max - Agent recommended model
|
||||
QWQ_PLUS = "qwq-plus"
|
||||
|
||||
# MiniMax
|
||||
MINIMAX_M2_7 = "MiniMax-M2.7" # MiniMax M2.7 - Latest
|
||||
MINIMAX_TEXT_01 = "MiniMax-Text-01" # MiniMax 多模态 (vision)
|
||||
MINIMAX_M3 = "MiniMax-M3" # MiniMax M3 - Latest (default)
|
||||
MINIMAX_M2_7 = "MiniMax-M2.7" # MiniMax M2.7
|
||||
MINIMAX_M2_7_HIGHSPEED = "MiniMax-M2.7-highspeed" # MiniMax M2.7 highspeed
|
||||
MINIMAX_M2_5 = "MiniMax-M2.5" # MiniMax M2.5
|
||||
MINIMAX_M2_1 = "MiniMax-M2.1" # MiniMax M2.1
|
||||
MINIMAX_M2_1_LIGHTNING = "MiniMax-M2.1-lightning" # MiniMax M2.1 极速版
|
||||
MINIMAX_M2 = "MiniMax-M2" # MiniMax M2
|
||||
MINIMAX_TEXT_01 = "MiniMax-Text-01" # MiniMax multimodal (vision)
|
||||
MINIMAX_ABAB6_5 = "abab6.5-chat" # MiniMax abab6.5
|
||||
|
||||
# GLM (智谱AI)
|
||||
GLM_5_1 = "glm-5.1" # 智谱 GLM-5.1 - Agent recommended model (default)
|
||||
GLM_5_TURBO = "glm-5-turbo" # 智谱 GLM-5-Turbo
|
||||
GLM_5 = "glm-5" # 智谱 GLM-5
|
||||
GLM_5V_TURBO = "glm-5v-turbo" # 智谱多模态 (vision)
|
||||
# GLM (Zhipu AI)
|
||||
GLM_5_2 = "glm-5.2" # GLM-5.2 - Agent recommended model (default)
|
||||
GLM_5_1 = "glm-5.1" # GLM-5.1
|
||||
GLM_5_TURBO = "glm-5-turbo" # GLM-5-Turbo
|
||||
GLM_5 = "glm-5" # GLM-5
|
||||
GLM_5V_TURBO = "glm-5v-turbo" # Zhipu multimodal (vision)
|
||||
GLM_4 = "glm-4"
|
||||
GLM_4_PLUS = "glm-4-plus"
|
||||
GLM_4_flash = "glm-4-flash"
|
||||
@@ -132,13 +134,22 @@ GLM_4_ALLTOOLS = "glm-4-alltools"
|
||||
GLM_4_0520 = "glm-4-0520"
|
||||
GLM_4_AIR = "glm-4-air"
|
||||
GLM_4_AIRX = "glm-4-airx"
|
||||
GLM_4_7 = "glm-4.7" # 智谱 GLM-4.7 - Agent推荐模型
|
||||
GLM_4_7 = "glm-4.7" # GLM-4.7 - Agent recommended model
|
||||
|
||||
# Kimi (Moonshot)
|
||||
MOONSHOT = "moonshot"
|
||||
KIMI_K2_7_CODE = "kimi-k2.7-code" # Kimi K2.7 Code - Agent recommended model (default)
|
||||
KIMI_K2_7_CODE_HIGHSPEED = "kimi-k2.7-code-highspeed" # Kimi K2.7 Code highspeed
|
||||
KIMI_K2 = "kimi-k2"
|
||||
KIMI_K2_5 = "kimi-k2.5"
|
||||
KIMI_K2_6 = "kimi-k2.6" # Kimi K2.6 - Agent recommended model (default)
|
||||
KIMI_K2_6 = "kimi-k2.6"
|
||||
|
||||
# Xiaomi MiMo
|
||||
MIMO_V2_5_PRO = "mimo-v2.5-pro" # MiMo V2.5 Pro - flagship, long context (default recommendation)
|
||||
MIMO_V2_5 = "mimo-v2.5" # MiMo V2.5 - multimodal (text/image/audio/video)
|
||||
MIMO_V2_PRO = "mimo-v2-pro" # MiMo V2 Pro
|
||||
MIMO_V2_OMNI = "mimo-v2-omni" # MiMo V2 Omni - multimodal
|
||||
MIMO_V2_FLASH = "mimo-v2-flash" # MiMo V2 Flash - high-speed
|
||||
|
||||
# Doubao (Volcengine Ark)
|
||||
DOUBAO = "doubao"
|
||||
@@ -147,11 +158,11 @@ DOUBAO_SEED_2_PRO = "doubao-seed-2-0-pro-260215"
|
||||
DOUBAO_SEED_2_LITE = "doubao-seed-2-0-lite-260215"
|
||||
DOUBAO_SEED_2_MINI = "doubao-seed-2-0-mini-260215"
|
||||
|
||||
# ModelScope(魔搭社区)
|
||||
# ModelScope
|
||||
QWEN3_235B_A22B_INSTRUCT_2507 = "Qwen/Qwen3-235B-A22B-Instruct-2507"
|
||||
QWEN3_5_27B = "Qwen/Qwen3.5-27B"
|
||||
|
||||
# 其他模型
|
||||
# Other models
|
||||
WEN_XIN = "wenxin"
|
||||
WEN_XIN_4 = "wenxin-4"
|
||||
XUNFEI = "xunfei"
|
||||
@@ -180,10 +191,13 @@ MODEL_LIST = [
|
||||
ERNIE_45_TURBO_VL, ERNIE_45_TURBO_VL_32K,
|
||||
|
||||
# MiniMax
|
||||
MiniMax, MINIMAX_M2_7, MINIMAX_M2_7_HIGHSPEED, MINIMAX_M2_5, MINIMAX_M2_1, MINIMAX_M2_1_LIGHTNING, MINIMAX_M2, MINIMAX_ABAB6_5,
|
||||
MiniMax, MINIMAX_M3, MINIMAX_M2_7, MINIMAX_M2_7_HIGHSPEED, MINIMAX_ABAB6_5,
|
||||
|
||||
# Xiaomi MiMo
|
||||
MIMO, MIMO_V2_5_PRO, MIMO_V2_5, MIMO_V2_PRO, MIMO_V2_OMNI, MIMO_V2_FLASH,
|
||||
|
||||
# Claude
|
||||
CLAUDE3, CLAUDE_4_6_SONNET, CLAUDE_4_7_OPUS, CLAUDE_4_6_OPUS, CLAUDE_4_OPUS, CLAUDE_4_5_SONNET, CLAUDE_4_SONNET, CLAUDE_3_OPUS, CLAUDE_3_OPUS_0229,
|
||||
CLAUDE3, CLAUDE_4_8_OPUS, CLAUDE_4_7_OPUS, CLAUDE_FABLE_5, CLAUDE_4_6_SONNET, CLAUDE_4_6_OPUS, CLAUDE_4_OPUS, CLAUDE_4_5_SONNET, CLAUDE_4_SONNET, CLAUDE_3_OPUS, CLAUDE_3_OPUS_0229,
|
||||
CLAUDE_35_SONNET, CLAUDE_35_SONNET_1022, CLAUDE_35_SONNET_0620, CLAUDE_3_SONNET, CLAUDE_3_HAIKU,
|
||||
"claude", "claude-3-haiku", "claude-3-sonnet", "claude-3-opus", "claude-3.5-sonnet",
|
||||
|
||||
@@ -201,19 +215,19 @@ MODEL_LIST = [
|
||||
GPT_54, GPT_55, GPT_54_MINI, GPT_54_NANO,
|
||||
O1, O1_MINI,
|
||||
|
||||
# GLM (智谱AI)
|
||||
ZHIPU_AI, GLM_5_1, GLM_5_TURBO, GLM_5, GLM_4, GLM_4_PLUS, GLM_4_flash, GLM_4_LONG, GLM_4_ALLTOOLS,
|
||||
# GLM (Zhipu AI)
|
||||
ZHIPU_AI, GLM_5_2, GLM_5_1, GLM_5_TURBO, GLM_5, GLM_4, GLM_4_PLUS, GLM_4_flash, GLM_4_LONG, GLM_4_ALLTOOLS,
|
||||
GLM_4_0520, GLM_4_AIR, GLM_4_AIRX, GLM_4_7,
|
||||
|
||||
# Qwen (通义千问)
|
||||
QWEN37_MAX, QWEN36_PLUS, QWEN35_PLUS, QWEN3_MAX, QWEN_MAX, QWEN_PLUS, QWEN_TURBO, QWEN_LONG,
|
||||
# Qwen
|
||||
QWEN37_PLUS, QWEN37_MAX, QWEN36_PLUS, QWEN35_PLUS, QWEN3_MAX, QWEN_MAX, QWEN_PLUS, QWEN_TURBO, QWEN_LONG,
|
||||
|
||||
# Doubao (豆包)
|
||||
# Doubao
|
||||
DOUBAO, DOUBAO_SEED_2_CODE, DOUBAO_SEED_2_PRO, DOUBAO_SEED_2_LITE, DOUBAO_SEED_2_MINI,
|
||||
|
||||
# Kimi (Moonshot)
|
||||
MOONSHOT, "moonshot-v1-8k", "moonshot-v1-32k", "moonshot-v1-128k",
|
||||
KIMI_K2_6, KIMI_K2_5, KIMI_K2,
|
||||
KIMI_K2_7_CODE, KIMI_K2_7_CODE_HIGHSPEED, KIMI_K2_6, KIMI_K2_5, KIMI_K2,
|
||||
|
||||
# ModelScope
|
||||
MODELSCOPE,
|
||||
@@ -221,14 +235,19 @@ MODEL_LIST = [
|
||||
# LinkAI
|
||||
LINKAI_35, LINKAI_4_TURBO, LINKAI_4o,
|
||||
|
||||
# 其他模型
|
||||
# Other models
|
||||
WEN_XIN, WEN_XIN_4, XUNFEI,
|
||||
]
|
||||
|
||||
MODEL_LIST = MODEL_LIST + GITEE_AI_MODEL_LIST + MODELSCOPE_MODEL_LIST
|
||||
|
||||
# channel
|
||||
FEISHU = "feishu"
|
||||
DINGTALK = "dingtalk"
|
||||
WECOM_BOT = "wecom_bot"
|
||||
QQ = "qq"
|
||||
WEIXIN = "weixin"
|
||||
WECHAT_KF = "wechat_kf"
|
||||
TELEGRAM = "telegram"
|
||||
SLACK = "slack"
|
||||
DISCORD = "discord"
|
||||
|
||||
179
common/i18n.py
Normal file
179
common/i18n.py
Normal file
@@ -0,0 +1,179 @@
|
||||
# encoding:utf-8
|
||||
|
||||
"""Lightweight global language detection and resolution.
|
||||
|
||||
This module is the single source of truth for the runtime UI language used
|
||||
across the CLI, startup logs, error messages, agent prompts and channel
|
||||
replies. It must NOT import project config (to avoid circular imports) and
|
||||
must stay dependency-free so it can run at the earliest startup phase.
|
||||
|
||||
Resolution priority (highest first):
|
||||
1. Explicit `cow_lang` from config.json — also covers Docker/CI, since any
|
||||
config key is overridable via its uppercase env var (e.g. COW_LANG=zh),
|
||||
handled by config.load_config() before resolution. COW_LANG is a private
|
||||
name to avoid clashing with the gettext-standard LANGUAGE variable.
|
||||
2. macOS `defaults read -g AppleLocale` (system-level preference; a Chinese
|
||||
system locale is a strong signal that beats a shell-default LANG)
|
||||
3. Standard locale env vars: LC_ALL > LC_MESSAGES > LANG
|
||||
4. Python locale module
|
||||
5. Default -> English
|
||||
|
||||
A value of "auto" (the default) triggers detection (steps 2-5). Explicitly
|
||||
setting "zh" or "en" locks the language and skips detection.
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
# Supported language codes
|
||||
ZH = "zh"
|
||||
EN = "en"
|
||||
SUPPORTED = (ZH, EN)
|
||||
DEFAULT_LANG = EN
|
||||
|
||||
# Resolved language cache; None until first resolution.
|
||||
_resolved_lang = None
|
||||
|
||||
|
||||
def _normalize(raw):
|
||||
"""Map an arbitrary locale-ish string to a supported code, or None.
|
||||
|
||||
Only Chinese is detected explicitly; everything else (including unknown
|
||||
or empty values) yields None so the caller can fall through to the next
|
||||
detection source.
|
||||
"""
|
||||
if not raw:
|
||||
return None
|
||||
value = str(raw).strip().lower().replace("_", "-")
|
||||
if value in ("auto", ""):
|
||||
return None
|
||||
# Chinese variants: zh, zh-cn, zh-hans, zh-hans-cn, zh-tw, zh-hk ...
|
||||
if value.startswith("zh") or value.startswith("chinese"):
|
||||
return ZH
|
||||
if value.startswith("en") or value.startswith("english"):
|
||||
return EN
|
||||
return None
|
||||
|
||||
|
||||
def _detect_from_env():
|
||||
"""Detect language from standard locale environment variables.
|
||||
|
||||
Note: on macOS, `LANG` is often a shell default (e.g. en_US.UTF-8 set by
|
||||
.zshrc) that does not reflect the user's real preference, so AppleLocale
|
||||
is checked first (see detect_language). On Linux these vars are the
|
||||
primary signal.
|
||||
|
||||
The cow_lang env override (COW_LANG=zh) is intentionally NOT read here:
|
||||
it sets config["cow_lang"] and is handled via the explicit config path,
|
||||
not auto-detection.
|
||||
"""
|
||||
for key in ("LC_ALL", "LC_MESSAGES", "LANG"):
|
||||
lang = _normalize(os.environ.get(key))
|
||||
if lang:
|
||||
return lang
|
||||
return None
|
||||
|
||||
|
||||
def _detect_from_macos():
|
||||
"""macOS fallback: read the system-wide AppleLocale preference.
|
||||
|
||||
On macOS the terminal often does NOT export LANG, yet the system locale
|
||||
is still meaningful (e.g. a Chinese Mac reports zh_CN). This recovers
|
||||
that signal so Chinese users are not misdetected as English.
|
||||
"""
|
||||
if sys.platform != "darwin":
|
||||
return None
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["defaults", "read", "-g", "AppleLocale"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=2,
|
||||
)
|
||||
if out.returncode == 0:
|
||||
return _normalize(out.stdout)
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def _detect_from_python_locale():
|
||||
"""Last-resort detection via Python's locale module."""
|
||||
try:
|
||||
import locale
|
||||
|
||||
for value in locale.getlocale():
|
||||
lang = _normalize(value)
|
||||
if lang:
|
||||
return lang
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def detect_language():
|
||||
"""Run full auto-detection and return a supported language code.
|
||||
|
||||
Order (auto-detection only; explicit config["cow_lang"] is resolved
|
||||
before this is reached):
|
||||
1. macOS AppleLocale (system-level preference; a Chinese system locale
|
||||
is a strong, low-false-positive signal that beats a shell-default
|
||||
LANG like en_US.UTF-8)
|
||||
2. locale env vars LC_ALL / LC_MESSAGES / LANG (primary signal on Linux)
|
||||
3. Python locale module
|
||||
4. default English
|
||||
"""
|
||||
if os.environ.get("CLOUD_DEPLOYMENT_ID"):
|
||||
return ZH
|
||||
return (
|
||||
_detect_from_macos()
|
||||
or _detect_from_env()
|
||||
or _detect_from_python_locale()
|
||||
or DEFAULT_LANG
|
||||
)
|
||||
|
||||
|
||||
def resolve_language(configured=None):
|
||||
"""Resolve the effective language from a configured value.
|
||||
|
||||
`configured` is the raw `cow_lang` value from config.json (may be None,
|
||||
"auto", "zh" or "en"). An explicit "zh"/"en" locks the result; "auto"
|
||||
or empty triggers detection. The result is cached globally.
|
||||
"""
|
||||
global _resolved_lang
|
||||
explicit = _normalize(configured)
|
||||
if explicit:
|
||||
_resolved_lang = explicit
|
||||
else:
|
||||
_resolved_lang = detect_language()
|
||||
return _resolved_lang
|
||||
|
||||
|
||||
def set_language(lang):
|
||||
"""Force the resolved language (used by tests or per-request overrides)."""
|
||||
global _resolved_lang
|
||||
normalized = _normalize(lang)
|
||||
_resolved_lang = normalized or DEFAULT_LANG
|
||||
return _resolved_lang
|
||||
|
||||
|
||||
def get_language():
|
||||
"""Return the currently resolved language, detecting lazily if needed."""
|
||||
global _resolved_lang
|
||||
if _resolved_lang is None:
|
||||
_resolved_lang = detect_language()
|
||||
return _resolved_lang
|
||||
|
||||
|
||||
def is_zh():
|
||||
return get_language() == ZH
|
||||
|
||||
|
||||
def t(zh_text, en_text):
|
||||
"""Pick a string by the current language. Tiny inline-translation helper.
|
||||
|
||||
Intended for one-off strings where a full message catalog is overkill:
|
||||
t("已中止", "Cancelled")
|
||||
"""
|
||||
return zh_text if get_language() == ZH else en_text
|
||||
@@ -1,8 +1,21 @@
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import io
|
||||
|
||||
|
||||
def _log_path():
|
||||
# Mirror config.get_data_root() without importing config (avoids a circular
|
||||
# import, since config imports this module). The desktop build sets
|
||||
# COW_DATA_DIR (e.g. ~/.cow); source deployments fall back to CWD.
|
||||
data_dir = os.environ.get("COW_DATA_DIR")
|
||||
if data_dir:
|
||||
data_dir = os.path.expanduser(data_dir)
|
||||
os.makedirs(data_dir, exist_ok=True)
|
||||
return os.path.join(data_dir, "run.log")
|
||||
return "run.log"
|
||||
|
||||
|
||||
def _reset_logger(log):
|
||||
for handler in log.handlers:
|
||||
handler.close()
|
||||
@@ -20,7 +33,7 @@ def _reset_logger(log):
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
)
|
||||
file_handle = logging.FileHandler("run.log", encoding="utf-8")
|
||||
file_handle = logging.FileHandler(_log_path(), encoding="utf-8")
|
||||
file_handle.setFormatter(
|
||||
logging.Formatter(
|
||||
"[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d] - %(message)s",
|
||||
|
||||
@@ -1,18 +1,21 @@
|
||||
import os
|
||||
import pathlib
|
||||
|
||||
from common.utils import expand_path
|
||||
from config import conf
|
||||
|
||||
|
||||
class TmpDir(object):
|
||||
"""A temporary directory that is deleted when the object is destroyed."""
|
||||
"""Temporary directory for transient artifacts (e.g. synthesized voice).
|
||||
|
||||
tmpFilePath = pathlib.Path("./tmp/")
|
||||
Resolves to ``<agent_workspace>/tmp`` (default ``~/cow/tmp``) so temp files
|
||||
land inside the agent workspace instead of a CWD-relative ``./tmp``, which
|
||||
is unreliable for the packaged desktop app where CWD is undefined.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
pathExists = os.path.exists(self.tmpFilePath)
|
||||
if not pathExists:
|
||||
os.makedirs(self.tmpFilePath)
|
||||
ws_root = expand_path(conf().get("agent_workspace", "~/cow"))
|
||||
self.tmpFilePath = os.path.join(ws_root, "tmp")
|
||||
os.makedirs(self.tmpFilePath, exist_ok=True)
|
||||
|
||||
def path(self):
|
||||
return str(self.tmpFilePath) + "/"
|
||||
|
||||
@@ -27,10 +27,14 @@ def compress_imgfile(file, max_size):
|
||||
img = Image.open(file)
|
||||
rgb_image = img.convert("RGB")
|
||||
quality = 95
|
||||
min_quality = 10
|
||||
while True:
|
||||
out_buf = io.BytesIO()
|
||||
rgb_image.save(out_buf, "JPEG", quality=quality)
|
||||
if fsize(out_buf) <= max_size:
|
||||
if fsize(out_buf) <= max_size or quality <= min_quality:
|
||||
# Stop at min_quality: further decrements would pass an invalid
|
||||
# quality (<1) to PIL and the loop would otherwise never terminate
|
||||
# for images that cannot be compressed below max_size.
|
||||
return out_buf
|
||||
quality -= 5
|
||||
|
||||
@@ -117,6 +121,18 @@ def expand_path(path: str) -> str:
|
||||
return expanded
|
||||
|
||||
|
||||
def is_cloud_deployment() -> bool:
|
||||
if os.environ.get("CLOUD_DEPLOYMENT_ID"):
|
||||
return True
|
||||
try:
|
||||
from config import conf
|
||||
if conf().get("cloud_deployment_id"):
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def get_cloud_headers(api_key: str) -> dict:
|
||||
"""
|
||||
Build standard headers for LinkAI API requests,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"channel_type": "weixin",
|
||||
"cow_lang": "auto",
|
||||
"channel_type": "web",
|
||||
"model": "deepseek-v4-flash",
|
||||
"deepseek_api_key": "",
|
||||
"deepseek_api_base": "https://api.deepseek.com/v1",
|
||||
@@ -39,5 +40,6 @@
|
||||
"agent_max_steps": 20,
|
||||
"enable_thinking": false,
|
||||
"reasoning_effort": "high",
|
||||
"knowledge": true
|
||||
"knowledge": true,
|
||||
"self_evolution_enabled": true
|
||||
}
|
||||
|
||||
476
config.py
476
config.py
@@ -5,198 +5,233 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import pickle
|
||||
import sys
|
||||
|
||||
from common.log import logger
|
||||
from common import i18n
|
||||
|
||||
# 将所有可用的配置项写在字典里, 请使用小写字母
|
||||
# 此处的配置值无实际意义,程序不会读取此处的配置,仅用于提示格式,请将配置加入到config.json中
|
||||
# All available config keys are listed in this dict (use lowercase keys).
|
||||
# The values here are placeholders only; the program does NOT read them.
|
||||
# They merely document the expected format — put real values in config.json.
|
||||
available_setting = {
|
||||
# openai api配置
|
||||
# global UI language for CLI, startup logs, error messages, agent prompts
|
||||
# and channel replies. Options: "auto" (detect from system locale, default),
|
||||
# "zh" (Chinese) or "en" (English). An explicit value locks the language.
|
||||
# value: auto/en/zh
|
||||
"cow_lang": "auto",
|
||||
# openai api config
|
||||
"open_ai_api_key": "", # openai api key
|
||||
# openai apibase,当use_azure_chatgpt为true时,需要设置对应的api base
|
||||
# openai api base; when use_azure_chatgpt is true, set the matching api base
|
||||
"open_ai_api_base": "https://api.openai.com/v1",
|
||||
"claude_api_base": "https://api.anthropic.com/v1", # claude api base
|
||||
"gemini_api_base": "https://generativelanguage.googleapis.com", # gemini api base
|
||||
"custom_api_key": "", # custom OpenAI-compatible provider api key (used when bot_type is "custom")
|
||||
"custom_api_base": "", # custom OpenAI-compatible provider api base (used when bot_type is "custom")
|
||||
"proxy": "", # openai使用的代理
|
||||
# chatgpt模型, 当use_azure_chatgpt为true时,其名称为Azure上model deployment名称
|
||||
"model": "gpt-3.5-turbo", # 可选择: gpt-4o, pt-4o-mini, gpt-4-turbo, claude-3-sonnet, wenxin, moonshot, qwen-turbo, xunfei, glm-4, minimax, gemini等模型,全部可选模型详见common/const.py文件
|
||||
"bot_type": "", # 可选配置,使用兼容openai格式的三方服务时候,需填"openai"或"custom"(custom模式下切换模型不会自动切换bot_type)。bot具体名称详见common/const.py文件,如不填根据model名称判断
|
||||
"use_azure_chatgpt": False, # 是否使用azure的chatgpt
|
||||
"azure_deployment_id": "", # azure 模型部署名称
|
||||
"azure_api_version": "", # azure api版本
|
||||
# Bot触发配置
|
||||
"single_chat_prefix": ["bot", "@bot"], # 私聊时文本需要包含该前缀才能触发机器人回复
|
||||
"single_chat_reply_prefix": "[bot] ", # 私聊时自动回复的前缀,用于区分真人
|
||||
"single_chat_reply_suffix": "", # 私聊时自动回复的后缀,\n 可以换行
|
||||
"group_chat_prefix": ["@bot"], # 群聊时包含该前缀则会触发机器人回复
|
||||
"no_need_at": False, # 群聊回复时是否不需要艾特
|
||||
"group_chat_reply_prefix": "", # 群聊时自动回复的前缀
|
||||
"group_chat_reply_suffix": "", # 群聊时自动回复的后缀,\n 可以换行
|
||||
"group_chat_keyword": [], # 群聊时包含该关键词则会触发机器人回复
|
||||
"group_at_off": False, # 是否关闭群聊时@bot的触发
|
||||
"group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"], # 开启自动回复的群名称列表
|
||||
"group_name_keyword_white_list": [], # 开启自动回复的群名称关键词列表
|
||||
"group_chat_in_one_session": ["ChatGPT测试群"], # 支持会话上下文共享的群名称
|
||||
"group_shared_session": False, # 群聊是否共享会话上下文(所有成员共享)。False时每个用户在群内有独立会话
|
||||
"nick_name_black_list": [], # 用户昵称黑名单
|
||||
"group_welcome_msg": "", # 配置新人进群固定欢迎语,不配置则使用随机风格欢迎
|
||||
"trigger_by_self": False, # 是否允许机器人触发
|
||||
"text_to_image": "dall-e-2", # 图片生成模型,可选 dall-e-2, dall-e-3
|
||||
# Azure OpenAI dall-e-3 配置
|
||||
"dalle3_image_style": "vivid", # 图片生成dalle3的风格,可选有 vivid, natural
|
||||
"dalle3_image_quality": "hd", # 图片生成dalle3的质量,可选有 standard, hd
|
||||
# Azure OpenAI DALL-E API 配置, 当use_azure_chatgpt为true时,用于将文字回复的资源和Dall-E的资源分开.
|
||||
"azure_openai_dalle_api_base": "", # [可选] azure openai 用于回复图片的资源 endpoint,默认使用 open_ai_api_base
|
||||
"azure_openai_dalle_api_key": "", # [可选] azure openai 用于回复图片的资源 key,默认使用 open_ai_api_key
|
||||
"azure_openai_dalle_deployment_id":"", # [可选] azure openai 用于回复图片的资源 deployment id,默认使用 text_to_image
|
||||
"image_proxy": True, # 是否需要图片代理,国内访问LinkAI时需要
|
||||
"image_create_prefix": ["画", "看", "找"], # 开启图片回复的前缀
|
||||
"concurrency_in_session": 1, # 同一会话最多有多少条消息在处理中,大于1可能乱序
|
||||
"image_create_size": "256x256", # 图片大小,可选有 256x256, 512x512, 1024x1024 (dall-e-3默认为1024x1024)
|
||||
"custom_api_key": "", # custom OpenAI-compatible provider api key (used when bot_type is "custom"); legacy single-provider field
|
||||
"custom_api_base": "", # custom OpenAI-compatible provider api base (used when bot_type is "custom"); legacy single-provider field
|
||||
# Multiple custom (OpenAI-compatible) providers. Activated via bot_type: "custom:<id>".
|
||||
# Each item: {"id": "3f2a9c1b", "name": "my-provider", "api_key": "sk-...", "api_base": "https://api.example.com/v1", "model": "model-name"}
|
||||
"custom_providers": [],
|
||||
"proxy": "", # proxy used by openai
|
||||
# chatgpt model; when use_azure_chatgpt is true, this is the Azure model deployment name
|
||||
"model": "gpt-3.5-turbo", # options: gpt-4o, gpt-4o-mini, gpt-4-turbo, claude-3-sonnet, wenxin, moonshot, qwen-turbo, xunfei, glm-4, minimax, gemini, etc. See common/const.py for the full list
|
||||
"bot_type": "", # optional; for OpenAI-compatible third-party services set "openai" or "custom" (in custom mode switching model won't auto-switch bot_type). See common/const.py for bot names; inferred from model name if left empty
|
||||
"use_azure_chatgpt": False, # whether to use Azure chatgpt
|
||||
"azure_deployment_id": "", # azure model deployment name
|
||||
"azure_api_version": "", # azure api version
|
||||
# Bot trigger config
|
||||
"single_chat_prefix": ["bot", "@bot"], # text must contain this prefix to trigger a reply in single chat
|
||||
"single_chat_reply_prefix": "[bot] ", # auto-reply prefix in single chat, used to distinguish from a real person
|
||||
"single_chat_reply_suffix": "", # auto-reply suffix in single chat; \n inserts a line break
|
||||
"group_chat_prefix": ["@bot"], # messages containing this prefix trigger a reply in group chat
|
||||
"no_need_at": False, # whether replying in group chat does not require an @mention
|
||||
"group_chat_reply_prefix": "", # auto-reply prefix in group chat
|
||||
"group_chat_reply_suffix": "", # auto-reply suffix in group chat; \n inserts a line break
|
||||
"group_chat_keyword": [], # messages containing this keyword trigger a reply in group chat
|
||||
"group_at_off": False, # whether to disable @bot triggering in group chat
|
||||
"group_name_white_list": ["group1", "group2"], # group names where auto-reply is enabled
|
||||
"group_name_keyword_white_list": [], # group-name keywords where auto-reply is enabled
|
||||
"group_chat_in_one_session": ["group1"], # group names that share conversation context
|
||||
"group_shared_session": False, # whether group chat shares conversation context (all members share). When False each user has an independent session in the group
|
||||
"nick_name_black_list": [], # user nickname blacklist
|
||||
"group_welcome_msg": "", # fixed welcome message for new group members; uses a random style when empty
|
||||
"trigger_by_self": False, # whether the bot can be triggered by itself
|
||||
"text_to_image": "dall-e-2", # image generation model, options: dall-e-2, dall-e-3
|
||||
# Azure OpenAI dall-e-3 config
|
||||
"dalle3_image_style": "vivid", # dalle3 image style, options: vivid, natural
|
||||
"dalle3_image_quality": "hd", # dalle3 image quality, options: standard, hd
|
||||
# Azure OpenAI DALL-E API config; when use_azure_chatgpt is true, separates the text-reply resource from the DALL-E resource
|
||||
"azure_openai_dalle_api_base": "", # [optional] azure openai endpoint for image replies; defaults to open_ai_api_base
|
||||
"azure_openai_dalle_api_key": "", # [optional] azure openai key for image replies; defaults to open_ai_api_key
|
||||
"azure_openai_dalle_deployment_id":"", # [optional] azure openai deployment id for image replies; defaults to text_to_image
|
||||
"image_proxy": True, # whether an image proxy is needed; required when accessing LinkAI from mainland China
|
||||
"image_create_prefix": ["画", "看", "找"], # prefixes that enable image replies
|
||||
"concurrency_in_session": 1, # max number of in-flight messages per session; values >1 may cause out-of-order replies
|
||||
"image_create_size": "256x256", # image size, options: 256x256, 512x512, 1024x1024 (dall-e-3 defaults to 1024x1024)
|
||||
"group_chat_exit_group": False,
|
||||
# chatgpt会话参数
|
||||
"expires_in_seconds": 3600, # 无操作会话的过期时间
|
||||
# 人格描述
|
||||
"character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。",
|
||||
"conversation_max_tokens": 1000, # 支持上下文记忆的最多字符数
|
||||
# chatgpt限流配置
|
||||
"rate_limit_chatgpt": 20, # chatgpt的调用频率限制
|
||||
"rate_limit_dalle": 50, # openai dalle的调用频率限制
|
||||
# chatgpt api参数 参考https://platform.openai.com/docs/api-reference/chat/create
|
||||
# chatgpt session params
|
||||
"expires_in_seconds": 3600, # idle session expiry time
|
||||
# persona description (only used in chat mode)
|
||||
"character_desc": "You are a helpful AI assistant. You aim to answer and solve any questions people have, and can communicate in multiple languages.",
|
||||
"conversation_max_tokens": 1000, # max characters of context memory
|
||||
# chatgpt rate limit config
|
||||
"rate_limit_chatgpt": 20, # chatgpt call rate limit
|
||||
"rate_limit_dalle": 50, # openai dalle call rate limit
|
||||
# chatgpt api params, see https://platform.openai.com/docs/api-reference/chat/create
|
||||
"temperature": 0.9,
|
||||
"top_p": 1,
|
||||
"frequency_penalty": 0,
|
||||
"presence_penalty": 0,
|
||||
"request_timeout": 180, # chatgpt请求超时时间,openai接口默认设置为600,对于难问题一般需要较长时间
|
||||
"timeout": 120, # chatgpt重试超时时间,在这个时间内,将会自动重试
|
||||
# Baidu 文心一言参数
|
||||
"baidu_wenxin_model": "eb-instant", # 默认使用ERNIE-Bot-turbo模型
|
||||
"request_timeout": 180, # chatgpt request timeout; the openai api defaults to 600, hard questions usually need longer
|
||||
"timeout": 120, # chatgpt retry timeout; will auto-retry within this window
|
||||
# Baidu Wenxin (ERNIE) params
|
||||
"baidu_wenxin_model": "eb-instant", # defaults to the ERNIE-Bot-turbo model
|
||||
"baidu_wenxin_api_key": "", # Baidu api key
|
||||
"baidu_wenxin_secret_key": "", # Baidu secret key
|
||||
"baidu_wenxin_prompt_enabled": False, # Enable prompt if you are using ernie character model
|
||||
# Baidu Qianfan / ERNIE OpenAI-compatible API
|
||||
"qianfan_api_key": "", # Baidu Qianfan API key in bce-v3 format
|
||||
"qianfan_api_base": "https://qianfan.baidubce.com/v2", # Qianfan OpenAI-compatible API base
|
||||
# 讯飞星火API
|
||||
"xunfei_app_id": "", # 讯飞应用ID
|
||||
"xunfei_api_key": "", # 讯飞 API key
|
||||
"xunfei_api_secret": "", # 讯飞 API secret
|
||||
"xunfei_domain": "", # 讯飞模型对应的domain参数,Spark4.0 Ultra为 4.0Ultra,其他模型详见: https://www.xfyun.cn/doc/spark/Web.html
|
||||
"xunfei_spark_url": "", # 讯飞模型对应的请求地址,Spark4.0 Ultra为 wss://spark-api.xf-yun.com/v4.0/chat,其他模型参考详见: https://www.xfyun.cn/doc/spark/Web.html
|
||||
# claude 配置
|
||||
# Xunfei Spark API
|
||||
"xunfei_app_id": "", # Xunfei app id
|
||||
"xunfei_api_key": "", # Xunfei API key
|
||||
"xunfei_api_secret": "", # Xunfei API secret
|
||||
"xunfei_domain": "", # Xunfei model domain param; for Spark4.0 Ultra it is 4.0Ultra, see https://www.xfyun.cn/doc/spark/Web.html for others
|
||||
"xunfei_spark_url": "", # Xunfei model request url; for Spark4.0 Ultra it is wss://spark-api.xf-yun.com/v4.0/chat, see https://www.xfyun.cn/doc/spark/Web.html for others
|
||||
# claude config
|
||||
"claude_api_cookie": "",
|
||||
"claude_uuid": "",
|
||||
# claude api key
|
||||
"claude_api_key": "",
|
||||
# 通义千问API, 获取方式查看文档 https://help.aliyun.com/document_detail/2587494.html
|
||||
# Tongyi Qianwen API, see https://help.aliyun.com/document_detail/2587494.html for how to obtain
|
||||
"qwen_access_key_id": "",
|
||||
"qwen_access_key_secret": "",
|
||||
"qwen_agent_key": "",
|
||||
"qwen_app_id": "",
|
||||
"qwen_node_id": "", # 流程编排模型用到的id,如果没有用到qwen_node_id,请务必保持为空字符串
|
||||
# 阿里灵积(通义新版sdk)模型api key
|
||||
"qwen_node_id": "", # id used by workflow-orchestration models; keep it an empty string if qwen_node_id is unused
|
||||
# Alibaba Lingji (Tongyi new sdk) model api key
|
||||
"dashscope_api_key": "",
|
||||
# Google Gemini Api Key
|
||||
"gemini_api_key": "",
|
||||
# Embedding 模型设置
|
||||
"embedding_provider": "", # 显式指定厂商:openai / linkai / dashscope / doubao / zhipu (与 bot_type 命名一致)
|
||||
"embedding_model": "", # 留空使用厂商默认 model
|
||||
"embedding_dimensions": 0, # 留空/0 使用厂商默认维度(推荐统一 1024)
|
||||
# 语音设置
|
||||
"speech_recognition": True, # 是否开启语音识别
|
||||
"group_speech_recognition": False, # 是否开启群组语音识别
|
||||
"voice_reply_voice": False, # 是否使用语音回复语音,需要设置对应语音合成引擎的api key
|
||||
"always_reply_voice": False, # 是否一直使用语音回复
|
||||
"voice_to_text": "openai", # 语音识别引擎,支持openai,baidu,google,azure,xunfei,ali
|
||||
"text_to_voice": "openai", # 语音合成引擎,支持openai,baidu,google,azure,xunfei,ali,pytts(offline),elevenlabs,edge(online)
|
||||
# Embedding model config
|
||||
"embedding_provider": "", # explicitly set the provider: openai / linkai / dashscope / doubao / zhipu (aligned with bot_type naming)
|
||||
"embedding_model": "", # leave empty to use the provider's default model
|
||||
"embedding_dimensions": 0, # leave empty/0 to use the provider's default dimension (1024 recommended for consistency)
|
||||
# voice config
|
||||
"speech_recognition": True, # whether to enable speech recognition
|
||||
"group_speech_recognition": False, # whether to enable group speech recognition
|
||||
"voice_reply_voice": False, # whether to reply to voice with voice; requires the matching TTS engine api key
|
||||
"always_reply_voice": False, # whether to always reply with voice
|
||||
"voice_to_text": "openai", # speech recognition engine: openai,baidu,google,azure,xunfei,ali
|
||||
"text_to_voice": "openai", # TTS engine: openai,baidu,google,azure,xunfei,ali,pytts(offline),elevenlabs,edge(online)
|
||||
"text_to_voice_model": "tts-1",
|
||||
"tts_voice_id": "alloy",
|
||||
# baidu 语音api配置, 使用百度语音识别和语音合成时需要
|
||||
# baidu voice api config; required when using Baidu speech recognition and TTS
|
||||
"baidu_app_id": "",
|
||||
"baidu_api_key": "",
|
||||
"baidu_secret_key": "",
|
||||
# 1536普通话(支持简单的英文识别) 1737英语 1637粤语 1837四川话 1936普通话远场
|
||||
# 1536 Mandarin (with basic English) 1737 English 1637 Cantonese 1837 Sichuanese 1936 Mandarin far-field
|
||||
"baidu_dev_pid": 1536,
|
||||
# azure 语音api配置, 使用azure语音识别和语音合成时需要
|
||||
# azure voice api config; required when using Azure speech recognition and TTS
|
||||
"azure_voice_api_key": "",
|
||||
"azure_voice_region": "japaneast",
|
||||
# elevenlabs 语音api配置
|
||||
"xi_api_key": "", # 获取ap的方法可以参考https://docs.elevenlabs.io/api-reference/quick-start/authentication
|
||||
"xi_voice_id": "", # ElevenLabs提供了9种英式、美式等英语发音id,分别是“Adam/Antoni/Arnold/Bella/Domi/Elli/Josh/Rachel/Sam”
|
||||
# 服务时间限制
|
||||
"chat_time_module": False, # 是否开启服务时间限制
|
||||
"chat_start_time": "00:00", # 服务开始时间
|
||||
"chat_stop_time": "24:00", # 服务结束时间
|
||||
# 翻译api
|
||||
"translate": "baidu", # 翻译api,支持baidu, youdao
|
||||
# baidu翻译api的配置
|
||||
"baidu_translate_app_id": "", # 百度翻译api的appid
|
||||
"baidu_translate_app_key": "", # 百度翻译api的秘钥
|
||||
# youdao翻译api的配置
|
||||
"youdao_translate_app_key": "", # 有道翻译api的应用ID
|
||||
"youdao_translate_app_secret": "", # 有道翻译api的应用密钥
|
||||
# wechatmp的配置
|
||||
"wechatmp_token": "", # 微信公众平台的Token
|
||||
"wechatmp_port": 8080, # 微信公众平台的端口,需要端口转发到80或443
|
||||
"wechatmp_app_id": "", # 微信公众平台的appID
|
||||
"wechatmp_app_secret": "", # 微信公众平台的appsecret
|
||||
"wechatmp_aes_key": "", # 微信公众平台的EncodingAESKey,加密模式需要
|
||||
# wechatcom的通用配置
|
||||
"wechatcom_corp_id": "", # 企业微信公司的corpID
|
||||
# wechatcomapp的配置
|
||||
"wechatcomapp_token": "", # 企业微信app的token
|
||||
"wechatcomapp_port": 9898, # 企业微信app的服务端口,不需要端口转发
|
||||
"wechatcomapp_secret": "", # 企业微信app的secret
|
||||
"wechatcomapp_agent_id": "", # 企业微信app的agent_id
|
||||
"wechatcomapp_aes_key": "", # 企业微信app的aes_key
|
||||
# 飞书配置
|
||||
"feishu_port": 80, # 飞书bot监听端口,仅webhook模式需要
|
||||
"feishu_app_id": "", # 飞书机器人应用APP Id
|
||||
"feishu_app_secret": "", # 飞书机器人APP secret
|
||||
"feishu_token": "", # 飞书 verification token,仅webhook模式需要
|
||||
"feishu_event_mode": "websocket", # 飞书事件接收模式: webhook(HTTP服务器) 或 websocket(长连接)
|
||||
# 飞书流式回复(基于官方 cardkit 流式卡片 API,需要机器人开通 cardkit:card:write 权限,且飞书客户端 7.20+)
|
||||
"feishu_stream_reply": True, # 是否开启流式回复(打字机效果)。失败/老客户端自动降级为非流式或升级提示
|
||||
# 钉钉配置
|
||||
"dingtalk_client_id": "", # 钉钉机器人Client ID
|
||||
"dingtalk_client_secret": "", # 钉钉机器人Client Secret
|
||||
# elevenlabs voice api config
|
||||
"xi_api_key": "", # see https://docs.elevenlabs.io/api-reference/quick-start/authentication for how to obtain the api key
|
||||
"xi_voice_id": "", # ElevenLabs offers 9 English voice ids: Adam/Antoni/Arnold/Bella/Domi/Elli/Josh/Rachel/Sam
|
||||
# service time limit
|
||||
"chat_time_module": False, # whether to enable service-time limiting
|
||||
"chat_start_time": "00:00", # service start time
|
||||
"chat_stop_time": "24:00", # service stop time
|
||||
# translation api
|
||||
"translate": "baidu", # translation api: baidu, youdao
|
||||
# baidu translation api config
|
||||
"baidu_translate_app_id": "", # baidu translation api appid
|
||||
"baidu_translate_app_key": "", # baidu translation api secret key
|
||||
# youdao translation api config
|
||||
"youdao_translate_app_key": "", # youdao translation api app id
|
||||
"youdao_translate_app_secret": "", # youdao translation api app secret
|
||||
# wechatmp config
|
||||
"wechatmp_token": "", # WeChat Official Account token
|
||||
"wechatmp_port": 8080, # WeChat Official Account port; needs port forwarding to 80 or 443
|
||||
"wechatmp_app_id": "", # WeChat Official Account appID
|
||||
"wechatmp_app_secret": "", # WeChat Official Account appsecret
|
||||
"wechatmp_aes_key": "", # WeChat Official Account EncodingAESKey; required in encrypted mode
|
||||
# wechatcom shared config
|
||||
"wechatcom_corp_id": "", # WeCom corp id
|
||||
# wechatcomapp config
|
||||
"wechatcomapp_token": "", # WeCom app token
|
||||
"wechatcomapp_port": 9898, # WeCom app service port; no port forwarding needed
|
||||
"wechatcomapp_secret": "", # WeCom app secret
|
||||
"wechatcomapp_agent_id": "", # WeCom app agent_id
|
||||
"wechatcomapp_aes_key": "", # WeCom app aes_key
|
||||
# WeChat Customer Service (wechat_kf) config
|
||||
"wechat_kf_corp_id": "", # corp_id of the company the WeChat Customer Service belongs to
|
||||
"wechat_kf_token": "", # WeChat Customer Service callback token
|
||||
"wechat_kf_port": 9888, # WeChat Customer Service callback service port
|
||||
"wechat_kf_secret": "", # WeChat Customer Service app secret
|
||||
"wechat_kf_aes_key": "", # WeChat Customer Service callback aes_key
|
||||
"wechat_kf_cursor_path": "~/.wechat_kf_cursors.json", # path for persisting the WeChat Customer Service sync_msg cursor
|
||||
# Feishu config
|
||||
"feishu_port": 80, # Feishu bot listening port; only needed in webhook mode
|
||||
"feishu_app_id": "", # Feishu bot app id
|
||||
"feishu_app_secret": "", # Feishu bot app secret
|
||||
"feishu_token": "", # Feishu verification token; only needed in webhook mode
|
||||
"feishu_event_mode": "websocket", # Feishu event mode: webhook(HTTP server) or websocket(long connection)
|
||||
# Feishu streaming reply (based on the official cardkit streaming-card API; requires the cardkit:card:write permission and Feishu client 7.20+)
|
||||
"feishu_stream_reply": True, # whether to enable streaming reply (typewriter effect); auto-downgrades to non-streaming or shows an upgrade prompt on failure/old clients
|
||||
# DingTalk config
|
||||
"dingtalk_client_id": "", # DingTalk bot Client ID
|
||||
"dingtalk_client_secret": "", # DingTalk bot Client Secret
|
||||
"dingtalk_card_enabled": False,
|
||||
# 企微智能机器人配置(长连接模式)
|
||||
"wecom_bot_id": "", # 企微智能机器人BotID
|
||||
"wecom_bot_secret": "", # 企微智能机器人长连接Secret
|
||||
# 微信配置
|
||||
"weixin_token": "", # 微信登录后获取的bot_token,留空则启动时自动扫码登录
|
||||
# WeCom smart bot config (long connection mode)
|
||||
"wecom_bot_id": "", # WeCom smart bot BotID
|
||||
"wecom_bot_secret": "", # WeCom smart bot long-connection secret
|
||||
# WeCom smart bot transport mode: "websocket" (long connection) or "webhook" (HTTP callback)
|
||||
"wecom_bot_mode": "websocket",
|
||||
"wecom_bot_token": "", # webhook mode: Token configured on the bot's receive-message URL
|
||||
"wecom_bot_encoding_aes_key": "", # webhook mode: EncodingAESKey configured on the bot's receive-message URL
|
||||
"wecom_bot_port": 9892, # webhook mode: local HTTP server port for the receive-message URL
|
||||
# Telegram config
|
||||
"telegram_token": "", # Bot token from @BotFather
|
||||
"telegram_proxy": "", # Optional HTTP/SOCKS5 proxy, e.g. http://127.0.0.1:7890 or socks5://127.0.0.1:1080 (empty falls back to env vars)
|
||||
"telegram_group_trigger": "mention_or_reply", # Group trigger: mention_or_reply(@ or reply, recommended) | mention_only(@ only) | all(every message)
|
||||
"telegram_register_commands": True, # Auto-register the BotFather command menu on startup (aligned with web slash commands)
|
||||
# Slack config (Socket Mode, no public IP required)
|
||||
"slack_bot_token": "", # Bot User OAuth Token, like xoxb-...
|
||||
"slack_app_token": "", # App-Level Token (generated after enabling Socket Mode), like xapp-...
|
||||
"slack_group_trigger": "mention_or_reply", # Channel trigger: mention_or_reply(@ or reply in thread, recommended) | mention_only(@ only) | all(every message)
|
||||
# Discord config (Gateway connection, no public IP required)
|
||||
"discord_token": "", # Discord Bot Token (generated on the Bot page of the Developer Portal)
|
||||
"discord_group_trigger": "mention_or_reply", # Channel trigger: mention_or_reply(@ or reply to bot, recommended) | mention_only(@ only) | all(every message)
|
||||
# WeChat config
|
||||
"weixin_token": "", # bot_token obtained after WeChat login; leave empty to auto scan-login on startup
|
||||
"weixin_base_url": "https://ilinkai.weixin.qq.com", # Weixin ilink API base URL
|
||||
"weixin_cdn_base_url": "https://novac2c.cdn.weixin.qq.com/c2c", # CDN base URL
|
||||
"weixin_credentials_path": "~/.weixin_cow_credentials.json", # credentials file path
|
||||
# chatgpt指令自定义触发词
|
||||
"clear_memory_commands": ["#清除记忆"], # 重置会话指令,必须以#开头
|
||||
# channel配置
|
||||
"channel_type": "", # 通道类型,支持多渠道同时运行。单个: "feishu",多个: "feishu, dingtalk" 或 ["feishu", "dingtalk"]。可选值: web,feishu,dingtalk,wecom_bot,weixin,wechatmp,wechatmp_service,wechatcom_app
|
||||
"web_console": True, # 是否自动启动Web控制台(默认启动)。设为False可禁用
|
||||
"subscribe_msg": "", # 订阅消息, 支持: wechatmp, wechatmp_service, wechatcom_app
|
||||
"debug": False, # 是否开启debug模式,开启后会打印更多日志
|
||||
"appdata_dir": "", # 数据目录
|
||||
# 插件配置
|
||||
"plugin_trigger_prefix": "$", # 规范插件提供聊天相关指令的前缀,建议不要和管理员指令前缀"#"冲突
|
||||
# 是否使用全局插件配置
|
||||
# custom trigger words for chatgpt commands
|
||||
"clear_memory_commands": ["#清除记忆"], # session-reset command; must start with #
|
||||
# channel config
|
||||
"channel_type": "", # channel type; supports running multiple channels at once. Single: "feishu", multiple: "feishu, dingtalk" or ["feishu", "dingtalk"]. Options: web,feishu,dingtalk,wecom_bot,weixin,wechatmp,wechatmp_service,wechatcom_app,wechat_kf,telegram,slack,discord
|
||||
"web_console": True, # whether to auto-start the Web console (on by default). Set False to disable
|
||||
"subscribe_msg": "", # subscribe message; supported by: wechatmp, wechatmp_service, wechatcom_app
|
||||
"debug": False, # whether to enable debug mode; prints more logs when on
|
||||
"appdata_dir": "", # data directory
|
||||
# plugin config
|
||||
"plugin_trigger_prefix": "$", # prefix for plugin chat commands; avoid clashing with the admin command prefix "#"
|
||||
# whether to use the global plugin config
|
||||
"use_global_plugin_config": False,
|
||||
"max_media_send_count": 3, # 单次最大发送媒体资源的个数
|
||||
"media_send_interval": 1, # 发送图片的事件间隔,单位秒
|
||||
# 智谱AI 平台配置
|
||||
"max_media_send_count": 3, # max number of media resources sent at once
|
||||
"media_send_interval": 1, # interval between sending images, in seconds
|
||||
# Zhipu AI platform config
|
||||
"zhipu_ai_api_key": "",
|
||||
"zhipu_ai_api_base": "https://open.bigmodel.cn/api/paas/v4",
|
||||
"moonshot_api_key": "",
|
||||
"moonshot_base_url": "https://api.moonshot.cn/v1",
|
||||
# 豆包(火山方舟) 平台配置
|
||||
# Doubao (Volcano Ark) platform config
|
||||
"ark_api_key": "",
|
||||
"ark_base_url": "https://ark.cn-beijing.volces.com/api/v3",
|
||||
# 魔搭社区 平台配置
|
||||
# ModelScope community platform config
|
||||
"modelscope_api_key": "",
|
||||
"modelscope_base_url": "https://api-inference.modelscope.cn/v1/chat/completions",
|
||||
# LinkAI平台配置
|
||||
# LinkAI platform config
|
||||
"use_linkai": False,
|
||||
"linkai_api_key": "",
|
||||
"linkai_app_code": "",
|
||||
@@ -209,18 +244,26 @@ available_setting = {
|
||||
"Minimax_base_url": "",
|
||||
"deepseek_api_key": "",
|
||||
"deepseek_api_base": "https://api.deepseek.com/v1",
|
||||
# Xiaomi MiMo LLM
|
||||
"mimo_api_key": "",
|
||||
"mimo_api_base": "https://api.xiaomimimo.com/v1",
|
||||
"web_host": "", # Web console bind address; empty means auto
|
||||
"web_port": 9899,
|
||||
"web_password": "", # Web console password; empty means no authentication required
|
||||
"web_session_expire_days": 30, # Auth session expiry in days
|
||||
"agent": True, # 是否开启Agent模式
|
||||
"agent_workspace": "~/cow", # agent工作空间路径,用于存储skills、memory等
|
||||
"agent_max_context_tokens": 50000, # Agent模式下最大上下文tokens
|
||||
"agent_max_context_turns": 20, # Agent模式下最大上下文记忆轮次
|
||||
"agent_max_steps": 20, # Agent模式下单次运行最大决策步数
|
||||
"web_file_serve_root": "~", # Root dir the /api/file endpoint may serve; "/" allows the whole filesystem
|
||||
"agent": True, # whether to enable Agent mode
|
||||
"agent_workspace": "~/cow", # agent workspace path, used to store skills, memory, etc.
|
||||
"agent_max_context_tokens": 50000, # max context tokens in Agent mode
|
||||
"agent_max_context_turns": 20, # max context memory turns in Agent mode
|
||||
"agent_max_steps": 20, # max decision steps per run in Agent mode
|
||||
"enable_thinking": False, # Enable deep-thinking mode for thinking-capable models
|
||||
"reasoning_effort": "high", # Reasoning depth under thinking mode: "high" or "max"
|
||||
"knowledge": True, # 是否开启知识库功能
|
||||
"knowledge": True, # whether to enable the knowledge base feature
|
||||
# Self-evolution: review idle conversations to learn memory/skills. Flat keys.
|
||||
"self_evolution_enabled": False, # switch to enable/disable self-evolution
|
||||
"self_evolution_idle_minutes": 10, # idle time before a session is reviewed
|
||||
"self_evolution_min_turns": 6, # min user turns (or context pressure) to trigger
|
||||
"skill": {}, # Per-skill runtime config; nested keys flatten to SKILL_<NAME>_<KEY> env vars at startup
|
||||
"mcp_servers": [], # MCP server list; each entry supports type "stdio" (local process) or "sse" (remote URL)
|
||||
}
|
||||
@@ -233,7 +276,7 @@ class Config(dict):
|
||||
d = {}
|
||||
for k, v in d.items():
|
||||
self[k] = v
|
||||
# user_datas: 用户数据,key为用户名,value为用户数据,也是dict
|
||||
# user_datas: per-user data; key is the username, value is the user's data (also a dict)
|
||||
self.user_datas = {}
|
||||
|
||||
def __getitem__(self, key):
|
||||
@@ -243,11 +286,11 @@ class Config(dict):
|
||||
return super().__setitem__(key, value)
|
||||
|
||||
def get(self, key, default=None):
|
||||
# 跳过以下划线开头的注释字段
|
||||
# skip comment fields starting with an underscore
|
||||
if key.startswith("_"):
|
||||
return super().get(key, default)
|
||||
|
||||
# 如果key不在available_setting中,直接走dict的get,返回config.json中实际加载的值(如不存在则返回default)
|
||||
# if the key is not in available_setting, fall back to dict.get and return the value actually loaded from config.json (or default if absent)
|
||||
if key not in available_setting:
|
||||
return super().get(key, default)
|
||||
|
||||
@@ -287,24 +330,37 @@ class Config(dict):
|
||||
config = Config()
|
||||
|
||||
|
||||
def _mask_value(val):
|
||||
"""Mask a sensitive string value, keeping first 3 and last 3 chars."""
|
||||
if not isinstance(val, str) or len(val) <= 8:
|
||||
return val
|
||||
return val[0:3] + "*" * 5 + val[-3:]
|
||||
|
||||
|
||||
def _mask_sensitive_recursive(obj):
|
||||
"""Recursively mask values whose keys contain 'key' or 'secret'."""
|
||||
if isinstance(obj, dict):
|
||||
masked = {}
|
||||
for k, v in obj.items():
|
||||
if ("key" in k or "secret" in k) and isinstance(v, str):
|
||||
masked[k] = _mask_value(v)
|
||||
else:
|
||||
masked[k] = _mask_sensitive_recursive(v)
|
||||
return masked
|
||||
elif isinstance(obj, list):
|
||||
return [_mask_sensitive_recursive(item) for item in obj]
|
||||
return obj
|
||||
|
||||
|
||||
def drag_sensitive(config):
|
||||
try:
|
||||
if isinstance(config, str):
|
||||
conf_dict: dict = json.loads(config)
|
||||
conf_dict_copy = copy.deepcopy(conf_dict)
|
||||
for key in conf_dict_copy:
|
||||
if "key" in key or "secret" in key:
|
||||
if isinstance(conf_dict_copy[key], str):
|
||||
conf_dict_copy[key] = conf_dict_copy[key][0:3] + "*" * 5 + conf_dict_copy[key][-3:]
|
||||
conf_dict_copy = _mask_sensitive_recursive(conf_dict)
|
||||
return json.dumps(conf_dict_copy, indent=4)
|
||||
|
||||
elif isinstance(config, dict):
|
||||
config_copy = copy.deepcopy(config)
|
||||
for key in config:
|
||||
if "key" in key or "secret" in key:
|
||||
if isinstance(config_copy[key], str):
|
||||
config_copy[key] = config_copy[key][0:3] + "*" * 5 + config_copy[key][-3:]
|
||||
return config_copy
|
||||
return _mask_sensitive_recursive(config)
|
||||
except Exception as e:
|
||||
logger.exception(e)
|
||||
return config
|
||||
@@ -314,7 +370,7 @@ def drag_sensitive(config):
|
||||
def load_config():
|
||||
global config
|
||||
|
||||
# 打印 ASCII Logo
|
||||
# print ASCII logo
|
||||
logger.info(" ____ _ _ ")
|
||||
logger.info(" / ___|_____ __ / \\ __ _ ___ _ __ | |_ ")
|
||||
logger.info("| | / _ \\ \\ /\\ / // _ \\ / _` |/ _ \\ '_ \\| __|")
|
||||
@@ -322,15 +378,21 @@ def load_config():
|
||||
logger.info(" \\____\\___/ \\_/\\_//_/ \\_\\__, |\\___|_| |_|\\__|")
|
||||
logger.info(" |___/ ")
|
||||
logger.info("")
|
||||
config_path = "./config.json"
|
||||
# User config lives in the data root: source deployments use CWD (./), while
|
||||
# the desktop build points COW_DATA_DIR at ~/.cow so config survives updates.
|
||||
config_path = os.path.join(get_data_root(), "config.json")
|
||||
if not os.path.exists(config_path):
|
||||
logger.info("配置文件不存在,将使用config-template.json模板")
|
||||
config_path = "./config-template.json"
|
||||
logger.info("config file not found, falling back to config-template.json")
|
||||
# Resolve the template via get_resource_root() so it works both from
|
||||
# source and from a frozen (PyInstaller) bundle, where the template
|
||||
# ships inside the bundle (sys._MEIPASS) and CWD may differ.
|
||||
template_path = os.path.join(get_resource_root(), "config-template.json")
|
||||
config_path = template_path if os.path.exists(template_path) else "./config-template.json"
|
||||
|
||||
config_str = read_file(config_path)
|
||||
logger.debug("[INIT] config str: {}".format(drag_sensitive(config_str)))
|
||||
|
||||
# 将json字符串反序列化为dict类型。
|
||||
# Deserialize the json string into a dict.
|
||||
# `object_pairs_hook` lets us catch users who accidentally typed the
|
||||
# same key twice (e.g. two `"tools"` blocks) — json.loads would
|
||||
# otherwise silently drop all but the last occurrence.
|
||||
@@ -347,7 +409,7 @@ def load_config():
|
||||
# Some online deployment platforms (e.g. Railway) deploy project from github directly. So you shouldn't put your secrets like api key in a config file, instead use environment variables to override the default config.
|
||||
for name, value in os.environ.items():
|
||||
name = name.lower()
|
||||
# 跳过以下划线开头的注释字段
|
||||
# skip comment fields starting with an underscore
|
||||
if name.startswith("_"):
|
||||
continue
|
||||
if name in available_setting:
|
||||
@@ -366,21 +428,26 @@ def load_config():
|
||||
logger.setLevel(logging.DEBUG)
|
||||
logger.debug("[INIT] set log level to DEBUG")
|
||||
|
||||
# Resolve the global UI language as early as possible so that every
|
||||
# downstream layer (logs, CLI, agent prompts, channel replies) shares it.
|
||||
resolved_lang = i18n.resolve_language(config.get("cow_lang", "auto"))
|
||||
|
||||
logger.info("[INIT] load config: {}".format(drag_sensitive(config)))
|
||||
|
||||
# 打印系统初始化信息
|
||||
# print system initialization info
|
||||
logger.info("[INIT] ========================================")
|
||||
logger.info("[INIT] System Initialization")
|
||||
logger.info("[INIT] ========================================")
|
||||
logger.info("[INIT] Language: {}".format(resolved_lang))
|
||||
logger.info("[INIT] Channel: {}".format(config.get("channel_type", "unknown")))
|
||||
logger.info("[INIT] Model: {}".format(config.get("model", "unknown")))
|
||||
|
||||
# Agent模式信息
|
||||
# Agent mode info
|
||||
if config.get("agent", True):
|
||||
workspace = config.get("agent_workspace", "~/cow")
|
||||
logger.info("[INIT] Mode: Agent (workspace: {})".format(workspace))
|
||||
else:
|
||||
logger.info("[INIT] Mode: Chat (在config.json中设置 \"agent\":true 可启用Agent模式)")
|
||||
logger.info("[INIT] Mode: Chat (set \"agent\":true in config.json to enable Agent mode)")
|
||||
|
||||
logger.info("[INIT] Debug: {}".format(config.get("debug", False)))
|
||||
logger.info("[INIT] ========================================")
|
||||
@@ -401,6 +468,8 @@ def load_config():
|
||||
"minimax_api_base": "MINIMAX_API_BASE",
|
||||
"deepseek_api_key": "DEEPSEEK_API_KEY",
|
||||
"deepseek_api_base": "DEEPSEEK_API_BASE",
|
||||
"mimo_api_key": "MIMO_API_KEY",
|
||||
"mimo_api_base": "MIMO_API_BASE",
|
||||
"qianfan_api_key": "QIANFAN_API_KEY",
|
||||
"qianfan_api_base": "QIANFAN_API_BASE",
|
||||
"zhipu_ai_api_key": "ZHIPU_AI_API_KEY",
|
||||
@@ -420,6 +489,11 @@ def load_config():
|
||||
"wechatmp_app_secret": "WECHATMP_APP_SECRET",
|
||||
"wechatcomapp_agent_id": "WECHATCOMAPP_AGENT_ID",
|
||||
"wechatcomapp_secret": "WECHATCOMAPP_SECRET",
|
||||
"wechatcom_corp_id": "WECHATCOM_CORP_ID",
|
||||
"wechat_kf_corp_id": "WECHAT_KF_CORP_ID",
|
||||
"wechat_kf_secret": "WECHAT_KF_SECRET",
|
||||
"wechat_kf_token": "WECHAT_KF_TOKEN",
|
||||
"wechat_kf_aes_key": "WECHAT_KF_AES_KEY",
|
||||
"qq_app_id": "QQ_APP_ID",
|
||||
"qq_app_secret": "QQ_APP_SECRET",
|
||||
"weixin_token": "WEIXIN_TOKEN",
|
||||
@@ -553,6 +627,34 @@ def get_root():
|
||||
return os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
def get_resource_root():
|
||||
"""Directory holding bundled read-only resources (e.g. config-template.json).
|
||||
|
||||
Under PyInstaller, data files live in sys._MEIPASS (the onedir _internal
|
||||
folder), which differs from get_root() — the latter is used for writable
|
||||
user data and should stay next to the executable, not inside the bundle.
|
||||
"""
|
||||
if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
|
||||
return sys._MEIPASS
|
||||
return os.path.dirname(os.path.abspath(__file__))
|
||||
|
||||
|
||||
def get_data_root():
|
||||
"""Directory for writable user data (config.json, user_datas.pkl, run.log).
|
||||
|
||||
The desktop build sets COW_DATA_DIR (e.g. ~/.cow) so data lives in the
|
||||
user's home rather than inside the read-only app bundle and survives app
|
||||
updates. When unset (source deployment), it falls back to get_root(), so
|
||||
existing behavior is unchanged.
|
||||
"""
|
||||
data_dir = os.environ.get("COW_DATA_DIR")
|
||||
if data_dir:
|
||||
data_dir = os.path.expanduser(data_dir)
|
||||
os.makedirs(data_dir, exist_ok=True)
|
||||
return data_dir
|
||||
return get_root()
|
||||
|
||||
|
||||
def read_file(path):
|
||||
with open(path, mode="r", encoding="utf-8-sig") as f:
|
||||
return f.read()
|
||||
@@ -563,13 +665,29 @@ def conf():
|
||||
|
||||
|
||||
def get_appdata_dir():
|
||||
data_path = os.path.join(get_root(), conf().get("appdata_dir", ""))
|
||||
data_path = os.path.join(get_data_root(), conf().get("appdata_dir", ""))
|
||||
if not os.path.exists(data_path):
|
||||
logger.info("[INIT] data path not exists, create it: {}".format(data_path))
|
||||
os.makedirs(data_path)
|
||||
return data_path
|
||||
|
||||
|
||||
def get_weixin_credentials_path():
|
||||
"""Resolve the Weixin credentials (token) file path.
|
||||
|
||||
Honors an explicit ``weixin_credentials_path`` from config. Otherwise the
|
||||
packaged desktop build (COW_DATA_DIR set) keeps it under the data dir
|
||||
(~/.cow) so all user data stays together, while source deployments retain
|
||||
the legacy ~/.weixin_cow_credentials.json default unchanged.
|
||||
"""
|
||||
configured = conf().get("weixin_credentials_path")
|
||||
if configured:
|
||||
return os.path.expanduser(configured)
|
||||
if os.environ.get("COW_DATA_DIR"):
|
||||
return os.path.join(get_data_root(), "weixin_credentials.json")
|
||||
return os.path.expanduser("~/.weixin_cow_credentials.json")
|
||||
|
||||
|
||||
def subscribe_msg():
|
||||
trigger_prefix = conf().get("single_chat_prefix", [""])[0]
|
||||
msg = conf().get("subscribe_msg", "")
|
||||
@@ -582,8 +700,8 @@ plugin_config = {}
|
||||
|
||||
def write_plugin_config(pconf: dict):
|
||||
"""
|
||||
写入插件全局配置
|
||||
:param pconf: 全量插件配置
|
||||
Write the global plugin config.
|
||||
:param pconf: the full plugin config
|
||||
"""
|
||||
global plugin_config
|
||||
for k in pconf:
|
||||
@@ -591,8 +709,8 @@ def write_plugin_config(pconf: dict):
|
||||
|
||||
def remove_plugin_config(name: str):
|
||||
"""
|
||||
移除待重新加载的插件全局配置
|
||||
:param name: 待重载的插件名
|
||||
Remove the global config of a plugin pending reload.
|
||||
:param name: name of the plugin to reload
|
||||
"""
|
||||
global plugin_config
|
||||
plugin_config.pop(name.lower(), None)
|
||||
@@ -600,12 +718,12 @@ def remove_plugin_config(name: str):
|
||||
|
||||
def pconf(plugin_name: str) -> dict:
|
||||
"""
|
||||
根据插件名称获取配置
|
||||
:param plugin_name: 插件名称
|
||||
:return: 该插件的配置项
|
||||
Get the config for a plugin by name.
|
||||
:param plugin_name: plugin name
|
||||
:return: the plugin's config
|
||||
"""
|
||||
return plugin_config.get(plugin_name.lower())
|
||||
|
||||
|
||||
# 全局配置,用于存放全局生效的状态
|
||||
# global config holding globally-effective state
|
||||
global_config = {"admin_users": []}
|
||||
|
||||
6
desktop/.gitignore
vendored
Normal file
6
desktop/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
node_modules/
|
||||
dist/
|
||||
release/
|
||||
*.log
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
81
desktop/README.md
Normal file
81
desktop/README.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# CowAgent Desktop
|
||||
|
||||
Cross-platform desktop client for CowAgent, built with Electron + React + TypeScript.
|
||||
|
||||
## Development
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Node.js 18+
|
||||
- npm or yarn
|
||||
- Python 3.7+ (for the backend)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
cd desktop
|
||||
npm install
|
||||
```
|
||||
|
||||
### Run in Development
|
||||
|
||||
Start the renderer dev server and Electron together:
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Or run them separately:
|
||||
|
||||
```bash
|
||||
# Terminal 1: Start Vite dev server
|
||||
npm run dev:renderer
|
||||
|
||||
# Terminal 2: Start Electron (after renderer is ready)
|
||||
npm run dev:main
|
||||
```
|
||||
|
||||
The app will automatically start the Python backend from the parent directory.
|
||||
|
||||
### Build
|
||||
|
||||
```bash
|
||||
# Build for current platform
|
||||
npm run dist
|
||||
|
||||
# Build for macOS only
|
||||
npm run dist:mac
|
||||
|
||||
# Build for Windows only
|
||||
npm run dist:win
|
||||
```
|
||||
|
||||
Build outputs are placed in the `release/` directory.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
desktop/
|
||||
├── src/
|
||||
│ ├── main/ # Electron main process
|
||||
│ │ ├── index.ts # Window management, IPC
|
||||
│ │ ├── python-manager.ts # Python backend lifecycle
|
||||
│ │ └── preload.ts # Context bridge for renderer
|
||||
│ └── renderer/ # React UI (Vite)
|
||||
│ └── src/
|
||||
│ ├── api/ # HTTP client for backend APIs
|
||||
│ ├── components/ # Reusable UI components
|
||||
│ ├── hooks/ # React hooks
|
||||
│ ├── pages/ # Page components
|
||||
│ └── types.ts # TypeScript types
|
||||
├── resources/ # App icons
|
||||
├── package.json # Dependencies and build config
|
||||
└── vite.config.ts # Vite config
|
||||
```
|
||||
|
||||
### How it Works
|
||||
|
||||
1. Electron main process starts and creates the app window
|
||||
2. It spawns the Python backend (`app.py`) as a child process
|
||||
3. The React UI communicates with the Python backend via HTTP APIs
|
||||
4. SSE (Server-Sent Events) is used for streaming chat responses and live logs
|
||||
79
desktop/build/build-backend.sh
Executable file
79
desktop/build/build-backend.sh
Executable file
@@ -0,0 +1,79 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Build the desktop backend into a self-contained onedir bundle via PyInstaller.
|
||||
# Run from anywhere; paths are resolved relative to the repo root.
|
||||
#
|
||||
# Usage:
|
||||
# bash desktop/build/build-backend.sh # build
|
||||
# PYTHON=python3.11 bash desktop/build/build-backend.sh # pick interpreter
|
||||
#
|
||||
# Output: desktop/build/dist/cowagent-backend/ (folder with the executable)
|
||||
set -euo pipefail
|
||||
|
||||
# --- resolve paths --------------------------------------------------------
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
BUILD_DIR="$SCRIPT_DIR"
|
||||
VENV_DIR="$BUILD_DIR/.venv-build"
|
||||
|
||||
# Prefer Python 3.11 when available: on 3.13+ web.py must be installed from a
|
||||
# GitHub git source (the PyPI build fails), which is flaky on some networks.
|
||||
# 3.11 installs web.py straight from PyPI and has the best PyInstaller support.
|
||||
if [ -z "${PYTHON:-}" ]; then
|
||||
for cand in \
|
||||
"/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11" \
|
||||
"python3.11" \
|
||||
"python3.12" \
|
||||
"python3"; do
|
||||
if command -v "$cand" >/dev/null 2>&1; then
|
||||
PYTHON="$cand"
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
# Prefer Python 3.11: it installs web.py from PyPI (no GitHub clone) and avoids
|
||||
# 3.13's removed-cgi compatibility shims. Override with PYTHON=... if needed.
|
||||
pick_python() {
|
||||
if [ -n "${PYTHON:-}" ]; then echo "$PYTHON"; return; fi
|
||||
for c in python3.11 python3.12 python3.10 python3; do
|
||||
if command -v "$c" >/dev/null 2>&1; then echo "$c"; return; fi
|
||||
done
|
||||
echo python3
|
||||
}
|
||||
PYTHON="$(pick_python)"
|
||||
|
||||
echo "==> Repo root: $ROOT"
|
||||
echo "==> Using Python: $($PYTHON --version 2>&1) ($PYTHON)"
|
||||
|
||||
# --- isolated build venv --------------------------------------------------
|
||||
if [ ! -d "$VENV_DIR" ]; then
|
||||
echo "==> Creating build venv at $VENV_DIR"
|
||||
"$PYTHON" -m venv "$VENV_DIR"
|
||||
fi
|
||||
# shellcheck disable=SC1091
|
||||
source "$VENV_DIR/bin/activate"
|
||||
|
||||
echo "==> Installing build dependencies"
|
||||
pip install -q --upgrade pip
|
||||
# Don't leave a half-populated venv behind if deps fail (e.g. flaky network):
|
||||
# the next run would otherwise reuse a broken venv.
|
||||
if ! pip install -q -r "$BUILD_DIR/requirements-desktop.txt"; then
|
||||
echo "!! Dependency install failed. Removing the build venv so a retry starts clean." >&2
|
||||
deactivate || true
|
||||
rm -rf "$VENV_DIR"
|
||||
exit 1
|
||||
fi
|
||||
pip install -q pyinstaller
|
||||
|
||||
# --- run pyinstaller from repo root so relative datas resolve -------------
|
||||
cd "$ROOT"
|
||||
echo "==> Running PyInstaller (onedir)"
|
||||
pyinstaller "$BUILD_DIR/cowagent-backend.spec" \
|
||||
--noconfirm \
|
||||
--distpath "$BUILD_DIR/dist" \
|
||||
--workpath "$BUILD_DIR/build-work"
|
||||
|
||||
echo ""
|
||||
echo "==> Done. Bundle at: $BUILD_DIR/dist/cowagent-backend/"
|
||||
du -sh "$BUILD_DIR/dist/cowagent-backend/" 2>/dev/null || true
|
||||
echo "==> Smoke test: COW_DESKTOP=1 \"$BUILD_DIR/dist/cowagent-backend/cowagent-backend\""
|
||||
145
desktop/build/cowagent-backend.spec
Normal file
145
desktop/build/cowagent-backend.spec
Normal file
@@ -0,0 +1,145 @@
|
||||
# -*- mode: python ; coding: utf-8 -*-
|
||||
"""
|
||||
PyInstaller spec for the CowAgent desktop backend (onedir).
|
||||
|
||||
Produces a self-contained `cowagent-backend` folder that the Electron app
|
||||
spawns directly, so end users don't need Python installed.
|
||||
|
||||
onedir is chosen over onefile because the backend reads data files via paths
|
||||
relative to the source tree (e.g. config-template.json, skills/, chat.html);
|
||||
onedir preserves that layout, while onefile's temp-extraction would break it.
|
||||
|
||||
Build from the repo root:
|
||||
pyinstaller desktop/build/cowagent-backend.spec --noconfirm
|
||||
"""
|
||||
import os
|
||||
from PyInstaller.utils.hooks import collect_submodules, collect_data_files
|
||||
|
||||
# Resolve the repo root from the spec's own location (desktop/build/ -> root),
|
||||
# independent of the current working directory. PyInstaller exposes SPECPATH.
|
||||
ROOT = os.path.abspath(os.path.join(SPECPATH, '..', '..'))
|
||||
|
||||
|
||||
def rp(*parts):
|
||||
"""Absolute path under the repo root."""
|
||||
return os.path.join(ROOT, *parts)
|
||||
|
||||
# --- Hidden imports -------------------------------------------------------
|
||||
# Channels are imported dynamically by channel_factory via string names, so
|
||||
# PyInstaller's static analysis can't see them. List every channel we ship
|
||||
# (Feishu is intentionally excluded — lark-oapi is dropped from the desktop
|
||||
# build to save ~116MB).
|
||||
hiddenimports = [
|
||||
# channels (dynamic import in channel/channel_factory.py)
|
||||
'channel.web.web_channel',
|
||||
'channel.terminal.terminal_channel',
|
||||
'channel.weixin.weixin_channel',
|
||||
'channel.wechatmp.wechatmp_channel',
|
||||
'channel.wechatcom.wechatcomapp_channel',
|
||||
'channel.wechat_kf.wechat_kf_channel',
|
||||
'channel.dingtalk.dingtalk_channel',
|
||||
'channel.wecom_bot.wecom_bot_channel',
|
||||
'channel.qq.qq_channel',
|
||||
'channel.telegram.telegram_channel',
|
||||
'channel.slack.slack_channel',
|
||||
'channel.discord.discord_channel',
|
||||
]
|
||||
|
||||
# Agent tools and model providers are imported lazily in places; collect their
|
||||
# submodules so nothing is missed at runtime.
|
||||
hiddenimports += collect_submodules('agent.tools')
|
||||
hiddenimports += collect_submodules('models')
|
||||
hiddenimports += collect_submodules('voice')
|
||||
hiddenimports += collect_submodules('bridge')
|
||||
|
||||
# Plugin framework: WebChannel -> ChatChannel imports `from plugins import *`,
|
||||
# so the framework package must be present even though desktop mode never loads
|
||||
# actual plugins (it's only ~tens of KB of code).
|
||||
hiddenimports += [
|
||||
'plugins',
|
||||
'plugins.event',
|
||||
'plugins.plugin',
|
||||
'plugins.plugin_manager',
|
||||
]
|
||||
|
||||
# Third-party SDKs that use lazy/conditional imports internally.
|
||||
hiddenimports += collect_submodules('dashscope')
|
||||
hiddenimports += [
|
||||
'tiktoken_ext',
|
||||
'tiktoken_ext.openai_public',
|
||||
]
|
||||
|
||||
# --- Data files -----------------------------------------------------------
|
||||
# Runtime-read files/dirs that must travel with the executable. Paths are
|
||||
# (source, dest_dir_in_bundle).
|
||||
datas = [
|
||||
(rp('config-template.json'), '.'),
|
||||
(rp('skills'), 'skills'),
|
||||
# Web console served on the backend port: ship chat.html plus its static
|
||||
# assets (~1.9MB) so the browser-based console works as a debug/fallback
|
||||
# entry alongside the Electron UI.
|
||||
(rp('channel', 'web', 'chat.html'), 'channel/web'),
|
||||
(rp('channel', 'web', 'static'), 'channel/web/static'),
|
||||
]
|
||||
|
||||
# Some libraries (tiktoken encodings, etc.) ship data files.
|
||||
datas += collect_data_files('tiktoken_ext', include_py_files=False)
|
||||
|
||||
# --- Excludes -------------------------------------------------------------
|
||||
# Keep the bundle lean: drop Feishu's heavy SDK, plugins (disabled in desktop
|
||||
# mode), tests/docs, and dev-only packages.
|
||||
excludes = [
|
||||
'lark_oapi', # Feishu — ~116MB, excluded from desktop build
|
||||
'tests',
|
||||
'pip',
|
||||
'wheel',
|
||||
'pytest',
|
||||
'playwright', # browser tool is opt-in, not bundled
|
||||
]
|
||||
|
||||
block_cipher = None
|
||||
|
||||
a = Analysis(
|
||||
[rp('app.py')],
|
||||
pathex=[ROOT],
|
||||
binaries=[],
|
||||
datas=datas,
|
||||
hiddenimports=hiddenimports,
|
||||
hookspath=[],
|
||||
hooksconfig={},
|
||||
runtime_hooks=[],
|
||||
excludes=excludes,
|
||||
win_no_prefer_redirects=False,
|
||||
win_private_assemblies=False,
|
||||
cipher=block_cipher,
|
||||
noarchive=False,
|
||||
)
|
||||
|
||||
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
|
||||
|
||||
exe = EXE(
|
||||
pyz,
|
||||
a.scripts,
|
||||
[],
|
||||
exclude_binaries=True,
|
||||
name='cowagent-backend',
|
||||
debug=False,
|
||||
bootloader_ignore_signals=False,
|
||||
strip=False,
|
||||
upx=False,
|
||||
console=True,
|
||||
disable_windowed_traceback=False,
|
||||
target_arch=None,
|
||||
codesign_identity=None,
|
||||
entitlements_file=None,
|
||||
)
|
||||
|
||||
coll = COLLECT(
|
||||
exe,
|
||||
a.binaries,
|
||||
a.datas,
|
||||
strip=False,
|
||||
upx=False,
|
||||
upx_exclude=[],
|
||||
name='cowagent-backend',
|
||||
)
|
||||
52
desktop/build/requirements-desktop.txt
Normal file
52
desktop/build/requirements-desktop.txt
Normal file
@@ -0,0 +1,52 @@
|
||||
# Desktop backend dependencies (slimmed down from the full requirements).
|
||||
#
|
||||
# Goal: keep the package light. The desktop client only needs the web channel
|
||||
# (which Electron talks to) plus the agent core; the remaining IM channels are
|
||||
# cheap (~27MB total) so we keep them, but Feishu's `lark-oapi` (~116MB) is
|
||||
# dropped — it is by far the heaviest dependency and not needed for a C-end
|
||||
# desktop app. Feishu is hidden in desktop mode (see COW_DESKTOP in app.py).
|
||||
|
||||
# ---- core ----
|
||||
numpy>=1.21
|
||||
aiohttp>=3.8.6,<3.10
|
||||
requests>=2.28.2
|
||||
chardet>=5.1.0
|
||||
Pillow
|
||||
python-dotenv>=1.0.0
|
||||
PyYAML>=6.0
|
||||
croniter>=2.0.0
|
||||
click>=8.0
|
||||
qrcode
|
||||
json-repair
|
||||
|
||||
# ---- web framework (web channel) ----
|
||||
# web.py 0.62 fails to build on Python 3.13+ (cgi removed); use the GitHub fix.
|
||||
web.py; python_version < "3.13"
|
||||
web.py @ git+https://github.com/webpy/webpy.git ; python_version >= "3.13"
|
||||
legacy-cgi; python_version >= "3.13"
|
||||
|
||||
# ---- AI model SDKs ----
|
||||
zai-sdk
|
||||
dashscope
|
||||
tenacity # used by some dashscope submodules (retry logic)
|
||||
google-generativeai
|
||||
tiktoken>=0.3.2
|
||||
|
||||
# ---- voice (TTS/ASR) — kept per product decision ----
|
||||
pydub>=0.25.1
|
||||
gTTS>=2.3.1
|
||||
|
||||
# ---- document parsing (web_fetch / knowledge) ----
|
||||
pypdf
|
||||
python-docx
|
||||
openpyxl
|
||||
python-pptx
|
||||
|
||||
# ---- IM channels (kept; lightweight). Feishu/lark-oapi intentionally excluded. ----
|
||||
wechatpy
|
||||
pycryptodome
|
||||
dingtalk_stream
|
||||
websocket-client>=1.4.0
|
||||
python-telegram-bot
|
||||
slack_bolt
|
||||
discord.py
|
||||
8300
desktop/package-lock.json
generated
Normal file
8300
desktop/package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user