fix: call super init

fix： baidu voice init params type error

.flake8

@@ -1,5 +1,5 @@
 [flake8]
-max-line-length = 88
+max-line-length = 176
 select = E303,W293,W291,W292,E305,E231,E302
 exclude =
     .tox,

.github/ISSUE_TEMPLATE.md

@@ -1,31 +0,0 @@
-### 前置确认
-
-1. 网络能够访问openai接口
-2. python 已安装：版本在 3.7 ~ 3.10 之间
-3. `git pull` 拉取最新代码
-4. 执行`pip3 install -r requirements.txt`，检查依赖是否满足
-5. 拓展功能请执行`pip3 install -r requirements-optional.txt`，检查依赖是否满足
-6. 在已有 issue 中未搜索到类似问题
-7. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
-
-
-### 问题描述
-
-> 简要说明、截图、复现步骤等，也可以是需求或想法
-
-
-
-
-### 终端日志 (如有报错)
-
-```
-[在此处粘贴终端日志, 可在主目录下`run.log`文件中找到]
-```
-
-
-
-### 环境
-
- - 操作系统类型  (Mac/Windows/Linux)：
- - Python版本  ( 执行 `python3 -V` )：  
- - pip版本  ( 依赖问题此项必填，执行 `pip3 -V`)：

.github/ISSUE_TEMPLATE/1.bug.yml

@@ -0,0 +1,133 @@
+name: Bug report 🐛
+description: 项目运行中遇到的Bug或问题。
+labels: ['status: needs check']
+body:
+  - type: markdown
+    attributes:
+      value: |
+        ### ⚠️ 前置确认
+        1. 网络能够访问openai接口
+        2. python 已安装：版本在 3.7 ~ 3.10 之间
+        3. `git pull` 拉取最新代码
+        4. 执行`pip3 install -r requirements.txt`，检查依赖是否满足
+        5. 拓展功能请执行`pip3 install -r requirements-optional.txt`，检查依赖是否满足
+        6. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
+  - type: checkboxes
+    attributes:
+      label: 前置确认
+      options:
+        - label: 我确认我运行的是最新版本的代码，并且安装了所需的依赖，在[FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs)中也未找到类似问题。
+          required: true
+  - type: checkboxes
+    attributes:
+      label: ⚠️ 搜索issues中是否已存在类似问题
+      description: >
+        请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框，搜索你的问题
+        或相关日志的关键词来查找是否存在类似问题。
+      options:
+        - label: 我已经搜索过issues和disscussions，没有跟我遇到的问题相关的issue
+          required: true
+  - type: markdown
+    attributes:
+      value: |
+        请在上方的`title`中填写你对你所遇到问题的简略总结，这将帮助其他人更好的找到相似问题，谢谢❤️。
+  - type: dropdown
+    attributes:
+      label: 操作系统类型?
+      description: >
+        请选择你运行程序的操作系统类型。
+      options:
+        - Windows
+        - Linux
+        - MacOS
+        - Docker
+        - Railway
+        - Windows Subsystem for Linux (WSL)
+        - Other (请在问题中说明)
+    validations:
+      required: true
+  - type: dropdown
+    attributes:
+      label: 运行的python版本是?
+      description: |
+        请选择你运行程序的`python`版本。
+        注意：在`python 3.7`中，有部分可选依赖无法安装。
+        经过长时间的观察，我们认为`python 3.8`是兼容性最好的版本。
+        `python 3.7`~`python 3.10`以外版本的issue，将视情况直接关闭。
+      options:
+        - python 3.7
+        - python 3.8
+        - python 3.9
+        - python 3.10
+        - other
+    validations:
+      required: true
+  - type: dropdown
+    attributes:
+      label: 使用的chatgpt-on-wechat版本是?
+      description: |
+        请确保你使用的是 [releases](https://github.com/zhayujie/chatgpt-on-wechat/releases) 中的最新版本。
+        如果你使用git, 请使用`git branch`命令来查看分支。
+      options:
+        - Latest Release
+        - Master (branch)
+    validations:
+      required: true
+  - type: dropdown
+    attributes:
+      label: 运行的`channel`类型是?
+      description: |
+        请确保你正确配置了该`channel`所需的配置项，所有可选的配置项都写在了[该文件中](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)，请将所需配置项填写在根目录下的`config.json`文件中。
+      options:
+        - wx(个人微信, itchat)
+        - wxy(个人微信, wechaty)
+        - wechatmp(公众号, 订阅号)
+        - wechatmp_service(公众号, 服务号)
+        - terminal
+        - other
+    validations:
+      required: true
+  - type: textarea
+    attributes:
+      label: 复现步骤 🕹
+      description: |
+        **⚠️ 不能复现将会关闭issue.**
+  - type: textarea
+    attributes:
+      label: 问题描述 😯
+      description: 详细描述出现的问题，或提供有关截图。
+  - type: textarea
+    attributes:
+      label: 终端日志 📒
+      description: |
+        在此处粘贴终端日志，可在主目录下`run.log`文件中找到，这会帮助我们更好的分析问题，注意隐去你的API key。
+        如果在配置文件中加入`"debug": true`，打印出的日志会更有帮助。
+
+        <details>
+        <summary><i>示例</i></summary>
+        ```log
+        [DEBUG][2023-04-16 00:23:22][plugin_manager.py:157] - Plugin SUMMARY triggered by event Event.ON_HANDLE_CONTEXT
+        [DEBUG][2023-04-16 00:23:22][main.py:221] - [Summary] on_handle_context. content: $总结前100条消息
+        [DEBUG][2023-04-16 00:23:24][main.py:240] - [Summary] limit: 100, duration: -1 seconds
+        [ERROR][2023-04-16 00:23:24][chat_channel.py:244] - Worker return exception: name 'start_date' is not defined
+        Traceback (most recent call last):
+          File "C:\ProgramData\Anaconda3\lib\concurrent\futures\thread.py", line 57, in run
+            result = self.fn(*self.args, **self.kwargs)
+          File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 132, in _handle
+            reply = self._generate_reply(context)
+          File "D:\project\chatgpt-on-wechat\channel\chat_channel.py", line 142, in _generate_reply
+            e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {
+          File "D:\project\chatgpt-on-wechat\plugins\plugin_manager.py", line 159, in emit_event
+            instance.handlers[e_context.event](e_context, *args, **kwargs)
+          File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 255, in on_handle_context
+            records = self._get_records(session_id, start_time, limit)
+          File "D:\project\chatgpt-on-wechat\plugins\summary\main.py", line 96, in _get_records
+            c.execute("SELECT * FROM chat_records WHERE sessionid=? and timestamp>? ORDER BY timestamp DESC LIMIT ?", (session_id, start_date, limit))
+        NameError: name 'start_date' is not defined
+        [INFO][2023-04-16 00:23:36][app.py:14] - signal 2 received, exiting...
+        ```
+        </details>
+      value: |
+        ```log
+        <此处粘贴终端日志>
+        ```

.github/ISSUE_TEMPLATE/2.feature.yml

@@ -0,0 +1,28 @@
+name: Feature request 🚀
+description: 提出你对项目的新想法或建议。
+labels: ['status: needs check']
+body:
+  - type: markdown
+    attributes:
+      value: |
+        请在上方的`title`中填写简略总结，谢谢❤️。
+  - type: checkboxes
+    attributes:
+      label: ⚠️ 搜索是否存在类似issue
+      description: >
+        请在 [历史issue](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中清空输入框，搜索关键词查找是否存在相似issue。
+      options:
+        - label: 我已经搜索过issues和disscussions，没有发现相似issue
+          required: true
+  - type: textarea
+    attributes:
+      label: 总结
+      description: 描述feature的功能。
+  - type: textarea
+    attributes:
+      label: 举例
+      description: 提供聊天示例，草图或相关网址。
+  - type: textarea
+    attributes:
+      label: 动机
+      description: 描述你提出该feature的动机，比如没有这项feature对你的使用造成了怎样的影响。 请提供更详细的场景描述，这可能会帮助我们发现并提出更好的解决方案。

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,6 @@
+blank_issues_enabled: false
+contact_links:
+  - name: 知识星球
+    url: https://public.zsxq.com/groups/88885848842852.html
+    about: 如果你想了解更多项目细节，并与开发者们交流更多关于AI技术的实践，欢迎加入星球
+

.gitignore

@@ -13,6 +13,7 @@ plugins.json
 itchat.pkl
 *.log
 user_datas.pkl
+chatgpt_tool_hub/
 plugins/**/
 !plugins/bdunit
 !plugins/dungeon
@@ -22,4 +23,5 @@ plugins/**/
 !plugins/banwords
 !plugins/banwords/**/
 !plugins/hello
-!plugins/role
+!plugins/role
+!plugins/keyword

README.md

@@ -2,27 +2,41 @@
 
 > ChatGPT近期以强大的对话和信息整合能力风靡全网，可以写代码、改论文、讲故事，几乎无所不能，这让人不禁有个大胆的想法，能否用他的对话模型把我们的微信打造成一个智能机器人，可以在与好友对话中给出意想不到的回应，而且再也不用担心女朋友影响我们 ~~打游戏~~ 工作了。
 
+最新版本支持的功能如下：
 
-基于ChatGPT的微信聊天机器人，通过 [ChatGPT](https://github.com/openai/openai-python) 接口生成对话内容，使用 [itchat](https://github.com/littlecodersh/ItChat) 实现微信消息的接收和自动回复。已实现的特性如下：
-
-- [x] **文本对话：** 接收私聊及群组中的微信消息，使用ChatGPT生成回复内容，完成自动回复
-- [x] **规则定制化：** 支持私聊中按指定规则触发自动回复，支持对群组设置自动回复白名单
-- [x] **图片生成：** 支持根据描述生成图片，支持图片修复
-- [x] **上下文记忆**：支持多轮对话记忆，且为每个好友维护独立的上下会话
-- [x] **语音识别：** 支持接收和处理语音消息，通过文字或语音回复
-- [x] **插件化：** 支持个性化插件，提供角色扮演、文字冒险、与操作系统交互、访问网络数据等能力
-
-> 目前支持微信和微信公众号部署，欢迎接入更多应用，参考 [Terminal代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/terminal/terminal_channel.py)实现接收和发送消息逻辑即可接入。 同时欢迎增加新的插件，参考 [插件说明文档](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins)。
+- [x] **多端部署：** 有多种部署方式可选择且功能完备，目前已支持个人微信，微信公众号和企业微信应用等部署方式
+- [x] **基础对话：** 私聊及群聊的消息智能回复，支持多轮会话上下文记忆，支持 GPT-3，GPT-3.5，GPT-4模型
+- [x] **语音识别：** 可识别语音消息，通过文字或语音回复，支持 azure, baidu, google, openai等多种语音模型
+- [x] **图片生成：** 支持图片生成 和 图生图（如照片修复），可选择 Dell-E, stable diffusion, replicate模型
+- [x] **丰富插件：** 支持个性化插件扩展，已实现多角色切换、文字冒险、敏感词过滤、聊天记录总结等插件
+- [X] **Tool工具：** 与操作系统和互联网交互，支持最新信息搜索、数学计算、天气和资讯查询、网页总结，基于 [chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub) 实现
 
+> 欢迎接入更多应用，参考 [Terminal代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/terminal/terminal_channel.py)实现接收和发送消息逻辑即可接入。 同时欢迎增加新的插件，参考 [插件说明文档](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins)。
 
 **一键部署:**
+- 个人微信
+  
+  [![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/qApznZ?referralCode=RC3znh)
 
-[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/qApznZ?referralCode=RC3znh)
+# 演示
 
+https://user-images.githubusercontent.com/26161723/233777277-e3b9928e-b88f-43e2-b0e0-3cbc923bc799.mp4
+
+Demo made by [Visionn](https://www.wangpc.cc/)
+
+# 交流群
+
+添加小助手微信进群：
+
+<img width="240" src="./docs/images/contact.jpg">
 
 # 更新日志
 
->**2023.04.05：** 支持微信个人号部署，兼容角色扮演等预设插件，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
+>**2023.06.12：** 接入 [LinkAI](https://chat.link-ai.tech/console) 平台，可在线创建 个人知识库，并接入微信中。Beta版本欢迎体验，使用参考 [接入文档](https://link-ai.tech/platform/link-app/wechat)。
+
+>**2023.04.26：** 支持企业微信应用号部署，兼容插件，并支持语音图片交互，私人助理理想选择，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatcom/README.md)。(contributed by [@lanvent](https://github.com/lanvent) in [#944](https://github.com/zhayujie/chatgpt-on-wechat/pull/944))
+
+>**2023.04.05：** 支持微信公众号部署，兼容插件，并支持语音图片交互，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
 
 >**2023.04.05：** 增加能让ChatGPT使用工具的`tool`插件，[使用文档](https://github.com/goldfishh/chatgpt-on-wechat/blob/master/plugins/tool/README.md)。工具相关issue可反馈至[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)。(contributed by [@goldfishh](https://github.com/goldfishh) in [#663](https://github.com/zhayujie/chatgpt-on-wechat/pull/663))
 
@@ -32,28 +46,7 @@
 
 >**2023.03.02：** 接入[ChatGPT API](https://platform.openai.com/docs/guides/chat) (gpt-3.5-turbo)，默认使用该模型进行对话，需升级openai依赖 (`pip3 install --upgrade openai`)。网络问题参考 [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
 
->**2023.02.09：** 扫码登录存在封号风险，请谨慎使用，参考[#58](https://github.com/AutumnWhj/ChatGPT-wechat-bot/issues/158)
-
->**2023.02.05：** 在openai官方接口方案中 (GPT-3模型) 实现上下文对话
-
->**2022.12.18：** 支持根据描述生成图片并发送，openai版本需大于0.25.0
-
->**2022.12.17：** 原来的方案是从 [ChatGPT页面](https://chat.openai.com/chat) 获取session_token，使用 [revChatGPT](https://github.com/acheong08/ChatGPT) 直接访问web接口，但随着ChatGPT接入Cloudflare人机验证，这一方案难以在服务器顺利运行。 所以目前使用的方案是调用 OpenAI 官方提供的 [API](https://beta.openai.com/docs/api-reference/introduction)，回复质量上基本接近于ChatGPT的内容，劣势是暂不支持有上下文记忆的对话，优势是稳定性和响应速度较好。
-
-# 使用效果
-
-### 个人聊天
-
-![single-chat-sample.jpg](docs/images/single-chat-sample.jpg)
-
-### 群组聊天
-
-![group-chat-sample.jpg](docs/images/group-chat-sample.jpg)
-
-### 图片生成
-
-![group-chat-sample.jpg](docs/images/image-create-sample.jpg)
-
+>**2023.02.09：** 扫码登录存在账号限制风险，请谨慎使用，参考[#58](https://github.com/AutumnWhj/ChatGPT-wechat-bot/issues/158)
 
 # 快速开始
 
@@ -63,7 +56,7 @@
 
 前往 [OpenAI注册页面](https://beta.openai.com/signup) 创建账号，参考这篇 [教程](https://www.pythonthree.com/register-openai-chatgpt/) 可以通过虚拟手机号来接收验证码。创建完账号则前往 [API管理页面](https://beta.openai.com/account/api-keys) 创建一个 API Key 并保存下来，后面需要在项目中配置这个key。
 
-> 项目中使用的对话模型是 davinci，计费方式是约每 750 字 (包含请求和回复) 消耗 $0.02，图片生成是每张消耗 $0.016，账号创建有免费的 $18 额度 (更新3.25: 最新注册的已经无免费额度了)，使用完可以更换邮箱重新注册。
+> 项目中默认使用的对话模型是 gpt3.5 turbo，计费方式是约每 500 汉字 (包含请求和回复) 消耗 $0.002，图片生成是每张消耗 $0.016。
 
 ### 2.运行环境
 
@@ -99,15 +92,13 @@ pip3 install -r requirements-optional.txt
 
 参考[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)
 
-使用`azure`语音功能需安装依赖（列在`requirements-optional.txt`内，但为便于`railway`部署已注释）:
+使用`azure`语音功能需安装依赖，并参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)的环境要求。
+:
 
 ```bash
 pip3 install azure-cognitiveservices-speech
 ```
 
-> 目前默认发布的镜像和`railway`部署，都基于`apline`，无法安装`azure`的依赖。若有需求请自行基于[`debian`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/docker/Dockerfile.debian.latest)打包。
-参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)
-
 ## 配置
 
 配置文件的模板在根目录的`config-template.json`中，需复制该模板创建最终生效的 `config.json` 文件：
@@ -116,7 +107,7 @@ pip3 install azure-cognitiveservices-speech
   cp config-template.json config.json
 ```
 
-然后在`config.json`中填入配置，以下是对默认配置的说明，可根据需要进行自定义修改：
+然后在`config.json`中填入配置，以下是对默认配置的说明，可根据需要进行自定义修改（请去掉注释）：
 
 ```bash
 # config.json文件内容示例
@@ -134,7 +125,10 @@ pip3 install azure-cognitiveservices-speech
   "speech_recognition": false,                                # 是否开启语音识别
   "group_speech_recognition": false,                          # 是否开启群组语音识别
   "use_azure_chatgpt": false,                                 # 是否使用Azure ChatGPT service代替openai ChatGPT service. 当设置为true时需要设置 open_ai_api_base，如 https://xxx.openai.azure.com/
-  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述,
+  "azure_deployment_id": "",                                  # 采用Azure ChatGPT时，模型部署名称
+  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述
+  # 订阅消息，公众号和企业微信channel中请填写，当被订阅时会自动回复，可使用特殊占位符。目前支持的占位符有{trigger_prefix}，在程序中它会自动替换成bot的触发词。
+  "subscribe_msg": "感谢您的关注！\n这里是ChatGPT，可以自由对话。\n支持语音对话。\n支持图片输出，画字开头的消息将按要求创作图片。\n支持角色扮演和文字冒险等丰富插件。\n输入{trigger_prefix}#help 查看详细指令。"
 }
 ```
 **配置说明：**
@@ -159,18 +153,19 @@ pip3 install azure-cognitiveservices-speech
 
 **4.其他配置**
 
-+ `model`: 模型名称，目前支持 `gpt-3.5-turbo`, `text-davinci-003`, `gpt-4`, `gpt-4-32k`  (其中gpt-4 api暂未开放)
++ `model`: 模型名称，目前支持 `gpt-3.5-turbo`, `text-davinci-003`, `gpt-4`, `gpt-4-32k`  (其中gpt-4 api暂未完全开放，申请通过后可使用)
 + `temperature`,`frequency_penalty`,`presence_penalty`: Chat API接口参数，详情参考[OpenAI官方文档。](https://platform.openai.com/docs/api-reference/chat)
 + `proxy`：由于目前 `openai` 接口国内无法访问，需配置代理客户端的地址，详情参考  [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
 + 对于图像生成，在满足个人或群组触发条件外，还需要额外的关键词前缀来触发，对应配置 `image_create_prefix `
-+ 关于OpenAI对话及图片接口的参数配置（内容自由度、回复字数限制、图片大小等），可以参考 [对话接口](https://beta.openai.com/docs/api-reference/completions) 和 [图像接口](https://beta.openai.com/docs/api-reference/completions)  文档直接在 [代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/bot/openai/open_ai_bot.py) `bot/openai/open_ai_bot.py` 中进行调整。
++ 关于OpenAI对话及图片接口的参数配置（内容自由度、回复字数限制、图片大小等），可以参考 [对话接口](https://beta.openai.com/docs/api-reference/completions) 和 [图像接口](https://beta.openai.com/docs/api-reference/completions)  文档，在[`config.py`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中检查哪些参数在本项目中是可配置的。
 + `conversation_max_tokens`：表示能够记忆的上下文最大字数（一问一答为一组对话，如果累积的对话字数超出限制，就会优先移除最早的一组对话）
 + `rate_limit_chatgpt`，`rate_limit_dalle`：每分钟最高问答速率、画图速率，超速后排队按序处理。
 + `clear_memory_commands`: 对话内指令，主动清空前文记忆，字符串数组可自定义指令别名。
 + `hot_reload`: 程序退出后，暂存微信扫码状态，默认关闭。
 + `character_desc` 配置中保存着你对机器人说的一段话，他会记住这段话并作为他的设定，你可以为他定制任何人格      (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
++ `subscribe_msg`：订阅消息，公众号和企业微信channel中请填写，当被订阅时会自动回复， 可使用特殊占位符。目前支持的占位符有{trigger_prefix}，在程序中它会自动替换成bot的触发词。
 
-**所有可选的配置项均在该[文件](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
+**本说明文档可能会未及时更新，当前所有可选的配置项均在该[`config.py`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
 
 ## 运行
 
@@ -203,7 +198,7 @@ nohup python3 app.py & tail -f nohup.out          # 在后台运行程序并通
 
 参考文档 [Docker部署](https://github.com/limccn/chatgpt-on-wechat/wiki/Docker%E9%83%A8%E7%BD%B2)   (Contributed by [limccn](https://github.com/limccn))。
 
-### 4. Railway部署(✅推荐)
+### 4. Railway部署 (✅推荐)
 > Railway每月提供5刀和最多500小时的免费额度。
 1. 进入 [Railway](https://railway.app/template/qApznZ?referralCode=RC3znh)。
 2. 点击 `Deploy Now` 按钮。
@@ -213,9 +208,12 @@ nohup python3 app.py & tail -f nohup.out          # 在后台运行程序并通
 
 FAQs： <https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs>
 
+或直接在线咨询 [项目小助手](https://chat.link-ai.tech/app/Kv2fXJcH)  (beta版本，语料完善中，回复仅供参考)
 
 ## 联系
 
-欢迎提交PR、Issues，以及Star支持一下。程序运行遇到问题优先查看 [常见问题列表](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) ，其次前往 [Issues](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中搜索。如果你想了解更多项目细节，并与开发者们交流更多关于AI技术的实践，欢迎加入星球:
+欢迎提交PR、Issues，以及Star支持一下。程序运行遇到问题可以查看 [常见问题列表](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) ，其次前往 [Issues](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中搜索。
+
+如果你想了解更多项目细节，与开发者们交流更多关于AI技术的实践，欢迎加入星球:
 
 <a href="https://public.zsxq.com/groups/88885848842852.html"><img width="360" src="./docs/images/planet.jpg"></a>

app.py

@@ -43,7 +43,7 @@ def run():
             # os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:9001'
 
         channel = channel_factory.create_channel(channel_name)
-        if channel_name in ["wx", "wxy", "terminal", "wechatmp", "wechatmp_service"]:
+        if channel_name in ["wx", "wxy", "terminal", "wechatmp", "wechatmp_service", "wechatcom_app"]:
             PluginManager().load_plugins()
 
         # startup channel

bot/baidu/baidu_unit_bot.py

@@ -10,10 +10,7 @@ from bridge.reply import Reply, ReplyType
 class BaiduUnitBot(Bot):
     def reply(self, query, context=None):
         token = self.get_token()
-        url = (
-            "https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token="
-            + token
-        )
+        url = "https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token=" + token
         post_data = (
             '{"version":"3.0","service_id":"S73177","session_id":"","log_id":"7758521","skill_ids":["1221886"],"request":{"terminal_id":"88888","query":"'
             + query
@@ -32,12 +29,7 @@ class BaiduUnitBot(Bot):
     def get_token(self):
         access_key = "YOUR_ACCESS_KEY"
         secret_key = "YOUR_SECRET_KEY"
-        host = (
-            "https://aip.baidubce.com/oauth/2.0/token?grant_type=client_credentials&client_id="
-            + access_key
-            + "&client_secret="
-            + secret_key
-        )
+        host = "https://aip.baidubce.com/oauth/2.0/token?grant_type=client_credentials&client_id=" + access_key + "&client_secret=" + secret_key
         response = requests.get(host)
         if response:
             print(response.json())

bot/bot_factory.py

@@ -33,4 +33,9 @@ def create_bot(bot_type):
         from bot.chatgpt.chat_gpt_bot import AzureChatGPTBot
 
         return AzureChatGPTBot()
+
+    elif bot_type == const.LINKAI:
+        from bot.linkai.link_ai_bot import LinkAIBot
+        return LinkAIBot()
+
     raise RuntimeError

bot/chatgpt/chat_gpt_bot.py

@@ -4,6 +4,7 @@ import time
 
 import openai
 import openai.error
+import requests
 
 from bot.bot import Bot
 from bot.chatgpt.chat_gpt_session import ChatGPTSession
@@ -30,23 +31,15 @@ class ChatGPTBot(Bot, OpenAIImage):
         if conf().get("rate_limit_chatgpt"):
             self.tb4chatgpt = TokenBucket(conf().get("rate_limit_chatgpt", 20))
 
-        self.sessions = SessionManager(
-            ChatGPTSession, model=conf().get("model") or "gpt-3.5-turbo"
-        )
+        self.sessions = SessionManager(ChatGPTSession, model=conf().get("model") or "gpt-3.5-turbo")
         self.args = {
             "model": conf().get("model") or "gpt-3.5-turbo",  # 对话模型的名称
             "temperature": conf().get("temperature", 0.9),  # 值在[0,1]之间，越大表示回复越具有不确定性
             # "max_tokens":4096,  # 回复最大的字符数
-            "top_p": 1,
-            "frequency_penalty": conf().get(
-                "frequency_penalty", 0.0
-            ),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
-            "presence_penalty": conf().get(
-                "presence_penalty", 0.0
-            ),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
-            "request_timeout": conf().get(
-                "request_timeout", None
-            ),  # 请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
+            "top_p": conf().get("top_p", 1),
+            "frequency_penalty": conf().get("frequency_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "presence_penalty": conf().get("presence_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "request_timeout": conf().get("request_timeout", None),  # 请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
             "timeout": conf().get("request_timeout", None),  # 重试超时时间，在这个时间内，将会自动重试
         }
 
@@ -73,12 +66,16 @@ class ChatGPTBot(Bot, OpenAIImage):
             logger.debug("[CHATGPT] session query={}".format(session.messages))
 
             api_key = context.get("openai_api_key")
-
+            model = context.get("gpt_model")
+            new_args = None
+            if model:
+                new_args = self.args.copy()
+                new_args["model"] = model
             # if context.get('stream'):
             #     # reply in stream
             #     return self.reply_text_stream(query, new_query, session_id)
 
-            reply_content = self.reply_text(session, api_key)
+            reply_content = self.reply_text(session, api_key, args=new_args)
             logger.debug(
                 "[CHATGPT] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(
                     session.messages,
@@ -87,15 +84,10 @@ class ChatGPTBot(Bot, OpenAIImage):
                     reply_content["completion_tokens"],
                 )
             )
-            if (
-                reply_content["completion_tokens"] == 0
-                and len(reply_content["content"]) > 0
-            ):
+            if reply_content["completion_tokens"] == 0 and len(reply_content["content"]) > 0:
                 reply = Reply(ReplyType.ERROR, reply_content["content"])
             elif reply_content["completion_tokens"] > 0:
-                self.sessions.session_reply(
-                    reply_content["content"], session_id, reply_content["total_tokens"]
-                )
+                self.sessions.session_reply(reply_content["content"], session_id, reply_content["total_tokens"])
                 reply = Reply(ReplyType.TEXT, reply_content["content"])
             else:
                 reply = Reply(ReplyType.ERROR, reply_content["content"])
@@ -114,7 +106,7 @@ class ChatGPTBot(Bot, OpenAIImage):
             reply = Reply(ReplyType.ERROR, "Bot不支持处理{}类型的消息".format(context.type))
             return reply
 
-    def reply_text(self, session: ChatGPTSession, api_key=None, retry_count=0) -> dict:
+    def reply_text(self, session: ChatGPTSession, api_key=None, args=None, retry_count=0) -> dict:
         """
         call openai's ChatCompletion to get the answer
         :param session: a conversation session
@@ -126,9 +118,9 @@ class ChatGPTBot(Bot, OpenAIImage):
             if conf().get("rate_limit_chatgpt") and not self.tb4chatgpt.get_token():
                 raise openai.error.RateLimitError("RateLimitError: rate limit exceeded")
             # if api_key == None, the default openai.api_key will be used
-            response = openai.ChatCompletion.create(
-                api_key=api_key, messages=session.messages, **self.args
-            )
+            if args is None:
+                args = self.args
+            response = openai.ChatCompletion.create(api_key=api_key, messages=session.messages, **args)
             # logger.info("[ChatGPT] reply={}, total_tokens={}".format(response.choices[0]['message']['content'], response["usage"]["total_tokens"]))
             return {
                 "total_tokens": response["usage"]["total_tokens"],
@@ -142,24 +134,29 @@ class ChatGPTBot(Bot, OpenAIImage):
                 logger.warn("[CHATGPT] RateLimitError: {}".format(e))
                 result["content"] = "提问太快啦，请休息一下再问我吧"
                 if need_retry:
-                    time.sleep(5)
+                    time.sleep(20)
             elif isinstance(e, openai.error.Timeout):
                 logger.warn("[CHATGPT] Timeout: {}".format(e))
                 result["content"] = "我没有收到你的消息"
                 if need_retry:
                     time.sleep(5)
+            elif isinstance(e, openai.error.APIError):
+                logger.warn("[CHATGPT] Bad Gateway: {}".format(e))
+                result["content"] = "请再问我一次"
+                if need_retry:
+                    time.sleep(10)
             elif isinstance(e, openai.error.APIConnectionError):
                 logger.warn("[CHATGPT] APIConnectionError: {}".format(e))
                 need_retry = False
                 result["content"] = "我连接不到你的网络"
             else:
-                logger.warn("[CHATGPT] Exception: {}".format(e))
+                logger.exception("[CHATGPT] Exception: {}".format(e))
                 need_retry = False
                 self.sessions.clear_session(session.session_id)
 
             if need_retry:
                 logger.warn("[CHATGPT] 第{}次重试".format(retry_count + 1))
-                return self.reply_text(session, api_key, retry_count + 1)
+                return self.reply_text(session, api_key, args, retry_count + 1)
             else:
                 return result
 
@@ -170,3 +167,26 @@ class AzureChatGPTBot(ChatGPTBot):
         openai.api_type = "azure"
         openai.api_version = "2023-03-15-preview"
         self.args["deployment_id"] = conf().get("azure_deployment_id")
+
+    def create_img(self, query, retry_count=0, api_key=None):
+        api_version = "2022-08-03-preview"
+        url = "{}dalle/text-to-image?api-version={}".format(openai.api_base, api_version)
+        api_key = api_key or openai.api_key
+        headers = {"api-key": api_key, "Content-Type": "application/json"}
+        try:
+            body = {"caption": query, "resolution": conf().get("image_create_size", "256x256")}
+            submission = requests.post(url, headers=headers, json=body)
+            operation_location = submission.headers["Operation-Location"]
+            retry_after = submission.headers["Retry-after"]
+            status = ""
+            image_url = ""
+            while status != "Succeeded":
+                logger.info("waiting for image create..., " + status + ",retry after " + retry_after + " seconds")
+                time.sleep(int(retry_after))
+                response = requests.get(operation_location, headers=headers)
+                status = response.json()["status"]
+            image_url = response.json()["result"]["contentUrl"]
+            return True, image_url
+        except Exception as e:
+            logger.error("create image error: {}".format(e))
+            return False, "图片生成失败"

bot/chatgpt/chat_gpt_session.py

@@ -25,9 +25,7 @@ class ChatGPTSession(Session):
             precise = False
             if cur_tokens is None:
                 raise e
-            logger.debug(
-                "Exception when counting tokens precisely for query: {}".format(e)
-            )
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
         while cur_tokens > max_tokens:
             if len(self.messages) > 2:
                 self.messages.pop(1)
@@ -39,16 +37,10 @@ class ChatGPTSession(Session):
                     cur_tokens = cur_tokens - max_tokens
                 break
             elif len(self.messages) == 2 and self.messages[1]["role"] == "user":
-                logger.warn(
-                    "user message exceed max_tokens. total_tokens={}".format(cur_tokens)
-                )
+                logger.warn("user message exceed max_tokens. total_tokens={}".format(cur_tokens))
                 break
             else:
-                logger.debug(
-                    "max_tokens={}, total_tokens={}, len(messages)={}".format(
-                        max_tokens, cur_tokens, len(self.messages)
-                    )
-                )
+                logger.debug("max_tokens={}, total_tokens={}, len(messages)={}".format(max_tokens, cur_tokens, len(self.messages)))
                 break
             if precise:
                 cur_tokens = self.calc_tokens()
@@ -65,27 +57,24 @@ def num_tokens_from_messages(messages, model):
     """Returns the number of tokens used by a list of messages."""
     import tiktoken
 
+    if model == "gpt-3.5-turbo" or model == "gpt-35-turbo":
+        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
+    elif model == "gpt-4":
+        return num_tokens_from_messages(messages, model="gpt-4-0314")
+
     try:
         encoding = tiktoken.encoding_for_model(model)
     except KeyError:
         logger.debug("Warning: model not found. Using cl100k_base encoding.")
         encoding = tiktoken.get_encoding("cl100k_base")
-    if model == "gpt-3.5-turbo" or model == "gpt-35-turbo":
-        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
-    elif model == "gpt-4":
-        return num_tokens_from_messages(messages, model="gpt-4-0314")
-    elif model == "gpt-3.5-turbo-0301":
-        tokens_per_message = (
-            4  # every message follows <|start|>{role/name}\n{content}<|end|>\n
-        )
+    if model == "gpt-3.5-turbo-0301":
+        tokens_per_message = 4  # every message follows <|start|>{role/name}\n{content}<|end|>\n
         tokens_per_name = -1  # if there's a name, the role is omitted
     elif model == "gpt-4-0314":
         tokens_per_message = 3
         tokens_per_name = 1
     else:
-        logger.warn(
-            f"num_tokens_from_messages() is not implemented for model {model}. Returning num tokens assuming gpt-3.5-turbo-0301."
-        )
+        logger.warn(f"num_tokens_from_messages() is not implemented for model {model}. Returning num tokens assuming gpt-3.5-turbo-0301.")
         return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
     num_tokens = 0
     for message in messages:

bot/linkai/link_ai_bot.py

@@ -0,0 +1,108 @@
+# access LinkAI knowledge base platform
+# docs: https://link-ai.tech/platform/link-app/wechat
+
+import time
+
+import requests
+
+from bot.bot import Bot
+from bot.chatgpt.chat_gpt_session import ChatGPTSession
+from bot.openai.open_ai_image import OpenAIImage
+from bot.session_manager import SessionManager
+from bridge.context import Context, ContextType
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from config import conf
+
+
+class LinkAIBot(Bot, OpenAIImage):
+    # authentication failed
+    AUTH_FAILED_CODE = 401
+    NO_QUOTA_CODE = 406
+
+    def __init__(self):
+        super().__init__()
+        self.base_url = "https://api.link-ai.chat/v1"
+        self.sessions = SessionManager(ChatGPTSession, model=conf().get("model") or "gpt-3.5-turbo")
+
+    def reply(self, query, context: Context = None) -> Reply:
+        if context.type == ContextType.TEXT:
+            return self._chat(query, context)
+        elif context.type == ContextType.IMAGE_CREATE:
+            ok, retstring = self.create_img(query, 0)
+            reply = None
+            if ok:
+                reply = Reply(ReplyType.IMAGE_URL, retstring)
+            else:
+                reply = Reply(ReplyType.ERROR, retstring)
+            return reply
+        else:
+            reply = Reply(ReplyType.ERROR, "Bot不支持处理{}类型的消息".format(context.type))
+            return reply
+
+    def _chat(self, query, context, retry_count=0):
+        if retry_count >= 2:
+            # exit from retry 2 times
+            logger.warn("[LINKAI] failed after maximum number of retry times")
+            return Reply(ReplyType.ERROR, "请再问我一次吧")
+
+        try:
+            # load config
+            if context.get("generate_breaked_by"):
+                logger.info(f"[LINKAI] won't set appcode because a plugin ({context['generate_breaked_by']}) affected the context")
+                app_code = None
+            else:
+                app_code = conf().get("linkai_app_code")
+            linkai_api_key = conf().get("linkai_api_key")
+
+            session_id = context["session_id"]
+
+            session = self.sessions.session_query(query, session_id)
+
+            # remove system message
+            if app_code and session.messages[0].get("role") == "system":
+                session.messages.pop(0)
+
+            logger.info(f"[LINKAI] query={query}, app_code={app_code}")
+
+            body = {
+                "appCode": app_code,
+                "messages": session.messages,
+                "model": conf().get("model") or "gpt-3.5-turbo",  # 对话模型的名称
+                "temperature": conf().get("temperature"),
+                "top_p": conf().get("top_p", 1),
+                "frequency_penalty": conf().get("frequency_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+                "presence_penalty": conf().get("presence_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            }
+            headers = {"Authorization": "Bearer " + linkai_api_key}
+
+            # do http request
+            res = requests.post(url=self.base_url + "/chat/completion", json=body, headers=headers).json()
+
+            if not res or not res["success"]:
+                if res.get("code") == self.AUTH_FAILED_CODE:
+                    logger.exception(f"[LINKAI] please check your linkai_api_key, res={res}")
+                    return Reply(ReplyType.ERROR, "请再问我一次吧")
+
+                elif res.get("code") == self.NO_QUOTA_CODE:
+                    logger.exception(f"[LINKAI] please check your account quota, https://chat.link-ai.tech/console/account")
+                    return Reply(ReplyType.ERROR, "提问太快啦，请休息一下再问我吧")
+
+                else:
+                    # retry
+                    time.sleep(2)
+                    logger.warn(f"[LINKAI] do retry, times={retry_count}")
+                    return self._chat(query, context, retry_count + 1)
+
+            # execute success
+            reply_content = res["data"]["content"]
+            logger.info(f"[LINKAI] reply={reply_content}")
+            self.sessions.session_reply(reply_content, session_id)
+            return Reply(ReplyType.TEXT, reply_content)
+
+        except Exception as e:
+            logger.exception(e)
+            # retry
+            time.sleep(2)
+            logger.warn(f"[LINKAI] do retry, times={retry_count}")
+            return self._chat(query, context, retry_count + 1)

bot/openai/open_ai_bot.py

@@ -28,23 +28,15 @@ class OpenAIBot(Bot, OpenAIImage):
         if proxy:
             openai.proxy = proxy
 
-        self.sessions = SessionManager(
-            OpenAISession, model=conf().get("model") or "text-davinci-003"
-        )
+        self.sessions = SessionManager(OpenAISession, model=conf().get("model") or "text-davinci-003")
         self.args = {
             "model": conf().get("model") or "text-davinci-003",  # 对话模型的名称
             "temperature": conf().get("temperature", 0.9),  # 值在[0,1]之间，越大表示回复越具有不确定性
             "max_tokens": 1200,  # 回复最大的字符数
             "top_p": 1,
-            "frequency_penalty": conf().get(
-                "frequency_penalty", 0.0
-            ),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
-            "presence_penalty": conf().get(
-                "presence_penalty", 0.0
-            ),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
-            "request_timeout": conf().get(
-                "request_timeout", None
-            ),  # 请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
+            "frequency_penalty": conf().get("frequency_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "presence_penalty": conf().get("presence_penalty", 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "request_timeout": conf().get("request_timeout", None),  # 请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
             "timeout": conf().get("request_timeout", None),  # 重试超时时间，在这个时间内，将会自动重试
             "stop": ["\n\n\n"],
         }
@@ -71,17 +63,13 @@ class OpenAIBot(Bot, OpenAIImage):
                         result["content"],
                     )
                     logger.debug(
-                        "[OPEN_AI] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(
-                            str(session), session_id, reply_content, completion_tokens
-                        )
+                        "[OPEN_AI] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(str(session), session_id, reply_content, completion_tokens)
                     )
 
                     if total_tokens == 0:
                         reply = Reply(ReplyType.ERROR, reply_content)
                     else:
-                        self.sessions.session_reply(
-                            reply_content, session_id, total_tokens
-                        )
+                        self.sessions.session_reply(reply_content, session_id, total_tokens)
                         reply = Reply(ReplyType.TEXT, reply_content)
                 return reply
             elif context.type == ContextType.IMAGE_CREATE:
@@ -96,9 +84,7 @@ class OpenAIBot(Bot, OpenAIImage):
     def reply_text(self, session: OpenAISession, retry_count=0):
         try:
             response = openai.Completion.create(prompt=str(session), **self.args)
-            res_content = (
-                response.choices[0]["text"].strip().replace("<|endoftext|>", "")
-            )
+            res_content = response.choices[0]["text"].strip().replace("<|endoftext|>", "")
             total_tokens = response["usage"]["total_tokens"]
             completion_tokens = response["usage"]["completion_tokens"]
             logger.info("[OPEN_AI] reply={}".format(res_content))
@@ -114,7 +100,7 @@ class OpenAIBot(Bot, OpenAIImage):
                 logger.warn("[OPEN_AI] RateLimitError: {}".format(e))
                 result["content"] = "提问太快啦，请休息一下再问我吧"
                 if need_retry:
-                    time.sleep(5)
+                    time.sleep(20)
             elif isinstance(e, openai.error.Timeout):
                 logger.warn("[OPEN_AI] Timeout: {}".format(e))
                 result["content"] = "我没有收到你的消息"

bot/openai/open_ai_image.py

@@ -15,17 +15,16 @@ class OpenAIImage(object):
         if conf().get("rate_limit_dalle"):
             self.tb4dalle = TokenBucket(conf().get("rate_limit_dalle", 50))
 
-    def create_img(self, query, retry_count=0):
+    def create_img(self, query, retry_count=0, api_key=None):
         try:
             if conf().get("rate_limit_dalle") and not self.tb4dalle.get_token():
                 return False, "请求太快了，请休息一下再问我吧"
             logger.info("[OPEN_AI] image_query={}".format(query))
             response = openai.Image.create(
+                api_key=api_key,
                 prompt=query,  # 图片描述
                 n=1,  # 每次生成图片的数量
-                size=conf().get(
-                    "image_create_size", "256x256"
-                ),  # 图片大小,可选有 256x256, 512x512, 1024x1024
+                size=conf().get("image_create_size", "256x256"),  # 图片大小,可选有 256x256, 512x512, 1024x1024
             )
             image_url = response["data"][0]["url"]
             logger.info("[OPEN_AI] image_url={}".format(image_url))
@@ -34,11 +33,7 @@ class OpenAIImage(object):
             logger.warn(e)
             if retry_count < 1:
                 time.sleep(5)
-                logger.warn(
-                    "[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(
-                        retry_count + 1
-                    )
-                )
+                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count + 1))
                 return self.create_img(query, retry_count + 1)
             else:
                 return False, "提问太快啦，请休息一下再问我吧"

bot/openai/open_ai_session.py

@@ -36,9 +36,7 @@ class OpenAISession(Session):
             precise = False
             if cur_tokens is None:
                 raise e
-            logger.debug(
-                "Exception when counting tokens precisely for query: {}".format(e)
-            )
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
         while cur_tokens > max_tokens:
             if len(self.messages) > 1:
                 self.messages.pop(0)
@@ -50,18 +48,10 @@ class OpenAISession(Session):
                     cur_tokens = len(str(self))
                 break
             elif len(self.messages) == 1 and self.messages[0]["role"] == "user":
-                logger.warn(
-                    "user question exceed max_tokens. total_tokens={}".format(
-                        cur_tokens
-                    )
-                )
+                logger.warn("user question exceed max_tokens. total_tokens={}".format(cur_tokens))
                 break
             else:
-                logger.debug(
-                    "max_tokens={}, total_tokens={}, len(conversation)={}".format(
-                        max_tokens, cur_tokens, len(self.messages)
-                    )
-                )
+                logger.debug("max_tokens={}, total_tokens={}, len(conversation)={}".format(max_tokens, cur_tokens, len(self.messages)))
                 break
             if precise:
                 cur_tokens = self.calc_tokens()

bot/session_manager.py

@@ -55,9 +55,7 @@ class SessionManager(object):
             return self.sessioncls(session_id, system_prompt, **self.session_args)
 
         if session_id not in self.sessions:
-            self.sessions[session_id] = self.sessioncls(
-                session_id, system_prompt, **self.session_args
-            )
+            self.sessions[session_id] = self.sessioncls(session_id, system_prompt, **self.session_args)
         elif system_prompt is not None:  # 如果有新的system_prompt，更新并重置session
             self.sessions[session_id].set_system_prompt(system_prompt)
         session = self.sessions[session_id]
@@ -71,9 +69,7 @@ class SessionManager(object):
             total_tokens = session.discard_exceeding(max_tokens, None)
             logger.debug("prompt tokens used={}".format(total_tokens))
         except Exception as e:
-            logger.debug(
-                "Exception when counting tokens precisely for prompt: {}".format(str(e))
-            )
+            logger.debug("Exception when counting tokens precisely for prompt: {}".format(str(e)))
         return session
 
     def session_reply(self, reply, session_id, total_tokens=None):
@@ -82,17 +78,9 @@ class SessionManager(object):
         try:
             max_tokens = conf().get("conversation_max_tokens", 1000)
             tokens_cnt = session.discard_exceeding(max_tokens, total_tokens)
-            logger.debug(
-                "raw total_tokens={}, savesession tokens={}".format(
-                    total_tokens, tokens_cnt
-                )
-            )
+            logger.debug("raw total_tokens={}, savesession tokens={}".format(total_tokens, tokens_cnt))
         except Exception as e:
-            logger.debug(
-                "Exception when counting tokens precisely for session: {}".format(
-                    str(e)
-                )
-            )
+            logger.debug("Exception when counting tokens precisely for session: {}".format(str(e)))
         return session
 
     def clear_session(self, session_id):

bridge/bridge.py

@@ -1,11 +1,12 @@
-from bot import bot_factory
+from bot.bot_factory import create_bot
 from bridge.context import Context
 from bridge.reply import Reply
 from common import const
 from common.log import logger
 from common.singleton import singleton
 from config import conf
-from voice import voice_factory
+from translate.factory import create_translator
+from voice.factory import create_voice
 
 
 @singleton
@@ -15,23 +16,28 @@ class Bridge(object):
             "chat": const.CHATGPT,
             "voice_to_text": conf().get("voice_to_text", "openai"),
             "text_to_voice": conf().get("text_to_voice", "google"),
+            "translate": conf().get("translate", "baidu"),
         }
         model_type = conf().get("model")
         if model_type in ["text-davinci-003"]:
             self.btype["chat"] = const.OPEN_AI
         if conf().get("use_azure_chatgpt", False):
             self.btype["chat"] = const.CHATGPTONAZURE
+        if conf().get("use_linkai") and conf().get("linkai_api_key"):
+            self.btype["chat"] = const.LINKAI
         self.bots = {}
 
     def get_bot(self, typename):
         if self.bots.get(typename) is None:
             logger.info("create bot {} for {}".format(self.btype[typename], typename))
             if typename == "text_to_voice":
-                self.bots[typename] = voice_factory.create_voice(self.btype[typename])
+                self.bots[typename] = create_voice(self.btype[typename])
             elif typename == "voice_to_text":
-                self.bots[typename] = voice_factory.create_voice(self.btype[typename])
+                self.bots[typename] = create_voice(self.btype[typename])
             elif typename == "chat":
-                self.bots[typename] = bot_factory.create_bot(self.btype[typename])
+                self.bots[typename] = create_bot(self.btype[typename])
+            elif typename == "translate":
+                self.bots[typename] = create_translator(self.btype[typename])
         return self.bots[typename]
 
     def get_bot_type(self, typename):
@@ -45,3 +51,6 @@ class Bridge(object):
 
     def fetch_text_to_voice(self, text) -> Reply:
         return self.get_bot("text_to_voice").textToVoice(text)
+
+    def fetch_translate(self, text, from_lang="", to_lang="en") -> Reply:
+        return self.get_bot("translate").translate(text, from_lang, to_lang)

bridge/context.py

@@ -8,6 +8,8 @@ class ContextType(Enum):
     VOICE = 2  # 音频消息
     IMAGE = 3  # 图片消息
     IMAGE_CREATE = 10  # 创建图片命令
+    JOIN_GROUP = 20  # 加入群聊
+    PATPAT = 21  # 拍了拍
 
     def __str__(self):
         return self.name
@@ -58,6 +60,4 @@ class Context:
             del self.kwargs[key]
 
     def __str__(self):
-        return "Context(type={}, content={}, kwargs={})".format(
-            self.type, self.content, self.kwargs
-        )
+        return "Context(type={}, content={}, kwargs={})".format(self.type, self.content, self.kwargs)

channel/channel_factory.py

@@ -29,4 +29,8 @@ def create_channel(channel_type):
         from channel.wechatmp.wechatmp_channel import WechatMPChannel
 
         return WechatMPChannel(passive_reply=False)
+    elif channel_type == "wechatcom_app":
+        from channel.wechatcom.wechatcomapp_channel import WechatComAppChannel
+
+        return WechatComAppChannel()
     raise RuntimeError

channel/chat_channel.py

@@ -48,14 +48,15 @@ class ChatChannel(Channel):
         if first_in:  # context首次传入时，receiver是None，根据类型设置receiver
             config = conf()
             cmsg = context["msg"]
+            user_data = conf().get_user_data(cmsg.from_user_id)
+            context["openai_api_key"] = user_data.get("openai_api_key")
+            context["gpt_model"] = user_data.get("gpt_model")
             if context.get("isgroup", False):
                 group_name = cmsg.other_user_nickname
                 group_id = cmsg.other_user_id
 
                 group_name_white_list = config.get("group_name_white_list", [])
-                group_name_keyword_white_list = config.get(
-                    "group_name_keyword_white_list", []
-                )
+                group_name_keyword_white_list = config.get("group_name_keyword_white_list", [])
                 if any(
                     [
                         group_name in group_name_white_list,
@@ -63,9 +64,7 @@ class ChatChannel(Channel):
                         check_contain(group_name, group_name_keyword_white_list),
                     ]
                 ):
-                    group_chat_in_one_session = conf().get(
-                        "group_chat_in_one_session", []
-                    )
+                    group_chat_in_one_session = conf().get("group_chat_in_one_session", [])
                     session_id = cmsg.actual_user_id
                     if any(
                         [
@@ -81,17 +80,11 @@ class ChatChannel(Channel):
             else:
                 context["session_id"] = cmsg.other_user_id
                 context["receiver"] = cmsg.other_user_id
-            e_context = PluginManager().emit_event(
-                EventContext(
-                    Event.ON_RECEIVE_MESSAGE, {"channel": self, "context": context}
-                )
-            )
+            e_context = PluginManager().emit_event(EventContext(Event.ON_RECEIVE_MESSAGE, {"channel": self, "context": context}))
             context = e_context["context"]
             if e_context.is_pass() or context is None:
                 return context
-            if cmsg.from_user_id == self.user_id and not config.get(
-                "trigger_by_self", True
-            ):
+            if cmsg.from_user_id == self.user_id and not config.get("trigger_by_self", True):
                 logger.debug("[WX]self message skipped")
                 return None
 
@@ -114,28 +107,22 @@ class ChatChannel(Channel):
                     logger.info("[WX]receive group at")
                     if not conf().get("group_at_off", False):
                         flag = True
-                    pattern = f"@{self.name}(\u2005|\u0020)"
+                    pattern = f"@{re.escape(self.name)}(\u2005|\u0020)"
                     content = re.sub(pattern, r"", content)
 
                 if not flag:
                     if context["origin_ctype"] == ContextType.VOICE:
-                        logger.info(
-                            "[WX]receive group voice, but checkprefix didn't match"
-                        )
+                        logger.info("[WX]receive group voice, but checkprefix didn't match")
                     return None
             else:  # 单聊
-                match_prefix = check_prefix(
-                    content, conf().get("single_chat_prefix", [""])
-                )
+                match_prefix = check_prefix(content, conf().get("single_chat_prefix", [""]))
                 if match_prefix is not None:  # 判断如果匹配到自定义前缀，则返回过滤掉前缀+空格后的内容
                     content = content.replace(match_prefix, "", 1).strip()
-                elif (
-                    context["origin_ctype"] == ContextType.VOICE
-                ):  # 如果源消息是私聊的语音消息，允许不匹配前缀，放宽条件
+                elif context["origin_ctype"] == ContextType.VOICE:  # 如果源消息是私聊的语音消息，允许不匹配前缀，放宽条件
                     pass
                 else:
                     return None
-
+            content = content.strip()
             img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
             if img_match_prefix:
                 content = content.replace(img_match_prefix, "", 1)
@@ -143,18 +130,10 @@ class ChatChannel(Channel):
             else:
                 context.type = ContextType.TEXT
             context.content = content.strip()
-            if (
-                "desire_rtype" not in context
-                and conf().get("always_reply_voice")
-                and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE
-            ):
+            if "desire_rtype" not in context and conf().get("always_reply_voice") and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
                 context["desire_rtype"] = ReplyType.VOICE
         elif context.type == ContextType.VOICE:
-            if (
-                "desire_rtype" not in context
-                and conf().get("voice_reply_voice")
-                and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE
-            ):
+            if "desire_rtype" not in context and conf().get("voice_reply_voice") and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
                 context["desire_rtype"] = ReplyType.VOICE
 
         return context
@@ -182,15 +161,10 @@ class ChatChannel(Channel):
         )
         reply = e_context["reply"]
         if not e_context.is_pass():
-            logger.debug(
-                "[WX] ready to handle context: type={}, content={}".format(
-                    context.type, context.content
-                )
-            )
-            if (
-                context.type == ContextType.TEXT
-                or context.type == ContextType.IMAGE_CREATE
-            ):  # 文字和图片消息
+            logger.debug("[WX] ready to handle context: type={}, content={}".format(context.type, context.content))
+            if e_context.is_break():
+                context["generate_breaked_by"] = e_context["breaked_by"]
+            if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:  # 文字和图片消息
                 reply = super().build_reply_content(context.content, context)
             elif context.type == ContextType.VOICE:  # 语音消息
                 cmsg = context["msg"]
@@ -214,9 +188,7 @@ class ChatChannel(Channel):
                     # logger.warning("[WX]delete temp file error: " + str(e))
 
                 if reply.type == ReplyType.TEXT:
-                    new_context = self._compose_context(
-                        ContextType.TEXT, reply.content, **context.kwargs
-                    )
+                    new_context = self._compose_context(ContextType.TEXT, reply.content, **context.kwargs)
                     if new_context:
                         reply = self._generate_reply(new_context)
                     else:
@@ -246,48 +218,24 @@ class ChatChannel(Channel):
 
                 if reply.type == ReplyType.TEXT:
                     reply_text = reply.content
-                    if (
-                        desire_rtype == ReplyType.VOICE
-                        and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE
-                    ):
+                    if desire_rtype == ReplyType.VOICE and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
                         reply = super().build_text_to_voice(reply.content)
                         return self._decorate_reply(context, reply)
                     if context.get("isgroup", False):
-                        reply_text = (
-                            "@"
-                            + context["msg"].actual_user_nickname
-                            + " "
-                            + reply_text.strip()
-                        )
-                        reply_text = (
-                            conf().get("group_chat_reply_prefix", "") + reply_text
-                        )
+                        reply_text = "@" + context["msg"].actual_user_nickname + "\n" + reply_text.strip()
+                        reply_text = conf().get("group_chat_reply_prefix", "") + reply_text
                     else:
-                        reply_text = (
-                            conf().get("single_chat_reply_prefix", "") + reply_text
-                        )
+                        reply_text = conf().get("single_chat_reply_prefix", "") + reply_text
                     reply.content = reply_text
                 elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
                     reply.content = "[" + str(reply.type) + "]\n" + reply.content
-                elif (
-                    reply.type == ReplyType.IMAGE_URL
-                    or reply.type == ReplyType.VOICE
-                    or reply.type == ReplyType.IMAGE
-                ):
+                elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
                     pass
                 else:
                     logger.error("[WX] unknown reply type: {}".format(reply.type))
                     return
-            if (
-                desire_rtype
-                and desire_rtype != reply.type
-                and reply.type not in [ReplyType.ERROR, ReplyType.INFO]
-            ):
-                logger.warning(
-                    "[WX] desire_rtype: {}, but reply type: {}".format(
-                        context.get("desire_rtype"), reply.type
-                    )
-                )
+            if desire_rtype and desire_rtype != reply.type and reply.type not in [ReplyType.ERROR, ReplyType.INFO]:
+                logger.warning("[WX] desire_rtype: {}, but reply type: {}".format(context.get("desire_rtype"), reply.type))
             return reply
 
     def _send_reply(self, context: Context, reply: Reply):
@@ -300,9 +248,7 @@ class ChatChannel(Channel):
             )
             reply = e_context["reply"]
             if not e_context.is_pass() and reply and reply.type:
-                logger.debug(
-                    "[WX] ready to send reply: {}, context: {}".format(reply, context)
-                )
+                logger.debug("[WX] ready to send reply: {}, context: {}".format(reply, context))
                 self._send(reply, context)
 
     def _send(self, reply: Reply, context: Context, retry_cnt=0):
@@ -328,9 +274,7 @@ class ChatChannel(Channel):
             try:
                 worker_exception = worker.exception()
                 if worker_exception:
-                    self._fail_callback(
-                        session_id, exception=worker_exception, **kwargs
-                    )
+                    self._fail_callback(session_id, exception=worker_exception, **kwargs)
                 else:
                     self._success_callback(session_id, **kwargs)
             except CancelledError as e:
@@ -366,24 +310,14 @@ class ChatChannel(Channel):
                         if not context_queue.empty():
                             context = context_queue.get()
                             logger.debug("[WX] consume context: {}".format(context))
-                            future: Future = self.handler_pool.submit(
-                                self._handle, context
-                            )
-                            future.add_done_callback(
-                                self._thread_pool_callback(session_id, context=context)
-                            )
+                            future: Future = self.handler_pool.submit(self._handle, context)
+                            future.add_done_callback(self._thread_pool_callback(session_id, context=context))
                             if session_id not in self.futures:
                                 self.futures[session_id] = []
                             self.futures[session_id].append(future)
-                        elif (
-                            semaphore._initial_value == semaphore._value + 1
-                        ):  # 除了当前，没有任务再申请到信号量，说明所有任务都处理完毕
-                            self.futures[session_id] = [
-                                t for t in self.futures[session_id] if not t.done()
-                            ]
-                            assert (
-                                len(self.futures[session_id]) == 0
-                            ), "thread pool error"
+                        elif semaphore._initial_value == semaphore._value + 1:  # 除了当前，没有任务再申请到信号量，说明所有任务都处理完毕
+                            self.futures[session_id] = [t for t in self.futures[session_id] if not t.done()]
+                            assert len(self.futures[session_id]) == 0, "thread pool error"
                             del self.sessions[session_id]
                         else:
                             semaphore.release()
@@ -397,9 +331,7 @@ class ChatChannel(Channel):
                     future.cancel()
                 cnt = self.sessions[session_id][0].qsize()
                 if cnt > 0:
-                    logger.info(
-                        "Cancel {} messages in session {}".format(cnt, session_id)
-                    )
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
                 self.sessions[session_id][0] = Dequeue()
 
     def cancel_all_session(self):
@@ -409,9 +341,7 @@ class ChatChannel(Channel):
                     future.cancel()
                 cnt = self.sessions[session_id][0].qsize()
                 if cnt > 0:
-                    logger.info(
-                        "Cancel {} messages in session {}".format(cnt, session_id)
-                    )
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
                 self.sessions[session_id][0] = Dequeue()
 
 

channel/terminal/terminal_channel.py

@@ -77,9 +77,7 @@ class TerminalChannel(ChatChannel):
             if check_prefix(prompt, trigger_prefixs) is None:
                 prompt = trigger_prefixs[0] + prompt  # 给没触发的消息加上触发前缀
 
-            context = self._compose_context(
-                ContextType.TEXT, prompt, msg=TerminalMessage(msg_id, prompt)
-            )
+            context = self._compose_context(ContextType.TEXT, prompt, msg=TerminalMessage(msg_id, prompt))
             if context:
                 self.produce(context)
             else:

channel/wechat/wechat_channel.py

@@ -23,23 +23,27 @@ from common.time_check import time_checker
 from config import conf, get_appdata_dir
 from lib import itchat
 from lib.itchat.content import *
-from plugins import *
 
 
-@itchat.msg_register([TEXT, VOICE, PICTURE])
+@itchat.msg_register([TEXT, VOICE, PICTURE, NOTE])
 def handler_single_msg(msg):
-    # logger.debug("handler_single_msg: {}".format(msg))
-    if msg["Type"] == PICTURE and msg["MsgType"] == 47:
+    try:
+        cmsg = WechatMessage(msg, False)
+    except NotImplementedError as e:
+        logger.debug("[WX]single message {} skipped: {}".format(msg["MsgId"], e))
         return None
-    WechatChannel().handle_single(WeChatMessage(msg))
+    WechatChannel().handle_single(cmsg)
     return None
 
 
-@itchat.msg_register([TEXT, VOICE, PICTURE], isGroupChat=True)
+@itchat.msg_register([TEXT, VOICE, PICTURE, NOTE], isGroupChat=True)
 def handler_group_msg(msg):
-    if msg["Type"] == PICTURE and msg["MsgType"] == 47:
+    try:
+        cmsg = WechatMessage(msg, True)
+    except NotImplementedError as e:
+        logger.debug("[WX]group message {} skipped: {}".format(msg["MsgId"], e))
         return None
-    WechatChannel().handle_group(WeChatMessage(msg, True))
+    WechatChannel().handle_group(cmsg)
     return None
 
 
@@ -51,10 +55,7 @@ def _check(func):
             return
         self.receivedMsgs[msgId] = cmsg
         create_time = cmsg.create_time  # 消息时间戳
-        if (
-            conf().get("hot_reload") == True
-            and int(create_time) < int(time.time()) - 60
-        ):  # 跳过1分钟前的历史消息
+        if conf().get("hot_reload") == True and int(create_time) < int(time.time()) - 60:  # 跳过1分钟前的历史消息
             logger.debug("[WX]history message {} skipped".format(msgId))
             return
         return func(self, cmsg)
@@ -83,15 +84,9 @@ def qrCallback(uuid, status, qrcode):
         url = f"https://login.weixin.qq.com/l/{uuid}"
 
         qr_api1 = "https://api.isoyu.com/qr/?m=1&e=L&p=20&url={}".format(url)
-        qr_api2 = (
-            "https://api.qrserver.com/v1/create-qr-code/?size=400×400&data={}".format(
-                url
-            )
-        )
+        qr_api2 = "https://api.qrserver.com/v1/create-qr-code/?size=400×400&data={}".format(url)
         qr_api3 = "https://api.pwmqr.com/qrcode/create/?url={}".format(url)
-        qr_api4 = "https://my.tv.sohu.com/user/a/wvideo/getQRCode.do?text={}".format(
-            url
-        )
+        qr_api4 = "https://my.tv.sohu.com/user/a/wvideo/getQRCode.do?text={}".format(url)
         print("You can also scan QRCode in any website below:")
         print(qr_api3)
         print(qr_api4)
@@ -117,30 +112,15 @@ class WechatChannel(ChatChannel):
         # login by scan QRCode
         hotReload = conf().get("hot_reload", False)
         status_path = os.path.join(get_appdata_dir(), "itchat.pkl")
-        try:
-            itchat.auto_login(
-                enableCmdQR=2,
-                hotReload=hotReload,
-                statusStorageDir=status_path,
-                qrCallback=qrCallback,
-            )
-        except Exception as e:
-            if hotReload:
-                logger.error("Hot reload failed, try to login without hot reload")
-                itchat.logout()
-                os.remove(status_path)
-                itchat.auto_login(
-                    enableCmdQR=2, hotReload=hotReload, qrCallback=qrCallback
-                )
-            else:
-                raise e
+        itchat.auto_login(
+            enableCmdQR=2,
+            hotReload=hotReload,
+            statusStorageDir=status_path,
+            qrCallback=qrCallback,
+        )
         self.user_id = itchat.instance.storageClass.userName
         self.name = itchat.instance.storageClass.nickName
-        logger.info(
-            "Wechat login success, user_id: {}, nickname: {}".format(
-                self.user_id, self.name
-            )
-        )
+        logger.info("Wechat login success, user_id: {}, nickname: {}".format(self.user_id, self.name))
         # start message listener
         itchat.run()
 
@@ -165,15 +145,13 @@ class WechatChannel(ChatChannel):
             logger.debug("[WX]receive voice msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.IMAGE:
             logger.debug("[WX]receive image msg: {}".format(cmsg.content))
+        elif cmsg.ctype == ContextType.PATPAT:
+            logger.debug("[WX]receive patpat msg: {}".format(cmsg.content))
+        elif cmsg.ctype == ContextType.TEXT:
+            logger.debug("[WX]receive text msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
         else:
-            logger.debug(
-                "[WX]receive text msg: {}, cmsg={}".format(
-                    json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg
-                )
-            )
-        context = self._compose_context(
-            cmsg.ctype, cmsg.content, isgroup=False, msg=cmsg
-        )
+            logger.debug("[WX]receive msg: {}, cmsg={}".format(cmsg.content, cmsg))
+        context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=False, msg=cmsg)
         if context:
             self.produce(context)
 
@@ -186,12 +164,14 @@ class WechatChannel(ChatChannel):
             logger.debug("[WX]receive voice for group msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.IMAGE:
             logger.debug("[WX]receive image for group msg: {}".format(cmsg.content))
-        else:
+        elif cmsg.ctype in [ContextType.JOIN_GROUP, ContextType.PATPAT]:
+            logger.debug("[WX]receive note msg: {}".format(cmsg.content))
+        elif cmsg.ctype == ContextType.TEXT:
             # logger.debug("[WX]receive group msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
             pass
-        context = self._compose_context(
-            cmsg.ctype, cmsg.content, isgroup=True, msg=cmsg
-        )
+        else:
+            logger.debug("[WX]receive group msg: {}".format(cmsg.content))
+        context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=True, msg=cmsg)
         if context:
             self.produce(context)
 

channel/wechat/wechat_message.py

@@ -1,3 +1,5 @@
+import re
+
 from bridge.context import ContextType
 from channel.chat_message import ChatMessage
 from common.log import logger
@@ -6,7 +8,7 @@ from lib import itchat
 from lib.itchat.content import *
 
 
-class WeChatMessage(ChatMessage):
+class WechatMessage(ChatMessage):
     def __init__(self, itchat_msg, is_group=False):
         super().__init__(itchat_msg)
         self.msg_id = itchat_msg["MsgId"]
@@ -24,10 +26,24 @@ class WeChatMessage(ChatMessage):
             self.ctype = ContextType.IMAGE
             self.content = TmpDir().path() + itchat_msg["FileName"]  # content直接存临时目录路径
             self._prepare_fn = lambda: itchat_msg.download(self.content)
+        elif itchat_msg["Type"] == NOTE and itchat_msg["MsgType"] == 10000:
+            if is_group and ("加入群聊" in itchat_msg["Content"] or "加入了群聊" in itchat_msg["Content"]):
+                self.ctype = ContextType.JOIN_GROUP
+                self.content = itchat_msg["Content"]
+                # 这里只能得到nickname， actual_user_id还是机器人的id
+                if "加入了群聊" in itchat_msg["Content"]:
+                    self.actual_user_nickname = re.findall(r"\"(.*?)\"", itchat_msg["Content"])[-1]
+                elif "加入群聊" in itchat_msg["Content"]:
+                    self.actual_user_nickname = re.findall(r"\"(.*?)\"", itchat_msg["Content"])[0]
+            elif "拍了拍我" in itchat_msg["Content"]:
+                self.ctype = ContextType.PATPAT
+                self.content = itchat_msg["Content"]
+                if is_group:
+                    self.actual_user_nickname = re.findall(r"\"(.*?)\"", itchat_msg["Content"])[0]
+            else:
+                raise NotImplementedError("Unsupported note message: " + itchat_msg["Content"])
         else:
-            raise NotImplementedError(
-                "Unsupported message type: {}".format(itchat_msg["Type"])
-            )
+            raise NotImplementedError("Unsupported message type: Type:{} MsgType:{}".format(itchat_msg["Type"], itchat_msg["MsgType"]))
 
         self.from_user_id = itchat_msg["FromUserName"]
         self.to_user_id = itchat_msg["ToUserName"]
@@ -58,4 +74,5 @@ class WeChatMessage(ChatMessage):
         if self.is_group:
             self.is_at = itchat_msg["IsAt"]
             self.actual_user_id = itchat_msg["ActualUserName"]
-            self.actual_user_nickname = itchat_msg["ActualNickName"]
+            if self.ctype not in [ContextType.JOIN_GROUP, ContextType.PATPAT]:
+                self.actual_user_nickname = itchat_msg["ActualNickName"]

channel/wechat/wechaty_channel.py

@@ -60,13 +60,9 @@ class WechatyChannel(ChatChannel):
         receiver_id = context["receiver"]
         loop = asyncio.get_event_loop()
         if context["isgroup"]:
-            receiver = asyncio.run_coroutine_threadsafe(
-                self.bot.Room.find(receiver_id), loop
-            ).result()
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Room.find(receiver_id), loop).result()
         else:
-            receiver = asyncio.run_coroutine_threadsafe(
-                self.bot.Contact.find(receiver_id), loop
-            ).result()
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Contact.find(receiver_id), loop).result()
         msg = None
         if reply.type == ReplyType.TEXT:
             msg = reply.content
@@ -83,9 +79,7 @@ class WechatyChannel(ChatChannel):
             voiceLength = int(any_to_sil(file_path, sil_file))
             if voiceLength >= 60000:
                 voiceLength = 60000
-                logger.info(
-                    "[WX] voice too long, length={}, set to 60s".format(voiceLength)
-                )
+                logger.info("[WX] voice too long, length={}, set to 60s".format(voiceLength))
             # 发送语音
             t = int(time.time())
             msg = FileBox.from_file(sil_file, name=str(t) + ".sil")
@@ -98,9 +92,7 @@ class WechatyChannel(ChatChannel):
                     os.remove(sil_file)
             except Exception as e:
                 pass
-            logger.info(
-                "[WX] sendVoice={}, receiver={}".format(reply.content, receiver)
-            )
+            logger.info("[WX] sendVoice={}, receiver={}".format(reply.content, receiver))
         elif reply.type == ReplyType.IMAGE_URL:  # 从网络下载图片
             img_url = reply.content
             t = int(time.time())
@@ -111,9 +103,7 @@ class WechatyChannel(ChatChannel):
             image_storage = reply.content
             image_storage.seek(0)
             t = int(time.time())
-            msg = FileBox.from_base64(
-                base64.b64encode(image_storage.read()), str(t) + ".png"
-            )
+            msg = FileBox.from_base64(base64.b64encode(image_storage.read()), str(t) + ".png")
             asyncio.run_coroutine_threadsafe(receiver.say(msg), loop).result()
             logger.info("[WX] sendImage, receiver={}".format(receiver))
 

channel/wechat/wechaty_message.py

@@ -45,16 +45,12 @@ class WechatyMessage(ChatMessage, aobject):
 
             def func():
                 loop = asyncio.get_event_loop()
-                asyncio.run_coroutine_threadsafe(
-                    voice_file.to_file(self.content), loop
-                ).result()
+                asyncio.run_coroutine_threadsafe(voice_file.to_file(self.content), loop).result()
 
             self._prepare_fn = func
 
         else:
-            raise NotImplementedError(
-                "Unsupported message type: {}".format(wechaty_msg.type())
-            )
+            raise NotImplementedError("Unsupported message type: {}".format(wechaty_msg.type()))
 
         from_contact = wechaty_msg.talker()  # 获取消息的发送者
         self.from_user_id = from_contact.contact_id
@@ -73,9 +69,7 @@ class WechatyMessage(ChatMessage, aobject):
             self.to_user_id = to_contact.contact_id
             self.to_user_nickname = to_contact.name
 
-        if (
-            self.is_group or wechaty_msg.is_self()
-        ):  # 如果是群消息，other_user设置为群，如果是私聊消息，而且自己发的，就设置成对方。
+        if self.is_group or wechaty_msg.is_self():  # 如果是群消息，other_user设置为群，如果是私聊消息，而且自己发的，就设置成对方。
             self.other_user_id = self.to_user_id
             self.other_user_nickname = self.to_user_nickname
         else:
@@ -86,7 +80,7 @@ class WechatyMessage(ChatMessage, aobject):
             self.is_at = await wechaty_msg.mention_self()
             if not self.is_at:  # 有时候复制粘贴的消息，不算做@，但是内容里面会有@xxx，这里做一下兼容
                 name = wechaty_msg.wechaty.user_self().name
-                pattern = f"@{name}(\u2005|\u0020)"
+                pattern = f"@{re.escape(name)}(\u2005|\u0020)"
                 if re.search(pattern, self.content):
                     logger.debug(f"wechaty message {self.msg_id} include at")
                     self.is_at = True

channel/wechatcom/README.md

@@ -0,0 +1,85 @@
+# 企业微信应用号channel
+
+企业微信官方提供了客服、应用等API，本channel使用的是企业微信的自建应用API的能力。
+
+因为未来可能还会开发客服能力，所以本channel的类型名叫作`wechatcom_app`。
+
+`wechatcom_app` channel支持插件系统和图片声音交互等能力，除了无法加入群聊，作为个人使用的私人助理已绰绰有余。
+
+## 开始之前
+
+- 在企业中确认自己拥有在企业内自建应用的权限。
+- 如果没有权限或者是个人用户，也可创建未认证的企业。操作方式：登录手机企业微信，选择`创建/加入企业`来创建企业，类型请选择企业，企业名称可随意填写。
+    未认证的企业有100人的服务人数上限，其他功能与认证企业没有差异。
+
+本channel需安装的依赖与公众号一致，需要安装`wechatpy`和`web.py`，它们包含在`requirements-optional.txt`中。
+
+此外，如果你是`Linux`系统，除了`ffmpeg`还需要安装`amr`编码器，否则会出现找不到编码器的错误，无法正常使用语音功能。
+
+- Ubuntu/Debian
+
+```bash
+apt-get install libavcodec-extra
+```
+
+- Alpine
+
+需自行编译`ffmpeg`，在编译参数里加入`amr`编码器的支持
+
+## 使用方法
+
+1.查看企业ID
+
+- 扫码登陆[企业微信后台](https://work.weixin.qq.com)
+- 选择`我的企业`，点击`企业信息`，记住该`企业ID`
+
+2.创建自建应用
+
+- 选择应用管理, 在自建区选创建应用来创建企业自建应用
+- 上传应用logo，填写应用名称等项
+- 创建应用后进入应用详情页面，记住`AgentId`和`Secert`
+
+3.配置应用
+
+- 在详情页点击`企业可信IP`的配置(没看到可以不管)，填入你服务器的公网IP，如果不知道可以先不填
+- 点击`接收消息`下的启用API接收消息
+- `URL`填写格式为`http://url:port/wxcomapp`，`port`是程序监听的端口，默认是9898
+    如果是未认证的企业，url可直接使用服务器的IP。如果是认证企业，需要使用备案的域名，可使用二级域名。
+- `Token`可随意填写，停留在这个页面
+- 在程序根目录`config.json`中增加配置（**去掉注释**），`wechatcomapp_aes_key`是当前页面的`wechatcomapp_aes_key`
+
+```python
+    "channel_type": "wechatcom_app",
+    "wechatcom_corp_id": "",  # 企业微信公司的corpID
+    "wechatcomapp_token": "",  # 企业微信app的token
+    "wechatcomapp_port": 9898,  # 企业微信app的服务端口, 不需要端口转发
+    "wechatcomapp_secret": "",  # 企业微信app的secret
+    "wechatcomapp_agent_id": "",  # 企业微信app的agent_id
+    "wechatcomapp_aes_key": "",  # 企业微信app的aes_key
+```
+
+- 运行程序，在页面中点击保存，保存成功说明验证成功
+
+4.连接个人微信
+
+选择`我的企业`，点击`微信插件`，下面有个邀请关注的二维码。微信扫码后，即可在微信中看到对应企业，在这里你便可以和机器人沟通。
+
+向机器人发送消息，如果日志里出现报错:
+
+```bash
+Error code: 60020, message: "not allow to access from your ip, ...from ip: xx.xx.xx.xx"
+```
+
+意思是IP不可信，需要参考上一步的`企业可信IP`配置，把这里的IP加进去。
+
+~~### Railway部署方式~~（2023-06-08已失效）
+
+~~公众号不能在`Railway`上部署，但企业微信应用[可以](https://railway.app/template/-FHS--?referralCode=RC3znh)!~~
+
+~~填写配置后，将部署完成后的网址```**.railway.app/wxcomapp```，填写在上一步的URL中。发送信息后观察日志，把报错的IP加入到可信IP。（每次重启后都需要加入可信IP）~~
+
+## 测试体验
+
+AIGC开放社区中已经部署了多个可免费使用的Bot，扫描下方的二维码会自动邀请你来体验。
+
+<img width="200" src="../../docs/images/aigcopen.png">

channel/wechatcom/wechatcomapp_channel.py

@@ -0,0 +1,178 @@
+# -*- coding=utf-8 -*-
+import io
+import os
+import time
+
+import requests
+import web
+from wechatpy.enterprise import create_reply, parse_message
+from wechatpy.enterprise.crypto import WeChatCrypto
+from wechatpy.enterprise.exceptions import InvalidCorpIdException
+from wechatpy.exceptions import InvalidSignatureException, WeChatClientException
+
+from bridge.context import Context
+from bridge.reply import Reply, ReplyType
+from channel.chat_channel import ChatChannel
+from channel.wechatcom.wechatcomapp_client import WechatComAppClient
+from channel.wechatcom.wechatcomapp_message import WechatComAppMessage
+from common.log import logger
+from common.singleton import singleton
+from common.utils import compress_imgfile, fsize, split_string_by_utf8_length
+from config import conf, subscribe_msg
+from voice.audio_convert import any_to_amr, split_audio
+
+MAX_UTF8_LEN = 2048
+
+
+@singleton
+class WechatComAppChannel(ChatChannel):
+    NOT_SUPPORT_REPLYTYPE = []
+
+    def __init__(self):
+        super().__init__()
+        self.corp_id = conf().get("wechatcom_corp_id")
+        self.secret = conf().get("wechatcomapp_secret")
+        self.agent_id = conf().get("wechatcomapp_agent_id")
+        self.token = conf().get("wechatcomapp_token")
+        self.aes_key = conf().get("wechatcomapp_aes_key")
+        print(self.corp_id, self.secret, self.agent_id, self.token, self.aes_key)
+        logger.info(
+            "[wechatcom] init: corp_id: {}, secret: {}, agent_id: {}, token: {}, aes_key: {}".format(self.corp_id, self.secret, self.agent_id, self.token, self.aes_key)
+        )
+        self.crypto = WeChatCrypto(self.token, self.aes_key, self.corp_id)
+        self.client = WechatComAppClient(self.corp_id, self.secret)
+
+    def startup(self):
+        # start message listener
+        urls = ("/wxcomapp", "channel.wechatcom.wechatcomapp_channel.Query")
+        app = web.application(urls, globals(), autoreload=False)
+        port = conf().get("wechatcomapp_port", 9898)
+        web.httpserver.runsimple(app.wsgifunc(), ("0.0.0.0", port))
+
+    def send(self, reply: Reply, context: Context):
+        receiver = context["receiver"]
+        if reply.type in [ReplyType.TEXT, ReplyType.ERROR, ReplyType.INFO]:
+            reply_text = reply.content
+            texts = split_string_by_utf8_length(reply_text, MAX_UTF8_LEN)
+            if len(texts) > 1:
+                logger.info("[wechatcom] text too long, split into {} parts".format(len(texts)))
+            for i, text in enumerate(texts):
+                self.client.message.send_text(self.agent_id, receiver, text)
+                if i != len(texts) - 1:
+                    time.sleep(0.5)  # 休眠0.5秒，防止发送过快乱序
+            logger.info("[wechatcom] Do send text to {}: {}".format(receiver, reply_text))
+        elif reply.type == ReplyType.VOICE:
+            try:
+                media_ids = []
+                file_path = reply.content
+                amr_file = os.path.splitext(file_path)[0] + ".amr"
+                any_to_amr(file_path, amr_file)
+                duration, files = split_audio(amr_file, 60 * 1000)
+                if len(files) > 1:
+                    logger.info("[wechatcom] voice too long {}s > 60s , split into {} parts".format(duration / 1000.0, len(files)))
+                for path in files:
+                    response = self.client.media.upload("voice", open(path, "rb"))
+                    logger.debug("[wechatcom] upload voice response: {}".format(response))
+                    media_ids.append(response["media_id"])
+            except WeChatClientException as e:
+                logger.error("[wechatcom] upload voice failed: {}".format(e))
+                return
+            try:
+                os.remove(file_path)
+                if amr_file != file_path:
+                    os.remove(amr_file)
+            except Exception:
+                pass
+            for media_id in media_ids:
+                self.client.message.send_voice(self.agent_id, receiver, media_id)
+                time.sleep(1)
+            logger.info("[wechatcom] sendVoice={}, receiver={}".format(reply.content, receiver))
+        elif reply.type == ReplyType.IMAGE_URL:  # 从网络下载图片
+            img_url = reply.content
+            pic_res = requests.get(img_url, stream=True)
+            image_storage = io.BytesIO()
+            for block in pic_res.iter_content(1024):
+                image_storage.write(block)
+            sz = fsize(image_storage)
+            if sz >= 10 * 1024 * 1024:
+                logger.info("[wechatcom] image too large, ready to compress, sz={}".format(sz))
+                image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
+                logger.info("[wechatcom] image compressed, sz={}".format(fsize(image_storage)))
+            image_storage.seek(0)
+            try:
+                response = self.client.media.upload("image", image_storage)
+                logger.debug("[wechatcom] upload image response: {}".format(response))
+            except WeChatClientException as e:
+                logger.error("[wechatcom] upload image failed: {}".format(e))
+                return
+
+            self.client.message.send_image(self.agent_id, receiver, response["media_id"])
+            logger.info("[wechatcom] sendImage url={}, receiver={}".format(img_url, receiver))
+        elif reply.type == ReplyType.IMAGE:  # 从文件读取图片
+            image_storage = reply.content
+            sz = fsize(image_storage)
+            if sz >= 10 * 1024 * 1024:
+                logger.info("[wechatcom] image too large, ready to compress, sz={}".format(sz))
+                image_storage = compress_imgfile(image_storage, 10 * 1024 * 1024 - 1)
+                logger.info("[wechatcom] image compressed, sz={}".format(fsize(image_storage)))
+            image_storage.seek(0)
+            try:
+                response = self.client.media.upload("image", image_storage)
+                logger.debug("[wechatcom] upload image response: {}".format(response))
+            except WeChatClientException as e:
+                logger.error("[wechatcom] upload image failed: {}".format(e))
+                return
+            self.client.message.send_image(self.agent_id, receiver, response["media_id"])
+            logger.info("[wechatcom] sendImage, receiver={}".format(receiver))
+
+
+class Query:
+    def GET(self):
+        channel = WechatComAppChannel()
+        params = web.input()
+        logger.info("[wechatcom] receive params: {}".format(params))
+        try:
+            signature = params.msg_signature
+            timestamp = params.timestamp
+            nonce = params.nonce
+            echostr = params.echostr
+            echostr = channel.crypto.check_signature(signature, timestamp, nonce, echostr)
+        except InvalidSignatureException:
+            raise web.Forbidden()
+        return echostr
+
+    def POST(self):
+        channel = WechatComAppChannel()
+        params = web.input()
+        logger.info("[wechatcom] receive params: {}".format(params))
+        try:
+            signature = params.msg_signature
+            timestamp = params.timestamp
+            nonce = params.nonce
+            message = channel.crypto.decrypt_message(web.data(), signature, timestamp, nonce)
+        except (InvalidSignatureException, InvalidCorpIdException):
+            raise web.Forbidden()
+        msg = parse_message(message)
+        logger.debug("[wechatcom] receive message: {}, msg= {}".format(message, msg))
+        if msg.type == "event":
+            if msg.event == "subscribe":
+                reply_content = subscribe_msg()
+                if reply_content:
+                    reply = create_reply(reply_content, msg).render()
+                    res = channel.crypto.encrypt_message(reply, nonce, timestamp)
+                    return res
+        else:
+            try:
+                wechatcom_msg = WechatComAppMessage(msg, client=channel.client)
+            except NotImplementedError as e:
+                logger.debug("[wechatcom] " + str(e))
+                return "success"
+            context = channel._compose_context(
+                wechatcom_msg.ctype,
+                wechatcom_msg.content,
+                isgroup=False,
+                msg=wechatcom_msg,
+            )
+            if context:
+                channel.produce(context)
+        return "success"

channel/wechatcom/wechatcomapp_client.py

@@ -0,0 +1,21 @@
+import threading
+import time
+
+from wechatpy.enterprise import WeChatClient
+
+
+class WechatComAppClient(WeChatClient):
+    def __init__(self, corp_id, secret, access_token=None, session=None, timeout=None, auto_retry=True):
+        super(WechatComAppClient, self).__init__(corp_id, secret, access_token, session, timeout, auto_retry)
+        self.fetch_access_token_lock = threading.Lock()
+
+    def fetch_access_token(self):  # 重载父类方法，加锁避免多线程重复获取access_token
+        with self.fetch_access_token_lock:
+            access_token = self.session.get(self.access_token_key)
+            if access_token:
+                if not self.expires_at:
+                    return access_token
+                timestamp = time.time()
+                if self.expires_at - timestamp > 60:
+                    return access_token
+            return super().fetch_access_token()

channel/wechatcom/wechatcomapp_message.py

@@ -0,0 +1,52 @@
+from wechatpy.enterprise import WeChatClient
+
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.log import logger
+from common.tmp_dir import TmpDir
+
+
+class WechatComAppMessage(ChatMessage):
+    def __init__(self, msg, client: WeChatClient, is_group=False):
+        super().__init__(msg)
+        self.msg_id = msg.id
+        self.create_time = msg.time
+        self.is_group = is_group
+
+        if msg.type == "text":
+            self.ctype = ContextType.TEXT
+            self.content = msg.content
+        elif msg.type == "voice":
+            self.ctype = ContextType.VOICE
+            self.content = TmpDir().path() + msg.media_id + "." + msg.format  # content直接存临时目录路径
+
+            def download_voice():
+                # 如果响应状态码是200，则将响应内容写入本地文件
+                response = client.media.download(msg.media_id)
+                if response.status_code == 200:
+                    with open(self.content, "wb") as f:
+                        f.write(response.content)
+                else:
+                    logger.info(f"[wechatcom] Failed to download voice file, {response.content}")
+
+            self._prepare_fn = download_voice
+        elif msg.type == "image":
+            self.ctype = ContextType.IMAGE
+            self.content = TmpDir().path() + msg.media_id + ".png"  # content直接存临时目录路径
+
+            def download_image():
+                # 如果响应状态码是200，则将响应内容写入本地文件
+                response = client.media.download(msg.media_id)
+                if response.status_code == 200:
+                    with open(self.content, "wb") as f:
+                        f.write(response.content)
+                else:
+                    logger.info(f"[wechatcom] Failed to download image file, {response.content}")
+
+            self._prepare_fn = download_image
+        else:
+            raise NotImplementedError("Unsupported message type: Type:{} ".format(msg.type))
+
+        self.from_user_id = msg.source
+        self.to_user_id = msg.target
+        self.other_user_id = msg.source

channel/wechatmp/README.md

@@ -1,57 +1,100 @@
 # 微信公众号channel
 
 鉴于个人微信号在服务器上通过itchat登录有封号风险，这里新增了微信公众号channel，提供无风险的服务。
-目前支持订阅号（个人）和服务号（企业）两种类型的公众号，它们的主要区别就是被动回复和主动回复。
-个人微信订阅号有许多接口限制，目前仅支持最基本的文本对话和语音输入，支持加载插件，支持私有api_key。
-暂未实现图片输入输出、语音输出等交互形式。
+目前支持订阅号和服务号两种类型的公众号，它们都支持文本交互，语音和图片输入。其中个人主体的微信订阅号由于无法通过微信认证，存在回复时间限制，每天的图片和声音回复次数也有限制。
 
 ## 使用方法（订阅号，服务号类似）
 
 在开始部署前，你需要一个拥有公网IP的服务器，以提供微信服务器和我们自己服务器的连接。或者你需要进行内网穿透，否则微信服务器无法将消息发送给我们的服务器。
 
-此外，需要在我们的服务器上安装python的web框架web.py。
+此外，需要在我们的服务器上安装python的web框架web.py和wechatpy。
 以ubuntu为例(在ubuntu 22.04上测试):
 ```
 pip3 install web.py
+pip3 install wechatpy
 ```
 
 然后在[微信公众平台](https://mp.weixin.qq.com)注册一个自己的公众号，类型选择订阅号，主体为个人即可。
 
-然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。这里的`URL`是`example.com/wx`的形式，不可以使用IP，`Token`是你自己编的一个特定的令牌。消息加解密方式目前选择的是明文模式。
+然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。`URL`填写格式为`http://url/wx`，可使用IP（成功几率看脸），`Token`是你自己编的一个特定的令牌。消息加解密方式如果选择了需要加密的模式，需要在配置中填写`wechatmp_aes_key`。
 
 相关的服务器验证代码已经写好，你不需要再添加任何代码。你只需要在本项目根目录的`config.json`中添加
 ```
-"channel_type": "wechatmp",
-"wechatmp_token": "Token",  # 微信公众平台的Token
-"wechatmp_port": 8080,      # 微信公众平台的端口,需要端口转发到80或443
-"wechatmp_app_id": "",      # 微信公众平台的appID，仅服务号需要
-"wechatmp_app_secret": "",  # 微信公众平台的appsecret，仅服务号需要
+"channel_type": "wechatmp",     # 如果通过了微信认证，将"wechatmp"替换为"wechatmp_service"，可极大的优化使用体验
+"wechatmp_token": "xxxx",       # 微信公众平台的Token
+"wechatmp_port": 8080,          # 微信公众平台的端口,需要端口转发到80或443
+"wechatmp_app_id": "xxxx",      # 微信公众平台的appID
+"wechatmp_app_secret": "xxxx",  # 微信公众平台的appsecret
+"wechatmp_aes_key": "",         # 微信公众平台的EncodingAESKey，加密模式需要
+"single_chat_prefix": [""],     # 推荐设置，任意对话都可以触发回复，不添加前缀
+"single_chat_reply_prefix": "", # 推荐设置，回复不设置前缀
+"plugin_trigger_prefix": "&",   # 推荐设置，在手机微信客户端中，$%^等符号与中文连在一起时会自动显示一段较大的间隔，用户体验不好。请不要使用管理员指令前缀"#"，这会造成未知问题。
 ```
-然后运行`python3 app.py`启动web服务器。这里会默认监听8080端口，但是微信公众号的服务器配置只支持80/443端口，有两种方法来解决这个问题。第一个是推荐的方法，使用端口转发命令将80端口转发到8080端口（443同理，注意需要支持SSL，也就是https的访问，在`wechatmp_channel.py`需要修改相应的证书路径）：
+然后运行`python3 app.py`启动web服务器。这里会默认监听8080端口，但是微信公众号的服务器配置只支持80/443端口，有两种方法来解决这个问题。第一个是推荐的方法，使用端口转发命令将80端口转发到8080端口：
 ```
 sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
 sudo iptables-save > /etc/iptables/rules.v4
 ```
-第二个方法是让python程序直接监听80端口。这样可能会导致权限问题，在linux上需要使用`sudo`。然而这会导致后续缓存文件的权限问题，因此不是推荐的方法。
-最后在刚才的“服务器配置”中点击`提交`即可验证你的服务器。
+第二个方法是让python程序直接监听80端口，在配置文件中设置`"wechatmp_port": 80` ，在linux上需要使用`sudo python3 app.py`启动程序。然而这会导致一系列环境和权限问题，因此不是推荐的方法。
 
+443端口同理，注意需要支持SSL，也就是https的访问，在`wechatmp_channel.py`中需要修改相应的证书路径。
+
+程序启动并监听端口后，在刚才的“服务器配置”中点击`提交`即可验证你的服务器。
 随后在[微信公众平台](https://mp.weixin.qq.com)启用服务器，关闭手动填写规则的自动回复，即可实现ChatGPT的自动回复。
 
+之后需要在公众号开发信息下将本机IP加入到IP白名单。
+
+不然在启用后，发送语音、图片等消息可能会遇到如下报错：
+```
+'errcode': 40164, 'errmsg': 'invalid ip xx.xx.xx.xx not in whitelist rid
+```
+
+
 ## 个人微信公众号的限制
 由于人微信公众号不能通过微信认证，所以没有客服接口，因此公众号无法主动发出消息，只能被动回复。而微信官方对被动回复有5秒的时间限制，最多重试2次，因此最多只有15秒的自动回复时间窗口。因此如果问题比较复杂或者我们的服务器比较忙，ChatGPT的回答就没办法及时回复给用户。为了解决这个问题，这里做了回答缓存，它需要你在回复超时后，再次主动发送任意文字（例如1）来尝试拿到回答缓存。为了优化使用体验，目前设置了两分钟（120秒）的timeout，用户在至多两分钟后即可得到查询到回复或者错误原因。
 
-另外，由于微信官方的限制，自动回复有长度限制。因此这里将ChatGPT的回答拆分，分成每段600字回复（限制大约在700字）。
+另外，由于微信官方的限制，自动回复有长度限制。因此这里将ChatGPT的回答进行了拆分，以满足限制。
 
 ## 私有api_key
-公共api有访问频率限制（免费账号每分钟最多20次ChatGPT的API调用），这在服务多人的时候会遇到问题。因此这里多加了一个设置私有api_key的功能。目前通过godcmd插件的命令来设置私有api_key。
+公共api有访问频率限制（免费账号每分钟最多3次ChatGPT的API调用），这在服务多人的时候会遇到问题。因此这里多加了一个设置私有api_key的功能。目前通过godcmd插件的命令来设置私有api_key。
 
 ## 语音输入
 利用微信自带的语音识别功能，提供语音输入能力。需要在公众号管理页面的“设置与开发”->“接口权限”页面开启“接收语音识别结果”。
 
-## 测试范围
-目前在`RoboStyle`这个公众号上进行了测试（基于[wechatmp分支](https://github.com/JS00000/chatgpt-on-wechat/tree/wechatmp)），感兴趣的可以关注并体验。开启了godcmd, Banwords, role, dungeon, finish这五个插件，其他的插件还没有测试。百度的接口暂未测试。语音对话没有测试。图片直接以链接形式回复（没有临时素材上传接口的权限）。
+## 语音回复
+请在配置文件中添加以下词条：
+```
+  "voice_reply_voice": true,
+```
+这样公众号将会用语音回复语音消息，实现语音对话。
+
+默认的语音合成引擎是`google`，它是免费使用的。
+
+如果要选择其他的语音合成引擎，请添加以下配置项：
+```
+"text_to_voice": "pytts"
+```
+
+pytts是本地的语音合成引擎。还支持baidu,azure，这些你需要自行配置相关的依赖和key。
+
+如果使用pytts，在ubuntu上需要安装如下依赖：
+```
+sudo apt update
+sudo apt install espeak
+sudo apt install ffmpeg
+python3 -m pip install pyttsx3
+```
+不是很建议开启pytts语音回复，因为它是离线本地计算，算的慢会拖垮服务器，且声音不好听。
+
+## 图片回复
+现在认证公众号和非认证公众号都可以实现的图片和语音回复。但是非认证公众号使用了永久素材接口，每天有1000次的调用上限（每个月有10次重置机会，程序中已设定遇到上限会自动重置），且永久素材库存也有上限。因此对于非认证公众号，我们会在回复图片或者语音消息后的10秒内从永久素材库存内删除该素材。
+
+## 测试
+目前在`RoboStyle`这个公众号上进行了测试（基于[wechatmp分支](https://github.com/JS00000/chatgpt-on-wechat/tree/wechatmp)），感兴趣的可以关注并体验。开启了godcmd, Banwords, role, dungeon, finish这五个插件，其他的插件还没有详尽测试。百度的接口暂未测试。[wechatmp-stable分支](https://github.com/JS00000/chatgpt-on-wechat/tree/wechatmp-stable)是较稳定的上个版本，但也缺少最新的功能支持。
 
 ## TODO
-* 服务号交互完善
-* 服务号使用临时素材接口，提供图片回复能力
-* 插件测试
+ - [x] 语音输入
+ - [x] 图片输入
+ - [x] 使用临时素材接口提供认证公众号的图片和语音回复
+ - [x] 使用永久素材接口提供未认证公众号的图片和语音回复
+ - [ ] 高并发支持

channel/wechatmp/ServiceAccount.py

@@ -1,70 +0,0 @@
-import time
-
-import web
-
-import channel.wechatmp.receive as receive
-import channel.wechatmp.reply as reply
-from bridge.context import *
-from channel.wechatmp.common import *
-from channel.wechatmp.wechatmp_channel import WechatMPChannel
-from common.log import logger
-from config import conf
-
-
-# This class is instantiated once per query
-class Query:
-    def GET(self):
-        return verify_server(web.input())
-
-    def POST(self):
-        # Make sure to return the instance that first created, @singleton will do that.
-        channel = WechatMPChannel()
-        try:
-            webData = web.data()
-            # logger.debug("[wechatmp] Receive request:\n" + webData.decode("utf-8"))
-            wechatmp_msg = receive.parse_xml(webData)
-            if wechatmp_msg.msg_type == "text" or wechatmp_msg.msg_type == "voice":
-                from_user = wechatmp_msg.from_user_id
-                message = wechatmp_msg.content.decode("utf-8")
-                message_id = wechatmp_msg.msg_id
-
-                logger.info(
-                    "[wechatmp] {}:{} Receive post query {} {}: {}".format(
-                        web.ctx.env.get("REMOTE_ADDR"),
-                        web.ctx.env.get("REMOTE_PORT"),
-                        from_user,
-                        message_id,
-                        message,
-                    )
-                )
-                context = channel._compose_context(
-                    ContextType.TEXT, message, isgroup=False, msg=wechatmp_msg
-                )
-                if context:
-                    # set private openai_api_key
-                    # if from_user is not changed in itchat, this can be placed at chat_channel
-                    user_data = conf().get_user_data(from_user)
-                    context["openai_api_key"] = user_data.get(
-                        "openai_api_key"
-                    )  # None or user openai_api_key
-                    channel.produce(context)
-                # The reply will be sent by channel.send() in another thread
-                return "success"
-
-            elif wechatmp_msg.msg_type == "event":
-                logger.info(
-                    "[wechatmp] Event {} from {}".format(
-                        wechatmp_msg.Event, wechatmp_msg.from_user_id
-                    )
-                )
-                content = subscribe_msg()
-                replyMsg = reply.TextMsg(
-                    wechatmp_msg.from_user_id, wechatmp_msg.to_user_id, content
-                )
-                return replyMsg.send()
-            else:
-                logger.info("暂且不处理")
-                return "success"
-        except Exception as exc:
-            logger.exception(exc)
-            return exc

channel/wechatmp/SubscribeAccount.py

@@ -1,232 +0,0 @@
-import time
-
-import web
-
-import channel.wechatmp.receive as receive
-import channel.wechatmp.reply as reply
-from bridge.context import *
-from channel.wechatmp.common import *
-from channel.wechatmp.wechatmp_channel import WechatMPChannel
-from common.log import logger
-from config import conf
-
-
-# This class is instantiated once per query
-class Query:
-    def GET(self):
-        return verify_server(web.input())
-
-    def POST(self):
-        # Make sure to return the instance that first created, @singleton will do that.
-        channel = WechatMPChannel()
-        try:
-            query_time = time.time()
-            webData = web.data()
-            logger.debug("[wechatmp] Receive request:\n" + webData.decode("utf-8"))
-            wechatmp_msg = receive.parse_xml(webData)
-            if wechatmp_msg.msg_type == "text" or wechatmp_msg.msg_type == "voice":
-                from_user = wechatmp_msg.from_user_id
-                to_user = wechatmp_msg.to_user_id
-                message = wechatmp_msg.content.decode("utf-8")
-                message_id = wechatmp_msg.msg_id
-
-                logger.info(
-                    "[wechatmp] {}:{} Receive post query {} {}: {}".format(
-                        web.ctx.env.get("REMOTE_ADDR"),
-                        web.ctx.env.get("REMOTE_PORT"),
-                        from_user,
-                        message_id,
-                        message,
-                    )
-                )
-                supported = True
-                if "【收到不支持的消息类型，暂无法显示】" in message:
-                    supported = False  # not supported, used to refresh
-                cache_key = from_user
-
-                reply_text = ""
-                # New request
-                if (
-                    cache_key not in channel.cache_dict
-                    and cache_key not in channel.running
-                ):
-                    # The first query begin, reset the cache
-                    context = channel._compose_context(
-                        ContextType.TEXT, message, isgroup=False, msg=wechatmp_msg
-                    )
-                    logger.debug(
-                        "[wechatmp] context: {} {}".format(context, wechatmp_msg)
-                    )
-                    if message_id in channel.received_msgs:  # received and finished
-                        # no return because of bandwords or other reasons
-                        return "success"
-                    if supported and context:
-                        # set private openai_api_key
-                        # if from_user is not changed in itchat, this can be placed at chat_channel
-                        user_data = conf().get_user_data(from_user)
-                        context["openai_api_key"] = user_data.get(
-                            "openai_api_key"
-                        )  # None or user openai_api_key
-                        channel.received_msgs[message_id] = wechatmp_msg
-                        channel.running.add(cache_key)
-                        channel.produce(context)
-                    else:
-                        trigger_prefix = conf().get("single_chat_prefix", [""])[0]
-                        if trigger_prefix or not supported:
-                            if trigger_prefix:
-                                content = textwrap.dedent(
-                                    f"""\
-                                    请输入'{trigger_prefix}'接你想说的话跟我说话。
-                                    例如:
-                                    {trigger_prefix}你好，很高兴见到你。"""
-                                )
-                            else:
-                                content = textwrap.dedent(
-                                    """\
-                                    你好，很高兴见到你。
-                                    请跟我说话吧。"""
-                                )
-                        else:
-                            logger.error(f"[wechatmp] unknown error")
-                            content = textwrap.dedent(
-                                """\
-                                未知错误，请稍后再试"""
-                            )
-                        replyMsg = reply.TextMsg(
-                            wechatmp_msg.from_user_id, wechatmp_msg.to_user_id, content
-                        )
-                        return replyMsg.send()
-                    channel.query1[cache_key] = False
-                    channel.query2[cache_key] = False
-                    channel.query3[cache_key] = False
-                # User request again, and the answer is not ready
-                elif (
-                    cache_key in channel.running
-                    and channel.query1.get(cache_key) == True
-                    and channel.query2.get(cache_key) == True
-                    and channel.query3.get(cache_key) == True
-                ):
-                    channel.query1[
-                        cache_key
-                    ] = False  # To improve waiting experience, this can be set to True.
-                    channel.query2[
-                        cache_key
-                    ] = False  # To improve waiting experience, this can be set to True.
-                    channel.query3[cache_key] = False
-                # User request again, and the answer is ready
-                elif cache_key in channel.cache_dict:
-                    # Skip the waiting phase
-                    channel.query1[cache_key] = True
-                    channel.query2[cache_key] = True
-                    channel.query3[cache_key] = True
-
-                assert not (
-                    cache_key in channel.cache_dict and cache_key in channel.running
-                )
-
-                if channel.query1.get(cache_key) == False:
-                    # The first query from wechat official server
-                    logger.debug("[wechatmp] query1 {}".format(cache_key))
-                    channel.query1[cache_key] = True
-                    cnt = 0
-                    while cache_key in channel.running and cnt < 45:
-                        cnt = cnt + 1
-                        time.sleep(0.1)
-                    if cnt == 45:
-                        # waiting for timeout (the POST query will be closed by wechat official server)
-                        time.sleep(1)
-                        # and do nothing
-                        return
-                    else:
-                        pass
-                elif channel.query2.get(cache_key) == False:
-                    # The second query from wechat official server
-                    logger.debug("[wechatmp] query2 {}".format(cache_key))
-                    channel.query2[cache_key] = True
-                    cnt = 0
-                    while cache_key in channel.running and cnt < 45:
-                        cnt = cnt + 1
-                        time.sleep(0.1)
-                    if cnt == 45:
-                        # waiting for timeout (the POST query will be closed by wechat official server)
-                        time.sleep(1)
-                        # and do nothing
-                        return
-                    else:
-                        pass
-                elif channel.query3.get(cache_key) == False:
-                    # The third query from wechat official server
-                    logger.debug("[wechatmp] query3 {}".format(cache_key))
-                    channel.query3[cache_key] = True
-                    cnt = 0
-                    while cache_key in channel.running and cnt < 40:
-                        cnt = cnt + 1
-                        time.sleep(0.1)
-                    if cnt == 40:
-                        # Have waiting for 3x5 seconds
-                        # return timeout message
-                        reply_text = "【正在思考中，回复任意文字尝试获取回复】"
-                        logger.info(
-                            "[wechatmp] Three queries has finished For {}: {}".format(
-                                from_user, message_id
-                            )
-                        )
-                        replyPost = reply.TextMsg(from_user, to_user, reply_text).send()
-                        return replyPost
-                    else:
-                        pass
-
-                if (
-                    cache_key not in channel.cache_dict
-                    and cache_key not in channel.running
-                ):
-                    # no return because of bandwords or other reasons
-                    return "success"
-
-                # if float(time.time()) - float(query_time) > 4.8:
-                #     reply_text = "【正在思考中，回复任意文字尝试获取回复】"
-                #     logger.info("[wechatmp] Timeout for {} {}, return".format(from_user, message_id))
-                #     replyPost = reply.TextMsg(from_user, to_user, reply_text).send()
-                #     return replyPost
-
-                if cache_key in channel.cache_dict:
-                    content = channel.cache_dict[cache_key]
-                    if len(content.encode("utf8")) <= MAX_UTF8_LEN:
-                        reply_text = channel.cache_dict[cache_key]
-                        channel.cache_dict.pop(cache_key)
-                    else:
-                        continue_text = "\n【未完待续，回复任意文字以继续】"
-                        splits = split_string_by_utf8_length(
-                            content,
-                            MAX_UTF8_LEN - len(continue_text.encode("utf-8")),
-                            max_split=1,
-                        )
-                        reply_text = splits[0] + continue_text
-                        channel.cache_dict[cache_key] = splits[1]
-                logger.info(
-                    "[wechatmp] {}:{} Do send {}".format(
-                        web.ctx.env.get("REMOTE_ADDR"),
-                        web.ctx.env.get("REMOTE_PORT"),
-                        reply_text,
-                    )
-                )
-                replyPost = reply.TextMsg(from_user, to_user, reply_text).send()
-                return replyPost
-
-            elif wechatmp_msg.msg_type == "event":
-                logger.info(
-                    "[wechatmp] Event {} from {}".format(
-                        wechatmp_msg.content, wechatmp_msg.from_user_id
-                    )
-                )
-                content = subscribe_msg()
-                replyMsg = reply.TextMsg(
-                    wechatmp_msg.from_user_id, wechatmp_msg.to_user_id, content
-                )
-                return replyMsg.send()
-            else:
-                logger.info("暂且不处理")
-                return "success"
-        except Exception as exc:
-            logger.exception(exc)
-            return exc

channel/wechatmp/active_reply.py

@@ -0,0 +1,75 @@
+import time
+
+import web
+from wechatpy import parse_message
+from wechatpy.replies import create_reply
+
+from bridge.context import *
+from bridge.reply import *
+from channel.wechatmp.common import *
+from channel.wechatmp.wechatmp_channel import WechatMPChannel
+from channel.wechatmp.wechatmp_message import WeChatMPMessage
+from common.log import logger
+from config import conf, subscribe_msg
+
+
+# This class is instantiated once per query
+class Query:
+    def GET(self):
+        return verify_server(web.input())
+
+    def POST(self):
+        # Make sure to return the instance that first created, @singleton will do that.
+        try:
+            args = web.input()
+            verify_server(args)
+            channel = WechatMPChannel()
+            message = web.data()
+            encrypt_func = lambda x: x
+            if args.get("encrypt_type") == "aes":
+                logger.debug("[wechatmp] Receive encrypted post data:\n" + message.decode("utf-8"))
+                if not channel.crypto:
+                    raise Exception("Crypto not initialized, Please set wechatmp_aes_key in config.json")
+                message = channel.crypto.decrypt_message(message, args.msg_signature, args.timestamp, args.nonce)
+                encrypt_func = lambda x: channel.crypto.encrypt_message(x, args.nonce, args.timestamp)
+            else:
+                logger.debug("[wechatmp] Receive post data:\n" + message.decode("utf-8"))
+            msg = parse_message(message)
+            if msg.type in ["text", "voice", "image"]:
+                wechatmp_msg = WeChatMPMessage(msg, client=channel.client)
+                from_user = wechatmp_msg.from_user_id
+                content = wechatmp_msg.content
+                message_id = wechatmp_msg.msg_id
+
+                logger.info(
+                    "[wechatmp] {}:{} Receive post query {} {}: {}".format(
+                        web.ctx.env.get("REMOTE_ADDR"),
+                        web.ctx.env.get("REMOTE_PORT"),
+                        from_user,
+                        message_id,
+                        content,
+                    )
+                )
+                if msg.type == "voice" and wechatmp_msg.ctype == ContextType.TEXT and conf().get("voice_reply_voice", False):
+                    context = channel._compose_context(wechatmp_msg.ctype, content, isgroup=False, desire_rtype=ReplyType.VOICE, msg=wechatmp_msg)
+                else:
+                    context = channel._compose_context(wechatmp_msg.ctype, content, isgroup=False, msg=wechatmp_msg)
+                if context:
+                    channel.produce(context)
+                # The reply will be sent by channel.send() in another thread
+                return "success"
+            elif msg.type == "event":
+                logger.info("[wechatmp] Event {} from {}".format(msg.event, msg.source))
+                if msg.event in ["subscribe", "subscribe_scan"]:
+                    reply_text = subscribe_msg()
+                    if reply_text:
+                        replyPost = create_reply(reply_text, msg)
+                        return encrypt_func(replyPost.render())
+                else:
+                    return "success"
+            else:
+                logger.info("暂且不处理")
+            return "success"
+        except Exception as exc:
+            logger.exception(exc)
+            return exc

channel/wechatmp/common.py

@@ -1,5 +1,7 @@
-import hashlib
-import textwrap
+import web
+from wechatpy.crypto import WeChatCrypto
+from wechatpy.exceptions import InvalidSignatureException
+from wechatpy.utils import check_signature
 
 from config import conf
 
@@ -12,57 +14,14 @@ class WeChatAPIException(Exception):
 
 def verify_server(data):
     try:
-        if len(data) == 0:
-            return "None"
         signature = data.signature
         timestamp = data.timestamp
         nonce = data.nonce
-        echostr = data.echostr
+        echostr = data.get("echostr", None)
         token = conf().get("wechatmp_token")  # 请按照公众平台官网\基本配置中信息填写
-
-        data_list = [token, timestamp, nonce]
-        data_list.sort()
-        sha1 = hashlib.sha1()
-        # map(sha1.update, data_list) #python2
-        sha1.update("".join(data_list).encode("utf-8"))
-        hashcode = sha1.hexdigest()
-        print("handle/GET func: hashcode, signature: ", hashcode, signature)
-        if hashcode == signature:
-            return echostr
-        else:
-            return ""
-    except Exception as Argument:
-        return Argument
-
-
-def subscribe_msg():
-    trigger_prefix = conf().get("single_chat_prefix", [""])[0]
-    msg = textwrap.dedent(
-        f"""\
-                    感谢您的关注！
-                    这里是ChatGPT，可以自由对话。
-                    资源有限，回复较慢，请勿着急。
-                    支持通用表情输入。
-                    暂时不支持图片输入。
-                    支持图片输出，画字开头的问题将回复图片链接。
-                    支持角色扮演和文字冒险两种定制模式对话。
-                    输入'{trigger_prefix}#帮助' 查看详细指令。"""
-    )
-    return msg
-
-
-def split_string_by_utf8_length(string, max_length, max_split=0):
-    encoded = string.encode("utf-8")
-    start, end = 0, 0
-    result = []
-    while end < len(encoded):
-        if max_split > 0 and len(result) >= max_split:
-            result.append(encoded[start:].decode("utf-8"))
-            break
-        end = start + max_length
-        # 如果当前字节不是 UTF-8 编码的开始字节，则向前查找直到找到开始字节为止
-        while end < len(encoded) and (encoded[end] & 0b11000000) == 0b10000000:
-            end -= 1
-        result.append(encoded[start:end].decode("utf-8"))
-        start = end
-    return result
+        check_signature(token, signature, timestamp, nonce)
+        return echostr
+    except InvalidSignatureException:
+        raise web.Forbidden("Invalid signature")
+    except Exception as e:
+        raise web.Forbidden(str(e))

channel/wechatmp/passive_reply.py

@@ -0,0 +1,209 @@
+import asyncio
+import time
+
+import web
+from wechatpy import parse_message
+from wechatpy.replies import ImageReply, VoiceReply, create_reply
+import textwrap
+from bridge.context import *
+from bridge.reply import *
+from channel.wechatmp.common import *
+from channel.wechatmp.wechatmp_channel import WechatMPChannel
+from channel.wechatmp.wechatmp_message import WeChatMPMessage
+from common.log import logger
+from common.utils import split_string_by_utf8_length
+from config import conf, subscribe_msg
+
+
+# This class is instantiated once per query
+class Query:
+    def GET(self):
+        return verify_server(web.input())
+
+    def POST(self):
+        try:
+            args = web.input()
+            verify_server(args)
+            request_time = time.time()
+            channel = WechatMPChannel()
+            message = web.data()
+            encrypt_func = lambda x: x
+            if args.get("encrypt_type") == "aes":
+                logger.debug("[wechatmp] Receive encrypted post data:\n" + message.decode("utf-8"))
+                if not channel.crypto:
+                    raise Exception("Crypto not initialized, Please set wechatmp_aes_key in config.json")
+                message = channel.crypto.decrypt_message(message, args.msg_signature, args.timestamp, args.nonce)
+                encrypt_func = lambda x: channel.crypto.encrypt_message(x, args.nonce, args.timestamp)
+            else:
+                logger.debug("[wechatmp] Receive post data:\n" + message.decode("utf-8"))
+            msg = parse_message(message)
+            if msg.type in ["text", "voice", "image"]:
+                wechatmp_msg = WeChatMPMessage(msg, client=channel.client)
+                from_user = wechatmp_msg.from_user_id
+                content = wechatmp_msg.content
+                message_id = wechatmp_msg.msg_id
+
+                supported = True
+                if "【收到不支持的消息类型，暂无法显示】" in content:
+                    supported = False  # not supported, used to refresh
+
+                # New request
+                if (
+                    from_user not in channel.cache_dict
+                    and from_user not in channel.running
+                    or content.startswith("#")
+                    and message_id not in channel.request_cnt  # insert the godcmd
+                ):
+                    # The first query begin
+                    if msg.type == "voice" and wechatmp_msg.ctype == ContextType.TEXT and conf().get("voice_reply_voice", False):
+                        context = channel._compose_context(wechatmp_msg.ctype, content, isgroup=False, desire_rtype=ReplyType.VOICE, msg=wechatmp_msg)
+                    else:
+                        context = channel._compose_context(wechatmp_msg.ctype, content, isgroup=False, msg=wechatmp_msg)
+                    logger.debug("[wechatmp] context: {} {} {}".format(context, wechatmp_msg, supported))
+
+                    if supported and context:
+                        channel.running.add(from_user)
+                        channel.produce(context)
+                    else:
+                        trigger_prefix = conf().get("single_chat_prefix", [""])[0]
+                        if trigger_prefix or not supported:
+                            if trigger_prefix:
+                                reply_text = textwrap.dedent(
+                                    f"""\
+                                    请输入'{trigger_prefix}'接你想说的话跟我说话。
+                                    例如:
+                                    {trigger_prefix}你好，很高兴见到你。"""
+                                )
+                            else:
+                                reply_text = textwrap.dedent(
+                                    """\
+                                    你好，很高兴见到你。
+                                    请跟我说话吧。"""
+                                )
+                        else:
+                            logger.error(f"[wechatmp] unknown error")
+                            reply_text = textwrap.dedent(
+                                """\
+                                未知错误，请稍后再试"""
+                            )
+
+                        replyPost = create_reply(reply_text, msg)
+                        return encrypt_func(replyPost.render())
+
+                # Wechat official server will request 3 times (5 seconds each), with the same message_id.
+                # Because the interval is 5 seconds, here assumed that do not have multithreading problems.
+                request_cnt = channel.request_cnt.get(message_id, 0) + 1
+                channel.request_cnt[message_id] = request_cnt
+                logger.info(
+                    "[wechatmp] Request {} from {} {} {}:{}\n{}".format(
+                        request_cnt, from_user, message_id, web.ctx.env.get("REMOTE_ADDR"), web.ctx.env.get("REMOTE_PORT"), content
+                    )
+                )
+
+                task_running = True
+                waiting_until = request_time + 4
+                while time.time() < waiting_until:
+                    if from_user in channel.running:
+                        time.sleep(0.1)
+                    else:
+                        task_running = False
+                        break
+
+                reply_text = ""
+                if task_running:
+                    if request_cnt < 3:
+                        # waiting for timeout (the POST request will be closed by Wechat official server)
+                        time.sleep(2)
+                        # and do nothing, waiting for the next request
+                        return "success"
+                    else:  # request_cnt == 3:
+                        # return timeout message
+                        reply_text = "【正在思考中，回复任意文字尝试获取回复】"
+                        replyPost = create_reply(reply_text, msg)
+                        return encrypt_func(replyPost.render())
+
+                # reply is ready
+                channel.request_cnt.pop(message_id)
+
+                # no return because of bandwords or other reasons
+                if from_user not in channel.cache_dict and from_user not in channel.running:
+                    return "success"
+
+                # Only one request can access to the cached data
+                try:
+                    (reply_type, reply_content) = channel.cache_dict.pop(from_user)
+                except KeyError:
+                    return "success"
+
+                if reply_type == "text":
+                    if len(reply_content.encode("utf8")) <= MAX_UTF8_LEN:
+                        reply_text = reply_content
+                    else:
+                        continue_text = "\n【未完待续，回复任意文字以继续】"
+                        splits = split_string_by_utf8_length(
+                            reply_content,
+                            MAX_UTF8_LEN - len(continue_text.encode("utf-8")),
+                            max_split=1,
+                        )
+                        reply_text = splits[0] + continue_text
+                        channel.cache_dict[from_user] = ("text", splits[1])
+
+                    logger.info(
+                        "[wechatmp] Request {} do send to {} {}: {}\n{}".format(
+                            request_cnt,
+                            from_user,
+                            message_id,
+                            content,
+                            reply_text,
+                        )
+                    )
+                    replyPost = create_reply(reply_text, msg)
+                    return encrypt_func(replyPost.render())
+
+                elif reply_type == "voice":
+                    media_id = reply_content
+                    asyncio.run_coroutine_threadsafe(channel.delete_media(media_id), channel.delete_media_loop)
+                    logger.info(
+                        "[wechatmp] Request {} do send to {} {}: {} voice media_id {}".format(
+                            request_cnt,
+                            from_user,
+                            message_id,
+                            content,
+                            media_id,
+                        )
+                    )
+                    replyPost = VoiceReply(message=msg)
+                    replyPost.media_id = media_id
+                    return encrypt_func(replyPost.render())
+
+                elif reply_type == "image":
+                    media_id = reply_content
+                    asyncio.run_coroutine_threadsafe(channel.delete_media(media_id), channel.delete_media_loop)
+                    logger.info(
+                        "[wechatmp] Request {} do send to {} {}: {} image media_id {}".format(
+                            request_cnt,
+                            from_user,
+                            message_id,
+                            content,
+                            media_id,
+                        )
+                    )
+                    replyPost = ImageReply(message=msg)
+                    replyPost.media_id = media_id
+                    return encrypt_func(replyPost.render())
+
+            elif msg.type == "event":
+                logger.info("[wechatmp] Event {} from {}".format(msg.event, msg.source))
+                if msg.event in ["subscribe", "subscribe_scan"]:
+                    reply_text = subscribe_msg()
+                    if reply_text:
+                        replyPost = create_reply(reply_text, msg)
+                        return encrypt_func(replyPost.render())
+                else:
+                    return "success"
+            else:
+                logger.info("暂且不处理")
+            return "success"
+        except Exception as exc:
+            logger.exception(exc)
+            return exc

channel/wechatmp/receive.py

@@ -1,47 +0,0 @@
-# -*- coding: utf-8 -*-#
-# filename: receive.py
-import xml.etree.ElementTree as ET
-
-from bridge.context import ContextType
-from channel.chat_message import ChatMessage
-from common.log import logger
-
-
-def parse_xml(web_data):
-    if len(web_data) == 0:
-        return None
-    xmlData = ET.fromstring(web_data)
-    return WeChatMPMessage(xmlData)
-
-
-class WeChatMPMessage(ChatMessage):
-    def __init__(self, xmlData):
-        super().__init__(xmlData)
-        self.to_user_id = xmlData.find("ToUserName").text
-        self.from_user_id = xmlData.find("FromUserName").text
-        self.create_time = xmlData.find("CreateTime").text
-        self.msg_type = xmlData.find("MsgType").text
-        try:
-            self.msg_id = xmlData.find("MsgId").text
-        except:
-            self.msg_id = self.from_user_id + self.create_time
-        self.is_group = False
-
-        # reply to other_user_id
-        self.other_user_id = self.from_user_id
-
-        if self.msg_type == "text":
-            self.ctype = ContextType.TEXT
-            self.content = xmlData.find("Content").text.encode("utf-8")
-        elif self.msg_type == "voice":
-            self.ctype = ContextType.TEXT
-            self.content = xmlData.find("Recognition").text.encode("utf-8")  # 接收语音识别结果
-        elif self.msg_type == "image":
-            # not implemented
-            self.pic_url = xmlData.find("PicUrl").text
-            self.media_id = xmlData.find("MediaId").text
-        elif self.msg_type == "event":
-            self.content = xmlData.find("Event").text
-        else:  # video, shortvideo, location, link
-            # not implemented
-            pass

channel/wechatmp/reply.py

@@ -1,55 +0,0 @@
-# -*- coding: utf-8 -*-#
-# filename: reply.py
-import time
-
-
-class Msg(object):
-    def __init__(self):
-        pass
-
-    def send(self):
-        return "success"
-
-
-class TextMsg(Msg):
-    def __init__(self, toUserName, fromUserName, content):
-        self.__dict = dict()
-        self.__dict["ToUserName"] = toUserName
-        self.__dict["FromUserName"] = fromUserName
-        self.__dict["CreateTime"] = int(time.time())
-        self.__dict["Content"] = content
-
-    def send(self):
-        XmlForm = """
-            <xml>
-                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
-                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
-                <CreateTime>{CreateTime}</CreateTime>
-                <MsgType><![CDATA[text]]></MsgType>
-                <Content><![CDATA[{Content}]]></Content>
-            </xml>
-            """
-        return XmlForm.format(**self.__dict)
-
-
-class ImageMsg(Msg):
-    def __init__(self, toUserName, fromUserName, mediaId):
-        self.__dict = dict()
-        self.__dict["ToUserName"] = toUserName
-        self.__dict["FromUserName"] = fromUserName
-        self.__dict["CreateTime"] = int(time.time())
-        self.__dict["MediaId"] = mediaId
-
-    def send(self):
-        XmlForm = """
-            <xml>
-                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
-                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
-                <CreateTime>{CreateTime}</CreateTime>
-                <MsgType><![CDATA[image]]></MsgType>
-                <Image>
-                <MediaId><![CDATA[{MediaId}]]></MediaId>
-                </Image>
-            </xml>
-            """
-        return XmlForm.format(**self.__dict)

channel/wechatmp/wechatmp_channel.py

@@ -1,19 +1,26 @@
 # -*- coding: utf-8 -*-
-import json
+import asyncio
+import imghdr
+import io
+import os
 import threading
 import time
 
 import requests
 import web
+from wechatpy.crypto import WeChatCrypto
+from wechatpy.exceptions import WeChatClientException
 
 from bridge.context import *
 from bridge.reply import *
 from channel.chat_channel import ChatChannel
 from channel.wechatmp.common import *
-from common.expired_dict import ExpiredDict
+from channel.wechatmp.wechatmp_client import WechatMPClient
 from common.log import logger
 from common.singleton import singleton
+from common.utils import split_string_by_utf8_length
 from config import conf
+from voice.audio_convert import any_to_mp3
 
 # If using SSL, uncomment the following lines, and modify the certificate path.
 # from cheroot.server import HTTPServer
@@ -28,111 +35,182 @@ class WechatMPChannel(ChatChannel):
     def __init__(self, passive_reply=True):
         super().__init__()
         self.passive_reply = passive_reply
-        self.running = set()
-        self.received_msgs = ExpiredDict(60 * 60 * 24)
+        self.NOT_SUPPORT_REPLYTYPE = []
+        appid = conf().get("wechatmp_app_id")
+        secret = conf().get("wechatmp_app_secret")
+        token = conf().get("wechatmp_token")
+        aes_key = conf().get("wechatmp_aes_key")
+        self.client = WechatMPClient(appid, secret)
+        self.crypto = None
+        if aes_key:
+            self.crypto = WeChatCrypto(token, aes_key, appid)
         if self.passive_reply:
-            self.NOT_SUPPORT_REPLYTYPE = [ReplyType.IMAGE, ReplyType.VOICE]
+            # Cache the reply to the user's first message
             self.cache_dict = dict()
-            self.query1 = dict()
-            self.query2 = dict()
-            self.query3 = dict()
-        else:
-            # TODO support image
-            self.NOT_SUPPORT_REPLYTYPE = [ReplyType.IMAGE, ReplyType.VOICE]
-            self.app_id = conf().get("wechatmp_app_id")
-            self.app_secret = conf().get("wechatmp_app_secret")
-            self.access_token = None
-            self.access_token_expires_time = 0
-            self.access_token_lock = threading.Lock()
-            self.get_access_token()
+            # Record whether the current message is being processed
+            self.running = set()
+            # Count the request from wechat official server by message_id
+            self.request_cnt = dict()
+            # The permanent media need to be deleted to avoid media number limit
+            self.delete_media_loop = asyncio.new_event_loop()
+            t = threading.Thread(target=self.start_loop, args=(self.delete_media_loop,))
+            t.setDaemon(True)
+            t.start()
 
     def startup(self):
         if self.passive_reply:
-            urls = ("/wx", "channel.wechatmp.SubscribeAccount.Query")
+            urls = ("/wx", "channel.wechatmp.passive_reply.Query")
         else:
-            urls = ("/wx", "channel.wechatmp.ServiceAccount.Query")
+            urls = ("/wx", "channel.wechatmp.active_reply.Query")
         app = web.application(urls, globals(), autoreload=False)
         port = conf().get("wechatmp_port", 8080)
         web.httpserver.runsimple(app.wsgifunc(), ("0.0.0.0", port))
 
-    def wechatmp_request(self, method, url, **kwargs):
-        r = requests.request(method=method, url=url, **kwargs)
-        r.raise_for_status()
-        r.encoding = "utf-8"
-        ret = r.json()
-        if "errcode" in ret and ret["errcode"] != 0:
-            raise WeChatAPIException("{}".format(ret))
-        return ret
+    def start_loop(self, loop):
+        asyncio.set_event_loop(loop)
+        loop.run_forever()
 
-    def get_access_token(self):
-        # return the access_token
-        if self.access_token:
-            if self.access_token_expires_time - time.time() > 60:
-                return self.access_token
-
-        # Get new access_token
-        # Do not request access_token in parallel! Only the last obtained is valid.
-        if self.access_token_lock.acquire(blocking=False):
-            # Wait for other threads that have previously obtained access_token to complete the request
-            # This happens every 2 hours, so it doesn't affect the experience very much
-            time.sleep(1)
-            self.access_token = None
-            url = "https://api.weixin.qq.com/cgi-bin/token"
-            params = {
-                "grant_type": "client_credential",
-                "appid": self.app_id,
-                "secret": self.app_secret,
-            }
-            data = self.wechatmp_request(method="get", url=url, params=params)
-            self.access_token = data["access_token"]
-            self.access_token_expires_time = int(time.time()) + data["expires_in"]
-            logger.info("[wechatmp] access_token: {}".format(self.access_token))
-            self.access_token_lock.release()
-        else:
-            # Wait for token update
-            while self.access_token_lock.locked():
-                time.sleep(0.1)
-        return self.access_token
+    async def delete_media(self, media_id):
+        logger.debug("[wechatmp] permanent media {} will be deleted in 10s".format(media_id))
+        await asyncio.sleep(10)
+        self.client.material.delete(media_id)
+        logger.info("[wechatmp] permanent media {} has been deleted".format(media_id))
 
     def send(self, reply: Reply, context: Context):
+        receiver = context["receiver"]
         if self.passive_reply:
-            receiver = context["receiver"]
-            self.cache_dict[receiver] = reply.content
-            logger.info("[send] reply to {} saved to cache: {}".format(receiver, reply))
+            if reply.type == ReplyType.TEXT or reply.type == ReplyType.INFO or reply.type == ReplyType.ERROR:
+                reply_text = reply.content
+                logger.info("[wechatmp] text cached, receiver {}\n{}".format(receiver, reply_text))
+                self.cache_dict[receiver] = ("text", reply_text)
+            elif reply.type == ReplyType.VOICE:
+                try:
+                    voice_file_path = reply.content
+                    with open(voice_file_path, "rb") as f:
+                        # support: <2M, <60s, mp3/wma/wav/amr
+                        response = self.client.material.add("voice", f)
+                        logger.debug("[wechatmp] upload voice response: {}".format(response))
+                        # 根据文件大小估计一个微信自动审核的时间，审核结束前返回将会导致语音无法播放，这个估计有待验证
+                        f_size = os.fstat(f.fileno()).st_size
+                        time.sleep(1.0 + 2 * f_size / 1024 / 1024)
+                        # todo check media_id
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload voice failed: {}".format(e))
+                    return
+                media_id = response["media_id"]
+                logger.info("[wechatmp] voice uploaded, receiver {}, media_id {}".format(receiver, media_id))
+                self.cache_dict[receiver] = ("voice", media_id)
+
+            elif reply.type == ReplyType.IMAGE_URL:  # 从网络下载图片
+                img_url = reply.content
+                pic_res = requests.get(img_url, stream=True)
+                image_storage = io.BytesIO()
+                for block in pic_res.iter_content(1024):
+                    image_storage.write(block)
+                image_storage.seek(0)
+                image_type = imghdr.what(image_storage)
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
+                content_type = "image/" + image_type
+                try:
+                    response = self.client.material.add("image", (filename, image_storage, content_type))
+                    logger.debug("[wechatmp] upload image response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload image failed: {}".format(e))
+                    return
+                media_id = response["media_id"]
+                logger.info("[wechatmp] image uploaded, receiver {}, media_id {}".format(receiver, media_id))
+                self.cache_dict[receiver] = ("image", media_id)
+            elif reply.type == ReplyType.IMAGE:  # 从文件读取图片
+                image_storage = reply.content
+                image_storage.seek(0)
+                image_type = imghdr.what(image_storage)
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
+                content_type = "image/" + image_type
+                try:
+                    response = self.client.material.add("image", (filename, image_storage, content_type))
+                    logger.debug("[wechatmp] upload image response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload image failed: {}".format(e))
+                    return
+                media_id = response["media_id"]
+                logger.info("[wechatmp] image uploaded, receiver {}, media_id {}".format(receiver, media_id))
+                self.cache_dict[receiver] = ("image", media_id)
         else:
-            receiver = context["receiver"]
-            reply_text = reply.content
-            url = "https://api.weixin.qq.com/cgi-bin/message/custom/send"
-            params = {"access_token": self.get_access_token()}
-            json_data = {
-                "touser": receiver,
-                "msgtype": "text",
-                "text": {"content": reply_text},
-            }
-            self.wechatmp_request(
-                method="post",
-                url=url,
-                params=params,
-                data=json.dumps(json_data, ensure_ascii=False).encode("utf8"),
-            )
-            logger.info("[send] Do send to {}: {}".format(receiver, reply_text))
+            if reply.type == ReplyType.TEXT or reply.type == ReplyType.INFO or reply.type == ReplyType.ERROR:
+                reply_text = reply.content
+                texts = split_string_by_utf8_length(reply_text, MAX_UTF8_LEN)
+                if len(texts) > 1:
+                    logger.info("[wechatmp] text too long, split into {} parts".format(len(texts)))
+                for i, text in enumerate(texts):
+                    self.client.message.send_text(receiver, text)
+                    if i != len(texts) - 1:
+                        time.sleep(0.5)  # 休眠0.5秒，防止发送过快乱序
+                logger.info("[wechatmp] Do send text to {}: {}".format(receiver, reply_text))
+            elif reply.type == ReplyType.VOICE:
+                try:
+                    file_path = reply.content
+                    file_name = os.path.basename(file_path)
+                    file_type = os.path.splitext(file_name)[1]
+                    if file_type == ".mp3":
+                        file_type = "audio/mpeg"
+                    elif file_type == ".amr":
+                        file_type = "audio/amr"
+                    else:
+                        mp3_file = os.path.splitext(file_path)[0] + ".mp3"
+                        any_to_mp3(file_path, mp3_file)
+                        file_path = mp3_file
+                        file_name = os.path.basename(file_path)
+                        file_type = "audio/mpeg"
+                    logger.info("[wechatmp] file_name: {}, file_type: {} ".format(file_name, file_type))
+                    # support: <2M, <60s, AMR\MP3
+                    response = self.client.media.upload("voice", (file_name, open(file_path, "rb"), file_type))
+                    logger.debug("[wechatmp] upload voice response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload voice failed: {}".format(e))
+                    return
+                self.client.message.send_voice(receiver, response["media_id"])
+                logger.info("[wechatmp] Do send voice to {}".format(receiver))
+            elif reply.type == ReplyType.IMAGE_URL:  # 从网络下载图片
+                img_url = reply.content
+                pic_res = requests.get(img_url, stream=True)
+                image_storage = io.BytesIO()
+                for block in pic_res.iter_content(1024):
+                    image_storage.write(block)
+                image_storage.seek(0)
+                image_type = imghdr.what(image_storage)
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
+                content_type = "image/" + image_type
+                try:
+                    response = self.client.media.upload("image", (filename, image_storage, content_type))
+                    logger.debug("[wechatmp] upload image response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload image failed: {}".format(e))
+                    return
+                self.client.message.send_image(receiver, response["media_id"])
+                logger.info("[wechatmp] Do send image to {}".format(receiver))
+            elif reply.type == ReplyType.IMAGE:  # 从文件读取图片
+                image_storage = reply.content
+                image_storage.seek(0)
+                image_type = imghdr.what(image_storage)
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + image_type
+                content_type = "image/" + image_type
+                try:
+                    response = self.client.media.upload("image", (filename, image_storage, content_type))
+                    logger.debug("[wechatmp] upload image response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload image failed: {}".format(e))
+                    return
+                self.client.message.send_image(receiver, response["media_id"])
+                logger.info("[wechatmp] Do send image to {}".format(receiver))
         return
 
     def _success_callback(self, session_id, context, **kwargs):  # 线程异常结束时的回调函数
-        logger.debug(
-            "[wechatmp] Success to generate reply, msgId={}".format(
-                context["msg"].msg_id
-            )
-        )
+        logger.debug("[wechatmp] Success to generate reply, msgId={}".format(context["msg"].msg_id))
         if self.passive_reply:
             self.running.remove(session_id)
 
     def _fail_callback(self, session_id, exception, context, **kwargs):  # 线程异常结束时的回调函数
-        logger.exception(
-            "[wechatmp] Fail to generate reply to user, msgId={}, exception={}".format(
-                context["msg"].msg_id, exception
-            )
-        )
+        logger.exception("[wechatmp] Fail to generate reply to user, msgId={}, exception={}".format(context["msg"].msg_id, exception))
         if self.passive_reply:
             assert session_id not in self.cache_dict
             self.running.remove(session_id)

channel/wechatmp/wechatmp_client.py

@@ -0,0 +1,49 @@
+import threading
+import time
+
+from wechatpy.client import WeChatClient
+from wechatpy.exceptions import APILimitedException
+
+from channel.wechatmp.common import *
+from common.log import logger
+
+
+class WechatMPClient(WeChatClient):
+    def __init__(self, appid, secret, access_token=None, session=None, timeout=None, auto_retry=True):
+        super(WechatMPClient, self).__init__(appid, secret, access_token, session, timeout, auto_retry)
+        self.fetch_access_token_lock = threading.Lock()
+        self.clear_quota_lock = threading.Lock()
+        self.last_clear_quota_time = -1
+
+    def clear_quota(self):
+        return self.post("clear_quota", data={"appid": self.appid})
+
+    def clear_quota_v2(self):
+        return self.post("clear_quota/v2", params={"appid": self.appid, "appsecret": self.secret})
+
+    def fetch_access_token(self):  # 重载父类方法，加锁避免多线程重复获取access_token
+        with self.fetch_access_token_lock:
+            access_token = self.session.get(self.access_token_key)
+            if access_token:
+                if not self.expires_at:
+                    return access_token
+                timestamp = time.time()
+                if self.expires_at - timestamp > 60:
+                    return access_token
+            return super().fetch_access_token()
+
+    def _request(self, method, url_or_endpoint, **kwargs):  # 重载父类方法，遇到API限流时，清除quota后重试
+        try:
+            return super()._request(method, url_or_endpoint, **kwargs)
+        except APILimitedException as e:
+            logger.error("[wechatmp] API quata has been used up. {}".format(e))
+            if self.last_clear_quota_time == -1 or time.time() - self.last_clear_quota_time > 60:
+                with self.clear_quota_lock:
+                    if self.last_clear_quota_time == -1 or time.time() - self.last_clear_quota_time > 60:
+                        self.last_clear_quota_time = time.time()
+                        response = self.clear_quota_v2()
+                        logger.debug("[wechatmp] API quata has been cleard, {}".format(response))
+                return super()._request(method, url_or_endpoint, **kwargs)
+            else:
+                logger.error("[wechatmp] last clear quota time is {}, less than 60s, skip clear quota")
+                raise e

channel/wechatmp/wechatmp_message.py

@@ -0,0 +1,56 @@
+# -*- coding: utf-8 -*-#
+
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.log import logger
+from common.tmp_dir import TmpDir
+
+
+class WeChatMPMessage(ChatMessage):
+    def __init__(self, msg, client=None):
+        super().__init__(msg)
+        self.msg_id = msg.id
+        self.create_time = msg.time
+        self.is_group = False
+
+        if msg.type == "text":
+            self.ctype = ContextType.TEXT
+            self.content = msg.content
+        elif msg.type == "voice":
+            if msg.recognition == None:
+                self.ctype = ContextType.VOICE
+                self.content = TmpDir().path() + msg.media_id + "." + msg.format  # content直接存临时目录路径
+
+                def download_voice():
+                    # 如果响应状态码是200，则将响应内容写入本地文件
+                    response = client.media.download(msg.media_id)
+                    if response.status_code == 200:
+                        with open(self.content, "wb") as f:
+                            f.write(response.content)
+                    else:
+                        logger.info(f"[wechatmp] Failed to download voice file, {response.content}")
+
+                self._prepare_fn = download_voice
+            else:
+                self.ctype = ContextType.TEXT
+                self.content = msg.recognition
+        elif msg.type == "image":
+            self.ctype = ContextType.IMAGE
+            self.content = TmpDir().path() + msg.media_id + ".png"  # content直接存临时目录路径
+
+            def download_image():
+                # 如果响应状态码是200，则将响应内容写入本地文件
+                response = client.media.download(msg.media_id)
+                if response.status_code == 200:
+                    with open(self.content, "wb") as f:
+                        f.write(response.content)
+                else:
+                    logger.info(f"[wechatmp] Failed to download image file, {response.content}")
+
+            self._prepare_fn = download_image
+        else:
+            raise NotImplementedError("Unsupported message type: Type:{} ".format(msg.type))
+
+        self.from_user_id = msg.source
+        self.to_user_id = msg.target
+        self.other_user_id = msg.source

common/const.py

@@ -3,3 +3,6 @@ OPEN_AI = "openAI"
 CHATGPT = "chatGPT"
 BAIDU = "baidu"
 CHATGPTONAZURE = "chatGPTOnAzure"
+LINKAI = "linkai"
+
+VERSION = "1.3.0"

common/time_check.py

@@ -13,23 +13,15 @@ def time_checker(f):
         if chat_time_module:
             chat_start_time = _config.get("chat_start_time", "00:00")
             chat_stopt_time = _config.get("chat_stop_time", "24:00")
-            time_regex = re.compile(
-                r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$"
-            )  # 时间匹配，包含24:00
+            time_regex = re.compile(r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$")  # 时间匹配，包含24:00
 
             starttime_format_check = time_regex.match(chat_start_time)  # 检查停止时间格式
             stoptime_format_check = time_regex.match(chat_stopt_time)  # 检查停止时间格式
             chat_time_check = chat_start_time < chat_stopt_time  # 确定启动时间<停止时间
 
             # 时间格式检查
-            if not (
-                starttime_format_check and stoptime_format_check and chat_time_check
-            ):
-                logger.warn(
-                    "时间格式不正确,请在config.json中修改您的CHAT_START_TIME/CHAT_STOP_TIME,否则可能会影响您正常使用,开始({})-结束({})".format(
-                        starttime_format_check, stoptime_format_check
-                    )
-                )
+            if not (starttime_format_check and stoptime_format_check and chat_time_check):
+                logger.warn("时间格式不正确,请在config.json中修改您的CHAT_START_TIME/CHAT_STOP_TIME,否则可能会影响您正常使用,开始({})-结束({})".format(starttime_format_check, stoptime_format_check))
             if chat_start_time > "23:59":
                 logger.error("启动时间可能存在问题，请修改!")
 

common/utils.py

@@ -0,0 +1,51 @@
+import io
+import os
+
+from PIL import Image
+
+
+def fsize(file):
+    if isinstance(file, io.BytesIO):
+        return file.getbuffer().nbytes
+    elif isinstance(file, str):
+        return os.path.getsize(file)
+    elif hasattr(file, "seek") and hasattr(file, "tell"):
+        pos = file.tell()
+        file.seek(0, os.SEEK_END)
+        size = file.tell()
+        file.seek(pos)
+        return size
+    else:
+        raise TypeError("Unsupported type")
+
+
+def compress_imgfile(file, max_size):
+    if fsize(file) <= max_size:
+        return file
+    file.seek(0)
+    img = Image.open(file)
+    rgb_image = img.convert("RGB")
+    quality = 95
+    while True:
+        out_buf = io.BytesIO()
+        rgb_image.save(out_buf, "JPEG", quality=quality)
+        if fsize(out_buf) <= max_size:
+            return out_buf
+        quality -= 5
+
+
+def split_string_by_utf8_length(string, max_length, max_split=0):
+    encoded = string.encode("utf-8")
+    start, end = 0, 0
+    result = []
+    while end < len(encoded):
+        if max_split > 0 and len(result) >= max_split:
+            result.append(encoded[start:].decode("utf-8"))
+            break
+        end = min(start + max_length, len(encoded))
+        # 如果当前字节不是 UTF-8 编码的开始字节，则向前查找直到找到开始字节为止
+        while end < len(encoded) and (encoded[end] & 0b11000000) == 0b10000000:
+            end -= 1
+        result.append(encoded[start:end].decode("utf-8"))
+        start = end
+    return result

config-template.json

@@ -27,5 +27,9 @@
   "voice_reply_voice": false,
   "conversation_max_tokens": 1000,
   "expires_in_seconds": 3600,
-  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"
+  "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",
+  "subscribe_msg": "感谢您的关注！\n这里是ChatGPT，可以自由对话。\n支持语音对话。\n支持图片输入。\n支持图片输出，画字开头的消息将按要求创作图片。\n支持tool、角色扮演和文字冒险等丰富的插件。\n输入{trigger_prefix}#help 查看详细指令。",
+  "use_linkai": false,
+  "linkai_api_key": "",
+  "linkai_app_code": ""
 }

config.py

@@ -8,6 +8,7 @@ import pickle
 from common.log import logger
 
 # 将所有可用的配置项写在字典里, 请使用小写字母
+# 此处的配置值无实际意义，程序不会读取此处的配置，仅用于提示格式，请将配置加入到config.json中
 available_setting = {
     # openai api配置
     "open_ai_api_key": "",  # openai api key
@@ -66,6 +67,11 @@ available_setting = {
     "chat_time_module": False,  # 是否开启服务时间限制
     "chat_start_time": "00:00",  # 服务开始时间
     "chat_stop_time": "24:00",  # 服务结束时间
+    # 翻译api
+    "translate": "baidu",  # 翻译api，支持baidu
+    # baidu翻译api的配置
+    "baidu_translate_app_id": "",  # 百度翻译api的appid
+    "baidu_translate_app_key": "",  # 百度翻译api的秘钥
     # itchat的配置
     "hot_reload": False,  # 是否开启热重载
     # wechaty的配置
@@ -73,22 +79,40 @@ available_setting = {
     # wechatmp的配置
     "wechatmp_token": "",  # 微信公众平台的Token
     "wechatmp_port": 8080,  # 微信公众平台的端口,需要端口转发到80或443
-    "wechatmp_app_id": "",  # 微信公众平台的appID，仅服务号需要
-    "wechatmp_app_secret": "",  # 微信公众平台的appsecret，仅服务号需要
+    "wechatmp_app_id": "",  # 微信公众平台的appID
+    "wechatmp_app_secret": "",  # 微信公众平台的appsecret
+    "wechatmp_aes_key": "",  # 微信公众平台的EncodingAESKey，加密模式需要
+    # wechatcom的通用配置
+    "wechatcom_corp_id": "",  # 企业微信公司的corpID
+    # wechatcomapp的配置
+    "wechatcomapp_token": "",  # 企业微信app的token
+    "wechatcomapp_port": 9898,  # 企业微信app的服务端口,不需要端口转发
+    "wechatcomapp_secret": "",  # 企业微信app的secret
+    "wechatcomapp_agent_id": "",  # 企业微信app的agent_id
+    "wechatcomapp_aes_key": "",  # 企业微信app的aes_key
     # chatgpt指令自定义触发词
     "clear_memory_commands": ["#清除记忆"],  # 重置会话指令，必须以#开头
     # channel配置
-    "channel_type": "wx",  # 通道类型，支持：{wx,wxy,terminal,wechatmp,wechatmp_service}
+    "channel_type": "wx",  # 通道类型，支持：{wx,wxy,terminal,wechatmp,wechatmp_service,wechatcom_app}
+    "subscribe_msg": "",  # 订阅消息, 支持: wechatmp, wechatmp_service, wechatcom_app
     "debug": False,  # 是否开启debug模式，开启后会打印更多日志
     "appdata_dir": "",  # 数据目录
     # 插件配置
     "plugin_trigger_prefix": "$",  # 规范插件提供聊天相关指令的前缀，建议不要和管理员指令前缀"#"冲突
+    # 知识库平台配置
+    "use_linkai": False,
+    "linkai_api_key": "",
+    "linkai_app_code": ""
 }
 
 
 class Config(dict):
-    def __init__(self, d: dict = {}):
-        super().__init__(d)
+    def __init__(self, d=None):
+        super().__init__()
+        if d is None:
+            d = {}
+        for k, v in d.items():
+            self[k] = v
         # user_datas: 用户数据，key为用户名，value为用户数据，也是dict
         self.user_datas = {}
 
@@ -157,9 +181,7 @@ def load_config():
     for name, value in os.environ.items():
         name = name.lower()
         if name in available_setting:
-            logger.info(
-                "[INIT] override config by environ args: {}={}".format(name, value)
-            )
+            logger.info("[INIT] override config by environ args: {}={}".format(name, value))
             try:
                 config[name] = eval(value)
             except:
@@ -198,3 +220,9 @@ def get_appdata_dir():
         logger.info("[INIT] data path not exists, create it: {}".format(data_path))
         os.makedirs(data_path)
     return data_path
+
+
+def subscribe_msg():
+    trigger_prefix = conf().get("single_chat_prefix", [""])[0]
+    msg = conf().get("subscribe_msg", "")
+    return msg.format(trigger_prefix=trigger_prefix)

docker/Dockerfile.alpine

@@ -22,8 +22,8 @@ RUN apk add --no-cache \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json ${BUILD_PREFIX}/config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache -r requirements.txt \
-    && pip install --no-cache -r requirements-optional.txt \
+    && pip install --no-cache -r requirements.txt --extra-index-url https://alpine-wheels.github.io/index\
+    && pip install --no-cache -r requirements-optional.txt --extra-index-url https://alpine-wheels.github.io/index\
     && apk del curl wget
 
 WORKDIR ${BUILD_PREFIX}

docker/Dockerfile.alpine.latest

@@ -1,4 +1,4 @@
-FROM python:3.10-slim
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -9,25 +9,21 @@ ENV BUILD_PREFIX=/app
 
 ADD . ${BUILD_PREFIX}
 
-RUN apt-get update \
-    &&apt-get install -y --no-install-recommends  bash \
-    ffmpeg espeak \
+RUN apk add --no-cache bash ffmpeg espeak \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache -r requirements.txt \
-    && pip install --no-cache -r requirements-optional.txt \
-    && pip install azure-cognitiveservices-speech
+    && pip install --no-cache -r requirements.txt --extra-index-url https://alpine-wheels.github.io/index\
+    && pip install --no-cache -r requirements-optional.txt --extra-index-url https://alpine-wheels.github.io/index
 
 WORKDIR ${BUILD_PREFIX}
 
 ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
-    && groupadd -r noroot \
-    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
     && chown -R noroot:noroot ${BUILD_PREFIX}
 
 USER noroot
 
-ENTRYPOINT ["docker/entrypoint.sh"]
+ENTRYPOINT ["/entrypoint.sh"]

docker/Dockerfile.debian

@@ -31,9 +31,10 @@ WORKDIR ${BUILD_PREFIX}
 ADD ./entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
+    && mkdir -p /home/noroot \
     && groupadd -r noroot \
     && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
-    && chown -R noroot:noroot ${BUILD_PREFIX}
+    && chown -R noroot:noroot /home/noroot ${BUILD_PREFIX} /usr/local/lib
 
 USER noroot
 

docker/Dockerfile.latest

@@ -1,29 +1,35 @@
-FROM python:3.10-alpine
+FROM python:3.10-slim
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
 
 ARG CHATGPT_ON_WECHAT_VER
 
+RUN echo /etc/apt/sources.list
+# RUN sed -i 's/deb.debian.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apt/sources.list
 ENV BUILD_PREFIX=/app
 
 ADD . ${BUILD_PREFIX}
 
-RUN apk add --no-cache bash ffmpeg espeak \
+RUN apt-get update \
+    &&apt-get install -y --no-install-recommends bash ffmpeg espeak libavcodec-extra\
     && cd ${BUILD_PREFIX} \
     && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
     && pip install --no-cache -r requirements.txt \
-    && pip install --no-cache -r requirements-optional.txt
+    && pip install --no-cache -r requirements-optional.txt \
+    && pip install azure-cognitiveservices-speech
 
 WORKDIR ${BUILD_PREFIX}
 
 ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
-    && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
-    && chown -R noroot:noroot ${BUILD_PREFIX}
+    && mkdir -p /home/noroot \
+    && groupadd -r noroot \
+    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && chown -R noroot:noroot /home/noroot ${BUILD_PREFIX} /usr/local/lib
 
 USER noroot
 
-ENTRYPOINT ["docker/entrypoint.sh"]
+ENTRYPOINT ["/entrypoint.sh"]

docker/entrypoint.sh

@@ -38,9 +38,9 @@ if [ "$CHATGPT_ON_WECHAT_EXEC" == "" ] ; then
 fi
 
 # modify content in config.json
-if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
-    echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
-fi
+# if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
+#     echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
+# fi
 
 
 # go to prefix dir

lib/itchat/components/login.py

@@ -43,6 +43,7 @@ def login(self, enableCmdQR=False, picDir=None, qrCallback=None,
         logger.warning('itchat has already logged in.')
         return
     self.isLogging = True
+    logger.info('Ready to login.')
     while self.isLogging:
         uuid = push_login(self)
         if uuid:
@@ -84,7 +85,7 @@ def login(self, enableCmdQR=False, picDir=None, qrCallback=None,
     if hasattr(loginCallback, '__call__'):
         r = loginCallback()
     else:
-        utils.clear_screen()
+        # utils.clear_screen()
         if os.path.exists(picDir or config.DEFAULT_QR):
             os.remove(picDir or config.DEFAULT_QR)
         logger.info('Login successfully as %s' % self.storageClass.nickName)
@@ -195,13 +196,17 @@ def process_login_info(core, loginContent):
     core.loginInfo['logintime'] = int(time.time() * 1e3)
     core.loginInfo['BaseRequest'] = {}
     cookies = core.s.cookies.get_dict()
-    skey = re.findall('<skey>(.*?)</skey>', r.text, re.S)[0]
-    pass_ticket = re.findall(
-        '<pass_ticket>(.*?)</pass_ticket>', r.text, re.S)[0]
-    core.loginInfo['skey'] = core.loginInfo['BaseRequest']['Skey'] = skey
+    res = re.findall('<skey>(.*?)</skey>', r.text, re.S)
+    skey = res[0] if res else None
+    res = re.findall(
+        '<pass_ticket>(.*?)</pass_ticket>', r.text, re.S)
+    pass_ticket = res[0] if res else None
+    if skey is not None:
+        core.loginInfo['skey'] = core.loginInfo['BaseRequest']['Skey'] = skey
     core.loginInfo['wxsid'] = core.loginInfo['BaseRequest']['Sid'] = cookies["wxsid"]
     core.loginInfo['wxuin'] = core.loginInfo['BaseRequest']['Uin'] = cookies["wxuin"]
-    core.loginInfo['pass_ticket'] = pass_ticket
+    if pass_ticket is not None:
+        core.loginInfo['pass_ticket'] = pass_ticket
     # A question : why pass_ticket == DeviceID ?
     #               deviceID is only a randomly generated number
 
@@ -317,6 +322,8 @@ def start_receiving(self, exitCallback=None, getReceivingFnOnly=False):
                 retryCount += 1
                 logger.error(traceback.format_exc())
                 if self.receivingRetryCount < retryCount:
+                    logger.error("Having tried %s times, but still failed. " % (
+                        retryCount) + "Stop trying...")
                     self.alive = False
                 else:
                     time.sleep(1)
@@ -363,7 +370,7 @@ def sync_check(self):
     regx = r'window.synccheck={retcode:"(\d+)",selector:"(\d+)"}'
     pm = re.search(regx, r.text)
     if pm is None or pm.group(1) != '0':
-        logger.debug('Unexpected sync check result: %s' % r.text)
+        logger.error('Unexpected sync check result: %s' % r.text)
         return None
     return pm.group(2)
 

lib/itchat/components/register.py

@@ -25,9 +25,12 @@ def auto_login(self, hotReload=False, statusStorageDir='itchat.pkl',
     self.useHotReload = hotReload
     self.hotReloadDir = statusStorageDir
     if hotReload:
-        if self.load_login_status(statusStorageDir,
-                loginCallback=loginCallback, exitCallback=exitCallback):
+        rval=self.load_login_status(statusStorageDir,
+                loginCallback=loginCallback, exitCallback=exitCallback)
+        if rval:
             return
+        logger.error('Hot reload failed, logging in normally, error={}'.format(rval))
+        self.logout()
         self.login(enableCmdQR=enableCmdQR, picDir=picDir, qrCallback=qrCallback,
             loginCallback=loginCallback, exitCallback=exitCallback)
         self.dump_login_status(statusStorageDir)

nixpacks.toml

@@ -1,7 +1,7 @@
-providers = ['python']
-
 [phases.setup]
 nixPkgs = ['python310']
-cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak','python -m venv /opt/venv && . /opt/venv/bin/activate && pip install -r requirements-optional.txt']
+cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak libavcodec-extra']
+[phases.install]
+cmds = ['python -m venv /opt/venv && . /opt/venv/bin/activate && pip install -r requirements.txt && pip install -r requirements-optional.txt']
 [start]
 cmd = "python ./app.py"

plugins/banwords/banwords.py

@@ -50,9 +50,7 @@ class Banwords(Plugin):
                 self.reply_action = conf.get("reply_action", "ignore")
             logger.info("[Banwords] inited")
         except Exception as e:
-            logger.warn(
-                "[Banwords] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/banwords ."
-            )
+            logger.warn("[Banwords] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/banwords .")
             raise e
 
     def on_handle_context(self, e_context: EventContext):
@@ -72,9 +70,7 @@ class Banwords(Plugin):
                 return
         elif self.action == "replace":
             if self.searchr.ContainsAny(content):
-                reply = Reply(
-                    ReplyType.INFO, "发言中包含敏感词，请重试: \n" + self.searchr.Replace(content)
-                )
+                reply = Reply(ReplyType.INFO, "发言中包含敏感词，请重试: \n" + self.searchr.Replace(content))
                 e_context["reply"] = reply
                 e_context.action = EventAction.BREAK_PASS
                 return
@@ -94,12 +90,10 @@ class Banwords(Plugin):
                 return
         elif self.reply_action == "replace":
             if self.searchr.ContainsAny(content):
-                reply = Reply(
-                    ReplyType.INFO, "已替换回复中的敏感词: \n" + self.searchr.Replace(content)
-                )
+                reply = Reply(ReplyType.INFO, "已替换回复中的敏感词: \n" + self.searchr.Replace(content))
                 e_context["reply"] = reply
                 e_context.action = EventAction.CONTINUE
                 return
 
     def get_help_text(self, **kwargs):
-        return Banwords.desc
+        return "过滤消息中的敏感词。"

plugins/bdunit/bdunit.py

@@ -76,9 +76,7 @@ class BDunit(Plugin):
         Returns:
             string: access_token
         """
-        url = "https://aip.baidubce.com/oauth/2.0/token?client_id={}&client_secret={}&grant_type=client_credentials".format(
-            self.api_key, self.secret_key
-        )
+        url = "https://aip.baidubce.com/oauth/2.0/token?client_id={}&client_secret={}&grant_type=client_credentials".format(self.api_key, self.secret_key)
         payload = ""
         headers = {"Content-Type": "application/json", "Accept": "application/json"}
 
@@ -94,10 +92,7 @@ class BDunit(Plugin):
         :returns: UNIT 解析结果。如果解析失败，返回 None
         """
 
-        url = (
-            "https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token="
-            + self.access_token
-        )
+        url = "https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token=" + self.access_token
         request = {
             "query": query,
             "user_id": str(get_mac())[:32],
@@ -124,10 +119,7 @@ class BDunit(Plugin):
         :param query: 用户的指令字符串
         :returns: UNIT 解析结果。如果解析失败，返回 None
         """
-        url = (
-            "https://aip.baidubce.com/rpc/2.0/unit/service/chat?access_token="
-            + self.access_token
-        )
+        url = "https://aip.baidubce.com/rpc/2.0/unit/service/chat?access_token=" + self.access_token
         request = {"query": query, "user_id": str(get_mac())[:32]}
         body = {
             "log_id": str(uuid.uuid1()),
@@ -170,11 +162,7 @@ class BDunit(Plugin):
         if parsed and "result" in parsed and "response_list" in parsed["result"]:
             response_list = parsed["result"]["response_list"]
             for response in response_list:
-                if (
-                    "schema" in response
-                    and "intent" in response["schema"]
-                    and response["schema"]["intent"] == intent
-                ):
+                if "schema" in response and "intent" in response["schema"] and response["schema"]["intent"] == intent:
                     return True
             return False
         else:
@@ -198,12 +186,7 @@ class BDunit(Plugin):
                     logger.warning(e)
                     return []
             for response in response_list:
-                if (
-                    "schema" in response
-                    and "intent" in response["schema"]
-                    and "slots" in response["schema"]
-                    and response["schema"]["intent"] == intent
-                ):
+                if "schema" in response and "intent" in response["schema"] and "slots" in response["schema"] and response["schema"]["intent"] == intent:
                     return response["schema"]["slots"]
             return []
         else:
@@ -239,11 +222,7 @@ class BDunit(Plugin):
                 if (
                     "schema" in response
                     and "intent_confidence" in response["schema"]
-                    and (
-                        not answer
-                        or response["schema"]["intent_confidence"]
-                        > answer["schema"]["intent_confidence"]
-                    )
+                    and (not answer or response["schema"]["intent_confidence"] > answer["schema"]["intent_confidence"])
                 ):
                     answer = response
             return answer["action_list"][0]["say"]
@@ -267,11 +246,7 @@ class BDunit(Plugin):
                     logger.warning(e)
                     return ""
             for response in response_list:
-                if (
-                    "schema" in response
-                    and "intent" in response["schema"]
-                    and response["schema"]["intent"] == intent
-                ):
+                if "schema" in response and "intent" in response["schema"] and response["schema"]["intent"] == intent:
                     try:
                         return response["action_list"][0]["say"]
                     except Exception as e:

plugins/dungeon/dungeon.py

@@ -64,7 +64,7 @@ class Dungeon(Plugin):
         if e_context["context"].type != ContextType.TEXT:
             return
         bottype = Bridge().get_bot_type("chat")
-        if bottype not in (const.CHATGPT, const.OPEN_AI):
+        if bottype not in [const.OPEN_AI, const.CHATGPT, const.CHATGPTONAZURE, const.LINKAI]:
             return
         bot = Bridge().get_bot("chat")
         content = e_context["context"].content[:]
@@ -84,9 +84,7 @@ class Dungeon(Plugin):
                 if len(clist) > 1:
                     story = clist[1]
                 else:
-                    story = (
-                        "你在树林里冒险，指不定会从哪里蹦出来一些奇怪的东西，你握紧手上的手枪，希望这次冒险能够找到一些值钱的东西，你往树林深处走去。"
-                    )
+                    story = "你在树林里冒险，指不定会从哪里蹦出来一些奇怪的东西，你握紧手上的手枪，希望这次冒险能够找到一些值钱的东西，你往树林深处走去。"
                 self.games[sessionid] = StoryTeller(bot, sessionid, story)
                 reply = Reply(ReplyType.INFO, "冒险开始，你可以输入任意内容，让故事继续下去。故事背景是：" + story)
                 e_context["reply"] = reply
@@ -102,11 +100,7 @@ class Dungeon(Plugin):
         if kwargs.get("verbose") != True:
             return help_text
         trigger_prefix = conf().get("plugin_trigger_prefix", "$")
-        help_text = (
-            f"{trigger_prefix}开始冒险 "
-            + "背景故事: 开始一个基于{背景故事}的文字冒险，之后你的所有消息会协助完善这个故事。\n"
-            + f"{trigger_prefix}停止冒险: 结束游戏。\n"
-        )
+        help_text = f"{trigger_prefix}开始冒险 " + "背景故事: 开始一个基于{背景故事}的文字冒险，之后你的所有消息会协助完善这个故事。\n" + f"{trigger_prefix}停止冒险: 结束游戏。\n"
         if kwargs.get("verbose") == True:
             help_text += f"\n命令例子: '{trigger_prefix}开始冒险 你在树林里冒险，指不定会从哪里蹦出来一些奇怪的东西，你握紧手上的手枪，希望这次冒险能够找到一些值钱的东西，你往树林深处走去。'"
         return help_text

plugins/event.py

@@ -50,3 +50,6 @@ class EventContext:
 
     def is_pass(self):
         return self.action == EventAction.BREAK_PASS
+
+    def is_break(self):
+        return self.action == EventAction.BREAK or self.action == EventAction.BREAK_PASS

plugins/godcmd/godcmd.py

@@ -41,6 +41,18 @@ COMMANDS = {
         "alias": ["reset_openai_api_key"],
         "desc": "重置为默认的api_key",
     },
+    "set_gpt_model": {
+        "alias": ["set_gpt_model"],
+        "desc": "设置你的私有模型",
+    },
+    "reset_gpt_model": {
+        "alias": ["reset_gpt_model"],
+        "desc": "重置你的私有模型",
+    },
+    "gpt_model": {
+        "alias": ["gpt_model"],
+        "desc": "查询你使用的模型",
+    },
     "id": {
         "alias": ["id", "用户"],
         "desc": "获取用户id",  # wechaty和wechatmp的用户id不会变化，可用于绑定管理员
@@ -140,9 +152,7 @@ def get_help_text(isadmin, isgroup):
         if plugins[plugin].enabled and not plugins[plugin].hidden:
             namecn = plugins[plugin].namecn
             help_text += "\n%s:" % namecn
-            help_text += (
-                PluginManager().instances[plugin].get_help_text(verbose=False).strip()
-            )
+            help_text += PluginManager().instances[plugin].get_help_text(verbose=False).strip()
 
     if ADMIN_COMMANDS and isadmin:
         help_text += "\n\n管理员指令：\n"
@@ -191,9 +201,7 @@ class Godcmd(Plugin):
                     COMMANDS["reset"]["alias"].append(custom_command)
 
         self.password = gconf["password"]
-        self.admin_users = gconf[
-            "admin_users"
-        ]  # 预存的管理员账号，这些账号不需要认证。itchat的用户名每次都会变，不可用
+        self.admin_users = gconf["admin_users"]  # 预存的管理员账号，这些账号不需要认证。itchat的用户名每次都会变，不可用
         self.isrunning = True  # 机器人是否运行中
 
         self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
@@ -209,6 +217,13 @@ class Godcmd(Plugin):
         content = e_context["context"].content
         logger.debug("[Godcmd] on_handle_context. content: %s" % content)
         if content.startswith("#"):
+            if len(content) == 1:
+                reply = Reply()
+                reply.type = ReplyType.ERROR
+                reply.content = f"空指令，输入#help查看指令列表\n"
+                e_context["reply"] = reply
+                e_context.action = EventAction.BREAK_PASS
+                return
             # msg = e_context['context']['msg']
             channel = e_context["channel"]
             user = e_context["context"]["receiver"]
@@ -241,11 +256,7 @@ class Godcmd(Plugin):
                             if not plugincls.enabled:
                                 continue
                             if query_name == name or query_name == plugincls.namecn:
-                                ok, result = True, PluginManager().instances[
-                                    name
-                                ].get_help_text(
-                                    isgroup=isgroup, isadmin=isadmin, verbose=True
-                                )
+                                ok, result = True, PluginManager().instances[name].get_help_text(isgroup=isgroup, isadmin=isadmin, verbose=True)
                                 break
                         if not ok:
                             result = "插件不存在或未启用"
@@ -265,8 +276,28 @@ class Godcmd(Plugin):
                         ok, result = True, "你的OpenAI私有api_key已清除"
                     except Exception as e:
                         ok, result = False, "你没有设置私有api_key"
+                elif cmd == "set_gpt_model":
+                    if len(args) == 1:
+                        user_data = conf().get_user_data(user)
+                        user_data["gpt_model"] = args[0]
+                        ok, result = True, "你的GPT模型已设置为" + args[0]
+                    else:
+                        ok, result = False, "请提供一个GPT模型"
+                elif cmd == "gpt_model":
+                    user_data = conf().get_user_data(user)
+                    model = conf().get("model")
+                    if "gpt_model" in user_data:
+                        model = user_data["gpt_model"]
+                    ok, result = True, "你的GPT模型为" + str(model)
+                elif cmd == "reset_gpt_model":
+                    try:
+                        user_data = conf().get_user_data(user)
+                        user_data.pop("gpt_model")
+                        ok, result = True, "你的GPT模型已重置"
+                    except Exception as e:
+                        ok, result = False, "你没有设置私有GPT模型"
                 elif cmd == "reset":
-                    if bottype in (const.CHATGPT, const.OPEN_AI):
+                    if bottype in [const.OPEN_AI, const.CHATGPT, const.CHATGPTONAZURE, const.LINKAI]:
                         bot.sessions.clear_session(session_id)
                         channel.cancel_session(session_id)
                         ok, result = True, "会话已重置"
@@ -278,11 +309,7 @@ class Godcmd(Plugin):
                     if isgroup:
                         ok, result = False, "群聊不可执行管理员指令"
                     else:
-                        cmd = next(
-                            c
-                            for c, info in ADMIN_COMMANDS.items()
-                            if cmd in info["alias"]
-                        )
+                        cmd = next(c for c, info in ADMIN_COMMANDS.items() if cmd in info["alias"])
                         if cmd == "stop":
                             self.isrunning = False
                             ok, result = True, "服务已暂停"
@@ -293,7 +320,7 @@ class Godcmd(Plugin):
                             load_config()
                             ok, result = True, "配置已重载"
                         elif cmd == "resetall":
-                            if bottype in (const.CHATGPT, const.OPEN_AI):
+                            if bottype in [const.OPEN_AI, const.CHATGPT, const.CHATGPTONAZURE, const.LINKAI]:
                                 channel.cancel_all_session()
                                 bot.sessions.clear_all_session()
                                 ok, result = True, "重置所有会话成功"
@@ -318,18 +345,14 @@ class Godcmd(Plugin):
                             PluginManager().activate_plugins()
                             if len(new_plugins) > 0:
                                 result += "\n发现新插件：\n"
-                                result += "\n".join(
-                                    [f"{p.name}_v{p.version}" for p in new_plugins]
-                                )
+                                result += "\n".join([f"{p.name}_v{p.version}" for p in new_plugins])
                             else:
                                 result += ", 未发现新插件"
                         elif cmd == "setpri":
                             if len(args) != 2:
                                 ok, result = False, "请提供插件名和优先级"
                             else:
-                                ok = PluginManager().set_plugin_priority(
-                                    args[0], int(args[1])
-                                )
+                                ok = PluginManager().set_plugin_priority(args[0], int(args[1]))
                                 if ok:
                                     result = "插件" + args[0] + "优先级已设置为" + args[1]
                                 else:

plugins/hello/hello.py

@@ -23,7 +23,25 @@ class Hello(Plugin):
         logger.info("[Hello] inited")
 
     def on_handle_context(self, e_context: EventContext):
-        if e_context["context"].type != ContextType.TEXT:
+        if e_context["context"].type not in [
+            ContextType.TEXT,
+            ContextType.JOIN_GROUP,
+            ContextType.PATPAT,
+        ]:
+            return
+
+        if e_context["context"].type == ContextType.JOIN_GROUP:
+            e_context["context"].type = ContextType.TEXT
+            msg: ChatMessage = e_context["context"]["msg"]
+            e_context["context"].content = f'请你随机使用一种风格说一句问候语来欢迎新用户"{msg.actual_user_nickname}"加入群聊。'
+            e_context.action = EventAction.CONTINUE  # 事件继续，交付给下个插件或默认逻辑
+            return
+
+        if e_context["context"].type == ContextType.PATPAT:
+            e_context["context"].type = ContextType.TEXT
+            msg: ChatMessage = e_context["context"]["msg"]
+            e_context["context"].content = f"请你随机使用一种风格介绍你自己，并告诉用户输入#help可以查看帮助信息。"
+            e_context.action = EventAction.CONTINUE  # 事件继续，交付给下个插件或默认逻辑
             return
 
         content = e_context["context"].content
@@ -33,9 +51,7 @@ class Hello(Plugin):
             reply.type = ReplyType.TEXT
             msg: ChatMessage = e_context["context"]["msg"]
             if e_context["context"]["isgroup"]:
-                reply.content = (
-                    f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
-                )
+                reply.content = f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
             else:
                 reply.content = f"Hello, {msg.from_user_nickname}"
             e_context["reply"] = reply

plugins/keyword/README.md

@@ -0,0 +1,13 @@
+# 目的
+关键字匹配并回复
+
+# 试用场景
+目前是在微信公众号下面使用过。
+
+# 使用步骤
+1. 复制 `config.json.template` 为 `config.json`
+2. 在关键字 `keyword` 新增需要关键字匹配的内容
+3. 重启程序做验证
+
+# 验证结果
+![结果](test-keyword.png)

plugins/keyword/__init__.py

@@ -0,0 +1 @@
+from .keyword import *

plugins/keyword/config.json.template

@@ -0,0 +1,5 @@
+{
+  "keyword": {
+    "关键字匹配": "测试成功"
+  }
+}

plugins/keyword/keyword.py

@@ -0,0 +1,65 @@
+# encoding:utf-8
+
+import json
+import os
+
+import plugins
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from plugins import *
+
+
+@plugins.register(
+    name="Keyword",
+    desire_priority=900,
+    hidden=True,
+    desc="关键词匹配过滤",
+    version="0.1",
+    author="fengyege.top",
+)
+class Keyword(Plugin):
+    def __init__(self):
+        super().__init__()
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            conf = None
+            if not os.path.exists(config_path):
+                logger.debug(f"[keyword]不存在配置文件{config_path}")
+                conf = {"keyword": {}}
+                with open(config_path, "w", encoding="utf-8") as f:
+                    json.dump(conf, f, indent=4)
+            else:
+                logger.debug(f"[keyword]加载配置文件{config_path}")
+                with open(config_path, "r", encoding="utf-8") as f:
+                    conf = json.load(f)
+            # 加载关键词
+            self.keyword = conf["keyword"]
+
+            logger.info("[keyword] {}".format(self.keyword))
+            self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+            logger.info("[keyword] inited.")
+        except Exception as e:
+            logger.warn("[keyword] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/keyword .")
+            raise e
+
+    def on_handle_context(self, e_context: EventContext):
+        if e_context["context"].type != ContextType.TEXT:
+            return
+
+        content = e_context["context"].content.strip()
+        logger.debug("[keyword] on_handle_context. content: %s" % content)
+        if content in self.keyword:
+            logger.debug(f"[keyword] 匹配到关键字【{content}】")
+            reply_text = self.keyword[content]
+
+            reply = Reply()
+            reply.type = ReplyType.TEXT
+            reply.content = reply_text
+            e_context["reply"] = reply
+            e_context.action = EventAction.BREAK_PASS  # 事件结束，并跳过处理context的默认逻辑
+
+    def get_help_text(self, **kwargs):
+        help_text = "关键词过滤"
+        return help_text

plugins/plugin_manager.py

@@ -31,23 +31,14 @@ class PluginManager:
             plugincls.desc = kwargs.get("desc")
             plugincls.author = kwargs.get("author")
             plugincls.path = self.current_plugin_path
-            plugincls.version = (
-                kwargs.get("version") if kwargs.get("version") != None else "1.0"
-            )
-            plugincls.namecn = (
-                kwargs.get("namecn") if kwargs.get("namecn") != None else name
-            )
-            plugincls.hidden = (
-                kwargs.get("hidden") if kwargs.get("hidden") != None else False
-            )
+            plugincls.version = kwargs.get("version") if kwargs.get("version") != None else "1.0"
+            plugincls.namecn = kwargs.get("namecn") if kwargs.get("namecn") != None else name
+            plugincls.hidden = kwargs.get("hidden") if kwargs.get("hidden") != None else False
             plugincls.enabled = True
             if self.current_plugin_path == None:
                 raise Exception("Plugin path not set")
             self.plugins[name.upper()] = plugincls
-            logger.info(
-                "Plugin %s_v%s registered, path=%s"
-                % (name, plugincls.version, plugincls.path)
-            )
+            logger.info("Plugin %s_v%s registered, path=%s" % (name, plugincls.version, plugincls.path))
 
         return wrapper
 
@@ -62,9 +53,7 @@ class PluginManager:
         if os.path.exists("./plugins/plugins.json"):
             with open("./plugins/plugins.json", "r", encoding="utf-8") as f:
                 pconf = json.load(f)
-                pconf["plugins"] = SortedDict(
-                    lambda k, v: v["priority"], pconf["plugins"], reverse=True
-                )
+                pconf["plugins"] = SortedDict(lambda k, v: v["priority"], pconf["plugins"], reverse=True)
         else:
             modified = True
             pconf = {"plugins": SortedDict(lambda k, v: v["priority"], reverse=True)}
@@ -90,26 +79,16 @@ class PluginManager:
                         if plugin_path in self.loaded:
                             if self.loaded[plugin_path] == None:
                                 logger.info("reload module %s" % plugin_name)
-                                self.loaded[plugin_path] = importlib.reload(
-                                    sys.modules[import_path]
-                                )
-                                dependent_module_names = [
-                                    name
-                                    for name in sys.modules.keys()
-                                    if name.startswith(import_path + ".")
-                                ]
+                                self.loaded[plugin_path] = importlib.reload(sys.modules[import_path])
+                                dependent_module_names = [name for name in sys.modules.keys() if name.startswith(import_path + ".")]
                                 for name in dependent_module_names:
                                     logger.info("reload module %s" % name)
                                     importlib.reload(sys.modules[name])
                         else:
-                            self.loaded[plugin_path] = importlib.import_module(
-                                import_path
-                            )
+                            self.loaded[plugin_path] = importlib.import_module(import_path)
                         self.current_plugin_path = None
                     except Exception as e:
-                        logger.exception(
-                            "Failed to import plugin %s: %s" % (plugin_name, e)
-                        )
+                        logger.exception("Failed to import plugin %s: %s" % (plugin_name, e))
                         continue
         pconf = self.pconf
         news = [self.plugins[name] for name in self.plugins]
@@ -119,9 +98,7 @@ class PluginManager:
             rawname = plugincls.name
             if rawname not in pconf["plugins"]:
                 modified = True
-                logger.info(
-                    "Plugin %s not found in pconfig, adding to pconfig..." % name
-                )
+                logger.info("Plugin %s not found in pconfig, adding to pconfig..." % name)
                 pconf["plugins"][rawname] = {
                     "enabled": plugincls.enabled,
                     "priority": plugincls.priority,
@@ -136,9 +113,7 @@ class PluginManager:
 
     def refresh_order(self):
         for event in self.listening_plugins.keys():
-            self.listening_plugins[event].sort(
-                key=lambda name: self.plugins[name].priority, reverse=True
-            )
+            self.listening_plugins[event].sort(key=lambda name: self.plugins[name].priority, reverse=True)
 
     def activate_plugins(self):  # 生成新开启的插件实例
         failed_plugins = []
@@ -148,7 +123,7 @@ class PluginManager:
                     try:
                         instance = plugincls()
                     except Exception as e:
-                        logger.error("Failed to init %s, diabled. %s" % (name, e))
+                        logger.exception("Failed to init %s, diabled. %s" % (name, e))
                         self.disable_plugin(name)
                         failed_plugins.append(name)
                         continue
@@ -184,15 +159,13 @@ class PluginManager:
     def emit_event(self, e_context: EventContext, *args, **kwargs):
         if e_context.event in self.listening_plugins:
             for name in self.listening_plugins[e_context.event]:
-                if (
-                    self.plugins[name].enabled
-                    and e_context.action == EventAction.CONTINUE
-                ):
-                    logger.debug(
-                        "Plugin %s triggered by event %s" % (name, e_context.event)
-                    )
+                if self.plugins[name].enabled and e_context.action == EventAction.CONTINUE:
+                    logger.debug("Plugin %s triggered by event %s" % (name, e_context.event))
                     instance = self.instances[name]
                     instance.handlers[e_context.event](e_context, *args, **kwargs)
+                    if e_context.is_break():
+                        e_context["breaked_by"] = name
+                        logger.debug("Plugin %s breaked event %s" % (name, e_context.event))
         return e_context
 
     def set_plugin_priority(self, name: str, priority: int):
@@ -262,9 +235,7 @@ class PluginManager:
                     source = json.load(f)
                 if repo in source["repo"]:
                     repo = source["repo"][repo]["url"]
-                    match = re.match(
-                        r"^(https?:\/\/|git@)([^\/:]+)[\/:]([^\/:]+)\/(.+).git$", repo
-                    )
+                    match = re.match(r"^(https?:\/\/|git@)([^\/:]+)[\/:]([^\/:]+)\/(.+).git$", repo)
                     if not match:
                         return False, "安装插件失败，source中的仓库地址不合法"
                 else:

plugins/role/role.py

@@ -69,13 +69,9 @@ class Role(Plugin):
             logger.info("[Role] inited")
         except Exception as e:
             if isinstance(e, FileNotFoundError):
-                logger.warn(
-                    f"[Role] init failed, {config_path} not found, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role ."
-                )
+                logger.warn(f"[Role] init failed, {config_path} not found, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role .")
             else:
-                logger.warn(
-                    "[Role] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role ."
-                )
+                logger.warn("[Role] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role .")
             raise e
 
     def get_role(self, name, find_closest=True, min_sim=0.35):
@@ -102,8 +98,8 @@ class Role(Plugin):
     def on_handle_context(self, e_context: EventContext):
         if e_context["context"].type != ContextType.TEXT:
             return
-        bottype = Bridge().get_bot_type("chat")
-        if bottype not in (const.CHATGPT, const.OPEN_AI):
+        btype = Bridge().get_bot_type("chat")
+        if btype not in [const.OPEN_AI, const.CHATGPT, const.CHATGPTONAZURE, const.LINKAI]:
             return
         bot = Bridge().get_bot("chat")
         content = e_context["context"].content[:]
@@ -143,9 +139,7 @@ class Role(Plugin):
                 else:
                     help_text = f"未知角色类型。\n"
                     help_text += "目前的角色类型有: \n"
-                    help_text += (
-                        "，".join([self.tags[tag][0] for tag in self.tags]) + "\n"
-                    )
+                    help_text += "，".join([self.tags[tag][0] for tag in self.tags]) + "\n"
             else:
                 help_text = f"请输入角色类型。\n"
                 help_text += "目前的角色类型有: \n"
@@ -158,9 +152,7 @@ class Role(Plugin):
             return
         logger.debug("[Role] on_handle_context. content: %s" % content)
         if desckey is not None:
-            if len(clist) == 1 or (
-                len(clist) > 1 and clist[1].lower() in ["help", "帮助"]
-            ):
+            if len(clist) == 1 or (len(clist) > 1 and clist[1].lower() in ["help", "帮助"]):
                 reply = Reply(ReplyType.INFO, self.get_help_text(verbose=True))
                 e_context["reply"] = reply
                 e_context.action = EventAction.BREAK_PASS
@@ -178,9 +170,7 @@ class Role(Plugin):
                     self.roles[role][desckey],
                     self.roles[role].get("wrapper", "%s"),
                 )
-                reply = Reply(
-                    ReplyType.INFO, f"预设角色为 {role}:\n" + self.roles[role][desckey]
-                )
+                reply = Reply(ReplyType.INFO, f"预设角色为 {role}:\n" + self.roles[role][desckey])
                 e_context["reply"] = reply
                 e_context.action = EventAction.BREAK_PASS
         elif customize == True:
@@ -199,17 +189,10 @@ class Role(Plugin):
         if not verbose:
             return help_text
         trigger_prefix = conf().get("plugin_trigger_prefix", "$")
-        help_text = (
-            f"使用方法:\n{trigger_prefix}角色"
-            + " 预设角色名: 设定角色为{预设角色名}。\n"
-            + f"{trigger_prefix}role"
-            + " 预设角色名: 同上，但使用英文设定。\n"
-        )
+        help_text = f"使用方法:\n{trigger_prefix}角色" + " 预设角色名: 设定角色为{预设角色名}。\n" + f"{trigger_prefix}role" + " 预设角色名: 同上，但使用英文设定。\n"
         help_text += f"{trigger_prefix}设定扮演" + " 角色设定: 设定自定义角色人设为{角色设定}。\n"
         help_text += f"{trigger_prefix}停止扮演: 清除设定的角色。\n"
-        help_text += (
-            f"{trigger_prefix}角色类型" + " 角色类型: 查看某类{角色类型}的所有预设角色，为所有时输出所有预设角色。\n"
-        )
+        help_text += f"{trigger_prefix}角色类型" + " 角色类型: 查看某类{角色类型}的所有预设角色，为所有时输出所有预设角色。\n"
         help_text += "\n目前的角色类型有: \n"
         help_text += "，".join([self.tags[tag][0] for tag in self.tags]) + "。\n"
         help_text += f"\n命令例子: \n{trigger_prefix}角色 写作助理\n"

plugins/tool/README.md

@@ -1,6 +1,11 @@
 ## 插件描述
-一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力  
-使用该插件需在机器人回复你的前提下，在对话内容前加$tool；仅输入$tool将返回tool插件帮助信息，用于测试插件是否加载成功  
+一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力   
+使用说明(默认trigger_prefix为$)：  
+```text
+#help tool: 查看tool帮助信息，可查看已加载工具列表  
+$tool 命令: 根据给出的{命令}使用一些可用工具尽力为你得到结果。  
+$tool reset: 重置工具。  
+```
 ### 本插件所有工具同步存放至专用仓库：[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)
 
 
@@ -9,9 +14,21 @@
 ### 1. python
 ###### python解释器，使用它来解释执行python指令，可以配合你想要chatgpt生成的代码输出结果或执行事务
 
-### 2. url-get
+### 2. 访问网页的工具汇总(默认url-get)
+
+#### 2.1 url-get
 ###### 往往用来获取某个网站具体内容，结果可能会被反爬策略影响
 
+#### 2.2 browser
+###### 浏览器，功能与2.1类似，但能更好模拟，不会被识别为爬虫影响获取网站内容
+
+> 注1：url-get默认配置、browser需额外配置，browser依赖google-chrome，你需要提前安装好
+
+> 注2：当检测到长文本时会进入summary tool总结长文本，tokens可能会大量消耗！
+
+这是debian端安装google-chrome教程，其他系统请自行查找
+> https://www.linuxjournal.com/content/how-can-you-install-google-browser-debian
+
 ### 3. terminal
 ###### 在你运行的电脑里执行shell命令，可以配合你想要chatgpt生成的代码使用，给予自然语言控制手段
 
@@ -38,51 +55,94 @@
 ### 5. wikipedia
 ###### 可以回答你想要知道确切的人事物
 
-### 6. news *
+### 6. news 新闻类工具集合
+
+> news更新：0.4版本对新闻类工具做了整合，配置文件只要加入`news`一个工具名就会自动加载所有新闻类工具
+
+#### 6.1. news-api *
 ###### 从全球 80,000 多个信息源中获取当前和历史新闻文章
 
-### 7. morning-news *
+#### 6.2. morning-news *
 ###### 每日60秒早报，每天凌晨一点更新，本工具使用了[alapi-每日60秒早报](https://alapi.cn/api/view/93)
 
+```text
+可配置参数：
+1. morning_news_use_llm: 是否使用LLM润色结果，默认false（可能会慢）
+```
+
 > 该tool每天返回内容相同
 
-### 8. bing-search *
+#### 6.3. finance-news
+###### 获取实时的金融财政新闻
+
+> 该工具需要解决browser tool 的google-chrome依赖安装
+
+
+
+### 7. bing-search *
 ###### bing搜索引擎，从此你不用再烦恼搜索要用哪些关键词
 
-### 9. wolfram-alpha *
+### 8. wolfram-alpha *
 ###### 知识搜索引擎、科学问答系统，常用于专业学科计算
 
-### 10. google-search *
+### 9. google-search *
 ###### google搜索引擎，申请流程较bing-search繁琐
 
-###### 注1：带*工具需要获取api-key才能使用，部分工具需要外网支持  
+### 10. arxiv
+###### 用于查找论文
+
+```text
+可配置参数：
+1. arxiv_summary: 是否使用总结工具，默认true, 当为false时会直接返回论文的标题、作者、发布时间、摘要、分类、备注、pdf链接等内容
+```
+
+> 0.4.2更新，例子：帮我找一篇吴恩达写的论文
+
+### 11. summary
+###### 总结工具，该工具必须输入一个本地文件的绝对路径
+
+> 该工具目前是和其他工具配合使用，暂未测试单独使用效果
+
+### 12. image2text
+###### 将图片转换成文字，底层调用imageCaption模型，该工具必须输入一个本地文件的绝对路径
+
+### 13. searxng-search *
+###### 一个私有化的搜索引擎工具
+
+> 安装教程：https://docs.searxng.org/admin/installation.html
+
+---
+
+###### 注1：带*工具需要获取api-key才能使用(在config.json内的kwargs添加项)，部分工具需要外网支持  
 #### [申请方法](https://github.com/goldfishh/chatgpt-tool-hub/blob/master/docs/apply_optional_tool.md)
 
 ## config.json 配置说明
 ###### 默认工具无需配置，其它工具需手动配置，一个例子：
 ```json
 {
-  "tools": ["wikipedia"],  // 填入你想用到的额外工具名
+  "tools": ["wikipedia", "你想要添加的其他工具"],  // 填入你想用到的额外工具名
   "kwargs": {
-      "request_timeout": 60,  // openai接口超时时间
+      "debug": true, // 当你遇到问题求助时，需要配置
+      "request_timeout": 120,  // openai接口超时时间
       "no_default": false,  // 是否不使用默认的4个工具
-      "OPTIONAL_API_NAME": "OPTIONAL_API_KEY"  // 带*工具需要申请api-key，在这里填入，api_name参考前述`申请方法`
+      // 带*工具需要申请api-key，在这里填入，api_name参考前述`申请方法`
   }
 }
 
 ```
 注：config.json文件非必须，未创建仍可使用本tool；带*工具需在kwargs填入对应api-key键值对  
-- `tools`：本插件初始化时加载的工具, 目前可选集：["wikipedia", "wolfram-alpha", "bing-search", "google-search", "news", "morning-news"] & 默认工具，除wikipedia工具之外均需要申请api-key
+- `tools`：本插件初始化时加载的工具, 上述一级标题即是对应工具名称，带*工具必须在kwargs中配置相应api-key
 - `kwargs`：工具执行时的配置，一般在这里存放**api-key**，或环境配置
+  - `debug`: 输出chatgpt-tool-hub额外信息用于调试
   - `request_timeout`: 访问openai接口的超时时间，默认与wechat-on-chatgpt配置一致，可单独配置
   - `no_default`: 用于配置默认加载4个工具的行为，如果为true则仅使用tools列表工具，不加载默认工具
   - `top_k_results`: 控制所有有关搜索的工具返回条目数，数字越高则参考信息越多，但无用信息可能干扰判断，该值一般为2
   - `model_name`: 用于控制tool插件底层使用的llm模型，目前暂未测试3.5以外的模型，一般保持默认
 
+---
 
 ## 备注
 - 强烈建议申请搜索工具搭配使用，推荐bing-search
 - 虽然我会有意加入一些限制，但请不要使用本插件做危害他人的事情，请提前了解清楚某些内容是否会违反相关规定，建议提前做好过滤
 - 如有本插件问题，请将debug设置为true无上下文重新问一遍，如仍有问题请访问[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)建个issue，将日志贴进去，我无法处理不能复现的问题
 - 欢迎 star & 宣传，有能力请提pr
-

plugins/tool/tool.py

@@ -1,7 +1,7 @@
 import json
 import os
 
-from chatgpt_tool_hub.apps import load_app
+from chatgpt_tool_hub.apps import AppFactory
 from chatgpt_tool_hub.apps.app import App
 from chatgpt_tool_hub.tools.all_tool_list import get_all_tool_names
 
@@ -18,7 +18,7 @@ from plugins import *
 @plugins.register(
     name="tool",
     desc="Arming your ChatGPT bot with various tools",
-    version="0.3",
+    version="0.4",
     author="goldfishh",
     desire_priority=0,
 )
@@ -33,12 +33,17 @@ class Tool(Plugin):
 
     def get_help_text(self, verbose=False, **kwargs):
         help_text = "这是一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力。"
+        trigger_prefix = conf().get("plugin_trigger_prefix", "$")
         if not verbose:
             return help_text
-        trigger_prefix = conf().get("plugin_trigger_prefix", "$")
-        help_text += "使用说明：\n"
+        help_text += "\n使用说明：\n"
         help_text += f"{trigger_prefix}tool " + "命令: 根据给出的{命令}使用一些可用工具尽力为你得到结果。\n"
-        help_text += f"{trigger_prefix}tool reset: 重置工具。\n"
+        help_text += f"{trigger_prefix}tool reset: 重置工具。\n\n"
+        help_text += f"已加载工具列表: \n"
+        for idx, tool in enumerate(self.app.get_tool_list()):
+            if idx != 0:
+                help_text += ", "
+            help_text += f"{tool}"
         return help_text
 
     def on_handle_context(self, e_context: EventContext):
@@ -50,6 +55,7 @@ class Tool(Plugin):
             const.CHATGPT,
             const.OPEN_AI,
             const.CHATGPTONAZURE,
+            const.LINKAI,
         ):
             return
 
@@ -82,9 +88,7 @@ class Tool(Plugin):
                     return
                 elif content_list[1].startswith("reset"):
                     logger.debug("[tool]: remind")
-                    e_context[
-                        "context"
-                    ].content = "请你随机用一种聊天风格，提醒用户：如果想重置tool插件，reset之后不要加任何字符"
+                    e_context["context"].content = "请你随机用一种聊天风格，提醒用户：如果想重置tool插件，reset之后不要加任何字符"
 
                     e_context.action = EventAction.BREAK
                     return
@@ -93,18 +97,14 @@ class Tool(Plugin):
 
                 # Don't modify bot name
                 all_sessions = Bridge().get_bot("chat").sessions
-                user_session = all_sessions.session_query(
-                    query, e_context["context"]["session_id"]
-                ).messages
+                user_session = all_sessions.session_query(query, e_context["context"]["session_id"]).messages
 
                 # chatgpt-tool-hub will reply you with many tools
                 logger.debug("[tool]: just-go")
                 try:
                     _reply = self.app.ask(query, user_session)
                     e_context.action = EventAction.BREAK_PASS
-                    all_sessions.session_reply(
-                        _reply, e_context["context"]["session_id"]
-                    )
+                    all_sessions.session_reply(_reply, e_context["context"]["session_id"])
                 except Exception as e:
                     logger.exception(e)
                     logger.error(str(e))
@@ -131,17 +131,18 @@ class Tool(Plugin):
 
     def _build_tool_kwargs(self, kwargs: dict):
         tool_model_name = kwargs.get("model_name")
+        request_timeout = kwargs.get("request_timeout")
 
         return {
+            "debug": kwargs.get("debug", False),
             "openai_api_key": conf().get("open_ai_api_key", ""),
+            "open_ai_api_base": conf().get("open_ai_api_base", "https://api.openai.com/v1"),
             "proxy": conf().get("proxy", ""),
-            "request_timeout": str(conf().get("request_timeout", 60)),
+            "request_timeout": request_timeout if request_timeout else conf().get("request_timeout", 120),
             # note: 目前tool暂未对其他模型测试，但这里仍对配置来源做了优先级区分，一般插件配置可覆盖全局配置
-            "model_name": tool_model_name
-            if tool_model_name
-            else conf().get("model", "gpt-3.5-turbo"),
+            "model_name": tool_model_name if tool_model_name else conf().get("model", "gpt-3.5-turbo"),
             "no_default": kwargs.get("no_default", False),
-            "top_k_results": kwargs.get("top_k_results", 2),
+            "top_k_results": kwargs.get("top_k_results", 3),
             # for news tool
             "news_api_key": kwargs.get("news_api_key", ""),
             # for bing-search tool
@@ -150,15 +151,16 @@ class Tool(Plugin):
             "google_api_key": kwargs.get("google_api_key", ""),
             "google_cse_id": kwargs.get("google_cse_id", ""),
             # for searxng-search tool
-            "searx_host": kwargs.get("searx_host", ""),
+            "searx_search_host": kwargs.get("searx_search_host", ""),
             # for wolfram-alpha tool
             "wolfram_alpha_appid": kwargs.get("wolfram_alpha_appid", ""),
             # for morning-news tool
-            "zaobao_api_key": kwargs.get("zaobao_api_key", ""),
+            "morning_news_api_key": kwargs.get("morning_news_api_key", ""),
             # for visual_dl tool
             "cuda_device": kwargs.get("cuda_device", "cpu"),
-            # for browser tool
-            "phantomjs_exec_path": kwargs.get("phantomjs_exec_path", ""),
+            "think_depth": kwargs.get("think_depth", 3),
+            "arxiv_summary": kwargs.get("arxiv_summary", True),
+            "morning_news_use_llm": kwargs.get("morning_news_use_llm", False),
         }
 
     def _filter_tool_list(self, tool_list: list):
@@ -172,11 +174,12 @@ class Tool(Plugin):
 
     def _reset_app(self) -> App:
         tool_config = self._read_json()
+        app_kwargs = self._build_tool_kwargs(tool_config.get("kwargs", {}))
+
+        app = AppFactory()
+        app.init_env(**app_kwargs)
 
         # filter not support tool
         tool_list = self._filter_tool_list(tool_config.get("tools", []))
 
-        return load_app(
-            tools_list=tool_list,
-            **self._build_tool_kwargs(tool_config.get("kwargs", {})),
-        )
+        return app.create_app(tools_list=tool_list, **app_kwargs)

pyproject.toml

@@ -0,0 +1,8 @@
+[tool.black]
+line-length = 176
+target-version = ['py37']
+include = '\.pyi?$'
+extend-exclude = '.+/(dist|.venv|venv|build|lib)/.+'
+
+[tool.isort]
+profile = "black"

requirements-optional.txt

@@ -6,7 +6,9 @@ SpeechRecognition # google speech to text
 gTTS>=2.3.1 # google text to speech
 pyttsx3>=2.90 # pytsx text to speech
 baidu_aip>=4.16.10 # baidu voice
-# azure-cognitiveservices-speech # azure voice
+azure-cognitiveservices-speech # azure voice
+numpy<=1.24.2
+langid # language detect
 
 #install plugin
 dulwich
@@ -16,9 +18,11 @@ wechaty>=0.10.7
 wechaty_puppet>=0.4.23
 pysilk_mod>=1.6.0 # needed by send voice
 
-# wechatmp
+# wechatmp wechatcom
 web.py
+wechatpy
 
 # chatgpt-tool-hub plugin
+
 --extra-index-url https://pypi.python.org/simple
-chatgpt_tool_hub>=0.3.9
+chatgpt_tool_hub==0.4.4

requirements.txt

@@ -4,4 +4,5 @@ PyQRCode>=1.2.1
 qrcode>=7.4.2
 requests>=2.28.2
 chardet>=5.1.0
+Pillow
 pre-commit

translate/baidu/baidu_translate.py

@@ -0,0 +1,49 @@
+# -*- coding: utf-8 -*-
+
+import random
+from hashlib import md5
+
+import requests
+
+from config import conf
+from translate.translator import Translator
+
+
+class BaiduTranslator(Translator):
+    def __init__(self) -> None:
+        super().__init__()
+        endpoint = "http://api.fanyi.baidu.com"
+        path = "/api/trans/vip/translate"
+        self.url = endpoint + path
+        self.appid = conf().get("baidu_translate_app_id")
+        self.appkey = conf().get("baidu_translate_app_key")
+        if not self.appid or not self.appkey:
+            raise Exception("baidu translate appid or appkey not set")
+
+    # For list of language codes, please refer to `https://api.fanyi.baidu.com/doc/21`, need to convert to ISO 639-1 codes
+    def translate(self, query: str, from_lang: str = "", to_lang: str = "en") -> str:
+        if not from_lang:
+            from_lang = "auto"  # baidu suppport auto detect
+        salt = random.randint(32768, 65536)
+        sign = self.make_md5("{}{}{}{}".format(self.appid, query, salt, self.appkey))
+        headers = {"Content-Type": "application/x-www-form-urlencoded"}
+        payload = {"appid": self.appid, "q": query, "from": from_lang, "to": to_lang, "salt": salt, "sign": sign}
+
+        retry_cnt = 3
+        while retry_cnt:
+            r = requests.post(self.url, params=payload, headers=headers)
+            result = r.json()
+            errcode = result.get("error_code", "52000")
+            if errcode != "52000":
+                if errcode == "52001" or errcode == "52002":
+                    retry_cnt -= 1
+                    continue
+                else:
+                    raise Exception(result["error_msg"])
+            else:
+                break
+        text = "\n".join([item["dst"] for item in result["trans_result"]])
+        return text
+
+    def make_md5(self, s, encoding="utf-8"):
+        return md5(s.encode(encoding)).hexdigest()

translate/factory.py

@@ -0,0 +1,6 @@
+def create_translator(voice_type):
+    if voice_type == "baidu":
+        from translate.baidu.baidu_translate import BaiduTranslator
+
+        return BaiduTranslator()
+    raise RuntimeError

translate/translator.py

@@ -0,0 +1,12 @@
+"""
+Voice service abstract class
+"""
+
+
+class Translator(object):
+    # please use https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes to specify language
+    def translate(self, query: str, from_lang: str = "", to_lang: str = "en") -> str:
+        """
+        Translate text from one language to another
+        """
+        raise NotImplementedError

voice/audio_convert.py

@@ -1,7 +1,13 @@
 import shutil
 import wave
 
-import pysilk
+from common.log import logger
+
+try:
+    import pysilk
+except ImportError:
+    logger.warn("import pysilk failed, wechaty voice message will not be supported.")
+
 from pydub import AudioSegment
 
 sil_supports = [8000, 12000, 16000, 24000, 32000, 44100, 48000]  # slk转wav时，支持的采样率
@@ -34,6 +40,20 @@ def get_pcm_from_wav(wav_path):
     return wav.readframes(wav.getnframes())
 
 
+def any_to_mp3(any_path, mp3_path):
+    """
+    把任意格式转成mp3文件
+    """
+    if any_path.endswith(".mp3"):
+        shutil.copy2(any_path, mp3_path)
+        return
+    if any_path.endswith(".sil") or any_path.endswith(".silk") or any_path.endswith(".slk"):
+        sil_to_wav(any_path, any_path)
+        any_path = mp3_path
+    audio = AudioSegment.from_file(any_path)
+    audio.export(mp3_path, format="mp3")
+
+
 def any_to_wav(any_path, wav_path):
     """
     把任意格式转成wav文件
@@ -41,11 +61,7 @@ def any_to_wav(any_path, wav_path):
     if any_path.endswith(".wav"):
         shutil.copy2(any_path, wav_path)
         return
-    if (
-        any_path.endswith(".sil")
-        or any_path.endswith(".silk")
-        or any_path.endswith(".slk")
-    ):
+    if any_path.endswith(".sil") or any_path.endswith(".silk") or any_path.endswith(".slk"):
         return sil_to_wav(any_path, wav_path)
     audio = AudioSegment.from_file(any_path)
     audio.export(wav_path, format="wav")
@@ -55,60 +71,33 @@ def any_to_sil(any_path, sil_path):
     """
     把任意格式转成sil文件
     """
-    if (
-        any_path.endswith(".sil")
-        or any_path.endswith(".silk")
-        or any_path.endswith(".slk")
-    ):
+    if any_path.endswith(".sil") or any_path.endswith(".silk") or any_path.endswith(".slk"):
         shutil.copy2(any_path, sil_path)
         return 10000
-    if any_path.endswith(".wav"):
-        return pcm_to_sil(any_path, sil_path)
-    if any_path.endswith(".mp3"):
-        return mp3_to_sil(any_path, sil_path)
-    raise NotImplementedError("Not support file type: {}".format(any_path))
-
-
-def mp3_to_wav(mp3_path, wav_path):
-    """
-    把mp3格式转成pcm文件
-    """
-    audio = AudioSegment.from_mp3(mp3_path)
-    audio.export(wav_path, format="wav")
-
-
-def pcm_to_sil(pcm_path, silk_path):
-    """
-    wav 文件转成 silk
-    return 声音长度，毫秒
-    """
-    audio = AudioSegment.from_wav(pcm_path)
+    audio = AudioSegment.from_file(any_path)
     rate = find_closest_sil_supports(audio.frame_rate)
     # Convert to PCM_s16
     pcm_s16 = audio.set_sample_width(2)
     pcm_s16 = pcm_s16.set_frame_rate(rate)
     wav_data = pcm_s16.raw_data
     silk_data = pysilk.encode(wav_data, data_rate=rate, sample_rate=rate)
-    with open(silk_path, "wb") as f:
+    with open(sil_path, "wb") as f:
         f.write(silk_data)
     return audio.duration_seconds * 1000
 
 
-def mp3_to_sil(mp3_path, silk_path):
+def any_to_amr(any_path, amr_path):
     """
-    mp3 文件转成 silk
-    return 声音长度，毫秒
+    把任意格式转成amr文件
     """
-    audio = AudioSegment.from_mp3(mp3_path)
-    rate = find_closest_sil_supports(audio.frame_rate)
-    # Convert to PCM_s16
-    pcm_s16 = audio.set_sample_width(2)
-    pcm_s16 = pcm_s16.set_frame_rate(rate)
-    wav_data = pcm_s16.raw_data
-    silk_data = pysilk.encode(wav_data, data_rate=rate, sample_rate=rate)
-    # Save the silk file
-    with open(silk_path, "wb") as f:
-        f.write(silk_data)
+    if any_path.endswith(".amr"):
+        shutil.copy2(any_path, amr_path)
+        return
+    if any_path.endswith(".sil") or any_path.endswith(".silk") or any_path.endswith(".slk"):
+        raise NotImplementedError("Not support file type: {}".format(any_path))
+    audio = AudioSegment.from_file(any_path)
+    audio = audio.set_frame_rate(8000)  # only support 8000
+    audio.export(amr_path, format="amr")
     return audio.duration_seconds * 1000
 
 
@@ -119,3 +108,26 @@ def sil_to_wav(silk_path, wav_path, rate: int = 24000):
     wav_data = pysilk.decode_file(silk_path, to_wav=True, sample_rate=rate)
     with open(wav_path, "wb") as f:
         f.write(wav_data)
+
+
+def split_audio(file_path, max_segment_length_ms=60000):
+    """
+    分割音频文件
+    """
+    audio = AudioSegment.from_file(file_path)
+    audio_length_ms = len(audio)
+    if audio_length_ms <= max_segment_length_ms:
+        return audio_length_ms, [file_path]
+    segments = []
+    for start_ms in range(0, audio_length_ms, max_segment_length_ms):
+        end_ms = min(audio_length_ms, start_ms + max_segment_length_ms)
+        segment = audio[start_ms:end_ms]
+        segments.append(segment)
+    file_prefix = file_path[: file_path.rindex(".")]
+    format = file_path[file_path.rindex(".") + 1 :]
+    files = []
+    for i, segment in enumerate(segments):
+        path = f"{file_prefix}_{i+1}" + f".{format}"
+        segment.export(path, format=format)
+        files.append(path)
+    return audio_length_ms, files

voice/azure/azure_voice.py

@@ -6,6 +6,7 @@ import os
 import time
 
 import azure.cognitiveservices.speech as speechsdk
+from langid import classify
 
 from bridge.reply import Reply, ReplyType
 from common.log import logger
@@ -30,7 +31,15 @@ class AzureVoice(Voice):
             config = None
             if not os.path.exists(config_path):  # 如果没有配置文件，创建本地配置文件
                 config = {
-                    "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural",
+                    "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural",  # 识别不出时的默认语音
+                    "auto_detect": True,  # 是否自动检测语言
+                    "speech_synthesis_zh": "zh-CN-XiaozhenNeural",
+                    "speech_synthesis_en": "en-US-JacobNeural",
+                    "speech_synthesis_ja": "ja-JP-AoiNeural",
+                    "speech_synthesis_ko": "ko-KR-SoonBokNeural",
+                    "speech_synthesis_de": "de-DE-LouisaNeural",
+                    "speech_synthesis_fr": "fr-FR-BrigitteNeural",
+                    "speech_synthesis_es": "es-ES-LaiaNeural",
                     "speech_recognition_language": "zh-CN",
                 }
                 with open(config_path, "w") as fw:
@@ -38,51 +47,49 @@ class AzureVoice(Voice):
             else:
                 with open(config_path, "r") as fr:
                     config = json.load(fr)
+            self.config = config
             self.api_key = conf().get("azure_voice_api_key")
             self.api_region = conf().get("azure_voice_region")
-            self.speech_config = speechsdk.SpeechConfig(
-                subscription=self.api_key, region=self.api_region
-            )
-            self.speech_config.speech_synthesis_voice_name = config[
-                "speech_synthesis_voice_name"
-            ]
-            self.speech_config.speech_recognition_language = config[
-                "speech_recognition_language"
-            ]
+            self.speech_config = speechsdk.SpeechConfig(subscription=self.api_key, region=self.api_region)
+            self.speech_config.speech_synthesis_voice_name = self.config["speech_synthesis_voice_name"]
+            self.speech_config.speech_recognition_language = self.config["speech_recognition_language"]
         except Exception as e:
             logger.warn("AzureVoice init failed: %s, ignore " % e)
 
     def voiceToText(self, voice_file):
         audio_config = speechsdk.AudioConfig(filename=voice_file)
-        speech_recognizer = speechsdk.SpeechRecognizer(
-            speech_config=self.speech_config, audio_config=audio_config
-        )
+        speech_recognizer = speechsdk.SpeechRecognizer(speech_config=self.speech_config, audio_config=audio_config)
         result = speech_recognizer.recognize_once()
         if result.reason == speechsdk.ResultReason.RecognizedSpeech:
-            logger.info(
-                "[Azure] voiceToText voice file name={} text={}".format(
-                    voice_file, result.text
-                )
-            )
+            logger.info("[Azure] voiceToText voice file name={} text={}".format(voice_file, result.text))
             reply = Reply(ReplyType.TEXT, result.text)
         else:
-            logger.error("[Azure] voiceToText error, result={}".format(result))
+            cancel_details = result.cancellation_details
+            logger.error("[Azure] voiceToText error, result={}, errordetails={}".format(result, cancel_details.error_details))
             reply = Reply(ReplyType.ERROR, "抱歉，语音识别失败")
         return reply
 
     def textToVoice(self, text):
-        fileName = TmpDir().path() + "reply-" + str(int(time.time())) + ".wav"
+        if self.config.get("auto_detect"):
+            lang = classify(text)[0]
+            key = "speech_synthesis_" + lang
+            if key in self.config:
+                logger.info("[Azure] textToVoice auto detect language={}, voice={}".format(lang, self.config[key]))
+                self.speech_config.speech_synthesis_voice_name = self.config[key]
+            else:
+                self.speech_config.speech_synthesis_voice_name = self.config["speech_synthesis_voice_name"]
+        else:
+            self.speech_config.speech_synthesis_voice_name = self.config["speech_synthesis_voice_name"]
+        # Avoid the same filename under multithreading
+        fileName = TmpDir().path() + "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".wav"
         audio_config = speechsdk.AudioConfig(filename=fileName)
-        speech_synthesizer = speechsdk.SpeechSynthesizer(
-            speech_config=self.speech_config, audio_config=audio_config
-        )
+        speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=self.speech_config, audio_config=audio_config)
         result = speech_synthesizer.speak_text(text)
         if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
-            logger.info(
-                "[Azure] textToVoice text={} voice file name={}".format(text, fileName)
-            )
+            logger.info("[Azure] textToVoice text={} voice file name={}".format(text, fileName))
             reply = Reply(ReplyType.VOICE, fileName)
         else:
-            logger.error("[Azure] textToVoice error, result={}".format(result))
+            cancel_details = result.cancellation_details
+            logger.error("[Azure] textToVoice error, result={}, errordetails={}".format(result, cancel_details.error_details))
             reply = Reply(ReplyType.ERROR, "抱歉，语音合成失败")
         return reply

voice/azure/config.json.template

@@ -1,4 +1,12 @@
 {
   "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural",
+  "auto_detect": true,
+  "speech_synthesis_zh": "zh-CN-YunxiNeural",
+  "speech_synthesis_en": "en-US-JacobNeural",
+  "speech_synthesis_ja": "ja-JP-AoiNeural",
+  "speech_synthesis_ko": "ko-KR-SoonBokNeural",
+  "speech_synthesis_de": "de-DE-LouisaNeural",
+  "speech_synthesis_fr": "fr-FR-BrigitteNeural",
+  "speech_synthesis_es": "es-ES-LaiaNeural",
   "speech_recognition_language": "zh-CN"
 }

voice/baidu/baidu_voice.py

@@ -43,9 +43,9 @@ class BaiduVoice(Voice):
                 with open(config_path, "r") as fr:
                     bconf = json.load(fr)
 
-            self.app_id = conf().get("baidu_app_id")
-            self.api_key = conf().get("baidu_api_key")
-            self.secret_key = conf().get("baidu_secret_key")
+            self.app_id = str(conf().get("baidu_app_id"))
+            self.api_key = str(conf().get("baidu_api_key"))
+            self.secret_key = str(conf().get("baidu_secret_key"))
             self.dev_id = conf().get("baidu_dev_pid")
             self.lang = bconf["lang"]
             self.ctp = bconf["ctp"]
@@ -82,12 +82,11 @@ class BaiduVoice(Voice):
             {"spd": self.spd, "pit": self.pit, "vol": self.vol, "per": self.per},
         )
         if not isinstance(result, dict):
-            fileName = TmpDir().path() + "reply-" + str(int(time.time())) + ".mp3"
+            # Avoid the same filename under multithreading
+            fileName = TmpDir().path() + "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".mp3"
             with open(fileName, "wb") as f:
                 f.write(result)
-            logger.info(
-                "[Baidu] textToVoice text={} voice file name={}".format(text, fileName)
-            )
+            logger.info("[Baidu] textToVoice text={} voice file name={}".format(text, fileName))
             reply = Reply(ReplyType.VOICE, fileName)
         else:
             logger.error("[Baidu] textToVoice error={}".format(result))

voice/google/google_voice.py

@@ -24,11 +24,7 @@ class GoogleVoice(Voice):
             audio = self.recognizer.record(source)
         try:
             text = self.recognizer.recognize_google(audio, language="zh-CN")
-            logger.info(
-                "[Google] voiceToText text={} voice file name={}".format(
-                    text, voice_file
-                )
-            )
+            logger.info("[Google] voiceToText text={} voice file name={}".format(text, voice_file))
             reply = Reply(ReplyType.TEXT, text)
         except speech_recognition.UnknownValueError:
             reply = Reply(ReplyType.ERROR, "抱歉，我听不懂")
@@ -39,12 +35,11 @@ class GoogleVoice(Voice):
 
     def textToVoice(self, text):
         try:
-            mp3File = TmpDir().path() + "reply-" + str(int(time.time())) + ".mp3"
+            # Avoid the same filename under multithreading
+            mp3File = TmpDir().path() + "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".mp3"
             tts = gTTS(text=text, lang="zh")
             tts.save(mp3File)
-            logger.info(
-                "[Google] textToVoice text={} voice file name={}".format(text, mp3File)
-            )
+            logger.info("[Google] textToVoice text={} voice file name={}".format(text, mp3File))
             reply = Reply(ReplyType.VOICE, mp3File)
         except Exception as e:
             reply = Reply(ReplyType.ERROR, str(e))

voice/openai/openai_voice.py

@@ -22,11 +22,7 @@ class OpenaiVoice(Voice):
             result = openai.Audio.transcribe("whisper-1", file)
             text = result["text"]
             reply = Reply(ReplyType.TEXT, text)
-            logger.info(
-                "[Openai] voiceToText text={} voice file name={}".format(
-                    text, voice_file
-                )
-            )
+            logger.info("[Openai] voiceToText text={} voice file name={}".format(text, voice_file))
         except Exception as e:
             reply = Reply(ReplyType.ERROR, str(e))
         finally:

voice/pytts/pytts_voice.py

@@ -2,6 +2,8 @@
 pytts voice service (offline)
 """
 
+import os
+import sys
 import time
 
 import pyttsx3
@@ -20,19 +22,42 @@ class PyttsVoice(Voice):
         self.engine.setProperty("rate", 125)
         # 音量
         self.engine.setProperty("volume", 1.0)
-        for voice in self.engine.getProperty("voices"):
-            if "Chinese" in voice.name:
-                self.engine.setProperty("voice", voice.id)
+        if sys.platform == "win32":
+            for voice in self.engine.getProperty("voices"):
+                if "Chinese" in voice.name:
+                    self.engine.setProperty("voice", voice.id)
+        else:
+            self.engine.setProperty("voice", "zh")
+            # If the problem of espeak is fixed, using runAndWait() and remove this startLoop()
+            # TODO: check if this is work on win32
+            self.engine.startLoop(useDriverLoop=False)
 
     def textToVoice(self, text):
         try:
-            wavFile = TmpDir().path() + "reply-" + str(int(time.time())) + ".wav"
+            # Avoid the same filename under multithreading
+            wavFileName = "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".wav"
+            wavFile = TmpDir().path() + wavFileName
+            logger.info("[Pytts] textToVoice text={} voice file name={}".format(text, wavFile))
+
             self.engine.save_to_file(text, wavFile)
-            self.engine.runAndWait()
-            logger.info(
-                "[Pytts] textToVoice text={} voice file name={}".format(text, wavFile)
-            )
+
+            if sys.platform == "win32":
+                self.engine.runAndWait()
+            else:
+                # In ubuntu, runAndWait do not really wait until the file created.
+                # It will return once the task queue is empty, but the task is still running in coroutine.
+                # And if you call runAndWait() and time.sleep() twice, it will stuck, so do not use this.
+                # If you want to fix this, add self._proxy.setBusy(True) in line 127 in espeak.py, at the beginning of the function save_to_file.
+                # self.engine.runAndWait()
+
+                # Before espeak fix this problem, we iterate the generator and control the waiting by ourself.
+                # But this is not the canonical way to use it, for example if the file already exists it also cannot wait.
+                self.engine.iterate()
+                while self.engine.isBusy() or wavFileName not in os.listdir(TmpDir().path()):
+                    time.sleep(0.1)
+
             reply = Reply(ReplyType.VOICE, wavFile)
+
         except Exception as e:
             reply = Reply(ReplyType.ERROR, str(e))
         finally: