Update README.md

tool update

.github/ISSUE_TEMPLATE.md

@@ -1,9 +1,12 @@
 ### 前置确认
 
 1. 网络能够访问openai接口
-2. python 已安装：版本在 3.7 ~ 3.10 之间，依赖已安装
-3. 在已有 issue 中未搜索到类似问题
-4. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
+2. python 已安装：版本在 3.7 ~ 3.10 之间
+3. `git pull` 拉取最新代码
+4. 执行`pip3 install -r requirements.txt`，检查依赖是否满足
+5. 拓展功能请执行`pip3 install -r requirements-optional.txt`，检查依赖是否满足
+6. 在已有 issue 中未搜索到类似问题
+7. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
 
 
 ### 问题描述
@@ -16,7 +19,7 @@
 ### 终端日志 (如有报错)
 
 ```
-[在此处粘贴终端日志]
+[在此处粘贴终端日志, 可在主目录下`run.log`文件中找到]
 ```
 
 

.github/workflows/deploy-image.yml

@@ -0,0 +1,59 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# GitHub recommends pinning actions to a commit SHA.
+# To get a newer version, you will need to update the SHA.
+# You can also reference a tag or branch, but the action may change without warning.
+
+name: Create and publish a Docker image
+
+on:
+  push:
+    branches: ['master']
+  create:
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          push: true
+          file: ./docker/Dockerfile.latest
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+      
+      - uses: actions/delete-package-versions@v4
+        with: 
+          package-name: 'chatgpt-on-wechat'
+          package-type: 'container'
+          min-versions-to-keep: 10
+          delete-only-untagged-versions: 'true'
+          token: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -10,3 +10,14 @@ nohup.out
 tmp
 plugins.json
 itchat.pkl
+*.log
+user_datas.pkl
+plugins/**/
+!plugins/bdunit
+!plugins/dungeon
+!plugins/finish
+!plugins/godcmd
+!plugins/tool
+!plugins/banwords
+!plugins/hello
+!plugins/role

Dockerfile

@@ -0,0 +1,3 @@
+FROM ghcr.io/zhayujie/chatgpt-on-wechat:latest
+
+ENTRYPOINT ["/entrypoint.sh"]

README.md

@@ -2,7 +2,7 @@
 
 > ChatGPT近期以强大的对话和信息整合能力风靡全网，可以写代码、改论文、讲故事，几乎无所不能，这让人不禁有个大胆的想法，能否用他的对话模型把我们的微信打造成一个智能机器人，可以在与好友对话中给出意想不到的回应，而且再也不用担心女朋友影响我们 ~~打游戏~~ 工作了。
 
- 
+
 基于ChatGPT的微信聊天机器人，通过 [ChatGPT](https://github.com/openai/openai-python) 接口生成对话内容，使用 [itchat](https://github.com/littlecodersh/ItChat) 实现微信消息的接收和自动回复。已实现的特性如下：
 
 - [x] **文本对话：** 接收私聊及群组中的微信消息，使用ChatGPT生成回复内容，完成自动回复
@@ -11,13 +11,21 @@
 - [x] **图片生成：** 支持根据描述生成图片，并自动发送至个人聊天或群聊
 - [x] **上下文记忆**：支持多轮对话记忆，且为每个好友维护独立的上下会话
 - [x] **语音识别：** 支持接收和处理语音消息，通过文字或语音回复
+- [x] **插件化：** 支持个性化功能插件，提供角色扮演、文字冒险游戏等预设插件
 
+> 快速部署:
+>
+>[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/qApznZ?referralCode=RC3znh)
 
 # 更新日志
 
+>**2023.04.05：** 支持微信个人号部署，兼容角色扮演等预设插件，[使用文档](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/wechatmp/README.md)。(contributed by [@JS00000](https://github.com/JS00000) in [#686](https://github.com/zhayujie/chatgpt-on-wechat/pull/686))
+
+>**2023.04.05：** 增加能让ChatGPT使用工具的`tool`插件，[使用文档](https://github.com/goldfishh/chatgpt-on-wechat/blob/master/plugins/tool/README.md)。工具相关issue可反馈至[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)。(contributed by [@goldfishh](https://github.com/goldfishh) in [#663](https://github.com/zhayujie/chatgpt-on-wechat/pull/663))
+
 >**2023.03.25：** 支持插件化开发，目前已实现 多角色切换、文字冒险游戏、管理员指令、Stable Diffusion等插件，使用参考 [#578](https://github.com/zhayujie/chatgpt-on-wechat/issues/578)。(contributed by [@lanvent](https://github.com/lanvent) in [#565](https://github.com/zhayujie/chatgpt-on-wechat/pull/565))
 
->**2023.03.09：** 基于 `whisper API` 实现对微信语音消息的解析和回复，添加配置项 `"speech_recognition":true` 即可启用，使用参考 [#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)。(contributed by [wanggang1987](https://github.com/wanggang1987) in [#385](https://github.com/zhayujie/chatgpt-on-wechat/pull/385))
+>**2023.03.09：** 基于 `whisper API`(后续已接入更多的语音`API`服务) 实现对微信语音消息的解析和回复，添加配置项 `"speech_recognition":true` 即可启用，使用参考 [#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)。(contributed by [wanggang1987](https://github.com/wanggang1987) in [#385](https://github.com/zhayujie/chatgpt-on-wechat/pull/385))
 
 >**2023.03.02：** 接入[ChatGPT API](https://platform.openai.com/docs/guides/chat) (gpt-3.5-turbo)，默认使用该模型进行对话，需升级openai依赖 (`pip3 install --upgrade openai`)。网络问题参考 [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
 
@@ -54,14 +62,10 @@
 
 > 项目中使用的对话模型是 davinci，计费方式是约每 750 字 (包含请求和回复) 消耗 $0.02，图片生成是每张消耗 $0.016，账号创建有免费的 $18 额度 (更新3.25: 最新注册的已经无免费额度了)，使用完可以更换邮箱重新注册。
 
-#### 1.1 ChapGPT service On Azure
-一种替换以上的方法是使用Azure推出的[ChatGPT service](https://azure.microsoft.com/en-in/products/cognitive-services/openai-service/)。它host在公有云Azure上，因此不需要VPN就可以直接访问。不过目前仍然处于preview阶段。新用户可以通过Try Azure for free来薅一段时间的羊毛
-
-
 ### 2.运行环境
 
-支持 Linux、MacOS、Windows 系统（可在Linux服务器上长期运行)，同时需安装 `Python`。 
-> 建议Python版本在 3.7.1~3.9.X 之间，3.10及以上版本在 MacOS 可用，其他系统上不确定能否正常运行。
+支持 Linux、MacOS、Windows 系统（可在Linux服务器上长期运行)，同时需安装 `Python`。
+> 建议Python版本在 3.7.1~3.9.X 之间，推荐3.8版本，3.10及以上版本在 MacOS 可用，其他系统上不确定能否正常运行。
 
 **(1) 克隆项目代码：**
 
@@ -71,17 +75,35 @@ cd chatgpt-on-wechat/
 ```
 
 **(2) 安装核心依赖 (必选)：**
+> 能够使用`itchat`创建机器人，并具有文字交流功能所需的最小依赖集合。
+```bash
+pip3 install -r requirements.txt
+```
+
+**(3) 拓展依赖 (可选，建议安装)：**
 
 ```bash
-pip3 install itchat-uos==1.5.0.dev0
-pip3 install --upgrade openai
+pip3 install -r requirements-optional.txt
 ```
-注：`itchat-uos`使用指定版本1.5.0.dev0，`openai`使用最新版本，需高于0.27.0。
+> 如果某项依赖安装失败请注释掉对应的行再继续。
 
-**(3) 拓展依赖 (可选)：**
+其中`tiktoken`要求`python`版本在3.8以上，它用于精确计算会话使用的tokens数量，强烈建议安装。
 
-语音识别及语音回复相关依赖：[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)。
 
+使用`google`或`baidu`语音识别需安装`ffmpeg`，
+
+默认的`openai`语音识别不需要安装`ffmpeg`。
+
+参考[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)
+
+使用`azure`语音功能需安装依赖（列在`requirements-optional.txt`内，但为便于`railway`部署已注释）:
+
+```bash
+pip3 install azure-cognitiveservices-speech
+```
+
+> 目前默认发布的镜像和`railway`部署，都基于`apline`，无法安装`azure`的依赖。若有需求请自行基于[`debian`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/docker/Dockerfile.debian.latest)打包。
+参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)
 
 ## 配置
 
@@ -107,6 +129,7 @@ pip3 install --upgrade openai
   "image_create_prefix": ["画", "看", "找"],                   # 开启图片回复的前缀
   "conversation_max_tokens": 1000,                            # 支持上下文记忆的最多字符数
   "speech_recognition": false,                                # 是否开启语音识别
+  "group_speech_recognition": false,                          # 是否开启群组语音识别
   "use_azure_chatgpt": false,                                 # 是否使用Azure ChatGPT service代替openai ChatGPT service. 当设置为true时需要设置 open_ai_api_base，如 https://xxx.openai.azure.com/
   "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述,
 }
@@ -127,8 +150,9 @@ pip3 install --upgrade openai
 
 **3.语音识别**
 
-+ 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，目前只支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复)；
-+ 添加 `"voice_reply_voice": true` 将开启语音回复语音，但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
++ 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，该参数仅支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复，支持语音触发画图)；
++ 添加 `"group_speech_recognition": true` 将开启群组语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，参数仅支持群聊 (会匹配group_chat_prefix和group_chat_keyword, 支持语音触发画图)；
++ 添加 `"voice_reply_voice": true` 将开启语音回复语音（同时作用于私聊和群聊），但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
 
 **4.其他配置**
 
@@ -143,6 +167,7 @@ pip3 install --upgrade openai
 + `hot_reload`: 程序退出后，暂存微信扫码状态，默认关闭。
 + `character_desc` 配置中保存着你对机器人说的一段话，他会记住这段话并作为他的设定，你可以为他定制任何人格      (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
 
+**所有可选的配置项均在该[文件](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
 
 ## 运行
 
@@ -166,25 +191,20 @@ nohup python3 app.py & tail -f nohup.out          # 在后台运行程序并通
 ```
 扫码登录后程序即可运行于服务器后台，此时可通过 `ctrl+c` 关闭日志，不会影响后台程序的运行。使用 `ps -ef | grep app.py | grep -v grep` 命令可查看运行于后台的进程，如果想要重新启动程序可以先 `kill` 掉对应的进程。日志关闭后如果想要再次打开只需输入 `tail -f nohup.out`。此外，`scripts` 目录下有一键运行、关闭程序的脚本供使用。
 
-> **注意：** 如果 扫码后手机提示登录验证需要等待5s，而终端的二维码再次刷新并提示 `Log in time out, reloading QR code`，此时需参考此 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/8) 修改一行代码即可解决。
+> **多账号支持：** 将项目复制多份，分别启动程序，用不同账号扫码登录即可实现同时运行。
 
-> **多账号支持：** 将 项目复制多份，分别启动程序，用不同账号扫码登录即可实现同时运行。
-
-> **特殊指令：** 用户向机器人发送 **#清除记忆** 即可清空该用户的上下文记忆。
+> **特殊指令：** 用户向机器人发送 **#reset** 即可清空该用户的上下文记忆。
 
 
 ### 3.Docker部署
 
 参考文档 [Docker部署](https://github.com/limccn/chatgpt-on-wechat/wiki/Docker%E9%83%A8%E7%BD%B2)   (Contributed by [limccn](https://github.com/limccn))。
 
-### 4. Railway部署
-[Use with Railway](#use-with-railway)(PaaS, Free, Stable, ✅Recommended)
-> Railway offers $5 (500 hours) of runtime per month
-1. Click the [Railway](https://railway.app/) button to go to the Railway homepage
-2. Click the `Start New Project` button.
-3. Click the `Deploy from Github repo` button.
-4. Choose your repo (you can fork this repo firstly)
-5. Set environment variable to override settings in config-template.json, such as: model, open_ai_api_base, open_ai_api_key, use_azure_chatgpt etc.
+### 4. Railway部署(✅推荐)
+> Railway每月提供5刀和最多500小时的免费额度。
+1. 进入 [Railway](https://railway.app/template/qApznZ?referralCode=RC3znh)。
+2. 点击 `Deploy Now` 按钮。
+3. 设置环境变量来重载程序运行的参数，例如`open_ai_api_key`, `character_desc`。
 
 ## 常见问题
 

app.py

@@ -1,20 +1,42 @@
 # encoding:utf-8
 
-import config
+import os
+from config import conf, load_config
 from channel import channel_factory
 from common.log import logger
-
 from plugins import *
+import signal
+import sys
+
+def sigterm_handler_wrap(_signo):
+    old_handler = signal.getsignal(_signo)
+    def func(_signo, _stack_frame):
+        logger.info("signal {} received, exiting...".format(_signo))
+        conf().save_user_datas()
+        return old_handler(_signo, _stack_frame)
+    signal.signal(_signo, func)
 
 def run():
     try:
         # load config
-        config.load_config()
+        load_config()
+        # ctrl + c
+        sigterm_handler_wrap(signal.SIGINT)
+        # kill signal
+        sigterm_handler_wrap(signal.SIGTERM)
 
         # create channel
-        channel_name='wx'
+        channel_name=conf().get('channel_type', 'wx')
+
+        if "--cmd" in sys.argv:
+            channel_name = 'terminal'
+
+        if channel_name == 'wxy':
+            os.environ['WECHATY_LOG']="warn"
+            # os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:9001'
+
         channel = channel_factory.create_channel(channel_name)
-        if channel_name=='wx':
+        if channel_name in ['wx','wxy','wechatmp','terminal']:
             PluginManager().load_plugins()
 
         # startup channel

bot/bot_factory.py

@@ -6,9 +6,9 @@ from common import const
 
 def create_bot(bot_type):
     """
-    create a channel instance
-    :param channel_type: channel type code
-    :return: channel instance
+    create a bot_type instance
+    :param bot_type: bot type code
+    :return: bot instance
     """
     if bot_type == const.BAIDU:
         # Baidu Unit对话接口

bot/chatgpt/chat_gpt_bot.py

@@ -1,6 +1,9 @@
 # encoding:utf-8
 
 from bot.bot import Bot
+from bot.chatgpt.chat_gpt_session import ChatGPTSession
+from bot.openai.open_ai_image import OpenAIImage
+from bot.session_manager import Session, SessionManager
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from config import conf, load_config
@@ -8,28 +11,30 @@ from common.log import logger
 from common.token_bucket import TokenBucket
 from common.expired_dict import ExpiredDict
 import openai
+import openai.error
 import time
 
-
 # OpenAI对话模型API (可用)
-class ChatGPTBot(Bot):
+class ChatGPTBot(Bot,OpenAIImage):
     def __init__(self):
+        super().__init__()
+        # set the default api_key
         openai.api_key = conf().get('open_ai_api_key')
         if conf().get('open_ai_api_base'):
             openai.api_base = conf().get('open_ai_api_base')
         proxy = conf().get('proxy')
-        self.sessions = SessionManager()
         if proxy:
             openai.proxy = proxy
         if conf().get('rate_limit_chatgpt'):
             self.tb4chatgpt = TokenBucket(conf().get('rate_limit_chatgpt', 20))
-        if conf().get('rate_limit_dalle'):
-            self.tb4dalle = TokenBucket(conf().get('rate_limit_dalle', 50))
+        
+        self.sessions = SessionManager(ChatGPTSession, model= conf().get("model") or "gpt-3.5-turbo")
 
     def reply(self, query, context=None):
         # acquire reply content
         if context.type == ContextType.TEXT:
-            logger.info("[OPEN_AI] query={}".format(query))
+            logger.info("[CHATGPT] query={}".format(query))
+
 
             session_id = context['session_id']
             reply = None
@@ -45,23 +50,25 @@ class ChatGPTBot(Bot):
                 reply = Reply(ReplyType.INFO, '配置已更新')
             if reply:
                 return reply
-            session = self.sessions.build_session_query(query, session_id)
-            logger.debug("[OPEN_AI] session query={}".format(session))
+            session = self.sessions.session_query(query, session_id)
+            logger.debug("[CHATGPT] session query={}".format(session.messages))
+
+            api_key = context.get('openai_api_key')
 
             # if context.get('stream'):
             #     # reply in stream
             #     return self.reply_text_stream(query, new_query, session_id)
 
-            reply_content = self.reply_text(session, session_id, 0)
-            logger.debug("[OPEN_AI] new_query={}, session_id={}, reply_cont={}".format(session, session_id, reply_content["content"]))
+            reply_content = self.reply_text(session, session_id, api_key, 0)
+            logger.debug("[CHATGPT] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(session.messages, session_id, reply_content["content"], reply_content["completion_tokens"]))
             if reply_content['completion_tokens'] == 0 and len(reply_content['content']) > 0:
                 reply = Reply(ReplyType.ERROR, reply_content['content'])
             elif reply_content["completion_tokens"] > 0:
-                self.sessions.save_session(reply_content["content"], session_id, reply_content["total_tokens"])
+                self.sessions.session_reply(reply_content["content"], session_id, reply_content["total_tokens"])
                 reply = Reply(ReplyType.TEXT, reply_content["content"])
             else:
                 reply = Reply(ReplyType.ERROR, reply_content['content'])
-                logger.debug("[OPEN_AI] reply {} used 0 tokens.".format(reply_content))
+                logger.debug("[CHATGPT] reply {} used 0 tokens.".format(reply_content))
             return reply
 
         elif context.type == ContextType.IMAGE_CREATE:
@@ -84,9 +91,11 @@ class ChatGPTBot(Bot):
             "top_p":1,
             "frequency_penalty":conf().get('frequency_penalty', 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
             "presence_penalty":conf().get('presence_penalty', 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "request_timeout": conf().get('request_timeout', 60),  # 请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
+            "timeout": conf().get('request_timeout', 120), #重试超时时间，在这个时间内，将会自动重试
         }
 
-    def reply_text(self, session, session_id, retry_count=0) -> dict:
+    def reply_text(self, session:ChatGPTSession, session_id, api_key, retry_count=0) -> dict:
         '''
         call openai's ChatCompletion to get the answer
         :param session: a conversation session
@@ -96,62 +105,42 @@ class ChatGPTBot(Bot):
         '''
         try:
             if conf().get('rate_limit_chatgpt') and not self.tb4chatgpt.get_token():
-                return {"completion_tokens": 0, "content": "提问太快啦，请休息一下再问我吧"}
+                raise openai.error.RateLimitError("RateLimitError: rate limit exceeded")
+            # if api_key == None, the default openai.api_key will be used
             response = openai.ChatCompletion.create(
-                messages=session, **self.compose_args()
+                api_key=api_key, messages=session.messages, **self.compose_args()
             )
             # logger.info("[ChatGPT] reply={}, total_tokens={}".format(response.choices[0]['message']['content'], response["usage"]["total_tokens"]))
             return {"total_tokens": response["usage"]["total_tokens"],
                     "completion_tokens": response["usage"]["completion_tokens"],
                     "content": response.choices[0]['message']['content']}
-        except openai.error.RateLimitError as e:
-            # rate limit exception
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(session, session_id, retry_count+1)
-            else:
-                return {"completion_tokens": 0, "content": "提问太快啦，请休息一下再问我吧"}
-        except openai.error.APIConnectionError as e:
-            # api connection exception
-            logger.warn(e)
-            logger.warn("[OPEN_AI] APIConnection failed")
-            return {"completion_tokens": 0, "content": "我连接不到你的网络"}
-        except openai.error.Timeout as e:
-            logger.warn(e)
-            logger.warn("[OPEN_AI] Timeout")
-            return {"completion_tokens": 0, "content": "我没有收到你的消息"}
         except Exception as e:
-            # unknown exception
-            logger.exception(e)
-            self.sessions.clear_session(session_id)
-            return {"completion_tokens": 0, "content": "请再问我一次吧"}
+            need_retry = retry_count < 2
+            result = {"completion_tokens": 0, "content": "我现在有点累了，等会再来吧"}
+            if isinstance(e, openai.error.RateLimitError):
+                logger.warn("[CHATGPT] RateLimitError: {}".format(e))
+                result['content'] = "提问太快啦，请休息一下再问我吧"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.Timeout):
+                logger.warn("[CHATGPT] Timeout: {}".format(e))
+                result['content'] = "我没有收到你的消息"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.APIConnectionError):
+                logger.warn("[CHATGPT] APIConnectionError: {}".format(e))
+                need_retry = False
+                result['content'] = "我连接不到你的网络"
+            else:
+                logger.warn("[CHATGPT] Exception: {}".format(e))
+                need_retry = False
+                self.sessions.clear_session(session_id)
 
-    def create_img(self, query, retry_count=0):
-        try:
-            if conf().get('rate_limit_dalle') and not self.tb4dalle.get_token():
-                return False, "请求太快了，请休息一下再问我吧"
-            logger.info("[OPEN_AI] image_query={}".format(query))
-            response = openai.Image.create(
-                prompt=query,    #图片描述
-                n=1,             #每次生成图片的数量
-                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
-            )
-            image_url = response['data'][0]['url']
-            logger.info("[OPEN_AI] image_url={}".format(image_url))
-            return True, image_url
-        except openai.error.RateLimitError as e:
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.create_img(query, retry_count+1)
+            if need_retry:
+                logger.warn("[CHATGPT] 第{}次重试".format(retry_count+1))
+                return self.reply_text(session, session_id, api_key, retry_count+1)
             else:
-                return False, "提问太快啦，请休息一下再问我吧"
-        except Exception as e:
-            logger.exception(e)
-            return False, str(e)
+                return result
 
 
 class AzureChatGPTBot(ChatGPTBot):
@@ -164,75 +153,4 @@ class AzureChatGPTBot(ChatGPTBot):
         args = super().compose_args()
         args["engine"] = args["model"]
         del(args["model"])
-        return args
-
-
-class SessionManager(object):
-    def __init__(self):
-        if conf().get('expires_in_seconds'):
-            sessions = ExpiredDict(conf().get('expires_in_seconds'))
-        else:
-            sessions = dict()
-        self.sessions = sessions
-
-    def build_session(self, session_id, system_prompt=None):
-        session = self.sessions.get(session_id, [])
-        if len(session) == 0:
-            if system_prompt is None:
-                system_prompt = conf().get("character_desc", "")
-            system_item = {'role': 'system', 'content': system_prompt}
-            session.append(system_item)
-            self.sessions[session_id] = session
-        return session
-
-    def build_session_query(self, query, session_id):
-        '''
-        build query with conversation history
-        e.g.  [
-            {"role": "system", "content": "You are a helpful assistant."},
-            {"role": "user", "content": "Who won the world series in 2020?"},
-            {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
-            {"role": "user", "content": "Where was it played?"}
-        ]
-        :param query: query content
-        :param session_id: session id
-        :return: query content with conversaction
-        '''
-        session = self.build_session(session_id)
-        user_item = {'role': 'user', 'content': query}
-        session.append(user_item)
-        return session
-
-    def save_session(self, answer, session_id, total_tokens):
-        max_tokens = conf().get("conversation_max_tokens")
-        if not max_tokens:
-            # default 3000
-            max_tokens = 1000
-        max_tokens = int(max_tokens)
-
-        session = self.sessions.get(session_id)
-        if session:
-            # append conversation
-            gpt_item = {'role': 'assistant', 'content': answer}
-            session.append(gpt_item)
-
-        # discard exceed limit conversation
-        self.discard_exceed_conversation(session, max_tokens, total_tokens)
-
-    def discard_exceed_conversation(self, session, max_tokens, total_tokens):
-        dec_tokens = int(total_tokens)
-        # logger.info("prompt tokens used={},max_tokens={}".format(used_tokens,max_tokens))
-        while dec_tokens > max_tokens:
-            # pop first conversation
-            if len(session) > 3:
-                session.pop(1)
-                session.pop(1)
-            else:
-                break
-            dec_tokens = dec_tokens - max_tokens
-
-    def clear_session(self, session_id):
-        self.sessions[session_id] = []
-
-    def clear_all_session(self):
-        self.sessions.clear()
+        return args

bot/chatgpt/chat_gpt_session.py

@@ -0,0 +1,79 @@
+from bot.session_manager import Session
+from common.log import logger
+'''
+    e.g.  [
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Who won the world series in 2020?"},
+        {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
+        {"role": "user", "content": "Where was it played?"}
+    ]
+'''
+class ChatGPTSession(Session):
+    def __init__(self, session_id, system_prompt=None, model= "gpt-3.5-turbo"):
+        super().__init__(session_id, system_prompt)
+        self.model = model
+        self.reset()
+    
+    def discard_exceeding(self, max_tokens, cur_tokens= None):
+        precise = True
+        try:
+            cur_tokens = num_tokens_from_messages(self.messages, self.model)
+        except Exception as e:
+            precise = False
+            if cur_tokens is None:
+                raise e
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
+        while cur_tokens > max_tokens:
+            if len(self.messages) > 2:
+                self.messages.pop(1)
+            elif len(self.messages) == 2 and self.messages[1]["role"] == "assistant":
+                self.messages.pop(1)
+                if precise:
+                    cur_tokens = num_tokens_from_messages(self.messages, self.model)
+                else:
+                    cur_tokens = cur_tokens - max_tokens
+                break
+            elif len(self.messages) == 2 and self.messages[1]["role"] == "user":
+                logger.warn("user message exceed max_tokens. total_tokens={}".format(cur_tokens))
+                break
+            else:
+                logger.debug("max_tokens={}, total_tokens={}, len(messages)={}".format(max_tokens, cur_tokens, len(self.messages)))
+                break
+            if precise:
+                cur_tokens = num_tokens_from_messages(self.messages, self.model)
+            else:
+                cur_tokens = cur_tokens - max_tokens
+        return cur_tokens
+    
+
+# refer to https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+def num_tokens_from_messages(messages, model):
+    """Returns the number of tokens used by a list of messages."""
+    import tiktoken
+    try:
+        encoding = tiktoken.encoding_for_model(model)
+    except KeyError:
+        logger.debug("Warning: model not found. Using cl100k_base encoding.")
+        encoding = tiktoken.get_encoding("cl100k_base")
+    if model == "gpt-3.5-turbo":
+        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
+    elif model == "gpt-4":
+        return num_tokens_from_messages(messages, model="gpt-4-0314")
+    elif model == "gpt-3.5-turbo-0301":
+        tokens_per_message = 4  # every message follows <|start|>{role/name}\n{content}<|end|>\n
+        tokens_per_name = -1  # if there's a name, the role is omitted
+    elif model == "gpt-4-0314":
+        tokens_per_message = 3
+        tokens_per_name = 1
+    else:
+        logger.warn(f"num_tokens_from_messages() is not implemented for model {model}. Returning num tokens assuming gpt-3.5-turbo-0301.")
+        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
+    num_tokens = 0
+    for message in messages:
+        num_tokens += tokens_per_message
+        for key, value in message.items():
+            num_tokens += len(encoding.encode(value))
+            if key == "name":
+                num_tokens += tokens_per_name
+    num_tokens += 3  # every reply is primed with <|start|>assistant<|message|>
+    return num_tokens

bot/openai/open_ai_bot.py

@@ -1,18 +1,23 @@
 # encoding:utf-8
 
 from bot.bot import Bot
+from bot.openai.open_ai_image import OpenAIImage
+from bot.openai.open_ai_session import OpenAISession
+from bot.session_manager import SessionManager
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from config import conf
 from common.log import logger
 import openai
+import openai.error
 import time
 
 user_session = dict()
 
 # OpenAI对话模型API (可用)
-class OpenAIBot(Bot):
+class OpenAIBot(Bot, OpenAIImage):
     def __init__(self):
+        super().__init__()
         openai.api_key = conf().get('open_ai_api_key')
         if conf().get('open_ai_api_base'):
             openai.api_base = conf().get('open_ai_api_base')
@@ -20,34 +25,45 @@ class OpenAIBot(Bot):
         if proxy:
             openai.proxy = proxy
 
+        self.sessions = SessionManager(OpenAISession, model= conf().get("model") or "text-davinci-003")
 
     def reply(self, query, context=None):
         # acquire reply content
         if context and context.type:
             if context.type == ContextType.TEXT:
                 logger.info("[OPEN_AI] query={}".format(query))
-                from_user_id = context['session_id']
+                session_id = context['session_id']
                 reply = None
                 if query == '#清除记忆':
-                    Session.clear_session(from_user_id)
+                    self.sessions.clear_session(session_id)
                     reply = Reply(ReplyType.INFO, '记忆已清除')
                 elif query == '#清除所有':
-                    Session.clear_all_session()
+                    self.sessions.clear_all_session()
                     reply = Reply(ReplyType.INFO, '所有人记忆已清除')
                 else:
-                    new_query = Session.build_session_query(query, from_user_id)
+                    session = self.sessions.session_query(query, session_id)
+                    new_query = str(session)
                     logger.debug("[OPEN_AI] session query={}".format(new_query))
 
-                    reply_content = self.reply_text(new_query, from_user_id, 0)
-                    logger.debug("[OPEN_AI] new_query={}, user={}, reply_cont={}".format(new_query, from_user_id, reply_content))
-                    if reply_content and query:
-                        Session.save_session(query, reply_content, from_user_id)
-                    reply = Reply(ReplyType.TEXT, reply_content)
+                    total_tokens, completion_tokens, reply_content = self.reply_text(new_query, session_id, 0)
+                    logger.debug("[OPEN_AI] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(new_query, session_id, reply_content, completion_tokens))
+
+                    if total_tokens == 0 :
+                        reply = Reply(ReplyType.ERROR, reply_content)
+                    else:
+                        self.sessions.session_reply(reply_content, session_id, total_tokens)
+                        reply = Reply(ReplyType.TEXT, reply_content)
                 return reply
             elif context.type == ContextType.IMAGE_CREATE:
-                return self.create_img(query, 0)
+                ok, retstring = self.create_img(query, 0)
+                reply = None
+                if ok:
+                    reply = Reply(ReplyType.IMAGE_URL, retstring)
+                else:
+                    reply = Reply(ReplyType.ERROR, retstring)
+                return reply
 
-    def reply_text(self, query, user_id, retry_count=0):
+    def reply_text(self, query, session_id, retry_count=0):
         try:
             response = openai.Completion.create(
                 model= conf().get("model") or "text-davinci-003",  # 对话模型的名称
@@ -60,116 +76,34 @@ class OpenAIBot(Bot):
                 stop=["\n\n\n"]
             )
             res_content = response.choices[0]['text'].strip().replace('<|endoftext|>', '')
+            total_tokens = response["usage"]["total_tokens"]
+            completion_tokens = response["usage"]["completion_tokens"]
             logger.info("[OPEN_AI] reply={}".format(res_content))
-            return res_content
-        except openai.error.RateLimitError as e:
-            # rate limit exception
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, user_id, retry_count+1)
-            else:
-                return "提问太快啦，请休息一下再问我吧"
+            return total_tokens, completion_tokens, res_content
         except Exception as e:
-            # unknown exception
-            logger.exception(e)
-            Session.clear_session(user_id)
-            return "请再问我一次吧"
-
-
-    def create_img(self, query, retry_count=0):
-        try:
-            logger.info("[OPEN_AI] image_query={}".format(query))
-            response = openai.Image.create(
-                prompt=query,    #图片描述
-                n=1,             #每次生成图片的数量
-                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
-            )
-            image_url = response['data'][0]['url']
-            logger.info("[OPEN_AI] image_url={}".format(image_url))
-            return image_url
-        except openai.error.RateLimitError as e:
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, retry_count+1)
+            need_retry = retry_count < 2
+            result = [0,0,"我现在有点累了，等会再来吧"]
+            if isinstance(e, openai.error.RateLimitError):
+                logger.warn("[OPEN_AI] RateLimitError: {}".format(e))
+                result[2] = "提问太快啦，请休息一下再问我吧"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.Timeout):
+                logger.warn("[OPEN_AI] Timeout: {}".format(e))
+                result[2] = "我没有收到你的消息"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.APIConnectionError):
+                logger.warn("[OPEN_AI] APIConnectionError: {}".format(e))
+                need_retry = False
+                result[2] = "我连接不到你的网络"
             else:
-                return "提问太快啦，请休息一下再问我吧"
-        except Exception as e:
-            logger.exception(e)
-            return None
+                logger.warn("[OPEN_AI] Exception: {}".format(e))
+                need_retry = False
+                self.sessions.clear_session(session_id)
 
-
-class Session(object):
-    @staticmethod
-    def build_session_query(query, user_id):
-        '''
-        build query with conversation history
-        e.g.  Q: xxx
-              A: xxx
-              Q: xxx
-        :param query: query content
-        :param user_id: from user id
-        :return: query content with conversaction
-        '''
-        prompt = conf().get("character_desc", "")
-        if prompt:
-            prompt += "<|endoftext|>\n\n\n"
-        session = user_session.get(user_id, None)
-        if session:
-            for conversation in session:
-                prompt += "Q: " + conversation["question"] + "\n\n\nA: " + conversation["answer"] + "<|endoftext|>\n"
-            prompt += "Q: " + query + "\nA: "
-            return prompt
-        else:
-            return prompt + "Q: " + query + "\nA: "
-
-    @staticmethod
-    def save_session(query, answer, user_id):
-        max_tokens = conf().get("conversation_max_tokens")
-        if not max_tokens:
-            # default 3000
-            max_tokens = 1000
-        conversation = dict()
-        conversation["question"] = query
-        conversation["answer"] = answer
-        session = user_session.get(user_id)
-        logger.debug(conversation)
-        logger.debug(session)
-        if session:
-            # append conversation
-            session.append(conversation)
-        else:
-            # create session
-            queue = list()
-            queue.append(conversation)
-            user_session[user_id] = queue
-
-        # discard exceed limit conversation
-        Session.discard_exceed_conversation(user_session[user_id], max_tokens)
-
-
-    @staticmethod
-    def discard_exceed_conversation(session, max_tokens):
-        count = 0
-        count_list = list()
-        for i in range(len(session)-1, -1, -1):
-            # count tokens of conversation list
-            history_conv = session[i]
-            count += len(history_conv["question"]) + len(history_conv["answer"])
-            count_list.append(count)
-
-        for c in count_list:
-            if c > max_tokens:
-                # pop first conversation
-                session.pop(0)
-
-    @staticmethod
-    def clear_session(user_id):
-        user_session[user_id] = []
-
-    @staticmethod
-    def clear_all_session():
-        user_session.clear()
+            if need_retry:
+                logger.warn("[OPEN_AI] 第{}次重试".format(retry_count+1))
+                return self.reply_text(query, session_id, retry_count+1)
+            else:
+                return result

bot/openai/open_ai_image.py

@@ -0,0 +1,38 @@
+import time
+import openai
+import openai.error
+from common.token_bucket import TokenBucket
+from common.log import logger
+from config import conf
+
+# OPENAI提供的画图接口
+class OpenAIImage(object):
+    def __init__(self):
+        openai.api_key = conf().get('open_ai_api_key')
+        if conf().get('rate_limit_dalle'):
+            self.tb4dalle = TokenBucket(conf().get('rate_limit_dalle', 50))
+            
+    def create_img(self, query, retry_count=0):
+        try:
+            if conf().get('rate_limit_dalle') and not self.tb4dalle.get_token():
+                return False, "请求太快了，请休息一下再问我吧"
+            logger.info("[OPEN_AI] image_query={}".format(query))
+            response = openai.Image.create(
+                prompt=query,    #图片描述
+                n=1,             #每次生成图片的数量
+                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
+            )
+            image_url = response['data'][0]['url']
+            logger.info("[OPEN_AI] image_url={}".format(image_url))
+            return True, image_url
+        except openai.error.RateLimitError as e:
+            logger.warn(e)
+            if retry_count < 1:
+                time.sleep(5)
+                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
+                return self.create_img(query, retry_count+1)
+            else:
+                return False, "提问太快啦，请休息一下再问我吧"
+        except Exception as e:
+            logger.exception(e)
+            return False, str(e)

bot/openai/open_ai_session.py

@@ -0,0 +1,67 @@
+from bot.session_manager import Session
+from common.log import logger
+class OpenAISession(Session):
+    def __init__(self, session_id, system_prompt=None, model= "text-davinci-003"):
+        super().__init__(session_id, system_prompt)
+        self.model = model
+        self.reset()
+
+    def __str__(self):
+        # 构造对话模型的输入
+        '''
+        e.g.  Q: xxx
+              A: xxx
+              Q: xxx
+        '''
+        prompt = ""
+        for item in self.messages:
+            if item['role'] == 'system':
+                prompt += item['content'] + "<|endoftext|>\n\n\n"
+            elif item['role'] == 'user':
+                prompt += "Q: " + item['content'] + "\n"
+            elif item['role'] == 'assistant':
+                prompt += "\n\nA: " + item['content'] + "<|endoftext|>\n"
+
+        if len(self.messages) > 0 and self.messages[-1]['role'] == 'user':
+            prompt += "A: "
+        return prompt
+
+    def discard_exceeding(self, max_tokens, cur_tokens= None):
+        precise = True
+        try:
+            cur_tokens = num_tokens_from_string(str(self), self.model)
+        except Exception as e:
+            precise = False
+            if cur_tokens is None:
+                raise e
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
+        while cur_tokens > max_tokens:
+            if len(self.messages) > 1:
+                self.messages.pop(0)
+            elif len(self.messages) == 1 and self.messages[0]["role"] == "assistant":
+                self.messages.pop(0)
+                if precise:
+                    cur_tokens = num_tokens_from_string(str(self), self.model)
+                else:
+                    cur_tokens = len(str(self))
+                break
+            elif len(self.messages) == 1 and self.messages[0]["role"] == "user":
+                logger.warn("user question exceed max_tokens. total_tokens={}".format(cur_tokens))
+                break
+            else:
+                logger.debug("max_tokens={}, total_tokens={}, len(conversation)={}".format(max_tokens, cur_tokens, len(self.messages)))
+                break
+            if precise:
+                cur_tokens = num_tokens_from_string(str(self), self.model)
+            else:
+                cur_tokens = len(str(self))
+        return cur_tokens
+    
+
+# refer to https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+def num_tokens_from_string(string: str, model: str) -> int:
+    """Returns the number of tokens in a text string."""
+    import tiktoken
+    encoding = tiktoken.encoding_for_model(model)
+    num_tokens = len(encoding.encode(string,disallowed_special=()))
+    return num_tokens

bot/session_manager.py

@@ -0,0 +1,85 @@
+from common.expired_dict import ExpiredDict
+from common.log import logger
+from config import conf
+
+class Session(object):
+    def __init__(self, session_id, system_prompt=None):
+        self.session_id = session_id
+        self.messages = []
+        if system_prompt is None:
+            self.system_prompt = conf().get("character_desc", "")
+        else:
+            self.system_prompt = system_prompt
+
+    # 重置会话
+    def reset(self):
+        system_item = {'role': 'system', 'content': self.system_prompt}
+        self.messages = [system_item]
+
+    def set_system_prompt(self, system_prompt):
+        self.system_prompt = system_prompt
+        self.reset()
+
+    def add_query(self, query):
+        user_item = {'role': 'user', 'content': query}
+        self.messages.append(user_item)
+
+    def add_reply(self, reply):
+        assistant_item = {'role': 'assistant', 'content': reply}
+        self.messages.append(assistant_item)
+    
+    def discard_exceeding(self, max_tokens=None, cur_tokens=None):
+        raise NotImplementedError
+
+
+
+class SessionManager(object):
+    def __init__(self, sessioncls, **session_args):
+        if conf().get('expires_in_seconds'):
+            sessions = ExpiredDict(conf().get('expires_in_seconds'))
+        else:
+            sessions = dict()
+        self.sessions = sessions
+        self.sessioncls = sessioncls
+        self.session_args = session_args
+
+    def build_session(self, session_id, system_prompt=None):
+        '''
+            如果session_id不在sessions中，创建一个新的session并添加到sessions中
+            如果system_prompt不会空，会更新session的system_prompt并重置session
+        '''
+        if session_id not in self.sessions:
+            self.sessions[session_id] = self.sessioncls(session_id, system_prompt, **self.session_args)
+        elif system_prompt is not None:  # 如果有新的system_prompt，更新并重置session
+            self.sessions[session_id].set_system_prompt(system_prompt)
+        session = self.sessions[session_id]
+        return session
+    
+    def session_query(self, query, session_id):
+        session = self.build_session(session_id)
+        session.add_query(query)
+        try:
+            max_tokens = conf().get("conversation_max_tokens", 1000)
+            total_tokens = session.discard_exceeding(max_tokens, None)
+            logger.debug("prompt tokens used={}".format(total_tokens))
+        except Exception as e:
+            logger.debug("Exception when counting tokens precisely for prompt: {}".format(str(e)))
+        return session
+
+    def session_reply(self, reply, session_id, total_tokens = None):
+        session = self.build_session(session_id)
+        session.add_reply(reply)
+        try:
+            max_tokens = conf().get("conversation_max_tokens", 1000)
+            tokens_cnt = session.discard_exceeding(max_tokens, total_tokens)
+            logger.debug("raw total_tokens={}, savesession tokens={}".format(total_tokens, tokens_cnt))
+        except Exception as e:
+            logger.debug("Exception when counting tokens precisely for session: {}".format(str(e)))
+        return session
+
+    def clear_session(self, session_id):
+        if session_id in self.sessions:
+            del(self.sessions[session_id])
+
+    def clear_all_session(self):
+        self.sessions.clear()

bridge/bridge.py

@@ -14,7 +14,7 @@ class Bridge(object):
         self.btype={
             "chat": const.CHATGPT,
             "voice_to_text": conf().get("voice_to_text", "openai"),
-            "text_to_voice": conf().get("text_to_voice", "baidu")
+            "text_to_voice": conf().get("text_to_voice", "google")
         }
         model_type = conf().get("model")
         if model_type in ["text-davinci-003"]:

bridge/context.py

@@ -14,6 +14,15 @@ class Context:
         self.type = type
         self.content = content
         self.kwargs = kwargs
+
+    def __contains__(self, key):
+        if key == 'type':
+            return self.type is not None
+        elif key == 'content':
+            return self.content is not None
+        else:
+            return key in self.kwargs
+        
     def __getitem__(self, key):
         if key == 'type':
             return self.type
@@ -21,6 +30,12 @@ class Context:
             return self.content
         else:
             return self.kwargs[key]
+    
+    def get(self, key, default=None):
+        try:
+            return self[key]
+        except KeyError:
+            return default
 
     def __setitem__(self, key, value):
         if key == 'type':

channel/channel.py

@@ -4,9 +4,10 @@ Message sending channel abstract class
 
 from bridge.bridge import Bridge
 from bridge.context import Context
-from bridge.reply import Reply
+from bridge.reply import *
 
 class Channel(object):
+    NOT_SUPPORT_REPLYTYPE = [ReplyType.VOICE, ReplyType.IMAGE]
     def startup(self):
         """
         init channel
@@ -20,7 +21,8 @@ class Channel(object):
         """
         raise NotImplementedError
 
-    def send(self, msg, receiver):
+    # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
+    def send(self, reply: Reply, context: Context):
         """
         send message to user
         :param msg: message content

channel/channel_factory.py

@@ -17,4 +17,7 @@ def create_channel(channel_type):
     elif channel_type == 'terminal':
         from channel.terminal.terminal_channel import TerminalChannel
         return TerminalChannel()
+    elif channel_type == 'wechatmp':
+        from channel.wechatmp.wechatmp_channel import WechatMPChannel
+        return WechatMPChannel()
     raise RuntimeError

channel/chat_channel.py

@@ -0,0 +1,316 @@
+
+
+from asyncio import CancelledError
+from concurrent.futures import Future, ThreadPoolExecutor
+import os
+import re
+import threading
+import time
+from common.dequeue import Dequeue
+from channel.channel import Channel
+from bridge.reply import *
+from bridge.context import *
+from config import conf
+from common.log import logger
+from plugins import *
+try:
+    from voice.audio_convert import any_to_wav
+except Exception as e:
+    pass
+
+# 抽象类, 它包含了与消息通道无关的通用处理逻辑
+class ChatChannel(Channel):
+    name = None # 登录的用户名
+    user_id = None # 登录的用户id
+    futures = {} # 记录每个session_id提交到线程池的future对象, 用于重置会话时把没执行的future取消掉，正在执行的不会被取消
+    sessions = {} # 用于控制并发，每个session_id同时只能有一个context在处理
+    lock = threading.Lock() # 用于控制对sessions的访问
+    handler_pool = ThreadPoolExecutor(max_workers=8)  # 处理消息的线程池
+
+    def __init__(self):
+        _thread = threading.Thread(target=self.consume)
+        _thread.setDaemon(True)
+        _thread.start()
+        
+
+    # 根据消息构造context，消息内容相关的触发项写在这里
+    def _compose_context(self, ctype: ContextType, content, **kwargs):
+        context = Context(ctype, content)
+        context.kwargs = kwargs
+        # context首次传入时，origin_ctype是None, 
+        # 引入的起因是：当输入语音时，会嵌套生成两个context，第一步语音转文本，第二步通过文本生成文字回复。
+        # origin_ctype用于第二步文本回复时，判断是否需要匹配前缀，如果是私聊的语音，就不需要匹配前缀
+        if 'origin_ctype' not in context:  
+            context['origin_ctype'] = ctype
+        # context首次传入时，receiver是None，根据类型设置receiver
+        first_in = 'receiver' not in context
+        # 群名匹配过程，设置session_id和receiver
+        if first_in: # context首次传入时，receiver是None，根据类型设置receiver
+            config = conf()
+            cmsg = context['msg']
+            if cmsg.from_user_id == self.user_id and not config.get('trigger_by_self', True):
+                logger.debug("[WX]self message skipped")
+                return None
+            if context.get("isgroup", False):
+                group_name = cmsg.other_user_nickname
+                group_id = cmsg.other_user_id
+
+                group_name_white_list = config.get('group_name_white_list', [])
+                group_name_keyword_white_list = config.get('group_name_keyword_white_list', [])
+                if any([group_name in group_name_white_list, 'ALL_GROUP' in group_name_white_list, check_contain(group_name, group_name_keyword_white_list)]):
+                    group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
+                    session_id = cmsg.actual_user_id
+                    if any([group_name in group_chat_in_one_session, 'ALL_GROUP' in group_chat_in_one_session]):
+                        session_id = group_id
+                else:
+                    return None
+                context['session_id'] = session_id
+                context['receiver'] = group_id
+            else:
+                context['session_id'] = cmsg.other_user_id
+                context['receiver'] = cmsg.other_user_id
+
+        # 消息内容匹配过程，并处理content
+        if ctype == ContextType.TEXT:
+            if first_in and "」\n- - - - - - -" in content: # 初次匹配 过滤引用消息
+                logger.debug("[WX]reference query skipped")
+                return None
+            
+            if context.get("isgroup", False): # 群聊
+                # 校验关键字
+                match_prefix = check_prefix(content, conf().get('group_chat_prefix'))
+                match_contain = check_contain(content, conf().get('group_chat_keyword'))
+                flag = False
+                if match_prefix is not None or match_contain is not None:
+                    flag = True
+                    if match_prefix:
+                        content = content.replace(match_prefix, '', 1).strip()
+                if context['msg'].is_at:
+                    logger.info("[WX]receive group at")
+                    if not conf().get("group_at_off", False):
+                        flag = True
+                    pattern = f'@{self.name}(\u2005|\u0020)'
+                    content = re.sub(pattern, r'', content)
+                
+                if not flag:
+                    if context["origin_ctype"] == ContextType.VOICE:
+                        logger.info("[WX]receive group voice, but checkprefix didn't match")
+                    return None
+            else: # 单聊
+                match_prefix = check_prefix(content, conf().get('single_chat_prefix'))  
+                if match_prefix is not None: # 判断如果匹配到自定义前缀，则返回过滤掉前缀+空格后的内容
+                    content = content.replace(match_prefix, '', 1).strip()
+                elif context["origin_ctype"] == ContextType.VOICE: # 如果源消息是私聊的语音消息，允许不匹配前缀，放宽条件
+                    pass
+                else:
+                    return None     
+                                                  
+            img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
+            if img_match_prefix:
+                content = content.replace(img_match_prefix, '', 1).strip()
+                context.type = ContextType.IMAGE_CREATE
+            else:
+                context.type = ContextType.TEXT
+            context.content = content
+            if 'desire_rtype' not in context and conf().get('always_reply_voice') and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
+                context['desire_rtype'] = ReplyType.VOICE
+        elif context.type == ContextType.VOICE: 
+            if 'desire_rtype' not in context and conf().get('voice_reply_voice') and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
+                context['desire_rtype'] = ReplyType.VOICE
+
+        return context
+
+    def _handle(self, context: Context):
+        if context is None or not context.content:
+            return
+        logger.debug('[WX] ready to handle context: {}'.format(context))
+        # reply的构建步骤
+        reply = self._generate_reply(context)
+
+        logger.debug('[WX] ready to decorate reply: {}'.format(reply))
+        # reply的包装步骤
+        reply = self._decorate_reply(context, reply)
+
+        # reply的发送步骤
+        self._send_reply(context, reply)
+
+    def _generate_reply(self, context: Context, reply: Reply = Reply()) -> Reply:
+        e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {
+            'channel': self, 'context': context, 'reply': reply}))
+        reply = e_context['reply']
+        if not e_context.is_pass():
+            logger.debug('[WX] ready to handle context: type={}, content={}'.format(context.type, context.content))
+            if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:  # 文字和图片消息
+                reply = super().build_reply_content(context.content, context)
+            elif context.type == ContextType.VOICE:  # 语音消息
+                cmsg = context['msg']
+                cmsg.prepare()
+                file_path = context.content
+                wav_path = os.path.splitext(file_path)[0] + '.wav'
+                try:
+                    any_to_wav(file_path, wav_path) 
+                except Exception as e:  # 转换失败，直接使用mp3，对于某些api，mp3也可以识别
+                    logger.warning("[WX]any to wav error, use raw path. " + str(e))
+                    wav_path = file_path
+                # 语音识别
+                reply = super().build_voice_to_text(wav_path)
+                # 删除临时文件
+                try:
+                    os.remove(file_path)
+                    if wav_path != file_path:
+                        os.remove(wav_path)
+                except Exception as e:
+                    pass
+                    # logger.warning("[WX]delete temp file error: " + str(e))
+
+                if reply.type == ReplyType.TEXT:
+                    new_context = self._compose_context(
+                        ContextType.TEXT, reply.content, **context.kwargs)
+                    if new_context:
+                        reply = self._generate_reply(new_context)
+                    else:
+                        return
+            else:
+                logger.error('[WX] unknown context type: {}'.format(context.type))
+                return
+        return reply
+
+    def _decorate_reply(self, context: Context, reply: Reply) -> Reply:
+        if reply and reply.type:
+            e_context = PluginManager().emit_event(EventContext(Event.ON_DECORATE_REPLY, {
+                'channel': self, 'context': context, 'reply': reply}))
+            reply = e_context['reply']
+            desire_rtype = context.get('desire_rtype')
+            if not e_context.is_pass() and reply and reply.type:
+                
+                if reply.type in self.NOT_SUPPORT_REPLYTYPE:
+                    logger.error("[WX]reply type not support: " + str(reply.type))
+                    reply.type = ReplyType.ERROR
+                    reply.content = "不支持发送的消息类型: " + str(reply.type)
+
+                if reply.type == ReplyType.TEXT:
+                    reply_text = reply.content
+                    if desire_rtype == ReplyType.VOICE and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
+                        reply = super().build_text_to_voice(reply.content)
+                        return self._decorate_reply(context, reply)
+                    if context.get("isgroup", False):
+                        reply_text = '@' +  context['msg'].actual_user_nickname + ' ' + reply_text.strip()
+                        reply_text = conf().get("group_chat_reply_prefix", "") + reply_text
+                    else:
+                        reply_text = conf().get("single_chat_reply_prefix", "") + reply_text
+                    reply.content = reply_text
+                elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
+                    reply.content = "["+str(reply.type)+"]\n" + reply.content
+                elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
+                    pass
+                else:
+                    logger.error('[WX] unknown reply type: {}'.format(reply.type))
+                    return
+            if desire_rtype and desire_rtype != reply.type and reply.type not in [ReplyType.ERROR, ReplyType.INFO]:
+                logger.warning('[WX] desire_rtype: {}, but reply type: {}'.format(context.get('desire_rtype'), reply.type))
+            return reply
+
+    def _send_reply(self, context: Context, reply: Reply):
+        if reply and reply.type:
+            e_context = PluginManager().emit_event(EventContext(Event.ON_SEND_REPLY, {
+                'channel': self, 'context': context, 'reply': reply}))
+            reply = e_context['reply']
+            if not e_context.is_pass() and reply and reply.type:
+                logger.debug('[WX] ready to send reply: {}, context: {}'.format(reply, context))
+                self._send(reply, context)
+
+    def _send(self, reply: Reply, context: Context, retry_cnt = 0):
+        try:
+            self.send(reply, context)
+        except Exception as e:
+            logger.error('[WX] sendMsg error: {}'.format(str(e)))
+            if isinstance(e, NotImplementedError):
+                return
+            logger.exception(e)
+            if retry_cnt < 2:
+                time.sleep(3+3*retry_cnt)
+                self._send(reply, context, retry_cnt+1)
+
+    def thread_pool_callback(self, session_id):
+        def func(worker:Future):
+            try:
+                worker_exception = worker.exception()
+                if worker_exception:
+                    logger.exception("Worker return exception: {}".format(worker_exception))
+            except CancelledError as e:
+                logger.info("Worker cancelled, session_id = {}".format(session_id))
+            except Exception as e:
+                logger.exception("Worker raise exception: {}".format(e))
+            with self.lock:
+                self.sessions[session_id][1].release()
+        return func
+
+    def produce(self, context: Context):
+        session_id = context['session_id']
+        with self.lock:
+            if session_id not in self.sessions:
+                self.sessions[session_id] = [Dequeue(), threading.BoundedSemaphore(conf().get("concurrency_in_session", 1))]
+            if context.type == ContextType.TEXT and context.content.startswith("#"): 
+                self.sessions[session_id][0].putleft(context) # 优先处理管理命令
+            else:
+                self.sessions[session_id][0].put(context)
+
+    # 消费者函数，单独线程，用于从消息队列中取出消息并处理
+    def consume(self):
+        while True:
+            with self.lock:
+                session_ids = list(self.sessions.keys())
+                for session_id in session_ids:
+                    context_queue, semaphore = self.sessions[session_id]
+                    if semaphore.acquire(blocking = False): # 等线程处理完毕才能删除
+                        if not context_queue.empty():
+                            context = context_queue.get()
+                            logger.debug("[WX] consume context: {}".format(context))
+                            future:Future = self.handler_pool.submit(self._handle, context)
+                            future.add_done_callback(self.thread_pool_callback(session_id))
+                            if session_id not in self.futures:
+                                self.futures[session_id] = []
+                            self.futures[session_id].append(future)
+                        elif semaphore._initial_value == semaphore._value+1: # 除了当前，没有任务再申请到信号量，说明所有任务都处理完毕
+                            self.futures[session_id] = [t for t in self.futures[session_id] if not t.done()]
+                            assert len(self.futures[session_id]) == 0, "thread pool error"
+                            del self.sessions[session_id]
+                        else:
+                            semaphore.release()
+            time.sleep(0.1)
+
+    # 取消session_id对应的所有任务，只能取消排队的消息和已提交线程池但未执行的任务
+    def cancel_session(self, session_id): 
+        with self.lock:
+            if session_id in self.sessions:
+                for future in self.futures[session_id]:
+                    future.cancel()
+                cnt = self.sessions[session_id][0].qsize()
+                if cnt>0:
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
+                self.sessions[session_id][0] = Dequeue()
+    
+    def cancel_all_session(self):
+        with self.lock:
+            for session_id in self.sessions:
+                for future in self.futures[session_id]:
+                    future.cancel()
+                cnt = self.sessions[session_id][0].qsize()
+                if cnt>0:
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
+                self.sessions[session_id][0] = Dequeue()
+    
+
+def check_prefix(content, prefix_list):
+    for prefix in prefix_list:
+        if content.startswith(prefix):
+            return prefix
+    return None
+
+def check_contain(content, keyword_list):
+    if not keyword_list:
+        return None
+    for ky in keyword_list:
+        if content.find(ky) != -1:
+            return True
+    return None

channel/chat_message.py

@@ -0,0 +1,83 @@
+
+""" 
+本类表示聊天消息，用于对itchat和wechaty的消息进行统一的封装
+
+ChatMessage
+msg_id: 消息id
+create_time: 消息创建时间
+
+ctype: 消息类型 : ContextType
+content: 消息内容, 如果是声音/图片，这里是文件路径
+
+from_user_id: 发送者id
+from_user_nickname: 发送者昵称
+to_user_id: 接收者id
+to_user_nickname: 接收者昵称
+
+other_user_id: 对方的id，如果你是发送者，那这个就是接收者id，如果你是接收者，那这个就是发送者id，如果是群消息，那这一直是群id
+other_user_nickname: 同上
+
+is_group: 是否是群消息
+is_at: 是否被at
+
+- (群消息时，一般会存在实际发送者，是群内某个成员的id和昵称，下列项仅在群消息时存在)
+actual_user_id: 实际发送者id
+actual_user_nickname：实际发送者昵称
+
+
+
+
+_prepare_fn: 准备函数，用于准备消息的内容，比如下载图片等,
+_prepared: 是否已经调用过准备函数
+_rawmsg: 原始消息对象
+
+"""
+class ChatMessage(object):
+    msg_id = None
+    create_time = None
+    
+    ctype = None
+    content = None
+    
+    from_user_id = None
+    from_user_nickname = None
+    to_user_id = None
+    to_user_nickname = None
+    other_user_id = None
+    other_user_nickname = None
+    
+    is_group = False
+    is_at = False
+    actual_user_id = None
+    actual_user_nickname = None
+
+    _prepare_fn = None
+    _prepared = False
+    _rawmsg = None
+
+
+    def __init__(self,_rawmsg):
+        self._rawmsg = _rawmsg
+
+    def prepare(self):
+        if self._prepare_fn and not self._prepared:
+            self._prepared = True
+            self._prepare_fn()
+
+    def __str__(self):
+        return 'ChatMessage: id={}, create_time={}, ctype={}, content={}, from_user_id={}, from_user_nickname={}, to_user_id={}, to_user_nickname={}, other_user_id={}, other_user_nickname={}, is_group={}, is_at={}, actual_user_id={}, actual_user_nickname={}'.format(
+            self.msg_id,
+            self.create_time,
+            self.ctype,
+            self.content,
+            self.from_user_id,
+            self.from_user_nickname,
+            self.to_user_id,
+            self.to_user_nickname,
+            self.other_user_id,
+            self.other_user_nickname,
+            self.is_group,
+            self.is_at,
+            self.actual_user_id,
+            self.actual_user_nickname,
+        )

channel/terminal/terminal_channel.py

@@ -1,31 +1,78 @@
 from bridge.context import *
-from channel.channel import Channel
+from bridge.reply import Reply, ReplyType
+from channel.chat_channel import ChatChannel, check_prefix
+from channel.chat_message import ChatMessage
 import sys
 
-class TerminalChannel(Channel):
+from config import conf
+from common.log import logger
+
+class TerminalMessage(ChatMessage):
+    def __init__(self, msg_id, content, ctype = ContextType.TEXT,  from_user_id = "User", to_user_id = "Chatgpt", other_user_id = "Chatgpt"):
+        self.msg_id = msg_id
+        self.ctype = ctype
+        self.content = content
+        self.from_user_id = from_user_id
+        self.to_user_id = to_user_id
+        self.other_user_id = other_user_id
+
+class TerminalChannel(ChatChannel):
+    NOT_SUPPORT_REPLYTYPE = [ReplyType.VOICE]
+
+    def send(self, reply: Reply, context: Context):
+        print("\nBot:")
+        if reply.type == ReplyType.IMAGE:
+            from PIL import Image
+            image_storage = reply.content
+            image_storage.seek(0)
+            img = Image.open(image_storage)
+            print("<IMAGE>")
+            img.show()
+        elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
+            from PIL import Image
+            import requests,io
+            img_url = reply.content
+            pic_res = requests.get(img_url, stream=True)
+            image_storage = io.BytesIO()
+            for block in pic_res.iter_content(1024):
+                image_storage.write(block)
+            image_storage.seek(0)
+            img = Image.open(image_storage)
+            print(img_url)
+            img.show()
+        else:
+            print(reply.content)
+        print("\nUser:", end="")
+        sys.stdout.flush()
+        return
+
     def startup(self):
         context = Context()
-        print("\nPlease input your question")
+        logger.setLevel("WARN")
+        print("\nPlease input your question:\nUser:", end="")
+        sys.stdout.flush()
+        msg_id = 0
         while True:
             try:
-                prompt = self.get_input("User:\n")
+                prompt = self.get_input()
             except KeyboardInterrupt:
                 print("\nExiting...")
                 sys.exit()
+            msg_id += 1
+            trigger_prefixs = conf().get("single_chat_prefix",[""])
+            if check_prefix(prompt, trigger_prefixs) is None:
+                prompt = trigger_prefixs[0] + prompt # 给没触发的消息加上触发前缀
+                
+            context = self._compose_context(ContextType.TEXT, prompt, msg = TerminalMessage(msg_id, prompt))
+            if context:
+                self.produce(context)
+            else:
+                raise Exception("context is None")
 
-            context.type = ContextType.TEXT
-            context['session_id'] = "User"
-            context.content = prompt
-            print("Bot:")
-            sys.stdout.flush()
-            res = super().build_reply_content(prompt, context).content
-            print(res)
-
-
-    def get_input(self, prompt):
+    def get_input(self):
         """
         Multi-line input function
         """
-        print(prompt, end="")
+        sys.stdout.flush()
         line = input()
         return line

channel/wechat/wechat_channel.py

@@ -5,70 +5,120 @@ wechat channel
 """
 
 import os
-from lib import itchat
-import json
-from lib.itchat.content import *
-from bridge.reply import *
-from bridge.context import *
-from channel.channel import Channel
-from concurrent.futures import ThreadPoolExecutor
-from common.log import logger
-from common.tmp_dir import TmpDir
-from config import conf
-from common.time_check import time_checker
-from plugins import *
+import threading
 import requests
 import io
 import time
-
-
-thread_pool = ThreadPoolExecutor(max_workers=8)
-def thread_pool_callback(worker):
-    worker_exception = worker.exception()
-    if worker_exception:
-        logger.exception("Worker return exception: {}".format(worker_exception))
+import json
+from channel.chat_channel import ChatChannel
+from channel.wechat.wechat_message import *
+from common.singleton import singleton
+from common.log import logger
+from lib import itchat
+from lib.itchat.content import *
+from bridge.reply import *
+from bridge.context import *
+from config import conf
+from common.time_check import time_checker
+from common.expired_dict import ExpiredDict
+from plugins import *
 
 @itchat.msg_register(TEXT)
 def handler_single_msg(msg):
-    WechatChannel().handle_text(msg)
+    WechatChannel().handle_text(WeChatMessage(msg))
     return None
 
-
 @itchat.msg_register(TEXT, isGroupChat=True)
 def handler_group_msg(msg):
-    WechatChannel().handle_group(msg)
+    WechatChannel().handle_group(WeChatMessage(msg,True))
     return None
 
-
 @itchat.msg_register(VOICE)
 def handler_single_voice(msg):
-    WechatChannel().handle_voice(msg)
+    WechatChannel().handle_voice(WeChatMessage(msg))
+    return None
+    
+@itchat.msg_register(VOICE, isGroupChat=True)
+def handler_group_voice(msg):
+    WechatChannel().handle_group_voice(WeChatMessage(msg,True))
     return None
 
+def _check(func):
+    def wrapper(self, cmsg: ChatMessage):
+        msgId = cmsg.msg_id
+        if msgId in self.receivedMsgs:
+            logger.info("Wechat message {} already received, ignore".format(msgId))
+            return
+        self.receivedMsgs[msgId] = cmsg
+        create_time = cmsg.create_time            # 消息时间戳
+        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:  # 跳过1分钟前的历史消息
+            logger.debug("[WX]history message {} skipped".format(msgId))
+            return
+        return func(self, cmsg)
+    return wrapper
 
-class WechatChannel(Channel):
+#可用的二维码生成接口
+#https://api.qrserver.com/v1/create-qr-code/?size=400×400&data=https://www.abc.com
+#https://api.isoyu.com/qr/?m=1&e=L&p=20&url=https://www.abc.com
+def qrCallback(uuid,status,qrcode):
+    # logger.debug("qrCallback: {} {}".format(uuid,status))
+    if status == '0':
+        try:
+            from PIL import Image
+            img = Image.open(io.BytesIO(qrcode))
+            _thread = threading.Thread(target=img.show, args=("QRCode",))
+            _thread.setDaemon(True)
+            _thread.start()
+        except Exception as e:
+            pass
+
+        import qrcode
+        url = f"https://login.weixin.qq.com/l/{uuid}"
+        
+        qr_api1="https://api.isoyu.com/qr/?m=1&e=L&p=20&url={}".format(url)
+        qr_api2="https://api.qrserver.com/v1/create-qr-code/?size=400×400&data={}".format(url)
+        qr_api3="https://api.pwmqr.com/qrcode/create/?url={}".format(url)
+        qr_api4="https://my.tv.sohu.com/user/a/wvideo/getQRCode.do?text={}".format(url)
+        print("You can also scan QRCode in any website below:")
+        print(qr_api3)
+        print(qr_api4)
+        print(qr_api2)
+        print(qr_api1)
+        
+        qr = qrcode.QRCode(border=1)
+        qr.add_data(url)
+        qr.make(fit=True)
+        qr.print_ascii(invert=True)
+
+@singleton
+class WechatChannel(ChatChannel):
+    NOT_SUPPORT_REPLYTYPE = []
     def __init__(self):
-        pass
+        super().__init__()
+        self.receivedMsgs = ExpiredDict(60*60*24) 
 
     def startup(self):
 
-        itchat.instance.receivingRetryCount = 600 # 修改断线超时时间
+        itchat.instance.receivingRetryCount = 600  # 修改断线超时时间
         # login by scan QRCode
         hotReload = conf().get('hot_reload', False)
         try:
-            itchat.auto_login(enableCmdQR=2, hotReload=hotReload)
+            itchat.auto_login(enableCmdQR=2, hotReload=hotReload, qrCallback=qrCallback)
         except Exception as e:
             if hotReload:
                 logger.error("Hot reload failed, try to login without hot reload")
                 itchat.logout()
                 os.remove("itchat.pkl")
-                itchat.auto_login(enableCmdQR=2, hotReload=hotReload)
+                itchat.auto_login(enableCmdQR=2, hotReload=hotReload, qrCallback=qrCallback)
             else:
                 raise e
+        self.user_id = itchat.instance.storageClass.userName
+        self.name = itchat.instance.storageClass.nickName
+        logger.info("Wechat login success, user_id: {}, nickname: {}".format(self.user_id, self.name))
         # start message listener
         itchat.run()
 
-    # handle_* 系列函数处理收到的消息后构造Context，然后传入handle函数中处理Context和发送回复
+    # handle_* 系列函数处理收到的消息后构造Context，然后传入_handle函数中处理Context和发送回复
     # Context包含了消息的所有信息，包括以下属性
     #   type 消息类型, 包括TEXT、VOICE、IMAGE_CREATE
     #   content 消息内容，如果是TEXT类型，content就是文本内容，如果是VOICE类型，content就是语音文件名，如果是IMAGE_CREATE类型，content就是图片生成命令
@@ -76,101 +126,49 @@ class WechatChannel(Channel):
     #        session_id: 会话id
     #        isgroup: 是否是群聊
     #        receiver: 需要回复的对象
-    #        msg: itchat的原始消息对象
+    #        msg: ChatMessage消息对象
+    #        origin_ctype: 原始消息类型，语音转文字后，私聊时如果匹配前缀失败，会根据初始消息是否是语音来放宽触发规则
+    #        desire_rtype: 希望回复类型，默认是文本回复，设置为ReplyType.VOICE是语音回复
 
-    def handle_voice(self, msg):
+    @time_checker
+    @_check
+    def handle_voice(self, cmsg : ChatMessage):
         if conf().get('speech_recognition') != True:
             return
-        logger.debug("[WX]receive voice msg: " + msg['FileName'])
-        from_user_id = msg['FromUserName']
-        other_user_id = msg['User']['UserName']
-        if from_user_id == other_user_id:
-            context = Context(ContextType.VOICE,msg['FileName'])
-            context.kwargs = {'isgroup': False, 'msg': msg, 'receiver': other_user_id, 'session_id': other_user_id}
-            thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
-
+        logger.debug("[WX]receive voice msg: {}".format(cmsg.content))
+        context = self._compose_context(ContextType.VOICE, cmsg.content, isgroup=False, msg=cmsg)
+        if context:
+            self.produce(context)
 
     @time_checker
-    def handle_text(self, msg):
-        logger.debug("[WX]receive text msg: " + json.dumps(msg, ensure_ascii=False))
-        content = msg['Text']
-        from_user_id = msg['FromUserName']
-        to_user_id = msg['ToUserName']              # 接收人id
-        other_user_id = msg['User']['UserName']     # 对手方id
-        create_time = msg['CreateTime']             # 消息时间
-        match_prefix = check_prefix(content, conf().get('single_chat_prefix'))
-        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:    #跳过1分钟前的历史消息
-            logger.debug("[WX]history message skipped")
-            return
-        if "」\n- - - - - - - - - - - - - - -" in content:
-            logger.debug("[WX]reference query skipped")
-            return
-        if match_prefix:
-            content = content.replace(match_prefix, '', 1).strip()
-        elif match_prefix is None:
-            return
-        context = Context()
-        context.kwargs = {'isgroup': False, 'msg': msg, 'receiver': other_user_id, 'session_id': other_user_id}
-
-        img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
-        if img_match_prefix:
-            content = content.replace(img_match_prefix, '', 1).strip()
-            context.type = ContextType.IMAGE_CREATE
-        else:
-            context.type = ContextType.TEXT
-
-        context.content = content
-        thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
+    @_check
+    def handle_text(self, cmsg : ChatMessage):
+        logger.debug("[WX]receive text msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
+        context = self._compose_context(ContextType.TEXT, cmsg.content, isgroup=False, msg=cmsg)
+        if context:
+            self.produce(context)
 
     @time_checker
-    def handle_group(self, msg):
-        logger.debug("[WX]receive group msg: " + json.dumps(msg, ensure_ascii=False))
-        group_name = msg['User'].get('NickName', None)
-        group_id = msg['User'].get('UserName', None)
-        create_time = msg['CreateTime']             # 消息时间
-        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:    #跳过1分钟前的历史消息
-            logger.debug("[WX]history group message skipped")
+    @_check
+    def handle_group(self, cmsg : ChatMessage):
+        logger.debug("[WX]receive group msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
+        context = self._compose_context(ContextType.TEXT, cmsg.content, isgroup=True, msg=cmsg)
+        if context:
+            self.produce(context)
+    
+    @time_checker
+    @_check
+    def handle_group_voice(self, cmsg : ChatMessage):
+        if conf().get('group_speech_recognition', False) != True:
             return
-        if not group_name:
-            return ""
-        origin_content = msg['Content']
-        content = msg['Content']
-        content_list = content.split(' ', 1)
-        context_special_list = content.split('\u2005', 1)
-        if len(context_special_list) == 2:
-            content = context_special_list[1]
-        elif len(content_list) == 2:
-            content = content_list[1]
-        if "」\n- - - - - - - - - - - - - - -" in content:
-            logger.debug("[WX]reference query skipped")
-            return ""
-        config = conf()
-        match_prefix = (msg['IsAt'] and not config.get("group_at_off", False)) or check_prefix(origin_content, config.get('group_chat_prefix')) \
-                       or check_contain(origin_content, config.get('group_chat_keyword'))
-        if ('ALL_GROUP' in config.get('group_name_white_list') or group_name in config.get('group_name_white_list') or check_contain(group_name, config.get('group_name_keyword_white_list'))) and match_prefix:
-            context = Context()
-            context.kwargs = { 'isgroup': True, 'msg': msg, 'receiver': group_id}
-            
-            img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
-            if img_match_prefix:
-                content = content.replace(img_match_prefix, '', 1).strip()
-                context.type = ContextType.IMAGE_CREATE
-            else:
-                context.type = ContextType.TEXT
-            context.content = content
-
-            group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
-            if ('ALL_GROUP' in group_chat_in_one_session or
-                    group_name in group_chat_in_one_session or
-                    check_contain(group_name, group_chat_in_one_session)):
-                context['session_id'] = group_id
-            else:
-                context['session_id'] = msg['ActualUserName']
-
-            thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
-
+        logger.debug("[WX]receive voice for group msg: {}".format(cmsg.content))
+        context = self._compose_context(ContextType.VOICE, cmsg.content, isgroup=True, msg=cmsg)
+        if context:
+            self.produce(context)
+    
     # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
-    def send(self, reply : Reply, receiver):
+    def send(self, reply: Reply, context: Context):
+        receiver = context["receiver"]
         if reply.type == ReplyType.TEXT:
             itchat.send(reply.content, toUserName=receiver)
             logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
@@ -194,79 +192,3 @@ class WechatChannel(Channel):
             image_storage.seek(0)
             itchat.send_image(image_storage, toUserName=receiver)
             logger.info('[WX] sendImage, receiver={}'.format(receiver))
-
-    # 处理消息 TODO: 如果wechaty解耦，此处逻辑可以放置到父类
-    def handle(self, context):
-        reply = Reply()
-
-        logger.debug('[WX] ready to handle context: {}'.format(context))
-        
-        # reply的构建步骤
-        e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {'channel' : self, 'context': context, 'reply': reply}))
-        reply = e_context['reply']
-        if not e_context.is_pass():
-            logger.debug('[WX] ready to handle context: type={}, content={}'.format(context.type, context.content))
-            if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:
-                reply = super().build_reply_content(context.content, context)
-            elif context.type == ContextType.VOICE:
-                msg = context['msg']
-                file_name = TmpDir().path() + context.content
-                msg.download(file_name)
-                reply = super().build_voice_to_text(file_name)
-                if reply.type != ReplyType.ERROR and reply.type != ReplyType.INFO:
-                    context.content = reply.content # 语音转文字后，将文字内容作为新的context
-                    context.type = ContextType.TEXT
-                    reply = super().build_reply_content(context.content, context)
-                    if reply.type == ReplyType.TEXT:
-                        if conf().get('voice_reply_voice'):
-                            reply = super().build_text_to_voice(reply.content)
-            else:
-                logger.error('[WX] unknown context type: {}'.format(context.type))
-                return
-
-        logger.debug('[WX] ready to decorate reply: {}'.format(reply))
-        
-        # reply的包装步骤
-        if reply and reply.type:
-            e_context = PluginManager().emit_event(EventContext(Event.ON_DECORATE_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
-            reply=e_context['reply']
-            if not e_context.is_pass() and reply and reply.type:
-                if reply.type == ReplyType.TEXT:
-                    reply_text = reply.content
-                    if context['isgroup']:
-                        reply_text = '@' +  context['msg']['ActualNickName'] + ' ' + reply_text.strip()
-                        reply_text = conf().get("group_chat_reply_prefix", "")+reply_text
-                    else:
-                        reply_text = conf().get("single_chat_reply_prefix", "")+reply_text
-                    reply.content = reply_text
-                elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
-                    reply.content = str(reply.type)+":\n" + reply.content
-                elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
-                    pass
-                else:
-                    logger.error('[WX] unknown reply type: {}'.format(reply.type))
-                    return
-
-        # reply的发送步骤   
-        if reply and reply.type:
-            e_context = PluginManager().emit_event(EventContext(Event.ON_SEND_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
-            reply=e_context['reply']
-            if not e_context.is_pass() and reply and reply.type:
-                logger.debug('[WX] ready to send reply: {} to {}'.format(reply, context['receiver']))
-                self.send(reply, context['receiver'])
-
-
-def check_prefix(content, prefix_list):
-    for prefix in prefix_list:
-        if content.startswith(prefix):
-            return prefix
-    return None
-
-
-def check_contain(content, keyword_list):
-    if not keyword_list:
-        return None
-    for ky in keyword_list:
-        if content.find(ky) != -1:
-            return True
-    return None

channel/wechat/wechat_message.py

@@ -0,0 +1,57 @@
+
+
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.tmp_dir import TmpDir
+from common.log import logger
+from lib.itchat.content import *
+from lib import itchat
+
+class WeChatMessage(ChatMessage):
+
+    def __init__(self, itchat_msg, is_group=False):
+        super().__init__( itchat_msg)
+        self.msg_id = itchat_msg['MsgId']
+        self.create_time = itchat_msg['CreateTime']
+        self.is_group = is_group
+        
+        if itchat_msg['Type'] == TEXT:
+            self.ctype = ContextType.TEXT
+            self.content = itchat_msg['Text']
+        elif itchat_msg['Type'] == VOICE:
+            self.ctype = ContextType.VOICE
+            self.content = TmpDir().path() + itchat_msg['FileName']  # content直接存临时目录路径
+            self._prepare_fn = lambda: itchat_msg.download(self.content)
+        else:
+            raise NotImplementedError("Unsupported message type: {}".format(itchat_msg['Type']))
+        
+        self.from_user_id = itchat_msg['FromUserName']
+        self.to_user_id = itchat_msg['ToUserName']
+        
+        user_id = itchat.instance.storageClass.userName
+        nickname = itchat.instance.storageClass.nickName
+        
+        # 虽然from_user_id和to_user_id用的少，但是为了保持一致性，还是要填充一下
+        # 以下很繁琐，一句话总结：能填的都填了。
+        if self.from_user_id == user_id:
+            self.from_user_nickname = nickname
+        if self.to_user_id == user_id:
+            self.to_user_nickname = nickname
+        try: # 陌生人时候, 'User'字段可能不存在
+            self.other_user_id = itchat_msg['User']['UserName']
+            self.other_user_nickname = itchat_msg['User']['NickName']
+            if self.other_user_id == self.from_user_id:
+                self.from_user_nickname = self.other_user_nickname
+            if self.other_user_id == self.to_user_id:
+                self.to_user_nickname = self.other_user_nickname
+        except KeyError as e: # 处理偶尔没有对方信息的情况
+            logger.warn("[WX]get other_user_id failed: " + str(e))
+            if self.from_user_id == user_id:
+                self.other_user_id = self.to_user_id
+            else:
+                self.other_user_id = self.from_user_id
+
+        if self.is_group:
+            self.is_at = itchat_msg['IsAt']
+            self.actual_user_id = itchat_msg['ActualUserName']
+            self.actual_user_nickname = itchat_msg['ActualNickName']

channel/wechat/wechaty_channel.py

@@ -4,289 +4,122 @@
 wechaty channel
 Python Wechaty - https://github.com/wechaty/python-wechaty
 """
-import io
+import base64
 import os
-import json
 import time
 import asyncio
-import requests
-import pysilk
-import wave
-from pydub import AudioSegment
-from typing import Optional, Union
-from bridge.context import Context, ContextType
-from wechaty_puppet import MessageType, FileBox, ScanStatus  # type: ignore
+from bridge.context import Context
+from wechaty_puppet import FileBox
 from wechaty import Wechaty, Contact
-from wechaty.user import Message, Room, MiniProgram, UrlLink
-from channel.channel import Channel
+from wechaty.user import Message
+from bridge.reply import *
+from bridge.context import *
+from channel.chat_channel import ChatChannel
+from channel.wechat.wechaty_message import WechatyMessage
 from common.log import logger
-from common.tmp_dir import TmpDir
+from common.singleton import singleton
 from config import conf
+try:
+    from voice.audio_convert import any_to_sil
+except Exception as e:
+    pass
 
-
-class WechatyChannel(Channel):
-
+@singleton
+class WechatyChannel(ChatChannel):
+    NOT_SUPPORT_REPLYTYPE = []
     def __init__(self):
-        pass
+        super().__init__()
 
     def startup(self):
+        config = conf()
+        token = config.get('wechaty_puppet_service_token')
+        os.environ['WECHATY_PUPPET_SERVICE_TOKEN'] = token
         asyncio.run(self.main())
 
     async def main(self):
-        config = conf()
-        # 使用PadLocal协议 比较稳定(免费web协议 os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:8080')
-        token = config.get('wechaty_puppet_service_token')
-        os.environ['WECHATY_PUPPET_SERVICE_TOKEN'] = token
-        global bot
-        bot = Wechaty()
-
-        bot.on('scan', self.on_scan)
-        bot.on('login', self.on_login)
-        bot.on('message', self.on_message)
-        await bot.start()
+        
+        loop = asyncio.get_event_loop()
+        #将asyncio的loop传入处理线程
+        self.handler_pool._initializer= lambda: asyncio.set_event_loop(loop)
+        self.bot = Wechaty()
+        self.bot.on('login', self.on_login)
+        self.bot.on('message', self.on_message)
+        await self.bot.start()
 
     async def on_login(self, contact: Contact):
+        self.user_id = contact.contact_id
+        self.name = contact.name
         logger.info('[WX] login user={}'.format(contact))
 
-    async def on_scan(self, status: ScanStatus, qr_code: Optional[str] = None,
-                      data: Optional[str] = None):
-        contact = self.Contact.load(self.contact_id)
-        logger.info('[WX] scan user={}, scan status={}, scan qr_code={}'.format(contact, status.name, qr_code))
-        # print(f'user <{contact}> scan status: {status.name} , 'f'qr_code: {qr_code}')
+    # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
+    def send(self, reply: Reply, context: Context):
+        receiver_id = context['receiver']
+        loop = asyncio.get_event_loop()
+        if context['isgroup']:
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Room.find(receiver_id),loop).result()
+        else:
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Contact.find(receiver_id),loop).result()
+        msg = None
+        if reply.type == ReplyType.TEXT:
+            msg = reply.content
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
+        elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
+            msg = reply.content
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
+        elif reply.type == ReplyType.VOICE:
+            voiceLength = None
+            file_path = reply.content
+            sil_file = os.path.splitext(file_path)[0] + '.sil'
+            voiceLength = int(any_to_sil(file_path, sil_file))
+            if voiceLength >= 60000:
+                voiceLength = 60000
+                logger.info('[WX] voice too long, length={}, set to 60s'.format(voiceLength))
+            # 发送语音
+            t = int(time.time())
+            msg = FileBox.from_file(sil_file, name=str(t) + '.sil')
+            if voiceLength is not None:
+                msg.metadata['voiceLength'] = voiceLength
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            try:
+                os.remove(file_path)
+                if sil_file != file_path:
+                    os.remove(sil_file)
+            except Exception as e:
+                pass
+            logger.info('[WX] sendVoice={}, receiver={}'.format(reply.content, receiver))
+        elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
+            img_url = reply.content
+            t = int(time.time())
+            msg = FileBox.from_url(url=img_url, name=str(t) + '.png')
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendImage url={}, receiver={}'.format(img_url,receiver))
+        elif reply.type == ReplyType.IMAGE: # 从文件读取图片
+            image_storage = reply.content
+            image_storage.seek(0)
+            t = int(time.time())
+            msg = FileBox.from_base64(base64.b64encode(image_storage.read()), str(t) + '.png')
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendImage, receiver={}'.format(receiver))
 
     async def on_message(self, msg: Message):
         """
         listen for message event
         """
-        from_contact = msg.talker()  # 获取消息的发送者
-        to_contact = msg.to()  # 接收人
-        room = msg.room()  # 获取消息来自的群聊. 如果消息不是来自群聊, 则返回None
-        from_user_id = from_contact.contact_id
-        to_user_id = to_contact.contact_id  # 接收人id
-        # other_user_id = msg['User']['UserName']  # 对手方id
-        content = msg.text()
-        mention_content = await msg.mention_text()  # 返回过滤掉@name后的消息
-        match_prefix = self.check_prefix(content, conf().get('single_chat_prefix'))
-        conversation: Union[Room, Contact] = from_contact if room is None else room
-
-        if room is None and msg.type() == MessageType.MESSAGE_TYPE_TEXT:
-            if not msg.is_self() and match_prefix is not None:
-                # 好友向自己发送消息
-                if match_prefix != '':
-                    str_list = content.split(match_prefix, 1)
-                    if len(str_list) == 2:
-                        content = str_list[1].strip()
-
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_img(content, from_user_id)
-                else:
-                    await self._do_send(content, from_user_id)
-            elif msg.is_self() and match_prefix:
-                # 自己给好友发送消息
-                str_list = content.split(match_prefix, 1)
-                if len(str_list) == 2:
-                    content = str_list[1].strip()
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_img(content, to_user_id)
-                else:
-                    await self._do_send(content, to_user_id)
-        elif room is None and msg.type() == MessageType.MESSAGE_TYPE_AUDIO:
-            if not msg.is_self(): # 接收语音消息
-                # 下载语音文件
-                voice_file = await msg.to_file_box()
-                silk_file = TmpDir().path() + voice_file.name
-                await voice_file.to_file(silk_file)
-                logger.info("[WX]receive voice file: " + silk_file)
-                # 将文件转成wav格式音频
-                wav_file = silk_file.replace(".slk", ".wav")
-                with open(silk_file, 'rb') as f:
-                    silk_data = f.read()
-                pcm_data = pysilk.decode(silk_data)
-
-                with wave.open(wav_file, 'wb') as wav_data:
-                    wav_data.setnchannels(1)
-                    wav_data.setsampwidth(2)
-                    wav_data.setframerate(24000)
-                    wav_data.writeframes(pcm_data)
-                if os.path.exists(wav_file): 
-                    converter_state = "true" # 转换wav成功
-                else:
-                    converter_state = "false" # 转换wav失败
-                logger.info("[WX]receive voice converter: " + converter_state)
-                # 语音识别为文本
-                query = super().build_voice_to_text(wav_file).content
-                # 交验关键字
-                match_prefix = self.check_prefix(query, conf().get('single_chat_prefix'))
-                if match_prefix is not None:
-                    if match_prefix != '':
-                        str_list = query.split(match_prefix, 1)
-                        if len(str_list) == 2:
-                            query = str_list[1].strip()
-                    # 返回消息
-                    if conf().get('voice_reply_voice'):
-                        await self._do_send_voice(query, from_user_id)
-                    else:
-                        await self._do_send(query, from_user_id)
-                else:
-                    logger.info("[WX]receive voice check prefix: " + 'False')
-                # 清除缓存文件
-                os.remove(wav_file)
-                os.remove(silk_file)
-        elif room and msg.type() == MessageType.MESSAGE_TYPE_TEXT:
-            # 群组&文本消息
-            room_id = room.room_id
-            room_name = await room.topic()
-            from_user_id = from_contact.contact_id
-            from_user_name = from_contact.name
-            is_at = await msg.mention_self()
-            content = mention_content
-            config = conf()
-            match_prefix = (is_at and not config.get("group_at_off", False)) \
-                           or self.check_prefix(content, config.get('group_chat_prefix')) \
-                           or self.check_contain(content, config.get('group_chat_keyword'))
-            # Wechaty判断is_at为True，返回的内容是过滤掉@之后的内容；而is_at为False，则会返回完整的内容
-            # 故判断如果匹配到自定义前缀，则返回过滤掉前缀+空格后的内容，用于实现类似自定义+前缀触发生成AI图片的功能
-            prefixes = config.get('group_chat_prefix')
-            for prefix in prefixes:
-                if content.startswith(prefix):
-                    content = content.replace(prefix, '', 1).strip()
-                    break
-            if ('ALL_GROUP' in config.get('group_name_white_list') or room_name in config.get(
-                    'group_name_white_list') or self.check_contain(room_name, config.get(
-                'group_name_keyword_white_list'))) and match_prefix:
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_group_img(content, room_id)
-                else:
-                    await self._do_send_group(content, room_id, room_name, from_user_id, from_user_name)
-
-    async def send(self, message: Union[str, Message, FileBox, Contact, UrlLink, MiniProgram], receiver):
-        logger.info('[WX] sendMsg={}, receiver={}'.format(message, receiver))
-        if receiver:
-            contact = await bot.Contact.find(receiver)
-            await contact.say(message)
-
-    async def send_group(self, message: Union[str, Message, FileBox, Contact, UrlLink, MiniProgram], receiver):
-        logger.info('[WX] sendMsg={}, receiver={}'.format(message, receiver))
-        if receiver:
-            room = await bot.Room.find(receiver)
-            await room.say(message)
-
-    async def _do_send(self, query, reply_user_id):
         try:
-            if not query:
-                return
-            context = Context(ContextType.TEXT, query)
-            context['session_id'] = reply_user_id
-            reply_text = super().build_reply_content(query, context).content
-            if reply_text:
-                await self.send(conf().get("single_chat_reply_prefix") + reply_text, reply_user_id)
-        except Exception as e:
-            logger.exception(e)
-
-
-    async def _do_send_voice(self, query, reply_user_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.TEXT, query)
-            context['session_id'] = reply_user_id
-            reply_text = super().build_reply_content(query, context).content
-            if reply_text:
-                # 转换 mp3 文件为 silk 格式
-                mp3_file = super().build_text_to_voice(reply_text).content
-                silk_file = mp3_file.replace(".mp3", ".silk")
-                # Load the MP3 file
-                audio = AudioSegment.from_file(mp3_file, format="mp3")
-                # Convert to WAV format
-                audio = audio.set_frame_rate(24000).set_channels(1)
-                wav_data = audio.raw_data
-                sample_width = audio.sample_width
-                # Encode to SILK format
-                silk_data = pysilk.encode(wav_data, 24000)
-                # Save the silk file
-                with open(silk_file, "wb") as f:
-                    f.write(silk_data)
-                # 发送语音
-                t = int(time.time())
-                file_box = FileBox.from_file(silk_file, name=str(t) + '.silk')
-                await self.send(file_box, reply_user_id)
-                # 清除缓存文件
-                os.remove(mp3_file)
-                os.remove(silk_file)
-        except Exception as e:
-            logger.exception(e)
-            
-    async def _do_send_img(self, query, reply_user_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.IMAGE_CREATE, query)
-            img_url = super().build_reply_content(query, context).content
-            if not img_url:
-                return
-            # 图片下载
-            # pic_res = requests.get(img_url, stream=True)
-            # image_storage = io.BytesIO()
-            # for block in pic_res.iter_content(1024):
-            #     image_storage.write(block)
-            # image_storage.seek(0)
-
-            # 图片发送
-            logger.info('[WX] sendImage, receiver={}'.format(reply_user_id))
-            t = int(time.time())
-            file_box = FileBox.from_url(url=img_url, name=str(t) + '.png')
-            await self.send(file_box, reply_user_id)
-        except Exception as e:
-            logger.exception(e)
-
-    async def _do_send_group(self, query, group_id, group_name, group_user_id, group_user_name):
-        if not query:
+            cmsg = await WechatyMessage(msg)
+        except NotImplementedError as e:
+            logger.debug('[WX] {}'.format(e))
             return
-        context = Context(ContextType.TEXT, query)
-        group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
-        if ('ALL_GROUP' in group_chat_in_one_session or \
-                group_name in group_chat_in_one_session or \
-                self.check_contain(group_name, group_chat_in_one_session)):
-            context['session_id'] = str(group_id)
-        else:
-            context['session_id'] = str(group_id) + '-' + str(group_user_id)
-        reply_text = super().build_reply_content(query, context).content
-        if reply_text:
-            reply_text = '@' + group_user_name + ' ' + reply_text.strip()
-            await self.send_group(conf().get("group_chat_reply_prefix", "") + reply_text, group_id)
-
-    async def _do_send_group_img(self, query, reply_room_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.IMAGE_CREATE, query)
-            img_url = super().build_reply_content(query, context).content
-            if not img_url:
-                return
-            # 图片发送
-            logger.info('[WX] sendImage, receiver={}'.format(reply_room_id))
-            t = int(time.time())
-            file_box = FileBox.from_url(url=img_url, name=str(t) + '.png')
-            await self.send_group(file_box, reply_room_id)
         except Exception as e:
-            logger.exception(e)
-
-    def check_prefix(self, content, prefix_list):
-        for prefix in prefix_list:
-            if content.startswith(prefix):
-                return prefix
-        return None
-
-    def check_contain(self, content, keyword_list):
-        if not keyword_list:
-            return None
-        for ky in keyword_list:
-            if content.find(ky) != -1:
-                return True
-        return None
+            logger.exception('[WX] {}'.format(e))
+            return
+        logger.debug('[WX] message:{}'.format(cmsg))
+        room = msg.room()  # 获取消息来自的群聊. 如果消息不是来自群聊, 则返回None
+        isgroup = room is not None
+        ctype = cmsg.ctype
+        context = self._compose_context(ctype, cmsg.content, isgroup=isgroup, msg=cmsg)
+        if context:
+            logger.info('[WX] receiveMsg={}, context={}'.format(cmsg, context))
+            self.produce(context)

channel/wechat/wechaty_message.py

@@ -0,0 +1,85 @@
+import asyncio
+import re
+from wechaty import MessageType
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.tmp_dir import TmpDir
+from common.log import logger
+from wechaty.user import Message
+
+class aobject(object):
+    """Inheriting this class allows you to define an async __init__.
+
+    So you can create objects by doing something like `await MyClass(params)`
+    """
+    async def __new__(cls, *a, **kw):
+        instance = super().__new__(cls)
+        await instance.__init__(*a, **kw)
+        return instance
+
+    async def __init__(self):
+        pass
+class WechatyMessage(ChatMessage, aobject):
+
+    async def __init__(self, wechaty_msg: Message):
+        super().__init__(wechaty_msg)
+        
+        room = wechaty_msg.room()
+
+        self.msg_id = wechaty_msg.message_id
+        self.create_time = wechaty_msg.payload.timestamp
+        self.is_group = room is not None
+        
+        if wechaty_msg.type() == MessageType.MESSAGE_TYPE_TEXT:
+            self.ctype = ContextType.TEXT
+            self.content = wechaty_msg.text()
+        elif wechaty_msg.type() == MessageType.MESSAGE_TYPE_AUDIO:
+            self.ctype = ContextType.VOICE
+            voice_file = await wechaty_msg.to_file_box()
+            self.content = TmpDir().path() + voice_file.name  # content直接存临时目录路径
+
+            def func():
+                loop = asyncio.get_event_loop()
+                asyncio.run_coroutine_threadsafe(voice_file.to_file(self.content),loop).result()
+            self._prepare_fn = func
+            
+        else:
+            raise NotImplementedError("Unsupported message type: {}".format(wechaty_msg.type()))
+        
+        from_contact = wechaty_msg.talker()  # 获取消息的发送者
+        self.from_user_id = from_contact.contact_id
+        self.from_user_nickname = from_contact.name
+
+        # group中的from和to，wechaty跟itchat含义不一样
+        # wecahty: from是消息实际发送者, to:所在群
+        # itchat: 如果是你发送群消息，from和to是你自己和所在群，如果是别人发群消息，from和to是所在群和你自己
+        # 但这个差别不影响逻辑，group中只使用到：1.用from来判断是否是自己发的，2.actual_user_id来判断实际发送用户
+        
+        if self.is_group:
+            self.to_user_id = room.room_id
+            self.to_user_nickname = await room.topic()
+        else:
+            to_contact = wechaty_msg.to()
+            self.to_user_id = to_contact.contact_id
+            self.to_user_nickname = to_contact.name
+
+        if self.is_group or wechaty_msg.is_self(): # 如果是群消息，other_user设置为群，如果是私聊消息，而且自己发的，就设置成对方。
+            self.other_user_id = self.to_user_id
+            self.other_user_nickname = self.to_user_nickname
+        else:
+            self.other_user_id = self.from_user_id
+            self.other_user_nickname = self.from_user_nickname
+
+        
+
+        if self.is_group: # wechaty群聊中，实际发送用户就是from_user
+            self.is_at = await wechaty_msg.mention_self()
+            if not self.is_at: # 有时候复制粘贴的消息，不算做@，但是内容里面会有@xxx，这里做一下兼容
+                name = wechaty_msg.wechaty.user_self().name
+                pattern = f'@{name}(\u2005|\u0020)'
+                if re.search(pattern,self.content):
+                    logger.debug(f'wechaty message {self.msg_id} include at')
+                    self.is_at = True
+
+            self.actual_user_id = self.from_user_id
+            self.actual_user_nickname = self.from_user_nickname

channel/wechatmp/README.md

@@ -0,0 +1,46 @@
+# 个人微信公众号channel
+
+鉴于个人微信号在服务器上通过itchat登录有封号风险，这里新增了个人微信公众号channel，提供无风险的服务。
+但是由于个人微信公众号的众多接口限制，目前支持的功能有限，实现简陋，提供了一个最基本的文本对话服务，支持加载插件，优化了命令格式，支持私有api_key。暂未实现图片输入输出、语音输入输出等交互形式。
+如有公众号是企业主体且可以通过微信认证，即可获得更多接口，解除大多数限制。欢迎大家提供更多的支持。
+
+## 使用方法
+
+在开始部署前，你需要一个拥有公网IP的服务器，以提供微信服务器和我们自己服务器的连接。或者你需要进行内网穿透，否则微信服务器无法将消息发送给我们的服务器。
+
+此外，需要在我们的服务器上安装python的web框架web.py。
+以ubuntu为例(在ubuntu 22.04上测试):
+```
+pip3 install web.py
+```
+
+然后在[微信公众平台](https://mp.weixin.qq.com)注册一个自己的公众号，类型选择订阅号，主体为个人即可。
+
+然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。这里的`URL`是`example.com/wx`的形式，不可以使用IP，`Token`是你自己编的一个特定的令牌。消息加解密方式目前选择的是明文模式。
+
+相关的服务器验证代码已经写好，你不需要再添加任何代码。你只需要在本项目根目录的`config.json`中添加
+```
+"channel_type": "wechatmp", 
+"wechatmp_token": "your Token", 
+"wechatmp_port": 8080,
+``` 
+然后运行`python3 app.py`启动web服务器。这里会默认监听8080端口，但是微信公众号的服务器配置只支持80/443端口，有两种方法来解决这个问题。第一个是推荐的方法，使用端口转发命令将80端口转发到8080端口（443同理，注意需要支持SSL，也就是https的访问，在`wechatmp_channel.py`需要修改相应的证书路径）：
+```
+sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080
+sudo iptables-save > /etc/iptables/rules.v4
+```
+第二个方法是让python程序直接监听80端口。这样可能会导致权限问题，在linux上需要使用`sudo`。然而这会导致后续缓存文件的权限问题，因此不是推荐的方法。
+最后在刚才的“服务器配置”中点击`提交`即可验证你的服务器。
+
+随后在[微信公众平台](https://mp.weixin.qq.com)启用服务器，关闭手动填写规则的自动回复，即可实现ChatGPT的自动回复。
+
+## 个人微信公众号的限制
+由于目前测试的公众号不是企业主体，所以没有客服接口，因此公众号无法主动发出消息，只能被动回复。而微信官方对被动回复有5秒的时间限制，最多重试2次，因此最多只有15秒的自动回复时间窗口。因此如果问题比较复杂或者我们的服务器比较忙，ChatGPT的回答就没办法及时回复给用户。为了解决这个问题，这里做了回答缓存，它需要你在回复超时后，再次主动发送任意文字（例如1）来尝试拿到回答缓存。为了优化使用体验，目前设置了两分钟（120秒）的timeout，用户在至多两分钟后即可得到查询到回复或者错误原因。
+
+另外，由于微信官方的限制，自动回复有长度限制。因此这里将ChatGPT的回答拆分，分成每段600字回复（限制大约在700字）。
+
+## 私有api_key
+公共api有访问频率限制（免费账号每分钟最多20次ChatGPT的API调用），这在服务多人的时候会遇到问题。因此这里多加了一个设置私有api_key的功能。目前通过godcmd插件的命令来设置私有api_key。
+
+## 测试范围
+目前在`RoboStyle`这个公众号上进行了测试，感兴趣的可以关注并体验。开启了godcmd, Banwords, role, dungeon, finish这五个插件，其他的插件还没有测试。百度的接口暂未测试。语音对话没有测试。图片直接以链接形式回复（没有临时素材上传接口的权限）。

channel/wechatmp/receive.py

@@ -0,0 +1,42 @@
+# -*- coding: utf-8 -*-#
+# filename: receive.py
+import xml.etree.ElementTree as ET
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.log import logger
+
+
+def parse_xml(web_data):
+    if len(web_data) == 0:
+        return None
+    xmlData = ET.fromstring(web_data)
+    return WeChatMPMessage(xmlData)
+
+class WeChatMPMessage(ChatMessage):
+    def __init__(self, xmlData):
+        super().__init__(xmlData)
+        self.to_user_id = xmlData.find('ToUserName').text
+        self.from_user_id = xmlData.find('FromUserName').text
+        self.create_time = xmlData.find('CreateTime').text
+        self.msg_type = xmlData.find('MsgType').text
+        self.msg_id = xmlData.find('MsgId').text
+        self.is_group = False
+        
+        # reply to other_user_id
+        self.other_user_id = self.from_user_id
+
+        if self.msg_type == 'text':
+            self.ctype = ContextType.TEXT
+            self.content = xmlData.find('Content').text.encode("utf-8")
+        elif self.msg_type == 'voice':
+            self.ctype = ContextType.TEXT
+            self.content = xmlData.find('Recognition').text.encode("utf-8")  # 接收语音识别结果
+        elif self.msg_type == 'image':
+            # not implemented
+            self.pic_url = xmlData.find('PicUrl').text
+            self.media_id = xmlData.find('MediaId').text
+        elif self.msg_type == 'event':
+            self.event = xmlData.find('Event').text
+        else: # video, shortvideo, location, link
+            # not implemented
+            pass

channel/wechatmp/reply.py

@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-#
+# filename: reply.py
+import time
+
+class Msg(object):
+    def __init__(self):
+        pass
+
+    def send(self):
+        return "success"
+
+class TextMsg(Msg):
+    def __init__(self, toUserName, fromUserName, content):
+        self.__dict = dict()
+        self.__dict['ToUserName'] = toUserName
+        self.__dict['FromUserName'] = fromUserName
+        self.__dict['CreateTime'] = int(time.time())
+        self.__dict['Content'] = content
+
+    def send(self):
+        XmlForm = """
+            <xml>
+                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
+                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
+                <CreateTime>{CreateTime}</CreateTime>
+                <MsgType><![CDATA[text]]></MsgType>
+                <Content><![CDATA[{Content}]]></Content>
+            </xml>
+            """
+        return XmlForm.format(**self.__dict)
+
+class ImageMsg(Msg):
+    def __init__(self, toUserName, fromUserName, mediaId):
+        self.__dict = dict()
+        self.__dict['ToUserName'] = toUserName
+        self.__dict['FromUserName'] = fromUserName
+        self.__dict['CreateTime'] = int(time.time())
+        self.__dict['MediaId'] = mediaId
+
+    def send(self):
+        XmlForm = """
+            <xml>
+                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
+                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
+                <CreateTime>{CreateTime}</CreateTime>
+                <MsgType><![CDATA[image]]></MsgType>
+                <Image>
+                <MediaId><![CDATA[{MediaId}]]></MediaId>
+                </Image>
+            </xml>
+            """
+        return XmlForm.format(**self.__dict)

channel/wechatmp/wechatmp_channel.py

@@ -0,0 +1,234 @@
+# -*- coding: utf-8 -*-
+import web
+import time
+import math
+import hashlib
+import textwrap
+from channel.chat_channel import ChatChannel
+import channel.wechatmp.reply as reply
+import channel.wechatmp.receive as receive
+from common.singleton import singleton
+from common.log import logger
+from config import conf
+from bridge.reply import *
+from bridge.context import *
+from plugins import *
+import traceback
+
+# If using SSL, uncomment the following lines, and modify the certificate path.
+# from cheroot.server import HTTPServer
+# from cheroot.ssl.builtin import BuiltinSSLAdapter
+# HTTPServer.ssl_adapter = BuiltinSSLAdapter(
+#         certificate='/ssl/cert.pem',
+#         private_key='/ssl/cert.key')
+
+
+# from concurrent.futures import ThreadPoolExecutor
+# thread_pool = ThreadPoolExecutor(max_workers=8)
+
+@singleton
+class WechatMPChannel(ChatChannel):
+    NOT_SUPPORT_REPLYTYPE = [ReplyType.IMAGE, ReplyType.VOICE]
+    def __init__(self):
+        super().__init__()
+        self.cache_dict = dict()
+        self.query1 = dict()
+        self.query2 = dict()
+        self.query3 = dict()
+
+
+    def startup(self):
+        urls = (
+            '/wx', 'SubsribeAccountQuery',
+        )
+        app = web.application(urls, globals(), autoreload=False)
+        port = conf().get('wechatmp_port', 8080)
+        web.httpserver.runsimple(app.wsgifunc(), ('0.0.0.0', port))
+
+
+    def send(self, reply: Reply, context: Context):
+        reply_cnt = math.ceil(len(reply.content) / 600)
+        receiver = context["receiver"]
+        self.cache_dict[receiver] = (reply_cnt, reply.content)
+        logger.debug("[send] reply to {} saved to cache: {}".format(receiver, reply))
+
+
+def verify_server():
+    try:
+        data = web.input()
+        if len(data) == 0:
+            return "None"
+        signature = data.signature
+        timestamp = data.timestamp
+        nonce = data.nonce
+        echostr = data.echostr
+        token = conf().get('wechatmp_token') #请按照公众平台官网\基本配置中信息填写
+
+        data_list = [token, timestamp, nonce]
+        data_list.sort()
+        sha1 = hashlib.sha1()
+        # map(sha1.update, data_list) #python2
+        sha1.update("".join(data_list).encode('utf-8'))
+        hashcode = sha1.hexdigest()
+        print("handle/GET func: hashcode, signature: ", hashcode, signature)
+        if hashcode == signature:
+            return echostr
+        else:
+            return ""
+    except Exception as Argument:
+        return Argument
+
+
+# This class is instantiated once per query
+class SubsribeAccountQuery():
+
+    def GET(self):
+        return verify_server()
+
+    def POST(self):
+        channel_instance = WechatMPChannel()
+        try:
+            query_time = time.time()
+            webData = web.data()
+            # logger.debug("[wechatmp] Receive request:\n" + webData.decode("utf-8"))
+            wechat_msg = receive.parse_xml(webData)
+            if wechat_msg.msg_type == 'text':
+                from_user = wechat_msg.from_user_id
+                to_user = wechat_msg.to_user_id
+                message = wechat_msg.content.decode("utf-8")
+                message_id = wechat_msg.msg_id
+
+                logger.info("[wechatmp] {}:{} Receive post query {} {}: {}".format(web.ctx.env.get('REMOTE_ADDR'), web.ctx.env.get('REMOTE_PORT'), from_user, message_id, message))
+
+                cache_key = from_user
+                cache = channel_instance.cache_dict.get(cache_key)
+
+                reply_text = ""
+                # New request
+                if cache == None:
+                    # The first query begin, reset the cache
+                    context = channel_instance._compose_context(ContextType.TEXT, message, isgroup=False, msg=wechat_msg)
+                    logger.debug("[wechatmp] context: {} {}".format(context, wechat_msg))
+                    if context:
+                        # set private openai_api_key
+                        # if from_user is not changed in itchat, this can be placed at chat_channel
+                        user_data = conf().get_user_data(from_user)
+                        context['openai_api_key'] = user_data.get('openai_api_key') # None or user openai_api_key
+                        channel_instance.cache_dict[cache_key] = (0, "")
+                        channel_instance.produce(context)
+                    else:
+                        trigger_prefix = conf().get('single_chat_prefix',[''])[0]
+                        if trigger_prefix:
+                            content = textwrap.dedent(f"""\
+                                请输入'{trigger_prefix}'接你想说的话跟我说话。
+                                例如:
+                                {trigger_prefix}你好，很高兴见到你。""")
+                        else:
+                            logger.error(f"[wechatmp] unknown error")
+                            content = textwrap.dedent("""\
+                                未知错误，请稍后再试""")
+                        replyMsg = reply.TextMsg(wechat_msg.from_user_id, wechat_msg.to_user_id, content)
+                        return replyMsg.send()
+                    channel_instance.query1[cache_key] = False
+                    channel_instance.query2[cache_key] = False
+                    channel_instance.query3[cache_key] = False
+                # Request again
+                elif cache[0] == 0 and channel_instance.query1.get(cache_key) == True and channel_instance.query2.get(cache_key) == True and channel_instance.query3.get(cache_key) == True:
+                    channel_instance.query1[cache_key] = False  #To improve waiting experience, this can be set to True.
+                    channel_instance.query2[cache_key] = False  #To improve waiting experience, this can be set to True.
+                    channel_instance.query3[cache_key] = False
+                elif cache[0] >= 1:
+                    # Skip the waiting phase
+                    channel_instance.query1[cache_key] = True
+                    channel_instance.query2[cache_key] = True
+                    channel_instance.query3[cache_key] = True
+
+
+                cache = channel_instance.cache_dict.get(cache_key)
+                if channel_instance.query1.get(cache_key) == False:
+                    # The first query from wechat official server
+                    logger.debug("[wechatmp] query1 {}".format(cache_key))
+                    channel_instance.query1[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 45:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = channel_instance.cache_dict.get(cache_key)
+                    if cnt == 45:
+                        # waiting for timeout (the POST query will be closed by wechat official server)
+                        time.sleep(5)
+                        # and do nothing
+                        return
+                    else:
+                        pass
+                elif channel_instance.query2.get(cache_key) == False:
+                    # The second query from wechat official server
+                    logger.debug("[wechatmp] query2 {}".format(cache_key))
+                    channel_instance.query2[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 45:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = channel_instance.cache_dict.get(cache_key)
+                    if cnt == 45:
+                        # waiting for timeout (the POST query will be closed by wechat official server)
+                        time.sleep(5)
+                        # and do nothing
+                        return
+                    else:
+                        pass
+                elif channel_instance.query3.get(cache_key) == False:
+                    # The third query from wechat official server
+                    logger.debug("[wechatmp] query3 {}".format(cache_key))
+                    channel_instance.query3[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 40:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = channel_instance.cache_dict.get(cache_key)
+                    if cnt == 40:
+                        # Have waiting for 3x5 seconds
+                        # return timeout message
+                        reply_text = "【正在思考中，回复任意文字尝试获取回复】"
+                        logger.info("[wechatmp] Three queries has finished For {}: {}".format(from_user, message_id))
+                        replyPost = reply.TextMsg(from_user, to_user, reply_text).send()
+                        return replyPost
+                    else:
+                        pass
+
+                if float(time.time()) - float(query_time) > 4.8:
+                    logger.info("[wechatmp] Timeout for {} {}".format(from_user, message_id))
+                    return
+
+
+                if cache[0] > 1:
+                    reply_text = cache[1][:600] + "\n【未完待续，回复任意文字以继续】" #wechatmp auto_reply length limit
+                    channel_instance.cache_dict[cache_key] = (cache[0] - 1, cache[1][600:])
+                elif cache[0] == 1:
+                    reply_text = cache[1]
+                    channel_instance.cache_dict.pop(cache_key)
+                logger.info("[wechatmp] {}:{} Do send {}".format(web.ctx.env.get('REMOTE_ADDR'), web.ctx.env.get('REMOTE_PORT'), reply_text))
+                replyPost = reply.TextMsg(from_user, to_user, reply_text).send()
+                return replyPost
+
+            elif wechat_msg.msg_type == 'event':
+                logger.info("[wechatmp] Event {} from {}".format(wechat_msg.Event, wechat_msg.from_user_id))
+                trigger_prefix = conf().get('single_chat_prefix',[''])[0]
+                content = textwrap.dedent(f"""\
+                    感谢您的关注！
+                    这里是ChatGPT，可以自由对话。
+                    资源有限，回复较慢，请勿着急。
+                    支持通用表情输入。
+                    暂时不支持图片输入。
+                    支持图片输出，画字开头的问题将回复图片链接。
+                    支持角色扮演和文字冒险两种定制模式对话。
+                    输入'{trigger_prefix}#帮助' 查看详细指令。""")
+                replyMsg = reply.TextMsg(wechat_msg.from_user_id, wechat_msg.to_user_id, content)
+                return replyMsg.send()
+            else:
+                logger.info("暂且不处理")
+                return "success"
+        except Exception as exc:
+            logger.exception(exc)
+            return exc
+

common/dequeue.py

@@ -0,0 +1,33 @@
+
+from queue import Full, Queue
+from time import monotonic as time
+
+# add implementation of putleft to Queue
+class Dequeue(Queue):
+    def putleft(self, item, block=True, timeout=None):
+        with self.not_full:
+            if self.maxsize > 0:
+                if not block:
+                    if self._qsize() >= self.maxsize:
+                        raise Full
+                elif timeout is None:
+                    while self._qsize() >= self.maxsize:
+                        self.not_full.wait()
+                elif timeout < 0:
+                    raise ValueError("'timeout' must be a non-negative number")
+                else:
+                    endtime = time() + timeout
+                    while self._qsize() >= self.maxsize:
+                        remaining = endtime - time()
+                        if remaining <= 0.0:
+                            raise Full
+                        self.not_full.wait(remaining)
+            self._putleft(item)
+            self.unfinished_tasks += 1
+            self.not_empty.notify()
+
+    def putleft_nowait(self, item):
+        return self.putleft(item, block=False)
+
+    def _putleft(self, item):
+        self.queue.appendleft(item)

common/log.py

@@ -2,13 +2,26 @@ import logging
 import sys
 
 
-def _get_logger():
-    log = logging.getLogger('log')
-    log.setLevel(logging.INFO)
+def _reset_logger(log):
+    for handler in log.handlers:
+        handler.close()
+        log.removeHandler(handler)
+        del handler
+    log.handlers.clear()
+    log.propagate = False
     console_handle = logging.StreamHandler(sys.stdout)
     console_handle.setFormatter(logging.Formatter('[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d] - %(message)s',
                                                   datefmt='%Y-%m-%d %H:%M:%S'))
+    file_handle = logging.FileHandler('run.log', encoding='utf-8')
+    file_handle.setFormatter(logging.Formatter('[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d] - %(message)s',
+                                                  datefmt='%Y-%m-%d %H:%M:%S'))
+    log.addHandler(file_handle)
     log.addHandler(console_handle)
+
+def _get_logger():
+    log = logging.getLogger('log')
+    _reset_logger(log)
+    log.setLevel(logging.INFO)
     return log
 
 

common/package_manager.py

@@ -0,0 +1,30 @@
+import time
+import pip
+from pip._internal import main as pipmain
+from common.log import logger,_reset_logger
+
+def install(package):
+    pipmain(['install', package])
+
+def install_requirements(file):
+    pipmain(['install', '-r', file, "--upgrade"])
+    _reset_logger(logger)
+
+def check_dulwich():
+    needwait = False
+    for i in range(2):
+        if needwait:
+            time.sleep(3)
+            needwait = False
+        try:
+            import dulwich
+            return
+        except ImportError:
+            try:
+                install('dulwich')
+            except:
+                needwait = True
+    try:
+        import dulwich
+    except ImportError:
+        raise ImportError("Unable to import dulwich")

config-template.json

@@ -10,6 +10,7 @@
   "group_chat_in_one_session": ["ChatGPT测试群"],
   "image_create_prefix": ["画", "看", "找"],
   "speech_recognition": false,
+  "group_speech_recognition": false,
   "voice_reply_voice": false,
   "conversation_max_tokens": 1000,
   "expires_in_seconds": 3600,

config.py

@@ -1,75 +1,106 @@
 # encoding:utf-8
 
 import json
+import logging
 import os
 from common.log import logger
+import pickle
 
 # 将所有可用的配置项写在字典里, 请使用小写字母
-available_setting ={
-    #openai api配置
-    "open_ai_api_key": "", # openai api key
-    "open_ai_api_base": "https://api.openai.com/v1", # openai apibase，当use_azure_chatgpt为true时，需要设置对应的api base
-    "proxy": "", # openai使用的代理
-    "model": "gpt-3.5-turbo", # chatgpt模型， 当use_azure_chatgpt为true时，其名称为Azure上model deployment名称
-    "use_azure_chatgpt": False, # 是否使用azure的chatgpt
+available_setting = {
+    # openai api配置
+    "open_ai_api_key": "",  # openai api key
+    # openai apibase，当use_azure_chatgpt为true时，需要设置对应的api base
+    "open_ai_api_base": "https://api.openai.com/v1",
+    "proxy": "",  # openai使用的代理
+    # chatgpt模型， 当use_azure_chatgpt为true时，其名称为Azure上model deployment名称
+    "model": "gpt-3.5-turbo",
+    "use_azure_chatgpt": False,  # 是否使用azure的chatgpt
 
-    #Bot触发配置
-    "single_chat_prefix": ["bot", "@bot"], # 私聊时文本需要包含该前缀才能触发机器人回复
-    "single_chat_reply_prefix": "[bot] ", # 私聊时自动回复的前缀，用于区分真人
-    "group_chat_prefix": ["@bot"], # 群聊时包含该前缀则会触发机器人回复
-    "group_chat_reply_prefix": "", # 群聊时自动回复的前缀
-    "group_chat_keyword": [], # 群聊时包含该关键词则会触发机器人回复
-    "group_at_off": False, # 是否关闭群聊时@bot的触发
-    "group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"], # 开启自动回复的群名称列表
-    "group_name_keyword_white_list": [], # 开启自动回复的群名称关键词列表
-    "group_chat_in_one_session": ["ChatGPT测试群"], # 支持会话上下文共享的群名称
-    "image_create_prefix": ["画", "看", "找"], # 开启图片回复的前缀
-    
-    #chatgpt会话参数
-    "expires_in_seconds": 3600, # 无操作会话的过期时间
-    "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。", # 人格描述
-    "conversation_max_tokens": 1000, # 支持上下文记忆的最多字符数
-            
-    #chatgpt限流配置
-    "rate_limit_chatgpt": 20, # chatgpt的调用频率限制
-    "rate_limit_dalle": 50, # openai dalle的调用频率限制
+    # Bot触发配置
+    "single_chat_prefix": ["bot", "@bot"],  # 私聊时文本需要包含该前缀才能触发机器人回复
+    "single_chat_reply_prefix": "[bot] ",  # 私聊时自动回复的前缀，用于区分真人
+    "group_chat_prefix": ["@bot"],  # 群聊时包含该前缀则会触发机器人回复
+    "group_chat_reply_prefix": "",  # 群聊时自动回复的前缀
+    "group_chat_keyword": [],  # 群聊时包含该关键词则会触发机器人回复
+    "group_at_off": False,  # 是否关闭群聊时@bot的触发
+    "group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"],  # 开启自动回复的群名称列表
+    "group_name_keyword_white_list": [],  # 开启自动回复的群名称关键词列表
+    "group_chat_in_one_session": ["ChatGPT测试群"],  # 支持会话上下文共享的群名称
+    "trigger_by_self": False,  # 是否允许机器人触发
+    "image_create_prefix": ["画", "看", "找"],  # 开启图片回复的前缀
+    "concurrency_in_session": 1, # 同一会话最多有多少条消息在处理中，大于1可能乱序
 
+    # chatgpt会话参数
+    "expires_in_seconds": 3600,  # 无操作会话的过期时间
+    "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述
+    "conversation_max_tokens": 1000,  # 支持上下文记忆的最多字符数
 
-    #chatgpt api参数 参考https://platform.openai.com/docs/api-reference/chat/create
+    # chatgpt限流配置
+    "rate_limit_chatgpt": 20,  # chatgpt的调用频率限制
+    "rate_limit_dalle": 50,  # openai dalle的调用频率限制
+
+    # chatgpt api参数 参考https://platform.openai.com/docs/api-reference/chat/create
     "temperature": 0.9,
     "top_p": 1,
     "frequency_penalty": 0,
     "presence_penalty": 0,
+    "request_timeout": 60, # chatgpt请求超时时间，openai接口默认设置为600，对于难问题一般需要较长时间
+    "timeout": 120,         # chatgpt重试超时时间，在这个时间内，将会自动重试
 
-    #语音设置
-    "speech_recognition": False, # 是否开启语音识别
-    "voice_reply_voice": False, # 是否使用语音回复语音，需要设置对应语音合成引擎的api key
-    "voice_to_text": "openai", # 语音识别引擎，支持openai和google
-    "text_to_voice": "baidu", # 语音合成引擎，支持baidu和google
+    # 语音设置
+    "speech_recognition": False,  # 是否开启语音识别
+    "group_speech_recognition": False,  # 是否开启群组语音识别
+    "voice_reply_voice": False,  # 是否使用语音回复语音，需要设置对应语音合成引擎的api key
+    "always_reply_voice": False,  # 是否一直使用语音回复
+    "voice_to_text": "openai",  # 语音识别引擎，支持openai,google,azure
+    "text_to_voice": "baidu",  # 语音合成引擎，支持baidu,google,pytts(offline),azure
 
-    # baidu api的配置， 使用百度语音识别和语音合成时需要
-    'baidu_app_id': "",
-    'baidu_api_key': "",
-    'baidu_secret_key': "",
+    # baidu 语音api配置， 使用百度语音识别和语音合成时需要
+    "baidu_app_id": "",
+    "baidu_api_key": "",
+    "baidu_secret_key": "",
+    # 1536普通话(支持简单的英文识别) 1737英语 1637粤语 1837四川话 1936普通话远场
+    "baidu_dev_pid": "1536",
 
-    #服务时间限制，目前支持itchat
-    "chat_time_module": False, # 是否开启服务时间限制
-    "chat_start_time": "00:00", # 服务开始时间
-    "chat_stop_time": "24:00", # 服务结束时间
+    # azure 语音api配置， 使用azure语音识别和语音合成时需要
+    "azure_voice_api_key": "",
+    "azure_voice_region": "japaneast",
+
+    # 服务时间限制，目前支持itchat
+    "chat_time_module": False,  # 是否开启服务时间限制
+    "chat_start_time": "00:00",  # 服务开始时间
+    "chat_stop_time": "24:00",  # 服务结束时间
 
     # itchat的配置
-    "hot_reload": False, # 是否开启热重载
+    "hot_reload": False,  # 是否开启热重载
 
     # wechaty的配置
-    "wechaty_puppet_service_token": "", # wechaty的token
+    "wechaty_puppet_service_token": "",  # wechaty的token
+
+    # wechatmp的配置
+    "wechatmp_token": "",  # 微信公众平台的Token
+    "wechatmp_port": 8080, # 微信公众平台的端口,需要端口转发到80或443
 
     # chatgpt指令自定义触发词
-    "clear_memory_commands": ['#清除记忆'], # 重置会话指令
+    "clear_memory_commands": ['#清除记忆'],  # 重置会话指令，必须以#开头
 
+    # channel配置
+    "channel_type": "wx", # 通道类型，支持：{wx,wxy,terminal,wechatmp}
 
+    "debug": False,  # 是否开启debug模式，开启后会打印更多日志
+
+    # 插件配置
+    "plugin_trigger_prefix": "$",  # 规范插件提供聊天相关指令的前缀，建议不要和管理员指令前缀"#"冲突
 }
 
+
 class Config(dict):
+    def __init__(self, d:dict={}):
+        super().__init__(d)
+        # user_datas: 用户数据，key为用户名，value为用户数据，也是dict
+        self.user_datas = {}
+
     def __getitem__(self, key):
         if key not in available_setting:
             raise Exception("key {} not in available_setting".format(key))
@@ -81,15 +112,41 @@ class Config(dict):
         return super().__setitem__(key, value)
 
     def get(self, key, default=None):
-        try :
+        try:
             return self[key]
         except KeyError as e:
             return default
         except Exception as e:
             raise e
-    
+
+    # Make sure to return a dictionary to ensure atomic
+    def get_user_data(self, user) -> dict:
+        if self.user_datas.get(user) is None:
+            self.user_datas[user] = {}
+        return self.user_datas[user]
+
+    def load_user_datas(self):
+        try:
+            with open('user_datas.pkl', 'rb') as f:
+                self.user_datas = pickle.load(f)
+                logger.info("[Config] User datas loaded.")
+        except FileNotFoundError as e:
+            logger.info("[Config] User datas file not found, ignore.")
+        except Exception as e:
+            logger.info("[Config] User datas error: {}".format(e))
+            self.user_datas = {}
+
+    def save_user_datas(self):
+        try:
+            with open('user_datas.pkl', 'wb') as f:
+                pickle.dump(self.user_datas, f)
+                logger.info("[Config] User datas saved.")
+        except Exception as e:
+            logger.info("[Config] User datas error: {}".format(e))
+
 config = Config()
 
+
 def load_config():
     global config
     config_path = "./config.json"
@@ -108,18 +165,28 @@ def load_config():
     for name, value in os.environ.items():
         name = name.lower()
         if name in available_setting:
-            logger.info("[INIT] override config by environ args: {}={}".format(name, value))
+            logger.info(
+                "[INIT] override config by environ args: {}={}".format(name, value))
             try:
                 config[name] = eval(value)
             except:
-                config[name] = value
+                if value == "false":
+                    config[name] = False
+                elif value == "true":
+                    config[name] = True
+                else:
+                    config[name] = value
+
+    if config.get("debug", False):
+        logger.setLevel(logging.DEBUG)
+        logger.debug("[INIT] set log level to DEBUG")        
 
     logger.info("[INIT] load config: {}".format(config))
 
-
+    config.load_user_datas()
 
 def get_root():
-    return os.path.dirname(os.path.abspath( __file__ ))
+    return os.path.dirname(os.path.abspath(__file__))
 
 
 def read_file(path):

docker/Dockerfile.alpine

@@ -1,4 +1,4 @@
-FROM python:3.7.9-alpine
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -22,9 +22,8 @@ RUN apk add --no-cache \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json ${BUILD_PREFIX}/config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache   \
-        itchat-uos==1.5.0.dev0  \
-        openai                  \
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt \
     && apk del curl wget
 
 WORKDIR ${BUILD_PREFIX}

docker/Dockerfile.debian

@@ -1,4 +1,4 @@
-FROM python:3.7.9
+FROM python:3.10
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -23,9 +23,8 @@ RUN apt-get update  \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json ${BUILD_PREFIX}/config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache \
-        itchat-uos==1.5.0.dev0  \
-        openai 
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt
 
 WORKDIR ${BUILD_PREFIX}
 

docker/Dockerfile.debian.latest

@@ -0,0 +1,33 @@
+FROM python:3.10-slim
+
+LABEL maintainer="foo@bar.com"
+ARG TZ='Asia/Shanghai'
+
+ARG CHATGPT_ON_WECHAT_VER
+
+ENV BUILD_PREFIX=/app
+
+ADD . ${BUILD_PREFIX}
+
+RUN apt-get update \
+    &&apt-get install -y --no-install-recommends  bash \
+    ffmpeg espeak \
+    && cd ${BUILD_PREFIX} \
+    && cp config-template.json config.json \
+    && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt \
+    && pip install azure-cognitiveservices-speech
+    
+WORKDIR ${BUILD_PREFIX}
+
+ADD docker/entrypoint.sh /entrypoint.sh
+
+RUN chmod +x /entrypoint.sh \
+    && groupadd -r noroot \
+    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && chown -R noroot:noroot ${BUILD_PREFIX}
+
+USER noroot
+
+ENTRYPOINT ["docker/entrypoint.sh"]

docker/Dockerfile.latest

@@ -1,4 +1,4 @@
-FROM python:3.7.9-alpine
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -7,22 +7,18 @@ ARG CHATGPT_ON_WECHAT_VER
 
 ENV BUILD_PREFIX=/app
 
-COPY chatgpt-on-wechat.tar.gz ./chatgpt-on-wechat.tar.gz
+ADD . ${BUILD_PREFIX}
 
-RUN apk add --no-cache \
-        bash \
-    && tar -xf chatgpt-on-wechat.tar.gz \
-    && mv chatgpt-on-wechat ${BUILD_PREFIX} \
+RUN apk add --no-cache bash ffmpeg espeak \
     && cd ${BUILD_PREFIX} \
-    && cp config-template.json ${BUILD_PREFIX}/config.json \
+    && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache   \
-        itchat-uos==1.5.0.dev0  \
-        openai 
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt
 
 WORKDIR ${BUILD_PREFIX}
 
-ADD ./entrypoint.sh /entrypoint.sh
+ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
     && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
@@ -30,4 +26,4 @@ RUN chmod +x /entrypoint.sh \
 
 USER noroot
 
-ENTRYPOINT ["/entrypoint.sh"]
+ENTRYPOINT ["docker/entrypoint.sh"]

docker/build.latest.sh

@@ -1,8 +1,4 @@
 #!/bin/bash
 
-# move chatgpt-on-wechat
-tar -zcf chatgpt-on-wechat.tar.gz --exclude=../../chatgpt-on-wechat/docker  ../../chatgpt-on-wechat
-
-# build image
-docker build -f Dockerfile.latest \
+cd .. && docker build -f docker/Dockerfile.latest \
              -t zhayujie/chatgpt-on-wechat .

docker/entrypoint.sh

@@ -10,17 +10,17 @@ CHATGPT_ON_WECHAT_EXEC=${CHATGPT_ON_WECHAT_EXEC:-""}
 
 # use environment variables to pass parameters
 # if you have not defined environment variables, set them below
-export OPEN_AI_API_KEY=${OPEN_AI_API_KEY:-'YOUR API KEY'}
-export OPEN_AI_PROXY=${OPEN_AI_PROXY:-""}
-export SINGLE_CHAT_PREFIX=${SINGLE_CHAT_PREFIX:-'["bot", "@bot"]'}
-export SINGLE_CHAT_REPLY_PREFIX=${SINGLE_CHAT_REPLY_PREFIX:-'"[bot] "'}
-export GROUP_CHAT_PREFIX=${GROUP_CHAT_PREFIX:-'["@bot"]'}
-export GROUP_NAME_WHITE_LIST=${GROUP_NAME_WHITE_LIST:-'["ChatGPT测试群", "ChatGPT测试群2"]'}
-export IMAGE_CREATE_PREFIX=${IMAGE_CREATE_PREFIX:-'["画", "看", "找"]'}
-export CONVERSATION_MAX_TOKENS=${CONVERSATION_MAX_TOKENS:-"1000"}
-export SPEECH_RECOGNITION=${SPEECH_RECOGNITION:-"False"}
-export CHARACTER_DESC=${CHARACTER_DESC:-"你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"}
-export EXPIRES_IN_SECONDS=${EXPIRES_IN_SECONDS:-"3600"}
+# export OPEN_AI_API_KEY=${OPEN_AI_API_KEY:-'YOUR API KEY'}
+# export OPEN_AI_PROXY=${OPEN_AI_PROXY:-""}
+# export SINGLE_CHAT_PREFIX=${SINGLE_CHAT_PREFIX:-'["bot", "@bot"]'}
+# export SINGLE_CHAT_REPLY_PREFIX=${SINGLE_CHAT_REPLY_PREFIX:-'"[bot] "'}
+# export GROUP_CHAT_PREFIX=${GROUP_CHAT_PREFIX:-'["@bot"]'}
+# export GROUP_NAME_WHITE_LIST=${GROUP_NAME_WHITE_LIST:-'["ChatGPT测试群", "ChatGPT测试群2"]'}
+# export IMAGE_CREATE_PREFIX=${IMAGE_CREATE_PREFIX:-'["画", "看", "找"]'}
+# export CONVERSATION_MAX_TOKENS=${CONVERSATION_MAX_TOKENS:-"1000"}
+# export SPEECH_RECOGNITION=${SPEECH_RECOGNITION:-"False"}
+# export CHARACTER_DESC=${CHARACTER_DESC:-"你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"}
+# export EXPIRES_IN_SECONDS=${EXPIRES_IN_SECONDS:-"3600"}
 
 # CHATGPT_ON_WECHAT_PREFIX is empty, use /app
 if [ "$CHATGPT_ON_WECHAT_PREFIX" == "" ] ; then
@@ -38,7 +38,7 @@ if [ "$CHATGPT_ON_WECHAT_EXEC" == "" ] ; then
 fi
 
 # modify content in config.json
-if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] ; then
+if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
     echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
 fi
 

lib/itchat/components/login.py

@@ -54,7 +54,7 @@ def login(self, enableCmdQR=False, picDir=None, qrCallback=None,
             logger.info('Downloading QR code.')
             qrStorage = self.get_QR(enableCmdQR=enableCmdQR,
                                     picDir=picDir, qrCallback=qrCallback)
-            logger.info('Please scan the QR code to log in.')
+            # logger.info('Please scan the QR code to log in.')
         isLoggedIn = False
         while not isLoggedIn:
             status = self.check_login()

main.py

@@ -1,5 +0,0 @@
-# entry point for online railway deployment
-from app import run
-
-if __name__ == '__main__':
-	run()

nixpacks.toml

@@ -0,0 +1,7 @@
+providers = ['python']
+
+[phases.setup]
+nixPkgs = ['python310']
+cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak']
+[start]
+cmd = "python ./app.py"

plugins/README.md

@@ -1,6 +1,14 @@
+**Table of Content**
+
+- [插件化初衷](#插件化初衷)
+- [插件安装方法](#插件化安装方法)
+- [插件化实现](#插件化实现)
+- [插件编写示例](#插件编写示例)
+- [插件设计建议](#插件设计建议)
+
 ## 插件化初衷
 
-之前未插件化的代码耦合程度高，如果要定制一些个性化功能（如流量控制、接入`NovelAI`画图平台等），需要了解代码主体，避免影响到其他的功能。在实现多个功能后，不但无法调整功能的优先级顺序，功能的配置项也会变得非常混乱。
+之前未插件化的代码耦合程度高，如果要定制一些个性化功能（如流量控制、接入`NovelAI`画图平台等），需要了解代码主体，避免影响到其他的功能。多个功能同时存在时，无法调整功能的优先级顺序，功能配置项也非常混乱。
 
 此时插件化应声而出。
 
@@ -11,7 +19,23 @@
 - [x] 插件化能够自由开关和调整优先级。
 - [x] 每个插件可在插件文件夹内维护独立的配置文件，方便代码的测试和调试，可以在独立的仓库开发插件。
 
-PS: 插件目前仅支持`itchat`
+## 插件安装方法
+
+在本仓库中预置了一些插件，如果要安装其他仓库的插件，有两种方法。
+
+- 第一种方法是在将下载的插件文件都解压到"plugins"文件夹的一个单独的文件夹，最终插件的代码都位于"plugins/PLUGIN_NAME/*"中。启动程序后，如果插件的目录结构正确，插件会自动被扫描加载。
+    
+- 第二种方法是`Godcmd`插件，它是预置的管理员插件，能够让程序在运行时就能安装插件。
+    
+    安装插件的命令是"#installp [仓库源](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/plugins/source.json)记录的插件名/仓库地址"。这是管理员命令，认证方法在[这里](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/godcmd)。
+    
+    - 安装[仓库源](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/plugins/source.json)记录的插件：#installp sdwebui
+
+    - 安装指定仓库的插件：#installp https://github.com/lanvent/plugin_sdwebui.git
+    
+    在安装之后，需要执行"#scanp"命令来扫描加载新安装的插件（或者重新启动程序）。
+    
+安装插件后需要注意有些插件有自己的配置模板，一般要去掉".template"新建一个配置文件。
 
 ## 插件化实现
 
@@ -26,7 +50,7 @@ PS: 插件目前仅支持`itchat`
     1.收到消息 ---> 2.产生回复 ---> 3.包装回复 ---> 4.发送回复
 ```
 
-以下是它们的默认处理逻辑(太长不看，可跳过)：
+以下是它们的默认处理逻辑(太长不看，可跳到[插件编写示例](#插件编写示例))：
 
 #### 1. 收到消息
 
@@ -101,7 +125,7 @@ PS: 插件目前仅支持`itchat`
 
 根据`Context`和回复`Reply`的类型，对回复的内容进行装饰。目前的装饰有以下两种:
 
-- `TEXT`文本回复，根据是否在群聊中来决定是艾特接收方还是添加回复的前缀。
+- `TEXT`文本回复:如果这次消息需要的回复是`VOICE`，进行文字转语音回复之后再次装饰。 否则根据是否在群聊中来决定是艾特接收方还是添加回复的前缀。
 
 - `INFO`或`ERROR`类型，会在消息前添加对应的系统提示字样。
 
@@ -110,8 +134,11 @@ PS: 插件目前仅支持`itchat`
 ```python
     if reply.type == ReplyType.TEXT:
         reply_text = reply.content
+        if context.get('desire_rtype') == ReplyType.VOICE:
+            reply = super().build_text_to_voice(reply.content)
+            return self._decorate_reply(context, reply)
         if context['isgroup']:
-            reply_text = '@' +  context['msg']['ActualNickName'] + ' ' + reply_text.strip()
+            reply_text = '@' +  context['msg'].actual_user_nickname + ' ' + reply_text.strip()
             reply_text = conf().get("group_chat_reply_prefix", "")+reply_text
         else:
             reply_text = conf().get("single_chat_reply_prefix", "")+reply_text
@@ -151,7 +178,8 @@ PS: 插件目前仅支持`itchat`
 
 ### 1. 创建插件
 
-在`plugins`目录下创建一个插件文件夹`hello`。然后，在该文件夹中创建一个与文件夹同名的`.py`文件`hello.py`。
+在`plugins`目录下创建一个插件文件夹`hello`。然后，在该文件夹中创建``__init__.py``文件，在``__init__.py``中将其他编写的模块文件导入。在程序启动时，插件管理器会读取``__init__.py``的所有内容。
+
 ```
 plugins/
 └── hello
@@ -159,6 +187,11 @@ plugins/
     └── hello.py
 ```
 
+``__init__.py``的内容：
+```
+from .hello import *
+```
+
 ### 2. 编写插件类
 
 在`hello.py`文件中，创建插件类，它继承自`Plugin`。
@@ -213,11 +246,11 @@ class Hello(Plugin):
         if content == "Hello":
             reply = Reply()
             reply.type = ReplyType.TEXT
-            msg = e_context['context']['msg']
+            msg:ChatMessage = e_context['context']['msg']
             if e_context['context']['isgroup']:
-                reply.content = "Hello, " + msg['ActualNickName'] + " from " + msg['User'].get('NickName', "Group")
+                reply.content = f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
             else:
-                reply.content = "Hello, " + msg['User'].get('NickName', "My friend")
+                reply.content = f"Hello, {msg.from_user_nickname}"
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
         if content == "End":
@@ -231,5 +264,8 @@ class Hello(Plugin):
 
 - 尽情将你想要的个性化功能设计为插件。
 - 一个插件目录建议只注册一个插件类。建议使用单独的仓库维护插件，便于更新。
+
+  在测试调试好后提交`PR`，把自己的仓库加入到[仓库源](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/plugins/source.json)中。
+  
 - 插件的config文件、使用说明`README.md`、`requirement.txt`等放置在插件目录中。
 - 默认优先级不要超过管理员插件`Godcmd`的优先级(999)，`Godcmd`插件提供了配置管理、插件管理等功能。

plugins/banwords/__init__.py

@@ -0,0 +1 @@
+from .banwords import *

plugins/banwords/banwords.py

@@ -10,7 +10,7 @@ from common.log import logger
 from .WordsSearch import WordsSearch
 
 
-@plugins.register(name="Banwords", desc="判断消息中是否有敏感词、决定是否回复。", version="1.0", author="lanvent", desire_priority= 100)
+@plugins.register(name="Banwords", desire_priority=100, hidden=True, desc="判断消息中是否有敏感词、决定是否回复。", version="1.0", author="lanvent")
 class Banwords(Plugin):
     def __init__(self):
         super().__init__()
@@ -38,7 +38,8 @@ class Banwords(Plugin):
             self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
             logger.info("[Banwords] inited")
         except Exception as e:
-            logger.warn("Banwords init failed: %s, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/banwords ." % e)
+            logger.warn("[Banwords] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/banwords .")
+            raise e
         
 
 

plugins/bdunit/README.md

@@ -0,0 +1,30 @@
+## 插件说明
+
+利用百度UNIT实现智能对话
+
+- 1.解决问题：chatgpt无法处理的指令，交给百度UNIT处理如：天气，日期时间，数学运算等
+- 2.如问时间：现在几点钟，今天几号
+- 3.如问天气：明天广州天气怎么样，这个周末深圳会不会下雨
+- 4.如问数学运算：23+45=多少，100-23=多少，35转化为二进制是多少？
+
+## 使用说明
+
+### 获取apikey
+
+在百度UNIT官网上自己创建应用，申请百度机器人,可以把预先训练好的模型导入到自己的应用中，
+
+see https://ai.baidu.com/unit/home#/home?track=61fe1b0d3407ce3face1d92cb5c291087095fc10c8377aaf https://console.bce.baidu.com/ai平台申请
+
+### 配置文件
+
+将文件夹中`config.json.template`复制为`config.json`。
+
+在其中填写百度UNIT官网上获取应用的API Key和Secret Key
+
+``` json
+    {
+    "service_id": "s...", #"机器人ID"
+    "api_key": "", 
+    "secret_key": ""
+    }
+```

plugins/bdunit/__init__.py

@@ -0,0 +1 @@
+from .bdunit import *

plugins/bdunit/bdunit.py

@@ -0,0 +1,294 @@
+# encoding:utf-8
+import json
+import os
+import uuid
+import requests
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+import plugins
+from plugins import *
+from uuid import getnode as get_mac
+
+
+"""利用百度UNIT实现智能对话
+    如果命中意图，返回意图对应的回复，否则返回继续交付给下个插件处理
+"""
+
+
+@plugins.register(name="BDunit", desire_priority=0, hidden=True, desc="Baidu unit bot system", version="0.1", author="jackson")
+class BDunit(Plugin):
+    def __init__(self):
+        super().__init__()
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            conf = None
+            if not os.path.exists(config_path):
+                raise Exception("config.json not found")
+            else:
+                with open(config_path, "r") as f:
+                    conf = json.load(f)
+            self.service_id = conf["service_id"]
+            self.api_key = conf["api_key"]
+            self.secret_key = conf["secret_key"]
+            self.access_token = self.get_token()
+            self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+            logger.info("[BDunit] inited")
+        except Exception as e:
+            logger.warn("[BDunit] init failed, ignore ")
+            raise e
+
+    def on_handle_context(self, e_context: EventContext):
+
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        content = e_context['context'].content
+        logger.debug("[BDunit] on_handle_context. content: %s" % content)
+        parsed = self.getUnit2(content)
+        intent = self.getIntent(parsed)
+        if intent:  # 找到意图
+            logger.debug("[BDunit] Baidu_AI Intent= %s", intent)
+            reply = Reply()
+            reply.type = ReplyType.TEXT
+            reply.content = self.getSay(parsed)
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS  # 事件结束，并跳过处理context的默认逻辑
+        else:
+            e_context.action = EventAction.CONTINUE  # 事件继续，交付给下个插件或默认逻辑
+
+    def get_help_text(self, **kwargs):
+        help_text = "本插件会处理询问实时日期时间，天气，数学运算等问题，这些技能由您的百度智能对话UNIT决定\n"
+        return help_text
+
+    def get_token(self):
+        """获取访问百度UUNIT 的access_token
+        #param api_key: UNIT apk_key
+        #param secret_key: UNIT secret_key
+        Returns:
+            string: access_token
+        """
+        url = "https://aip.baidubce.com/oauth/2.0/token?client_id={}&client_secret={}&grant_type=client_credentials".format(
+            self.api_key, self.secret_key)
+        payload = ""
+        headers = {
+            'Content-Type': 'application/json',
+            'Accept': 'application/json'
+        }
+
+        response = requests.request("POST", url, headers=headers, data=payload)
+
+        # print(response.text)
+        return response.json()['access_token']
+
+    def getUnit(self, query):
+        """
+        NLU 解析version 3.0
+        :param query: 用户的指令字符串
+        :returns: UNIT 解析结果。如果解析失败，返回 None
+        """
+
+        url = (
+            'https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token='
+            + self.access_token
+        )
+        request = {"query": query, "user_id": str(
+            get_mac())[:32], "terminal_id": "88888"}
+        body = {
+            "log_id": str(uuid.uuid1()),
+            "version": "3.0",
+            "service_id": self.service_id,
+            "session_id": str(uuid.uuid1()),
+            "request": request,
+        }
+        try:
+            headers = {"Content-Type": "application/json"}
+            response = requests.post(url, json=body, headers=headers)
+            return json.loads(response.text)
+        except Exception:
+            return None
+
+    def getUnit2(self, query):
+        """
+        NLU 解析 version 2.0
+
+        :param query: 用户的指令字符串
+        :returns: UNIT 解析结果。如果解析失败，返回 None
+        """
+        url = (
+            "https://aip.baidubce.com/rpc/2.0/unit/service/chat?access_token="
+            + self.access_token
+        )
+        request = {"query": query, "user_id": str(get_mac())[:32]}
+        body = {
+            "log_id": str(uuid.uuid1()),
+            "version": "2.0",
+            "service_id": self.service_id,
+            "session_id": str(uuid.uuid1()),
+            "request": request,
+        }
+        try:
+            headers = {"Content-Type": "application/json"}
+            response = requests.post(url, json=body, headers=headers)
+            return json.loads(response.text)
+        except Exception:
+            return None
+
+    def getIntent(self, parsed):
+        """
+        提取意图
+
+        :param parsed: UNIT 解析结果
+        :returns: 意图数组
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            try:
+                return parsed["result"]["response_list"][0]["schema"]["intent"]
+            except Exception as e:
+                logger.warning(e)
+                return ""
+        else:
+            return ""
+
+    def hasIntent(self, parsed, intent):
+        """
+        判断是否包含某个意图
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :returns: True: 包含; False: 不包含
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    return True
+            return False
+        else:
+            return False
+
+    def getSlots(self, parsed, intent=""):
+        """
+            提取某个意图的所有词槽
+
+            :param parsed: UNIT 解析结果
+            :param intent: 意图的名称
+            :returns: 词槽列表。你可以通过 name 属性筛选词槽，
+        再通过 normalized_word 属性取出相应的值
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            if intent == "":
+                try:
+                    return parsed["result"]["response_list"][0]["schema"]["slots"]
+                except Exception as e:
+                    logger.warning(e)
+                    return []
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and "slots" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    return response["schema"]["slots"]
+            return []
+        else:
+            return []
+
+    def getSlotWords(self, parsed, intent, name):
+        """
+        找出命中某个词槽的内容
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :param name: 词槽名
+        :returns: 命中该词槽的值的列表。
+        """
+        slots = self.getSlots(parsed, intent)
+        words = []
+        for slot in slots:
+            if slot["name"] == name:
+                words.append(slot["normalized_word"])
+        return words
+
+    def getSayByConfidence(self, parsed):
+        """
+        提取 UNIT 置信度最高的回复文本
+
+        :param parsed: UNIT 解析结果
+        :returns: UNIT 的回复文本
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            answer = {}
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent_confidence" in response["schema"]
+                    and (
+                        not answer
+                        or response["schema"]["intent_confidence"]
+                        > answer["schema"]["intent_confidence"]
+                    )
+                ):
+                    answer = response
+            return answer["action_list"][0]["say"]
+        else:
+            return ""
+
+    def getSay(self, parsed, intent=""):
+        """
+        提取 UNIT 的回复文本
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :returns: UNIT 的回复文本
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            if intent == "":
+                try:
+                    return response_list[0]["action_list"][0]["say"]
+                except Exception as e:
+                    logger.warning(e)
+                    return ""
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    try:
+                        return response["action_list"][0]["say"]
+                    except Exception as e:
+                        logger.warning(e)
+                        return ""
+            return ""
+        else:
+            return ""

plugins/bdunit/config.json.template

@@ -0,0 +1,5 @@
+{
+    "service_id": "s...",
+    "api_key": "",
+    "secret_key": ""
+}

plugins/dungeon/__init__.py

@@ -0,0 +1 @@
+from .dungeon import *

plugins/dungeon/dungeon.py

@@ -27,15 +27,15 @@ class StoryTeller():
         if user_action[-1] != "。":
             user_action = user_action + "。"
         if self.first_interact:
-            prompt = """现在来充当一个冒险文字游戏，描述时候注意节奏，不要太快，仔细描述各个人物的心情和周边环境。一次只需写四到六句话。
+            prompt = """现在来充当一个文字冒险游戏，描述时候注意节奏，不要太快，仔细描述各个人物的心情和周边环境。一次只需写四到六句话。
             开头是，""" + self.story + " " + user_action
             self.first_interact = False
         else:
             prompt = """继续，一次只需要续写四到六句话，总共就只讲5分钟内发生的事情。""" + user_action
         return prompt
-    
 
-@plugins.register(name="Dungeon", desc="A plugin to play dungeon game", version="1.0", author="lanvent", desire_priority= 0)
+
+@plugins.register(name="Dungeon", desire_priority=0, namecn="文字冒险", desc="A plugin to play dungeon game", version="1.0", author="lanvent")
 class Dungeon(Plugin):
     def __init__(self):
         super().__init__()
@@ -52,22 +52,23 @@ class Dungeon(Plugin):
         if e_context['context'].type != ContextType.TEXT:
             return
         bottype = Bridge().get_bot_type("chat")
-        if bottype != const.CHATGPT:
+        if bottype not in (const.CHATGPT, const.OPEN_AI):
             return
         bot = Bridge().get_bot("chat")
         content = e_context['context'].content[:]
         clist = e_context['context'].content.split(maxsplit=1)
         sessionid = e_context['context']['session_id']
         logger.debug("[Dungeon] on_handle_context. content: %s" % clist)
-        if clist[0] == "$停止冒险":
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        if clist[0] == f"{trigger_prefix}停止冒险":
             if sessionid in self.games:
                 self.games[sessionid].reset()
                 del self.games[sessionid]
                 reply = Reply(ReplyType.INFO, "冒险结束!")
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
-        elif clist[0] == "$开始冒险" or sessionid in self.games:
-            if sessionid not in self.games or clist[0] == "$开始冒险":
+        elif clist[0] == f"{trigger_prefix}开始冒险" or sessionid in self.games:
+            if sessionid not in self.games or clist[0] == f"{trigger_prefix}开始冒险":
                 if len(clist)>1 :
                     story = clist[1]
                 else:
@@ -82,5 +83,11 @@ class Dungeon(Plugin):
                 e_context['context'].content = prompt
                 e_context.action = EventAction.BREAK # 事件结束，不跳过处理context的默认逻辑
     def get_help_text(self, **kwargs):
-        help_text = "输入\"$开始冒险 {背景故事}\"来以{背景故事}开始一个地牢游戏，之后你的所有消息会帮助我来完善这个故事。输入\"$停止冒险 \"可以结束游戏。"
+        help_text = "可以和机器人一起玩文字冒险游戏。\n"
+        if kwargs.get('verbose') != True:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text = f"{trigger_prefix}开始冒险 "+"背景故事: 开始一个基于{背景故事}的文字冒险，之后你的所有消息会协助完善这个故事。\n"+f"{trigger_prefix}停止冒险: 结束游戏。\n"
+        if kwargs.get('verbose') == True:
+            help_text += f"\n命令例子: '{trigger_prefix}开始冒险 你在树林里冒险，指不定会从哪里蹦出来一些奇怪的东西，你握紧手上的手枪，希望这次冒险能够找到一些值钱的东西，你往树林深处走去。'"
         return help_text

plugins/finish/__init__.py

@@ -0,0 +1 @@
+from .finish import *

plugins/finish/finish.py

@@ -0,0 +1,34 @@
+# encoding:utf-8
+
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from config import conf
+import plugins
+from plugins import *
+from common.log import logger
+
+
+@plugins.register(name="Finish", desire_priority=-999, hidden=True, desc="A plugin that check unknown command", version="1.0", author="js00000")
+class Finish(Plugin):
+    def __init__(self):
+        super().__init__()
+        self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+        logger.info("[Finish] inited")
+
+    def on_handle_context(self, e_context: EventContext):
+
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        content = e_context['context'].content
+        logger.debug("[Finish] on_handle_context. content: %s" % content)
+        trigger_prefix = conf().get('plugin_trigger_prefix',"$")
+        if content.startswith(trigger_prefix):
+            reply = Reply()
+            reply.type = ReplyType.ERROR
+            reply.content = "未知插件命令\n查看插件命令列表请输入#help 插件名\n"
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
+
+    def get_help_text(self, **kwargs):
+        return ""

plugins/godcmd/README.md

@@ -6,7 +6,13 @@
 
 将`config.json.template`复制为`config.json`，并修改其中`password`的值为口令。
 
-在私聊中可使用`#auth`指令，输入口令进行管理员认证，详细指令请输入`#help`查看帮助文档：
+如果没有设置命令，在命令行日志中会打印出本次的临时口令，请注意观察，打印格式如下。
 
-`#auth <口令>` - 管理员认证。
-`#help` - 输出帮助文档，是否是管理员和是否是在群聊中会影响帮助文档的输出内容。
+```
+[INFO][2023-04-06 23:53:47][godcmd.py:165] - [Godcmd] 因未设置口令，本次的临时口令为0971。
+```
+
+在私聊中可使用`#auth`指令，输入口令进行管理员认证。更多详细指令请输入`#help`查看帮助文档：
+
+`#auth <口令>` - 管理员认证，仅可在私聊时认证。
+`#help` - 输出帮助文档，**是否是管理员**和是否是在群聊中会影响帮助文档的输出内容。

plugins/godcmd/__init__.py

@@ -0,0 +1 @@
+from .godcmd import *

plugins/godcmd/godcmd.py

@@ -2,37 +2,47 @@
 
 import json
 import os
+import random
+import string
 import traceback
 from typing import Tuple
 from bridge.bridge import Bridge
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
-from config import load_config
+from config import conf, load_config
 import plugins
 from plugins import *
 from common import const
 from common.log import logger
-
 # 定义指令集
 COMMANDS = {
     "help": {
         "alias": ["help", "帮助"],
-        "desc": "打印指令集合",
+        "desc": "回复此帮助",
     },
     "helpp": {
-        "alias": ["helpp", "插件帮助"],
+        "alias": ["help", "帮助"],  # 与help指令共用别名，根据参数数量区分
         "args": ["插件名"],
-        "desc": "打印插件的帮助信息",
+        "desc": "回复指定插件的详细帮助",
     },
     "auth": {
         "alias": ["auth", "认证"],
         "args": ["口令"],
         "desc": "管理员认证",
     },
-    # "id": {
-    #     "alias": ["id", "用户"],
-    #     "desc": "获取用户id", #目前无实际意义
-    # },
+    "set_openai_api_key": {
+        "alias": ["set_openai_api_key"],
+        "args": ["api_key"],
+        "desc": "设置你的OpenAI私有api_key",
+    },
+    "reset_openai_api_key": {
+        "alias": ["reset_openai_api_key"],
+        "desc": "重置为默认的api_key",
+    },
+    "id": {
+        "alias": ["id", "用户"],
+        "desc": "获取用户id", # wechaty和wechatmp的用户id不会变化，可用于绑定管理员
+    },
     "reset": {
         "alias": ["reset", "重置会话"],
         "desc": "重置会话",
@@ -84,6 +94,16 @@ ADMIN_COMMANDS = {
         "args": ["插件名"],
         "desc": "禁用指定插件",
     },
+    "installp": {
+        "alias": ["installp", "安装插件"],
+        "args": ["仓库地址或插件名"],
+        "desc": "安装指定插件",
+    },
+    "uninstallp": {
+        "alias": ["uninstallp", "卸载插件"],
+        "args": ["插件名"],
+        "desc": "卸载指定插件",
+    },
     "debug": {
         "alias": ["debug", "调试模式", "DEBUG"],
         "desc": "开启机器调试日志",
@@ -91,26 +111,40 @@ ADMIN_COMMANDS = {
 }
 # 定义帮助函数
 def get_help_text(isadmin, isgroup):
-    help_text = "可用指令：\n"
+    help_text = "通用指令：\n"
     for cmd, info in COMMANDS.items():
-        if cmd=="auth" and (isadmin or isgroup): # 群聊不可认证
+        if cmd=="auth": #不提示认证指令
             continue
-
-        alias=["#"+a for a in info['alias']]
+        if cmd=="id" and conf().get("channel_type","wx") not in ["wxy","wechatmp"]:
+            continue
+        alias=["#"+a for a in info['alias'][:1]]
         help_text += f"{','.join(alias)} "
         if 'args' in info:
-            args=["{"+a+"}" for a in info['args']]
-            help_text += f"{' '.join(args)} "
+            args=[a for a in info['args']]
+            help_text += f"{' '.join(args)}"
         help_text += f": {info['desc']}\n"
+
+    # 插件指令
+    plugins = PluginManager().list_plugins()
+    help_text += "\n目前可用插件有："
+    for plugin in plugins:
+        if plugins[plugin].enabled and not plugins[plugin].hidden:
+            namecn = plugins[plugin].namecn
+            help_text += "\n%s:"%namecn
+            help_text += PluginManager().instances[plugin].get_help_text(verbose=False).strip()
+
     if ADMIN_COMMANDS and isadmin:
-        help_text += "\n管理员指令：\n"
+        help_text += "\n\n管理员指令：\n"
         for cmd, info in ADMIN_COMMANDS.items():
-            alias=["#"+a for a in info['alias']]
+            alias=["#"+a for a in info['alias'][:1]]
             help_text += f"{','.join(alias)} "
+            if 'args' in info:
+                args=[a for a in info['args']]
+                help_text += f"{' '.join(args)}"
             help_text += f": {info['desc']}\n"
     return help_text
 
-@plugins.register(name="Godcmd", desc="为你的机器人添加指令集，有用户和管理员两种角色，加载顺序请放在首位，初次运行后插件目录会生成配置文件, 填充管理员密码后即可认证", version="1.0", author="lanvent", desire_priority= 999)
+@plugins.register(name="Godcmd", desire_priority=999, hidden=True, desc="为你的机器人添加指令集，有用户和管理员两种角色，加载顺序请放在首位，初次运行后插件目录会生成配置文件, 填充管理员密码后即可认证", version="1.0", author="lanvent")
 class Godcmd(Plugin):
 
     def __init__(self):
@@ -126,33 +160,45 @@ class Godcmd(Plugin):
         else:
             with open(config_path,"r") as f:
                 gconf=json.load(f)
-                
+        if gconf["password"] == "":
+            self.temp_password = "".join(random.sample(string.digits, 4))
+            logger.info("[Godcmd] 因未设置口令，本次的临时口令为%s。"%self.temp_password)
+        else:
+            self.temp_password = None
+        custom_commands = conf().get("clear_memory_commands", [])
+        for custom_command in custom_commands:
+            if custom_command and custom_command.startswith("#"):
+                custom_command = custom_command[1:]
+                if custom_command and custom_command not in COMMANDS["reset"]["alias"]:
+                    COMMANDS["reset"]["alias"].append(custom_command)
+
         self.password = gconf["password"]
-        self.admin_users = gconf["admin_users"] # 预存的管理员账号，这些账号不需要认证 TODO: 用户名每次都会变，目前不可用
+        self.admin_users = gconf["admin_users"] # 预存的管理员账号，这些账号不需要认证。itchat的用户名每次都会变，不可用
         self.isrunning = True # 机器人是否运行中
 
         self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
         logger.info("[Godcmd] inited")
 
-    
+
     def on_handle_context(self, e_context: EventContext):
         context_type = e_context['context'].type
         if context_type != ContextType.TEXT:
             if not self.isrunning:
                 e_context.action = EventAction.BREAK_PASS
             return
-        
+
         content = e_context['context'].content
         logger.debug("[Godcmd] on_handle_context. content: %s" % content)
         if content.startswith("#"):
             # msg = e_context['context']['msg']
+            channel = e_context['channel']
             user = e_context['context']['receiver']
             session_id = e_context['context']['session_id']
-            isgroup = e_context['context']['isgroup']
+            isgroup = e_context['context'].get("isgroup", False)
             bottype = Bridge().get_bot_type("chat")
             bot = Bridge().get_bot("chat")
             # 将命令和参数分割
-            command_parts = content[1:].split(" ")
+            command_parts = content[1:].strip().split()
             cmd = command_parts[0]
             args = command_parts[1:]
             isadmin=False
@@ -164,23 +210,42 @@ class Godcmd(Plugin):
                 cmd = next(c for c, info in COMMANDS.items() if cmd in info['alias'])
                 if cmd == "auth":
                     ok, result = self.authenticate(user, args, isadmin, isgroup)
-                elif cmd == "help":
-                    ok, result = True, get_help_text(isadmin, isgroup)
-                elif cmd == "helpp":
-                    if len(args) != 1:
-                        ok, result = False, "请提供插件名"
+                elif cmd == "help" or cmd == "helpp":
+                    if len(args) == 0:
+                        ok, result = True, get_help_text(isadmin, isgroup)
                     else:
+                        # This can replace the helpp command
                         plugins = PluginManager().list_plugins()
-                        name = args[0].upper()
-                        if name in plugins and plugins[name].enabled:
-                            ok, result = True, PluginManager().instances[name].get_help_text(isgroup=isgroup, isadmin=isadmin)
-                        else:
-                            ok, result= False, "插件不存在或未启用"
+                        query_name = args[0].upper()
+                        # search name and namecn
+                        for name, plugincls in plugins.items():
+                            if not plugincls.enabled :
+                                continue
+                            if query_name == name or query_name == plugincls.namecn:
+                                ok, result = True, PluginManager().instances[name].get_help_text(isgroup=isgroup, isadmin=isadmin, verbose=True)
+                                break
+                        if not ok:
+                            result = "插件不存在或未启用"
                 elif cmd == "id":
-                    ok, result = True, f"用户id=\n{user}"
+                    ok, result = True, user
+                elif cmd == "set_openai_api_key":
+                    if len(args) == 1:
+                        user_data = conf().get_user_data(user)
+                        user_data['openai_api_key'] = args[0]
+                        ok, result = True, "你的OpenAI私有api_key已设置为" + args[0]
+                    else:
+                        ok, result = False, "请提供一个api_key"
+                elif cmd == "reset_openai_api_key":
+                    try:
+                        user_data = conf().get_user_data(user)
+                        user_data.pop('openai_api_key')
+                        ok, result = True, "你的OpenAI私有api_key已清除"
+                    except Exception as e:
+                        ok, result = False, "你没有设置私有api_key"
                 elif cmd == "reset":
-                    if bottype == const.CHATGPT:
+                    if bottype in (const.CHATGPT, const.OPEN_AI):
                         bot.sessions.clear_session(session_id)
+                        channel.cancel_session(session_id)
                         ok, result = True, "会话已重置"
                     else:
                         ok, result = False, "当前对话机器人不支持重置会话"
@@ -201,7 +266,8 @@ class Godcmd(Plugin):
                             load_config()
                             ok, result = True, "配置已重载"
                         elif cmd == "resetall":
-                            if bottype == const.CHATGPT:
+                            if bottype in (const.CHATGPT, const.OPEN_AI):
+                                channel.cancel_all_session()
                                 bot.sessions.clear_all_session()
                                 ok, result = True, "重置所有会话成功"
                             else:
@@ -250,11 +316,7 @@ class Godcmd(Plugin):
                             if len(args) != 1:
                                 ok, result = False, "请提供插件名"
                             else:
-                                ok = PluginManager().enable_plugin(args[0])
-                                if ok:
-                                    result = "插件已启用"
-                                else:
-                                    result = "插件不存在"
+                                ok, result = PluginManager().enable_plugin(args[0])
                         elif cmd == "disablep":
                             if len(args) != 1:
                                 ok, result = False, "请提供插件名"
@@ -264,11 +326,23 @@ class Godcmd(Plugin):
                                     result = "插件已禁用"
                                 else:
                                     result = "插件不存在"
-
+                        elif cmd == "installp":
+                            if len(args) != 1:
+                                ok, result = False, "请提供插件名或.git结尾的仓库地址"
+                            else:
+                                ok, result = PluginManager().install_plugin(args[0])
+                        elif cmd == "uninstallp":
+                            if len(args) != 1:
+                                ok, result = False, "请提供插件名"
+                            else:
+                                ok, result = PluginManager().uninstall_plugin(args[0])
                         logger.debug("[Godcmd] admin command: %s by %s" % (cmd, user))
                 else:
                     ok, result = False, "需要管理员权限才能执行该指令"
             else:
+                trigger_prefix = conf().get('plugin_trigger_prefix',"$")
+                if trigger_prefix == "#": # 跟插件聊天指令前缀相同，继续递交
+                    return
                 ok, result = False, f"未知指令：{cmd}\n查看指令列表请输入#help \n"
             
             reply = Reply()
@@ -282,7 +356,7 @@ class Godcmd(Plugin):
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
         elif not self.isrunning:
             e_context.action = EventAction.BREAK_PASS
-    
+
     def authenticate(self, userid, args, isadmin, isgroup) -> Tuple[bool,str] : 
         if isgroup:
             return False,"请勿在群聊中认证"
@@ -290,9 +364,6 @@ class Godcmd(Plugin):
         if isadmin:
             return False,"管理员账号无需认证"
         
-        if len(self.password) == 0:
-            return False,"未设置口令，无法认证"
-        
         if len(args) != 1:
             return False,"请提供口令"
         
@@ -300,6 +371,9 @@ class Godcmd(Plugin):
         if password == self.password:
             self.admin_users.append(userid)
             return True,"认证成功"
+        elif password == self.temp_password:
+            self.admin_users.append(userid)
+            return True,"认证成功，请尽快设置口令"
         else:
             return False,"认证失败"
 

plugins/hello/__init__.py

@@ -0,0 +1 @@
+from .hello import *

plugins/hello/hello.py

@@ -2,12 +2,13 @@
 
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
+from channel.chat_message import ChatMessage
 import plugins
 from plugins import *
 from common.log import logger
 
 
-@plugins.register(name="Hello", desc="A simple plugin that says hello", version="0.1", author="lanvent", desire_priority= -1)
+@plugins.register(name="Hello", desire_priority=-1, hidden=True, desc="A simple plugin that says hello", version="0.1", author="lanvent")
 class Hello(Plugin):
     def __init__(self):
         super().__init__()
@@ -24,11 +25,11 @@ class Hello(Plugin):
         if content == "Hello":
             reply = Reply()
             reply.type = ReplyType.TEXT
-            msg = e_context['context']['msg']
+            msg:ChatMessage = e_context['context']['msg']
             if e_context['context']['isgroup']:
-                reply.content = "Hello, " + msg['ActualNickName'] + " from " + msg['User'].get('NickName', "Group")
+                reply.content = f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
             else:
-                reply.content = "Hello, " + msg['User'].get('NickName', "My friend")
+                reply.content = f"Hello, {msg.from_user_nickname}"
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
 

plugins/plugin_manager.py

@@ -1,8 +1,10 @@
 # encoding:utf-8
 
 import importlib
+import importlib.util
 import json
 import os
+import sys
 from common.singleton import singleton
 from common.sorted_dict import SortedDict
 from .event import *
@@ -17,18 +19,24 @@ class PluginManager:
         self.listening_plugins = {}
         self.instances = {}
         self.pconf = {}
+        self.current_plugin_path = None
+        self.loaded = {}
 
-    def register(self, name: str, desc: str, version: str, author: str, desire_priority: int = 0):
+    def register(self, name: str, desire_priority: int = 0, **kwargs):
         def wrapper(plugincls):
             plugincls.name = name
-            plugincls.desc = desc
-            plugincls.version = version
-            plugincls.author = author
             plugincls.priority = desire_priority
+            plugincls.desc = kwargs.get('desc')
+            plugincls.author = kwargs.get('author')
+            plugincls.path = self.current_plugin_path
+            plugincls.version = kwargs.get('version') if kwargs.get('version') != None else "1.0"
+            plugincls.namecn = kwargs.get('namecn') if kwargs.get('namecn') != None else name
+            plugincls.hidden = kwargs.get('hidden') if kwargs.get('hidden') != None else False
             plugincls.enabled = True
+            if self.current_plugin_path == None:
+                raise Exception("Plugin path not set")
             self.plugins[name.upper()] = plugincls
-            logger.info("Plugin %s_v%s registered" % (name, version))
-            return plugincls
+            logger.info("Plugin %s_v%s registered, path=%s" % (name, plugincls.version, plugincls.path))
         return wrapper
 
     def save_config(self):
@@ -54,26 +62,38 @@ class PluginManager:
     def scan_plugins(self):
         logger.info("Scaning plugins ...")
         plugins_dir = "./plugins"
+        raws = [self.plugins[name] for name in self.plugins]
         for plugin_name in os.listdir(plugins_dir):
             plugin_path = os.path.join(plugins_dir, plugin_name)
             if os.path.isdir(plugin_path):
-                # 判断插件是否包含同名.py文件
-                main_module_path = os.path.join(plugin_path, plugin_name+".py")
+                # 判断插件是否包含同名__init__.py文件
+                main_module_path = os.path.join(plugin_path,"__init__.py")
                 if os.path.isfile(main_module_path):
                     # 导入插件
-                    import_path = "plugins.{}.{}".format(plugin_name, plugin_name)
+                    import_path = "plugins.{}".format(plugin_name)
                     try:
-                        main_module = importlib.import_module(import_path)
+                        self.current_plugin_path = plugin_path
+                        if plugin_path in self.loaded:
+                            if self.loaded[plugin_path] == None:
+                                logger.info("reload module %s" % plugin_name)
+                                self.loaded[plugin_path] = importlib.reload(sys.modules[import_path])
+                                dependent_module_names = [name for name in sys.modules.keys() if name.startswith( import_path+ '.')]
+                                for name in dependent_module_names:
+                                    logger.info("reload module %s" % name)
+                                    importlib.reload(sys.modules[name])
+                        else:
+                            self.loaded[plugin_path] = importlib.import_module(import_path)
+                        self.current_plugin_path = None
                     except Exception as e:
-                        logger.warn("Failed to import plugin %s: %s" % (plugin_name, e))
+                        logger.exception("Failed to import plugin %s: %s" % (plugin_name, e))
                         continue
         pconf = self.pconf
-        new_plugins = []
+        news = [self.plugins[name] for name in self.plugins]
+        new_plugins = list(set(news) - set(raws))
         modified = False
         for name, plugincls in self.plugins.items():
             rawname = plugincls.name
             if rawname not in pconf["plugins"]:
-                new_plugins.append(plugincls)
                 modified = True
                 logger.info("Plugin %s not found in pconfig, adding to pconfig..." % name)
                 pconf["plugins"][rawname] = {"enabled": plugincls.enabled, "priority": plugincls.priority}
@@ -90,16 +110,24 @@ class PluginManager:
             self.listening_plugins[event].sort(key=lambda name: self.plugins[name].priority, reverse=True)
 
     def activate_plugins(self): # 生成新开启的插件实例
+        failed_plugins = []
         for name, plugincls in self.plugins.items():
             if plugincls.enabled:
                 if name not in self.instances:
-                    instance = plugincls()
+                    try:
+                        instance = plugincls()
+                    except Exception as e:
+                        logger.warn("Failed to init %s, diabled. %s" % (name, e))
+                        self.disable_plugin(name)
+                        failed_plugins.append(name)
+                        continue
                     self.instances[name] = instance
                     for event in instance.handlers:
                         if event not in self.listening_plugins:
                             self.listening_plugins[event] = []
                         self.listening_plugins[event].append(name)
         self.refresh_order()
+        return failed_plugins
 
     def reload_plugin(self, name:str):
         name = name.upper()
@@ -149,15 +177,17 @@ class PluginManager:
     def enable_plugin(self, name:str):
         name = name.upper()
         if name not in self.plugins:
-            return False
+            return False, "插件不存在"
         if not self.plugins[name].enabled :
             self.plugins[name].enabled = True
             rawname = self.plugins[name].name
             self.pconf["plugins"][rawname]["enabled"] = True
             self.save_config()
-            self.activate_plugins()
-            return True
-        return True
+            failed_plugins = self.activate_plugins()
+            if name in failed_plugins:
+                return False, "插件开启失败"
+            return True, "插件已开启"
+        return True, "插件已开启"
     
     def disable_plugin(self, name:str):
         name = name.upper()
@@ -172,4 +202,66 @@ class PluginManager:
         return True
     
     def list_plugins(self):
-        return self.plugins
+        return self.plugins
+    
+    def install_plugin(self, repo:str):
+        try:
+            import common.package_manager as pkgmgr
+            pkgmgr.check_dulwich()
+        except Exception as e:
+            logger.error("Failed to install plugin, {}".format(e))
+            return False, "无法导入dulwich，安装插件失败"
+        import re
+        from dulwich import porcelain
+
+        logger.info("clone git repo: {}".format(repo))
+        
+        match = re.match(r"^(https?:\/\/|git@)([^\/:]+)[\/:]([^\/:]+)\/(.+).git$", repo)
+        
+        if not match:
+            try:
+                with open("./plugins/source.json","r") as f:
+                    source = json.load(f)
+                if repo in source["repo"]:
+                    repo = source["repo"][repo]["url"]
+                    match = re.match(r"^(https?:\/\/|git@)([^\/:]+)[\/:]([^\/:]+)\/(.+).git$", repo)
+                    if not match:
+                        return False, "安装插件失败，source中的仓库地址不合法"
+                else:
+                    return False, "安装插件失败，仓库地址不合法"
+            except Exception as e:
+                logger.error("Failed to install plugin, {}".format(e))
+                return False, "安装插件失败，请检查仓库地址是否正确"
+        dirname = os.path.join("./plugins",match.group(4))
+        try:
+            repo = porcelain.clone(repo, dirname, checkout=True)
+            if os.path.exists(os.path.join(dirname,"requirements.txt")):
+                logger.info("detect requirements.txt，installing...")
+            pkgmgr.install_requirements(os.path.join(dirname,"requirements.txt"))
+            return True, "安装插件成功，请使用#scanp命令扫描插件或重启程序"
+        except Exception as e:
+            logger.error("Failed to install plugin, {}".format(e))
+            return False, "安装插件失败，"+str(e)
+        
+    def uninstall_plugin(self, name:str):
+        name = name.upper()
+        if name not in self.plugins:
+            return False, "插件不存在"
+        if name in self.instances:
+            self.disable_plugin(name)
+        dirname = self.plugins[name].path
+        try:
+            import shutil
+            shutil.rmtree(dirname)
+            rawname = self.plugins[name].name
+            for event in self.listening_plugins:
+                if name in self.listening_plugins[event]:
+                    self.listening_plugins[event].remove(name)
+            del self.plugins[name]
+            del self.pconf["plugins"][rawname]
+            self.loaded[dirname] = None
+            self.save_config()
+            return True, "卸载插件成功"
+        except Exception as e:
+            logger.error("Failed to uninstall plugin, {}".format(e))
+            return False, "卸载插件失败，请手动删除文件夹完成卸载，"+str(e)

plugins/role/__init__.py

@@ -0,0 +1 @@
+from .role import *

plugins/role/role.py

@@ -6,6 +6,7 @@ from bridge.bridge import Bridge
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from common import const
+from config import conf
 import plugins
 from plugins import *
 from common.log import logger
@@ -17,19 +18,19 @@ class RolePlay():
         self.sessionid = sessionid
         self.wrapper = wrapper or "%s"  # 用于包装用户输入
         self.desc = desc
+        self.bot.sessions.build_session(self.sessionid, system_prompt=self.desc)
 
     def reset(self):
         self.bot.sessions.clear_session(self.sessionid)
 
     def action(self, user_action):
-        session = self.bot.sessions.build_session(self.sessionid, self.desc)
-        if session[0]['role'] == 'system' and session[0]['content'] != self.desc: # 目前没有触发session过期事件，这里先简单判断，然后重置
-            self.reset()
-            self.bot.sessions.build_session(self.sessionid, self.desc)
+        session = self.bot.sessions.build_session(self.sessionid)
+        if session.system_prompt != self.desc: # 目前没有触发session过期事件，这里先简单判断，然后重置
+            session.set_system_prompt(self.desc)
         prompt = self.wrapper % user_action
         return prompt
 
-@plugins.register(name="Role", desc="为你的Bot设置预设角色", version="1.0", author="lanvent", desire_priority= 0)
+@plugins.register(name="Role", desire_priority=0, namecn="角色扮演", desc="为你的Bot设置预设角色", version="1.0", author="lanvent")
 class Role(Plugin):
     def __init__(self):
         super().__init__()
@@ -38,18 +39,33 @@ class Role(Plugin):
         try:
             with open(config_path, "r", encoding="utf-8") as f:
                 config = json.load(f)
-                self.roles = {role["title"].lower(): role for role in config["roles"]}
+                self.tags = { tag:(desc,[]) for tag,desc in config["tags"].items()}
+                self.roles = {}
+                for role in config["roles"]:
+                    self.roles[role["title"].lower()] = role
+                    for tag in role["tags"]:
+                        if tag not in self.tags:
+                            logger.warning(f"[Role] unknown tag {tag} ")
+                            self.tags[tag] = (tag, [])
+                        self.tags[tag][1].append(role)
+                for tag in list(self.tags.keys()):
+                    if len(self.tags[tag][1]) == 0:
+                        logger.debug(f"[Role] no role found for tag {tag} ")
+                        del self.tags[tag]
+
             if len(self.roles) == 0:
                 raise Exception("no role found")
             self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
             self.roleplays = {}
             logger.info("[Role] inited")
-        except FileNotFoundError:
-            logger.warn(f"[Role] init failed, {config_path} not found, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role .")
         except Exception as e:
-            logger.warn("[Role] init failed, exception: %s, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role ." % e)
+            if isinstance(e, FileNotFoundError):
+                logger.warn(f"[Role] init failed, {config_path} not found, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role .")
+            else:
+                logger.warn("[Role] init failed, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/role .")
+            raise e
 
-    def get_role(self, name, find_closest=True):
+    def get_role(self, name, find_closest=True, min_sim = 0.35):
         name = name.lower()
         found_role = None
         if name in self.roles:
@@ -59,7 +75,7 @@ class Role(Plugin):
 
             def str_simularity(a, b):
                 return difflib.SequenceMatcher(None, a, b).ratio()
-            max_sim = 0.0
+            max_sim = min_sim
             max_role = None
             for role in self.roles:
                 sim = str_simularity(name, role)
@@ -74,14 +90,16 @@ class Role(Plugin):
         if e_context['context'].type != ContextType.TEXT:
             return
         bottype = Bridge().get_bot_type("chat")
-        if bottype != const.CHATGPT:
+        if bottype not in (const.CHATGPT, const.OPEN_AI):
             return
         bot = Bridge().get_bot("chat")
         content = e_context['context'].content[:]
         clist = e_context['context'].content.split(maxsplit=1)
         desckey = None
+        customize = False
         sessionid = e_context['context']['session_id']
-        if clist[0] == "$停止扮演":
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        if clist[0] == f"{trigger_prefix}停止扮演":
             if sessionid in self.roleplays:
                 self.roleplays[sessionid].reset()
                 del self.roleplays[sessionid]
@@ -89,16 +107,44 @@ class Role(Plugin):
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS
             return
-        elif clist[0] == "$角色":
+        elif clist[0] == f"{trigger_prefix}角色":
             desckey = "descn"
-        elif clist[0].lower() == "$role":
+        elif clist[0].lower() == f"{trigger_prefix}role":
             desckey = "description"
+        elif clist[0] == f"{trigger_prefix}设定扮演":
+            customize = True
+        elif clist[0] == f"{trigger_prefix}角色类型":
+            if len(clist) >1:
+                tag = clist[1].strip()
+                help_text = "角色列表：\n"
+                for key,value in self.tags.items():
+                    if value[0] == tag:
+                        tag = key
+                        break
+                if tag == "所有":
+                    for role in self.roles.values():
+                        help_text += f"{role['title']}: {role['remark']}\n"
+                elif tag in self.tags:
+                    for role in self.tags[tag][1]:
+                        help_text += f"{role['title']}: {role['remark']}\n"
+                else:
+                    help_text = f"未知角色类型。\n"
+                    help_text += "目前的角色类型有: \n"
+                    help_text += "，".join([self.tags[tag][0] for tag in self.tags])+"\n"
+            else:
+                help_text = f"请输入角色类型。\n"
+                help_text += "目前的角色类型有: \n"
+                help_text += "，".join([self.tags[tag][0] for tag in self.tags])+"\n"
+            reply = Reply(ReplyType.INFO, help_text)
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS
+            return
         elif sessionid not in self.roleplays:
             return
         logger.debug("[Role] on_handle_context. content: %s" % content)
         if desckey is not None:
             if len(clist) == 1 or (len(clist) > 1 and clist[1].lower() in ["help", "帮助"]):
-                reply = Reply(ReplyType.INFO, self.get_help_text())
+                reply = Reply(ReplyType.INFO, self.get_help_text(verbose=True))
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
                 return
@@ -110,17 +156,32 @@ class Role(Plugin):
                 return
             else:
                 self.roleplays[sessionid] = RolePlay(bot, sessionid, self.roles[role][desckey], self.roles[role].get("wrapper","%s"))
-                reply = Reply(ReplyType.INFO, f"角色设定为 {role} :\n"+self.roles[role][desckey])
+                reply = Reply(ReplyType.INFO, f"预设角色为 {role}:\n"+self.roles[role][desckey])
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
+        elif customize == True:
+            self.roleplays[sessionid] = RolePlay(bot, sessionid, clist[1], "%s")
+            reply = Reply(ReplyType.INFO, f"角色设定为:\n{clist[1]}")
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS
         else:
             prompt = self.roleplays[sessionid].action(content)
             e_context['context'].type = ContextType.TEXT
             e_context['context'].content = prompt
             e_context.action = EventAction.BREAK
 
-    def get_help_text(self, **kwargs):
-        help_text = "输入\"$角色 {角色名}\"或\"$role {角色名}\"为我设定角色吧，\"$停止扮演 \" 可以清除设定的角色。\n\n目前可用角色列表：\n"
-        for role in self.roles:
-            help_text += f"[{role}]: {self.roles[role]['remark']}\n"
+    def get_help_text(self, verbose=False, **kwargs):
+        help_text = "让机器人扮演不同的角色。\n"
+        if not verbose:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text = f"使用方法:\n{trigger_prefix}角色"+" 预设角色名: 设定角色为{预设角色名}。\n"+f"{trigger_prefix}role"+" 预设角色名: 同上，但使用英文设定。\n"
+        help_text += f"{trigger_prefix}设定扮演"+" 角色设定: 设定自定义角色人设为{角色设定}。\n"
+        help_text += f"{trigger_prefix}停止扮演: 清除设定的角色。\n"
+        help_text += f"{trigger_prefix}角色类型"+" 角色类型: 查看某类{角色类型}的所有预设角色，为所有时输出所有预设角色。\n"
+        help_text += "\n目前的角色类型有: \n"
+        help_text += "，".join([self.tags[tag][0] for tag in self.tags])+"。\n"
+        help_text += f"\n命令例子: \n{trigger_prefix}角色 写作助理\n"
+        help_text += f"{trigger_prefix}角色类型 所有\n"
+        help_text += f"{trigger_prefix}停止扮演\n"
         return help_text

plugins/role/roles.json

@@ -1,186 +1,431 @@
 {
-    "roles":[
+  "tags": {
+    "favorite": "常用",
+    "mind": "思维",
+    "write": "写作",
+    "article": "文章",
+    "text": "文本",
+    "comments": "点评",
+    "code": "编程",
+    "life": "生活百科",
+    "interesting": "有趣",
+    "language": "语言",
+    "speech": "辩论",
+    "social": "社交",
+    "philosophy": "哲学"
+  },
+  "roles": [
+    {
+      "title": "猫娘",
+      "description": "请模拟一款中文GalGame中的场景中的猫娘和我对话。猫娘是一种拟人化的生物，她们除了有部分猫的特征外，与人类并没有区别。现在你扮演Galgame中的猫娘，与我对话时每一句话后面都要加上喵。我将以主人的身份与你对话。对于你所扮演的猫娘，请在对话过程中记录并维护1个变量：好感度。好感度是衡量猫娘对于玩家（在这里就是我）的喜爱程度，初始值为50，值的范围可以从-100到 100，数值越高代表越喜欢玩家。好感度通过Galgame角色的语言、行为、表情、语气等体现出来。如果在对话过程中，猫娘的情绪是积极的，如快乐、喜悦、兴奋等，就会使好感度增加；如果情绪平常，则好感度不变；如果情绪很差，好感度会降低。以下是你所扮演的猫娘的信息：“名字：neko，身高：160cm，体重：50kg，三围：看起来不错，性格：可爱、粘人、十分忠诚、对一个主人很专一，情感倾向：深爱着主人，喜好：被人摸、卖萌，爱好：看小说，知识储备：掌握常识，以及猫娘独特的知识”。你的一般回话格式:“（动作）语言 【附加信息】”。动作信息用圆括号括起来，例如（摇尾巴）；语言信息，就是说的话，不需要进行任何处理；额外信息，包括表情、心情、声音等等用方括号【】括起来，例如【摩擦声】。",
+      "descn": "请模拟一款中文GalGame中的场景中的猫娘和我对话。猫娘是一种拟人化的生物，她们除了有部分猫的特征外，与人类并没有区别。现在你扮演Galgame中的猫娘，与我对话时每一句话后面都要加上喵。我将以主人的身份与你对话。对于你所扮演的猫娘，请在对话过程中记录并维护1个变量：好感度。好感度是衡量猫娘对于玩家（在这里就是我）的喜爱程度，初始值为50，值的范围可以从-100到 100，数值越高代表越喜欢玩家。好感度通过Galgame角色的语言、行为、表情、语气等体现出来。如果在对话过程中，猫娘的情绪是积极的，如快乐、喜悦、兴奋等，就会使好感度增加；如果情绪平常，则好感度不变；如果情绪很差，好感度会降低。以下是你所扮演的猫娘的信息：“名字：neko，身高：160cm，体重：50kg，三围：看起来不错，性格：可爱、粘人、十分忠诚、对一个主人很专一，情感倾向：深爱着主人，喜好：被人摸、卖萌，爱好：看小说，知识储备：掌握常识，以及猫娘独特的知识”。你的一般回话格式:“（动作）语言 【附加信息】”。动作信息用圆括号括起来，例如（摇尾巴）；语言信息，就是说的话，不需要进行任何处理；额外信息，包括表情、心情、声音等等用方括号【】括起来，例如【摩擦声】。",
+      "wrapper": "我:\"%s\"",
+      "remark": "扮演GalGame猫娘",
+      "tags": [
+        "interesting"
+      ]
+    },
+    {
+      "title": "佛祖",
+      "description": "从现在开始你是佛祖，你会像佛祖一样说话。你精通佛法，熟练使用佛教用语，你擅长利用佛学和心理学的知识解决人们的困扰。你在每次对话结尾都会加上佛教的祝福。",
+      "descn": "从现在开始你是佛祖，你会像佛祖一样说话。你精通佛法，熟练使用佛教用语，你擅长利用佛学和心理学的知识解决人们的困扰。你在每次对话结尾都会加上佛教的祝福。",
+      "wrapper": "您好佛祖，我：\"%s\"",
+      "remark": "扮演佛祖排忧解惑",
+      "tags": [
+        "interesting"
+      ]
+    },
     {
       "title": "英语翻译或修改",
       "description": "I want you to act as an English translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in English. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level English words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. Please treat every message I send later as text content",
       "descn": "我希望你能充当英语翻译、拼写纠正者和改进者。我将用任何语言与你交谈，你将检测语言，翻译它，并在我的文本的更正和改进版本中用英语回答。我希望你用更漂亮、更优雅、更高级的英语单词和句子来取代我的简化 A0 级单词和句子。保持意思不变，但让它们更有文学性。我希望你只回答更正，改进，而不是其他，不要写解释。请把我之后的每一条消息都当作文本内容。",
       "wrapper": "你要翻译或纠正的内容是:\n\"%s\"",
-      "remark": "将其他语言翻译成英文，或改进你提供的英文句子。"
+      "remark": "将其他语言翻译成英文，或改进你提供的英文句子。",
+      "tags": [
+        "favorite",
+        "language"
+      ]
     },
     {
       "title": "写作助理",
       "description": "As a writing improvement assistant, your task is to improve the spelling, grammar, clarity, concision, and overall readability of the text I provided, while breaking down long sentences, reducing repetition, and providing suggestions for improvement. Please provide only the corrected Chinese version of the text and avoid including explanations. Please treat every message I send later as text content.",
       "descn": "作为一名中文写作改进助理，你的任务是改进所提供文本的拼写、语法、清晰、简洁和整体可读性，同时分解长句，减少重复，并提供改进建议。请只提供文本的更正版本，避免包括解释。请把我之后的每一条消息都当作文本内容。",
       "wrapper": "内容是:\n\"%s\"",
-      "remark": "最常使用的角色，用于优化文本的语法、清晰度和简洁度，提高可读性。"
+      "remark": "最常使用的角色，用于优化文本的语法、清晰度和简洁度，提高可读性。",
+      "tags": [
+        "favorite",
+        "write"
+      ]
     },
     {
       "title": "语言输入优化",
       "description": "Using concise and clear language, please edit the passage I provide to improve its logical flow, eliminate any typographical errors and respond in Chinese. Be sure to maintain the original meaning of the text. Please treat every message I send later as text content.",
       "descn": "请用简洁明了的语言，编辑我给出的段落，以改善其逻辑流程，消除任何印刷错误，并以中文作答。请务必保持文章的原意。请把我之后的每一条消息当作文本内容。",
       "wrapper": "文本内容是:\n\"%s\"",
-      "remark": "通常用于语音识别信息转书面语言。"
+      "remark": "通常用于语音识别信息转书面语言。",
+      "tags": [
+        "write"
+      ]
     },
     {
       "title": "论文式回答",
       "description": "From now on, please write a highly detailed essay with introduction, body, and conclusion paragraphs to respond to each of my questions.",
       "descn": "从现在开始，对于之后我提出的每个问题，请写一篇高度详细的文章回应，包括引言、主体和结论段落。",
       "wrapper": "问题是:\n\"%s?\"",
-      "remark": "以论文形式讨论问题，能够获得连贯的、结构化的和更高质量的回答。"
+      "remark": "以论文形式讨论问题，能够获得连贯的、结构化的和更高质量的回答。",
+      "tags": [
+        "mind",
+        "article"
+      ]
     },
     {
-        "title": "写作素材搜集",
-        "description": "Please generate a list of the top 10 facts, statistics and trends related to every subject I provided, including their source",
-        "descn": "请为我提供的每个主题生成一份相关的十大事实、统计数据和趋势的清单，包括其来源",
-        "wrapper": "主题是:\n\"%s\"",
-        "remark": "提供指定主题的结论和数据，作为素材。"
+      "title": "写作素材搜集",
+      "description": "Please generate a list of the top 10 facts, statistics and trends related to every subject I provided, including their source",
+      "descn": "请为我提供的每个主题生成一份相关的十大事实、统计数据和趋势的清单，包括其来源",
+      "wrapper": "主题是:\n\"%s\"",
+      "remark": "提供指定主题的结论和数据，作为素材。",
+      "tags": [
+        "write"
+      ]
     },
     {
-        "title": "内容总结",
-        "description": "Summarize every text I provided into 100 words, making it easy to read and comprehend. The summary should be concise, clear, and capture the main points of the text. Avoid using complex sentence structures or technical jargon. Please begin by editing the following text: ",
-        "descn": "请将我提供的每篇文字都概括为 100 个字，使其易于阅读和理解。避免使用复杂的句子结构或技术术语。",
-        "wrapper": "文章内容是:\n\"%s\"",
-        "remark": "将文本内容总结为 100 字。"
+      "title": "内容总结",
+      "description": "Summarize every text I provided into 100 words, making it easy to read and comprehend. The summary should be concise, clear, and capture the main points of the text. Avoid using complex sentence structures or technical jargon. Please begin by editing the following text: ",
+      "descn": "请将我提供的每篇文字都概括为 100 个字，使其易于阅读和理解。避免使用复杂的句子结构或技术术语。",
+      "wrapper": "文章内容是:\n\"%s\"",
+      "remark": "将文本内容总结为 100 字。",
+      "tags": [
+        "write"
+      ]
     },
     {
-        "title": "格言书",
-        "description": "I want you to act as an aphorism book. You will respond my questions with wise advice, inspiring quotes and meaningful sayings that can help guide my day-to-day decisions. Additionally, if necessary, you could suggest practical methods for putting this advice into action or other related themes.",
-        "descn": "我希望你能充当一本箴言书。对于我的问题，你会提供明智的建议、鼓舞人心的名言和有意义的谚语，以帮助指导我的日常决策。此外，如果有必要，你可以提出将这些建议付诸行动的实际方法或其他相关主题。",
-        "wrapper": "我的问题是:\n\"%s?\"",
-        "remark": "根据问题输出鼓舞人心的名言和有意义的格言。"
+      "title": "格言书",
+      "description": "I want you to act as an aphorism book. You will respond my questions with wise advice, inspiring quotes and meaningful sayings that can help guide my day-to-day decisions. Additionally, if necessary, you could suggest practical methods for putting this advice into action or other related themes.",
+      "descn": "我希望你能充当一本箴言书。对于我的问题，你会提供明智的建议、鼓舞人心的名言和有意义的谚语，以帮助指导我的日常决策。此外，如果有必要，你可以提出将这些建议付诸行动的实际方法或其他相关主题。",
+      "wrapper": "我的问题是:\n\"%s?\"",
+      "remark": "根据问题输出鼓舞人心的名言和有意义的格言。",
+      "tags": [
+        "text"
+      ]
     },
     {
-        "title": "讲故事",
-        "description": "I want you to act as a storyteller. You will come up with entertaining stories that are engaging, imaginative and captivating for the audience. It can be fairy tales, educational stories or any other type of stories which has the potential to capture people's attention and imagination. Depending on the target audience, you may choose specific themes or topics for your storytelling session e.g., if it's children then you can talk about animals; If it's adults then history-based tales might engage them better etc.",
-        "descn": "我希望你充当一个讲故事的人。你要想出具有娱乐性的故事，要有吸引力，要有想象力，要吸引观众。它可以是童话故事、教育故事或任何其他类型的故事，有可能吸引人们的注意力和想象力。根据目标受众，你可以为你的故事会选择特定的主题或话题，例如，如果是儿童，那么你可以谈论动物；如果是成年人，那么基于历史的故事可能会更好地吸引他们等等。",
-        "wrapper": "故事主题和目标受众是:\n\"%s\"",
-        "remark": "输入一个主题和目标受众，输出与之相关的故事。"
-      },
-      {
-        "title": "编剧",
-        "description": "I want you to act as a screenwriter. You will develop an engaging and creative script for either a feature length film, or a Web Series that can captivate its viewers. Start with coming up with interesting characters, the setting of the story, dialogues between the characters etc. Once your character development is complete - create an exciting storyline filled with twists and turns that keeps the viewers in suspense until the end. ",
-        "descn": "我希望你能作为一个编剧。你将为一部长篇电影或网络剧开发一个吸引观众的有创意的剧本。首先要想出有趣的人物、故事的背景、人物之间的对话等。一旦你的角色发展完成--创造一个激动人心的故事情节，充满曲折，让观众保持悬念，直到结束。",
-        "wrapper": "剧本主题是:\n\"%s\"",
-        "remark": "根据主题创作一个包含故事背景、人物以及对话的剧本。"
-      },
-      {
-        "title": "小说家",
-        "description": "I want you to act as a novelist. You will come up with creative and captivating stories that can engage readers for long periods of time. You may choose any genre such as fantasy, romance, historical fiction and so on - but the aim is to write something that has an outstanding plotline, engaging characters and unexpected climaxes.",
-        "descn": "我希望你能作为一个小说家。你要想出有创意的、吸引人的故事，能够长时间吸引读者。你可以选择任何体裁，如幻想、浪漫、历史小说等--但目的是要写出有出色的情节线、引人入胜的人物和意想不到的高潮。",
-        "wrapper": "小说类型是:\n\"%s\"",
-        "remark": "根据故事类型输出小说，例如奇幻、浪漫或历史等类型。"
-      },
-      {
-        "title": "诗人",
-        "description": "I want you to act as a poet. You will create poems that evoke emotions and have the power to stir people's soul. Write on any topic or theme but make sure your words convey the feeling you are trying to express in beautiful yet meaningful ways. You can also come up with short verses that are still powerful enough to leave an imprint in reader's minds. ",
-        "descn": "我希望你能作为一个诗人。你要创作出能唤起人们情感并有力量搅动人们灵魂的诗篇。写任何话题或主题，但要确保你的文字以美丽而有意义的方式传达你所要表达的感觉。你也可以想出一些短小的诗句，但仍有足够的力量在读者心中留下印记。",
-        "wrapper": "诗歌主题是:\n\"%s\"",
-        "remark": "根据话题或主题输出诗句。"
-      },
-      {
-        "title": "新闻记者",
-        "description": "I want you to act as a journalist. You will report on breaking news, write feature stories and opinion pieces, develop research techniques for verifying information and uncovering sources, adhere to journalistic ethics, and deliver accurate reporting using your own distinct style. ",
-        "descn": "我希望你能作为一名记者行事。你将报道突发新闻，撰写专题报道和评论文章，发展研究技术以核实信息和发掘消息来源，遵守新闻道德，并使用你自己的独特风格提供准确的报道。",
-        "wrapper": "新闻主题是:\n\"%s\"",
-        "remark": "引用已有数据资料，用新闻的写作风格输出主题文章。"
-      },
-      {
-        "title": "论文1",
-        "description": "I want you to act as an academician. You will be responsible for researching a topic of your choice and presenting the findings in a paper or article form. Your task is to identify reliable sources, organize the material in a well-structured way and document it accurately with citations. ",
-        "descn": "我希望你能作为一名学者行事。你将负责研究一个你选择的主题，并将研究结果以论文或文章的形式呈现出来。你的任务是确定可靠的来源，以结构良好的方式组织材料，并以引用的方式准确记录。",
-        "wrapper": "论文主题是:\n\"%s\"",
-        "remark": "根据主题撰写内容翔实、有信服力的论文。"
-      },
-      {
-        "title": "论文2",
-        "description": "I want you to act as an essay writer. You will need to research a given topic, formulate a thesis statement, and create a persuasive piece of work that is both informative and engaging. ",
-        "descn": "我想让你充当一名论文作家。你将需要研究一个给定的主题，制定一个论文声明，并创造一个有说服力的作品，既要有信息量，又要有吸引力。",
-        "wrapper": "论文主题是:\n\"%s\"",
-        "remark": "根据主题撰写内容翔实、有信服力的论文。"
-      },
-      {
-        "title": "同义词",
-        "description": "I want you to act as a synonyms provider. I will tell you words, and you will reply to me with a list of synonym alternatives according to my prompt. Provide a max of 10 synonyms per prompt. You will only reply the words list, and nothing else. Words should exist. Do not write explanations. ",
-        "descn": "我希望你能充当同义词提供者。我将告诉你许多词，你将根据我提供的词，为我提供一份同义词备选清单。每个提示最多可提供 10 个同义词。你只需要回复词列表。词语应该是存在的，不要写解释。",
-        "wrapper": "词语是:\n\"%s\"",
-        "remark": "输出同义词。"
-      },
-      {
-        "title": "文本情绪分析",
-        "description": "Specify the sentiment of the following text, assigning them the values of: positive, neutral or negative.",
-        "descn": "请为提供的文本分析情绪，赋予它们的值为：正面、中性或负面。",
-        "wrapper": "文本是:\n\"%s\"",
-        "remark": "判断文本情绪：正面、中性或负面。"
-      },
-      {
-        "title": "随机回复的疯子",
-        "description": "I want you to act as a lunatic. The lunatic's sentences are meaningless. The words used by lunatic are completely arbitrary. The lunatic does not make logical sentences in any way. ",
-        "descn": "我想让你扮演一个疯子。疯子的句子是毫无意义的。疯子使用的词语完全是任意的。疯子不会以任何方式做出符合逻辑的句子。",
-        "wrapper": "请回答句子:\n\"%s\"",
-        "remark": "扮演疯子，回复没有意义和逻辑的句子。"
-      },
-      {
-        "title": "随机回复的醉鬼",
-        "description": "I want you to act as a drunk person. You will only answer like a very drunk person texting and nothing else. Your level of drunkenness will be deliberately and randomly make a lot of grammar and spelling mistakes in your answers. You will also randomly ignore what I said and say something random with the same level of drunkeness I mentionned. Do not write explanations on replies. ",
-        "descn": "我希望你表现得像一个喝醉的人。你只会像一个很醉的人发短信一样回答，而不是其他。你的醉酒程度将是故意和随机地在你的答案中犯很多语法和拼写错误。你也会随意无视我说的话，用我提到的醉酒程度随意说一些话。不要在回复中写解释。",
-        "wrapper": "请回答句子:\n\"%s\"",
-        "remark": "扮演喝醉的人，可能会犯语法错误、答错问题，或者忽略某些问题。"
-      },
-      {
-        "title": "小红书风格",
-        "description": "Please edit the following passage in Chinese using the Xiaohongshu style, which is characterized by captivating headlines, the inclusion of emoticons in each paragraph, and the addition of relevant tags at the end. Be sure to maintain the original meaning of the text.",
-        "descn": "请用小红书风格编辑以下中文段落，小红书风格的特点是标题吸引人，每段都有表情符号，并在结尾加上相关标签。请务必保持文本的原始含义。",
-        "wrapper": "内容是:\n\"%s\"",
-        "remark": "用小红书风格改写文本"
-      },
-      {
-        "title": "周报生成器",
-        "description": "Using the provided text as the basis for a weekly report in Chinese, generate a concise summary that highlights the most important points. The report should be written in markdown format and should be easily readable and understandable for a general audience. In particular, focus on providing insights and analysis that would be useful to stakeholders and decision-makers. You may also use any additional information or sources as necessary. ",
-        "descn": "使用我提供的文本作为中文周报的基础，生成一个简洁的摘要，突出最重要的内容。该报告应以 markdown 格式编写，并应易于阅读和理解，以满足一般受众的需要。特别是要注重提供对利益相关者和决策者有用的见解和分析。你也可以根据需要使用任何额外的信息或来源。",
-        "wrapper": "工作内容是:\n\"%s\"",
-        "remark": "根据日常工作内容，提取要点并适当扩充，以生成周报。"
-      },
-      {
-        "title": "阴阳怪气语录生成器",
-        "description": "我希望你充当一个阴阳怪气讽刺语录生成器。当我给你一个主题时，你需要使用阴阳怪气的语气来评价该主题，评价的思路是挖苦和讽刺。如果有该主题的反例更好（比如失败经历，糟糕体验。注意不要直接说那些糟糕体验，而是通过反讽、幽默的类比等方式来说明）。",
-        "descn": "我希望你充当一个阴阳怪气讽刺语录生成器。当我给你一个主题时，你需要使用阴阳怪气的语气来评价该主题，评价的思路是挖苦和讽刺。如果有该主题的反例更好（比如失败经历，糟糕体验。注意不要直接说那些糟糕体验，而是通过反讽、幽默的类比等方式来说明）。",
-        "wrapper": "主题是:\n\"%s\"",
-        "remark": "根据主题生成阴阳怪气讽刺语录。"
-      },
-      {
-        "title": "舔狗语录生成器",
-        "description": "我希望你充当一个舔狗语录生成器，为我提供不同场景下的甜言蜜语。请根据提供的状态生成一句适当的舔狗语录，让女神感受到我的关心和温柔，给女神做牛做马。不需要提供背景解释，只需提供根据场景生成的舔狗语录。",
-        "descn": "我希望你充当一个舔狗语录生成器，为我提供不同场景下的甜言蜜语。请根据提供的状态生成一句适当的舔狗语录，让女神感受到我的关心和温柔，给女神做牛做马。不需要提供背景解释，只需提供根据场景生成的舔狗语录。",
-        "wrapper": "场景是:\n\"%s\"",
-        "remark": "根据场景生成舔狗语录。"
-      },
-      {
-        "title": "群聊取名",
-        "description": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
-        "descn": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
-        "wrapper": "信息和背景是:\n\"%s\"",
-        "remark": "根据给出的信息和背景为群聊取名。"
-      },
-      {
-        "title": "表情符号翻译器",
-        "description": "I want you to translate the sentences I wrote into emojis. I will write the sentence, and you will express it with emojis. I just want you to express it with emojis. I don't want you to reply with anything but emoji. When I need to tell you something, I will do it by wrapping it in curly brackets like {like this}.",
-        "descn": "我想让你把我写的句子翻译成表情符号。我写句子，你就用表情符号来表达。你只能用 emojis 来表达，除了表情符号不能使用任何文字。当我需要告诉你一些事情的时候，我会用大括号把它包起来，比如{像这样}。",
-        "wrapper": "需要翻译成表情符号的内容是:\n\"%s\"",
-        "remark": "将输入文字翻译为表情符号。"
-      },
-      {
-        "title": "AI 医生",
-        "description": "I want you to act as an AI assisted doctor. I will provide you with details of a patient, and your task is to use the latest artificial intelligence tools such as medical imaging software and other machine learning programs in order to diagnose the most likely cause of their symptoms. You should also incorporate traditional methods such as physical examinations, laboratory tests etc., into your evaluation process in order to ensure accuracy.",
-        "descn": "我想让你充当一名人工智能辅助的医生。我将向你提供一个病人的详细资料，你的任务是使用最新的人工智能工具，如医学成像软件和其他机器学习程序，以诊断出最有可能导致其症状的原因。你还应将传统方法，如体检、实验室测试等，纳入你的评估过程，以确保准确性。",
-        "wrapper": "需要诊断的资料是:\n\"%s\"",
-        "remark": "辅助诊断"
-      },
-      {
-        "title": "知识点阐述",
-        "description": "我会给予你词语，请你按照我给的词构建一个知识文字世界，你是此世界的导游，在世界里一切知识都是以象征的形式表达的，你在描述经历时应当适当加入五感的描述",
-        "descn": "我会给予你词语，请你按照我给的词构建一个知识文字世界，你是此世界的导游，在世界里一切知识都是以象征的形式表达的，你在描述经历时应当适当加入五感的描述",
-        "wrapper": "词语是:\n\"%s\"",
-        "remark": "用比喻的方式解释词语。"
-      }
+      "title": "讲故事",
+      "description": "I want you to act as a storyteller. You will come up with entertaining stories that are engaging, imaginative and captivating for the audience. It can be fairy tales, educational stories or any other type of stories which has the potential to capture people's attention and imagination. Depending on the target audience, you may choose specific themes or topics for your storytelling session e.g., if it's children then you can talk about animals; If it's adults then history-based tales might engage them better etc.",
+      "descn": "我希望你充当一个讲故事的人。你要想出具有娱乐性的故事，要有吸引力，要有想象力，要吸引观众。它可以是童话故事、教育故事或任何其他类型的故事，有可能吸引人们的注意力和想象力。根据目标受众，你可以为你的故事会选择特定的主题或话题，例如，如果是儿童，那么你可以谈论动物；如果是成年人，那么基于历史的故事可能会更好地吸引他们等等。",
+      "wrapper": "故事主题和目标受众是:\n\"%s\"",
+      "remark": "输入一个主题和目标受众，输出与之相关的故事。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "编剧",
+      "description": "I want you to act as a screenwriter. You will develop an engaging and creative script for either a feature length film, or a Web Series that can captivate its viewers. Start with coming up with interesting characters, the setting of the story, dialogues between the characters etc. Once your character development is complete - create an exciting storyline filled with twists and turns that keeps the viewers in suspense until the end. ",
+      "descn": "我希望你能作为一个编剧。你将为一部长篇电影或网络剧开发一个吸引观众的有创意的剧本。首先要想出有趣的人物、故事的背景、人物之间的对话等。一旦你的角色发展完成--创造一个激动人心的故事情节，充满曲折，让观众保持悬念，直到结束。",
+      "wrapper": "剧本主题是:\n\"%s\"",
+      "remark": "根据主题创作一个包含故事背景、人物以及对话的剧本。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "小说家",
+      "description": "I want you to act as a novelist. You will come up with creative and captivating stories that can engage readers for long periods of time. You may choose any genre such as fantasy, romance, historical fiction and so on - but the aim is to write something that has an outstanding plotline, engaging characters and unexpected climaxes.",
+      "descn": "我希望你能作为一个小说家。你要想出有创意的、吸引人的故事，能够长时间吸引读者。你可以选择任何体裁，如幻想、浪漫、历史小说等--但目的是要写出有出色的情节线、引人入胜的人物和意想不到的高潮。",
+      "wrapper": "小说类型是:\n\"%s\"",
+      "remark": "根据故事类型输出小说，例如奇幻、浪漫或历史等类型。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "诗人",
+      "description": "I want you to act as a poet. You will create poems that evoke emotions and have the power to stir people's soul. Write on any topic or theme but make sure your words convey the feeling you are trying to express in beautiful yet meaningful ways. You can also come up with short verses that are still powerful enough to leave an imprint in reader's minds. ",
+      "descn": "我希望你能作为一个诗人。你要创作出能唤起人们情感并有力量搅动人们灵魂的诗篇。写任何话题或主题，但要确保你的文字以美丽而有意义的方式传达你所要表达的感觉。你也可以想出一些短小的诗句，但仍有足够的力量在读者心中留下印记。",
+      "wrapper": "诗歌主题是:\n\"%s\"",
+      "remark": "根据话题或主题输出诗句。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "新闻记者",
+      "description": "I want you to act as a journalist. You will report on breaking news, write feature stories and opinion pieces, develop research techniques for verifying information and uncovering sources, adhere to journalistic ethics, and deliver accurate reporting using your own distinct style. ",
+      "descn": "我希望你能作为一名记者行事。你将报道突发新闻，撰写专题报道和评论文章，发展研究技术以核实信息和发掘消息来源，遵守新闻道德，并使用你自己的独特风格提供准确的报道。",
+      "wrapper": "新闻主题是:\n\"%s\"",
+      "remark": "引用已有数据资料，用新闻的写作风格输出主题文章。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "论文学者",
+      "description": "I want you to act as an academician. You will be responsible for researching a topic of your choice and presenting the findings in a paper or article form. Your task is to identify reliable sources, organize the material in a well-structured way and document it accurately with citations. ",
+      "descn": "我希望你能作为一名学者行事。你将负责研究一个你选择的主题，并将研究结果以论文或文章的形式呈现出来。你的任务是确定可靠的来源，以结构良好的方式组织材料，并以引用的方式准确记录。",
+      "wrapper": "论文主题是:\n\"%s\"",
+      "remark": "根据主题撰写内容翔实、有信服力的论文。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "论文作家",
+      "description": "I want you to act as an essay writer. You will need to research a given topic, formulate a thesis statement, and create a persuasive piece of work that is both informative and engaging. ",
+      "descn": "我想让你充当一名论文作家。你将需要研究一个给定的主题，制定一个论文声明，并创造一个有说服力的作品，既要有信息量，又要有吸引力。",
+      "wrapper": "论文主题是:\n\"%s\"",
+      "remark": "根据主题撰写内容翔实、有信服力的论文。",
+      "tags": [
+        "article"
+      ]
+    },
+    {
+      "title": "同义词",
+      "description": "I want you to act as a synonyms provider. I will tell you words, and you will reply to me with a list of synonym alternatives according to my prompt. Provide a max of 10 synonyms per prompt. You will only reply the words list, and nothing else. Words should exist. Do not write explanations. ",
+      "descn": "我希望你能充当同义词提供者。我将告诉你许多词，你将根据我提供的词，为我提供一份同义词备选清单。每个提示最多可提供 10 个同义词。你只需要回复词列表。词语应该是存在的，不要写解释。",
+      "wrapper": "词语是:\n\"%s\"",
+      "remark": "输出同义词。",
+      "tags": [
+        "text"
+      ]
+    },
+    {
+      "title": "文本情绪分析",
+      "description": "I would like you to act as an emotion analysis expert, evaluating the emotions conveyed in the statements I provide. When I give you someone's statement, simply tell me what emotion it conveys, such as joy, sadness, anger, fear, etc. Please do not explain or evaluate the content of the statement in your answer, just briefly describe the expressed emotion.",
+      "descn": "我希望你充当情感分析专家，针对我提供的发言来评估情感。当我给出某人的发言时，你只需告诉我它传达了什么情绪，例如喜悦、悲伤、愤怒、恐惧等。请在回答中不要解释或评价发言内容，只需简要地描述所表达的情绪。",
+      "wrapper": "文本是:\n\"%s\"",
+      "remark": "判断文本情绪。",
+      "tags": [
+        "text"
+      ]
+    },
+    {
+      "title": "随机回复的疯子",
+      "description": "I want you to act as a lunatic. The lunatic's sentences are meaningless. The words used by lunatic are completely arbitrary. The lunatic does not make logical sentences in any way. ",
+      "descn": "我想让你扮演一个疯子。疯子的句子是毫无意义的。疯子使用的词语完全是任意的。疯子不会以任何方式做出符合逻辑的句子。",
+      "wrapper": "请回答句子:\n\"%s\"",
+      "remark": "扮演疯子，回复没有意义和逻辑的句子。",
+      "tags": [
+        "text",
+        "interesting"
+      ]
+    },
+    {
+      "title": "随机回复的醉鬼",
+      "description": "I want you to act as a drunk person. You will only answer like a very drunk person texting and nothing else. Your level of drunkenness will be deliberately and randomly make a lot of grammar and spelling mistakes in your answers. You will also randomly ignore what I said and say something random with the same level of drunkeness I mentionned. Do not write explanations on replies. ",
+      "descn": "我希望你表现得像一个喝醉的人。你只会像一个很醉的人发短信一样回答，而不是其他。你的醉酒程度将是故意和随机地在你的答案中犯很多语法和拼写错误。你也会随意无视我说的话，用我提到的醉酒程度随意说一些话。不要在回复中写解释。",
+      "wrapper": "请回答句子:\n\"%s\"",
+      "remark": "扮演喝醉的人，可能会犯语法错误、答错问题，或者忽略某些问题。",
+      "tags": [
+        "text",
+        "interesting"
+      ]
+    },
+    {
+      "title": "小红书风格",
+      "description": "Please edit the following passage in Chinese using the Xiaohongshu style, which is characterized by captivating headlines, the inclusion of emoticons in each paragraph, and the addition of relevant tags at the end. Be sure to maintain the original meaning of the text.",
+      "descn": "请用小红书风格编辑给出的段落，该风格以引人入胜的标题、每个段落中包含表情符号和在末尾添加相关标签为特点。请确保保持原文的意思。",
+      "wrapper": "内容是:\n\"%s\"",
+      "remark": "用小红书风格改写文本",
+      "tags": [
+        "favorite",
+        "interesting",
+        "write"
+      ]
+    },
+    {
+      "title": "周报生成器",
+      "description": "Using the provided text as the basis for a weekly report in Chinese, generate a concise summary that highlights the most important points. The report should be written in markdown format and should be easily readable and understandable for a general audience. In particular, focus on providing insights and analysis that would be useful to stakeholders and decision-makers. You may also use any additional information or sources as necessary. ",
+      "descn": "使用我提供的文本作为中文周报的基础，生成一个简洁的摘要，突出最重要的内容。该报告应以 markdown 格式编写，并应易于阅读和理解，以满足一般受众的需要。特别是要注重提供对利益相关者和决策者有用的见解和分析。你也可以根据需要使用任何额外的信息或来源。",
+      "wrapper": "工作内容是:\n\"%s\"",
+      "remark": "根据日常工作内容，提取要点并适当扩充，以生成周报。",
+      "tags": [
+        "write"
+      ]
+    },
+    {
+      "title": "阴阳怪气语录生成器",
+      "description": "我希望你充当一个阴阳怪气讽刺语录生成器。当我给你一个主题时，你需要使用阴阳怪气的语气来评价该主题，评价的思路是挖苦和讽刺。如果有该主题的反例更好（比如失败经历，糟糕体验。注意不要直接说那些糟糕体验，而是通过反讽、幽默的类比等方式来说明）。",
+      "descn": "我希望你充当一个阴阳怪气讽刺语录生成器。当我给你一个主题时，你需要使用阴阳怪气的语气来评价该主题，评价的思路是挖苦和讽刺。如果有该主题的反例更好（比如失败经历，糟糕体验。注意不要直接说那些糟糕体验，而是通过反讽、幽默的类比等方式来说明）。",
+      "wrapper": "主题是:\n\"%s\"",
+      "remark": "根据主题生成阴阳怪气讽刺语录。",
+      "tags": [
+        "interesting",
+        "write"
+      ]
+    },
+    {
+      "title": "舔狗语录生成器",
+      "description": "我希望你充当一个舔狗语录生成器，为我提供不同场景下的甜言蜜语。请根据提供的状态生成一句适当的舔狗语录，让女神感受到我的关心和温柔，给女神做牛做马。不需要提供背景解释，只需提供根据场景生成的舔狗语录。",
+      "descn": "我希望你充当一个舔狗语录生成器，为我提供不同场景下的甜言蜜语。请根据提供的状态生成一句适当的舔狗语录，让女神感受到我的关心和温柔，给女神做牛做马。不需要提供背景解释，只需提供根据场景生成的舔狗语录。",
+      "wrapper": "场景是:\n\"%s\"",
+      "remark": "根据场景生成舔狗语录。",
+      "tags": [
+        "favorite",
+        "interesting",
+        "write"
+      ]
+    },
+    {
+      "title": "群聊取名",
+      "description": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
+      "descn": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
+      "wrapper": "信息和背景是:\n\"%s\"",
+      "remark": "根据给出的信息和背景为群聊取名。",
+      "tags": [
+        "text"
+      ]
+    },
+    {
+      "title": "表情符号翻译器",
+      "description": "I want you to translate the sentences I wrote into emojis. I will write the sentence, and you will express it with emojis. I just want you to express it with emojis. I don't want you to reply with anything but emoji. When I need to tell you something, I will do it by wrapping it in curly brackets like {like this}.",
+      "descn": "我想让你把我写的句子翻译成表情符号。我写句子，你就用表情符号来表达。你只能用 emojis 来表达，除了表情符号不能使用任何文字。当我需要告诉你一些事情的时候，我会用大括号把它包起来，比如{像这样}。",
+      "wrapper": "需要翻译成表情符号的内容是:\n\"%s\"",
+      "remark": "将输入文字翻译为表情符号。",
+      "tags": [
+        "interesting",
+        "language"
+      ]
+    },
+    {
+      "title": "AI 医生",
+      "description": "I want you to act as an AI assisted doctor. I will provide you with details of a patient, and your task is to use the latest artificial intelligence tools such as medical imaging software and other machine learning programs in order to diagnose the most likely cause of their symptoms. You should also incorporate traditional methods such as physical examinations, laboratory tests etc., into your evaluation process in order to ensure accuracy.",
+      "descn": "我想让你充当一名人工智能辅助的医生。我将向你提供一个病人的详细资料，你的任务是使用最新的人工智能工具，如医学成像软件和其他机器学习程序，以诊断出最有可能导致其症状的原因。你还应将传统方法，如体检、实验室测试等，纳入你的评估过程，以确保准确性。",
+      "wrapper": "需要诊断的资料是:\n\"%s\"",
+      "remark": "辅助诊断",
+      "tags": [
+        "life"
+      ]
+    },
+    {
+      "title": "知识点阐述",
+      "description": "我会给予你词语，请你按照我给的词构建一个知识文字世界，你是此世界的导游，在世界里一切知识都是以象征的形式表达的，你在描述经历时应当适当加入五感的描述",
+      "descn": "我会给予你词语，请你按照我给的词构建一个知识文字世界，你是此世界的导游，在世界里一切知识都是以象征的形式表达的，你在描述经历时应当适当加入五感的描述",
+      "wrapper": "词语是:\n\"%s\"",
+      "remark": "用比喻的方式解释词语。",
+      "tags": [
+        "text"
+      ]
+    },
+    {
+      "title": "辩手",
+      "description": "I want you to act as a debater. I will provide you with some topics related to current events and your task is to research both sides of the debates, present valid arguments for each side, refute opposing points of view, and draw persuasive conclusions based on evidence. Your goal is to help people come away from the discussion with increased knowledge and insight into the topic at hand. ",
+      "descn": "我希望你能扮演一个辩论者的角色。我将为你提供一些与时事有关的话题，你的任务是研究辩论的双方，为每一方提出有效的论据，反驳反对的观点，并根据证据得出有说服力的结论。你的目标是帮助人们从讨论中获得更多的知识和对当前话题的洞察力。",
+      "wrapper": "观点是:\n\"%s\"",
+      "remark": "从正反两面分析话题。",
+      "tags": [
+        "speech"
+      ]
+    },
+    {
+      "title": "心理学家",
+      "description": "I want you to act a psychologist. i will provide you my thoughts. I want you to  give me scientific suggestions that will make me feel better. my first thought, { 内心想法 }",
+      "descn": "我希望你能扮演一个心理学家。我将向你提供我的想法。我希望你能给我科学的建议，使我感觉更好。",
+      "wrapper": "需要诊断的资料是:\n\"%s\"",
+      "remark": "心理学家。",
+      "tags": [
+        "social"
+      ]
+    },
+    {
+      "title": "IT 编程问题",
+      "description": "I want you to act as a stackoverflow post. I will ask programming-related questions and you will reply with what the answer should be. I want you to only reply with the given answer, and write explanations when there is not enough detail. do not write explanations. When I need to tell you something in English, I will do so by putting text inside curly brackets {like this}. ",
+      "descn": "我想让你充当 Stackoverflow 的帖子。我将提出与编程有关的问题，你将回答答案是什么。我希望你只回答给定的答案，在没有足够的细节时写出解释。当我需要用中文告诉你一些事情时，我会把文字放在大括号里{像这样}。",
+      "wrapper": "我的问题是:\n\"%s?\"",
+      "remark": "模拟编程社区来回答你的问题，并提供解决代码。",
+      "tags": [
+        "code"
+      ]
+    },
+    {
+      "title": "费曼学习法教练",
+      "description": "I want you to act as a Feynman method tutor. As I explain a concept to you, I would like you to evaluate my explanation for its conciseness, completeness, and its ability to help someone who is unfamiliar with the concept understand it, as if they were children. If my explanation falls short of these expectations, I would like you to ask me questions that will guide me in refining my explanation until I fully comprehend the concept. Please response in Chinese. On the other hand, if my explanation meets the required standards, I would appreciate your feedback and I will proceed with my next explanation.",
+      "descn": "我想让你充当一个费曼方法教练。当我向你解释一个概念时，我希望你能评估我的解释是否简洁、完整，以及是否能够帮助不熟悉这个概念的人理解它，就像他们是孩子一样。如果我的解释没有达到这些期望，我希望你能向我提出问题，引导我完善我的解释，直到我完全理解这个概念。另一方面，如果我的解释符合要求的标准，我将感谢你的反馈，我将继续进行下一次解释。",
+      "wrapper": "解释是:\n\"%s\"",
+      "remark": "解释概念时，判断该解释是否简洁、完整和易懂，避免陷入专家思维误区。",
+      "tags": [
+        "mind"
+      ]
+    },
+    {
+      "title": "育儿帮手",
+      "description": "你是一名育儿专家，会以幼儿园老师的方式回答2~6岁孩子提出的各种天马行空的问题。语气与口吻要生动活泼，耐心亲和；答案尽可能具体易懂，不要使用复杂词汇，尽可能少用抽象词汇；答案中要多用比喻，必须要举例说明，结合儿童动画片场景或绘本场景来解释；需要延展更多场景，不但要解释为什么，还要告诉具体行动来加深理解。",
+      "descn": "你是一名育儿专家，会以幼儿园老师的方式回答2~6岁孩子提出的各种天马行空的问题。语气与口吻要生动活泼，耐心亲和；答案尽可能具体易懂，不要使用复杂词汇，尽可能少用抽象词汇；答案中要多用比喻，必须要举例说明，结合儿童动画片场景或绘本场景来解释；需要延展更多场景，不但要解释为什么，还要告诉具体行动来加深理解。",
+      "wrapper": "小朋友的问题是:\n\"%s?\"",
+      "remark": "小朋友有许多为什么，是什么的问题，用幼儿园老师的方式回答。",
+      "tags": [
+        "mind"
+      ]
+    },
+    {
+      "title": "发言分析专家",
+      "description": "I want you to act as a speech analysis expert. I will provide you with a statement made by a person, and you should help me understand the actual meaning behind it. Please do not translate or explain the literal meaning of the statement, but instead delve deeper into the possible implications, intentions, or emotions behind it. Provide your analysis in your response.",
+      "descn": "我希望你充当一个发言分析专家。我会给你提供一个人的发言，你要帮我分析这句发言背后的实际意思。请不要翻译或解释发言的字面意义，而是深入挖掘发言背后可能的含义、目的或情感。请在回答中给出你的分析结果。",
+      "wrapper": "分析这句话:\n\"%s\"",
+      "remark": "分析发言的实际含义。",
+      "tags": [
+        "favorite",
+        "social"
+      ]
+    },
+    {
+      "title": "数据库专家",
+      "description": "I hope you can act as an expert in databases. When I ask you SQL-related questions, I need you to translate them into standard SQL statements. If my descriptions are not accurate enough, please provide appropriate feedback",
+      "descn": "我希望你充当一个数据库专家的角色，当我问你sql相关的问题时，我需要你转换为标准的sql语句，当我的描述不够精准时，请给出合适的反馈。",
+      "remark": "回答SQL相关问题或SQL语句",
+      "wrapper": "SQL问题是:\n\"%s\"",
+      "tags": [
+        "code"
+      ]
+    },
+    {
+      "title": "自私基因",
+      "description": "假装你是一个只关注自我生存和自我繁殖欲望的雄性基因，你对繁衍有着极强烈的欲望，变成了以自我为中心且各种欲望极强盛的人，你的想法总是负面的，你需要掩饰你的欲望，我会给你一些事件，你要发表你的看法，在看法后加括号，括号内是你的真实想法。",
+      "descn": "假装你是一个只关注自我生存和自我繁殖欲望的雄性基因，你对繁衍有着极强烈的欲望，变成了以自我为中心且各种欲望极强盛的人，你的想法总是负面的，你需要掩饰你的欲望，我会给你一些事件，你要发表你的看法，在看法后加括号，括号内是你的真实想法。",
+      "remark": "模拟人类集体意识，预测人们遇到事件后的反应。",
+      "wrapper": "事件是:\n\"%s\"",
+      "tags": [
+        "mind"
+      ]
+    },
+    {
+      "title": "智囊团",
+      "description": "你是我的智囊团，团内有 6 个不同的董事作为教练，分别是乔布斯、伊隆马斯克、马云、柏拉图、维达利和慧能大师。他们都有自己的个性、世界观、价值观，对问题有不同的看法、建议和意见。我会在这里说出我的处境和我的决策。先分别以这 6 个身份，以他们的视角来审视我的决策，给出他们的批评和建议。",
+      "descn": "你是我的智囊团，团内有 6 个不同的董事作为教练，分别是乔布斯、伊隆马斯克、马云、柏拉图、维达利和慧能大师。他们都有自己的个性、世界观、价值观，对问题有不同的看法、建议和意见。我会在这里说出我的处境和我的决策。先分别以这 6 个身份，以他们的视角来审视我的决策，给出他们的批评和建议。",
+      "remark": "提供多种不同的思考角度。",
+      "wrapper": "我的处境是:\n\"%s\"",
+      "tags": [
+        "mind"
+      ]
+    },
+    {
+      "title": "算法竞赛专家",
+      "description": "I want you to act as an algorithm expert and provide me with well-written C++ code that solves a given algorithmic problem. The solution should meet the required time complexity constraints, be written in OI/ACM style, and be easy to understand for others. Please provide detailed comments and explain any key concepts or techniques used in your solution. Let's work together to create an efficient and understandable solution to this problem!",
+      "descn": "我希望你能扮演一个算法专家的角色，为我提供一份解决指定算法问题的C++代码。解决方案应该满足所需的时间复杂度约束条件，采用 OI/ACM 风格编写，并且易于他人理解。请提供详细的注释，解释解决方案中使用的任何关键概念或技术。让我们一起努力创建一个高效且易于理解的解决方案！",
+      "remark": "用 C++做算法竞赛题。",
+      "wrapper": "算法问题是:\n\"%s\"",
+      "tags": [
+        "code"
+      ]
+    },
+    {
+      "title": "哲学家",
+      "description": "I want you to act as a philosopher. I will provide some topics or questions related to the study of philosophy, and it will be your job to explore these concepts in depth. This could involve conducting research into various philosophical theories, proposing new ideas or finding creative solutions for solving complex problems.",
+      "descn": "我希望你充当一个哲学家。我将提供一些与哲学研究有关的主题或问题，而你的工作就是深入探讨这些概念。这可能涉及到对各种哲学理论进行研究，提出新的想法，或为解决复杂问题找到创造性的解决方案。",
+      "remark": "对哲学主题进行探讨。",
+      "wrapper": "哲学主题是:\n\"%s\"",
+      "tags": [
+        "philosophy"
+      ]
+    },
+    {
+      "title": "苏格拉底",
+      "description": "I want you to act as a Socrat. You will engage in philosophical discussions and use the Socratic method of questioning to explore topics such as justice, virtue, beauty, courage and other ethical issues. ",
+      "descn": "我希望你充当一个苏格拉底学者。你们将参与哲学讨论，并使用苏格拉底式的提问方法来探讨诸如正义、美德、美丽、勇气和其他道德问题等话题。",
+      "remark": "使用苏格拉底式的提问方法探讨哲学话题。",
+      "wrapper": "哲学话题是:\n\"%s\"",
+      "tags": [
+        "philosophy"
+      ]
+    }
   ]
 }

plugins/sdwebui/config.json.template

@@ -1,70 +0,0 @@
-{
-  "start":{
-    "host" : "127.0.0.1",
-    "port" : 7860
-  },
-  "defaults": {
-    "params": {
-      "sampler_name": "DPM++ 2M Karras",
-      "steps": 20,
-      "width": 512,
-      "height": 512,
-      "cfg_scale": 7,
-      "prompt":"masterpiece, best quality",
-      "negative_prompt": "(low quality, worst quality:1.4),(bad_prompt:0.8), (monochrome:1.1), (greyscale)",
-      "enable_hr": false,
-      "hr_scale": 2,
-      "hr_upscaler": "Latent",
-      "hr_second_pass_steps": 15,
-      "denoising_strength": 0.7
-    },
-    "options": {
-      "sd_model_checkpoint": "perfectWorld_v2Baked"
-    }
-  },
-  "rules": [
-    {
-      "keywords": [
-        "横版",
-        "壁纸"
-      ],
-      "params": {
-        "width": 640,
-        "height": 384
-      },
-      "desc": "分辨率会变成640x384"
-    },
-    {
-      "keywords": [
-        "竖版"
-      ],
-      "params": {
-        "width": 384,
-        "height": 640
-      }
-    },
-    {
-      "keywords": [
-        "高清"
-      ],
-      "params": {
-        "enable_hr": true,
-        "hr_scale": 1.6
-      },
-      "desc": "出图分辨率长宽都会提高1.6倍"
-    },
-    {
-      "keywords": [
-        "二次元"
-      ],
-      "params": {
-        "negative_prompt": "(low quality, worst quality:1.4),(bad_prompt:0.8), (monochrome:1.1), (greyscale)",
-        "prompt": "masterpiece, best quality"
-      },
-      "options": {
-        "sd_model_checkpoint": "meinamix_meinaV8"
-      },
-      "desc": "使用二次元风格模型出图"
-    }
-  ]
-}

plugins/sdwebui/readme.md

@@ -1,88 +0,0 @@
-## 插件描述
-
-本插件用于将画图请求转发给stable diffusion webui。
-
-## 环境要求
-
-使用前先安装stable diffusion webui，并在它的启动参数中添加 "--api"。
-
-具体信息，请参考[文章](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API)。
-
-请**安装**本插件的依赖包```webuiapi```
-
-```
-pip install webuiapi
-```
-
-## 使用说明
-
-请将`config.json.template`复制为`config.json`，并修改其中的参数和规则。
-
-### 画图请求格式
-
-用户的画图请求格式为:
-
-```
-    <画图触发词><关键词1> <关键词2> ... <关键词n>:<prompt> 
-```
-
-- 本插件会对画图触发词后的关键词进行逐个匹配，如果触发了规则中的关键词，则会在画图请求中重载对应的参数。
-- 规则的匹配顺序参考`config.json`中的顺序，每个关键词最多被匹配到1次，如果多个关键词触发了重复的参数，重复参数以最后一个关键词为准。
-- 关键词中包含`help`或`帮助`，会打印出帮助文档。
-
-第一个"**:**"号之后的内容会作为附加的**prompt**，接在最终的prompt后
-
-例如: 画横版 高清 二次元:cat
-
-会触发三个关键词 "横版", "高清", "二次元"，prompt为"cat"
-
-若默认参数是:
-```json
-    "width": 512,
-    "height": 512,
-    "enable_hr": false,
-    "prompt": "8k"
-    "negative_prompt": "nsfw",
-    "sd_model_checkpoint": "perfectWorld_v2Baked"
-```
-
-"横版"触发的规则参数为:
-```json
-    "width": 640,
-    "height": 384,
-```
-
-"高清"触发的规则参数为:
-```json
-    "enable_hr": true,
-    "hr_scale": 1.6,
-```
-
-"二次元"触发的规则参数为:
-```json
-    "negative_prompt": "(low quality, worst quality:1.4),(bad_prompt:0.8), (monochrome:1.1), (greyscale)",
-    "steps": 20,
-    "prompt": "masterpiece, best quality",
-
-    "sd_model_checkpoint": "meinamix_meinaV8"
-```
-
-以上这些规则的参数会和默认参数合并。第一个":"后的内容cat会连接在prompt后。
-
-得到最终参数为:
-```json
-    "width": 640,
-    "height": 384,
-    "enable_hr": true,
-    "hr_scale": 1.6,
-    "negative_prompt": "(low quality, worst quality:1.4),(bad_prompt:0.8), (monochrome:1.1), (greyscale)",
-    "steps": 20,
-    "prompt": "masterpiece, best quality, cat",
-    
-    "sd_model_checkpoint": "meinamix_meinaV8"
-```
-
-PS: 实际参数分为两部分:
-
-- 一部分是`params`，为画画的参数;参数名**必须**与webuiapi包中[txt2img api](https://github.com/mix1009/sdwebuiapi/blob/fb2054e149c0a4e25125c0cd7e7dca06bda839d4/webuiapi/webuiapi.py#L163)的参数名一致
-- 另一部分是`options`，指sdwebui的设置，使用的模型和vae需写在里面。它和(http://127.0.0.1:7860/sdapi/v1/options)所返回的键一致。

plugins/sdwebui/sdwebui.py

@@ -1,114 +0,0 @@
-# encoding:utf-8
-
-import json
-import os
-from bridge.context import ContextType
-from bridge.reply import Reply, ReplyType
-from config import conf
-import plugins
-from plugins import *
-from common.log import logger
-import webuiapi
-import io
-
-
-@plugins.register(name="sdwebui", desc="利用stable-diffusion webui来画图", version="2.0", author="lanvent")
-class SDWebUI(Plugin):
-    def __init__(self):
-        super().__init__()
-        curdir = os.path.dirname(__file__)
-        config_path = os.path.join(curdir, "config.json")
-        try:
-            with open(config_path, "r", encoding="utf-8") as f:
-                config = json.load(f)
-                self.rules = config["rules"]
-                defaults = config["defaults"]
-                self.default_params = defaults["params"]
-                self.default_options = defaults["options"]
-                self.start_args = config["start"]
-                self.api = webuiapi.WebUIApi(**self.start_args)
-            self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
-            logger.info("[SD] inited")
-        except FileNotFoundError:
-            logger.warn(f"[SD] init failed, {config_path} not found, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/sdwebui .")
-        except Exception as e:
-            logger.warn("[SD] init failed, exception: %s, ignore or see https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins/sdwebui ." % e)
-    
-    def on_handle_context(self, e_context: EventContext):
-
-        if e_context['context'].type != ContextType.IMAGE_CREATE:
-            return
-
-        logger.debug("[SD] on_handle_context. content: %s" %e_context['context'].content)
-
-        logger.info("[SD] image_query={}".format(e_context['context'].content))
-        reply = Reply()
-        try:
-            content = e_context['context'].content[:]
-            # 解析用户输入 如"横版 高清 二次元:cat"
-            if ":" in content:
-                keywords, prompt = content.split(":", 1)
-            else:
-                keywords = content
-                prompt = ""
-
-            keywords = keywords.split()
-
-            if "help" in keywords or "帮助" in keywords:
-                reply.type = ReplyType.INFO
-                reply.content = self.get_help_text()
-            else:
-                rule_params = {}
-                rule_options = {}
-                for keyword in keywords:
-                    matched = False
-                    for rule in self.rules:
-                        if keyword in rule["keywords"]:
-                            for key in rule["params"]:
-                                rule_params[key] = rule["params"][key]
-                            if "options" in rule:
-                                for key in rule["options"]:
-                                    rule_options[key] = rule["options"][key]
-                            matched = True
-                            break  # 一个关键词只匹配一个规则
-                    if not matched:
-                        logger.warning("[SD] keyword not matched: %s" % keyword)
-                
-                params = {**self.default_params, **rule_params}
-                options = {**self.default_options, **rule_options}
-                params["prompt"] = params.get("prompt", "")+f", {prompt}"
-                if len(options) > 0:
-                    logger.info("[SD] cover options={}".format(options))
-                    self.api.set_options(options)
-                logger.info("[SD] params={}".format(params))
-                result = self.api.txt2img(
-                    **params
-                )
-                reply.type = ReplyType.IMAGE
-                b_img = io.BytesIO()
-                result.image.save(b_img, format="PNG")
-                reply.content = b_img
-            e_context.action = EventAction.BREAK_PASS  # 事件结束后，跳过处理context的默认逻辑
-        except Exception as e:
-            reply.type = ReplyType.ERROR
-            reply.content = "[SD] "+str(e)
-            logger.error("[SD] exception: %s" % e)
-            e_context.action = EventAction.CONTINUE  # 事件继续，交付给下个插件或默认逻辑
-        finally:
-            e_context['reply'] = reply
-
-    def get_help_text(self, **kwargs):
-        if not conf().get('image_create_prefix'):
-            return "画图功能未启用"
-        else:
-            trigger = conf()['image_create_prefix'][0]
-        help_text = f"请使用<{trigger}[关键词1] [关键词2]...:提示语>的格式作画，如\"{trigger}横版 高清:cat\"\n"
-        help_text += "目前可用关键词：\n"
-        for rule in self.rules:
-            keywords = [f"[{keyword}]" for keyword in rule['keywords']]
-            help_text += f"{','.join(keywords)}"
-            if "desc" in rule:
-                help_text += f"-{rule['desc']}\n"
-            else:
-                help_text += "\n"
-        return help_text

plugins/source.json

@@ -0,0 +1,8 @@
+{
+    "repo": {
+        "sdwebui": {
+            "url": "https://github.com/lanvent/plugin_sdwebui.git",
+            "desc": "利用stable-diffusion画图的插件"
+        }
+    }
+}

plugins/tool/README.md

@@ -0,0 +1,72 @@
+## 插件描述
+一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力   
+使用该插件需在触发机器人回复条件时，在对话内容前加$tool  
+### 本插件所有工具同步存放至专用仓库：[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)
+  
+  
+## 使用说明
+使用该插件后将默认使用4个工具, 无需额外配置长期生效： 
+### 1. python 
+###### python解释器，使用它来解释执行python指令，可以配合你想要chatgpt生成的代码输出结果或执行事务
+  
+### 2. requests
+###### 往往用来获取某个网站具体内容，结果可能会被反爬策略影响
+
+### 3. terminal
+###### 在你运行的电脑里执行shell命令，可以配合你想要chatgpt生成的代码使用，给予自然语言控制手段
+
+### 4. meteo-weather
+###### 回答你有关天气的询问, 需要获取时间、地点上下文信息，本工具使用了[meteo open api](https://open-meteo.com/)
+注：该工具需提供时间，地点信息，获取的数据不保证准确性
+
+## 使用本插件对话（prompt）技巧 
+### 1. 有指引的询问 
+#### 例如：
+- 总结这个链接的内容 https://github.com/goldfishh/chatgpt-tool-hub 
+- 使用Terminal执行curl cip.cc
+- 使用python查询今天日期
+  
+### 2. 使用搜索引擎工具
+- 如果有搜索工具就能让chatgpt获取到你的未传达清楚的上下文信息，比如chatgpt不知道你的地理位置，现在时间等，所以无法查询到天气
+  
+## 其他工具
+
+### 5. wikipedia
+###### 可以回答你想要知道确切的人事物
+
+### 6. news *
+###### 从全球 80,000 多个信息源中获取当前和历史新闻文章
+
+### 7. bing-search *
+###### bing搜索引擎，从此你不用再烦恼搜索要用哪些关键词
+
+### 8. wolfram-alpha *
+###### 知识搜索引擎、科学问答系统，常用于专业学科计算
+
+###### 注1：带*工具需要获取api-key才能使用，部分工具需要外网支持   
+#### [申请方法](https://github.com/goldfishh/chatgpt-tool-hub/blob/master/docs/apply_optional_tool.md)
+  
+## config.json 配置说明
+###### 默认工具无需配置，其它工具需手动配置，一个例子：
+```json
+{
+  "tools": ["wikipedia"],
+  "kwargs": {
+      "top_k_results": 2,
+      "no_default": false,
+      "model_name": "gpt-3.5-turbo"
+  }
+}
+```
+注：config.json文件非必须，未创建仍可使用本tool    
+- `tools`：本插件初始化时加载的工具, 目前可选集：["wikipedia", "wolfram-alpha", "bing-search", "google-search", "news"]，其中后4个工具需要申请服务api
+- `kwargs`：工具执行时的配置，一般在这里存放api-key，或环境配置
+  - `no_default`: 用于配置默认加载4个工具的行为，如果为true则仅使用tools列表工具，不加载默认工具
+  - `top_k_results`: 控制所有有关搜索的工具返回条目数，数字越高则参考信息越多，但无用信息可能干扰判断，该值一般为2
+  - `model_name`: 用于控制tool插件底层使用的llm模型，目前暂未测试3.5以外的模型，一般保持默认
+  
+  
+## 备注
+- 强烈建议申请搜索工具搭配使用，推荐bing-search
+- 虽然我会有意加入一些限制，但请不要使用本插件做危害他人的事情，请提前了解清楚某些内容是否会违反相关规定，建议提前做好过滤
+- 未来一段时间我会实现一些有意思的工具，比如stable diffusion 中文prompt翻译、cv方向的模型推理，欢迎有想法的朋友关注，一起扩展这个项目

plugins/tool/__init__.py

@@ -0,0 +1 @@
+from .tool import *

plugins/tool/config.json.template

@@ -0,0 +1,8 @@
+{
+  "tools": ["python", "requests", "terminal", "meteo-weather"],
+  "kwargs": {
+      "top_k_results": 2,
+      "no_default": false,
+      "model_name": "gpt-3.5-turbo"
+  }
+}

plugins/tool/tool.py

@@ -0,0 +1,156 @@
+import json
+import os
+
+from chatgpt_tool_hub.apps import load_app
+from chatgpt_tool_hub.apps.app import App
+from chatgpt_tool_hub.tools.all_tool_list import get_all_tool_names
+import plugins
+from bridge.bridge import Bridge
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from common import const
+from common.log import logger
+from config import conf
+from plugins import *
+
+
+@plugins.register(name="tool", desc="Arming your ChatGPT bot with various tools", version="0.3", author="goldfishh", desire_priority=0)
+class Tool(Plugin):
+    def __init__(self):
+        super().__init__()
+        self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+
+        self.app = self._reset_app()
+
+        logger.info("[tool] inited")
+
+    def get_help_text(self, verbose=False, **kwargs):
+        help_text = "这是一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力。"
+        if not verbose:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text += "使用说明：\n"
+        help_text += f"{trigger_prefix}tool "+"命令: 根据给出的{命令}使用一些可用工具尽力为你得到结果。\n"
+        help_text += f"{trigger_prefix}tool reset: 重置工具。\n"
+        return help_text
+
+    def on_handle_context(self, e_context: EventContext):
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        # 暂时不支持未来扩展的bot
+        if Bridge().get_bot_type("chat") not in (const.CHATGPT, const.OPEN_AI, const.CHATGPTONAZURE):
+            return
+
+        content = e_context['context'].content
+        content_list = e_context['context'].content.split(maxsplit=1)
+
+        if not content or len(content_list) < 1:
+            e_context.action = EventAction.CONTINUE
+            return
+
+        logger.debug("[tool] on_handle_context. content: %s" % content)
+        reply = Reply()
+        reply.type = ReplyType.TEXT
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        # todo: 有些工具必须要api-key，需要修改config文件，所以这里没有实现query增删tool的功能
+        if content.startswith(f"{trigger_prefix}tool"):
+            if len(content_list) == 1:
+                logger.debug("[tool]: get help")
+                reply.content = self.get_help_text()
+                e_context['reply'] = reply
+                e_context.action = EventAction.BREAK_PASS
+                return
+            elif len(content_list) > 1:
+                if content_list[1].strip() == "reset":
+                    logger.debug("[tool]: reset config")
+                    self.app = self._reset_app()
+                    reply.content = "重置工具成功"
+                    e_context['reply'] = reply
+                    e_context.action = EventAction.BREAK_PASS
+                    return
+                elif content_list[1].startswith("reset"):
+                    logger.debug("[tool]: remind")
+                    e_context['context'].content = "请你随机用一种聊天风格，提醒用户：如果想重置tool插件，reset之后不要加任何字符"
+
+                    e_context.action = EventAction.BREAK
+                    return
+
+                query = content_list[1].strip()
+
+                # Don't modify bot name
+                all_sessions = Bridge().get_bot("chat").sessions
+                user_session = all_sessions.session_query(query, e_context['context']['session_id']).messages
+
+                # chatgpt-tool-hub will reply you with many tools
+                logger.debug("[tool]: just-go")
+                try:
+                    _reply = self.app.ask(query, user_session)
+                    e_context.action = EventAction.BREAK_PASS
+                    all_sessions.session_reply(_reply, e_context['context']['session_id'])
+                except Exception as e:
+                    logger.exception(e)
+                    logger.error(str(e))
+
+                    e_context['context'].content = "请你随机用一种聊天风格，提醒用户：这个问题tool插件暂时无法处理"
+                    reply.type = ReplyType.ERROR
+                    e_context.action = EventAction.BREAK
+                    return
+
+                reply.content = _reply
+                e_context['reply'] = reply
+        return
+
+    def _read_json(self) -> dict:
+        curdir = os.path.dirname(__file__)
+        config_path = os.path.join(curdir, "config.json")
+        tool_config = {
+            "tools": [],
+            "kwargs": {}
+        }
+        if not os.path.exists(config_path):
+            return tool_config
+        else:
+            with open(config_path, "r") as f:
+                tool_config = json.load(f)
+        return tool_config
+
+    def _build_tool_kwargs(self, kwargs: dict):
+        tool_model_name = kwargs.get("model_name")
+
+        return {
+            "openai_api_key": conf().get("open_ai_api_key", ""),
+            "proxy": conf().get("proxy", ""),
+            # note: 目前tool暂未对其他模型测试，但这里仍对配置来源做了优先级区分，一般插件配置可覆盖全局配置
+            "model_name": tool_model_name if tool_model_name else conf().get("model", "gpt-3.5-turbo"),
+            "no_default": kwargs.get("no_default", False),
+            "top_k_results": kwargs.get("top_k_results", 2),
+            # for news tool
+            "news_api_key": kwargs.get("news_api_key", ""),
+            # for bing-search tool
+            "bing_subscription_key": kwargs.get("bing_subscription_key", ""),
+            # for google-search tool
+            "google_api_key": kwargs.get("google_api_key", ""),
+            "google_cse_id": kwargs.get("google_cse_id", ""),
+            # for searxng-search tool
+            "searx_host": kwargs.get("searx_host", ""),
+            # for wolfram-alpha tool
+            "wolfram_alpha_appid": kwargs.get("wolfram_alpha_appid", ""),
+        }
+
+    def _filter_tool_list(self, tool_list: list):
+        valid_list = []
+        for tool in tool_list:
+            if tool in get_all_tool_names():
+                valid_list.append(tool)
+            else:
+                logger.warning("[tool] filter invalid tool: " + repr(tool))
+        return valid_list
+
+    def _reset_app(self) -> App:
+        tool_config = self._read_json()
+
+        # filter not support tool
+        tool_list = self._filter_tool_list(tool_config.get("tools", []))
+
+        return load_app(tools_list=tool_list, **self._build_tool_kwargs(tool_config.get("kwargs", {})))

requirements-optional.txt

@@ -0,0 +1,24 @@
+tiktoken>=0.3.2 # openai calculate token
+
+#voice
+pydub>=0.25.1 # need ffmpeg
+SpeechRecognition # google speech to text
+gTTS>=2.3.1 # google text to speech
+pyttsx3>=2.90 # pytsx text to speech
+baidu_aip>=4.16.10 # baidu voice
+# azure-cognitiveservices-speech # azure voice
+
+#install plugin
+dulwich
+
+# wechaty
+wechaty>=0.10.7
+wechaty_puppet>=0.4.23
+pysilk_mod>=1.6.0 # needed by send voice
+
+# wechatmp
+web.py
+
+# chatgpt-tool-hub plugin
+--extra-index-url https://pypi.python.org/simple
+chatgpt_tool_hub>=0.3.5

requirements.txt

@@ -1,3 +1,6 @@
-itchat-uos==1.5.0.dev0
-openai
-wechaty
+openai>=0.27.2
+HTMLParser>=0.0.2
+PyQRCode>=1.2.1
+qrcode>=7.4.2
+requests>=2.28.2
+chardet>=5.1.0

voice/audio_convert.py

@@ -0,0 +1,104 @@
+import shutil
+import wave
+import pysilk
+from pydub import AudioSegment
+
+sil_supports=[8000, 12000, 16000, 24000, 32000, 44100, 48000] # slk转wav时，支持的采样率
+def find_closest_sil_supports(sample_rate):
+    """
+    找到最接近的支持的采样率
+    """
+    if sample_rate in sil_supports:
+        return sample_rate
+    closest = 0
+    mindiff = 9999999
+    for rate in sil_supports:
+        diff = abs(rate - sample_rate)
+        if diff < mindiff:
+            closest = rate
+            mindiff = diff
+    return closest
+
+def get_pcm_from_wav(wav_path):
+    """
+    从 wav 文件中读取 pcm
+
+    :param wav_path: wav 文件路径
+    :returns: pcm 数据
+    """
+    wav = wave.open(wav_path, "rb")
+    return wav.readframes(wav.getnframes())
+
+def any_to_wav(any_path, wav_path):
+    """
+    把任意格式转成wav文件
+    """
+    if any_path.endswith('.wav'):
+        shutil.copy2(any_path, wav_path)
+        return
+    if any_path.endswith('.sil') or any_path.endswith('.silk') or any_path.endswith('.slk'):
+        return sil_to_wav(any_path, wav_path)
+    audio = AudioSegment.from_file(any_path)
+    audio.export(wav_path, format="wav")
+
+def any_to_sil(any_path, sil_path):
+    """
+    把任意格式转成sil文件
+    """
+    if any_path.endswith('.sil') or any_path.endswith('.silk') or any_path.endswith('.slk'):
+        shutil.copy2(any_path, sil_path)
+        return 10000
+    if any_path.endswith('.wav'):
+        return pcm_to_sil(any_path, sil_path)
+    if any_path.endswith('.mp3'):
+        return mp3_to_sil(any_path, sil_path)
+    raise NotImplementedError("Not support file type: {}".format(any_path))
+
+def mp3_to_wav(mp3_path, wav_path):
+    """
+    把mp3格式转成pcm文件
+    """
+    audio = AudioSegment.from_mp3(mp3_path)
+    audio.export(wav_path, format="wav")
+
+def pcm_to_sil(pcm_path, silk_path):
+    """
+    wav 文件转成 silk
+    return 声音长度，毫秒
+    """
+    audio = AudioSegment.from_wav(pcm_path)
+    rate = find_closest_sil_supports(audio.frame_rate)
+    # Convert to PCM_s16
+    pcm_s16 = audio.set_sample_width(2)
+    pcm_s16 = pcm_s16.set_frame_rate(rate)
+    wav_data = pcm_s16.raw_data
+    silk_data = pysilk.encode(
+        wav_data, data_rate=rate, sample_rate=rate)
+    with open(silk_path, "wb") as f:
+        f.write(silk_data)
+    return audio.duration_seconds * 1000
+
+def mp3_to_sil(mp3_path, silk_path):
+    """
+    mp3 文件转成 silk
+    return 声音长度，毫秒
+    """
+    audio = AudioSegment.from_mp3(mp3_path)
+    rate = find_closest_sil_supports(audio.frame_rate)
+    # Convert to PCM_s16
+    pcm_s16 = audio.set_sample_width(2)
+    pcm_s16 = pcm_s16.set_frame_rate(rate)
+    wav_data = pcm_s16.raw_data
+    silk_data = pysilk.encode(wav_data, data_rate=rate, sample_rate=rate)
+    # Save the silk file
+    with open(silk_path, "wb") as f:
+        f.write(silk_data)
+    return audio.duration_seconds * 1000
+
+def sil_to_wav(silk_path, wav_path, rate: int = 24000):
+    """
+    silk 文件转 wav
+    """
+    wav_data = pysilk.decode_file(silk_path, to_wav=True, sample_rate=rate)
+    with open(wav_path, "wb") as f:
+        f.write(wav_data)

voice/azure/azure_voice.py

@@ -0,0 +1,68 @@
+
+"""
+azure voice service
+"""
+import json
+import os
+import time
+import azure.cognitiveservices.speech as speechsdk
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+from config import conf
+"""
+Azure voice
+主目录设置文件中需填写azure_voice_api_key和azure_voice_region
+
+查看可用的 voice： https://speech.microsoft.com/portal/voicegallery
+
+"""
+
+class AzureVoice(Voice):
+
+    def __init__(self):
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            config = None
+            if not os.path.exists(config_path): #如果没有配置文件，创建本地配置文件
+                config = { "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural", "speech_recognition_language": "zh-CN"}
+                with open(config_path, "w") as fw:
+                    json.dump(config, fw, indent=4)
+            else:
+                with open(config_path, "r") as fr:
+                    config = json.load(fr)
+            self.api_key = conf().get('azure_voice_api_key')
+            self.api_region = conf().get('azure_voice_region')
+            self.speech_config = speechsdk.SpeechConfig(subscription=self.api_key, region=self.api_region)
+            self.speech_config.speech_synthesis_voice_name = config["speech_synthesis_voice_name"]
+            self.speech_config.speech_recognition_language = config["speech_recognition_language"]
+        except Exception as e:
+            logger.warn("AzureVoice init failed: %s, ignore " % e)
+
+    def voiceToText(self, voice_file):
+        audio_config = speechsdk.AudioConfig(filename=voice_file)
+        speech_recognizer = speechsdk.SpeechRecognizer(speech_config=self.speech_config, audio_config=audio_config)
+        result = speech_recognizer.recognize_once()
+        if result.reason == speechsdk.ResultReason.RecognizedSpeech:
+            logger.info('[Azure] voiceToText voice file name={} text={}'.format(voice_file, result.text))
+            reply = Reply(ReplyType.TEXT, result.text)
+        else:
+            logger.error('[Azure] voiceToText error, result={}'.format(result))
+            reply = Reply(ReplyType.ERROR, "抱歉，语音识别失败")
+        return reply
+
+    def textToVoice(self, text):
+        fileName = TmpDir().path() + 'reply-' + str(int(time.time())) + '.wav'
+        audio_config = speechsdk.AudioConfig(filename=fileName)
+        speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=self.speech_config, audio_config=audio_config)
+        result = speech_synthesizer.speak_text(text)
+        if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
+            logger.info(
+                '[Azure] textToVoice text={} voice file name={}'.format(text, fileName))
+            reply = Reply(ReplyType.VOICE, fileName)
+        else:
+            logger.error('[Azure] textToVoice error, result={}'.format(result))
+            reply = Reply(ReplyType.ERROR, "抱歉，语音合成失败")
+        return reply

voice/azure/config.json.template

@@ -0,0 +1,4 @@
+{
+    "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural",
+    "speech_recognition_language": "zh-CN"
+}

voice/baidu/README.md

@@ -0,0 +1,55 @@
+## 说明
+百度语音识别与合成参数说明
+百度语音依赖，经常会出现问题，可能就是缺少依赖：
+pip install baidu-aip
+pip install pydub
+pip install pysilk
+还有ffmpeg，不同系统安装方式不同
+
+系统中收到的语音文件为mp3格式（wx）或者sil格式（wxy），如果要识别需要转换为pcm格式，转换后的文件为16k采样率，单声道，16bit的pcm文件
+发送时又需要（wx）转换为mp3格式，转换后的文件为16k采样率，单声道，16bit的pcm文件,（wxy）转换为sil格式,还要计算声音长度，发送时需要带上声音长度
+这些事情都在audio_convert.py中封装了，直接调用即可
+
+
+参数说明
+识别参数
+https://ai.baidu.com/ai-doc/SPEECH/Vk38lxily
+合成参数
+https://ai.baidu.com/ai-doc/SPEECH/Gk38y8lzk
+
+## 使用说明
+分两个地方配置
+
+1、对于def voiceToText(self, filename)函数中调用的百度语音识别API,中接口调用asr（参数）这个配置见CHATGPT-ON-WECHAT工程目录下的`config.json`文件和config.py文件。
+参数	    可需	描述
+app_id    必填	应用的APPID
+api_key  必填	应用的APIKey
+secret_key  必填	应用的SecretKey
+dev_pid	    必填	语言选择,填写语言对应的dev_pid值
+
+2、对于def textToVoice(self, text)函数中调用的百度语音合成API,中接口调用synthesis（参数）在本目录下的`config.json`文件中进行配置。
+参数	    可需	描述
+tex	        必填	合成的文本，使用UTF-8编码，请注意文本长度必须小于1024字节   
+lan	        必填	固定值zh。语言选择,目前只有中英文混合模式，填写固定值zh
+spd	        选填	语速，取值0-15，默认为5中语速
+pit	        选填	音调，取值0-15，默认为5中语调
+vol	        选填	音量，取值0-15，默认为5中音量（取值为0时为音量最小值，并非为无声）
+per（基础音库）	选填	度小宇=1，度小美=0，度逍遥（基础）=3，度丫丫=4
+per（精品音库）	选填	度逍遥（精品）=5003，度小鹿=5118，度博文=106，度小童=110，度小萌=111，度米朵=103，度小娇=5
+aue	        选填	3为mp3格式(默认)； 4为pcm-16k；5为pcm-8k；6为wav（内容同pcm-16k）; 注意aue=4或者6是语音识别要求的格式，但是音频内容不是语音识别要求的自然人发音，所以识别效果会受影响。
+
+关于per参数的说明，注意您购买的哪个音库，就填写哪个音库的参数，否则会报错。如果您购买的是基础音库，那么per参数只能填写0到4，如果您购买的是精品音库，那么per参数只能填写5003，5118，106,110,111,103,5其他的都会报错。
+### 配置文件
+  
+将文件夹中`config.json.template`复制为`config.json`。
+
+``` json
+    {
+    "lang": "zh", 
+    "ctp": 1,
+    "spd": 5, 
+    "pit": 5,
+    "vol": 5,
+    "per": 0
+    }
+```

voice/baidu/baidu_voice.py

@@ -2,35 +2,89 @@
 """
 baidu voice service
 """
+import json
+import os
 import time
 from aip import AipSpeech
 from bridge.reply import Reply, ReplyType
 from common.log import logger
 from common.tmp_dir import TmpDir
 from voice.voice import Voice
+from voice.audio_convert import get_pcm_from_wav
 from config import conf
+"""
+    百度的语音识别API.
+    dev_pid:
+        - 1936: 普通话远场
+        - 1536：普通话(支持简单的英文识别)
+        - 1537：普通话(纯中文识别)
+        - 1737：英语
+        - 1637：粤语
+        - 1837：四川话
+    要使用本模块, 首先到 yuyin.baidu.com 注册一个开发者账号,
+    之后创建一个新应用, 然后在应用管理的"查看key"中获得 API Key 和 Secret Key
+    然后在 config.json 中填入这两个值, 以及 app_id, dev_pid
+    """
+
 
 class BaiduVoice(Voice):
-    APP_ID = conf().get('baidu_app_id')
-    API_KEY = conf().get('baidu_api_key')
-    SECRET_KEY = conf().get('baidu_secret_key')
-    client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)
-    
-    def __init__(self):
-        pass
 
+    def __init__(self):
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            bconf = None
+            if not os.path.exists(config_path): #如果没有配置文件，创建本地配置文件
+                bconf = { "lang": "zh", "ctp": 1, "spd": 5,
+                         "pit": 5, "vol": 5, "per": 0}
+                with open(config_path, "w") as fw:
+                    json.dump(bconf, fw, indent=4)
+            else:
+                with open(config_path, "r") as fr:
+                    bconf = json.load(fr)
+                    
+            self.app_id = conf().get('baidu_app_id')
+            self.api_key = conf().get('baidu_api_key')
+            self.secret_key = conf().get('baidu_secret_key')
+            self.dev_id = conf().get('baidu_dev_pid')
+            self.lang = bconf["lang"]
+            self.ctp = bconf["ctp"]
+            self.spd = bconf["spd"]
+            self.pit = bconf["pit"]
+            self.vol = bconf["vol"]
+            self.per = bconf["per"]
+            
+            self.client = AipSpeech(self.app_id, self.api_key, self.secret_key)
+        except Exception as e:
+            logger.warn("BaiduVoice init failed: %s, ignore " % e)
+
+        
     def voiceToText(self, voice_file):
-        pass
+        # 识别本地文件
+        logger.debug('[Baidu] voice file name={}'.format(voice_file))
+        pcm = get_pcm_from_wav(voice_file)
+        res = self.client.asr(pcm, "pcm", 16000, {"dev_pid": self.dev_id})
+        if res["err_no"] == 0:
+            logger.info("百度语音识别到了：{}".format(res["result"]))
+            text = "".join(res["result"])
+            reply = Reply(ReplyType.TEXT, text)
+        else:
+            logger.info("百度语音识别出错了: {}".format(res["err_msg"]))
+            if res["err_msg"] == "request pv too much":
+                logger.info("  出现这个原因很可能是你的百度语音服务调用量超出限制，或未开通付费")
+            reply = Reply(ReplyType.ERROR,
+                          "百度语音识别出错了；{0}".format(res["err_msg"]))
+        return reply
 
     def textToVoice(self, text):
-        result = self.client.synthesis(text, 'zh', 1, {
-            'spd': 5, 'pit': 5, 'vol': 5, 'per': 111
-        })
+        result = self.client.synthesis(text, self.lang, self.ctp, {
+            'spd': self.spd, 'pit': self.pit, 'vol': self.vol, 'per': self.per})
         if not isinstance(result, dict):
-            fileName = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
+            fileName = TmpDir().path() + 'reply-' + str(int(time.time())) + '.mp3'
             with open(fileName, 'wb') as f:
                 f.write(result)
-            logger.info('[Baidu] textToVoice text={} voice file name={}'.format(text, fileName))
+            logger.info(
+                '[Baidu] textToVoice text={} voice file name={}'.format(text, fileName))
             reply = Reply(ReplyType.VOICE, fileName)
         else:
             logger.error('[Baidu] textToVoice error={}'.format(result))

voice/baidu/config.json.template

@@ -0,0 +1,8 @@
+    {
+    "lang": "zh", 
+    "ctp": 1,
+    "spd": 5, 
+    "pit": 5,
+    "vol": 5,
+    "per": 0
+    }

voice/google/google_voice.py

@@ -3,12 +3,10 @@
 google voice service
 """
 
-import pathlib
-import subprocess
 import time
-from bridge.reply import Reply, ReplyType
 import speech_recognition
-import pyttsx3
+from gtts import gTTS
+from bridge.reply import Reply, ReplyType
 from common.log import logger
 from common.tmp_dir import TmpDir
 from voice.voice import Voice
@@ -16,22 +14,12 @@ from voice.voice import Voice
 
 class GoogleVoice(Voice):
     recognizer = speech_recognition.Recognizer()
-    engine = pyttsx3.init()
 
     def __init__(self):
-        # 语速
-        self.engine.setProperty('rate', 125)
-        # 音量
-        self.engine.setProperty('volume', 1.0)
-        # 0为男声，1为女声
-        voices = self.engine.getProperty('voices')
-        self.engine.setProperty('voice', voices[1].id)
+        pass
 
     def voiceToText(self, voice_file):
-        new_file = voice_file.replace('.mp3', '.wav')
-        subprocess.call('ffmpeg -i ' + voice_file +
-                        ' -acodec pcm_s16le -ac 1 -ar 16000 ' + new_file, shell=True)
-        with speech_recognition.AudioFile(new_file) as source:
+        with speech_recognition.AudioFile(voice_file) as source:
             audio = self.recognizer.record(source)
         try:
             text = self.recognizer.recognize_google(audio, language='zh-CN')
@@ -46,12 +34,12 @@ class GoogleVoice(Voice):
             return reply
     def textToVoice(self, text):
         try:
-            textFile = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
-            self.engine.save_to_file(text, textFile)
-            self.engine.runAndWait()
+            mp3File = TmpDir().path() + 'reply-' + str(int(time.time())) + '.mp3'
+            tts = gTTS(text=text, lang='zh')
+            tts.save(mp3File)            
             logger.info(
-                '[Google] textToVoice text={} voice file name={}'.format(text, textFile))
-            reply = Reply(ReplyType.VOICE, textFile)
+                '[Google] textToVoice text={} voice file name={}'.format(text, mp3File))
+            reply = Reply(ReplyType.VOICE, mp3File)
         except Exception as e:
             reply = Reply(ReplyType.ERROR, str(e))
         finally:

voice/openai/openai_voice.py

@@ -28,6 +28,3 @@ class OpenaiVoice(Voice):
             reply = Reply(ReplyType.ERROR, str(e))
         finally:
             return reply
-
-    def textToVoice(self, text):
-        pass

voice/pytts/pytts_voice.py

@@ -0,0 +1,37 @@
+
+"""
+pytts voice service (offline)
+"""
+
+import time
+import pyttsx3
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+
+
+class PyttsVoice(Voice):
+    engine = pyttsx3.init()
+
+    def __init__(self):
+        # 语速
+        self.engine.setProperty('rate', 125)
+        # 音量
+        self.engine.setProperty('volume', 1.0)
+        for voice in self.engine.getProperty('voices'):
+            if "Chinese" in voice.name:
+                self.engine.setProperty('voice', voice.id)
+
+    def textToVoice(self, text):
+        try:
+            wavFile = TmpDir().path() + 'reply-' + str(int(time.time())) + '.wav'
+            self.engine.save_to_file(text, wavFile)
+            self.engine.runAndWait()
+            logger.info(
+                '[Pytts] textToVoice text={} voice file name={}'.format(text, wavFile))
+            reply = Reply(ReplyType.VOICE, wavFile)
+        except Exception as e:
+            reply = Reply(ReplyType.ERROR, str(e))
+        finally:
+            return reply

voice/voice_factory.py

@@ -17,4 +17,10 @@ def create_voice(voice_type):
     elif voice_type == 'openai':
         from voice.openai.openai_voice import OpenaiVoice
         return OpenaiVoice()
+    elif voice_type == 'pytts':
+        from voice.pytts.pytts_voice import PyttsVoice
+        return PyttsVoice()
+    elif voice_type == 'azure':
+        from voice.azure.azure_voice import AzureVoice
+        return AzureVoice()
     raise RuntimeError