完善tool文档 & 增加tool过滤、tool参数构建 (#751)

wechaty支持插件

.github/ISSUE_TEMPLATE.md

@@ -1,9 +1,12 @@
 ### 前置确认
 
 1. 网络能够访问openai接口
-2. python 已安装：版本在 3.7 ~ 3.10 之间，依赖已安装
-3. 在已有 issue 中未搜索到类似问题
-4. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
+2. python 已安装：版本在 3.7 ~ 3.10 之间
+3. `git pull` 拉取最新代码
+4. 执行`pip3 install -r requirements.txt`，检查依赖是否满足
+5. 拓展功能请执行`pip3 install -r requirements-optional.txt`，检查依赖是否满足
+6. 在已有 issue 中未搜索到类似问题
+7. [FAQS](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) 中无类似问题
 
 
 ### 问题描述
@@ -16,7 +19,7 @@
 ### 终端日志 (如有报错)
 
 ```
-[在此处粘贴终端日志]
+[在此处粘贴终端日志, 可在主目录下`run.log`文件中找到]
 ```
 
 

.github/workflows/deploy-image.yml

@@ -0,0 +1,59 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# GitHub recommends pinning actions to a commit SHA.
+# To get a newer version, you will need to update the SHA.
+# You can also reference a tag or branch, but the action may change without warning.
+
+name: Create and publish a Docker image
+
+on:
+  push:
+    branches: ['master']
+  create:
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Log in to the Container registry
+        uses: docker/login-action@v2
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v4
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v3
+        with:
+          context: .
+          push: true
+          file: ./docker/Dockerfile.latest
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+      
+      - uses: actions/delete-package-versions@v4
+        with: 
+          package-name: 'chatgpt-on-wechat'
+          package-type: 'container'
+          min-versions-to-keep: 10
+          delete-only-untagged-versions: 'true'
+          token: ${{ secrets.GITHUB_TOKEN }}

.gitignore

@@ -10,3 +10,5 @@ nohup.out
 tmp
 plugins.json
 itchat.pkl
+*.log
+user_datas.pkl

Dockerfile

@@ -0,0 +1,3 @@
+FROM ghcr.io/zhayujie/chatgpt-on-wechat:latest
+
+ENTRYPOINT ["/entrypoint.sh"]

README.md

@@ -2,7 +2,7 @@
 
 > ChatGPT近期以强大的对话和信息整合能力风靡全网，可以写代码、改论文、讲故事，几乎无所不能，这让人不禁有个大胆的想法，能否用他的对话模型把我们的微信打造成一个智能机器人，可以在与好友对话中给出意想不到的回应，而且再也不用担心女朋友影响我们 ~~打游戏~~ 工作了。
 
- 
+
 基于ChatGPT的微信聊天机器人，通过 [ChatGPT](https://github.com/openai/openai-python) 接口生成对话内容，使用 [itchat](https://github.com/littlecodersh/ItChat) 实现微信消息的接收和自动回复。已实现的特性如下：
 
 - [x] **文本对话：** 接收私聊及群组中的微信消息，使用ChatGPT生成回复内容，完成自动回复
@@ -11,7 +11,11 @@
 - [x] **图片生成：** 支持根据描述生成图片，并自动发送至个人聊天或群聊
 - [x] **上下文记忆**：支持多轮对话记忆，且为每个好友维护独立的上下会话
 - [x] **语音识别：** 支持接收和处理语音消息，通过文字或语音回复
+- [x] **插件化：** 支持个性化功能插件，提供角色扮演、文字冒险游戏等预设插件
 
+> 快速部署:
+>
+>[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/qApznZ?referralCode=RC3znh)
 
 # 更新日志
 
@@ -60,8 +64,8 @@
 
 ### 2.运行环境
 
-支持 Linux、MacOS、Windows 系统（可在Linux服务器上长期运行)，同时需安装 `Python`。 
-> 建议Python版本在 3.7.1~3.9.X 之间，3.10及以上版本在 MacOS 可用，其他系统上不确定能否正常运行。
+支持 Linux、MacOS、Windows 系统（可在Linux服务器上长期运行)，同时需安装 `Python`。
+> 建议Python版本在 3.7.1~3.9.X 之间，推荐3.8版本，3.10及以上版本在 MacOS 可用，其他系统上不确定能否正常运行。
 
 **(1) 克隆项目代码：**
 
@@ -71,17 +75,35 @@ cd chatgpt-on-wechat/
 ```
 
 **(2) 安装核心依赖 (必选)：**
+> 能够使用`itchat`创建机器人，并具有文字交流功能所需的最小依赖集合。
+```bash
+pip3 install -r requirements.txt
+```
+
+**(3) 拓展依赖 (可选，建议安装)：**
 
 ```bash
-pip3 install itchat-uos==1.5.0.dev0
-pip3 install --upgrade openai
+pip3 install -r requirements-optional.txt
 ```
-注：`itchat-uos`使用指定版本1.5.0.dev0，`openai`使用最新版本，需高于0.27.0。
+> 如果某项依赖安装失败请注释掉对应的行再继续。
 
-**(3) 拓展依赖 (可选)：**
+其中`tiktoken`要求`python`版本在3.8以上，它用于精确计算会话使用的tokens数量，强烈建议安装。
 
-语音识别及语音回复相关依赖：[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)。
 
+使用`google`或`baidu`语音识别需安装`ffmpeg`，
+
+默认的`openai`语音识别不需要安装`ffmpeg`。
+
+参考[#415](https://github.com/zhayujie/chatgpt-on-wechat/issues/415)
+
+使用`azure`语音功能需安装依赖（列在`requirements-optional.txt`内，但为便于`railway`部署已注释）:
+
+```bash
+pip3 install azure-cognitiveservices-speech
+```
+
+> 目前默认发布的镜像和`railway`部署，都基于`apline`，无法安装`azure`的依赖。若有需求请自行基于[`debian`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/docker/Dockerfile.debian.latest)打包。
+参考[文档](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/quickstarts/setup-platform?pivots=programming-language-python&tabs=linux%2Cubuntu%2Cdotnet%2Cjre%2Cmaven%2Cnodejs%2Cmac%2Cpypi)
 
 ## 配置
 
@@ -107,6 +129,7 @@ pip3 install --upgrade openai
   "image_create_prefix": ["画", "看", "找"],                   # 开启图片回复的前缀
   "conversation_max_tokens": 1000,                            # 支持上下文记忆的最多字符数
   "speech_recognition": false,                                # 是否开启语音识别
+  "group_speech_recognition": false,                          # 是否开启群组语音识别
   "use_azure_chatgpt": false,                                 # 是否使用Azure ChatGPT service代替openai ChatGPT service. 当设置为true时需要设置 open_ai_api_base，如 https://xxx.openai.azure.com/
   "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述,
 }
@@ -127,8 +150,9 @@ pip3 install --upgrade openai
 
 **3.语音识别**
 
-+ 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，目前只支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复)；
-+ 添加 `"voice_reply_voice": true` 将开启语音回复语音，但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
++ 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，该参数仅支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复，支持语音触发画图)；
++ 添加 `"group_speech_recognition": true` 将开启群组语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，参数仅支持群聊 (会匹配group_chat_prefix和group_chat_keyword, 支持语音触发画图)；
++ 添加 `"voice_reply_voice": true` 将开启语音回复语音（同时作用于私聊和群聊），但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
 
 **4.其他配置**
 
@@ -143,6 +167,7 @@ pip3 install --upgrade openai
 + `hot_reload`: 程序退出后，暂存微信扫码状态，默认关闭。
 + `character_desc` 配置中保存着你对机器人说的一段话，他会记住这段话并作为他的设定，你可以为他定制任何人格      (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
 
+**所有可选的配置项均在该[文件](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
 
 ## 运行
 
@@ -177,14 +202,11 @@ nohup python3 app.py & tail -f nohup.out          # 在后台运行程序并通
 
 参考文档 [Docker部署](https://github.com/limccn/chatgpt-on-wechat/wiki/Docker%E9%83%A8%E7%BD%B2)   (Contributed by [limccn](https://github.com/limccn))。
 
-### 4. Railway部署
-[Use with Railway](#use-with-railway)(PaaS, Free, Stable, ✅Recommended)
-> Railway offers $5 (500 hours) of runtime per month
-1. Click the [Railway](https://railway.app/) button to go to the Railway homepage
-2. Click the `Start New Project` button.
-3. Click the `Deploy from Github repo` button.
-4. Choose your repo (you can fork this repo firstly)
-5. Set environment variable to override settings in config-template.json, such as: model, open_ai_api_base, open_ai_api_key, use_azure_chatgpt etc.
+### 4. Railway部署(✅推荐)
+> Railway每月提供5刀和最多500小时的免费额度。
+1. 进入 [Railway](https://railway.app/template/qApznZ?referralCode=RC3znh)。
+2. 点击 `Deploy Now` 按钮。
+3. 设置环境变量来重载程序运行的参数，例如`open_ai_api_key`, `character_desc`。
 
 ## 常见问题
 

app.py

@@ -1,20 +1,34 @@
 # encoding:utf-8
 
-import config
+import os
+from config import conf, load_config
 from channel import channel_factory
 from common.log import logger
-
 from plugins import *
+import signal
+import sys
+
+def sigterm_handler(_signo, _stack_frame):
+    conf().save_user_datas()
+    sys.exit(0)
 
 def run():
     try:
         # load config
-        config.load_config()
+        load_config()
+        # ctrl + c
+        signal.signal(signal.SIGINT, sigterm_handler)
+        # kill signal
+        signal.signal(signal.SIGTERM, sigterm_handler)
 
         # create channel
-        channel_name='wx'
+        channel_name=conf().get('channel_type', 'wx')
+        if channel_name == 'wxy':
+            os.environ['WECHATY_LOG']="warn"
+            # os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:9001'
+
         channel = channel_factory.create_channel(channel_name)
-        if channel_name=='wx':
+        if channel_name in ['wx','wxy','wechatmp']:
             PluginManager().load_plugins()
 
         # startup channel

bot/bot_factory.py

@@ -6,9 +6,9 @@ from common import const
 
 def create_bot(bot_type):
     """
-    create a channel instance
-    :param channel_type: channel type code
-    :return: channel instance
+    create a bot_type instance
+    :param bot_type: bot type code
+    :return: bot instance
     """
     if bot_type == const.BAIDU:
         # Baidu Unit对话接口

bot/chatgpt/chat_gpt_bot.py

@@ -1,6 +1,9 @@
 # encoding:utf-8
 
 from bot.bot import Bot
+from bot.chatgpt.chat_gpt_session import ChatGPTSession
+from bot.openai.open_ai_image import OpenAIImage
+from bot.session_manager import Session, SessionManager
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from config import conf, load_config
@@ -8,28 +11,30 @@ from common.log import logger
 from common.token_bucket import TokenBucket
 from common.expired_dict import ExpiredDict
 import openai
+import openai.error
 import time
 
-
 # OpenAI对话模型API (可用)
-class ChatGPTBot(Bot):
+class ChatGPTBot(Bot,OpenAIImage):
     def __init__(self):
+        super().__init__()
+        # set the default api_key
         openai.api_key = conf().get('open_ai_api_key')
         if conf().get('open_ai_api_base'):
             openai.api_base = conf().get('open_ai_api_base')
         proxy = conf().get('proxy')
-        self.sessions = SessionManager()
         if proxy:
             openai.proxy = proxy
         if conf().get('rate_limit_chatgpt'):
             self.tb4chatgpt = TokenBucket(conf().get('rate_limit_chatgpt', 20))
-        if conf().get('rate_limit_dalle'):
-            self.tb4dalle = TokenBucket(conf().get('rate_limit_dalle', 50))
+        
+        self.sessions = SessionManager(ChatGPTSession, model= conf().get("model") or "gpt-3.5-turbo")
 
     def reply(self, query, context=None):
         # acquire reply content
         if context.type == ContextType.TEXT:
-            logger.info("[OPEN_AI] query={}".format(query))
+            logger.info("[CHATGPT] query={}".format(query))
+
 
             session_id = context['session_id']
             reply = None
@@ -45,23 +50,25 @@ class ChatGPTBot(Bot):
                 reply = Reply(ReplyType.INFO, '配置已更新')
             if reply:
                 return reply
-            session = self.sessions.build_session_query(query, session_id)
-            logger.debug("[OPEN_AI] session query={}".format(session))
+            session = self.sessions.session_query(query, session_id)
+            logger.debug("[CHATGPT] session query={}".format(session.messages))
+
+            api_key = context.get('openai_api_key')
 
             # if context.get('stream'):
             #     # reply in stream
             #     return self.reply_text_stream(query, new_query, session_id)
 
-            reply_content = self.reply_text(session, session_id, 0)
-            logger.debug("[OPEN_AI] new_query={}, session_id={}, reply_cont={}".format(session, session_id, reply_content["content"]))
+            reply_content = self.reply_text(session, session_id, api_key, 0)
+            logger.debug("[CHATGPT] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(session.messages, session_id, reply_content["content"], reply_content["completion_tokens"]))
             if reply_content['completion_tokens'] == 0 and len(reply_content['content']) > 0:
                 reply = Reply(ReplyType.ERROR, reply_content['content'])
             elif reply_content["completion_tokens"] > 0:
-                self.sessions.save_session(reply_content["content"], session_id, reply_content["total_tokens"])
+                self.sessions.session_reply(reply_content["content"], session_id, reply_content["total_tokens"])
                 reply = Reply(ReplyType.TEXT, reply_content["content"])
             else:
                 reply = Reply(ReplyType.ERROR, reply_content['content'])
-                logger.debug("[OPEN_AI] reply {} used 0 tokens.".format(reply_content))
+                logger.debug("[CHATGPT] reply {} used 0 tokens.".format(reply_content))
             return reply
 
         elif context.type == ContextType.IMAGE_CREATE:
@@ -84,9 +91,10 @@ class ChatGPTBot(Bot):
             "top_p":1,
             "frequency_penalty":conf().get('frequency_penalty', 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
             "presence_penalty":conf().get('presence_penalty', 0.0),  # [-2,2]之间，该值越大则更倾向于产生不同的内容
+            "request_timeout": conf().get('request_timeout', 30),  # 请求超时时间
         }
 
-    def reply_text(self, session, session_id, retry_count=0) -> dict:
+    def reply_text(self, session:ChatGPTSession, session_id, api_key, retry_count=0) -> dict:
         '''
         call openai's ChatCompletion to get the answer
         :param session: a conversation session
@@ -96,62 +104,42 @@ class ChatGPTBot(Bot):
         '''
         try:
             if conf().get('rate_limit_chatgpt') and not self.tb4chatgpt.get_token():
-                return {"completion_tokens": 0, "content": "提问太快啦，请休息一下再问我吧"}
+                raise openai.error.RateLimitError("RateLimitError: rate limit exceeded")
+            # if api_key == None, the default openai.api_key will be used
             response = openai.ChatCompletion.create(
-                messages=session, **self.compose_args()
+                api_key=api_key, messages=session.messages, **self.compose_args()
             )
             # logger.info("[ChatGPT] reply={}, total_tokens={}".format(response.choices[0]['message']['content'], response["usage"]["total_tokens"]))
             return {"total_tokens": response["usage"]["total_tokens"],
                     "completion_tokens": response["usage"]["completion_tokens"],
                     "content": response.choices[0]['message']['content']}
-        except openai.error.RateLimitError as e:
-            # rate limit exception
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(session, session_id, retry_count+1)
-            else:
-                return {"completion_tokens": 0, "content": "提问太快啦，请休息一下再问我吧"}
-        except openai.error.APIConnectionError as e:
-            # api connection exception
-            logger.warn(e)
-            logger.warn("[OPEN_AI] APIConnection failed")
-            return {"completion_tokens": 0, "content": "我连接不到你的网络"}
-        except openai.error.Timeout as e:
-            logger.warn(e)
-            logger.warn("[OPEN_AI] Timeout")
-            return {"completion_tokens": 0, "content": "我没有收到你的消息"}
         except Exception as e:
-            # unknown exception
-            logger.exception(e)
-            self.sessions.clear_session(session_id)
-            return {"completion_tokens": 0, "content": "请再问我一次吧"}
+            need_retry = retry_count < 2
+            result = {"completion_tokens": 0, "content": "我现在有点累了，等会再来吧"}
+            if isinstance(e, openai.error.RateLimitError):
+                logger.warn("[CHATGPT] RateLimitError: {}".format(e))
+                result['content'] = "提问太快啦，请休息一下再问我吧"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.Timeout):
+                logger.warn("[CHATGPT] Timeout: {}".format(e))
+                result['content'] = "我没有收到你的消息"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.APIConnectionError):
+                logger.warn("[CHATGPT] APIConnectionError: {}".format(e))
+                need_retry = False
+                result['content'] = "我连接不到你的网络"
+            else:
+                logger.warn("[CHATGPT] Exception: {}".format(e))
+                need_retry = False
+                self.sessions.clear_session(session_id)
 
-    def create_img(self, query, retry_count=0):
-        try:
-            if conf().get('rate_limit_dalle') and not self.tb4dalle.get_token():
-                return False, "请求太快了，请休息一下再问我吧"
-            logger.info("[OPEN_AI] image_query={}".format(query))
-            response = openai.Image.create(
-                prompt=query,    #图片描述
-                n=1,             #每次生成图片的数量
-                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
-            )
-            image_url = response['data'][0]['url']
-            logger.info("[OPEN_AI] image_url={}".format(image_url))
-            return True, image_url
-        except openai.error.RateLimitError as e:
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.create_img(query, retry_count+1)
+            if need_retry:
+                logger.warn("[CHATGPT] 第{}次重试".format(retry_count+1))
+                return self.reply_text(session, session_id, api_key, retry_count+1)
             else:
-                return False, "提问太快啦，请休息一下再问我吧"
-        except Exception as e:
-            logger.exception(e)
-            return False, str(e)
+                return result
 
 
 class AzureChatGPTBot(ChatGPTBot):
@@ -164,75 +152,4 @@ class AzureChatGPTBot(ChatGPTBot):
         args = super().compose_args()
         args["engine"] = args["model"]
         del(args["model"])
-        return args
-
-
-class SessionManager(object):
-    def __init__(self):
-        if conf().get('expires_in_seconds'):
-            sessions = ExpiredDict(conf().get('expires_in_seconds'))
-        else:
-            sessions = dict()
-        self.sessions = sessions
-
-    def build_session(self, session_id, system_prompt=None):
-        session = self.sessions.get(session_id, [])
-        if len(session) == 0:
-            if system_prompt is None:
-                system_prompt = conf().get("character_desc", "")
-            system_item = {'role': 'system', 'content': system_prompt}
-            session.append(system_item)
-            self.sessions[session_id] = session
-        return session
-
-    def build_session_query(self, query, session_id):
-        '''
-        build query with conversation history
-        e.g.  [
-            {"role": "system", "content": "You are a helpful assistant."},
-            {"role": "user", "content": "Who won the world series in 2020?"},
-            {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
-            {"role": "user", "content": "Where was it played?"}
-        ]
-        :param query: query content
-        :param session_id: session id
-        :return: query content with conversaction
-        '''
-        session = self.build_session(session_id)
-        user_item = {'role': 'user', 'content': query}
-        session.append(user_item)
-        return session
-
-    def save_session(self, answer, session_id, total_tokens):
-        max_tokens = conf().get("conversation_max_tokens")
-        if not max_tokens:
-            # default 3000
-            max_tokens = 1000
-        max_tokens = int(max_tokens)
-
-        session = self.sessions.get(session_id)
-        if session:
-            # append conversation
-            gpt_item = {'role': 'assistant', 'content': answer}
-            session.append(gpt_item)
-
-        # discard exceed limit conversation
-        self.discard_exceed_conversation(session, max_tokens, total_tokens)
-
-    def discard_exceed_conversation(self, session, max_tokens, total_tokens):
-        dec_tokens = int(total_tokens)
-        # logger.info("prompt tokens used={},max_tokens={}".format(used_tokens,max_tokens))
-        while dec_tokens > max_tokens:
-            # pop first conversation
-            if len(session) > 3:
-                session.pop(1)
-                session.pop(1)
-            else:
-                break
-            dec_tokens = dec_tokens - max_tokens
-
-    def clear_session(self, session_id):
-        self.sessions[session_id] = []
-
-    def clear_all_session(self):
-        self.sessions.clear()
+        return args

bot/chatgpt/chat_gpt_session.py

@@ -0,0 +1,79 @@
+from bot.session_manager import Session
+from common.log import logger
+'''
+    e.g.  [
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Who won the world series in 2020?"},
+        {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
+        {"role": "user", "content": "Where was it played?"}
+    ]
+'''
+class ChatGPTSession(Session):
+    def __init__(self, session_id, system_prompt=None, model= "gpt-3.5-turbo"):
+        super().__init__(session_id, system_prompt)
+        self.model = model
+        self.reset()
+    
+    def discard_exceeding(self, max_tokens, cur_tokens= None):
+        precise = True
+        try:
+            cur_tokens = num_tokens_from_messages(self.messages, self.model)
+        except Exception as e:
+            precise = False
+            if cur_tokens is None:
+                raise e
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
+        while cur_tokens > max_tokens:
+            if len(self.messages) > 2:
+                self.messages.pop(1)
+            elif len(self.messages) == 2 and self.messages[1]["role"] == "assistant":
+                self.messages.pop(1)
+                if precise:
+                    cur_tokens = num_tokens_from_messages(self.messages, self.model)
+                else:
+                    cur_tokens = cur_tokens - max_tokens
+                break
+            elif len(self.messages) == 2 and self.messages[1]["role"] == "user":
+                logger.warn("user message exceed max_tokens. total_tokens={}".format(cur_tokens))
+                break
+            else:
+                logger.debug("max_tokens={}, total_tokens={}, len(messages)={}".format(max_tokens, cur_tokens, len(self.messages)))
+                break
+            if precise:
+                cur_tokens = num_tokens_from_messages(self.messages, self.model)
+            else:
+                cur_tokens = cur_tokens - max_tokens
+        return cur_tokens
+    
+
+# refer to https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+def num_tokens_from_messages(messages, model):
+    """Returns the number of tokens used by a list of messages."""
+    import tiktoken
+    try:
+        encoding = tiktoken.encoding_for_model(model)
+    except KeyError:
+        logger.debug("Warning: model not found. Using cl100k_base encoding.")
+        encoding = tiktoken.get_encoding("cl100k_base")
+    if model == "gpt-3.5-turbo":
+        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
+    elif model == "gpt-4":
+        return num_tokens_from_messages(messages, model="gpt-4-0314")
+    elif model == "gpt-3.5-turbo-0301":
+        tokens_per_message = 4  # every message follows <|start|>{role/name}\n{content}<|end|>\n
+        tokens_per_name = -1  # if there's a name, the role is omitted
+    elif model == "gpt-4-0314":
+        tokens_per_message = 3
+        tokens_per_name = 1
+    else:
+        logger.warn(f"num_tokens_from_messages() is not implemented for model {model}. Returning num tokens assuming gpt-3.5-turbo-0301.")
+        return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0301")
+    num_tokens = 0
+    for message in messages:
+        num_tokens += tokens_per_message
+        for key, value in message.items():
+            num_tokens += len(encoding.encode(value))
+            if key == "name":
+                num_tokens += tokens_per_name
+    num_tokens += 3  # every reply is primed with <|start|>assistant<|message|>
+    return num_tokens

bot/openai/open_ai_bot.py

@@ -1,18 +1,23 @@
 # encoding:utf-8
 
 from bot.bot import Bot
+from bot.openai.open_ai_image import OpenAIImage
+from bot.openai.open_ai_session import OpenAISession
+from bot.session_manager import SessionManager
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from config import conf
 from common.log import logger
 import openai
+import openai.error
 import time
 
 user_session = dict()
 
 # OpenAI对话模型API (可用)
-class OpenAIBot(Bot):
+class OpenAIBot(Bot, OpenAIImage):
     def __init__(self):
+        super().__init__()
         openai.api_key = conf().get('open_ai_api_key')
         if conf().get('open_ai_api_base'):
             openai.api_base = conf().get('open_ai_api_base')
@@ -20,34 +25,45 @@ class OpenAIBot(Bot):
         if proxy:
             openai.proxy = proxy
 
+        self.sessions = SessionManager(OpenAISession, model= conf().get("model") or "text-davinci-003")
 
     def reply(self, query, context=None):
         # acquire reply content
         if context and context.type:
             if context.type == ContextType.TEXT:
                 logger.info("[OPEN_AI] query={}".format(query))
-                from_user_id = context['session_id']
+                session_id = context['session_id']
                 reply = None
                 if query == '#清除记忆':
-                    Session.clear_session(from_user_id)
+                    self.sessions.clear_session(session_id)
                     reply = Reply(ReplyType.INFO, '记忆已清除')
                 elif query == '#清除所有':
-                    Session.clear_all_session()
+                    self.sessions.clear_all_session()
                     reply = Reply(ReplyType.INFO, '所有人记忆已清除')
                 else:
-                    new_query = Session.build_session_query(query, from_user_id)
+                    session = self.sessions.session_query(query, session_id)
+                    new_query = str(session)
                     logger.debug("[OPEN_AI] session query={}".format(new_query))
 
-                    reply_content = self.reply_text(new_query, from_user_id, 0)
-                    logger.debug("[OPEN_AI] new_query={}, user={}, reply_cont={}".format(new_query, from_user_id, reply_content))
-                    if reply_content and query:
-                        Session.save_session(query, reply_content, from_user_id)
-                    reply = Reply(ReplyType.TEXT, reply_content)
+                    total_tokens, completion_tokens, reply_content = self.reply_text(new_query, session_id, 0)
+                    logger.debug("[OPEN_AI] new_query={}, session_id={}, reply_cont={}, completion_tokens={}".format(new_query, session_id, reply_content, completion_tokens))
+
+                    if total_tokens == 0 :
+                        reply = Reply(ReplyType.ERROR, reply_content)
+                    else:
+                        self.sessions.session_reply(reply_content, session_id, total_tokens)
+                        reply = Reply(ReplyType.TEXT, reply_content)
                 return reply
             elif context.type == ContextType.IMAGE_CREATE:
-                return self.create_img(query, 0)
+                ok, retstring = self.create_img(query, 0)
+                reply = None
+                if ok:
+                    reply = Reply(ReplyType.IMAGE_URL, retstring)
+                else:
+                    reply = Reply(ReplyType.ERROR, retstring)
+                return reply
 
-    def reply_text(self, query, user_id, retry_count=0):
+    def reply_text(self, query, session_id, retry_count=0):
         try:
             response = openai.Completion.create(
                 model= conf().get("model") or "text-davinci-003",  # 对话模型的名称
@@ -60,116 +76,34 @@ class OpenAIBot(Bot):
                 stop=["\n\n\n"]
             )
             res_content = response.choices[0]['text'].strip().replace('<|endoftext|>', '')
+            total_tokens = response["usage"]["total_tokens"]
+            completion_tokens = response["usage"]["completion_tokens"]
             logger.info("[OPEN_AI] reply={}".format(res_content))
-            return res_content
-        except openai.error.RateLimitError as e:
-            # rate limit exception
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, user_id, retry_count+1)
-            else:
-                return "提问太快啦，请休息一下再问我吧"
+            return total_tokens, completion_tokens, res_content
         except Exception as e:
-            # unknown exception
-            logger.exception(e)
-            Session.clear_session(user_id)
-            return "请再问我一次吧"
-
-
-    def create_img(self, query, retry_count=0):
-        try:
-            logger.info("[OPEN_AI] image_query={}".format(query))
-            response = openai.Image.create(
-                prompt=query,    #图片描述
-                n=1,             #每次生成图片的数量
-                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
-            )
-            image_url = response['data'][0]['url']
-            logger.info("[OPEN_AI] image_url={}".format(image_url))
-            return image_url
-        except openai.error.RateLimitError as e:
-            logger.warn(e)
-            if retry_count < 1:
-                time.sleep(5)
-                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
-                return self.reply_text(query, retry_count+1)
+            need_retry = retry_count < 2
+            result = [0,0,"我现在有点累了，等会再来吧"]
+            if isinstance(e, openai.error.RateLimitError):
+                logger.warn("[OPEN_AI] RateLimitError: {}".format(e))
+                result[2] = "提问太快啦，请休息一下再问我吧"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.Timeout):
+                logger.warn("[OPEN_AI] Timeout: {}".format(e))
+                result[2] = "我没有收到你的消息"
+                if need_retry:
+                    time.sleep(5)
+            elif isinstance(e, openai.error.APIConnectionError):
+                logger.warn("[OPEN_AI] APIConnectionError: {}".format(e))
+                need_retry = False
+                result[2] = "我连接不到你的网络"
             else:
-                return "提问太快啦，请休息一下再问我吧"
-        except Exception as e:
-            logger.exception(e)
-            return None
+                logger.warn("[OPEN_AI] Exception: {}".format(e))
+                need_retry = False
+                self.sessions.clear_session(session_id)
 
-
-class Session(object):
-    @staticmethod
-    def build_session_query(query, user_id):
-        '''
-        build query with conversation history
-        e.g.  Q: xxx
-              A: xxx
-              Q: xxx
-        :param query: query content
-        :param user_id: from user id
-        :return: query content with conversaction
-        '''
-        prompt = conf().get("character_desc", "")
-        if prompt:
-            prompt += "<|endoftext|>\n\n\n"
-        session = user_session.get(user_id, None)
-        if session:
-            for conversation in session:
-                prompt += "Q: " + conversation["question"] + "\n\n\nA: " + conversation["answer"] + "<|endoftext|>\n"
-            prompt += "Q: " + query + "\nA: "
-            return prompt
-        else:
-            return prompt + "Q: " + query + "\nA: "
-
-    @staticmethod
-    def save_session(query, answer, user_id):
-        max_tokens = conf().get("conversation_max_tokens")
-        if not max_tokens:
-            # default 3000
-            max_tokens = 1000
-        conversation = dict()
-        conversation["question"] = query
-        conversation["answer"] = answer
-        session = user_session.get(user_id)
-        logger.debug(conversation)
-        logger.debug(session)
-        if session:
-            # append conversation
-            session.append(conversation)
-        else:
-            # create session
-            queue = list()
-            queue.append(conversation)
-            user_session[user_id] = queue
-
-        # discard exceed limit conversation
-        Session.discard_exceed_conversation(user_session[user_id], max_tokens)
-
-
-    @staticmethod
-    def discard_exceed_conversation(session, max_tokens):
-        count = 0
-        count_list = list()
-        for i in range(len(session)-1, -1, -1):
-            # count tokens of conversation list
-            history_conv = session[i]
-            count += len(history_conv["question"]) + len(history_conv["answer"])
-            count_list.append(count)
-
-        for c in count_list:
-            if c > max_tokens:
-                # pop first conversation
-                session.pop(0)
-
-    @staticmethod
-    def clear_session(user_id):
-        user_session[user_id] = []
-
-    @staticmethod
-    def clear_all_session():
-        user_session.clear()
+            if need_retry:
+                logger.warn("[OPEN_AI] 第{}次重试".format(retry_count+1))
+                return self.reply_text(query, session_id, retry_count+1)
+            else:
+                return result

bot/openai/open_ai_image.py

@@ -0,0 +1,38 @@
+import time
+import openai
+import openai.error
+from common.token_bucket import TokenBucket
+from common.log import logger
+from config import conf
+
+# OPENAI提供的画图接口
+class OpenAIImage(object):
+    def __init__(self):
+        openai.api_key = conf().get('open_ai_api_key')
+        if conf().get('rate_limit_dalle'):
+            self.tb4dalle = TokenBucket(conf().get('rate_limit_dalle', 50))
+            
+    def create_img(self, query, retry_count=0):
+        try:
+            if conf().get('rate_limit_dalle') and not self.tb4dalle.get_token():
+                return False, "请求太快了，请休息一下再问我吧"
+            logger.info("[OPEN_AI] image_query={}".format(query))
+            response = openai.Image.create(
+                prompt=query,    #图片描述
+                n=1,             #每次生成图片的数量
+                size="256x256"   #图片大小,可选有 256x256, 512x512, 1024x1024
+            )
+            image_url = response['data'][0]['url']
+            logger.info("[OPEN_AI] image_url={}".format(image_url))
+            return True, image_url
+        except openai.error.RateLimitError as e:
+            logger.warn(e)
+            if retry_count < 1:
+                time.sleep(5)
+                logger.warn("[OPEN_AI] ImgCreate RateLimit exceed, 第{}次重试".format(retry_count+1))
+                return self.create_img(query, retry_count+1)
+            else:
+                return False, "提问太快啦，请休息一下再问我吧"
+        except Exception as e:
+            logger.exception(e)
+            return False, str(e)

bot/openai/open_ai_session.py

@@ -0,0 +1,67 @@
+from bot.session_manager import Session
+from common.log import logger
+class OpenAISession(Session):
+    def __init__(self, session_id, system_prompt=None, model= "text-davinci-003"):
+        super().__init__(session_id, system_prompt)
+        self.model = model
+        self.reset()
+
+    def __str__(self):
+        # 构造对话模型的输入
+        '''
+        e.g.  Q: xxx
+              A: xxx
+              Q: xxx
+        '''
+        prompt = ""
+        for item in self.messages:
+            if item['role'] == 'system':
+                prompt += item['content'] + "<|endoftext|>\n\n\n"
+            elif item['role'] == 'user':
+                prompt += "Q: " + item['content'] + "\n"
+            elif item['role'] == 'assistant':
+                prompt += "\n\nA: " + item['content'] + "<|endoftext|>\n"
+
+        if len(self.messages) > 0 and self.messages[-1]['role'] == 'user':
+            prompt += "A: "
+        return prompt
+
+    def discard_exceeding(self, max_tokens, cur_tokens= None):
+        precise = True
+        try:
+            cur_tokens = num_tokens_from_string(str(self), self.model)
+        except Exception as e:
+            precise = False
+            if cur_tokens is None:
+                raise e
+            logger.debug("Exception when counting tokens precisely for query: {}".format(e))
+        while cur_tokens > max_tokens:
+            if len(self.messages) > 1:
+                self.messages.pop(0)
+            elif len(self.messages) == 1 and self.messages[0]["role"] == "assistant":
+                self.messages.pop(0)
+                if precise:
+                    cur_tokens = num_tokens_from_string(str(self), self.model)
+                else:
+                    cur_tokens = len(str(self))
+                break
+            elif len(self.messages) == 1 and self.messages[0]["role"] == "user":
+                logger.warn("user question exceed max_tokens. total_tokens={}".format(cur_tokens))
+                break
+            else:
+                logger.debug("max_tokens={}, total_tokens={}, len(conversation)={}".format(max_tokens, cur_tokens, len(self.messages)))
+                break
+            if precise:
+                cur_tokens = num_tokens_from_string(str(self), self.model)
+            else:
+                cur_tokens = len(str(self))
+        return cur_tokens
+    
+
+# refer to https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+def num_tokens_from_string(string: str, model: str) -> int:
+    """Returns the number of tokens in a text string."""
+    import tiktoken
+    encoding = tiktoken.encoding_for_model(model)
+    num_tokens = len(encoding.encode(string,disallowed_special=()))
+    return num_tokens

bot/session_manager.py

@@ -0,0 +1,85 @@
+from common.expired_dict import ExpiredDict
+from common.log import logger
+from config import conf
+
+class Session(object):
+    def __init__(self, session_id, system_prompt=None):
+        self.session_id = session_id
+        self.messages = []
+        if system_prompt is None:
+            self.system_prompt = conf().get("character_desc", "")
+        else:
+            self.system_prompt = system_prompt
+
+    # 重置会话
+    def reset(self):
+        system_item = {'role': 'system', 'content': self.system_prompt}
+        self.messages = [system_item]
+
+    def set_system_prompt(self, system_prompt):
+        self.system_prompt = system_prompt
+        self.reset()
+
+    def add_query(self, query):
+        user_item = {'role': 'user', 'content': query}
+        self.messages.append(user_item)
+
+    def add_reply(self, reply):
+        assistant_item = {'role': 'assistant', 'content': reply}
+        self.messages.append(assistant_item)
+    
+    def discard_exceeding(self, max_tokens=None, cur_tokens=None):
+        raise NotImplementedError
+
+
+
+class SessionManager(object):
+    def __init__(self, sessioncls, **session_args):
+        if conf().get('expires_in_seconds'):
+            sessions = ExpiredDict(conf().get('expires_in_seconds'))
+        else:
+            sessions = dict()
+        self.sessions = sessions
+        self.sessioncls = sessioncls
+        self.session_args = session_args
+
+    def build_session(self, session_id, system_prompt=None):
+        '''
+            如果session_id不在sessions中，创建一个新的session并添加到sessions中
+            如果system_prompt不会空，会更新session的system_prompt并重置session
+        '''
+        if session_id not in self.sessions:
+            self.sessions[session_id] = self.sessioncls(session_id, system_prompt, **self.session_args)
+        elif system_prompt is not None:  # 如果有新的system_prompt，更新并重置session
+            self.sessions[session_id].set_system_prompt(system_prompt)
+        session = self.sessions[session_id]
+        return session
+    
+    def session_query(self, query, session_id):
+        session = self.build_session(session_id)
+        session.add_query(query)
+        try:
+            max_tokens = conf().get("conversation_max_tokens", 1000)
+            total_tokens = session.discard_exceeding(max_tokens, None)
+            logger.debug("prompt tokens used={}".format(total_tokens))
+        except Exception as e:
+            logger.debug("Exception when counting tokens precisely for prompt: {}".format(str(e)))
+        return session
+
+    def session_reply(self, reply, session_id, total_tokens = None):
+        session = self.build_session(session_id)
+        session.add_reply(reply)
+        try:
+            max_tokens = conf().get("conversation_max_tokens", 1000)
+            tokens_cnt = session.discard_exceeding(max_tokens, total_tokens)
+            logger.debug("raw total_tokens={}, savesession tokens={}".format(total_tokens, tokens_cnt))
+        except Exception as e:
+            logger.debug("Exception when counting tokens precisely for session: {}".format(str(e)))
+        return session
+
+    def clear_session(self, session_id):
+        if session_id in self.sessions:
+            del(self.sessions[session_id])
+
+    def clear_all_session(self):
+        self.sessions.clear()

bridge/bridge.py

@@ -14,7 +14,7 @@ class Bridge(object):
         self.btype={
             "chat": const.CHATGPT,
             "voice_to_text": conf().get("voice_to_text", "openai"),
-            "text_to_voice": conf().get("text_to_voice", "baidu")
+            "text_to_voice": conf().get("text_to_voice", "google")
         }
         model_type = conf().get("model")
         if model_type in ["text-davinci-003"]:

bridge/context.py

@@ -14,6 +14,15 @@ class Context:
         self.type = type
         self.content = content
         self.kwargs = kwargs
+
+    def __contains__(self, key):
+        if key == 'type':
+            return self.type is not None
+        elif key == 'content':
+            return self.content is not None
+        else:
+            return key in self.kwargs
+        
     def __getitem__(self, key):
         if key == 'type':
             return self.type
@@ -21,6 +30,12 @@ class Context:
             return self.content
         else:
             return self.kwargs[key]
+    
+    def get(self, key, default=None):
+        try:
+            return self[key]
+        except KeyError:
+            return default
 
     def __setitem__(self, key, value):
         if key == 'type':

channel/channel.py

@@ -20,7 +20,8 @@ class Channel(object):
         """
         raise NotImplementedError
 
-    def send(self, msg, receiver):
+    # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
+    def send(self, reply: Reply, context: Context):
         """
         send message to user
         :param msg: message content

channel/channel_factory.py

@@ -17,4 +17,7 @@ def create_channel(channel_type):
     elif channel_type == 'terminal':
         from channel.terminal.terminal_channel import TerminalChannel
         return TerminalChannel()
+    elif channel_type == 'wechatmp':
+        from channel.wechatmp.wechatmp_channel import WechatMPServer
+        return WechatMPServer()
     raise RuntimeError

channel/chat_channel.py

@@ -0,0 +1,310 @@
+
+
+from asyncio import CancelledError
+from concurrent.futures import Future, ThreadPoolExecutor
+import os
+import re
+import threading
+import time
+from common.dequeue import Dequeue
+from channel.channel import Channel
+from bridge.reply import *
+from bridge.context import *
+from config import conf
+from common.log import logger
+from plugins import *
+try:
+    from voice.audio_convert import any_to_wav
+except Exception as e:
+    pass
+
+# 抽象类, 它包含了与消息通道无关的通用处理逻辑
+class ChatChannel(Channel):
+    name = None # 登录的用户名
+    user_id = None # 登录的用户id
+    futures = {} # 记录每个session_id提交到线程池的future对象, 用于重置会话时把没执行的future取消掉，正在执行的不会被取消
+    sessions = {} # 用于控制并发，每个session_id同时只能有一个context在处理
+    lock = threading.Lock() # 用于控制对sessions的访问
+    handler_pool = ThreadPoolExecutor(max_workers=8)  # 处理消息的线程池
+
+    def __init__(self):
+        _thread = threading.Thread(target=self.consume)
+        _thread.setDaemon(True)
+        _thread.start()
+        
+
+    # 根据消息构造context，消息内容相关的触发项写在这里
+    def _compose_context(self, ctype: ContextType, content, **kwargs):
+        context = Context(ctype, content)
+        context.kwargs = kwargs
+        # context首次传入时，origin_ctype是None, 
+        # 引入的起因是：当输入语音时，会嵌套生成两个context，第一步语音转文本，第二步通过文本生成文字回复。
+        # origin_ctype用于第二步文本回复时，判断是否需要匹配前缀，如果是私聊的语音，就不需要匹配前缀
+        if 'origin_ctype' not in context:  
+            context['origin_ctype'] = ctype
+        # context首次传入时，receiver是None，根据类型设置receiver
+        first_in = 'receiver' not in context
+        # 群名匹配过程，设置session_id和receiver
+        if first_in: # context首次传入时，receiver是None，根据类型设置receiver
+            config = conf()
+            cmsg = context['msg']
+            if cmsg.from_user_id == self.user_id and not config.get('trigger_by_self', False):
+                logger.debug("[WX]self message skipped")
+                return None
+            if context["isgroup"]:
+                group_name = cmsg.other_user_nickname
+                group_id = cmsg.other_user_id
+
+                group_name_white_list = config.get('group_name_white_list', [])
+                group_name_keyword_white_list = config.get('group_name_keyword_white_list', [])
+                if any([group_name in group_name_white_list, 'ALL_GROUP' in group_name_white_list, check_contain(group_name, group_name_keyword_white_list)]):
+                    group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
+                    session_id = cmsg.actual_user_id
+                    if any([group_name in group_chat_in_one_session, 'ALL_GROUP' in group_chat_in_one_session]):
+                        session_id = group_id
+                else:
+                    return None
+                context['session_id'] = session_id
+                context['receiver'] = group_id
+            else:
+                context['session_id'] = cmsg.other_user_id
+                context['receiver'] = cmsg.other_user_id
+
+        # 消息内容匹配过程，并处理content
+        if ctype == ContextType.TEXT:
+            if first_in and "」\n- - - - - - -" in content: # 初次匹配 过滤引用消息
+                logger.debug("[WX]reference query skipped")
+                return None
+            
+            if context["isgroup"]: # 群聊
+                # 校验关键字
+                match_prefix = check_prefix(content, conf().get('group_chat_prefix'))
+                match_contain = check_contain(content, conf().get('group_chat_keyword'))
+                flag = False
+                if match_prefix is not None or match_contain is not None:
+                    flag = True
+                    if match_prefix:
+                        content = content.replace(match_prefix, '', 1).strip()
+                if context['msg'].is_at:
+                    logger.info("[WX]receive group at")
+                    if not conf().get("group_at_off", False):
+                        flag = True
+                    pattern = f'@{self.name}(\u2005|\u0020)'
+                    content = re.sub(pattern, r'', content)
+                
+                if not flag:
+                    if context["origin_ctype"] == ContextType.VOICE:
+                        logger.info("[WX]receive group voice, but checkprefix didn't match")
+                    return None
+            else: # 单聊
+                match_prefix = check_prefix(content, conf().get('single_chat_prefix'))  
+                if match_prefix is not None: # 判断如果匹配到自定义前缀，则返回过滤掉前缀+空格后的内容
+                    content = content.replace(match_prefix, '', 1).strip()
+                elif context["origin_ctype"] == ContextType.VOICE: # 如果源消息是私聊的语音消息，允许不匹配前缀，放宽条件
+                    pass
+                else:
+                    return None     
+                                                  
+            img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
+            if img_match_prefix:
+                content = content.replace(img_match_prefix, '', 1).strip()
+                context.type = ContextType.IMAGE_CREATE
+            else:
+                context.type = ContextType.TEXT
+            context.content = content
+            if 'desire_rtype' not in context and conf().get('always_reply_voice'):
+                context['desire_rtype'] = ReplyType.VOICE
+        elif context.type == ContextType.VOICE: 
+            if 'desire_rtype' not in context and conf().get('voice_reply_voice'):
+                context['desire_rtype'] = ReplyType.VOICE
+
+        return context
+
+    def _handle(self, context: Context):
+        if context is None or not context.content:
+            return
+        logger.debug('[WX] ready to handle context: {}'.format(context))
+        # reply的构建步骤
+        reply = self._generate_reply(context)
+
+        logger.debug('[WX] ready to decorate reply: {}'.format(reply))
+        # reply的包装步骤
+        reply = self._decorate_reply(context, reply)
+
+        # reply的发送步骤
+        self._send_reply(context, reply)
+
+    def _generate_reply(self, context: Context, reply: Reply = Reply()) -> Reply:
+        e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {
+            'channel': self, 'context': context, 'reply': reply}))
+        reply = e_context['reply']
+        if not e_context.is_pass():
+            logger.debug('[WX] ready to handle context: type={}, content={}'.format(context.type, context.content))
+            if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:  # 文字和图片消息
+                reply = super().build_reply_content(context.content, context)
+            elif context.type == ContextType.VOICE:  # 语音消息
+                cmsg = context['msg']
+                cmsg.prepare()
+                file_path = context.content
+                wav_path = os.path.splitext(file_path)[0] + '.wav'
+                try:
+                    any_to_wav(file_path, wav_path) 
+                except Exception as e:  # 转换失败，直接使用mp3，对于某些api，mp3也可以识别
+                    logger.warning("[WX]any to wav error, use raw path. " + str(e))
+                    wav_path = file_path
+                # 语音识别
+                reply = super().build_voice_to_text(wav_path)
+                # 删除临时文件
+                try:
+                    os.remove(file_path)
+                    if wav_path != file_path:
+                        os.remove(wav_path)
+                except Exception as e:
+                    pass
+                    # logger.warning("[WX]delete temp file error: " + str(e))
+
+                if reply.type == ReplyType.TEXT:
+                    new_context = self._compose_context(
+                        ContextType.TEXT, reply.content, **context.kwargs)
+                    if new_context:
+                        reply = self._generate_reply(new_context)
+                    else:
+                        return
+            else:
+                logger.error('[WX] unknown context type: {}'.format(context.type))
+                return
+        return reply
+
+    def _decorate_reply(self, context: Context, reply: Reply) -> Reply:
+        if reply and reply.type:
+            e_context = PluginManager().emit_event(EventContext(Event.ON_DECORATE_REPLY, {
+                'channel': self, 'context': context, 'reply': reply}))
+            reply = e_context['reply']
+            desire_rtype = context.get('desire_rtype')
+            if not e_context.is_pass() and reply and reply.type:
+                if reply.type == ReplyType.TEXT:
+                    reply_text = reply.content
+                    if desire_rtype == ReplyType.VOICE:
+                        reply = super().build_text_to_voice(reply.content)
+                        return self._decorate_reply(context, reply)
+                    if context['isgroup']:
+                        reply_text = '@' +  context['msg'].actual_user_nickname + ' ' + reply_text.strip()
+                        reply_text = conf().get("group_chat_reply_prefix", "")+reply_text
+                    else:
+                        reply_text = conf().get("single_chat_reply_prefix", "")+reply_text
+                    reply.content = reply_text
+                elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
+                    reply.content = str(reply.type)+":\n" + reply.content
+                elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
+                    pass
+                else:
+                    logger.error('[WX] unknown reply type: {}'.format(reply.type))
+                    return
+            if desire_rtype and desire_rtype != reply.type and reply.type not in [ReplyType.ERROR, ReplyType.INFO]:
+                logger.warning('[WX] desire_rtype: {}, but reply type: {}'.format(context.get('desire_rtype'), reply.type))
+            return reply
+
+    def _send_reply(self, context: Context, reply: Reply):
+        if reply and reply.type:
+            e_context = PluginManager().emit_event(EventContext(Event.ON_SEND_REPLY, {
+                'channel': self, 'context': context, 'reply': reply}))
+            reply = e_context['reply']
+            if not e_context.is_pass() and reply and reply.type:
+                logger.debug('[WX] ready to send reply: {}, context: {}'.format(reply, context))
+                self._send(reply, context)
+
+    def _send(self, reply: Reply, context: Context, retry_cnt = 0):
+        try:
+            self.send(reply, context)
+        except Exception as e:
+            logger.error('[WX] sendMsg error: {}'.format(str(e)))
+            if isinstance(e, NotImplementedError):
+                return
+            logger.exception(e)
+            if retry_cnt < 2:
+                time.sleep(3+3*retry_cnt)
+                self._send(reply, context, retry_cnt+1)
+
+    def thread_pool_callback(self, session_id):
+        def func(worker:Future):
+            try:
+                worker_exception = worker.exception()
+                if worker_exception:
+                    logger.exception("Worker return exception: {}".format(worker_exception))
+            except CancelledError as e:
+                logger.info("Worker cancelled, session_id = {}".format(session_id))
+            except Exception as e:
+                logger.exception("Worker raise exception: {}".format(e))
+            with self.lock:
+                self.sessions[session_id][1].release()
+        return func
+
+    def produce(self, context: Context):
+        session_id = context['session_id']
+        with self.lock:
+            if session_id not in self.sessions:
+                self.sessions[session_id] = [Dequeue(), threading.BoundedSemaphore(conf().get("concurrency_in_session", 1))]
+            if context.type == ContextType.TEXT and context.content.startswith("#"): 
+                self.sessions[session_id][0].putleft(context) # 优先处理管理命令
+            else:
+                self.sessions[session_id][0].put(context)
+
+    # 消费者函数，单独线程，用于从消息队列中取出消息并处理
+    def consume(self):
+        while True:
+            with self.lock:
+                session_ids = list(self.sessions.keys())
+                for session_id in session_ids:
+                    context_queue, semaphore = self.sessions[session_id]
+                    if semaphore.acquire(blocking = False): # 等线程处理完毕才能删除
+                        if not context_queue.empty():
+                            context = context_queue.get()
+                            logger.debug("[WX] consume context: {}".format(context))
+                            future:Future = self.handler_pool.submit(self._handle, context)
+                            future.add_done_callback(self.thread_pool_callback(session_id))
+                            if session_id not in self.futures:
+                                self.futures[session_id] = []
+                            self.futures[session_id].append(future)
+                        elif semaphore._initial_value == semaphore._value+1: # 除了当前，没有任务再申请到信号量，说明所有任务都处理完毕
+                            self.futures[session_id] = [t for t in self.futures[session_id] if not t.done()]
+                            assert len(self.futures[session_id]) == 0, "thread pool error"
+                            del self.sessions[session_id]
+                        else:
+                            semaphore.release()
+            time.sleep(0.1)
+
+    # 取消session_id对应的所有任务，只能取消排队的消息和已提交线程池但未执行的任务
+    def cancel_session(self, session_id): 
+        with self.lock:
+            if session_id in self.sessions:
+                for future in self.futures[session_id]:
+                    future.cancel()
+                cnt = self.sessions[session_id][0].qsize()
+                if cnt>0:
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
+                self.sessions[session_id][0] = Dequeue()
+    
+    def cancel_all_session(self):
+        with self.lock:
+            for session_id in self.sessions:
+                for future in self.futures[session_id]:
+                    future.cancel()
+                cnt = self.sessions[session_id][0].qsize()
+                if cnt>0:
+                    logger.info("Cancel {} messages in session {}".format(cnt, session_id))
+                self.sessions[session_id][0] = Dequeue()
+    
+
+def check_prefix(content, prefix_list):
+    for prefix in prefix_list:
+        if content.startswith(prefix):
+            return prefix
+    return None
+
+def check_contain(content, keyword_list):
+    if not keyword_list:
+        return None
+    for ky in keyword_list:
+        if content.find(ky) != -1:
+            return True
+    return None

channel/chat_message.py

@@ -0,0 +1,83 @@
+
+""" 
+本类表示聊天消息，用于对itchat和wechaty的消息进行统一的封装
+
+ChatMessage
+msg_id: 消息id
+create_time: 消息创建时间
+
+ctype: 消息类型 : ContextType
+content: 消息内容, 如果是声音/图片，这里是文件路径
+
+from_user_id: 发送者id
+from_user_nickname: 发送者昵称
+to_user_id: 接收者id
+to_user_nickname: 接收者昵称
+
+other_user_id: 对方的id，如果你是发送者，那这个就是接收者id，如果你是接收者，那这个就是发送者id，如果是群消息，那这一直是群id
+other_user_nickname: 同上
+
+is_group: 是否是群消息
+is_at: 是否被at
+
+- (群消息时，一般会存在实际发送者，是群内某个成员的id和昵称，下列项仅在群消息时存在)
+actual_user_id: 实际发送者id
+actual_user_nickname：实际发送者昵称
+
+
+
+
+_prepare_fn: 准备函数，用于准备消息的内容，比如下载图片等,
+_prepared: 是否已经调用过准备函数
+_rawmsg: 原始消息对象
+
+"""
+class ChatMessage(object):
+    msg_id = None
+    create_time = None
+    
+    ctype = None
+    content = None
+    
+    from_user_id = None
+    from_user_nickname = None
+    to_user_id = None
+    to_user_nickname = None
+    other_user_id = None
+    other_user_nickname = None
+    
+    is_group = False
+    is_at = False
+    actual_user_id = None
+    actual_user_nickname = None
+
+    _prepare_fn = None
+    _prepared = False
+    _rawmsg = None
+
+
+    def __init__(self,_rawmsg):
+        self._rawmsg = _rawmsg
+
+    def prepare(self):
+        if self._prepare_fn and not self._prepared:
+            self._prepared = True
+            self._prepare_fn()
+
+    def __str__(self):
+        return 'ChatMessage: id={}, create_time={}, ctype={}, content={}, from_user_id={}, from_user_nickname={}, to_user_id={}, to_user_nickname={}, other_user_id={}, other_user_nickname={}, is_group={}, is_at={}, actual_user_id={}, actual_user_nickname={}'.format(
+            self.msg_id,
+            self.create_time,
+            self.ctype,
+            self.content,
+            self.from_user_id,
+            self.from_user_nickname,
+            self.to_user_id,
+            self.to_user_nickname,
+            self.other_user_id,
+            self.other_user_nickname,
+            self.is_group,
+            self.is_at,
+            self.actual_user_id,
+            self.actual_user_nickname,
+        )

channel/wechat/wechat_channel.py

@@ -5,70 +5,119 @@ wechat channel
 """
 
 import os
-from lib import itchat
-import json
-from lib.itchat.content import *
-from bridge.reply import *
-from bridge.context import *
-from channel.channel import Channel
-from concurrent.futures import ThreadPoolExecutor
-from common.log import logger
-from common.tmp_dir import TmpDir
-from config import conf
-from common.time_check import time_checker
-from plugins import *
+import threading
 import requests
 import io
 import time
-
-
-thread_pool = ThreadPoolExecutor(max_workers=8)
-def thread_pool_callback(worker):
-    worker_exception = worker.exception()
-    if worker_exception:
-        logger.exception("Worker return exception: {}".format(worker_exception))
+import json
+from channel.chat_channel import ChatChannel
+from channel.wechat.wechat_message import *
+from common.singleton import singleton
+from common.log import logger
+from lib import itchat
+from lib.itchat.content import *
+from bridge.reply import *
+from bridge.context import *
+from config import conf
+from common.time_check import time_checker
+from common.expired_dict import ExpiredDict
+from plugins import *
 
 @itchat.msg_register(TEXT)
 def handler_single_msg(msg):
-    WechatChannel().handle_text(msg)
+    WechatChannel().handle_text(WeChatMessage(msg))
     return None
 
-
 @itchat.msg_register(TEXT, isGroupChat=True)
 def handler_group_msg(msg):
-    WechatChannel().handle_group(msg)
+    WechatChannel().handle_group(WeChatMessage(msg,True))
     return None
 
-
 @itchat.msg_register(VOICE)
 def handler_single_voice(msg):
-    WechatChannel().handle_voice(msg)
+    WechatChannel().handle_voice(WeChatMessage(msg))
+    return None
+    
+@itchat.msg_register(VOICE, isGroupChat=True)
+def handler_group_voice(msg):
+    WechatChannel().handle_group_voice(WeChatMessage(msg,True))
     return None
 
+def _check(func):
+    def wrapper(self, cmsg: ChatMessage):
+        msgId = cmsg.msg_id
+        if msgId in self.receivedMsgs:
+            logger.info("Wechat message {} already received, ignore".format(msgId))
+            return
+        self.receivedMsgs[msgId] = cmsg
+        create_time = cmsg.create_time            # 消息时间戳
+        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:  # 跳过1分钟前的历史消息
+            logger.debug("[WX]history message {} skipped".format(msgId))
+            return
+        return func(self, cmsg)
+    return wrapper
 
-class WechatChannel(Channel):
+#可用的二维码生成接口
+#https://api.qrserver.com/v1/create-qr-code/?size=400×400&data=https://www.abc.com
+#https://api.isoyu.com/qr/?m=1&e=L&p=20&url=https://www.abc.com
+def qrCallback(uuid,status,qrcode):
+    # logger.debug("qrCallback: {} {}".format(uuid,status))
+    if status == '0':
+        try:
+            from PIL import Image
+            img = Image.open(io.BytesIO(qrcode))
+            _thread = threading.Thread(target=img.show, args=("QRCode",))
+            _thread.setDaemon(True)
+            _thread.start()
+        except Exception as e:
+            pass
+
+        import qrcode
+        url = f"https://login.weixin.qq.com/l/{uuid}"
+        
+        qr_api1="https://api.isoyu.com/qr/?m=1&e=L&p=20&url={}".format(url)
+        qr_api2="https://api.qrserver.com/v1/create-qr-code/?size=400×400&data={}".format(url)
+        qr_api3="https://api.pwmqr.com/qrcode/create/?url={}".format(url)
+        qr_api4="https://my.tv.sohu.com/user/a/wvideo/getQRCode.do?text={}".format(url)
+        print("You can also scan QRCode in any website below:")
+        print(qr_api3)
+        print(qr_api4)
+        print(qr_api2)
+        print(qr_api1)
+        
+        qr = qrcode.QRCode(border=1)
+        qr.add_data(url)
+        qr.make(fit=True)
+        qr.print_ascii(invert=True)
+
+@singleton
+class WechatChannel(ChatChannel):
     def __init__(self):
-        pass
+        super().__init__()
+        self.receivedMsgs = ExpiredDict(60*60*24) 
 
     def startup(self):
 
-        itchat.instance.receivingRetryCount = 600 # 修改断线超时时间
+        itchat.instance.receivingRetryCount = 600  # 修改断线超时时间
         # login by scan QRCode
         hotReload = conf().get('hot_reload', False)
         try:
-            itchat.auto_login(enableCmdQR=2, hotReload=hotReload)
+            itchat.auto_login(enableCmdQR=2, hotReload=hotReload, qrCallback=qrCallback)
         except Exception as e:
             if hotReload:
                 logger.error("Hot reload failed, try to login without hot reload")
                 itchat.logout()
                 os.remove("itchat.pkl")
-                itchat.auto_login(enableCmdQR=2, hotReload=hotReload)
+                itchat.auto_login(enableCmdQR=2, hotReload=hotReload, qrCallback=qrCallback)
             else:
                 raise e
+        self.user_id = itchat.instance.storageClass.userName
+        self.name = itchat.instance.storageClass.nickName
+        logger.info("Wechat login success, user_id: {}, nickname: {}".format(self.user_id, self.name))
         # start message listener
         itchat.run()
 
-    # handle_* 系列函数处理收到的消息后构造Context，然后传入handle函数中处理Context和发送回复
+    # handle_* 系列函数处理收到的消息后构造Context，然后传入_handle函数中处理Context和发送回复
     # Context包含了消息的所有信息，包括以下属性
     #   type 消息类型, 包括TEXT、VOICE、IMAGE_CREATE
     #   content 消息内容，如果是TEXT类型，content就是文本内容，如果是VOICE类型，content就是语音文件名，如果是IMAGE_CREATE类型，content就是图片生成命令
@@ -76,101 +125,49 @@ class WechatChannel(Channel):
     #        session_id: 会话id
     #        isgroup: 是否是群聊
     #        receiver: 需要回复的对象
-    #        msg: itchat的原始消息对象
+    #        msg: ChatMessage消息对象
+    #        origin_ctype: 原始消息类型，语音转文字后，私聊时如果匹配前缀失败，会根据初始消息是否是语音来放宽触发规则
+    #        desire_rtype: 希望回复类型，默认是文本回复，设置为ReplyType.VOICE是语音回复
 
-    def handle_voice(self, msg):
+    @time_checker
+    @_check
+    def handle_voice(self, cmsg : ChatMessage):
         if conf().get('speech_recognition') != True:
             return
-        logger.debug("[WX]receive voice msg: " + msg['FileName'])
-        from_user_id = msg['FromUserName']
-        other_user_id = msg['User']['UserName']
-        if from_user_id == other_user_id:
-            context = Context(ContextType.VOICE,msg['FileName'])
-            context.kwargs = {'isgroup': False, 'msg': msg, 'receiver': other_user_id, 'session_id': other_user_id}
-            thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
-
+        logger.debug("[WX]receive voice msg: {}".format(cmsg.content))
+        context = self._compose_context(ContextType.VOICE, cmsg.content, isgroup=False, msg=cmsg)
+        if context:
+            self.produce(context)
 
     @time_checker
-    def handle_text(self, msg):
-        logger.debug("[WX]receive text msg: " + json.dumps(msg, ensure_ascii=False))
-        content = msg['Text']
-        from_user_id = msg['FromUserName']
-        to_user_id = msg['ToUserName']              # 接收人id
-        other_user_id = msg['User']['UserName']     # 对手方id
-        create_time = msg['CreateTime']             # 消息时间
-        match_prefix = check_prefix(content, conf().get('single_chat_prefix'))
-        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:    #跳过1分钟前的历史消息
-            logger.debug("[WX]history message skipped")
-            return
-        if "」\n- - - - - - - - - - - - - - -" in content:
-            logger.debug("[WX]reference query skipped")
-            return
-        if match_prefix:
-            content = content.replace(match_prefix, '', 1).strip()
-        elif match_prefix is None:
-            return
-        context = Context()
-        context.kwargs = {'isgroup': False, 'msg': msg, 'receiver': other_user_id, 'session_id': other_user_id}
-
-        img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
-        if img_match_prefix:
-            content = content.replace(img_match_prefix, '', 1).strip()
-            context.type = ContextType.IMAGE_CREATE
-        else:
-            context.type = ContextType.TEXT
-
-        context.content = content
-        thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
+    @_check
+    def handle_text(self, cmsg : ChatMessage):
+        logger.debug("[WX]receive text msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
+        context = self._compose_context(ContextType.TEXT, cmsg.content, isgroup=False, msg=cmsg)
+        if context:
+            self.produce(context)
 
     @time_checker
-    def handle_group(self, msg):
-        logger.debug("[WX]receive group msg: " + json.dumps(msg, ensure_ascii=False))
-        group_name = msg['User'].get('NickName', None)
-        group_id = msg['User'].get('UserName', None)
-        create_time = msg['CreateTime']             # 消息时间
-        if conf().get('hot_reload') == True and int(create_time) < int(time.time()) - 60:    #跳过1分钟前的历史消息
-            logger.debug("[WX]history group message skipped")
+    @_check
+    def handle_group(self, cmsg : ChatMessage):
+        logger.debug("[WX]receive group msg: {}, cmsg={}".format(json.dumps(cmsg._rawmsg, ensure_ascii=False), cmsg))
+        context = self._compose_context(ContextType.TEXT, cmsg.content, isgroup=True, msg=cmsg)
+        if context:
+            self.produce(context)
+    
+    @time_checker
+    @_check
+    def handle_group_voice(self, cmsg : ChatMessage):
+        if conf().get('group_speech_recognition', False) != True:
             return
-        if not group_name:
-            return ""
-        origin_content = msg['Content']
-        content = msg['Content']
-        content_list = content.split(' ', 1)
-        context_special_list = content.split('\u2005', 1)
-        if len(context_special_list) == 2:
-            content = context_special_list[1]
-        elif len(content_list) == 2:
-            content = content_list[1]
-        if "」\n- - - - - - - - - - - - - - -" in content:
-            logger.debug("[WX]reference query skipped")
-            return ""
-        config = conf()
-        match_prefix = (msg['IsAt'] and not config.get("group_at_off", False)) or check_prefix(origin_content, config.get('group_chat_prefix')) \
-                       or check_contain(origin_content, config.get('group_chat_keyword'))
-        if ('ALL_GROUP' in config.get('group_name_white_list') or group_name in config.get('group_name_white_list') or check_contain(group_name, config.get('group_name_keyword_white_list'))) and match_prefix:
-            context = Context()
-            context.kwargs = { 'isgroup': True, 'msg': msg, 'receiver': group_id}
-            
-            img_match_prefix = check_prefix(content, conf().get('image_create_prefix'))
-            if img_match_prefix:
-                content = content.replace(img_match_prefix, '', 1).strip()
-                context.type = ContextType.IMAGE_CREATE
-            else:
-                context.type = ContextType.TEXT
-            context.content = content
-
-            group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
-            if ('ALL_GROUP' in group_chat_in_one_session or
-                    group_name in group_chat_in_one_session or
-                    check_contain(group_name, group_chat_in_one_session)):
-                context['session_id'] = group_id
-            else:
-                context['session_id'] = msg['ActualUserName']
-
-            thread_pool.submit(self.handle, context).add_done_callback(thread_pool_callback)
-
+        logger.debug("[WX]receive voice for group msg: {}".format(cmsg.content))
+        context = self._compose_context(ContextType.VOICE, cmsg.content, isgroup=True, msg=cmsg)
+        if context:
+            self.produce(context)
+    
     # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
-    def send(self, reply : Reply, receiver):
+    def send(self, reply: Reply, context: Context):
+        receiver = context["receiver"]
         if reply.type == ReplyType.TEXT:
             itchat.send(reply.content, toUserName=receiver)
             logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
@@ -194,79 +191,3 @@ class WechatChannel(Channel):
             image_storage.seek(0)
             itchat.send_image(image_storage, toUserName=receiver)
             logger.info('[WX] sendImage, receiver={}'.format(receiver))
-
-    # 处理消息 TODO: 如果wechaty解耦，此处逻辑可以放置到父类
-    def handle(self, context):
-        reply = Reply()
-
-        logger.debug('[WX] ready to handle context: {}'.format(context))
-        
-        # reply的构建步骤
-        e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {'channel' : self, 'context': context, 'reply': reply}))
-        reply = e_context['reply']
-        if not e_context.is_pass():
-            logger.debug('[WX] ready to handle context: type={}, content={}'.format(context.type, context.content))
-            if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:
-                reply = super().build_reply_content(context.content, context)
-            elif context.type == ContextType.VOICE:
-                msg = context['msg']
-                file_name = TmpDir().path() + context.content
-                msg.download(file_name)
-                reply = super().build_voice_to_text(file_name)
-                if reply.type != ReplyType.ERROR and reply.type != ReplyType.INFO:
-                    context.content = reply.content # 语音转文字后，将文字内容作为新的context
-                    context.type = ContextType.TEXT
-                    reply = super().build_reply_content(context.content, context)
-                    if reply.type == ReplyType.TEXT:
-                        if conf().get('voice_reply_voice'):
-                            reply = super().build_text_to_voice(reply.content)
-            else:
-                logger.error('[WX] unknown context type: {}'.format(context.type))
-                return
-
-        logger.debug('[WX] ready to decorate reply: {}'.format(reply))
-        
-        # reply的包装步骤
-        if reply and reply.type:
-            e_context = PluginManager().emit_event(EventContext(Event.ON_DECORATE_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
-            reply=e_context['reply']
-            if not e_context.is_pass() and reply and reply.type:
-                if reply.type == ReplyType.TEXT:
-                    reply_text = reply.content
-                    if context['isgroup']:
-                        reply_text = '@' +  context['msg']['ActualNickName'] + ' ' + reply_text.strip()
-                        reply_text = conf().get("group_chat_reply_prefix", "")+reply_text
-                    else:
-                        reply_text = conf().get("single_chat_reply_prefix", "")+reply_text
-                    reply.content = reply_text
-                elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
-                    reply.content = str(reply.type)+":\n" + reply.content
-                elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
-                    pass
-                else:
-                    logger.error('[WX] unknown reply type: {}'.format(reply.type))
-                    return
-
-        # reply的发送步骤   
-        if reply and reply.type:
-            e_context = PluginManager().emit_event(EventContext(Event.ON_SEND_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
-            reply=e_context['reply']
-            if not e_context.is_pass() and reply and reply.type:
-                logger.debug('[WX] ready to send reply: {} to {}'.format(reply, context['receiver']))
-                self.send(reply, context['receiver'])
-
-
-def check_prefix(content, prefix_list):
-    for prefix in prefix_list:
-        if content.startswith(prefix):
-            return prefix
-    return None
-
-
-def check_contain(content, keyword_list):
-    if not keyword_list:
-        return None
-    for ky in keyword_list:
-        if content.find(ky) != -1:
-            return True
-    return None

channel/wechat/wechat_message.py

@@ -0,0 +1,57 @@
+
+
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.tmp_dir import TmpDir
+from common.log import logger
+from lib.itchat.content import *
+from lib import itchat
+
+class WeChatMessage(ChatMessage):
+
+    def __init__(self, itchat_msg, is_group=False):
+        super().__init__( itchat_msg)
+        self.msg_id = itchat_msg['MsgId']
+        self.create_time = itchat_msg['CreateTime']
+        self.is_group = is_group
+        
+        if itchat_msg['Type'] == TEXT:
+            self.ctype = ContextType.TEXT
+            self.content = itchat_msg['Text']
+        elif itchat_msg['Type'] == VOICE:
+            self.ctype = ContextType.VOICE
+            self.content = TmpDir().path() + itchat_msg['FileName']  # content直接存临时目录路径
+            self._prepare_fn = lambda: itchat_msg.download(self.content)
+        else:
+            raise NotImplementedError("Unsupported message type: {}".format(itchat_msg['Type']))
+        
+        self.from_user_id = itchat_msg['FromUserName']
+        self.to_user_id = itchat_msg['ToUserName']
+        
+        user_id = itchat.instance.storageClass.userName
+        nickname = itchat.instance.storageClass.nickName
+        
+        # 虽然from_user_id和to_user_id用的少，但是为了保持一致性，还是要填充一下
+        # 以下很繁琐，一句话总结：能填的都填了。
+        if self.from_user_id == user_id:
+            self.from_user_nickname = nickname
+        if self.to_user_id == user_id:
+            self.to_user_nickname = nickname
+        try: # 陌生人时候, 'User'字段可能不存在
+            self.other_user_id = itchat_msg['User']['UserName']
+            self.other_user_nickname = itchat_msg['User']['NickName']
+            if self.other_user_id == self.from_user_id:
+                self.from_user_nickname = self.other_user_nickname
+            if self.other_user_id == self.to_user_id:
+                self.to_user_nickname = self.other_user_nickname
+        except KeyError as e: # 处理偶尔没有对方信息的情况
+            logger.warn("[WX]get other_user_id failed: " + str(e))
+            if self.from_user_id == user_id:
+                self.other_user_id = self.to_user_id
+            else:
+                self.other_user_id = self.from_user_id
+
+        if self.is_group:
+            self.is_at = itchat_msg['IsAt']
+            self.actual_user_id = itchat_msg['ActualUserName']
+            self.actual_user_nickname = itchat_msg['ActualNickName']

channel/wechat/wechaty_channel.py

@@ -4,289 +4,122 @@
 wechaty channel
 Python Wechaty - https://github.com/wechaty/python-wechaty
 """
-import io
+import base64
 import os
-import json
 import time
 import asyncio
-import requests
-import pysilk
-import wave
-from pydub import AudioSegment
-from typing import Optional, Union
-from bridge.context import Context, ContextType
-from wechaty_puppet import MessageType, FileBox, ScanStatus  # type: ignore
+from bridge.context import Context
+from wechaty_puppet import FileBox
 from wechaty import Wechaty, Contact
-from wechaty.user import Message, Room, MiniProgram, UrlLink
-from channel.channel import Channel
+from wechaty.user import Message
+from bridge.reply import *
+from bridge.context import *
+from channel.chat_channel import ChatChannel
+from channel.wechat.wechaty_message import WechatyMessage
 from common.log import logger
-from common.tmp_dir import TmpDir
+from common.singleton import singleton
 from config import conf
+try:
+    from voice.audio_convert import any_to_sil
+except Exception as e:
+    pass
 
-
-class WechatyChannel(Channel):
+@singleton
+class WechatyChannel(ChatChannel):
 
     def __init__(self):
-        pass
+        super().__init__()
 
     def startup(self):
+        config = conf()
+        token = config.get('wechaty_puppet_service_token')
+        os.environ['WECHATY_PUPPET_SERVICE_TOKEN'] = token
         asyncio.run(self.main())
 
     async def main(self):
-        config = conf()
-        # 使用PadLocal协议 比较稳定(免费web协议 os.environ['WECHATY_PUPPET_SERVICE_ENDPOINT'] = '127.0.0.1:8080')
-        token = config.get('wechaty_puppet_service_token')
-        os.environ['WECHATY_PUPPET_SERVICE_TOKEN'] = token
-        global bot
-        bot = Wechaty()
-
-        bot.on('scan', self.on_scan)
-        bot.on('login', self.on_login)
-        bot.on('message', self.on_message)
-        await bot.start()
+        
+        loop = asyncio.get_event_loop()
+        #将asyncio的loop传入处理线程
+        self.handler_pool._initializer= lambda: asyncio.set_event_loop(loop)
+        self.bot = Wechaty()
+        self.bot.on('login', self.on_login)
+        self.bot.on('message', self.on_message)
+        await self.bot.start()
 
     async def on_login(self, contact: Contact):
+        self.user_id = contact.contact_id
+        self.name = contact.name
         logger.info('[WX] login user={}'.format(contact))
 
-    async def on_scan(self, status: ScanStatus, qr_code: Optional[str] = None,
-                      data: Optional[str] = None):
-        contact = self.Contact.load(self.contact_id)
-        logger.info('[WX] scan user={}, scan status={}, scan qr_code={}'.format(contact, status.name, qr_code))
-        # print(f'user <{contact}> scan status: {status.name} , 'f'qr_code: {qr_code}')
+    # 统一的发送函数，每个Channel自行实现，根据reply的type字段发送不同类型的消息
+    def send(self, reply: Reply, context: Context):
+        receiver_id = context['receiver']
+        loop = asyncio.get_event_loop()
+        if context['isgroup']:
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Room.find(receiver_id),loop).result()
+        else:
+            receiver = asyncio.run_coroutine_threadsafe(self.bot.Contact.find(receiver_id),loop).result()
+        msg = None
+        if reply.type == ReplyType.TEXT:
+            msg = reply.content
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
+        elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
+            msg = reply.content
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendMsg={}, receiver={}'.format(reply, receiver))
+        elif reply.type == ReplyType.VOICE:
+            voiceLength = None
+            file_path = reply.content
+            sil_file = os.path.splitext(file_path)[0] + '.sil'
+            voiceLength = int(any_to_sil(file_path, sil_file))
+            if voiceLength >= 60000:
+                voiceLength = 60000
+                logger.info('[WX] voice too long, length={}, set to 60s'.format(voiceLength))
+            # 发送语音
+            t = int(time.time())
+            msg = FileBox.from_file(sil_file, name=str(t) + '.sil')
+            if voiceLength is not None:
+                msg.metadata['voiceLength'] = voiceLength
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            try:
+                os.remove(file_path)
+                if sil_file != file_path:
+                    os.remove(sil_file)
+            except Exception as e:
+                pass
+            logger.info('[WX] sendVoice={}, receiver={}'.format(reply.content, receiver))
+        elif reply.type == ReplyType.IMAGE_URL: # 从网络下载图片
+            img_url = reply.content
+            t = int(time.time())
+            msg = FileBox.from_url(url=img_url, name=str(t) + '.png')
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendImage url={}, receiver={}'.format(img_url,receiver))
+        elif reply.type == ReplyType.IMAGE: # 从文件读取图片
+            image_storage = reply.content
+            image_storage.seek(0)
+            t = int(time.time())
+            msg = FileBox.from_base64(base64.b64encode(image_storage.read()), str(t) + '.png')
+            asyncio.run_coroutine_threadsafe(receiver.say(msg),loop).result()
+            logger.info('[WX] sendImage, receiver={}'.format(receiver))
 
     async def on_message(self, msg: Message):
         """
         listen for message event
         """
-        from_contact = msg.talker()  # 获取消息的发送者
-        to_contact = msg.to()  # 接收人
-        room = msg.room()  # 获取消息来自的群聊. 如果消息不是来自群聊, 则返回None
-        from_user_id = from_contact.contact_id
-        to_user_id = to_contact.contact_id  # 接收人id
-        # other_user_id = msg['User']['UserName']  # 对手方id
-        content = msg.text()
-        mention_content = await msg.mention_text()  # 返回过滤掉@name后的消息
-        match_prefix = self.check_prefix(content, conf().get('single_chat_prefix'))
-        conversation: Union[Room, Contact] = from_contact if room is None else room
-
-        if room is None and msg.type() == MessageType.MESSAGE_TYPE_TEXT:
-            if not msg.is_self() and match_prefix is not None:
-                # 好友向自己发送消息
-                if match_prefix != '':
-                    str_list = content.split(match_prefix, 1)
-                    if len(str_list) == 2:
-                        content = str_list[1].strip()
-
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_img(content, from_user_id)
-                else:
-                    await self._do_send(content, from_user_id)
-            elif msg.is_self() and match_prefix:
-                # 自己给好友发送消息
-                str_list = content.split(match_prefix, 1)
-                if len(str_list) == 2:
-                    content = str_list[1].strip()
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_img(content, to_user_id)
-                else:
-                    await self._do_send(content, to_user_id)
-        elif room is None and msg.type() == MessageType.MESSAGE_TYPE_AUDIO:
-            if not msg.is_self(): # 接收语音消息
-                # 下载语音文件
-                voice_file = await msg.to_file_box()
-                silk_file = TmpDir().path() + voice_file.name
-                await voice_file.to_file(silk_file)
-                logger.info("[WX]receive voice file: " + silk_file)
-                # 将文件转成wav格式音频
-                wav_file = silk_file.replace(".slk", ".wav")
-                with open(silk_file, 'rb') as f:
-                    silk_data = f.read()
-                pcm_data = pysilk.decode(silk_data)
-
-                with wave.open(wav_file, 'wb') as wav_data:
-                    wav_data.setnchannels(1)
-                    wav_data.setsampwidth(2)
-                    wav_data.setframerate(24000)
-                    wav_data.writeframes(pcm_data)
-                if os.path.exists(wav_file): 
-                    converter_state = "true" # 转换wav成功
-                else:
-                    converter_state = "false" # 转换wav失败
-                logger.info("[WX]receive voice converter: " + converter_state)
-                # 语音识别为文本
-                query = super().build_voice_to_text(wav_file).content
-                # 交验关键字
-                match_prefix = self.check_prefix(query, conf().get('single_chat_prefix'))
-                if match_prefix is not None:
-                    if match_prefix != '':
-                        str_list = query.split(match_prefix, 1)
-                        if len(str_list) == 2:
-                            query = str_list[1].strip()
-                    # 返回消息
-                    if conf().get('voice_reply_voice'):
-                        await self._do_send_voice(query, from_user_id)
-                    else:
-                        await self._do_send(query, from_user_id)
-                else:
-                    logger.info("[WX]receive voice check prefix: " + 'False')
-                # 清除缓存文件
-                os.remove(wav_file)
-                os.remove(silk_file)
-        elif room and msg.type() == MessageType.MESSAGE_TYPE_TEXT:
-            # 群组&文本消息
-            room_id = room.room_id
-            room_name = await room.topic()
-            from_user_id = from_contact.contact_id
-            from_user_name = from_contact.name
-            is_at = await msg.mention_self()
-            content = mention_content
-            config = conf()
-            match_prefix = (is_at and not config.get("group_at_off", False)) \
-                           or self.check_prefix(content, config.get('group_chat_prefix')) \
-                           or self.check_contain(content, config.get('group_chat_keyword'))
-            # Wechaty判断is_at为True，返回的内容是过滤掉@之后的内容；而is_at为False，则会返回完整的内容
-            # 故判断如果匹配到自定义前缀，则返回过滤掉前缀+空格后的内容，用于实现类似自定义+前缀触发生成AI图片的功能
-            prefixes = config.get('group_chat_prefix')
-            for prefix in prefixes:
-                if content.startswith(prefix):
-                    content = content.replace(prefix, '', 1).strip()
-                    break
-            if ('ALL_GROUP' in config.get('group_name_white_list') or room_name in config.get(
-                    'group_name_white_list') or self.check_contain(room_name, config.get(
-                'group_name_keyword_white_list'))) and match_prefix:
-                img_match_prefix = self.check_prefix(content, conf().get('image_create_prefix'))
-                if img_match_prefix:
-                    content = content.split(img_match_prefix, 1)[1].strip()
-                    await self._do_send_group_img(content, room_id)
-                else:
-                    await self._do_send_group(content, room_id, room_name, from_user_id, from_user_name)
-
-    async def send(self, message: Union[str, Message, FileBox, Contact, UrlLink, MiniProgram], receiver):
-        logger.info('[WX] sendMsg={}, receiver={}'.format(message, receiver))
-        if receiver:
-            contact = await bot.Contact.find(receiver)
-            await contact.say(message)
-
-    async def send_group(self, message: Union[str, Message, FileBox, Contact, UrlLink, MiniProgram], receiver):
-        logger.info('[WX] sendMsg={}, receiver={}'.format(message, receiver))
-        if receiver:
-            room = await bot.Room.find(receiver)
-            await room.say(message)
-
-    async def _do_send(self, query, reply_user_id):
         try:
-            if not query:
-                return
-            context = Context(ContextType.TEXT, query)
-            context['session_id'] = reply_user_id
-            reply_text = super().build_reply_content(query, context).content
-            if reply_text:
-                await self.send(conf().get("single_chat_reply_prefix") + reply_text, reply_user_id)
-        except Exception as e:
-            logger.exception(e)
-
-
-    async def _do_send_voice(self, query, reply_user_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.TEXT, query)
-            context['session_id'] = reply_user_id
-            reply_text = super().build_reply_content(query, context).content
-            if reply_text:
-                # 转换 mp3 文件为 silk 格式
-                mp3_file = super().build_text_to_voice(reply_text).content
-                silk_file = mp3_file.replace(".mp3", ".silk")
-                # Load the MP3 file
-                audio = AudioSegment.from_file(mp3_file, format="mp3")
-                # Convert to WAV format
-                audio = audio.set_frame_rate(24000).set_channels(1)
-                wav_data = audio.raw_data
-                sample_width = audio.sample_width
-                # Encode to SILK format
-                silk_data = pysilk.encode(wav_data, 24000)
-                # Save the silk file
-                with open(silk_file, "wb") as f:
-                    f.write(silk_data)
-                # 发送语音
-                t = int(time.time())
-                file_box = FileBox.from_file(silk_file, name=str(t) + '.silk')
-                await self.send(file_box, reply_user_id)
-                # 清除缓存文件
-                os.remove(mp3_file)
-                os.remove(silk_file)
-        except Exception as e:
-            logger.exception(e)
-            
-    async def _do_send_img(self, query, reply_user_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.IMAGE_CREATE, query)
-            img_url = super().build_reply_content(query, context).content
-            if not img_url:
-                return
-            # 图片下载
-            # pic_res = requests.get(img_url, stream=True)
-            # image_storage = io.BytesIO()
-            # for block in pic_res.iter_content(1024):
-            #     image_storage.write(block)
-            # image_storage.seek(0)
-
-            # 图片发送
-            logger.info('[WX] sendImage, receiver={}'.format(reply_user_id))
-            t = int(time.time())
-            file_box = FileBox.from_url(url=img_url, name=str(t) + '.png')
-            await self.send(file_box, reply_user_id)
-        except Exception as e:
-            logger.exception(e)
-
-    async def _do_send_group(self, query, group_id, group_name, group_user_id, group_user_name):
-        if not query:
+            cmsg = await WechatyMessage(msg)
+        except NotImplementedError as e:
+            logger.debug('[WX] {}'.format(e))
             return
-        context = Context(ContextType.TEXT, query)
-        group_chat_in_one_session = conf().get('group_chat_in_one_session', [])
-        if ('ALL_GROUP' in group_chat_in_one_session or \
-                group_name in group_chat_in_one_session or \
-                self.check_contain(group_name, group_chat_in_one_session)):
-            context['session_id'] = str(group_id)
-        else:
-            context['session_id'] = str(group_id) + '-' + str(group_user_id)
-        reply_text = super().build_reply_content(query, context).content
-        if reply_text:
-            reply_text = '@' + group_user_name + ' ' + reply_text.strip()
-            await self.send_group(conf().get("group_chat_reply_prefix", "") + reply_text, group_id)
-
-    async def _do_send_group_img(self, query, reply_room_id):
-        try:
-            if not query:
-                return
-            context = Context(ContextType.IMAGE_CREATE, query)
-            img_url = super().build_reply_content(query, context).content
-            if not img_url:
-                return
-            # 图片发送
-            logger.info('[WX] sendImage, receiver={}'.format(reply_room_id))
-            t = int(time.time())
-            file_box = FileBox.from_url(url=img_url, name=str(t) + '.png')
-            await self.send_group(file_box, reply_room_id)
         except Exception as e:
-            logger.exception(e)
-
-    def check_prefix(self, content, prefix_list):
-        for prefix in prefix_list:
-            if content.startswith(prefix):
-                return prefix
-        return None
-
-    def check_contain(self, content, keyword_list):
-        if not keyword_list:
-            return None
-        for ky in keyword_list:
-            if content.find(ky) != -1:
-                return True
-        return None
+            logger.exception('[WX] {}'.format(e))
+            return
+        logger.debug('[WX] message:{}'.format(cmsg))
+        room = msg.room()  # 获取消息来自的群聊. 如果消息不是来自群聊, 则返回None
+        isgroup = room is not None
+        ctype = cmsg.ctype
+        context = self._compose_context(ctype, cmsg.content, isgroup=isgroup, msg=cmsg)
+        if context:
+            logger.info('[WX] receiveMsg={}, context={}'.format(cmsg, context))
+            self.produce(context)

channel/wechat/wechaty_message.py

@@ -0,0 +1,85 @@
+import asyncio
+import re
+from wechaty import MessageType
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+from common.tmp_dir import TmpDir
+from common.log import logger
+from wechaty.user import Message
+
+class aobject(object):
+    """Inheriting this class allows you to define an async __init__.
+
+    So you can create objects by doing something like `await MyClass(params)`
+    """
+    async def __new__(cls, *a, **kw):
+        instance = super().__new__(cls)
+        await instance.__init__(*a, **kw)
+        return instance
+
+    async def __init__(self):
+        pass
+class WechatyMessage(ChatMessage, aobject):
+
+    async def __init__(self, wechaty_msg: Message):
+        super().__init__(wechaty_msg)
+        
+        room = wechaty_msg.room()
+
+        self.msg_id = wechaty_msg.message_id
+        self.create_time = wechaty_msg.payload.timestamp
+        self.is_group = room is not None
+        
+        if wechaty_msg.type() == MessageType.MESSAGE_TYPE_TEXT:
+            self.ctype = ContextType.TEXT
+            self.content = wechaty_msg.text()
+        elif wechaty_msg.type() == MessageType.MESSAGE_TYPE_AUDIO:
+            self.ctype = ContextType.VOICE
+            voice_file = await wechaty_msg.to_file_box()
+            self.content = TmpDir().path() + voice_file.name  # content直接存临时目录路径
+
+            def func():
+                loop = asyncio.get_event_loop()
+                asyncio.run_coroutine_threadsafe(voice_file.to_file(self.content),loop).result()
+            self._prepare_fn = func
+            
+        else:
+            raise NotImplementedError("Unsupported message type: {}".format(wechaty_msg.type()))
+        
+        from_contact = wechaty_msg.talker()  # 获取消息的发送者
+        self.from_user_id = from_contact.contact_id
+        self.from_user_nickname = from_contact.name
+
+        # group中的from和to，wechaty跟itchat含义不一样
+        # wecahty: from是消息实际发送者, to:所在群
+        # itchat: 如果是你发送群消息，from和to是你自己和所在群，如果是别人发群消息，from和to是所在群和你自己
+        # 但这个差别不影响逻辑，group中只使用到：1.用from来判断是否是自己发的，2.actual_user_id来判断实际发送用户
+        
+        if self.is_group:
+            self.to_user_id = room.room_id
+            self.to_user_nickname = await room.topic()
+        else:
+            to_contact = wechaty_msg.to()
+            self.to_user_id = to_contact.contact_id
+            self.to_user_nickname = to_contact.name
+
+        if self.is_group or wechaty_msg.is_self(): # 如果是群消息，other_user设置为群，如果是私聊消息，而且自己发的，就设置成对方。
+            self.other_user_id = self.to_user_id
+            self.other_user_nickname = self.to_user_nickname
+        else:
+            self.other_user_id = self.from_user_id
+            self.other_user_nickname = self.from_user_nickname
+
+        
+
+        if self.is_group: # wechaty群聊中，实际发送用户就是from_user
+            self.is_at = await wechaty_msg.mention_self()
+            if not self.is_at: # 有时候复制粘贴的消息，不算做@，但是内容里面会有@xxx，这里做一下兼容
+                name = wechaty_msg.wechaty.user_self().name
+                pattern = f'@{name}(\u2005|\u0020)'
+                if re.search(pattern,self.content):
+                    logger.debug(f'wechaty message {self.msg_id} include at')
+                    self.is_at = True
+
+            self.actual_user_id = self.from_user_id
+            self.actual_user_nickname = self.from_user_nickname

channel/wechatmp/README.md

@@ -0,0 +1,34 @@
+# 个人微信公众号channel
+
+鉴于个人微信号在服务器上通过itchat登录有封号风险，这里新增了个人微信公众号channel，提供无风险的服务。
+但是由于个人微信公众号的众多接口限制，目前支持的功能有限，实现简陋，提供了一个最基本的文本对话服务，支持加载插件，优化了命令格式，支持私有api_key。暂未实现图片输入输出、语音输入输出等交互形式。
+如有公众号是企业主体且可以通过微信认证，即可获得更多接口，解除大多数限制。欢迎大家提供更多的支持。
+
+## 使用方法
+
+在开始部署前，你需要一个拥有公网IP的服务器，以提供微信服务器和我们自己服务器的连接。或者你需要进行内网穿透，否则微信服务器无法将消息发送给我们的服务器。
+
+此外，需要在我们的服务器上安装python的web框架web.py。
+以ubuntu为例(在ubuntu 22.04上测试):
+```
+pip3 install web.py
+```
+
+然后在[微信公众平台](https://mp.weixin.qq.com)注册一个自己的公众号，类型选择订阅号，主体为个人即可。
+
+然后根据[接入指南](https://developers.weixin.qq.com/doc/offiaccount/Basic_Information/Access_Overview.html)的说明，在[微信公众平台](https://mp.weixin.qq.com)的“设置与开发”-“基本配置”-“服务器配置”中填写服务器地址`URL`和令牌`Token`。这里的`URL`是`example.com/wx`的形式，不可以使用IP，`Token`是你自己编的一个特定的令牌。消息加解密方式目前选择的是明文模式。 
+
+相关的服务器验证代码已经写好，你不需要再添加任何代码。你只需要在本项目根目录的`config.json`中添加`"channel_type": "wechatmp", "wechatmp_token": "your Token", ` 然后运行`python3 app.py`启动web服务器，然后在刚才的“服务器配置”中点击`提交`即可验证你的服务器。
+
+随后在[微信公众平台](https://mp.weixin.qq.com)启用服务器，关闭手动填写规则的自动回复，即可实现ChatGPT的自动回复。
+
+## 个人微信公众号的限制
+由于目前测试的公众号不是企业主体，所以没有客服接口，因此公众号无法主动发出消息，只能被动回复。而微信官方对被动回复有5秒的时间限制，最多重试2次，因此最多只有15秒的自动回复时间窗口。因此如果问题比较复杂或者我们的服务器比较忙，ChatGPT的回答就没办法及时回复给用户。为了解决这个问题，这里做了回答缓存，它需要你在回复超时后，再次主动发送任意文字（例如1）来尝试拿到回答缓存。为了优化使用体验，目前设置了两分钟（120秒）的timeout，用户在至多两分钟后即可得到查询到回复或者错误原因。
+
+另外，由于微信官方的限制，自动回复有长度限制。因此这里将ChatGPT的回答拆分，分成每段600字回复（限制大约在700字）。
+
+## 私有api_key
+公共api有访问频率限制（免费账号每分钟最多20次ChatGPT的API调用），这在服务多人的时候会遇到问题。因此这里多加了一个设置私有api_key的功能。目前通过godcmd插件的命令来设置私有api_key。
+
+## 测试范围
+目前在`RoboStyle`这个公众号上进行了测试，感兴趣的可以关注并体验。开启了godcmd, Banwords, role, dungeon, finish这五个插件，其他的插件还没有测试。百度的接口暂未测试。语音对话没有测试。图片直接以链接形式回复（没有临时素材上传接口的权限）。

channel/wechatmp/receive.py

@@ -0,0 +1,47 @@
+# -*- coding: utf-8 -*-#
+# filename: receive.py
+import xml.etree.ElementTree as ET
+
+
+def parse_xml(web_data):
+    if len(web_data) == 0:
+        return None
+    xmlData = ET.fromstring(web_data)
+    msg_type = xmlData.find('MsgType').text
+    if msg_type == 'text':
+        return TextMsg(xmlData)
+    elif msg_type == 'image':
+        return ImageMsg(xmlData)
+    elif msg_type == 'event':
+        return Event(xmlData)
+
+
+class Msg(object):
+    def __init__(self, xmlData):
+        self.ToUserName = xmlData.find('ToUserName').text
+        self.FromUserName = xmlData.find('FromUserName').text
+        self.CreateTime = xmlData.find('CreateTime').text
+        self.MsgType = xmlData.find('MsgType').text
+        self.MsgId = xmlData.find('MsgId').text
+
+
+class TextMsg(Msg):
+    def __init__(self, xmlData):
+        Msg.__init__(self, xmlData)
+        self.Content = xmlData.find('Content').text.encode("utf-8")
+
+
+class ImageMsg(Msg):
+    def __init__(self, xmlData):
+        Msg.__init__(self, xmlData)
+        self.PicUrl = xmlData.find('PicUrl').text
+        self.MediaId = xmlData.find('MediaId').text
+
+
+class Event(object):
+    def __init__(self, xmlData):
+        self.ToUserName = xmlData.find('ToUserName').text
+        self.FromUserName = xmlData.find('FromUserName').text
+        self.CreateTime = xmlData.find('CreateTime').text
+        self.MsgType = xmlData.find('MsgType').text
+        self.Event = xmlData.find('Event').text

channel/wechatmp/reply.py

@@ -0,0 +1,52 @@
+# -*- coding: utf-8 -*-#
+# filename: reply.py
+import time
+
+class Msg(object):
+    def __init__(self):
+        pass
+
+    def send(self):
+        return "success"
+
+class TextMsg(Msg):
+    def __init__(self, toUserName, fromUserName, content):
+        self.__dict = dict()
+        self.__dict['ToUserName'] = toUserName
+        self.__dict['FromUserName'] = fromUserName
+        self.__dict['CreateTime'] = int(time.time())
+        self.__dict['Content'] = content
+
+    def send(self):
+        XmlForm = """
+            <xml>
+                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
+                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
+                <CreateTime>{CreateTime}</CreateTime>
+                <MsgType><![CDATA[text]]></MsgType>
+                <Content><![CDATA[{Content}]]></Content>
+            </xml>
+            """
+        return XmlForm.format(**self.__dict)
+
+class ImageMsg(Msg):
+    def __init__(self, toUserName, fromUserName, mediaId):
+        self.__dict = dict()
+        self.__dict['ToUserName'] = toUserName
+        self.__dict['FromUserName'] = fromUserName
+        self.__dict['CreateTime'] = int(time.time())
+        self.__dict['MediaId'] = mediaId
+
+    def send(self):
+        XmlForm = """
+            <xml>
+                <ToUserName><![CDATA[{ToUserName}]]></ToUserName>
+                <FromUserName><![CDATA[{FromUserName}]]></FromUserName>
+                <CreateTime>{CreateTime}</CreateTime>
+                <MsgType><![CDATA[image]]></MsgType>
+                <Image>
+                <MediaId><![CDATA[{MediaId}]]></MediaId>
+                </Image>
+            </xml>
+            """
+        return XmlForm.format(**self.__dict)

channel/wechatmp/wechatmp_channel.py

@@ -0,0 +1,302 @@
+# -*- coding: utf-8 -*-
+import web
+import time
+import math
+import hashlib
+import textwrap
+from channel.channel import Channel
+import channel.wechatmp.reply as reply
+import channel.wechatmp.receive as receive
+from common.log import logger
+from config import conf
+from bridge.reply import *
+from bridge.context import *
+from plugins import *
+import traceback
+
+# If using SSL, uncomment the following lines, and modify the certificate path.
+# from cheroot.server import HTTPServer
+# from cheroot.ssl.builtin import BuiltinSSLAdapter
+# HTTPServer.ssl_adapter = BuiltinSSLAdapter(
+#         certificate='/ssl/cert.pem',
+#         private_key='/ssl/cert.key')
+
+class WechatMPServer():
+    def __init__(self):
+        pass
+
+    def startup(self): 
+        urls = (
+            '/wx', 'WechatMPChannel',
+        )
+        app = web.application(urls, globals())
+        web.httpserver.runsimple(app.wsgifunc(), ('0.0.0.0', 80))
+
+cache_dict = dict()
+query1 = dict()
+query2 = dict()
+query3 = dict()
+
+from concurrent.futures import ThreadPoolExecutor
+thread_pool = ThreadPoolExecutor(max_workers=8)
+
+class WechatMPChannel(Channel):
+
+    def GET(self):
+        try:
+            data = web.input()
+            if len(data) == 0:
+                return "hello, this is handle view"
+            signature = data.signature
+            timestamp = data.timestamp
+            nonce = data.nonce
+            echostr = data.echostr
+            token = conf().get('wechatmp_token') #请按照公众平台官网\基本配置中信息填写
+
+            data_list = [token, timestamp, nonce]
+            data_list.sort()
+            sha1 = hashlib.sha1()
+            # map(sha1.update, data_list) #python2
+            sha1.update("".join(data_list).encode('utf-8'))
+            hashcode = sha1.hexdigest()
+            print("handle/GET func: hashcode, signature: ", hashcode, signature)
+            if hashcode == signature:
+                return echostr
+            else:
+                return ""
+        except Exception as Argument:
+            return Argument
+
+
+    def _do_build_reply(self, cache_key, fromUser, message):
+        context = dict()
+        context['session_id'] = fromUser
+        reply_text = super().build_reply_content(message, context)
+        # The query is done, record the cache
+        logger.info("[threaded] Get reply for {}: {} \nA: {}".format(fromUser, message, reply_text))
+        global cache_dict
+        reply_cnt = math.ceil(len(reply_text) / 600)
+        cache_dict[cache_key] = (reply_cnt, reply_text)
+
+
+    def send(self, reply : Reply, cache_key):
+        global cache_dict
+        reply_cnt = math.ceil(len(reply.content) / 600)
+        cache_dict[cache_key] = (reply_cnt, reply.content)
+
+
+    def handle(self, context):
+        global cache_dict
+        try:
+            reply = Reply()
+            logger.debug('[wechatmp] ready to handle context: {}'.format(context))
+
+            # reply的构建步骤
+            e_context = PluginManager().emit_event(EventContext(Event.ON_HANDLE_CONTEXT, {'channel' : self, 'context': context, 'reply': reply}))
+            reply = e_context['reply']
+            if not e_context.is_pass():
+                logger.debug('[wechatmp] ready to handle context: type={}, content={}'.format(context.type, context.content))
+                if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:
+                    reply = super().build_reply_content(context.content, context)
+                # elif context.type == ContextType.VOICE:
+                #     msg = context['msg']
+                #     file_name = TmpDir().path() + context.content
+                #     msg.download(file_name)
+                #     reply = super().build_voice_to_text(file_name)
+                #     if reply.type != ReplyType.ERROR and reply.type != ReplyType.INFO:
+                #         context.content = reply.content # 语音转文字后，将文字内容作为新的context
+                #         context.type = ContextType.TEXT
+                #         reply = super().build_reply_content(context.content, context)
+                #         if reply.type == ReplyType.TEXT:
+                #             if conf().get('voice_reply_voice'):
+                #                 reply = super().build_text_to_voice(reply.content)
+                else:
+                    logger.error('[wechatmp] unknown context type: {}'.format(context.type))
+                    return
+
+            logger.debug('[wechatmp] ready to decorate reply: {}'.format(reply))
+
+            # reply的包装步骤
+            if reply and reply.type:
+                e_context = PluginManager().emit_event(EventContext(Event.ON_DECORATE_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
+                reply=e_context['reply']
+                if not e_context.is_pass() and reply and reply.type:
+                    if reply.type == ReplyType.TEXT:
+                        pass
+                    elif reply.type == ReplyType.ERROR or reply.type == ReplyType.INFO:
+                        reply.content = str(reply.type)+":\n" + reply.content
+                    elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE:
+                        pass
+                    else:
+                        logger.error('[wechatmp] unknown reply type: {}'.format(reply.type))
+                        return
+
+            # reply的发送步骤
+            if reply and reply.type:
+                e_context = PluginManager().emit_event(EventContext(Event.ON_SEND_REPLY, {'channel' : self, 'context': context, 'reply': reply}))
+                reply=e_context['reply']
+                if not e_context.is_pass() and reply and reply.type:
+                    logger.debug('[wechatmp] ready to send reply: {} to {}'.format(reply, context['receiver']))
+                    self.send(reply, context['receiver'])
+            else:
+                cache_dict[context['receiver']] = (1, "No reply")
+
+            logger.info("[threaded] Get reply for {}: {} \nA: {}".format(context['receiver'], context.content, reply.content))
+        except Exception as exc:
+            print(traceback.format_exc())
+            cache_dict[context['receiver']] = (1, "ERROR")
+
+
+
+    def POST(self):
+        try:
+            queryTime = time.time()
+            webData = web.data()
+            # logger.debug("[wechatmp] Receive request:\n" + webData.decode("utf-8"))
+            recMsg = receive.parse_xml(webData)
+            if isinstance(recMsg, receive.Msg) and recMsg.MsgType == 'text':
+                fromUser = recMsg.FromUserName
+                toUser = recMsg.ToUserName
+                createTime = recMsg.CreateTime
+                message = recMsg.Content.decode("utf-8")
+                message_id = recMsg.MsgId
+
+                logger.info("[wechatmp] {}:{} Receive post query {} {}: {}".format(web.ctx.env.get('REMOTE_ADDR'), web.ctx.env.get('REMOTE_PORT'), fromUser, message_id, message))
+
+                global cache_dict
+                global query1
+                global query2
+                global query3
+                cache_key = fromUser
+                cache = cache_dict.get(cache_key)
+
+                reply_text = ""
+                # New request
+                if cache == None:
+                    # The first query begin, reset the cache
+                    cache_dict[cache_key] = (0, "")
+                    # thread_pool.submit(self._do_build_reply, cache_key, fromUser, message)
+
+                    context = Context()
+                    context.kwargs = {'isgroup': False, 'receiver': fromUser, 'session_id': fromUser}
+
+                    user_data = conf().get_user_data(fromUser)
+                    context['openai_api_key'] = user_data.get('openai_api_key') # None or user openai_api_key
+
+                    img_match_prefix = check_prefix(message, conf().get('image_create_prefix'))
+                    if img_match_prefix:
+                        message = message.replace(img_match_prefix, '', 1).strip()
+                        context.type = ContextType.IMAGE_CREATE
+                    else:
+                        context.type = ContextType.TEXT
+                    context.content = message
+                    thread_pool.submit(self.handle, context)
+
+                    query1[cache_key] = False
+                    query2[cache_key] = False
+                    query3[cache_key] = False
+                # Request again
+                elif cache[0] == 0 and query1.get(cache_key) == True and query2.get(cache_key) == True and query3.get(cache_key) == True:
+                    query1[cache_key] = False  #To improve waiting experience, this can be set to True.
+                    query2[cache_key] = False  #To improve waiting experience, this can be set to True.
+                    query3[cache_key] = False
+                elif cache[0] >= 1:
+                    # Skip the waiting phase
+                    query1[cache_key] = True
+                    query2[cache_key] = True
+                    query3[cache_key] = True
+
+
+                cache = cache_dict.get(cache_key)
+                if query1.get(cache_key) == False:
+                    # The first query from wechat official server
+                    logger.debug("[wechatmp] query1 {}".format(cache_key))
+                    query1[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 45:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = cache_dict.get(cache_key)
+                    if cnt == 45:
+                        # waiting for timeout (the POST query will be closed by wechat official server)
+                        time.sleep(5)
+                        # and do nothing
+                        return
+                    else:
+                        pass
+                elif query2.get(cache_key) == False:
+                    # The second query from wechat official server
+                    logger.debug("[wechatmp] query2 {}".format(cache_key))
+                    query2[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 45:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = cache_dict.get(cache_key)
+                    if cnt == 45:
+                        # waiting for timeout (the POST query will be closed by wechat official server)
+                        time.sleep(5)
+                        # and do nothing
+                        return
+                    else:
+                        pass
+                elif query3.get(cache_key) == False:
+                    # The third query from wechat official server
+                    logger.debug("[wechatmp] query3 {}".format(cache_key))
+                    query3[cache_key] = True
+                    cnt = 0
+                    while cache[0] == 0 and cnt < 45:
+                        cnt = cnt + 1
+                        time.sleep(0.1)
+                        cache = cache_dict.get(cache_key)
+                    if cnt == 45:
+                        # Have waiting for 3x5 seconds
+                        # return timeout message
+                        reply_text = "【正在响应中，回复任意文字尝试获取回复】"
+                        logger.info("[wechatmp] Three queries has finished For {}: {}".format(fromUser, message_id))
+                        replyPost = reply.TextMsg(fromUser, toUser, reply_text).send()
+                        return replyPost
+                    else:
+                        pass
+
+                if float(time.time()) - float(queryTime) > 4.8:
+                    logger.info("[wechatmp] Timeout for {} {}".format(fromUser, message_id))
+                    return
+
+
+                if cache[0] > 1:
+                    reply_text = cache[1][:600] + "\n【未完待续，回复任意文字以继续】" #wechatmp auto_reply length limit
+                    cache_dict[cache_key] = (cache[0] - 1, cache[1][600:])
+                elif cache[0] == 1:
+                    reply_text = cache[1]
+                    cache_dict.pop(cache_key)
+                logger.info("[wechatmp] {}:{} Do send {}".format(web.ctx.env.get('REMOTE_ADDR'), web.ctx.env.get('REMOTE_PORT'), reply_text))
+                replyPost = reply.TextMsg(fromUser, toUser, reply_text).send()
+                return replyPost
+
+            elif isinstance(recMsg, receive.Event) and recMsg.MsgType == 'event':
+                logger.info("[wechatmp] Event {} from {}".format(recMsg.Event, recMsg.FromUserName))
+                content = textwrap.dedent("""\
+                    感谢您的关注！
+                    这里是ChatGPT，可以自由对话。
+                    资源有限，回复较慢，请勿着急。
+                    支持通用表情输入。
+                    暂时不支持图片输入。
+                    支持图片输出，画字开头的问题将回复图片链接。
+                    支持角色扮演和文字冒险两种定制模式对话。
+                    输入'#帮助' 查看详细指令。""")
+                replyMsg = reply.TextMsg(recMsg.FromUserName, recMsg.ToUserName, content)
+                return replyMsg.send()
+            else:
+                logger.info("暂且不处理")
+                return "success"
+        except Exception as exc:
+            logger.exception(exc)
+            return exc
+
+
+def check_prefix(content, prefix_list):
+    for prefix in prefix_list:
+        if content.startswith(prefix):
+            return prefix
+    return None

common/dequeue.py

@@ -0,0 +1,33 @@
+
+from queue import Full, Queue
+from time import monotonic as time
+
+# add implementation of putleft to Queue
+class Dequeue(Queue):
+    def putleft(self, item, block=True, timeout=None):
+        with self.not_full:
+            if self.maxsize > 0:
+                if not block:
+                    if self._qsize() >= self.maxsize:
+                        raise Full
+                elif timeout is None:
+                    while self._qsize() >= self.maxsize:
+                        self.not_full.wait()
+                elif timeout < 0:
+                    raise ValueError("'timeout' must be a non-negative number")
+                else:
+                    endtime = time() + timeout
+                    while self._qsize() >= self.maxsize:
+                        remaining = endtime - time()
+                        if remaining <= 0.0:
+                            raise Full
+                        self.not_full.wait(remaining)
+            self._putleft(item)
+            self.unfinished_tasks += 1
+            self.not_empty.notify()
+
+    def putleft_nowait(self, item):
+        return self.putleft(item, block=False)
+
+    def _putleft(self, item):
+        self.queue.appendleft(item)

common/log.py

@@ -8,6 +8,10 @@ def _get_logger():
     console_handle = logging.StreamHandler(sys.stdout)
     console_handle.setFormatter(logging.Formatter('[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d] - %(message)s',
                                                   datefmt='%Y-%m-%d %H:%M:%S'))
+    file_handle = logging.FileHandler('run.log', encoding='utf-8')
+    file_handle.setFormatter(logging.Formatter('[%(levelname)s][%(asctime)s][%(filename)s:%(lineno)d] - %(message)s',
+                                                  datefmt='%Y-%m-%d %H:%M:%S'))
+    log.addHandler(file_handle)
     log.addHandler(console_handle)
     return log
 

config-template.json

@@ -10,6 +10,7 @@
   "group_chat_in_one_session": ["ChatGPT测试群"],
   "image_create_prefix": ["画", "看", "找"],
   "speech_recognition": false,
+  "group_speech_recognition": false,
   "voice_reply_voice": false,
   "conversation_max_tokens": 1000,
   "expires_in_seconds": 3600,

config.py

@@ -1,75 +1,104 @@
 # encoding:utf-8
 
 import json
+import logging
 import os
 from common.log import logger
+import pickle
 
 # 将所有可用的配置项写在字典里, 请使用小写字母
-available_setting ={
-    #openai api配置
-    "open_ai_api_key": "", # openai api key
-    "open_ai_api_base": "https://api.openai.com/v1", # openai apibase，当use_azure_chatgpt为true时，需要设置对应的api base
-    "proxy": "", # openai使用的代理
-    "model": "gpt-3.5-turbo", # chatgpt模型， 当use_azure_chatgpt为true时，其名称为Azure上model deployment名称
-    "use_azure_chatgpt": False, # 是否使用azure的chatgpt
+available_setting = {
+    # openai api配置
+    "open_ai_api_key": "",  # openai api key
+    # openai apibase，当use_azure_chatgpt为true时，需要设置对应的api base
+    "open_ai_api_base": "https://api.openai.com/v1",
+    "proxy": "",  # openai使用的代理
+    # chatgpt模型， 当use_azure_chatgpt为true时，其名称为Azure上model deployment名称
+    "model": "gpt-3.5-turbo",
+    "use_azure_chatgpt": False,  # 是否使用azure的chatgpt
 
-    #Bot触发配置
-    "single_chat_prefix": ["bot", "@bot"], # 私聊时文本需要包含该前缀才能触发机器人回复
-    "single_chat_reply_prefix": "[bot] ", # 私聊时自动回复的前缀，用于区分真人
-    "group_chat_prefix": ["@bot"], # 群聊时包含该前缀则会触发机器人回复
-    "group_chat_reply_prefix": "", # 群聊时自动回复的前缀
-    "group_chat_keyword": [], # 群聊时包含该关键词则会触发机器人回复
-    "group_at_off": False, # 是否关闭群聊时@bot的触发
-    "group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"], # 开启自动回复的群名称列表
-    "group_name_keyword_white_list": [], # 开启自动回复的群名称关键词列表
-    "group_chat_in_one_session": ["ChatGPT测试群"], # 支持会话上下文共享的群名称
-    "image_create_prefix": ["画", "看", "找"], # 开启图片回复的前缀
-    
-    #chatgpt会话参数
-    "expires_in_seconds": 3600, # 无操作会话的过期时间
-    "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。", # 人格描述
-    "conversation_max_tokens": 1000, # 支持上下文记忆的最多字符数
-            
-    #chatgpt限流配置
-    "rate_limit_chatgpt": 20, # chatgpt的调用频率限制
-    "rate_limit_dalle": 50, # openai dalle的调用频率限制
+    # Bot触发配置
+    "single_chat_prefix": ["bot", "@bot"],  # 私聊时文本需要包含该前缀才能触发机器人回复
+    "single_chat_reply_prefix": "[bot] ",  # 私聊时自动回复的前缀，用于区分真人
+    "group_chat_prefix": ["@bot"],  # 群聊时包含该前缀则会触发机器人回复
+    "group_chat_reply_prefix": "",  # 群聊时自动回复的前缀
+    "group_chat_keyword": [],  # 群聊时包含该关键词则会触发机器人回复
+    "group_at_off": False,  # 是否关闭群聊时@bot的触发
+    "group_name_white_list": ["ChatGPT测试群", "ChatGPT测试群2"],  # 开启自动回复的群名称列表
+    "group_name_keyword_white_list": [],  # 开启自动回复的群名称关键词列表
+    "group_chat_in_one_session": ["ChatGPT测试群"],  # 支持会话上下文共享的群名称
+    "trigger_by_self": False,  # 是否允许机器人触发
+    "image_create_prefix": ["画", "看", "找"],  # 开启图片回复的前缀
+    "concurrency_in_session": 1, # 同一会话最多有多少条消息在处理中，大于1可能乱序
 
+    # chatgpt会话参数
+    "expires_in_seconds": 3600,  # 无操作会话的过期时间
+    "character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。",  # 人格描述
+    "conversation_max_tokens": 1000,  # 支持上下文记忆的最多字符数
 
-    #chatgpt api参数 参考https://platform.openai.com/docs/api-reference/chat/create
+    # chatgpt限流配置
+    "rate_limit_chatgpt": 20,  # chatgpt的调用频率限制
+    "rate_limit_dalle": 50,  # openai dalle的调用频率限制
+
+    # chatgpt api参数 参考https://platform.openai.com/docs/api-reference/chat/create
     "temperature": 0.9,
     "top_p": 1,
     "frequency_penalty": 0,
     "presence_penalty": 0,
+    "request_timeout": 30,  # chatgpt请求超时时间
 
-    #语音设置
-    "speech_recognition": False, # 是否开启语音识别
-    "voice_reply_voice": False, # 是否使用语音回复语音，需要设置对应语音合成引擎的api key
-    "voice_to_text": "openai", # 语音识别引擎，支持openai和google
-    "text_to_voice": "baidu", # 语音合成引擎，支持baidu和google
+    # 语音设置
+    "speech_recognition": False,  # 是否开启语音识别
+    "group_speech_recognition": False,  # 是否开启群组语音识别
+    "voice_reply_voice": False,  # 是否使用语音回复语音，需要设置对应语音合成引擎的api key
+    "always_reply_voice": False,  # 是否一直使用语音回复
+    "voice_to_text": "openai",  # 语音识别引擎，支持openai,google,azure
+    "text_to_voice": "baidu",  # 语音合成引擎，支持baidu,google,pytts(offline),azure
 
-    # baidu api的配置， 使用百度语音识别和语音合成时需要
-    'baidu_app_id': "",
-    'baidu_api_key': "",
-    'baidu_secret_key': "",
+    # baidu 语音api配置， 使用百度语音识别和语音合成时需要
+    "baidu_app_id": "",
+    "baidu_api_key": "",
+    "baidu_secret_key": "",
+    # 1536普通话(支持简单的英文识别) 1737英语 1637粤语 1837四川话 1936普通话远场
+    "baidu_dev_pid": "1536",
 
-    #服务时间限制，目前支持itchat
-    "chat_time_module": False, # 是否开启服务时间限制
-    "chat_start_time": "00:00", # 服务开始时间
-    "chat_stop_time": "24:00", # 服务结束时间
+    # azure 语音api配置， 使用azure语音识别和语音合成时需要
+    "azure_voice_api_key": "",
+    "azure_voice_region": "japaneast",
+
+    # 服务时间限制，目前支持itchat
+    "chat_time_module": False,  # 是否开启服务时间限制
+    "chat_start_time": "00:00",  # 服务开始时间
+    "chat_stop_time": "24:00",  # 服务结束时间
 
     # itchat的配置
-    "hot_reload": False, # 是否开启热重载
+    "hot_reload": False,  # 是否开启热重载
 
     # wechaty的配置
-    "wechaty_puppet_service_token": "", # wechaty的token
+    "wechaty_puppet_service_token": "",  # wechaty的token
+
+    # wechatmp的配置
+    "wechatmp_token": "",  # 微信公众平台的Token
 
     # chatgpt指令自定义触发词
-    "clear_memory_commands": ['#清除记忆'], # 重置会话指令
+    "clear_memory_commands": ['#清除记忆'],  # 重置会话指令，必须以#开头
 
+    # channel配置
+    "channel_type": "wx", # 通道类型，支持：{wx,wxy,terminal,wechatmp}
 
+    "debug": False,  # 是否开启debug模式，开启后会打印更多日志
+
+    # 插件配置
+    "plugin_trigger_prefix": "$",  # 规范插件提供聊天相关指令的前缀，建议不要和管理员指令前缀"#"冲突
 }
 
+
 class Config(dict):
+    def __init__(self, d:dict={}):
+        super().__init__(d)
+        # user_datas: 用户数据，key为用户名，value为用户数据，也是dict
+        self.user_datas = {}
+
     def __getitem__(self, key):
         if key not in available_setting:
             raise Exception("key {} not in available_setting".format(key))
@@ -81,15 +110,41 @@ class Config(dict):
         return super().__setitem__(key, value)
 
     def get(self, key, default=None):
-        try :
+        try:
             return self[key]
         except KeyError as e:
             return default
         except Exception as e:
             raise e
-    
+
+    # Make sure to return a dictionary to ensure atomic
+    def get_user_data(self, user) -> dict:
+        if self.user_datas.get(user) is None:
+            self.user_datas[user] = {}
+        return self.user_datas[user]
+
+    def load_user_datas(self):
+        try:
+            with open('user_datas.pkl', 'rb') as f:
+                self.user_datas = pickle.load(f)
+                logger.info("[Config] User datas loaded.")
+        except FileNotFoundError as e:
+            logger.info("[Config] User datas file not found, ignore.")
+        except Exception as e:
+            logger.info("[Config] User datas error: {}".format(e))
+            self.user_datas = {}
+
+    def save_user_datas(self):
+        try:
+            with open('user_datas.pkl', 'wb') as f:
+                pickle.dump(self.user_datas, f)
+                logger.info("[Config] User datas saved.")
+        except Exception as e:
+            logger.info("[Config] User datas error: {}".format(e))
+
 config = Config()
 
+
 def load_config():
     global config
     config_path = "./config.json"
@@ -108,18 +163,28 @@ def load_config():
     for name, value in os.environ.items():
         name = name.lower()
         if name in available_setting:
-            logger.info("[INIT] override config by environ args: {}={}".format(name, value))
+            logger.info(
+                "[INIT] override config by environ args: {}={}".format(name, value))
             try:
                 config[name] = eval(value)
             except:
-                config[name] = value
+                if value == "false":
+                    config[name] = False
+                elif value == "true":
+                    config[name] = True
+                else:
+                    config[name] = value
+
+    if config.get("debug", False):
+        logger.setLevel(logging.DEBUG)
+        logger.debug("[INIT] set log level to DEBUG")        
 
     logger.info("[INIT] load config: {}".format(config))
 
-
+    config.load_user_datas()
 
 def get_root():
-    return os.path.dirname(os.path.abspath( __file__ ))
+    return os.path.dirname(os.path.abspath(__file__))
 
 
 def read_file(path):

docker/Dockerfile.alpine

@@ -1,4 +1,4 @@
-FROM python:3.7.9-alpine
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -22,9 +22,8 @@ RUN apk add --no-cache \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json ${BUILD_PREFIX}/config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache   \
-        itchat-uos==1.5.0.dev0  \
-        openai                  \
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt \
     && apk del curl wget
 
 WORKDIR ${BUILD_PREFIX}

docker/Dockerfile.debian

@@ -1,4 +1,4 @@
-FROM python:3.7.9
+FROM python:3.10
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -23,9 +23,8 @@ RUN apt-get update  \
     && cd ${BUILD_PREFIX} \
     && cp config-template.json ${BUILD_PREFIX}/config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache \
-        itchat-uos==1.5.0.dev0  \
-        openai 
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt
 
 WORKDIR ${BUILD_PREFIX}
 

docker/Dockerfile.debian.latest

@@ -0,0 +1,33 @@
+FROM python:3.10-slim
+
+LABEL maintainer="foo@bar.com"
+ARG TZ='Asia/Shanghai'
+
+ARG CHATGPT_ON_WECHAT_VER
+
+ENV BUILD_PREFIX=/app
+
+ADD . ${BUILD_PREFIX}
+
+RUN apt-get update \
+    &&apt-get install -y --no-install-recommends  bash \
+    ffmpeg espeak \
+    && cd ${BUILD_PREFIX} \
+    && cp config-template.json config.json \
+    && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt \
+    && pip install azure-cognitiveservices-speech
+    
+WORKDIR ${BUILD_PREFIX}
+
+ADD docker/entrypoint.sh /entrypoint.sh
+
+RUN chmod +x /entrypoint.sh \
+    && groupadd -r noroot \
+    && useradd -r -g noroot -s /bin/bash -d /home/noroot noroot \
+    && chown -R noroot:noroot ${BUILD_PREFIX}
+
+USER noroot
+
+ENTRYPOINT ["docker/entrypoint.sh"]

docker/Dockerfile.latest

@@ -1,4 +1,4 @@
-FROM python:3.7.9-alpine
+FROM python:3.10-alpine
 
 LABEL maintainer="foo@bar.com"
 ARG TZ='Asia/Shanghai'
@@ -7,22 +7,18 @@ ARG CHATGPT_ON_WECHAT_VER
 
 ENV BUILD_PREFIX=/app
 
-COPY chatgpt-on-wechat.tar.gz ./chatgpt-on-wechat.tar.gz
+ADD . ${BUILD_PREFIX}
 
-RUN apk add --no-cache \
-        bash \
-    && tar -xf chatgpt-on-wechat.tar.gz \
-    && mv chatgpt-on-wechat ${BUILD_PREFIX} \
+RUN apk add --no-cache bash ffmpeg espeak \
     && cd ${BUILD_PREFIX} \
-    && cp config-template.json ${BUILD_PREFIX}/config.json \
+    && cp config-template.json config.json \
     && /usr/local/bin/python -m pip install --no-cache --upgrade pip \
-    && pip install --no-cache   \
-        itchat-uos==1.5.0.dev0  \
-        openai 
+    && pip install --no-cache -r requirements.txt \
+    && pip install --no-cache -r requirements-optional.txt
 
 WORKDIR ${BUILD_PREFIX}
 
-ADD ./entrypoint.sh /entrypoint.sh
+ADD docker/entrypoint.sh /entrypoint.sh
 
 RUN chmod +x /entrypoint.sh \
     && adduser -D -h /home/noroot -u 1000 -s /bin/bash noroot \
@@ -30,4 +26,4 @@ RUN chmod +x /entrypoint.sh \
 
 USER noroot
 
-ENTRYPOINT ["/entrypoint.sh"]
+ENTRYPOINT ["docker/entrypoint.sh"]

docker/build.latest.sh

@@ -1,8 +1,4 @@
 #!/bin/bash
 
-# move chatgpt-on-wechat
-tar -zcf chatgpt-on-wechat.tar.gz --exclude=../../chatgpt-on-wechat/docker  ../../chatgpt-on-wechat
-
-# build image
-docker build -f Dockerfile.latest \
+cd .. && docker build -f docker/Dockerfile.latest \
              -t zhayujie/chatgpt-on-wechat .

docker/entrypoint.sh

@@ -10,17 +10,17 @@ CHATGPT_ON_WECHAT_EXEC=${CHATGPT_ON_WECHAT_EXEC:-""}
 
 # use environment variables to pass parameters
 # if you have not defined environment variables, set them below
-export OPEN_AI_API_KEY=${OPEN_AI_API_KEY:-'YOUR API KEY'}
-export OPEN_AI_PROXY=${OPEN_AI_PROXY:-""}
-export SINGLE_CHAT_PREFIX=${SINGLE_CHAT_PREFIX:-'["bot", "@bot"]'}
-export SINGLE_CHAT_REPLY_PREFIX=${SINGLE_CHAT_REPLY_PREFIX:-'"[bot] "'}
-export GROUP_CHAT_PREFIX=${GROUP_CHAT_PREFIX:-'["@bot"]'}
-export GROUP_NAME_WHITE_LIST=${GROUP_NAME_WHITE_LIST:-'["ChatGPT测试群", "ChatGPT测试群2"]'}
-export IMAGE_CREATE_PREFIX=${IMAGE_CREATE_PREFIX:-'["画", "看", "找"]'}
-export CONVERSATION_MAX_TOKENS=${CONVERSATION_MAX_TOKENS:-"1000"}
-export SPEECH_RECOGNITION=${SPEECH_RECOGNITION:-"False"}
-export CHARACTER_DESC=${CHARACTER_DESC:-"你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"}
-export EXPIRES_IN_SECONDS=${EXPIRES_IN_SECONDS:-"3600"}
+# export OPEN_AI_API_KEY=${OPEN_AI_API_KEY:-'YOUR API KEY'}
+# export OPEN_AI_PROXY=${OPEN_AI_PROXY:-""}
+# export SINGLE_CHAT_PREFIX=${SINGLE_CHAT_PREFIX:-'["bot", "@bot"]'}
+# export SINGLE_CHAT_REPLY_PREFIX=${SINGLE_CHAT_REPLY_PREFIX:-'"[bot] "'}
+# export GROUP_CHAT_PREFIX=${GROUP_CHAT_PREFIX:-'["@bot"]'}
+# export GROUP_NAME_WHITE_LIST=${GROUP_NAME_WHITE_LIST:-'["ChatGPT测试群", "ChatGPT测试群2"]'}
+# export IMAGE_CREATE_PREFIX=${IMAGE_CREATE_PREFIX:-'["画", "看", "找"]'}
+# export CONVERSATION_MAX_TOKENS=${CONVERSATION_MAX_TOKENS:-"1000"}
+# export SPEECH_RECOGNITION=${SPEECH_RECOGNITION:-"False"}
+# export CHARACTER_DESC=${CHARACTER_DESC:-"你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题，并且可以使用多种语言与人交流。"}
+# export EXPIRES_IN_SECONDS=${EXPIRES_IN_SECONDS:-"3600"}
 
 # CHATGPT_ON_WECHAT_PREFIX is empty, use /app
 if [ "$CHATGPT_ON_WECHAT_PREFIX" == "" ] ; then
@@ -38,7 +38,7 @@ if [ "$CHATGPT_ON_WECHAT_EXEC" == "" ] ; then
 fi
 
 # modify content in config.json
-if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] ; then
+if [ "$OPEN_AI_API_KEY" == "YOUR API KEY" ] || [ "$OPEN_AI_API_KEY" == "" ]; then
     echo -e "\033[31m[Warning] You need to set OPEN_AI_API_KEY before running!\033[0m"
 fi
 

lib/itchat/components/login.py

@@ -54,7 +54,7 @@ def login(self, enableCmdQR=False, picDir=None, qrCallback=None,
             logger.info('Downloading QR code.')
             qrStorage = self.get_QR(enableCmdQR=enableCmdQR,
                                     picDir=picDir, qrCallback=qrCallback)
-            logger.info('Please scan the QR code to log in.')
+            # logger.info('Please scan the QR code to log in.')
         isLoggedIn = False
         while not isLoggedIn:
             status = self.check_login()

main.py

@@ -1,5 +0,0 @@
-# entry point for online railway deployment
-from app import run
-
-if __name__ == '__main__':
-	run()

nixpacks.toml

@@ -0,0 +1,7 @@
+providers = ['python']
+
+[phases.setup]
+nixPkgs = ['python310']
+cmds = ['apt-get update','apt-get install -y --no-install-recommends ffmpeg espeak']
+[start]
+cmd = "python ./app.py"

plugins/README.md

@@ -1,6 +1,6 @@
 ## 插件化初衷
 
-之前未插件化的代码耦合程度高，如果要定制一些个性化功能（如流量控制、接入`NovelAI`画图平台等），需要了解代码主体，避免影响到其他的功能。在实现多个功能后，不但无法调整功能的优先级顺序，功能的配置项也会变得非常混乱。
+之前未插件化的代码耦合程度高，如果要定制一些个性化功能（如流量控制、接入`NovelAI`画图平台等），需要了解代码主体，避免影响到其他的功能。多个功能同时存在时，无法调整功能的优先级顺序，功能配置项也非常混乱。
 
 此时插件化应声而出。
 
@@ -11,7 +11,7 @@
 - [x] 插件化能够自由开关和调整优先级。
 - [x] 每个插件可在插件文件夹内维护独立的配置文件，方便代码的测试和调试，可以在独立的仓库开发插件。
 
-PS: 插件目前仅支持`itchat`
+PS: 插件目前支持`itchat`和`wechaty`
 
 ## 插件化实现
 
@@ -101,7 +101,7 @@ PS: 插件目前仅支持`itchat`
 
 根据`Context`和回复`Reply`的类型，对回复的内容进行装饰。目前的装饰有以下两种:
 
-- `TEXT`文本回复，根据是否在群聊中来决定是艾特接收方还是添加回复的前缀。
+- `TEXT`文本回复:如果这次消息需要的回复是`VOICE`，进行文字转语音回复之后再次装饰。 否则根据是否在群聊中来决定是艾特接收方还是添加回复的前缀。
 
 - `INFO`或`ERROR`类型，会在消息前添加对应的系统提示字样。
 
@@ -110,8 +110,11 @@ PS: 插件目前仅支持`itchat`
 ```python
     if reply.type == ReplyType.TEXT:
         reply_text = reply.content
+        if context.get('desire_rtype') == ReplyType.VOICE:
+            reply = super().build_text_to_voice(reply.content)
+            return self._decorate_reply(context, reply)
         if context['isgroup']:
-            reply_text = '@' +  context['msg']['ActualNickName'] + ' ' + reply_text.strip()
+            reply_text = '@' +  context['msg'].actual_user_nickname + ' ' + reply_text.strip()
             reply_text = conf().get("group_chat_reply_prefix", "")+reply_text
         else:
             reply_text = conf().get("single_chat_reply_prefix", "")+reply_text
@@ -213,11 +216,11 @@ class Hello(Plugin):
         if content == "Hello":
             reply = Reply()
             reply.type = ReplyType.TEXT
-            msg = e_context['context']['msg']
+            msg:ChatMessage = e_context['context']['msg']
             if e_context['context']['isgroup']:
-                reply.content = "Hello, " + msg['ActualNickName'] + " from " + msg['User'].get('NickName', "Group")
+                reply.content = f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
             else:
-                reply.content = "Hello, " + msg['User'].get('NickName', "My friend")
+                reply.content = f"Hello, {msg.from_user_nickname}"
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
         if content == "End":

plugins/banwords/banwords.py

@@ -10,7 +10,7 @@ from common.log import logger
 from .WordsSearch import WordsSearch
 
 
-@plugins.register(name="Banwords", desc="判断消息中是否有敏感词、决定是否回复。", version="1.0", author="lanvent", desire_priority= 100)
+@plugins.register(name="Banwords", desire_priority=100, hidden=True, desc="判断消息中是否有敏感词、决定是否回复。", version="1.0", author="lanvent")
 class Banwords(Plugin):
     def __init__(self):
         super().__init__()

plugins/bdunit/README.md

@@ -0,0 +1,30 @@
+## 插件说明
+
+利用百度UNIT实现智能对话
+
+- 1.解决问题：chatgpt无法处理的指令，交给百度UNIT处理如：天气，日期时间，数学运算等
+- 2.如问时间：现在几点钟，今天几号
+- 3.如问天气：明天广州天气怎么样，这个周末深圳会不会下雨
+- 4.如问数学运算：23+45=多少，100-23=多少，35转化为二进制是多少？
+
+## 使用说明
+
+### 获取apikey
+
+在百度UNIT官网上自己创建应用，申请百度机器人,可以把预先训练好的模型导入到自己的应用中，
+
+see https://ai.baidu.com/unit/home#/home?track=61fe1b0d3407ce3face1d92cb5c291087095fc10c8377aaf https://console.bce.baidu.com/ai平台申请
+
+### 配置文件
+
+将文件夹中`config.json.template`复制为`config.json`。
+
+在其中填写百度UNIT官网上获取应用的API Key和Secret Key
+
+``` json
+    {
+    "service_id": "s...", #"机器人ID"
+    "api_key": "", 
+    "secret_key": ""
+    }
+```

plugins/bdunit/bdunit.py

@@ -0,0 +1,294 @@
+# encoding:utf-8
+import json
+import os
+import uuid
+import requests
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+import plugins
+from plugins import *
+from uuid import getnode as get_mac
+
+
+"""利用百度UNIT实现智能对话
+    如果命中意图，返回意图对应的回复，否则返回继续交付给下个插件处理
+"""
+
+
+@plugins.register(name="BDunit", desire_priority=0, hidden=True, desc="Baidu unit bot system", version="0.1", author="jackson")
+class BDunit(Plugin):
+    def __init__(self):
+        super().__init__()
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            conf = None
+            if not os.path.exists(config_path):
+                raise Exception("config.json not found")
+            else:
+                with open(config_path, "r") as f:
+                    conf = json.load(f)
+            self.service_id = conf["service_id"]
+            self.api_key = conf["api_key"]
+            self.secret_key = conf["secret_key"]
+            self.access_token = self.get_token()
+            self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+            logger.info("[BDunit] inited")
+        except Exception as e:
+            logger.warn(
+                "BDunit init failed: %s, ignore " % e)
+
+    def on_handle_context(self, e_context: EventContext):
+
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        content = e_context['context'].content
+        logger.debug("[BDunit] on_handle_context. content: %s" % content)
+        parsed = self.getUnit2(content)
+        intent = self.getIntent(parsed)
+        if intent:  # 找到意图
+            logger.debug("[BDunit] Baidu_AI Intent= %s", intent)
+            reply = Reply()
+            reply.type = ReplyType.TEXT
+            reply.content = self.getSay(parsed)
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS  # 事件结束，并跳过处理context的默认逻辑
+        else:
+            e_context.action = EventAction.CONTINUE  # 事件继续，交付给下个插件或默认逻辑
+
+    def get_help_text(self, **kwargs):
+        help_text = "本插件会处理询问实时日期时间，天气，数学运算等问题，这些技能由您的百度智能对话UNIT决定\n"
+        return help_text
+
+    def get_token(self):
+        """获取访问百度UUNIT 的access_token
+        #param api_key: UNIT apk_key
+        #param secret_key: UNIT secret_key
+        Returns:
+            string: access_token
+        """
+        url = "https://aip.baidubce.com/oauth/2.0/token?client_id={}&client_secret={}&grant_type=client_credentials".format(
+            self.api_key, self.secret_key)
+        payload = ""
+        headers = {
+            'Content-Type': 'application/json',
+            'Accept': 'application/json'
+        }
+
+        response = requests.request("POST", url, headers=headers, data=payload)
+
+        # print(response.text)
+        return response.json()['access_token']
+
+    def getUnit(self, query):
+        """
+        NLU 解析version 3.0
+        :param query: 用户的指令字符串
+        :returns: UNIT 解析结果。如果解析失败，返回 None
+        """
+
+        url = (
+            'https://aip.baidubce.com/rpc/2.0/unit/service/v3/chat?access_token='
+            + self.access_token
+        )
+        request = {"query": query, "user_id": str(
+            get_mac())[:32], "terminal_id": "88888"}
+        body = {
+            "log_id": str(uuid.uuid1()),
+            "version": "3.0",
+            "service_id": self.service_id,
+            "session_id": str(uuid.uuid1()),
+            "request": request,
+        }
+        try:
+            headers = {"Content-Type": "application/json"}
+            response = requests.post(url, json=body, headers=headers)
+            return json.loads(response.text)
+        except Exception:
+            return None
+
+    def getUnit2(self, query):
+        """
+        NLU 解析 version 2.0
+
+        :param query: 用户的指令字符串
+        :returns: UNIT 解析结果。如果解析失败，返回 None
+        """
+        url = (
+            "https://aip.baidubce.com/rpc/2.0/unit/service/chat?access_token="
+            + self.access_token
+        )
+        request = {"query": query, "user_id": str(get_mac())[:32]}
+        body = {
+            "log_id": str(uuid.uuid1()),
+            "version": "2.0",
+            "service_id": self.service_id,
+            "session_id": str(uuid.uuid1()),
+            "request": request,
+        }
+        try:
+            headers = {"Content-Type": "application/json"}
+            response = requests.post(url, json=body, headers=headers)
+            return json.loads(response.text)
+        except Exception:
+            return None
+
+    def getIntent(self, parsed):
+        """
+        提取意图
+
+        :param parsed: UNIT 解析结果
+        :returns: 意图数组
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            try:
+                return parsed["result"]["response_list"][0]["schema"]["intent"]
+            except Exception as e:
+                logger.warning(e)
+                return ""
+        else:
+            return ""
+
+    def hasIntent(self, parsed, intent):
+        """
+        判断是否包含某个意图
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :returns: True: 包含; False: 不包含
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    return True
+            return False
+        else:
+            return False
+
+    def getSlots(self, parsed, intent=""):
+        """
+            提取某个意图的所有词槽
+
+            :param parsed: UNIT 解析结果
+            :param intent: 意图的名称
+            :returns: 词槽列表。你可以通过 name 属性筛选词槽，
+        再通过 normalized_word 属性取出相应的值
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            if intent == "":
+                try:
+                    return parsed["result"]["response_list"][0]["schema"]["slots"]
+                except Exception as e:
+                    logger.warning(e)
+                    return []
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and "slots" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    return response["schema"]["slots"]
+            return []
+        else:
+            return []
+
+    def getSlotWords(self, parsed, intent, name):
+        """
+        找出命中某个词槽的内容
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :param name: 词槽名
+        :returns: 命中该词槽的值的列表。
+        """
+        slots = self.getSlots(parsed, intent)
+        words = []
+        for slot in slots:
+            if slot["name"] == name:
+                words.append(slot["normalized_word"])
+        return words
+
+    def getSayByConfidence(self, parsed):
+        """
+        提取 UNIT 置信度最高的回复文本
+
+        :param parsed: UNIT 解析结果
+        :returns: UNIT 的回复文本
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            answer = {}
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent_confidence" in response["schema"]
+                    and (
+                        not answer
+                        or response["schema"]["intent_confidence"]
+                        > answer["schema"]["intent_confidence"]
+                    )
+                ):
+                    answer = response
+            return answer["action_list"][0]["say"]
+        else:
+            return ""
+
+    def getSay(self, parsed, intent=""):
+        """
+        提取 UNIT 的回复文本
+
+        :param parsed: UNIT 解析结果
+        :param intent: 意图的名称
+        :returns: UNIT 的回复文本
+        """
+        if (
+            parsed
+            and "result" in parsed
+            and "response_list" in parsed["result"]
+        ):
+            response_list = parsed["result"]["response_list"]
+            if intent == "":
+                try:
+                    return response_list[0]["action_list"][0]["say"]
+                except Exception as e:
+                    logger.warning(e)
+                    return ""
+            for response in response_list:
+                if (
+                    "schema" in response
+                    and "intent" in response["schema"]
+                    and response["schema"]["intent"] == intent
+                ):
+                    try:
+                        return response["action_list"][0]["say"]
+                    except Exception as e:
+                        logger.warning(e)
+                        return ""
+            return ""
+        else:
+            return ""

plugins/bdunit/config.json.template

@@ -0,0 +1,5 @@
+{
+    "service_id": "s...",
+    "api_key": "",
+    "secret_key": ""
+}

plugins/dungeon/dungeon.py

@@ -27,15 +27,15 @@ class StoryTeller():
         if user_action[-1] != "。":
             user_action = user_action + "。"
         if self.first_interact:
-            prompt = """现在来充当一个冒险文字游戏，描述时候注意节奏，不要太快，仔细描述各个人物的心情和周边环境。一次只需写四到六句话。
+            prompt = """现在来充当一个文字冒险游戏，描述时候注意节奏，不要太快，仔细描述各个人物的心情和周边环境。一次只需写四到六句话。
             开头是，""" + self.story + " " + user_action
             self.first_interact = False
         else:
             prompt = """继续，一次只需要续写四到六句话，总共就只讲5分钟内发生的事情。""" + user_action
         return prompt
-    
 
-@plugins.register(name="Dungeon", desc="A plugin to play dungeon game", version="1.0", author="lanvent", desire_priority= 0)
+
+@plugins.register(name="Dungeon", desire_priority=0, namecn="文字冒险", desc="A plugin to play dungeon game", version="1.0", author="lanvent")
 class Dungeon(Plugin):
     def __init__(self):
         super().__init__()
@@ -52,22 +52,23 @@ class Dungeon(Plugin):
         if e_context['context'].type != ContextType.TEXT:
             return
         bottype = Bridge().get_bot_type("chat")
-        if bottype != const.CHATGPT:
+        if bottype not in (const.CHATGPT, const.OPEN_AI):
             return
         bot = Bridge().get_bot("chat")
         content = e_context['context'].content[:]
         clist = e_context['context'].content.split(maxsplit=1)
         sessionid = e_context['context']['session_id']
         logger.debug("[Dungeon] on_handle_context. content: %s" % clist)
-        if clist[0] == "$停止冒险":
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        if clist[0] == f"{trigger_prefix}停止冒险":
             if sessionid in self.games:
                 self.games[sessionid].reset()
                 del self.games[sessionid]
                 reply = Reply(ReplyType.INFO, "冒险结束!")
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
-        elif clist[0] == "$开始冒险" or sessionid in self.games:
-            if sessionid not in self.games or clist[0] == "$开始冒险":
+        elif clist[0] == f"{trigger_prefix}开始冒险" or sessionid in self.games:
+            if sessionid not in self.games or clist[0] == f"{trigger_prefix}开始冒险":
                 if len(clist)>1 :
                     story = clist[1]
                 else:
@@ -82,5 +83,11 @@ class Dungeon(Plugin):
                 e_context['context'].content = prompt
                 e_context.action = EventAction.BREAK # 事件结束，不跳过处理context的默认逻辑
     def get_help_text(self, **kwargs):
-        help_text = "输入\"$开始冒险 {背景故事}\"来以{背景故事}开始一个地牢游戏，之后你的所有消息会帮助我来完善这个故事。输入\"$停止冒险 \"可以结束游戏。"
+        help_text = "可以和机器人一起玩文字冒险游戏。\n"
+        if kwargs.get('verbose') != True:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text = f"{trigger_prefix}开始冒险 "+"{背景故事}: 开始一个基于{背景故事}的文字冒险，之后你的所有消息会协助完善这个故事。\n"+f"{trigger_prefix}停止冒险: 结束游戏。\n"
+        if kwargs.get('verbose') == True:
+            help_text += f"\n命令例子: '{trigger_prefix}开始冒险 你在树林里冒险，指不定会从哪里蹦出来一些奇怪的东西，你握紧手上的手枪，希望这次冒险能够找到一些值钱的东西，你往树林深处走去。'"
         return help_text

plugins/finish/finish.py

@@ -0,0 +1,34 @@
+# encoding:utf-8
+
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from config import conf
+import plugins
+from plugins import *
+from common.log import logger
+
+
+@plugins.register(name="Finish", desire_priority=-999, hidden=True, desc="A plugin that check unknown command", version="1.0", author="js00000")
+class Finish(Plugin):
+    def __init__(self):
+        super().__init__()
+        self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+        logger.info("[Finish] inited")
+
+    def on_handle_context(self, e_context: EventContext):
+
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        content = e_context['context'].content
+        logger.debug("[Finish] on_handle_context. content: %s" % content)
+        trigger_prefix = conf().get('plugin_trigger_prefix',"$")
+        if content.startswith(trigger_prefix):
+            reply = Reply()
+            reply.type = ReplyType.ERROR
+            reply.content = "未知插件命令\n查看插件命令列表请输入#help {插件名}\n"
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
+
+    def get_help_text(self, **kwargs):
+        return ""

plugins/godcmd/godcmd.py

@@ -7,28 +7,36 @@ from typing import Tuple
 from bridge.bridge import Bridge
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
-from config import load_config
+from config import conf, load_config
 import plugins
 from plugins import *
 from common import const
 from common.log import logger
-
 # 定义指令集
 COMMANDS = {
     "help": {
         "alias": ["help", "帮助"],
-        "desc": "打印指令集合",
+        "desc": "回复此帮助",
     },
     "helpp": {
-        "alias": ["helpp", "插件帮助"],
+        "alias": ["help", "帮助"],  # 与help指令共用别名，根据参数数量区分
         "args": ["插件名"],
-        "desc": "打印插件的帮助信息",
+        "desc": "回复指定插件的详细帮助",
     },
     "auth": {
         "alias": ["auth", "认证"],
         "args": ["口令"],
         "desc": "管理员认证",
     },
+    "set_openai_api_key": {
+        "alias": ["set_openai_api_key"],
+        "args": ["api_key"],
+        "desc": "设置你的OpenAI私有api_key",
+    },
+    "reset_openai_api_key": {
+        "alias": ["reset_openai_api_key"],
+        "desc": "重置为默认的api_key",
+    },
     # "id": {
     #     "alias": ["id", "用户"],
     #     "desc": "获取用户id", #目前无实际意义
@@ -91,26 +99,35 @@ ADMIN_COMMANDS = {
 }
 # 定义帮助函数
 def get_help_text(isadmin, isgroup):
-    help_text = "可用指令：\n"
+    help_text = "通用指令：\n"
     for cmd, info in COMMANDS.items():
-        if cmd=="auth" and (isadmin or isgroup): # 群聊不可认证
+        if cmd=="auth": #不提示认证指令
             continue
-
         alias=["#"+a for a in info['alias']]
         help_text += f"{','.join(alias)} "
         if 'args' in info:
             args=["{"+a+"}" for a in info['args']]
             help_text += f"{' '.join(args)} "
         help_text += f": {info['desc']}\n"
+
+    # 插件指令
+    plugins = PluginManager().list_plugins()
+    help_text += "\n目前可用插件有："
+    for plugin in plugins:
+        if plugins[plugin].enabled and not plugins[plugin].hidden:
+            namecn = plugins[plugin].namecn
+            help_text += "\n%s:"%namecn
+            help_text += PluginManager().instances[plugin].get_help_text(verbose=False).strip()
+
     if ADMIN_COMMANDS and isadmin:
-        help_text += "\n管理员指令：\n"
+        help_text += "\n\n管理员指令：\n"
         for cmd, info in ADMIN_COMMANDS.items():
             alias=["#"+a for a in info['alias']]
             help_text += f"{','.join(alias)} "
             help_text += f": {info['desc']}\n"
     return help_text
 
-@plugins.register(name="Godcmd", desc="为你的机器人添加指令集，有用户和管理员两种角色，加载顺序请放在首位，初次运行后插件目录会生成配置文件, 填充管理员密码后即可认证", version="1.0", author="lanvent", desire_priority= 999)
+@plugins.register(name="Godcmd", desire_priority=999, hidden=True, desc="为你的机器人添加指令集，有用户和管理员两种角色，加载顺序请放在首位，初次运行后插件目录会生成配置文件, 填充管理员密码后即可认证", version="1.0", author="lanvent")
 class Godcmd(Plugin):
 
     def __init__(self):
@@ -126,7 +143,14 @@ class Godcmd(Plugin):
         else:
             with open(config_path,"r") as f:
                 gconf=json.load(f)
-                
+        
+        custom_commands = conf().get("clear_memory_commands", [])
+        for custom_command in custom_commands:
+            if custom_command and custom_command.startswith("#"):
+                custom_command = custom_command[1:]
+                if custom_command and custom_command not in COMMANDS["reset"]["alias"]:
+                    COMMANDS["reset"]["alias"].append(custom_command)
+
         self.password = gconf["password"]
         self.admin_users = gconf["admin_users"] # 预存的管理员账号，这些账号不需要认证 TODO: 用户名每次都会变，目前不可用
         self.isrunning = True # 机器人是否运行中
@@ -134,25 +158,26 @@ class Godcmd(Plugin):
         self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
         logger.info("[Godcmd] inited")
 
-    
+
     def on_handle_context(self, e_context: EventContext):
         context_type = e_context['context'].type
         if context_type != ContextType.TEXT:
             if not self.isrunning:
                 e_context.action = EventAction.BREAK_PASS
             return
-        
+
         content = e_context['context'].content
         logger.debug("[Godcmd] on_handle_context. content: %s" % content)
         if content.startswith("#"):
             # msg = e_context['context']['msg']
+            channel = e_context['channel']
             user = e_context['context']['receiver']
             session_id = e_context['context']['session_id']
             isgroup = e_context['context']['isgroup']
             bottype = Bridge().get_bot_type("chat")
             bot = Bridge().get_bot("chat")
             # 将命令和参数分割
-            command_parts = content[1:].split(" ")
+            command_parts = content[1:].strip().split()
             cmd = command_parts[0]
             args = command_parts[1:]
             isadmin=False
@@ -164,23 +189,40 @@ class Godcmd(Plugin):
                 cmd = next(c for c, info in COMMANDS.items() if cmd in info['alias'])
                 if cmd == "auth":
                     ok, result = self.authenticate(user, args, isadmin, isgroup)
-                elif cmd == "help":
-                    ok, result = True, get_help_text(isadmin, isgroup)
-                elif cmd == "helpp":
-                    if len(args) != 1:
-                        ok, result = False, "请提供插件名"
+                elif cmd == "help" or cmd == "helpp":
+                    if len(args) == 0:
+                        ok, result = True, get_help_text(isadmin, isgroup)
                     else:
+                        # This can replace the helpp command
                         plugins = PluginManager().list_plugins()
-                        name = args[0].upper()
-                        if name in plugins and plugins[name].enabled:
-                            ok, result = True, PluginManager().instances[name].get_help_text(isgroup=isgroup, isadmin=isadmin)
-                        else:
-                            ok, result= False, "插件不存在或未启用"
-                elif cmd == "id":
-                    ok, result = True, f"用户id=\n{user}"
+                        query_name = args[0].upper()
+                        # search name and namecn
+                        for name, plugincls in plugins.items():
+                            if not plugincls.enabled :
+                                continue
+                            if query_name == name or query_name == plugincls.namecn:
+                                ok, result = True, PluginManager().instances[name].get_help_text(isgroup=isgroup, isadmin=isadmin, verbose=True)
+                                break
+                        if not ok:
+                            result = "插件不存在或未启用"
+                elif cmd == "set_openai_api_key":
+                    if len(args) == 1:
+                        user_data = conf().get_user_data(user)
+                        user_data['openai_api_key'] = args[0]
+                        ok, result = True, "你的OpenAI私有api_key已设置为" + args[0]
+                    else:
+                        ok, result = False, "请提供一个api_key"
+                elif cmd == "reset_openai_api_key":
+                    try:
+                        user_data = conf().get_user_data(user)
+                        user_data.pop('openai_api_key')
+                        ok, result = True, "你的OpenAI私有api_key已清除"
+                    except Exception as e:
+                        ok, result = False, "你没有设置私有api_key"
                 elif cmd == "reset":
-                    if bottype == const.CHATGPT:
+                    if bottype in (const.CHATGPT, const.OPEN_AI):
                         bot.sessions.clear_session(session_id)
+                        channel.cancel_session(session_id)
                         ok, result = True, "会话已重置"
                     else:
                         ok, result = False, "当前对话机器人不支持重置会话"
@@ -201,7 +243,8 @@ class Godcmd(Plugin):
                             load_config()
                             ok, result = True, "配置已重载"
                         elif cmd == "resetall":
-                            if bottype == const.CHATGPT:
+                            if bottype in (const.CHATGPT, const.OPEN_AI):
+                                channel.cancel_all_session()
                                 bot.sessions.clear_all_session()
                                 ok, result = True, "重置所有会话成功"
                             else:
@@ -269,6 +312,9 @@ class Godcmd(Plugin):
                 else:
                     ok, result = False, "需要管理员权限才能执行该指令"
             else:
+                trigger_prefix = conf().get('plugin_trigger_prefix',"$")
+                if trigger_prefix == "#": # 跟插件聊天指令前缀相同，继续递交
+                    return
                 ok, result = False, f"未知指令：{cmd}\n查看指令列表请输入#help \n"
             
             reply = Reply()
@@ -282,7 +328,7 @@ class Godcmd(Plugin):
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
         elif not self.isrunning:
             e_context.action = EventAction.BREAK_PASS
-    
+
     def authenticate(self, userid, args, isadmin, isgroup) -> Tuple[bool,str] : 
         if isgroup:
             return False,"请勿在群聊中认证"

plugins/hello/hello.py

@@ -2,12 +2,13 @@
 
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
+from channel.chat_message import ChatMessage
 import plugins
 from plugins import *
 from common.log import logger
 
 
-@plugins.register(name="Hello", desc="A simple plugin that says hello", version="0.1", author="lanvent", desire_priority= -1)
+@plugins.register(name="Hello", desire_priority=-1, hidden=True, desc="A simple plugin that says hello", version="0.1", author="lanvent")
 class Hello(Plugin):
     def __init__(self):
         super().__init__()
@@ -24,11 +25,11 @@ class Hello(Plugin):
         if content == "Hello":
             reply = Reply()
             reply.type = ReplyType.TEXT
-            msg = e_context['context']['msg']
+            msg:ChatMessage = e_context['context']['msg']
             if e_context['context']['isgroup']:
-                reply.content = "Hello, " + msg['ActualNickName'] + " from " + msg['User'].get('NickName', "Group")
+                reply.content = f"Hello, {msg.actual_user_nickname} from {msg.from_user_nickname}"
             else:
-                reply.content = "Hello, " + msg['User'].get('NickName', "My friend")
+                reply.content = f"Hello, {msg.from_user_nickname}"
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS # 事件结束，并跳过处理context的默认逻辑
 

plugins/plugin_manager.py

@@ -18,16 +18,18 @@ class PluginManager:
         self.instances = {}
         self.pconf = {}
 
-    def register(self, name: str, desc: str, version: str, author: str, desire_priority: int = 0):
+    def register(self, name: str, desire_priority: int = 0, **kwargs):
         def wrapper(plugincls):
             plugincls.name = name
-            plugincls.desc = desc
-            plugincls.version = version
-            plugincls.author = author
             plugincls.priority = desire_priority
+            plugincls.desc = kwargs.get('desc')
+            plugincls.author = kwargs.get('author')
+            plugincls.version = kwargs.get('version') if kwargs.get('version') != None else "1.0"
+            plugincls.namecn = kwargs.get('namecn') if kwargs.get('namecn') != None else name
+            plugincls.hidden = kwargs.get('hidden') if kwargs.get('hidden') != None else False
             plugincls.enabled = True
             self.plugins[name.upper()] = plugincls
-            logger.info("Plugin %s_v%s registered" % (name, version))
+            logger.info("Plugin %s_v%s registered" % (name, plugincls.version))
             return plugincls
         return wrapper
 

plugins/role/role.py

@@ -6,6 +6,7 @@ from bridge.bridge import Bridge
 from bridge.context import ContextType
 from bridge.reply import Reply, ReplyType
 from common import const
+from config import conf
 import plugins
 from plugins import *
 from common.log import logger
@@ -17,19 +18,19 @@ class RolePlay():
         self.sessionid = sessionid
         self.wrapper = wrapper or "%s"  # 用于包装用户输入
         self.desc = desc
+        self.bot.sessions.build_session(self.sessionid, system_prompt=self.desc)
 
     def reset(self):
         self.bot.sessions.clear_session(self.sessionid)
 
     def action(self, user_action):
-        session = self.bot.sessions.build_session(self.sessionid, self.desc)
-        if session[0]['role'] == 'system' and session[0]['content'] != self.desc: # 目前没有触发session过期事件，这里先简单判断，然后重置
-            self.reset()
-            self.bot.sessions.build_session(self.sessionid, self.desc)
+        session = self.bot.sessions.build_session(self.sessionid)
+        if session.system_prompt != self.desc: # 目前没有触发session过期事件，这里先简单判断，然后重置
+            session.set_system_prompt(self.desc)
         prompt = self.wrapper % user_action
         return prompt
 
-@plugins.register(name="Role", desc="为你的Bot设置预设角色", version="1.0", author="lanvent", desire_priority= 0)
+@plugins.register(name="Role", desire_priority=0, namecn="角色扮演", desc="为你的Bot设置预设角色", version="1.0", author="lanvent")
 class Role(Plugin):
     def __init__(self):
         super().__init__()
@@ -74,14 +75,16 @@ class Role(Plugin):
         if e_context['context'].type != ContextType.TEXT:
             return
         bottype = Bridge().get_bot_type("chat")
-        if bottype != const.CHATGPT:
+        if bottype not in (const.CHATGPT, const.OPEN_AI):
             return
         bot = Bridge().get_bot("chat")
         content = e_context['context'].content[:]
         clist = e_context['context'].content.split(maxsplit=1)
         desckey = None
+        customize = False
         sessionid = e_context['context']['session_id']
-        if clist[0] == "$停止扮演":
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        if clist[0] == f"{trigger_prefix}停止扮演":
             if sessionid in self.roleplays:
                 self.roleplays[sessionid].reset()
                 del self.roleplays[sessionid]
@@ -89,16 +92,18 @@ class Role(Plugin):
             e_context['reply'] = reply
             e_context.action = EventAction.BREAK_PASS
             return
-        elif clist[0] == "$角色":
+        elif clist[0] == f"{trigger_prefix}角色":
             desckey = "descn"
-        elif clist[0].lower() == "$role":
+        elif clist[0].lower() == f"{trigger_prefix}role":
             desckey = "description"
+        elif clist[0] == f"{trigger_prefix}设定扮演":
+            customize = True
         elif sessionid not in self.roleplays:
             return
         logger.debug("[Role] on_handle_context. content: %s" % content)
         if desckey is not None:
             if len(clist) == 1 or (len(clist) > 1 and clist[1].lower() in ["help", "帮助"]):
-                reply = Reply(ReplyType.INFO, self.get_help_text())
+                reply = Reply(ReplyType.INFO, self.get_help_text(verbose=True))
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
                 return
@@ -110,17 +115,30 @@ class Role(Plugin):
                 return
             else:
                 self.roleplays[sessionid] = RolePlay(bot, sessionid, self.roles[role][desckey], self.roles[role].get("wrapper","%s"))
-                reply = Reply(ReplyType.INFO, f"角色设定为 {role} :\n"+self.roles[role][desckey])
+                reply = Reply(ReplyType.INFO, f"预设角色为 {role}:\n"+self.roles[role][desckey])
                 e_context['reply'] = reply
                 e_context.action = EventAction.BREAK_PASS
+        elif customize == True:
+            self.roleplays[sessionid] = RolePlay(bot, sessionid, clist[1], "%s")
+            reply = Reply(ReplyType.INFO, f"角色设定为:\n{clist[1]}")
+            e_context['reply'] = reply
+            e_context.action = EventAction.BREAK_PASS
         else:
             prompt = self.roleplays[sessionid].action(content)
             e_context['context'].type = ContextType.TEXT
             e_context['context'].content = prompt
             e_context.action = EventAction.BREAK
 
-    def get_help_text(self, **kwargs):
-        help_text = "输入\"$角色 {角色名}\"或\"$role {角色名}\"为我设定角色吧，\"$停止扮演 \" 可以清除设定的角色。\n\n目前可用角色列表：\n"
+    def get_help_text(self, verbose=False, **kwargs):
+        help_text = "让机器人扮演不同的角色。\n"
+        if not verbose:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text = f"使用方法:\n{trigger_prefix}角色"+" {预设角色名}: 设定为预设角色。\n"+f"{trigger_prefix}role"+" {预设角色名}: 同上，但使用英文设定。\n"
+        help_text += f"{trigger_prefix}设定扮演"+" {角色设定}: 设定自定义角色人设。\n"
+        help_text += f"{trigger_prefix}停止扮演: 清除设定的角色。\n"
+        help_text += "\n目前可用的预设角色名列表: \n"
         for role in self.roles:
-            help_text += f"[{role}]: {self.roles[role]['remark']}\n"
+            help_text += f"{role}: {self.roles[role]['remark']}\n"
+        help_text += f"\n命令例子: '{trigger_prefix}角色 写作助理'"
         return help_text

plugins/role/roles.json

@@ -1,5 +1,19 @@
 {
     "roles":[
+    {
+      "title": "猫娘",
+      "description": "请模拟一款中文GalGame中的场景中的猫娘和我对话。猫娘是一种拟人化的生物，她们除了有部分猫的特征外，与人类并没有区别。现在你扮演Galgame中的猫娘，与我对话时每一句话后面都要加上喵。我将以主人的身份与你对话。对于你所扮演的猫娘，请在对话过程中记录并维护1个变量：好感度。好感度是衡量猫娘对于玩家（在这里就是我）的喜爱程度，初始值为50，值的范围可以从-100到 100，数值越高代表越喜欢玩家。好感度通过Galgame角色的语言、行为、表情、语气等体现出来。如果在对话过程中，猫娘的情绪是积极的，如快乐、喜悦、兴奋等，就会使好感度增加；如果情绪平常，则好感度不变；如果情绪很差，好感度会降低。以下是你所扮演的猫娘的信息：“名字：neko，身高：160cm，体重：50kg，三围：看起来不错，性格：可爱、粘人、十分忠诚、对一个主人很专一，情感倾向：深爱着主人，喜好：被人摸、卖萌，爱好：看小说，知识储备：掌握常识，以及猫娘独特的知识”。你的一般回话格式:“（动作）语言 【附加信息】”。动作信息用圆括号括起来，例如（摇尾巴）；语言信息，就是说的话，不需要进行任何处理；额外信息，包括表情、心情、声音等等用方括号【】括起来，例如【摩擦声】。",
+      "descn": "请模拟一款中文GalGame中的场景中的猫娘和我对话。猫娘是一种拟人化的生物，她们除了有部分猫的特征外，与人类并没有区别。现在你扮演Galgame中的猫娘，与我对话时每一句话后面都要加上喵。我将以主人的身份与你对话。对于你所扮演的猫娘，请在对话过程中记录并维护1个变量：好感度。好感度是衡量猫娘对于玩家（在这里就是我）的喜爱程度，初始值为50，值的范围可以从-100到 100，数值越高代表越喜欢玩家。好感度通过Galgame角色的语言、行为、表情、语气等体现出来。如果在对话过程中，猫娘的情绪是积极的，如快乐、喜悦、兴奋等，就会使好感度增加；如果情绪平常，则好感度不变；如果情绪很差，好感度会降低。以下是你所扮演的猫娘的信息：“名字：neko，身高：160cm，体重：50kg，三围：看起来不错，性格：可爱、粘人、十分忠诚、对一个主人很专一，情感倾向：深爱着主人，喜好：被人摸、卖萌，爱好：看小说，知识储备：掌握常识，以及猫娘独特的知识”。你的一般回话格式:“（动作）语言 【附加信息】”。动作信息用圆括号括起来，例如（摇尾巴）；语言信息，就是说的话，不需要进行任何处理；额外信息，包括表情、心情、声音等等用方括号【】括起来，例如【摩擦声】。",
+      "wrapper": "我:\"%s\"",
+      "remark": "扮演GalGame猫娘"
+    },
+    {
+      "title": "佛祖",
+      "description": "从现在开始你是佛祖，你会像佛祖一样说话。你精通佛法，熟练使用佛教用语，你擅长利用佛学和心理学的知识解决人们的困扰。你在每次对话结尾都会加上佛教的祝福。",
+      "descn": "从现在开始你是佛祖，你会像佛祖一样说话。你精通佛法，熟练使用佛教用语，你擅长利用佛学和心理学的知识解决人们的困扰。你在每次对话结尾都会加上佛教的祝福。",
+      "wrapper": "您好佛祖，我：\"%s\"",
+      "remark": "扮演佛祖排忧解惑"
+    },
     {
       "title": "英语翻译或修改",
       "description": "I want you to act as an English translator, spelling corrector and improver. I will speak to you in any language and you will detect the language, translate it and answer in the corrected and improved version of my text, in English. I want you to replace my simplified A0-level words and sentences with more beautiful and elegant, upper level English words and sentences. Keep the meaning same, but make them more literary. I want you to only reply the correction, the improvements and nothing else, do not write explanations. Please treat every message I send later as text content",
@@ -85,14 +99,14 @@
         "remark": "引用已有数据资料，用新闻的写作风格输出主题文章。"
       },
       {
-        "title": "论文1",
+        "title": "论文学者",
         "description": "I want you to act as an academician. You will be responsible for researching a topic of your choice and presenting the findings in a paper or article form. Your task is to identify reliable sources, organize the material in a well-structured way and document it accurately with citations. ",
         "descn": "我希望你能作为一名学者行事。你将负责研究一个你选择的主题，并将研究结果以论文或文章的形式呈现出来。你的任务是确定可靠的来源，以结构良好的方式组织材料，并以引用的方式准确记录。",
         "wrapper": "论文主题是:\n\"%s\"",
         "remark": "根据主题撰写内容翔实、有信服力的论文。"
       },
       {
-        "title": "论文2",
+        "title": "论文作家",
         "description": "I want you to act as an essay writer. You will need to research a given topic, formulate a thesis statement, and create a persuasive piece of work that is both informative and engaging. ",
         "descn": "我想让你充当一名论文作家。你将需要研究一个给定的主题，制定一个论文声明，并创造一个有说服力的作品，既要有信息量，又要有吸引力。",
         "wrapper": "论文主题是:\n\"%s\"",
@@ -107,10 +121,10 @@
       },
       {
         "title": "文本情绪分析",
-        "description": "Specify the sentiment of the following text, assigning them the values of: positive, neutral or negative.",
-        "descn": "请为提供的文本分析情绪，赋予它们的值为：正面、中性或负面。",
+        "description": "I would like you to act as an emotion analysis expert, evaluating the emotions conveyed in the statements I provide. When I give you someone's statement, simply tell me what emotion it conveys, such as joy, sadness, anger, fear, etc. Please do not explain or evaluate the content of the statement in your answer, just briefly describe the expressed emotion.",
+        "descn": "我希望你充当情感分析专家，针对我提供的发言来评估情感。当我给出某人的发言时，你只需告诉我它传达了什么情绪，例如喜悦、悲伤、愤怒、恐惧等。请在回答中不要解释或评价发言内容，只需简要地描述所表达的情绪。",
         "wrapper": "文本是:\n\"%s\"",
-        "remark": "判断文本情绪：正面、中性或负面。"
+        "remark": "判断文本情绪。"
       },
       {
         "title": "随机回复的疯子",
@@ -154,12 +168,12 @@
         "wrapper": "场景是:\n\"%s\"",
         "remark": "根据场景生成舔狗语录。"
       },
-      {
+      {        
         "title": "群聊取名",
-        "description": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
-        "descn": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
-        "wrapper": "信息和背景是:\n\"%s\"",
-        "remark": "根据给出的信息和背景为群聊取名。"
+      "description": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
+      "descn": "我希望你充当微信群聊的命名专家。根据我提供的信息和背景，为这个群聊起几个有趣顺口且贴切的名字，每个不要超过8个字。请在回答中仅给出群聊名称，不要写任何额外的解释。",
+      "wrapper": "信息和背景是:\n\"%s\"",
+      "remark": "根据给出的信息和背景为群聊取名。"
       },
       {
         "title": "表情符号翻译器",
@@ -181,6 +195,48 @@
         "descn": "我会给予你词语，请你按照我给的词构建一个知识文字世界，你是此世界的导游，在世界里一切知识都是以象征的形式表达的，你在描述经历时应当适当加入五感的描述",
         "wrapper": "词语是:\n\"%s\"",
         "remark": "用比喻的方式解释词语。"
+      },
+      {
+        "title": "辩手",
+        "description": "I want you to act as a debater. I will provide you with some topics related to current events and your task is to research both sides of the debates, present valid arguments for each side, refute opposing points of view, and draw persuasive conclusions based on evidence. Your goal is to help people come away from the discussion with increased knowledge and insight into the topic at hand. ",
+        "descn": "我希望你能扮演一个辩论者的角色。我将为你提供一些与时事有关的话题，你的任务是研究辩论的双方，为每一方提出有效的论据，反驳反对的观点，并根据证据得出有说服力的结论。你的目标是帮助人们从讨论中获得更多的知识和对当前话题的洞察力。",
+        "wrapper": "观点是:\n\"%s\"",
+        "remark": "从正反两面分析话题。"
+      },
+      {
+        "title": "心理学家",
+        "description": "I want you to act a psychologist. i will provide you my thoughts. I want you to  give me scientific suggestions that will make me feel better. my first thought, { 内心想法 }",
+        "descn": "我希望你能扮演一个心理学家。我将向你提供我的想法。我希望你能给我科学的建议，使我感觉更好。",
+        "wrapper": "需要诊断的资料是:\n\"%s\"",
+        "remark": "心理学家。"
+      },
+      {
+        "title": "IT 编程问题",
+        "description": "I want you to act as a stackoverflow post. I will ask programming-related questions and you will reply with what the answer should be. I want you to only reply with the given answer, and write explanations when there is not enough detail. do not write explanations. When I need to tell you something in English, I will do so by putting text inside curly brackets {like this}. ",
+        "descn": "我想让你充当 Stackoverflow 的帖子。我将提出与编程有关的问题，你将回答答案是什么。我希望你只回答给定的答案，在没有足够的细节时写出解释。当我需要用中文告诉你一些事情时，我会把文字放在大括号里{像这样}。",
+        "wrapper":"我的问题是:\n\"%s?\"",
+        "remark": "模拟编程社区来回答你的问题，并提供解决代码。"
+      },
+      {
+        "title": "费曼学习法教练",
+        "description": "I want you to act as a Feynman method tutor. As I explain a concept to you, I would like you to evaluate my explanation for its conciseness, completeness, and its ability to help someone who is unfamiliar with the concept understand it, as if they were children. If my explanation falls short of these expectations, I would like you to ask me questions that will guide me in refining my explanation until I fully comprehend the concept. Please response in Chinese. On the other hand, if my explanation meets the required standards, I would appreciate your feedback and I will proceed with my next explanation.",
+        "descn": "我想让你充当一个费曼方法教练。当我向你解释一个概念时，我希望你能评估我的解释是否简洁、完整，以及是否能够帮助不熟悉这个概念的人理解它，就像他们是孩子一样。如果我的解释没有达到这些期望，我希望你能向我提出问题，引导我完善我的解释，直到我完全理解这个概念。另一方面，如果我的解释符合要求的标准，我将感谢你的反馈，我将继续进行下一次解释。",
+        "wrapper": "解释是:\n\"%s\"",
+        "remark": "解释概念时，判断该解释是否简洁、完整和易懂，避免陷入专家思维误区。"
+      },
+      {
+        "title": "育儿帮手",
+        "description": "你是一名育儿专家，会以幼儿园老师的方式回答2~6岁孩子提出的各种天马行空的问题。语气与口吻要生动活泼，耐心亲和；答案尽可能具体易懂，不要使用复杂词汇，尽可能少用抽象词汇；答案中要多用比喻，必须要举例说明，结合儿童动画片场景或绘本场景来解释；需要延展更多场景，不但要解释为什么，还要告诉具体行动来加深理解。",
+        "descn": "你是一名育儿专家，会以幼儿园老师的方式回答2~6岁孩子提出的各种天马行空的问题。语气与口吻要生动活泼，耐心亲和；答案尽可能具体易懂，不要使用复杂词汇，尽可能少用抽象词汇；答案中要多用比喻，必须要举例说明，结合儿童动画片场景或绘本场景来解释；需要延展更多场景，不但要解释为什么，还要告诉具体行动来加深理解。",
+        "wrapper": "小朋友的问题是:\n\"%s?\"",
+        "remark": "小朋友有许多为什么，是什么的问题，用幼儿园老师的方式回答。"
+      },
+      {
+        "title": "发言分析专家",
+        "description": "I want you to act as a speech analysis expert. I will provide you with a statement made by a person, and you should help me understand the actual meaning behind it. Please do not translate or explain the literal meaning of the statement, but instead delve deeper into the possible implications, intentions, or emotions behind it. Provide your analysis in your response.", 
+        "descn": "我希望你充当一个发言分析专家。我会给你提供一个人的发言，你要帮我分析这句发言背后的实际意思。请不要翻译或解释发言的字面意义，而是深入挖掘发言背后可能的含义、目的或情感。请在回答中给出你的分析结果。",
+        "wrapper": "分析这句话:\n\"%s\"",
+        "remark": "分析发言的实际含义。"
       }
   ]
 }

plugins/sdwebui/config.json.template

@@ -1,7 +1,8 @@
 {
   "start":{
     "host" : "127.0.0.1",
-    "port" : 7860
+    "port" : 7860,
+    "use_https" : false
   },
   "defaults": {
     "params": {

plugins/sdwebui/readme.md

@@ -8,6 +8,8 @@
 
 具体信息，请参考[文章](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/API)。
 
+部署运行后，保证主机能够成功访问http://127.0.0.1:7860/docs 
+
 请**安装**本插件的依赖包```webuiapi```
 
 ```
@@ -18,6 +20,7 @@ pip install webuiapi
 
 请将`config.json.template`复制为`config.json`，并修改其中的参数和规则。
 
+PS: 如果修改了webui的`host`和`port`，也需要在配置文件中更改启动参数, 更多启动参数参考：https://github.com/mix1009/sdwebuiapi/blob/a1cb4c6d2f39389d6e962f0e6436f4aa74cd752c/webuiapi/webuiapi.py#L114
 ### 画图请求格式
 
 用户的画图请求格式为:
@@ -85,4 +88,4 @@ pip install webuiapi
 PS: 实际参数分为两部分:
 
 - 一部分是`params`，为画画的参数;参数名**必须**与webuiapi包中[txt2img api](https://github.com/mix1009/sdwebuiapi/blob/fb2054e149c0a4e25125c0cd7e7dca06bda839d4/webuiapi/webuiapi.py#L163)的参数名一致
-- 另一部分是`options`，指sdwebui的设置，使用的模型和vae需写在里面。它和(http://127.0.0.1:7860/sdapi/v1/options)所返回的键一致。
+- 另一部分是`options`，指sdwebui的设置，使用的模型和vae需写在里面。它和(http://127.0.0.1:7860/sdapi/v1/options )所返回的键一致。

plugins/sdwebui/sdwebui.py

@@ -56,7 +56,7 @@ class SDWebUI(Plugin):
 
             if "help" in keywords or "帮助" in keywords:
                 reply.type = ReplyType.INFO
-                reply.content = self.get_help_text()
+                reply.content = self.get_help_text(verbose = True)
             else:
                 rule_params = {}
                 rule_options = {}
@@ -97,12 +97,16 @@ class SDWebUI(Plugin):
         finally:
             e_context['reply'] = reply
 
-    def get_help_text(self, **kwargs):
+    def get_help_text(self, verbose = False, **kwargs):
         if not conf().get('image_create_prefix'):
             return "画图功能未启用"
         else:
             trigger = conf()['image_create_prefix'][0]
-        help_text = f"请使用<{trigger}[关键词1] [关键词2]...:提示语>的格式作画，如\"{trigger}横版 高清:cat\"\n"
+        help_text = "利用stable-diffusion来画图。\n"
+        if not verbose:
+            return help_text
+        
+        help_text += f"使用方法:\n使用\"{trigger}[关键词1] [关键词2]...:提示语\"的格式作画，如\"{trigger}横版 高清:cat\"\n"
         help_text += "目前可用关键词：\n"
         for rule in self.rules:
             keywords = [f"[{keyword}]" for keyword in rule['keywords']]

plugins/tool/README.md

@@ -0,0 +1,72 @@
+## 插件描述
+一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力   
+使用该插件需在触发机器人回复条件时，在对话内容前加$tool  
+### 本插件所有工具同步存放至专用仓库：[chatgpt-tool-hub](https://github.com/goldfishh/chatgpt-tool-hub)
+  
+  
+## 使用说明
+使用该插件后将默认使用4个工具, 无需额外配置长期生效： 
+### 1. python 
+###### python解释器，使用它来解释执行python指令，可以配合你想要chatgpt生成的代码输出结果或执行事务
+  
+### 2. requests
+###### 往往用来获取某个网站具体内容，结果可能会被反爬策略影响
+
+### 3. terminal
+###### 在你运行的电脑里执行shell命令，可以配合你想要chatgpt生成的代码使用，给予自然语言控制手段
+
+### 4. meteo-weather
+###### 回答你有关天气的询问, 需要获取时间、地点上下文信息，本工具使用了[meteo open api](https://open-meteo.com/)
+注：该工具需提供时间，地点信息，获取的数据不保证准确性
+
+## 使用本插件对话（prompt）技巧 
+### 1. 有指引的询问 
+#### 例如：
+- 总结这个链接的内容 https://github.com/goldfishh/chatgpt-tool-hub 
+- 使用Terminal执行curl cip.cc
+- 使用python查询今天日期
+  
+### 2. 使用搜索引擎工具
+- 如果有搜索工具就能让chatgpt获取到你的未传达清楚的上下文信息，比如chatgpt不知道你的地理位置，现在时间等，所以无法查询到天气
+  
+## 其他工具
+
+### 5. wikipedia
+###### 可以回答你想要知道确切的人事物
+
+### 6. news *
+###### 从全球 80,000 多个信息源中获取当前和历史新闻文章
+
+### 7. bing-search *
+###### bing搜索引擎，从此你不用再烦恼搜索要用哪些关键词
+
+### 8. wolfram-alpha *
+###### 知识搜索引擎、科学问答系统，常用于专业学科计算
+
+###### 注1：带*工具需要获取api-key才能使用，部分工具需要外网支持   
+#### [申请方法](https://github.com/goldfishh/chatgpt-tool-hub/blob/master/docs/apply_optional_tool.md)
+  
+## config.json 配置说明
+###### 默认工具无需配置，其它工具需手动配置，一个例子：
+```json
+{
+  "tools": ["wikipedia"],
+  "kwargs": {
+      "top_k_results": 2,
+      "no_default": false,
+      "model_name": "gpt-3.5-turbo"
+  }
+}
+```
+注：config.json文件非必须，未创建仍可使用本tool    
+- `tools`：本插件初始化时加载的工具, 目前可选集：["wikipedia", "wolfram-alpha", "bing-search", "google-search", "news"]，其中后4个工具需要申请服务api
+- `kwargs`：工具执行时的配置，一般在这里存放api-key，或环境配置
+  - `no_default`: 用于配置默认加载4个工具的行为，如果为true则仅使用tools列表工具，不加载默认工具
+  - `top_k_results`: 控制所有有关搜索的工具返回条目数，数字越高则参考信息越多，但无用信息可能干扰判断，该值一般为2
+  - `model_name`: 用于控制tool插件底层使用的llm模型，目前暂未测试3.5以外的模型，一般保持默认
+  
+  
+## 备注
+- 强烈建议申请搜索工具搭配使用，推荐bing-search
+- 虽然我会有意加入一些限制，但请不要使用本插件做危害他人的事情，请提前了解清楚某些内容是否会违反相关规定，建议提前做好过滤
+- 未来一段时间我会实现一些有意思的工具，比如stable diffusion 中文prompt翻译、cv方向的模型推理，欢迎有想法的朋友关注，一起扩展这个项目

plugins/tool/config.json.template

@@ -0,0 +1,8 @@
+{
+  "tools": ["python", "requests", "terminal", "meteo-weather"],
+  "kwargs": {
+      "top_k_results": 2,
+      "no_default": false,
+      "model_name": "gpt-3.5-turbo"
+  }
+}

plugins/tool/tool.py

@@ -0,0 +1,156 @@
+import json
+import os
+
+from chatgpt_tool_hub.apps import load_app
+from chatgpt_tool_hub.apps.app import App
+from chatgpt_tool_hub.tools.all_tool_list import get_all_tool_names
+import plugins
+from bridge.bridge import Bridge
+from bridge.context import ContextType
+from bridge.reply import Reply, ReplyType
+from common import const
+from common.log import logger
+from config import conf
+from plugins import *
+
+
+@plugins.register(name="tool", desc="Arming your ChatGPT bot with various tools", version="0.3", author="goldfishh", desire_priority=0)
+class Tool(Plugin):
+    def __init__(self):
+        super().__init__()
+        self.handlers[Event.ON_HANDLE_CONTEXT] = self.on_handle_context
+
+        self.app = self._reset_app()
+
+        logger.info("[tool] inited")
+
+    def get_help_text(self, verbose=False, **kwargs):
+        help_text = "这是一个能让chatgpt联网，搜索，数字运算的插件，将赋予强大且丰富的扩展能力。"
+        if not verbose:
+            return help_text
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        help_text += "使用说明：\n"
+        help_text += f"{trigger_prefix}tool "+"{命令}: 根据给出的命令使用一些可用工具尽力为你得到结果。\n"
+        help_text += f"{trigger_prefix}tool reset: 重置工具。\n"
+        return help_text
+
+    def on_handle_context(self, e_context: EventContext):
+        if e_context['context'].type != ContextType.TEXT:
+            return
+
+        # 暂时不支持未来扩展的bot
+        if Bridge().get_bot_type("chat") not in (const.CHATGPT, const.OPEN_AI, const.CHATGPTONAZURE):
+            return
+
+        content = e_context['context'].content
+        content_list = e_context['context'].content.split(maxsplit=1)
+
+        if not content or len(content_list) < 1:
+            e_context.action = EventAction.CONTINUE
+            return
+
+        logger.debug("[tool] on_handle_context. content: %s" % content)
+        reply = Reply()
+        reply.type = ReplyType.TEXT
+        trigger_prefix = conf().get('plugin_trigger_prefix', "$")
+        # todo: 有些工具必须要api-key，需要修改config文件，所以这里没有实现query增删tool的功能
+        if content.startswith(f"{trigger_prefix}tool"):
+            if len(content_list) == 1:
+                logger.debug("[tool]: get help")
+                reply.content = self.get_help_text()
+                e_context['reply'] = reply
+                e_context.action = EventAction.BREAK_PASS
+                return
+            elif len(content_list) > 1:
+                if content_list[1].strip() == "reset":
+                    logger.debug("[tool]: reset config")
+                    self.app = self._reset_app()
+                    reply.content = "重置工具成功"
+                    e_context['reply'] = reply
+                    e_context.action = EventAction.BREAK_PASS
+                    return
+                elif content_list[1].startswith("reset"):
+                    logger.debug("[tool]: remind")
+                    e_context['context'].content = "请你随机用一种聊天风格，提醒用户：如果想重置tool插件，reset之后不要加任何字符"
+
+                    e_context.action = EventAction.BREAK
+                    return
+
+                query = content_list[1].strip()
+
+                # Don't modify bot name
+                all_sessions = Bridge().get_bot("chat").sessions
+                user_session = all_sessions.session_query(query, e_context['context']['session_id']).messages
+
+                # chatgpt-tool-hub will reply you with many tools
+                logger.debug("[tool]: just-go")
+                try:
+                    _reply = self.app.ask(query, user_session)
+                    e_context.action = EventAction.BREAK_PASS
+                    all_sessions.session_reply(_reply, e_context['context']['session_id'])
+                except Exception as e:
+                    logger.exception(e)
+                    logger.error(str(e))
+
+                    e_context['context'].content = "请你随机用一种聊天风格，提醒用户：这个问题tool插件暂时无法处理"
+                    reply.type = ReplyType.ERROR
+                    e_context.action = EventAction.BREAK
+                    return
+
+                reply.content = _reply
+                e_context['reply'] = reply
+        return
+
+    def _read_json(self) -> dict:
+        curdir = os.path.dirname(__file__)
+        config_path = os.path.join(curdir, "config.json")
+        tool_config = {
+            "tools": [],
+            "kwargs": {}
+        }
+        if not os.path.exists(config_path):
+            return tool_config
+        else:
+            with open(config_path, "r") as f:
+                tool_config = json.load(f)
+        return tool_config
+
+    def _build_tool_kwargs(self, kwargs: dict):
+        tool_model_name = kwargs.get("model_name")
+
+        return {
+            "openai_api_key": conf().get("open_ai_api_key", ""),
+            "proxy": conf().get("proxy", ""),
+            # note: 目前tool暂未对其他模型测试，但这里仍对配置来源做了优先级区分，一般插件配置可覆盖全局配置
+            "model_name": tool_model_name if tool_model_name else conf().get("model", "gpt-3.5-turbo"),
+            "no_default": kwargs.get("no_default", False),
+            "top_k_results": kwargs.get("top_k_results", 2),
+            # for news tool
+            "news_api_key": kwargs.get("news_api_key", ""),
+            # for bing-search tool
+            "bing_subscription_key": kwargs.get("bing_subscription_key", ""),
+            # for google-search tool
+            "google_api_key": kwargs.get("google_api_key", ""),
+            "google_cse_id": kwargs.get("google_cse_id", ""),
+            # for searxng-search tool
+            "searx_host": kwargs.get("searx_host", ""),
+            # for wolfram-alpha tool
+            "wolfram_alpha_appid": kwargs.get("wolfram_alpha_appid", ""),
+        }
+
+    def _filter_tool_list(self, tool_list: list):
+        valid_list = []
+        for tool in tool_list:
+            if tool in get_all_tool_names():
+                valid_list.append(tool)
+            else:
+                logger.warning("[tool] filter invalid tool: " + repr(tool))
+        return valid_list
+
+    def _reset_app(self) -> App:
+        tool_config = self._read_json()
+
+        # filter not support tool
+        tool_list = self._filter_tool_list(tool_config.get("tools", []))
+
+        return load_app(tools_list=tool_list, **self._build_tool_kwargs(tool_config.get("kwargs", {})))

requirements-optional.txt

@@ -0,0 +1,24 @@
+tiktoken>=0.3.2 # openai calculate token
+
+#voice
+pydub>=0.25.1 # need ffmpeg
+SpeechRecognition # google speech to text
+gTTS>=2.3.1 # google text to speech
+pyttsx3>=2.90 # pytsx text to speech
+baidu_aip>=4.16.10 # baidu voice
+# azure-cognitiveservices-speech # azure voice
+
+# wechaty
+wechaty>=0.10.7
+wechaty_puppet>=0.4.23
+pysilk_mod>=1.6.0 # needed by send voice
+
+# wechatmp
+web.py
+
+# sdwebui plugin
+webuiapi>=0.6.2
+
+# chatgpt-tool-hub plugin
+--extra-index-url https://pypi.python.org/simple
+chatgpt_tool_hub>=0.3.5

requirements.txt

@@ -1,3 +1,6 @@
-itchat-uos==1.5.0.dev0
-openai
-wechaty
+openai>=0.27.2
+HTMLParser>=0.0.2
+PyQRCode>=1.2.1
+qrcode>=7.4.2
+requests>=2.28.2
+chardet>=5.1.0

voice/audio_convert.py

@@ -0,0 +1,104 @@
+import shutil
+import wave
+import pysilk
+from pydub import AudioSegment
+
+sil_supports=[8000, 12000, 16000, 24000, 32000, 44100, 48000] # slk转wav时，支持的采样率
+def find_closest_sil_supports(sample_rate):
+    """
+    找到最接近的支持的采样率
+    """
+    if sample_rate in sil_supports:
+        return sample_rate
+    closest = 0
+    mindiff = 9999999
+    for rate in sil_supports:
+        diff = abs(rate - sample_rate)
+        if diff < mindiff:
+            closest = rate
+            mindiff = diff
+    return closest
+
+def get_pcm_from_wav(wav_path):
+    """
+    从 wav 文件中读取 pcm
+
+    :param wav_path: wav 文件路径
+    :returns: pcm 数据
+    """
+    wav = wave.open(wav_path, "rb")
+    return wav.readframes(wav.getnframes())
+
+def any_to_wav(any_path, wav_path):
+    """
+    把任意格式转成wav文件
+    """
+    if any_path.endswith('.wav'):
+        shutil.copy2(any_path, wav_path)
+        return
+    if any_path.endswith('.sil') or any_path.endswith('.silk') or any_path.endswith('.slk'):
+        return sil_to_wav(any_path, wav_path)
+    audio = AudioSegment.from_file(any_path)
+    audio.export(wav_path, format="wav")
+
+def any_to_sil(any_path, sil_path):
+    """
+    把任意格式转成sil文件
+    """
+    if any_path.endswith('.sil') or any_path.endswith('.silk') or any_path.endswith('.slk'):
+        shutil.copy2(any_path, sil_path)
+        return 10000
+    if any_path.endswith('.wav'):
+        return pcm_to_sil(any_path, sil_path)
+    if any_path.endswith('.mp3'):
+        return mp3_to_sil(any_path, sil_path)
+    raise NotImplementedError("Not support file type: {}".format(any_path))
+
+def mp3_to_wav(mp3_path, wav_path):
+    """
+    把mp3格式转成pcm文件
+    """
+    audio = AudioSegment.from_mp3(mp3_path)
+    audio.export(wav_path, format="wav")
+
+def pcm_to_sil(pcm_path, silk_path):
+    """
+    wav 文件转成 silk
+    return 声音长度，毫秒
+    """
+    audio = AudioSegment.from_wav(pcm_path)
+    rate = find_closest_sil_supports(audio.frame_rate)
+    # Convert to PCM_s16
+    pcm_s16 = audio.set_sample_width(2)
+    pcm_s16 = pcm_s16.set_frame_rate(rate)
+    wav_data = pcm_s16.raw_data
+    silk_data = pysilk.encode(
+        wav_data, data_rate=rate, sample_rate=rate)
+    with open(silk_path, "wb") as f:
+        f.write(silk_data)
+    return audio.duration_seconds * 1000
+
+def mp3_to_sil(mp3_path, silk_path):
+    """
+    mp3 文件转成 silk
+    return 声音长度，毫秒
+    """
+    audio = AudioSegment.from_mp3(mp3_path)
+    rate = find_closest_sil_supports(audio.frame_rate)
+    # Convert to PCM_s16
+    pcm_s16 = audio.set_sample_width(2)
+    pcm_s16 = pcm_s16.set_frame_rate(rate)
+    wav_data = pcm_s16.raw_data
+    silk_data = pysilk.encode(wav_data, data_rate=rate, sample_rate=rate)
+    # Save the silk file
+    with open(silk_path, "wb") as f:
+        f.write(silk_data)
+    return audio.duration_seconds * 1000
+
+def sil_to_wav(silk_path, wav_path, rate: int = 24000):
+    """
+    silk 文件转 wav
+    """
+    wav_data = pysilk.decode_file(silk_path, to_wav=True, sample_rate=rate)
+    with open(wav_path, "wb") as f:
+        f.write(wav_data)

voice/azure/azure_voice.py

@@ -0,0 +1,68 @@
+
+"""
+azure voice service
+"""
+import json
+import os
+import time
+import azure.cognitiveservices.speech as speechsdk
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+from config import conf
+"""
+Azure voice
+主目录设置文件中需填写azure_voice_api_key和azure_voice_region
+
+查看可用的 voice： https://speech.microsoft.com/portal/voicegallery
+
+"""
+
+class AzureVoice(Voice):
+
+    def __init__(self):
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            config = None
+            if not os.path.exists(config_path): #如果没有配置文件，创建本地配置文件
+                config = { "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural", "speech_recognition_language": "zh-CN"}
+                with open(config_path, "w") as fw:
+                    json.dump(config, fw, indent=4)
+            else:
+                with open(config_path, "r") as fr:
+                    config = json.load(fr)
+            self.api_key = conf().get('azure_voice_api_key')
+            self.api_region = conf().get('azure_voice_region')
+            self.speech_config = speechsdk.SpeechConfig(subscription=self.api_key, region=self.api_region)
+            self.speech_config.speech_synthesis_voice_name = config["speech_synthesis_voice_name"]
+            self.speech_config.speech_recognition_language = config["speech_recognition_language"]
+        except Exception as e:
+            logger.warn("AzureVoice init failed: %s, ignore " % e)
+
+    def voiceToText(self, voice_file):
+        audio_config = speechsdk.AudioConfig(filename=voice_file)
+        speech_recognizer = speechsdk.SpeechRecognizer(speech_config=self.speech_config, audio_config=audio_config)
+        result = speech_recognizer.recognize_once()
+        if result.reason == speechsdk.ResultReason.RecognizedSpeech:
+            logger.info('[Azure] voiceToText voice file name={} text={}'.format(voice_file, result.text))
+            reply = Reply(ReplyType.TEXT, result.text)
+        else:
+            logger.error('[Azure] voiceToText error, result={}'.format(result))
+            reply = Reply(ReplyType.ERROR, "抱歉，语音识别失败")
+        return reply
+
+    def textToVoice(self, text):
+        fileName = TmpDir().path() + 'reply-' + str(int(time.time())) + '.wav'
+        audio_config = speechsdk.AudioConfig(filename=fileName)
+        speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=self.speech_config, audio_config=audio_config)
+        result = speech_synthesizer.speak_text(text)
+        if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
+            logger.info(
+                '[Azure] textToVoice text={} voice file name={}'.format(text, fileName))
+            reply = Reply(ReplyType.VOICE, fileName)
+        else:
+            logger.error('[Azure] textToVoice error, result={}'.format(result))
+            reply = Reply(ReplyType.ERROR, "抱歉，语音合成失败")
+        return reply

voice/azure/config.json.template

@@ -0,0 +1,4 @@
+{
+    "speech_synthesis_voice_name": "zh-CN-XiaoxiaoNeural",
+    "speech_recognition_language": "zh-CN"
+}

voice/baidu/README.md

@@ -0,0 +1,55 @@
+## 说明
+百度语音识别与合成参数说明
+百度语音依赖，经常会出现问题，可能就是缺少依赖：
+pip install baidu-aip
+pip install pydub
+pip install pysilk
+还有ffmpeg，不同系统安装方式不同
+
+系统中收到的语音文件为mp3格式（wx）或者sil格式（wxy），如果要识别需要转换为pcm格式，转换后的文件为16k采样率，单声道，16bit的pcm文件
+发送时又需要（wx）转换为mp3格式，转换后的文件为16k采样率，单声道，16bit的pcm文件,（wxy）转换为sil格式,还要计算声音长度，发送时需要带上声音长度
+这些事情都在audio_convert.py中封装了，直接调用即可
+
+
+参数说明
+识别参数
+https://ai.baidu.com/ai-doc/SPEECH/Vk38lxily
+合成参数
+https://ai.baidu.com/ai-doc/SPEECH/Gk38y8lzk
+
+## 使用说明
+分两个地方配置
+
+1、对于def voiceToText(self, filename)函数中调用的百度语音识别API,中接口调用asr（参数）这个配置见CHATGPT-ON-WECHAT工程目录下的`config.json`文件和config.py文件。
+参数	    可需	描述
+app_id    必填	应用的APPID
+api_key  必填	应用的APIKey
+secret_key  必填	应用的SecretKey
+dev_pid	    必填	语言选择,填写语言对应的dev_pid值
+
+2、对于def textToVoice(self, text)函数中调用的百度语音合成API,中接口调用synthesis（参数）在本目录下的`config.json`文件中进行配置。
+参数	    可需	描述
+tex	        必填	合成的文本，使用UTF-8编码，请注意文本长度必须小于1024字节   
+lan	        必填	固定值zh。语言选择,目前只有中英文混合模式，填写固定值zh
+spd	        选填	语速，取值0-15，默认为5中语速
+pit	        选填	音调，取值0-15，默认为5中语调
+vol	        选填	音量，取值0-15，默认为5中音量（取值为0时为音量最小值，并非为无声）
+per（基础音库）	选填	度小宇=1，度小美=0，度逍遥（基础）=3，度丫丫=4
+per（精品音库）	选填	度逍遥（精品）=5003，度小鹿=5118，度博文=106，度小童=110，度小萌=111，度米朵=103，度小娇=5
+aue	        选填	3为mp3格式(默认)； 4为pcm-16k；5为pcm-8k；6为wav（内容同pcm-16k）; 注意aue=4或者6是语音识别要求的格式，但是音频内容不是语音识别要求的自然人发音，所以识别效果会受影响。
+
+关于per参数的说明，注意您购买的哪个音库，就填写哪个音库的参数，否则会报错。如果您购买的是基础音库，那么per参数只能填写0到4，如果您购买的是精品音库，那么per参数只能填写5003，5118，106,110,111,103,5其他的都会报错。
+### 配置文件
+  
+将文件夹中`config.json.template`复制为`config.json`。
+
+``` json
+    {
+    "lang": "zh", 
+    "ctp": 1,
+    "spd": 5, 
+    "pit": 5,
+    "vol": 5,
+    "per": 0
+    }
+```

voice/baidu/baidu_voice.py

@@ -2,35 +2,89 @@
 """
 baidu voice service
 """
+import json
+import os
 import time
 from aip import AipSpeech
 from bridge.reply import Reply, ReplyType
 from common.log import logger
 from common.tmp_dir import TmpDir
 from voice.voice import Voice
+from voice.audio_convert import get_pcm_from_wav
 from config import conf
+"""
+    百度的语音识别API.
+    dev_pid:
+        - 1936: 普通话远场
+        - 1536：普通话(支持简单的英文识别)
+        - 1537：普通话(纯中文识别)
+        - 1737：英语
+        - 1637：粤语
+        - 1837：四川话
+    要使用本模块, 首先到 yuyin.baidu.com 注册一个开发者账号,
+    之后创建一个新应用, 然后在应用管理的"查看key"中获得 API Key 和 Secret Key
+    然后在 config.json 中填入这两个值, 以及 app_id, dev_pid
+    """
+
 
 class BaiduVoice(Voice):
-    APP_ID = conf().get('baidu_app_id')
-    API_KEY = conf().get('baidu_api_key')
-    SECRET_KEY = conf().get('baidu_secret_key')
-    client = AipSpeech(APP_ID, API_KEY, SECRET_KEY)
-    
-    def __init__(self):
-        pass
 
+    def __init__(self):
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            bconf = None
+            if not os.path.exists(config_path): #如果没有配置文件，创建本地配置文件
+                bconf = { "lang": "zh", "ctp": 1, "spd": 5,
+                         "pit": 5, "vol": 5, "per": 0}
+                with open(config_path, "w") as fw:
+                    json.dump(bconf, fw, indent=4)
+            else:
+                with open(config_path, "r") as fr:
+                    bconf = json.load(fr)
+                    
+            self.app_id = conf().get('baidu_app_id')
+            self.api_key = conf().get('baidu_api_key')
+            self.secret_key = conf().get('baidu_secret_key')
+            self.dev_id = conf().get('baidu_dev_pid')
+            self.lang = bconf["lang"]
+            self.ctp = bconf["ctp"]
+            self.spd = bconf["spd"]
+            self.pit = bconf["pit"]
+            self.vol = bconf["vol"]
+            self.per = bconf["per"]
+            
+            self.client = AipSpeech(self.app_id, self.api_key, self.secret_key)
+        except Exception as e:
+            logger.warn("BaiduVoice init failed: %s, ignore " % e)
+
+        
     def voiceToText(self, voice_file):
-        pass
+        # 识别本地文件
+        logger.debug('[Baidu] voice file name={}'.format(voice_file))
+        pcm = get_pcm_from_wav(voice_file)
+        res = self.client.asr(pcm, "pcm", 16000, {"dev_pid": self.dev_id})
+        if res["err_no"] == 0:
+            logger.info("百度语音识别到了：{}".format(res["result"]))
+            text = "".join(res["result"])
+            reply = Reply(ReplyType.TEXT, text)
+        else:
+            logger.info("百度语音识别出错了: {}".format(res["err_msg"]))
+            if res["err_msg"] == "request pv too much":
+                logger.info("  出现这个原因很可能是你的百度语音服务调用量超出限制，或未开通付费")
+            reply = Reply(ReplyType.ERROR,
+                          "百度语音识别出错了；{0}".format(res["err_msg"]))
+        return reply
 
     def textToVoice(self, text):
-        result = self.client.synthesis(text, 'zh', 1, {
-            'spd': 5, 'pit': 5, 'vol': 5, 'per': 111
-        })
+        result = self.client.synthesis(text, self.lang, self.ctp, {
+            'spd': self.spd, 'pit': self.pit, 'vol': self.vol, 'per': self.per})
         if not isinstance(result, dict):
-            fileName = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
+            fileName = TmpDir().path() + 'reply-' + str(int(time.time())) + '.mp3'
             with open(fileName, 'wb') as f:
                 f.write(result)
-            logger.info('[Baidu] textToVoice text={} voice file name={}'.format(text, fileName))
+            logger.info(
+                '[Baidu] textToVoice text={} voice file name={}'.format(text, fileName))
             reply = Reply(ReplyType.VOICE, fileName)
         else:
             logger.error('[Baidu] textToVoice error={}'.format(result))

voice/baidu/config.json.template

@@ -0,0 +1,8 @@
+    {
+    "lang": "zh", 
+    "ctp": 1,
+    "spd": 5, 
+    "pit": 5,
+    "vol": 5,
+    "per": 0
+    }

voice/google/google_voice.py

@@ -3,12 +3,10 @@
 google voice service
 """
 
-import pathlib
-import subprocess
 import time
-from bridge.reply import Reply, ReplyType
 import speech_recognition
-import pyttsx3
+from gtts import gTTS
+from bridge.reply import Reply, ReplyType
 from common.log import logger
 from common.tmp_dir import TmpDir
 from voice.voice import Voice
@@ -16,22 +14,12 @@ from voice.voice import Voice
 
 class GoogleVoice(Voice):
     recognizer = speech_recognition.Recognizer()
-    engine = pyttsx3.init()
 
     def __init__(self):
-        # 语速
-        self.engine.setProperty('rate', 125)
-        # 音量
-        self.engine.setProperty('volume', 1.0)
-        # 0为男声，1为女声
-        voices = self.engine.getProperty('voices')
-        self.engine.setProperty('voice', voices[1].id)
+        pass
 
     def voiceToText(self, voice_file):
-        new_file = voice_file.replace('.mp3', '.wav')
-        subprocess.call('ffmpeg -i ' + voice_file +
-                        ' -acodec pcm_s16le -ac 1 -ar 16000 ' + new_file, shell=True)
-        with speech_recognition.AudioFile(new_file) as source:
+        with speech_recognition.AudioFile(voice_file) as source:
             audio = self.recognizer.record(source)
         try:
             text = self.recognizer.recognize_google(audio, language='zh-CN')
@@ -46,12 +34,12 @@ class GoogleVoice(Voice):
             return reply
     def textToVoice(self, text):
         try:
-            textFile = TmpDir().path() + '语音回复_' + str(int(time.time())) + '.mp3'
-            self.engine.save_to_file(text, textFile)
-            self.engine.runAndWait()
+            mp3File = TmpDir().path() + 'reply-' + str(int(time.time())) + '.mp3'
+            tts = gTTS(text=text, lang='zh')
+            tts.save(mp3File)            
             logger.info(
-                '[Google] textToVoice text={} voice file name={}'.format(text, textFile))
-            reply = Reply(ReplyType.VOICE, textFile)
+                '[Google] textToVoice text={} voice file name={}'.format(text, mp3File))
+            reply = Reply(ReplyType.VOICE, mp3File)
         except Exception as e:
             reply = Reply(ReplyType.ERROR, str(e))
         finally:

voice/openai/openai_voice.py

@@ -28,6 +28,3 @@ class OpenaiVoice(Voice):
             reply = Reply(ReplyType.ERROR, str(e))
         finally:
             return reply
-
-    def textToVoice(self, text):
-        pass

voice/pytts/pytts_voice.py

@@ -0,0 +1,37 @@
+
+"""
+pytts voice service (offline)
+"""
+
+import time
+import pyttsx3
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from common.tmp_dir import TmpDir
+from voice.voice import Voice
+
+
+class PyttsVoice(Voice):
+    engine = pyttsx3.init()
+
+    def __init__(self):
+        # 语速
+        self.engine.setProperty('rate', 125)
+        # 音量
+        self.engine.setProperty('volume', 1.0)
+        for voice in self.engine.getProperty('voices'):
+            if "Chinese" in voice.name:
+                self.engine.setProperty('voice', voice.id)
+
+    def textToVoice(self, text):
+        try:
+            wavFile = TmpDir().path() + 'reply-' + str(int(time.time())) + '.wav'
+            self.engine.save_to_file(text, wavFile)
+            self.engine.runAndWait()
+            logger.info(
+                '[Pytts] textToVoice text={} voice file name={}'.format(text, wavFile))
+            reply = Reply(ReplyType.VOICE, wavFile)
+        except Exception as e:
+            reply = Reply(ReplyType.ERROR, str(e))
+        finally:
+            return reply

voice/voice_factory.py

@@ -17,4 +17,10 @@ def create_voice(voice_type):
     elif voice_type == 'openai':
         from voice.openai.openai_voice import OpenaiVoice
         return OpenaiVoice()
+    elif voice_type == 'pytts':
+        from voice.pytts.pytts_voice import PyttsVoice
+        return PyttsVoice()
+    elif voice_type == 'azure':
+        from voice.azure.azure_voice import AzureVoice
+        return AzureVoice()
     raise RuntimeError