Merge pull request #1753 from MasterKeee/master

新增公众号的回复视频类型

README.md

@@ -1,19 +1,22 @@
 # 简介
 
-> chatgpt-on-wechat（简称CoW）项目是基于大模型的智能对话机器人，支持微信公众号、企业微信应用、微信、飞书、钉钉接入，可选择GPT3.5/GPT4.0/Claude/Gemini/LinkAI/ChatGLM/KIMI/文心一言/讯飞星火/通义千问/LinkAI，能处理文本、语音和图片，通过插件访问操作系统和互联网等外部资源，支持基于自有知识库定制企业AI应用。
+> chatgpt-on-wechat（简称CoW）项目是基于大模型的智能对话机器人，支持微信公众号、企业微信应用、飞书、钉钉接入，可选择GPT3.5/GPT4.0/Claude/Gemini/LinkAI/ChatGLM/KIMI/文心一言/讯飞星火/通义千问/LinkAI，能处理文本、语音和图片，通过插件访问操作系统和互联网等外部资源，支持基于自有知识库定制企业AI应用。
 
 最新版本支持的功能如下：
 
--  ✅   **多端部署：** 有多种部署方式可选择且功能完备，目前已支持微信生态下公众号、企业微信应用、飞书、钉钉等部署方式
+-  ✅   **多端部署：** 有多种部署方式可选择且功能完备，目前已支持微信公众号、企业微信应用、飞书、钉钉等部署方式
 -  ✅   **基础对话：** 私聊及群聊的消息智能回复，支持多轮会话上下文记忆，支持 GPT-3.5, GPT-4, GPT-4o, Claude-3, Gemini, 文心一言, 讯飞星火, 通义千问，ChatGLM-4，Kimi(月之暗面)
 -  ✅   **语音能力：** 可识别语音消息，通过文字或语音回复，支持 azure, baidu, google, openai(whisper/tts) 等多种语音模型
 -  ✅   **图像能力：** 支持图片生成、图片识别、图生图（如照片修复），可选择 Dall-E-3, stable diffusion, replicate, midjourney, CogView-3, vision模型
 -  ✅   **丰富插件：** 支持个性化插件扩展，已实现多角色切换、文字冒险、敏感词过滤、聊天记录总结、文档总结和对话、联网搜索等插件
 -  ✅   **知识库：** 通过上传知识库文件自定义专属机器人，可作为数字分身、智能客服、私域助手使用，基于 [LinkAI](https://link-ai.tech) 实现
 
-## 演示
+## 声明
 
-https://github.com/zhayujie/chatgpt-on-wechat/assets/26161723/d5154020-36e3-41db-8706-40ce9f3f1b1e
+1. 本项目遵循 [MIT开源协议](/LICENSE)，仅用于技术研究和学习，使用本项目时需遵守所在地法律法规、相关政策以及企业章程，禁止用于任何违法或侵犯他人权益的行为
+2. 境内使用该项目时，请使用国内厂商的大模型服务，并进行必要的内容安全审核及过滤
+3. 本项目主要接入协同办公平台，推荐使用公众号、企微自建应用、钉钉、飞书等接入通道，其他通道为历史产物已不维护
+4. 任何个人、团队和企业，无论以何种方式使用该项目、对何对象提供服务，所产生的一切后果，本项目均不承担任何责任
 
 ## 社区
 
@@ -25,7 +28,7 @@ https://github.com/zhayujie/chatgpt-on-wechat/assets/26161723/d5154020-36e3-41db
 
 # 企业服务
 
-<a href="https://link-ai.tech" target="_blank"><img width="800" src="https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/github-linkai-intro-v2.png"></a>
+<a href="https://link-ai.tech" target="_blank"><img width="800" src="https://cdn.link-ai.tech/image/link-ai-intro.jpg"></a>
 
 > [LinkAI](https://link-ai.tech/) 是面向企业和开发者的一站式AI应用平台，聚合多模态大模型、知识库、Agent 插件、工作流等能力，支持一键接入主流平台并进行管理，支持SaaS、私有化部署多种模式。
 >
@@ -39,7 +42,7 @@ https://github.com/zhayujie/chatgpt-on-wechat/assets/26161723/d5154020-36e3-41db
 
 # 🏷 更新日志
 
->**2024.05.14：** [1.6.5版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.5)，新增 gpt-4o 模型支持
+>**2024.06.04：** [1.6.6版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.6) 和 [1.6.5版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.5)，gpt-4o模型、钉钉流式卡片、讯飞语音识别/合成
 
 >**2024.04.26：** [1.6.0版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.0)，新增 Kimi 接入、gpt-4-turbo版本升级、文件总结和语音识别问题修复
 
@@ -139,7 +142,7 @@ pip3 install -r requirements-optional.txt
   "subscribe_msg": "感谢您的关注！\n这里是ChatGPT，可以自由对话。\n支持语音对话。\n支持图片输出，画字开头的消息将按要求创作图片。\n支持角色扮演和文字冒险等丰富插件。\n输入{trigger_prefix}#help 查看详细指令。",
   "use_linkai": false,                                        # 是否使用LinkAI接口，默认关闭，开启后可国内访问，使用知识库和MJ
   "linkai_api_key": "",                                       # LinkAI Api Key
-  "linkai_app_code": ""                                       # LinkAI 应用code
+  "linkai_app_code": ""                                       # LinkAI 应用或工作流code
 }
 ```
 **配置说明：**
@@ -160,7 +163,7 @@ pip3 install -r requirements-optional.txt
 
 + 添加 `"speech_recognition": true` 将开启语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，该参数仅支持私聊 (注意由于语音消息无法匹配前缀，一旦开启将对所有语音自动回复，支持语音触发画图)；
 + 添加 `"group_speech_recognition": true` 将开启群组语音识别，默认使用openai的whisper模型识别为文字，同时以文字回复，参数仅支持群聊 (会匹配group_chat_prefix和group_chat_keyword, 支持语音触发画图)；
-+ 添加 `"voice_reply_voice": true` 将开启语音回复语音（同时作用于私聊和群聊），但是需要配置对应语音合成平台的key，由于itchat协议的限制，只能发送语音mp3文件，若使用wechaty则回复的是微信语音。
++ 添加 `"voice_reply_voice": true` 将开启语音回复语音（同时作用于私聊和群聊）
 
 **4.其他配置**
 
@@ -172,7 +175,7 @@ pip3 install -r requirements-optional.txt
 + `conversation_max_tokens`：表示能够记忆的上下文最大字数（一问一答为一组对话，如果累积的对话字数超出限制，就会优先移除最早的一组对话）
 + `rate_limit_chatgpt`，`rate_limit_dalle`：每分钟最高问答速率、画图速率，超速后排队按序处理。
 + `clear_memory_commands`: 对话内指令，主动清空前文记忆，字符串数组可自定义指令别名。
-+ `hot_reload`: 程序退出后，暂存微信扫码状态，默认关闭。
++ `hot_reload`: 程序退出后，暂存等于状态，默认关闭。
 + `character_desc` 配置中保存着你对机器人说的一段话，他会记住这段话并作为他的设定，你可以为他定制任何人格      (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
 + `subscribe_msg`：订阅消息，公众号和企业微信channel中请填写，当被订阅时会自动回复， 可使用特殊占位符。目前支持的占位符有{trigger_prefix}，在程序中它会自动替换成bot的触发词。
 
@@ -180,7 +183,7 @@ pip3 install -r requirements-optional.txt
 
 + `use_linkai`: 是否使用LinkAI接口，开启后可国内访问，使用知识库和 `Midjourney` 绘画, 参考 [文档](https://link-ai.tech/platform/link-app/wechat)
 + `linkai_api_key`: LinkAI Api Key，可在 [控制台](https://link-ai.tech/console/interface) 创建
-+ `linkai_app_code`: LinkAI 应用code，选填
++ `linkai_app_code`: LinkAI 应用或工作流的code，选填
 
 **本说明文档可能会未及时更新，当前所有可选的配置项均在该[`config.py`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
 
@@ -194,7 +197,7 @@ pip3 install -r requirements-optional.txt
 python3 app.py                                    # windows环境下该命令通常为 python app.py
 ```
 
-终端输出二维码后，使用微信进行扫码，当输出 "Start auto replying" 时表示自动回复程序已经成功运行了（注意：用于登录的微信需要在支付处已完成实名认证）。扫码登录后你的账号就成为机器人了，可以在手机端通过配置的关键词触发自动回复 (任意好友发送消息给你，或是自己发消息给好友)，参考[#142](https://github.com/zhayujie/chatgpt-on-wechat/issues/142)。
+终端输出二维码后，进行扫码登录，当输出 "Start auto replying" 时表示自动回复程序已经成功运行了（注意：用于登录的账号需要在支付处已完成实名认证）。扫码登录后你的账号就成为机器人了，可以在手机端通过配置的关键词触发自动回复 (任意好友发送消息给你，或是自己发消息给好友)，参考[#142](https://github.com/zhayujie/chatgpt-on-wechat/issues/142)。
 
 ### 2.服务器部署
 

bot/baidu/baidu_wenxin.py

@@ -94,7 +94,7 @@ class BaiduWenxinBot(Bot):
             logger.warn("[BAIDU] Exception: {}".format(e))
             need_retry = False
             self.sessions.clear_session(session.session_id)
-            result = {"completion_tokens": 0, "content": "出错了: {}".format(e)}
+            result = {"total_tokens": 0, "completion_tokens": 0, "content": "出错了: {}".format(e)}
             return result
 
     def get_access_token(self):

bot/chatgpt/chat_gpt_bot.py

@@ -171,24 +171,48 @@ class AzureChatGPTBot(ChatGPTBot):
         self.args["deployment_id"] = conf().get("azure_deployment_id")
 
     def create_img(self, query, retry_count=0, api_key=None):
-        api_version = "2022-08-03-preview"
-        url = "{}dalle/text-to-image?api-version={}".format(openai.api_base, api_version)
-        api_key = api_key or openai.api_key
-        headers = {"api-key": api_key, "Content-Type": "application/json"}
-        try:
-            body = {"caption": query, "resolution": conf().get("image_create_size", "256x256")}
-            submission = requests.post(url, headers=headers, json=body)
-            operation_location = submission.headers["Operation-Location"]
-            retry_after = submission.headers["Retry-after"]
-            status = ""
-            image_url = ""
-            while status != "Succeeded":
-                logger.info("waiting for image create..., " + status + ",retry after " + retry_after + " seconds")
-                time.sleep(int(retry_after))
-                response = requests.get(operation_location, headers=headers)
-                status = response.json()["status"]
-            image_url = response.json()["result"]["contentUrl"]
-            return True, image_url
-        except Exception as e:
-            logger.error("create image error: {}".format(e))
-            return False, "图片生成失败"
+        text_to_image_model = conf().get("text_to_image")
+        if text_to_image_model == "dall-e-2":
+            api_version = "2023-06-01-preview"
+            endpoint = conf().get("azure_openai_dalle_api_base","open_ai_api_base")
+            # 检查endpoint是否以/结尾
+            if not endpoint.endswith("/"):
+                endpoint = endpoint + "/"
+            url = "{}openai/images/generations:submit?api-version={}".format(endpoint, api_version)
+            api_key = conf().get("azure_openai_dalle_api_key","open_ai_api_key")
+            headers = {"api-key": api_key, "Content-Type": "application/json"}
+            try:
+                body = {"prompt": query, "size": conf().get("image_create_size", "256x256"),"n": 1}
+                submission = requests.post(url, headers=headers, json=body)
+                operation_location = submission.headers['operation-location']
+                status = ""
+                while (status != "succeeded"):
+                    if retry_count > 3:
+                        return False, "图片生成失败"
+                    response = requests.get(operation_location, headers=headers)
+                    status = response.json()['status']
+                    retry_count += 1
+                image_url = response.json()['result']['data'][0]['url']
+                return True, image_url
+            except Exception as e:
+                logger.error("create image error: {}".format(e))
+                return False, "图片生成失败"
+        elif text_to_image_model == "dall-e-3":
+            api_version = conf().get("azure_api_version", "2024-02-15-preview")
+            endpoint = conf().get("azure_openai_dalle_api_base","open_ai_api_base")
+            # 检查endpoint是否以/结尾
+            if not endpoint.endswith("/"):
+                endpoint = endpoint + "/"
+            url = "{}openai/deployments/{}/images/generations?api-version={}".format(endpoint, conf().get("azure_openai_dalle_deployment_id","text_to_image"),api_version)
+            api_key = conf().get("azure_openai_dalle_api_key","open_ai_api_key")
+            headers = {"api-key": api_key, "Content-Type": "application/json"}
+            try:
+                body = {"prompt": query, "size": conf().get("image_create_size", "1024x1024"), "quality": conf().get("dalle3_image_quality", "standard")}
+                submission = requests.post(url, headers=headers, json=body)
+                image_url = submission.json()['data'][0]['url']
+                return True, image_url
+            except Exception as e:
+                logger.error("create image error: {}".format(e))
+                return False, "图片生成失败"
+        else:
+            return False, "图片生成失败，未配置text_to_image参数"

channel/chat_channel.py

@@ -86,14 +86,14 @@ class ChatChannel(Channel):
             if e_context.is_pass() or context is None:
                 return context
             if cmsg.from_user_id == self.user_id and not config.get("trigger_by_self", True):
-                logger.debug("[WX]self message skipped")
+                logger.debug("[chat_channel]self message skipped")
                 return None
 
         # 消息内容匹配过程，并处理content
         if ctype == ContextType.TEXT:
             if first_in and "」\n- - - - - - -" in content:  # 初次匹配 过滤引用消息
                 logger.debug(content)
-                logger.debug("[WX]reference query skipped")
+                logger.debug("[chat_channel]reference query skipped")
                 return None
 
             nick_name_black_list = conf().get("nick_name_black_list", [])
@@ -111,10 +111,10 @@ class ChatChannel(Channel):
                         nick_name = context["msg"].actual_user_nickname
                         if nick_name and nick_name in nick_name_black_list:
                             # 黑名单过滤
-                            logger.warning(f"[WX] Nickname {nick_name} in In BlackList, ignore")
+                            logger.warning(f"[chat_channel] Nickname {nick_name} in In BlackList, ignore")
                             return None
 
-                        logger.info("[WX]receive group at")
+                        logger.info("[chat_channel]receive group at")
                         if not conf().get("group_at_off", False):
                             flag = True
                         pattern = f"@{re.escape(self.name)}(\u2005|\u0020)"
@@ -130,13 +130,13 @@ class ChatChannel(Channel):
                         content = subtract_res
                 if not flag:
                     if context["origin_ctype"] == ContextType.VOICE:
-                        logger.info("[WX]receive group voice, but checkprefix didn't match")
+                        logger.info("[chat_channel]receive group voice, but checkprefix didn't match")
                     return None
             else:  # 单聊
                 nick_name = context["msg"].from_user_nickname
                 if nick_name and nick_name in nick_name_black_list:
                     # 黑名单过滤
-                    logger.warning(f"[WX] Nickname '{nick_name}' in In BlackList, ignore")
+                    logger.warning(f"[chat_channel] Nickname '{nick_name}' in In BlackList, ignore")
                     return None
 
                 match_prefix = check_prefix(content, conf().get("single_chat_prefix", [""]))
@@ -147,7 +147,7 @@ class ChatChannel(Channel):
                 else:
                     return None
             content = content.strip()
-            img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
+            img_match_prefix = check_prefix(content, conf().get("image_create_prefix",[""]))
             if img_match_prefix:
                 content = content.replace(img_match_prefix, "", 1)
                 context.type = ContextType.IMAGE_CREATE
@@ -159,17 +159,16 @@ class ChatChannel(Channel):
         elif context.type == ContextType.VOICE:
             if "desire_rtype" not in context and conf().get("voice_reply_voice") and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
                 context["desire_rtype"] = ReplyType.VOICE
-
         return context
 
     def _handle(self, context: Context):
         if context is None or not context.content:
             return
-        logger.debug("[WX] ready to handle context: {}".format(context))
+        logger.debug("[chat_channel] ready to handle context: {}".format(context))
         # reply的构建步骤
         reply = self._generate_reply(context)
 
-        logger.debug("[WX] ready to decorate reply: {}".format(reply))
+        logger.debug("[chat_channel] ready to decorate reply: {}".format(reply))
 
         # reply的包装步骤
         if reply and reply.content:
@@ -187,7 +186,7 @@ class ChatChannel(Channel):
         )
         reply = e_context["reply"]
         if not e_context.is_pass():
-            logger.debug("[WX] ready to handle context: type={}, content={}".format(context.type, context.content))
+            logger.debug("[chat_channel] ready to handle context: type={}, content={}".format(context.type, context.content))
             if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE:  # 文字和图片消息
                 context["channel"] = e_context["channel"]
                 reply = super().build_reply_content(context.content, context)
@@ -199,7 +198,7 @@ class ChatChannel(Channel):
                 try:
                     any_to_wav(file_path, wav_path)
                 except Exception as e:  # 转换失败，直接使用mp3，对于某些api，mp3也可以识别
-                    logger.warning("[WX]any to wav error, use raw path. " + str(e))
+                    logger.warning("[chat_channel]any to wav error, use raw path. " + str(e))
                     wav_path = file_path
                 # 语音识别
                 reply = super().build_voice_to_text(wav_path)
@@ -210,7 +209,7 @@ class ChatChannel(Channel):
                         os.remove(wav_path)
                 except Exception as e:
                     pass
-                    # logger.warning("[WX]delete temp file error: " + str(e))
+                    # logger.warning("[chat_channel]delete temp file error: " + str(e))
 
                 if reply.type == ReplyType.TEXT:
                     new_context = self._compose_context(ContextType.TEXT, reply.content, **context.kwargs)
@@ -228,7 +227,7 @@ class ChatChannel(Channel):
             elif context.type == ContextType.FUNCTION or context.type == ContextType.FILE:  # 文件消息及函数调用等，当前无默认逻辑
                 pass
             else:
-                logger.warning("[WX] unknown context type: {}".format(context.type))
+                logger.warning("[chat_channel] unknown context type: {}".format(context.type))
                 return
         return reply
 
@@ -244,7 +243,7 @@ class ChatChannel(Channel):
             desire_rtype = context.get("desire_rtype")
             if not e_context.is_pass() and reply and reply.type:
                 if reply.type in self.NOT_SUPPORT_REPLYTYPE:
-                    logger.error("[WX]reply type not support: " + str(reply.type))
+                    logger.error("[chat_channel]reply type not support: " + str(reply.type))
                     reply.type = ReplyType.ERROR
                     reply.content = "不支持发送的消息类型: " + str(reply.type)
 
@@ -265,10 +264,10 @@ class ChatChannel(Channel):
                 elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE or reply.type == ReplyType.FILE or reply.type == ReplyType.VIDEO or reply.type == ReplyType.VIDEO_URL:
                     pass
                 else:
-                    logger.error("[WX] unknown reply type: {}".format(reply.type))
+                    logger.error("[chat_channel] unknown reply type: {}".format(reply.type))
                     return
             if desire_rtype and desire_rtype != reply.type and reply.type not in [ReplyType.ERROR, ReplyType.INFO]:
-                logger.warning("[WX] desire_rtype: {}, but reply type: {}".format(context.get("desire_rtype"), reply.type))
+                logger.warning("[chat_channel] desire_rtype: {}, but reply type: {}".format(context.get("desire_rtype"), reply.type))
             return reply
 
     def _send_reply(self, context: Context, reply: Reply):
@@ -281,14 +280,14 @@ class ChatChannel(Channel):
             )
             reply = e_context["reply"]
             if not e_context.is_pass() and reply and reply.type:
-                logger.debug("[WX] ready to send reply: {}, context: {}".format(reply, context))
+                logger.debug("[chat_channel] ready to send reply: {}, context: {}".format(reply, context))
                 self._send(reply, context)
 
     def _send(self, reply: Reply, context: Context, retry_cnt=0):
         try:
             self.send(reply, context)
         except Exception as e:
-            logger.error("[WX] sendMsg error: {}".format(str(e)))
+            logger.error("[chat_channel] sendMsg error: {}".format(str(e)))
             if isinstance(e, NotImplementedError):
                 return
             logger.exception(e)
@@ -342,7 +341,7 @@ class ChatChannel(Channel):
                     if semaphore.acquire(blocking=False):  # 等线程处理完毕才能删除
                         if not context_queue.empty():
                             context = context_queue.get()
-                            logger.debug("[WX] consume context: {}".format(context))
+                            logger.debug("[chat_channel] consume context: {}".format(context))
                             future: Future = handler_pool.submit(self._handle, context)
                             future.add_done_callback(self._thread_pool_callback(session_id, context=context))
                             if session_id not in self.futures:

channel/dingtalk/dingtalk_channel.py

@@ -4,20 +4,81 @@
 @author huiwen
 @Date 2023/11/28
 """
-
+import copy
+import json
 # -*- coding=utf-8 -*-
+import logging
+import time
+
+import dingtalk_stream
+from dingtalk_stream import AckMessage
+from dingtalk_stream.card_replier import AICardReplier
+from dingtalk_stream.card_replier import AICardStatus
+from dingtalk_stream.card_replier import CardReplier
+
+from bridge.context import Context, ContextType
+from bridge.reply import Reply, ReplyType
+from channel.chat_channel import ChatChannel
 from channel.dingtalk.dingtalk_message import DingTalkMessage
-from bridge.context import Context
-from bridge.reply import Reply
+from common.expired_dict import ExpiredDict
 from common.log import logger
 from common.singleton import singleton
+from common.time_check import time_checker
 from config import conf
-from common.expired_dict import ExpiredDict
-from bridge.context import ContextType
-from channel.chat_channel import ChatChannel
-import logging
-from dingtalk_stream import AckMessage
-import dingtalk_stream
+
+
+class CustomAICardReplier(CardReplier):
+    def __init__(self, dingtalk_client, incoming_message):
+        super(AICardReplier, self).__init__(dingtalk_client, incoming_message)
+
+    def start(
+            self,
+            card_template_id: str,
+            card_data: dict,
+            recipients: list = None,
+            support_forward: bool = True,
+    ) -> str:
+        """
+        AI卡片的创建接口
+        :param support_forward:
+        :param recipients:
+        :param card_template_id:
+        :param card_data:
+        :return:
+        """
+        card_data_with_status = copy.deepcopy(card_data)
+        card_data_with_status["flowStatus"] = AICardStatus.PROCESSING
+        return self.create_and_send_card(
+            card_template_id,
+            card_data_with_status,
+            at_sender=True,
+            at_all=False,
+            recipients=recipients,
+            support_forward=support_forward,
+        )
+
+
+# 对 AICardReplier 进行猴子补丁
+AICardReplier.start = CustomAICardReplier.start
+
+
+def _check(func):
+    def wrapper(self, cmsg: DingTalkMessage):
+        msgId = cmsg.msg_id
+        if msgId in self.receivedMsgs:
+            logger.info("DingTalk message {} already received, ignore".format(msgId))
+            return
+        self.receivedMsgs[msgId] = True
+        create_time = cmsg.create_time  # 消息时间戳
+        if conf().get("hot_reload") == True and int(create_time) < int(time.time()) - 60:  # 跳过1分钟前的历史消息
+            logger.debug("[DingTalk] History message {} skipped".format(msgId))
+            return
+        if cmsg.my_msg and not cmsg.is_group:
+            logger.debug("[DingTalk] My message {} skipped".format(msgId))
+            return
+        return func(self, cmsg)
+
+    return wrapper
 
 
 @singleton
@@ -39,11 +100,13 @@ class DingTalkChanel(ChatChannel, dingtalk_stream.ChatbotHandler):
         super(dingtalk_stream.ChatbotHandler, self).__init__()
         self.logger = self.setup_logger()
         # 历史消息id暂存，用于幂等控制
-        self.receivedMsgs = ExpiredDict(60 * 60 * 7.1)
-        logger.info("[dingtalk] client_id={}, client_secret={} ".format(
+        self.receivedMsgs = ExpiredDict(conf().get("expires_in_seconds"))
+        logger.info("[DingTalk] client_id={}, client_secret={} ".format(
             self.dingtalk_client_id, self.dingtalk_client_secret))
         # 无需群校验和前缀
         conf()["group_name_white_list"] = ["ALL_GROUP"]
+        # 单聊无需前缀
+        conf()["single_chat_prefix"] = [""]
 
     def startup(self):
         credential = dingtalk_stream.Credential(self.dingtalk_client_id, self.dingtalk_client_secret)
@@ -51,50 +114,112 @@ class DingTalkChanel(ChatChannel, dingtalk_stream.ChatbotHandler):
         client.register_callback_handler(dingtalk_stream.chatbot.ChatbotMessage.TOPIC, self)
         client.start_forever()
 
+    async def process(self, callback: dingtalk_stream.CallbackMessage):
+        try:
+            incoming_message = dingtalk_stream.ChatbotMessage.from_dict(callback.data)
+            image_download_handler = self  # 传入方法所在的类实例
+            dingtalk_msg = DingTalkMessage(incoming_message, image_download_handler)
+
+            if dingtalk_msg.is_group:
+                self.handle_group(dingtalk_msg)
+            else:
+                self.handle_single(dingtalk_msg)
+            return AckMessage.STATUS_OK, 'OK'
+        except Exception as e:
+            logger.error(f"dingtalk process error={e}")
+            return AckMessage.STATUS_SYSTEM_EXCEPTION, 'ERROR'
+
+    @time_checker
+    @_check
     def handle_single(self, cmsg: DingTalkMessage):
         # 处理单聊消息
         if cmsg.ctype == ContextType.VOICE:
-            logger.debug("[dingtalk]receive voice msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive voice msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.IMAGE:
-            logger.debug("[dingtalk]receive image msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive image msg: {}".format(cmsg.content))
+        elif cmsg.ctype == ContextType.IMAGE_CREATE:
+            logger.debug("[DingTalk]receive image create msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.PATPAT:
-            logger.debug("[dingtalk]receive patpat msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.TEXT:
-            expression = cmsg.my_msg
-        cmsg.content = conf()["single_chat_prefix"][0] + cmsg.content
+            logger.debug("[DingTalk]receive text msg: {}".format(cmsg.content))
+        else:
+            logger.debug("[DingTalk]receive other msg: {}".format(cmsg.content))
         context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=False, msg=cmsg)
         if context:
             self.produce(context)
 
+
+    @time_checker
+    @_check
     def handle_group(self, cmsg: DingTalkMessage):
         # 处理群聊消息
         if cmsg.ctype == ContextType.VOICE:
-            logger.debug("[dingtalk]receive voice msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive voice msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.IMAGE:
-            logger.debug("[dingtalk]receive image msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive image msg: {}".format(cmsg.content))
+        elif cmsg.ctype == ContextType.IMAGE_CREATE:
+            logger.debug("[DingTalk]receive image create msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.PATPAT:
-            logger.debug("[dingtalk]receive patpat msg: {}".format(cmsg.content))
+            logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
         elif cmsg.ctype == ContextType.TEXT:
-            expression = cmsg.my_msg
-        cmsg.content = conf()["group_chat_prefix"][0] + cmsg.content
+            logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
+        else:
+            logger.debug("[DingTalk]receive other msg: {}".format(cmsg.content))
         context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=True, msg=cmsg)
         context['no_need_at'] = True
         if context:
             self.produce(context)
 
-    async def process(self, callback: dingtalk_stream.CallbackMessage):
-        try:
-            incoming_message = dingtalk_stream.ChatbotMessage.from_dict(callback.data)
-            dingtalk_msg = DingTalkMessage(incoming_message)
-            if incoming_message.conversation_type == '1':
-                self.handle_single(dingtalk_msg)
-            else:
-                self.handle_group(dingtalk_msg)
-            return AckMessage.STATUS_OK, 'OK'
-        except Exception as e:
-            logger.error(e)
-            return self.FAILED_MSG
 
     def send(self, reply: Reply, context: Context):
+        receiver = context["receiver"]
+        isgroup = context.kwargs['msg'].is_group
         incoming_message = context.kwargs['msg'].incoming_message
-        self.reply_text(reply.content, incoming_message)
+
+        if conf().get("dingtalk_card_enabled"):
+            logger.info("[Dingtalk] sendMsg={}, receiver={}".format(reply, receiver))
+            def reply_with_text():
+                self.reply_text(reply.content, incoming_message)
+            def reply_with_at_text():
+                self.reply_text("📢 您有一条新的消息，请查看。", incoming_message)
+            def reply_with_ai_markdown():
+                button_list, markdown_content = self.generate_button_markdown_content(context, reply)
+                self.reply_ai_markdown_button(incoming_message, markdown_content, button_list, "", "📌 内容由AI生成", "",[incoming_message.sender_staff_id])
+
+            if reply.type in [ReplyType.IMAGE_URL, ReplyType.IMAGE, ReplyType.TEXT]:
+                if isgroup:
+                    reply_with_ai_markdown()
+                    reply_with_at_text()
+                else:
+                    reply_with_ai_markdown()
+            else:
+                # 暂不支持其它类型消息回复
+                reply_with_text()
+        else:
+            self.reply_text(reply.content, incoming_message)
+
+
+    def generate_button_markdown_content(self, context, reply):
+        image_url = context.kwargs.get("image_url")
+        promptEn = context.kwargs.get("promptEn")
+        reply_text = reply.content
+        button_list = []
+        markdown_content = f"""
+{reply.content}
+                                """
+        if image_url is not None and promptEn is not None:
+            button_list = [
+                {"text": "查看原图", "url": image_url, "iosUrl": image_url, "color": "blue"}
+            ]
+            markdown_content = f"""
+{promptEn}
+
+!["图片"]({image_url})
+
+{reply_text}
+
+                                """
+        logger.debug(f"[Dingtalk] generate_button_markdown_content, button_list={button_list} , markdown_content={markdown_content}")
+
+        return button_list, markdown_content

channel/dingtalk/dingtalk_message.py

@@ -1,44 +1,83 @@
-from bridge.context import ContextType
-from channel.chat_message import ChatMessage
-import json
+import os
+
 import requests
-from common.log import logger
-from common.tmp_dir import TmpDir
-from common import utils
 from dingtalk_stream import ChatbotMessage
 
+from bridge.context import ContextType
+from channel.chat_message import ChatMessage
+# -*- coding=utf-8 -*-
+from common.log import logger
+from common.tmp_dir import TmpDir
+
+
 class DingTalkMessage(ChatMessage):
-    def __init__(self, event: ChatbotMessage):
+    def __init__(self, event: ChatbotMessage, image_download_handler):
         super().__init__(event)
-        
+        self.image_download_handler = image_download_handler
         self.msg_id = event.message_id
-        msg_type = event.message_type
-        self.incoming_message =event
+        self.message_type = event.message_type
+        self.incoming_message = event
         self.sender_staff_id = event.sender_staff_id
         self.other_user_id = event.conversation_id
         self.create_time = event.create_at
-        if event.conversation_type=="1":
+        self.image_content = event.image_content
+        self.rich_text_content = event.rich_text_content
+        if event.conversation_type == "1":
             self.is_group = False
         else:
             self.is_group = True
-        
 
-        if msg_type == "text":
+        if self.message_type == "text":
             self.ctype = ContextType.TEXT
-            
+
             self.content = event.text.content.strip()
-        elif msg_type == "audio":
-            
+        elif self.message_type == "audio":
             # 钉钉支持直接识别语音，所以此处将直接提取文字，当文字处理
             self.content = event.extensions['content']['recognition'].strip()
             self.ctype = ContextType.TEXT
+        elif (self.message_type == 'picture') or (self.message_type == 'richText'):
+            self.ctype = ContextType.IMAGE
+            # 钉钉图片类型或富文本类型消息处理
+            image_list = event.get_image_list()
+            if len(image_list) > 0:
+                download_code = image_list[0]
+                download_url = image_download_handler.get_image_download_url(download_code)
+                self.content = download_image_file(download_url, TmpDir().path())
+            else:
+                logger.debug(f"[Dingtalk] messageType :{self.message_type} , imageList isEmpty")
+
         if self.is_group:
             self.from_user_id = event.conversation_id
             self.actual_user_id = event.sender_id
         else:
             self.from_user_id = event.sender_id
+            self.actual_user_id = event.sender_id
         self.to_user_id = event.chatbot_user_id
         self.other_user_nickname = event.conversation_title
-        
-        user_id = event.sender_id
-        nickname =event.sender_nick
+
+
+def download_image_file(image_url, temp_dir):
+    headers = {
+        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36'
+    }
+    # 设置代理
+    # self.proxies
+    # , proxies=self.proxies
+    response = requests.get(image_url, headers=headers, stream=True, timeout=60 * 5)
+    if response.status_code == 200:
+
+        # 生成文件名
+        file_name = image_url.split("/")[-1].split("?")[0]
+
+        # 检查临时目录是否存在，如果不存在则创建
+        if not os.path.exists(temp_dir):
+            os.makedirs(temp_dir)
+
+        # 将文件保存到临时目录
+        file_path = os.path.join(temp_dir, file_name)
+        with open(file_path, 'wb') as file:
+            file.write(response.content)
+        return file_path
+    else:
+        logger.info(f"[Dingtalk] Failed to download image file, {response.content}")
+        return None

channel/feishu/feishu_channel.py

@@ -40,7 +40,7 @@ class FeiShuChanel(ChatChannel):
             self.feishu_app_id, self.feishu_app_secret, self.feishu_token))
         # 无需群校验和前缀
         conf()["group_name_white_list"] = ["ALL_GROUP"]
-        conf()["single_chat_prefix"] = []
+        conf()["single_chat_prefix"] = [""]
 
     def startup(self):
         urls = (

channel/wechat/wechat_channel.py

@@ -109,7 +109,7 @@ class WechatChannel(ChatChannel):
 
     def __init__(self):
         super().__init__()
-        self.receivedMsgs = ExpiredDict(60 * 60)
+        self.receivedMsgs = ExpiredDict(conf().get("expires_in_seconds"))
         self.auto_login_times = 0
 
     def startup(self):

channel/wechatmp/wechatmp_channel.py

@@ -140,6 +140,42 @@ class WechatMPChannel(ChatChannel):
                 media_id = response["media_id"]
                 logger.info("[wechatmp] image uploaded, receiver {}, media_id {}".format(receiver, media_id))
                 self.cache_dict[receiver].append(("image", media_id))
+            elif reply.type == ReplyType.VIDEO_URL:  # 从网络下载视频
+                video_url = reply.content
+                video_res = requests.get(video_url, stream=True)
+                video_storage = io.BytesIO()
+                for block in video_res.iter_content(1024):
+                    video_storage.write(block)
+                video_storage.seek(0)
+                video_type = 'mp4'
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
+                content_type = "video/" + video_type
+                try:
+                    response = self.client.material.add("video", (filename, video_storage, content_type))
+                    logger.debug("[wechatmp] upload video response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload video failed: {}".format(e))
+                    return
+                media_id = response["media_id"]
+                logger.info("[wechatmp] video uploaded, receiver {}, media_id {}".format(receiver, media_id))
+                self.cache_dict[receiver].append(("video", media_id))
+
+            elif reply.type == ReplyType.VIDEO:  # 从文件读取视频
+                video_storage = reply.content
+                video_storage.seek(0)
+                video_type = 'mp4'
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
+                content_type = "video/" + video_type
+                try:
+                    response = self.client.material.add("video", (filename, video_storage, content_type))
+                    logger.debug("[wechatmp] upload video response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload video failed: {}".format(e))
+                    return
+                media_id = response["media_id"]
+                logger.info("[wechatmp] video uploaded, receiver {}, media_id {}".format(receiver, media_id))
+                self.cache_dict[receiver].append(("video", media_id))
+
         else:
             if reply.type == ReplyType.TEXT or reply.type == ReplyType.INFO or reply.type == ReplyType.ERROR:
                 reply_text = reply.content
@@ -222,6 +258,38 @@ class WechatMPChannel(ChatChannel):
                     return
                 self.client.message.send_image(receiver, response["media_id"])
                 logger.info("[wechatmp] Do send image to {}".format(receiver))
+            elif reply.type == ReplyType.VIDEO_URL:  # 从网络下载视频
+                video_url = reply.content
+                video_res = requests.get(video_url, stream=True)
+                video_storage = io.BytesIO()
+                for block in video_res.iter_content(1024):
+                    video_storage.write(block)
+                video_storage.seek(0)
+                video_type = 'mp4'
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
+                content_type = "video/" + video_type
+                try:
+                    response = self.client.media.upload("video", (filename, video_storage, content_type))
+                    logger.debug("[wechatmp] upload video response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload video failed: {}".format(e))
+                    return
+                self.client.message.send_video(receiver, response["media_id"])
+                logger.info("[wechatmp] Do send video to {}".format(receiver))
+            elif reply.type == ReplyType.VIDEO:  # 从文件读取视频
+                video_storage = reply.content
+                video_storage.seek(0)
+                video_type = 'mp4'
+                filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
+                content_type = "video/" + video_type
+                try:
+                    response = self.client.media.upload("video", (filename, video_storage, content_type))
+                    logger.debug("[wechatmp] upload video response: {}".format(response))
+                except WeChatClientException as e:
+                    logger.error("[wechatmp] upload video failed: {}".format(e))
+                    return
+                self.client.message.send_video(receiver, response["media_id"])
+                logger.info("[wechatmp] Do send video to {}".format(receiver))
         return
 
     def _success_callback(self, session_id, context, **kwargs):  # 线程异常结束时的回调函数

common/time_check.py

@@ -1,7 +1,5 @@
-import hashlib
 import re
 import time
-
 import config
 from common.log import logger
 
@@ -10,31 +8,33 @@ def time_checker(f):
     def _time_checker(self, *args, **kwargs):
         _config = config.conf()
         chat_time_module = _config.get("chat_time_module", False)
+
         if chat_time_module:
             chat_start_time = _config.get("chat_start_time", "00:00")
-            chat_stopt_time = _config.get("chat_stop_time", "24:00")
-            time_regex = re.compile(r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$")  # 时间匹配，包含24:00
+            chat_stop_time = _config.get("chat_stop_time", "24:00")
 
-            starttime_format_check = time_regex.match(chat_start_time)  # 检查停止时间格式
-            stoptime_format_check = time_regex.match(chat_stopt_time)  # 检查停止时间格式
-            chat_time_check = chat_start_time < chat_stopt_time  # 确定启动时间<停止时间
+            time_regex = re.compile(r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$")
 
-            # 时间格式检查
-            if not (starttime_format_check and stoptime_format_check and chat_time_check):
-                logger.warn("时间格式不正确,请在config.json中修改您的CHAT_START_TIME/CHAT_STOP_TIME,否则可能会影响您正常使用,开始({})-结束({})".format(starttime_format_check, stoptime_format_check))
-            if chat_start_time > "23:59":
-                logger.error("启动时间可能存在问题，请修改!")
-
-            # 服务时间检查
-            now_time = time.strftime("%H:%M", time.localtime())
-            if chat_start_time <= now_time <= chat_stopt_time:  # 服务时间内，正常返回回答
-                f(self, *args, **kwargs)
+            if not (time_regex.match(chat_start_time) and time_regex.match(chat_stop_time)):
+                logger.warning("时间格式不正确，请在config.json中修改CHAT_START_TIME/CHAT_STOP_TIME。")
                 return None
+
+            now_time = time.strptime(time.strftime("%H:%M"), "%H:%M")
+            chat_start_time = time.strptime(chat_start_time, "%H:%M")
+            chat_stop_time = time.strptime(chat_stop_time, "%H:%M")
+            # 结束时间小于开始时间，跨天了
+            if chat_stop_time < chat_start_time and (chat_start_time <= now_time or now_time <= chat_stop_time):
+                f(self, *args, **kwargs)
+            # 结束大于开始时间代表，没有跨天
+            elif chat_start_time < chat_stop_time and chat_start_time <= now_time <= chat_stop_time:
+                f(self, *args, **kwargs)
             else:
-                if args[0]["Content"] == "#更新配置":  # 不在服务时间内也可以更新配置
+                # 定义匹配规则，如果以 #reconf 或者  #更新配置  结尾, 非服务时间可以修改开始/结束时间并重载配置
+                pattern = re.compile(r"^.*#(?:reconf|更新配置)$")
+                if args and pattern.match(args[0].content):
                     f(self, *args, **kwargs)
                 else:
-                    logger.info("非服务时间内,不接受访问")
+                    logger.info("非服务时间内，不接受访问")
                     return None
         else:
             f(self, *args, **kwargs)  # 未开启时间模块则直接回答

config.py

@@ -4,6 +4,7 @@ import json
 import logging
 import os
 import pickle
+import copy
 
 from common.log import logger
 
@@ -36,6 +37,13 @@ available_setting = {
     "group_welcome_msg": "",  # 配置新人进群固定欢迎语，不配置则使用随机风格欢迎 
     "trigger_by_self": False,  # 是否允许机器人触发
     "text_to_image": "dall-e-2",  # 图片生成模型，可选 dall-e-2, dall-e-3
+    # Azure OpenAI dall-e-3 配置
+    "dalle3_image_style": "vivid", # 图片生成dalle3的风格，可选有 vivid, natural
+    "dalle3_image_quality": "hd", # 图片生成dalle3的质量，可选有 standard, hd
+    # Azure OpenAI DALL-E API 配置, 当use_azure_chatgpt为true时,用于将文字回复的资源和Dall-E的资源分开.
+    "azure_openai_dalle_api_base": "", # [可选] azure openai 用于回复图片的资源 endpoint，默认使用 open_ai_api_base
+    "azure_openai_dalle_api_key": "", # [可选] azure openai 用于回复图片的资源 key，默认使用 open_ai_api_key
+    "azure_openai_dalle_deployment_id":"", # [可选] azure openai 用于回复图片的资源 deployment id，默认使用 text_to_image
     "image_proxy": True,  # 是否需要图片代理，国内访问LinkAI时需要
     "image_create_prefix": ["画", "看", "找"],  # 开启图片回复的前缀
     "concurrency_in_session": 1,  # 同一会话最多有多少条消息在处理中，大于1可能乱序
@@ -95,7 +103,7 @@ available_setting = {
     "baidu_api_key": "",
     "baidu_secret_key": "",
     # 1536普通话(支持简单的英文识别) 1737英语 1637粤语 1837四川话 1936普通话远场
-    "baidu_dev_pid": "1536",
+    "baidu_dev_pid": 1536,
     # azure 语音api配置， 使用azure语音识别和语音合成时需要
     "azure_voice_api_key": "",
     "azure_voice_region": "japaneast",
@@ -139,12 +147,13 @@ available_setting = {
     
     # 钉钉配置
     "dingtalk_client_id": "",  # 钉钉机器人Client ID 
-    "dingtalk_client_secret": "",  # 钉钉机器人Client Secret 
+    "dingtalk_client_secret": "",  # 钉钉机器人Client Secret
+    "dingtalk_card_enabled": False,
     
     # chatgpt指令自定义触发词
     "clear_memory_commands": ["#清除记忆"],  # 重置会话指令，必须以#开头
     # channel配置
-    "channel_type": "wx",  # 通道类型，支持：{wx,wxy,terminal,wechatmp,wechatmp_service,wechatcom_app}
+    "channel_type": "",  # 通道类型，支持：{wx,wxy,terminal,wechatmp,wechatmp_service,wechatcom_app,dingtalk}
     "subscribe_msg": "",  # 订阅消息, 支持: wechatmp, wechatmp_service, wechatcom_app
     "debug": False,  # 是否开启debug模式，开启后会打印更多日志
     "appdata_dir": "",  # 数据目录
@@ -224,6 +233,30 @@ class Config(dict):
 config = Config()
 
 
+def drag_sensitive(config):
+    try:
+        if isinstance(config, str):
+            conf_dict: dict = json.loads(config)
+            conf_dict_copy = copy.deepcopy(conf_dict)
+            for key in conf_dict_copy:
+                if "key" in key or "secret" in key:
+                    if isinstance(key, str):
+                        conf_dict_copy[key] = conf_dict_copy[key][0:3] + "*" * 5 + conf_dict_copy[key][-3:]
+            return json.dumps(conf_dict_copy, indent=4)
+
+        elif isinstance(config, dict):
+            config_copy = copy.deepcopy(config)
+            for key in config:
+                if "key" in key or "secret" in key:
+                    if isinstance(key, str):
+                        config_copy[key] = config_copy[key][0:3] + "*" * 5 + config_copy[key][-3:]
+            return config_copy
+    except Exception as e:
+        logger.exception(e)
+        return config
+    return config
+
+
 def load_config():
     global config
     config_path = "./config.json"
@@ -232,7 +265,7 @@ def load_config():
         config_path = "./config-template.json"
 
     config_str = read_file(config_path)
-    logger.debug("[INIT] config str: {}".format(config_str))
+    logger.debug("[INIT] config str: {}".format(drag_sensitive(config_str)))
 
     # 将json字符串反序列化为dict类型
     config = Config(json.loads(config_str))
@@ -257,13 +290,11 @@ def load_config():
         logger.setLevel(logging.DEBUG)
         logger.debug("[INIT] set log level to DEBUG")
 
-    logger.info("[INIT] load config: {}".format(config))
+    logger.info("[INIT] load config: {}".format(drag_sensitive(config)))
 
     config.load_user_datas()
 
 
-
-
 def get_root():
     return os.path.dirname(os.path.abspath(__file__))
 

docs/version/old-version.md

@@ -8,6 +8,6 @@
 
 2023.03.25： 支持插件化开发，目前已实现 多角色切换、文字冒险游戏、管理员指令、Stable Diffusion等插件，使用参考 #578。(contributed by @lanvent in #565)
 
-2023.03.09： 基于 whisper API(后续已接入更多的语音API服务) 实现对微信语音消息的解析和回复，添加配置项 "speech_recognition":true 即可启用，使用参考 #415。(contributed by wanggang1987 in #385)
+2023.03.09： 基于 whisper API(后续已接入更多的语音API服务) 实现对语音消息的解析和回复，添加配置项 "speech_recognition":true 即可启用，使用参考 #415。(contributed by wanggang1987 in #385)
 
 2023.02.09： 扫码登录存在账号限制风险，请谨慎使用，参考#58

plugins/linkai/linkai.py

@@ -9,6 +9,7 @@ from common.expired_dict import ExpiredDict
 from common import const
 import os
 from .utils import Util
+from config import plugin_config
 
 
 @plugins.register(
@@ -69,7 +70,7 @@ class LinkAI(Plugin):
             return
 
         if (context.type == ContextType.SHARING and self._is_summary_open(context)) or \
-                (context.type == ContextType.TEXT and LinkSummary().check_url(context.content)):
+                (context.type == ContextType.TEXT and self._is_summary_open(context) and LinkSummary().check_url(context.content)):
             if not LinkSummary().check_url(context.content):
                 return
             _send_info(e_context, "正在为你加速生成摘要，请稍后")
@@ -196,7 +197,7 @@ class LinkAI(Plugin):
         if context.kwargs.get("isgroup") and not self.sum_config.get("group_enabled"):
             return False
         support_type = self.sum_config.get("type") or ["FILE", "SHARING"]
-        if context.type.name not in support_type:
+        if context.type.name not in support_type and context.type.name != "TEXT":
             return False
         return True
 
@@ -253,6 +254,7 @@ class LinkAI(Plugin):
                     plugin_conf = json.load(f)
                     plugin_conf["midjourney"]["enabled"] = False
                     plugin_conf["summary"]["enabled"] = False
+                    plugin_config["linkai"] = plugin_conf
                     return plugin_conf
         except Exception as e:
             logger.exception(e)

plugins/source.json

@@ -27,6 +27,10 @@
     "Blackroom": {
       "url": "https://github.com/dividduang/blackroom.git",
       "desc": "小黑屋插件，被拉进小黑屋的人将不能使用@bot的功能的插件"
+    },
+    "midjourney": {
+      "url": "https://github.com/baojingyu/midjourney.git",
+      "desc": "利用midjourney实现ai绘图的的插件"
     }
   }
 }

voice/baidu/baidu_voice.py

@@ -62,7 +62,7 @@ class BaiduVoice(Voice):
         # 识别本地文件
         logger.debug("[Baidu] voice file name={}".format(voice_file))
         pcm = get_pcm_from_wav(voice_file)
-        res = self.client.asr(pcm, "pcm", 8000, {"dev_pid": self.dev_id})
+        res = self.client.asr(pcm, "pcm", 16000, {"dev_pid": self.dev_id})
         if res["err_no"] == 0:
             logger.info("百度语音识别到了：{}".format(res["result"]))
             text = "".join(res["result"])

voice/factory.py

@@ -46,4 +46,8 @@ def create_voice(voice_type):
         from voice.edge.edge_voice import EdgeVoice
 
         return EdgeVoice()
+    elif voice_type == "xunfei":
+        from voice.xunfei.xunfei_voice import XunfeiVoice
+
+        return XunfeiVoice()
     raise RuntimeError

voice/xunfei/config.json.template

@@ -0,0 +1,7 @@
+{
+  "APPID":"xxx71xxx",
+  "APIKey":"xxxx69058exxxxxx",
+  "APISecret":"xxxx697f0xxxxxx",
+  "BusinessArgsTTS":{"aue": "lame", "sfl": 1, "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"},
+  "BusinessArgsASR":{"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vad_eos":10000, "dwa": "wpgs"}
+}

voice/xunfei/xunfei_asr.py

@@ -0,0 +1,209 @@
+# -*- coding:utf-8 -*-
+#
+#  Author: njnuko 
+#  Email: njnuko@163.com 
+#
+#  这个文档是基于官方的demo来改的，固体官方demo文档请参考官网
+#
+#  语音听写流式 WebAPI 接口调用示例 接口文档（必看）：https://doc.xfyun.cn/rest_api/语音听写（流式版）.html
+#  webapi 听写服务参考帖子（必看）：http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=38947&extra=
+#  语音听写流式WebAPI 服务，热词使用方式：登陆开放平台https://www.xfyun.cn/后，找到控制台--我的应用---语音听写（流式）---服务管理--个性化热词，
+#  设置热词
+#  注意：热词只能在识别的时候会增加热词的识别权重，需要注意的是增加相应词条的识别率，但并不是绝对的，具体效果以您测试为准。
+#  语音听写流式WebAPI 服务，方言试用方法：登陆开放平台https://www.xfyun.cn/后，找到控制台--我的应用---语音听写（流式）---服务管理--识别语种列表
+#  可添加语种或方言，添加后会显示该方言的参数值
+#  错误码链接：https://www.xfyun.cn/document/error-code （code返回错误码时必看）
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+
+import websocket
+import datetime
+import hashlib
+import base64
+import hmac
+import json
+from urllib.parse import urlencode
+import time
+import ssl
+from wsgiref.handlers import format_date_time
+from datetime import datetime
+from time import mktime
+import _thread as thread
+import os
+import wave
+
+
+STATUS_FIRST_FRAME = 0  # 第一帧的标识
+STATUS_CONTINUE_FRAME = 1  # 中间帧标识
+STATUS_LAST_FRAME = 2  # 最后一帧的标识
+
+#############
+#whole_dict 是用来存储返回值的，由于带语音修正，所以用dict来存储，有更新的化pop之前的值，最后再合并
+global whole_dict
+#这个文档是官方文档改的，这个参数是用来做函数调用时用的
+global wsParam
+##############
+
+
+class Ws_Param(object):
+    # 初始化
+    def __init__(self, APPID, APIKey, APISecret,BusinessArgs, AudioFile):
+        self.APPID = APPID
+        self.APIKey = APIKey
+        self.APISecret = APISecret
+        self.AudioFile = AudioFile
+        self.BusinessArgs = BusinessArgs
+        # 公共参数(common)
+        self.CommonArgs = {"app_id": self.APPID}
+        # 业务参数(business)，更多个性化参数可在官网查看
+        #self.BusinessArgs = {"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vinfo":1,"vad_eos":10000}
+
+    # 生成url
+    def create_url(self):
+        url = 'wss://ws-api.xfyun.cn/v2/iat'
+        # 生成RFC1123格式的时间戳
+        now = datetime.now()
+        date = format_date_time(mktime(now.timetuple()))
+
+        # 拼接字符串
+        signature_origin = "host: " + "ws-api.xfyun.cn" + "\n"
+        signature_origin += "date: " + date + "\n"
+        signature_origin += "GET " + "/v2/iat " + "HTTP/1.1"
+        # 进行hmac-sha256进行加密
+        signature_sha = hmac.new(self.APISecret.encode('utf-8'), signature_origin.encode('utf-8'),
+                                 digestmod=hashlib.sha256).digest()
+        signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8')
+
+        authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % (
+            self.APIKey, "hmac-sha256", "host date request-line", signature_sha)
+        authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8')
+        # 将请求的鉴权参数组合为字典
+        v = {
+            "authorization": authorization,
+            "date": date,
+            "host": "ws-api.xfyun.cn"
+        }
+        # 拼接鉴权参数，生成url
+        url = url + '?' + urlencode(v)
+        #print("date: ",date)
+        #print("v: ",v)
+        # 此处打印出建立连接时候的url,参考本demo的时候可取消上方打印的注释，比对相同参数时生成的url与自己代码生成的url是否一致
+        #print('websocket url :', url)
+        return url
+
+
+# 收到websocket消息的处理
+def on_message(ws, message):
+    global whole_dict
+    try:
+        code = json.loads(message)["code"]
+        sid = json.loads(message)["sid"]
+        if code != 0:
+            errMsg = json.loads(message)["message"]
+            print("sid:%s call error:%s code is:%s" % (sid, errMsg, code))
+        else:
+            temp1 = json.loads(message)["data"]["result"]
+            data = json.loads(message)["data"]["result"]["ws"]
+            sn = temp1["sn"]
+            if "rg" in temp1.keys():
+                rep = temp1["rg"]
+                rep_start = rep[0]
+                rep_end = rep[1]
+                for sn in range(rep_start,rep_end+1):
+                    #print("before pop",whole_dict)
+                    #print("sn",sn)
+                    whole_dict.pop(sn,None)
+                    #print("after pop",whole_dict)
+                results = ""
+                for i in data:
+                    for w in i["cw"]:
+                        results += w["w"]
+                whole_dict[sn]=results
+                #print("after add",whole_dict)
+            else:
+                results = ""
+                for i in data:
+                    for w in i["cw"]:
+                        results += w["w"]
+                whole_dict[sn]=results
+            #print("sid:%s call success!,data is:%s" % (sid, json.dumps(data, ensure_ascii=False)))
+    except Exception as e:
+        print("receive msg,but parse exception:", e)
+
+
+
+# 收到websocket错误的处理
+def on_error(ws, error):
+    print("### error:", error)
+
+
+# 收到websocket关闭的处理
+def on_close(ws,a,b):
+    print("### closed ###")
+
+
+# 收到websocket连接建立的处理
+def on_open(ws):
+    global wsParam
+    def run(*args):
+        frameSize = 8000  # 每一帧的音频大小
+        intervel = 0.04  # 发送音频间隔(单位:s)
+        status = STATUS_FIRST_FRAME  # 音频的状态信息，标识音频是第一帧，还是中间帧、最后一帧
+
+        with wave.open(wsParam.AudioFile, "rb") as fp:
+            while True:
+                buf = fp.readframes(frameSize)
+                # 文件结束
+                if not buf:
+                    status = STATUS_LAST_FRAME
+                # 第一帧处理
+                # 发送第一帧音频，带business 参数
+                # appid 必须带上，只需第一帧发送
+                if status == STATUS_FIRST_FRAME:
+                    d = {"common": wsParam.CommonArgs,
+                         "business": wsParam.BusinessArgs,
+                         "data": {"status": 0, "format": "audio/L16;rate=16000","audio": str(base64.b64encode(buf), 'utf-8'), "encoding": "raw"}} 
+                    d = json.dumps(d)
+                    ws.send(d)
+                    status = STATUS_CONTINUE_FRAME
+                # 中间帧处理
+                elif status == STATUS_CONTINUE_FRAME:
+                    d = {"data": {"status": 1, "format": "audio/L16;rate=16000",
+                                  "audio": str(base64.b64encode(buf), 'utf-8'),
+                                  "encoding": "raw"}}
+                    ws.send(json.dumps(d))
+                # 最后一帧处理
+                elif status == STATUS_LAST_FRAME:
+                    d = {"data": {"status": 2, "format": "audio/L16;rate=16000",
+                                  "audio": str(base64.b64encode(buf), 'utf-8'),
+                                  "encoding": "raw"}}
+                    ws.send(json.dumps(d))
+                    time.sleep(1)
+                    break
+                # 模拟音频采样间隔
+                time.sleep(intervel)
+        ws.close()
+
+    thread.start_new_thread(run, ())
+
+#提供给xunfei_voice调用的函数
+def xunfei_asr(APPID,APISecret,APIKey,BusinessArgsASR,AudioFile):
+    global whole_dict
+    global wsParam
+    whole_dict = {}
+    wsParam1 = Ws_Param(APPID=APPID, APISecret=APISecret,
+                       APIKey=APIKey,BusinessArgs=BusinessArgsASR,
+                       AudioFile=AudioFile)
+    #wsParam是global变量，给上面on_open函数调用使用的
+    wsParam = wsParam1
+    websocket.enableTrace(False)
+    wsUrl = wsParam.create_url()
+    ws = websocket.WebSocketApp(wsUrl, on_message=on_message, on_error=on_error, on_close=on_close)
+    ws.on_open = on_open
+    ws.run_forever(sslopt={"cert_reqs": ssl.CERT_NONE})
+    #把字典的值合并起来做最后识别的输出
+    whole_words = ""
+    for i in sorted(whole_dict.keys()):
+        whole_words += whole_dict[i]
+    return whole_words
+
+     

voice/xunfei/xunfei_tts.py

@@ -0,0 +1,163 @@
+# -*- coding:utf-8 -*-
+#
+#  Author: njnuko
+#  Email: njnuko@163.com
+#
+#  这个文档是基于官方的demo来改的，固体官方demo文档请参考官网
+#
+#  语音听写流式 WebAPI 接口调用示例 接口文档（必看）：https://doc.xfyun.cn/rest_api/语音听写（流式版）.html
+#  webapi 听写服务参考帖子（必看）：http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=38947&extra=
+#  语音听写流式WebAPI 服务，热词使用方式：登陆开放平台https://www.xfyun.cn/后，找到控制台--我的应用---语音听写（流式）---服务管理--个性化热词，
+#  设置热词
+#  注意：热词只能在识别的时候会增加热词的识别权重，需要注意的是增加相应词条的识别率，但并不是绝对的，具体效果以您测试为准。
+#  语音听写流式WebAPI 服务，方言试用方法：登陆开放平台https://www.xfyun.cn/后，找到控制台--我的应用---语音听写（流式）---服务管理--识别语种列表
+#  可添加语种或方言，添加后会显示该方言的参数值
+#  错误码链接：https://www.xfyun.cn/document/error-code （code返回错误码时必看）
+# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
+import websocket
+import datetime
+import hashlib
+import base64
+import hmac
+import json
+from urllib.parse import urlencode
+import time
+import ssl
+from wsgiref.handlers import format_date_time
+from datetime import datetime
+from time import mktime
+import _thread as thread
+import os
+
+
+
+STATUS_FIRST_FRAME = 0  # 第一帧的标识
+STATUS_CONTINUE_FRAME = 1  # 中间帧标识
+STATUS_LAST_FRAME = 2  # 最后一帧的标识
+
+#############
+#这个参数是用来做输出文件路径的
+global outfile
+#这个文档是官方文档改的，这个参数是用来做函数调用时用的
+global wsParam
+##############
+
+
+class Ws_Param(object):
+    # 初始化
+    def __init__(self, APPID, APIKey, APISecret,BusinessArgs,Text):
+        self.APPID = APPID
+        self.APIKey = APIKey
+        self.APISecret = APISecret
+        self.BusinessArgs = BusinessArgs
+        self.Text = Text
+
+        # 公共参数(common)
+        self.CommonArgs = {"app_id": self.APPID}
+        # 业务参数(business)，更多个性化参数可在官网查看
+        #self.BusinessArgs = {"aue": "raw", "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"}
+        self.Data = {"status": 2, "text": str(base64.b64encode(self.Text.encode('utf-8')), "UTF8")}
+        #使用小语种须使用以下方式，此处的unicode指的是 utf16小端的编码方式，即"UTF-16LE"”
+        #self.Data = {"status": 2, "text": str(base64.b64encode(self.Text.encode('utf-16')), "UTF8")}
+
+    # 生成url
+    def create_url(self):
+        url = 'wss://tts-api.xfyun.cn/v2/tts'
+        # 生成RFC1123格式的时间戳
+        now = datetime.now()
+        date = format_date_time(mktime(now.timetuple()))
+
+        # 拼接字符串
+        signature_origin = "host: " + "ws-api.xfyun.cn" + "\n"
+        signature_origin += "date: " + date + "\n"
+        signature_origin += "GET " + "/v2/tts " + "HTTP/1.1"
+        # 进行hmac-sha256进行加密
+        signature_sha = hmac.new(self.APISecret.encode('utf-8'), signature_origin.encode('utf-8'),
+                                 digestmod=hashlib.sha256).digest()
+        signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8')
+
+        authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % (
+            self.APIKey, "hmac-sha256", "host date request-line", signature_sha)
+        authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8')
+        # 将请求的鉴权参数组合为字典
+        v = {
+            "authorization": authorization,
+            "date": date,
+            "host": "ws-api.xfyun.cn"
+        }
+        # 拼接鉴权参数，生成url
+        url = url + '?' + urlencode(v)
+        # print("date: ",date)
+        # print("v: ",v)
+        # 此处打印出建立连接时候的url,参考本demo的时候可取消上方打印的注释，比对相同参数时生成的url与自己代码生成的url是否一致
+        # print('websocket url :', url)
+        return url
+
+def on_message(ws, message):
+    #输出文件
+    global outfile
+    try:
+        message =json.loads(message)
+        code = message["code"]
+        sid = message["sid"]
+        audio = message["data"]["audio"]
+        audio = base64.b64decode(audio)
+        status = message["data"]["status"]
+        if status == 2:
+            print("ws is closed")
+            ws.close()
+        if code != 0:
+            errMsg = message["message"]
+            print("sid:%s call error:%s code is:%s" % (sid, errMsg, code))
+        else:
+
+            with open(outfile, 'ab') as f:
+                f.write(audio)
+
+    except Exception as e:
+        print("receive msg,but parse exception:", e)
+
+
+
+# 收到websocket连接建立的处理
+def on_open(ws):
+    global outfile
+    global wsParam
+    def run(*args):
+        d = {"common": wsParam.CommonArgs,
+             "business": wsParam.BusinessArgs,
+             "data": wsParam.Data,
+             }
+        d = json.dumps(d)
+        # print("------>开始发送文本数据")
+        ws.send(d)
+        if os.path.exists(outfile):
+            os.remove(outfile)
+
+    thread.start_new_thread(run, ())
+
+# 收到websocket错误的处理
+def on_error(ws, error):
+    print("### error:", error)
+
+
+
+# 收到websocket关闭的处理
+def on_close(ws):
+    print("### closed ###")
+
+
+
+def xunfei_tts(APPID, APIKey, APISecret,BusinessArgsTTS, Text, OutFile):
+    global outfile
+    global wsParam 
+    outfile = OutFile
+    wsParam1 = Ws_Param(APPID,APIKey,APISecret,BusinessArgsTTS,Text)
+    wsParam = wsParam1
+    websocket.enableTrace(False)
+    wsUrl = wsParam.create_url()
+    ws = websocket.WebSocketApp(wsUrl, on_message=on_message, on_error=on_error, on_close=on_close)
+    ws.on_open = on_open
+    ws.run_forever(sslopt={"cert_reqs": ssl.CERT_NONE})
+    return outfile
+     

voice/xunfei/xunfei_voice.py

@@ -0,0 +1,86 @@
+#####################################################################
+#    xunfei voice service
+#     Auth: njnuko
+#     Email: njnuko@163.com
+#
+#    要使用本模块, 首先到 xfyun.cn 注册一个开发者账号,
+#    之后创建一个新应用, 然后在应用管理的语音识别或者语音合同右边可以查看APPID API Key 和 Secret Key
+#    然后在 config.json 中填入这三个值
+#
+#    配置说明：
+# {
+#  "APPID":"xxx71xxx",
+#  "APIKey":"xxxx69058exxxxxx",  #讯飞xfyun.cn控制台语音合成或者听写界面的APIKey
+#  "APISecret":"xxxx697f0xxxxxx",  #讯飞xfyun.cn控制台语音合成或者听写界面的APIKey
+#  "BusinessArgsTTS":{"aue": "lame", "sfl": 1, "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"}, #语音合成的参数，具体可以参考xfyun.cn的文档
+#  "BusinessArgsASR":{"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vad_eos":10000, "dwa": "wpgs"}  #语音听写的参数，具体可以参考xfyun.cn的文档
+# }
+#####################################################################
+
+import json
+import os
+import time
+
+from bridge.reply import Reply, ReplyType
+from common.log import logger
+from common.tmp_dir import TmpDir
+from config import conf
+from voice.voice import Voice
+from .xunfei_asr import xunfei_asr
+from .xunfei_tts import xunfei_tts
+from voice.audio_convert import any_to_mp3
+import shutil
+from pydub import AudioSegment
+
+
+class XunfeiVoice(Voice):
+    def __init__(self):
+        try:
+            curdir = os.path.dirname(__file__)
+            config_path = os.path.join(curdir, "config.json")
+            conf = None
+            with open(config_path, "r") as fr:
+                conf = json.load(fr)
+            print(conf)
+            self.APPID = str(conf.get("APPID"))
+            self.APIKey = str(conf.get("APIKey"))
+            self.APISecret = str(conf.get("APISecret"))
+            self.BusinessArgsTTS = conf.get("BusinessArgsTTS")
+            self.BusinessArgsASR= conf.get("BusinessArgsASR")
+
+        except Exception as e:
+            logger.warn("XunfeiVoice init failed: %s, ignore " % e)
+
+    def voiceToText(self, voice_file):
+        # 识别本地文件
+        try:
+            logger.debug("[Xunfei] voice file name={}".format(voice_file))
+            #print("voice_file===========",voice_file)
+            #print("voice_file_type===========",type(voice_file))
+            #mp3_name, file_extension = os.path.splitext(voice_file)
+            #mp3_file = mp3_name + ".mp3"
+            #pcm_data=get_pcm_from_wav(voice_file)
+            #mp3_name, file_extension = os.path.splitext(voice_file)
+            #AudioSegment.from_wav(voice_file).export(mp3_file, format="mp3")
+            #shutil.copy2(voice_file, 'tmp/test1.wav')
+            #shutil.copy2(mp3_file, 'tmp/test1.mp3')
+            #print("voice and mp3 file",voice_file,mp3_file)
+            text = xunfei_asr(self.APPID,self.APISecret,self.APIKey,self.BusinessArgsASR,voice_file)
+            logger.info("讯飞语音识别到了: {}".format(text))
+            reply = Reply(ReplyType.TEXT, text)
+        except Exception as e:
+            logger.warn("XunfeiVoice init failed: %s, ignore " % e)
+            reply = Reply(ReplyType.ERROR, "讯飞语音识别出错了；{0}")
+        return reply
+
+    def textToVoice(self, text):
+        try:
+            # Avoid the same filename under multithreading
+            fileName = TmpDir().path() + "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".mp3"
+            return_file = xunfei_tts(self.APPID,self.APIKey,self.APISecret,self.BusinessArgsTTS,text,fileName)
+            logger.info("[Xunfei] textToVoice text={} voice file name={}".format(text, fileName))
+            reply = Reply(ReplyType.VOICE, fileName)
+        except Exception as e:
+            logger.error("[Xunfei] textToVoice error={}".format(fileName))
+            reply = Reply(ReplyType.ERROR, "抱歉，讯飞语音合成失败")
+        return reply