Compare commits

...

56 Commits
1.6.0 ... 1.6.6

Author SHA1 Message Date
zhayujie
405372d1a7 Merge pull request #1753 from MasterKeee/master
新增公众号的回复视频类型
2024-06-04 14:25:11 +08:00
Saboteur7
b8c5174da5 docs: xunfei voice comment 2024-06-04 13:49:44 +08:00
Saboteur7
1f6f9103d9 docs: update README.md 2024-06-04 12:50:59 +08:00
Saboteur7
6431487c7a fix: drag sensitive bug 2024-06-04 12:02:23 +08:00
Saboteur7
8b2d1189db Merge pull request #1999 from njnuko/voice-xunfei
add xunfei voice
2024-06-04 11:43:55 +08:00
Saboteur7
b777f27cb7 chore: remove some xunfei voice log 2024-06-04 11:42:05 +08:00
Saboteur7
b31c3b124a Merge pull request #1972 from Undertone0809/zeeland/add-logger-drag-sensitive
feat: add logger drag sensitive
2024-06-04 11:26:05 +08:00
Saboteur7
fa1e965fba feat: add dingtalk card switch 2024-06-04 11:23:45 +08:00
Saboteur7
91dc8b4d58 Merge pull request #1994 from baojingyu/feat-05-17
钉钉接入增加流式输出支持,语音、图片或富文本消息接收
2024-06-04 10:53:02 +08:00
Saboteur7
6d16ea8830 Update requirements.txt 2024-06-04 10:49:17 +08:00
Saboteur7
7db4253264 Update chat_channel.py 2024-06-04 10:47:56 +08:00
Saboteur7
4d2b7d9bf9 Update chat_channel.py 2024-06-04 10:47:05 +08:00
Saboteur7
8f6f4acb88 Update chat_channel.py 2024-06-04 10:43:19 +08:00
Saboteur7
f20d84cb37 Merge pull request #1809 from whw23/master
Azure OpenAI Dalle fix
2024-06-03 22:46:07 +08:00
Saboteur7
afbdf1d5d5 Merge pull request #2002 from 6vision/time_check
fix: time_check model
2024-06-03 22:40:01 +08:00
Haowei
bc8364d594 Merge branch 'zhayujie:master' into master 2024-05-25 23:34:47 +08:00
vision
c8d388f70f Merge pull request #2013 from 6vision/fix_baidu_voice
Changed sampling rate
2024-05-23 01:36:00 +08:00
6vision
be13cc3194 Changed sampling rate 2024-05-23 01:34:20 +08:00
vision
a46320e744 Merge pull request #2012 from 6vision/fix_issue_1959_
Fix issue 1959 wenxin模型返回报错
2024-05-22 21:45:20 +08:00
6vision
071709d263 fix: 1959-百度文心偶发报错336006 2024-05-22 16:01:46 +08:00
6vision
93a32ae5ff 修复模型请求异常时的bug 2024-05-22 15:57:22 +08:00
vision
eee96f226f Merge pull request #2005 from 6vision/fix_baidu_voice
fix: baidu voice bug
2024-05-21 22:38:54 +08:00
6vision
e19a8b479c fix: baidu voice bug 2024-05-21 22:32:35 +08:00
6vision
9ef459112e fix: time_check model 2024-05-20 20:37:00 +08:00
Haowei
e96474bd5c Merge branch 'zhayujie:master' into master 2024-05-20 16:53:02 +08:00
njnuko
6fed719e09 add Xunfei Voice
Signed-off-by: njnuko <njnuko@163.com>
2024-05-20 15:04:23 +08:00
zhayujie
99aac76618 docs: update readme 2024-05-18 19:03:17 +08:00
baojingyu
599f458201 Update plugins source.js add midjourney实现ai绘图的的插件 2024-05-17 15:38:19 +08:00
baojingyu
2f8099059c 修复chat_channel配置参数取值错误bug,优化dingtalk_channel回复打字机效果流式 AI卡片、dingtalk_message图片或富文本消息接收 2024-05-17 14:48:52 +08:00
zhayujie
e24f177832 Merge pull request #1993 from 6vision/fix_linkai_pconf
fix: linkai plugin config_template
2024-05-17 01:25:30 +08:00
6vision
48cc143e88 fix: linkai plugin config_template 2024-05-17 01:22:38 +08:00
zhayujie
b09b46c045 fix: summary switch bug 2024-05-14 17:48:18 +08:00
zhayujie
2c6583cc9c fix: summary switch bug 2024-05-14 17:26:10 +08:00
zhayujie
e381d1bfb8 feat: support gpt-4o model 2024-05-14 09:50:03 +08:00
zeeland
eac619d54f feat: add logger drag sensitive 2024-05-13 19:53:33 +08:00
zhayujie
a6ef3bc0ce fix: add channel login exception log 2024-05-08 12:54:13 +08:00
zhayujie
118122c541 docs: update README.md 2024-05-08 12:07:59 +08:00
zhayujie
bfdf33ac09 Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2024-05-07 11:37:53 +08:00
zhayujie
fa3370df5b fix: image model check 2024-05-07 11:37:27 +08:00
zhayujie
f1e51672c5 Merge pull request #1944 from alvinsuDL/patch-1
Update README.md
2024-05-07 11:20:43 +08:00
alvinsuDL
91f97b2728 Update README.md 2024-05-07 11:16:41 +08:00
zhayujie
2c542e03fe Merge branch 'master' of github.com:zhayujie/chatgpt-on-wechat 2024-05-07 11:10:41 +08:00
zhayujie
71a11b4267 feat: support mj client config 2024-05-07 11:09:49 +08:00
zhayujie
ea642757db docs: update README.md 2024-05-06 22:19:49 +08:00
zhayujie
fb72b601aa fix: model config 2024-05-03 19:41:12 +08:00
zhayujie
27e507e744 fix: update client sdk version 2024-05-03 19:10:27 +08:00
zhayujie
4db19f816f feat: update service url 2024-05-03 14:10:07 +08:00
Haowei
d40e915e2b Merge branch 'zhayujie:master' into master 2024-04-09 11:31:57 +08:00
Haowei
44fdadda08 Merge branch 'zhayujie:master' into master 2024-04-07 14:54:48 +08:00
unknown
dd95f849d4 Merge branch 'master' of https://github.com/whw23/chatgpt-on-wechat 2024-03-30 01:08:07 +08:00
unknown
22c7f8fe9e add dall-e-2 retry_count limit 2024-03-30 01:07:52 +08:00
Haowei
3d47be1f49 Merge branch 'zhayujie:master' into master 2024-03-30 00:54:38 +08:00
Haowei
748c53c774 Merge branch 'zhayujie:master' into master 2024-03-23 21:13:36 +08:00
Haowei
d0fd36e7e1 Merge branch 'zhayujie:master' into master 2024-03-20 15:31:31 +08:00
unknown
9d394adf24 1.修复Azure Openai Dalle请求 2.增加Azure Openai Dalle3 请求参数 3.将用于回复文字和回复Dalle3的Azure Openai资源分离开 2024-03-12 08:32:24 +08:00
MasterKeee
a0427b569e 新增公众号的回复视频类型 2024-02-19 00:45:53 +08:00
35 changed files with 991 additions and 202 deletions

114
README.md
View File

@@ -1,39 +1,48 @@
# 简介
> 项目是基于大模型的智能对话机器人,支持企业微信、微信公众号、飞书、钉钉接入可选择GPT3.5/GPT4.0/Claude/文心一言/讯飞星火/通义千问/Gemini/LinkAI/ChatGLM/KIMI能处理文本、语音和图片通过插件访问操作系统和互联网等外部资源支持基于自有知识库定制企业AI应用。
> chatgpt-on-wechat简称CoW项目是基于大模型的智能对话机器人,支持微信公众号、企业微信应用、飞书、钉钉接入可选择GPT3.5/GPT4.0/Claude/Gemini/LinkAI/ChatGLM/KIMI/文心一言/讯飞星火/通义千问/LinkAI能处理文本、语音和图片通过插件访问操作系统和互联网等外部资源支持基于自有知识库定制企业AI应用。
最新版本支持的功能如下:
- [x] **多端部署:** 有多种部署方式可选择且功能完备,目前已支持微信生态下公众号、企业微信应用、飞书、钉钉等部署方式
- [x] **基础对话:** 私聊及群聊的消息智能回复,支持多轮会话上下文记忆,支持 GPT-3.5, GPT-4, Claude-3, Gemini, 文心一言, 讯飞星火, 通义千问ChatGLM-4Kimi(月之暗面)
- [x] **语音能力:** 可识别语音消息,通过文字或语音回复,支持 azure, baidu, google, openai(whisper/tts) 等多种语音模型
- [x] **图像能力:** 支持图片生成、图片识别、图生图(如照片修复),可选择 Dall-E-3, stable diffusion, replicate, midjourney, CogView-3, vision模型
- [x] **丰富插件:** 支持个性化插件扩展,已实现多角色切换、文字冒险、敏感词过滤、聊天记录总结、文档总结和对话、联网搜索等插件
- [x] **知识库:** 通过上传知识库文件自定义专属机器人,可作为数字分身、智能客服、私域助手使用,基于 [LinkAI](https://link-ai.tech) 实现
- **多端部署:** 有多种部署方式可选择且功能完备,目前已支持微信公众号、企业微信应用、飞书、钉钉等部署方式
- **基础对话:** 私聊及群聊的消息智能回复,支持多轮会话上下文记忆,支持 GPT-3.5, GPT-4, GPT-4o, Claude-3, Gemini, 文心一言, 讯飞星火, 通义千问ChatGLM-4Kimi(月之暗面)
- **语音能力:** 可识别语音消息,通过文字或语音回复,支持 azure, baidu, google, openai(whisper/tts) 等多种语音模型
- **图像能力:** 支持图片生成、图片识别、图生图(如照片修复),可选择 Dall-E-3, stable diffusion, replicate, midjourney, CogView-3, vision模型
- **丰富插件:** 支持个性化插件扩展,已实现多角色切换、文字冒险、敏感词过滤、聊天记录总结、文档总结和对话、联网搜索等插件
- **知识库:** 通过上传知识库文件自定义专属机器人,可作为数字分身、智能客服、私域助手使用,基于 [LinkAI](https://link-ai.tech) 实现
# 演示
## 声明
https://github.com/zhayujie/chatgpt-on-wechat/assets/26161723/d5154020-36e3-41db-8706-40ce9f3f1b1e
1. 本项目遵循 [MIT开源协议](/LICENSE),仅用于技术研究和学习,使用本项目时需遵守所在地法律法规、相关政策以及企业章程,禁止用于任何违法或侵犯他人权益的行为
2. 境内使用该项目时,请使用国内厂商的大模型服务,并进行必要的内容安全审核及过滤
3. 本项目主要接入协同办公平台,推荐使用公众号、企微自建应用、钉钉、飞书等接入通道,其他通道为历史产物已不维护
4. 任何个人、团队和企业,无论以何种方式使用该项目、对何对象提供服务,所产生的一切后果,本项目均不承担任何责任
Demo made by [Visionn](https://www.wangpc.cc/)
# 商业支持
> 我们还提供企业级的 **AI应用平台**包含知识库、Agent插件、应用管理等能力支持多平台聚合的应用接入、客户端管理、对话管理以及提供
SaaS服务、私有化部署、稳定托管接入 等多种模式。
>
> 目前已在私域运营、智能客服、企业效率助手等场景积累了丰富的 AI 解决方案, 在电商、文教、健康、新消费等各行业沉淀了 AI 落地的最佳实践,致力于打造助力中小企业拥抱 AI 的一站式平台。
企业服务和商用咨询可联系产品顾问:
<img width="240" src="https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/product-manager-qrcode.jpg">
# 开源社区
## 社区
添加小助手微信加入开源项目交流群:
<img width="240" src="./docs/images/contact.jpg">
<img width="160" src="https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/open-community.png">
# 更新日志
<br>
# 企业服务
<a href="https://link-ai.tech" target="_blank"><img width="800" src="https://cdn.link-ai.tech/image/link-ai-intro.jpg"></a>
> [LinkAI](https://link-ai.tech/) 是面向企业和开发者的一站式AI应用平台聚合多模态大模型、知识库、Agent 插件、工作流等能力支持一键接入主流平台并进行管理支持SaaS、私有化部署多种模式。
>
> LinkAI 目前 已在私域运营、智能客服、企业效率助手等场景积累了丰富的 AI 解决方案, 在电商、文教、健康、新消费、科技制造等各行业沉淀了大模型落地应用的最佳实践,致力于帮助更多企业和开发者拥抱 AI 生产力。
**企业服务和产品咨询** 可联系产品顾问:
<img width="160" src="https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/github-product-consult.png">
<br>
# 🏷 更新日志
>**2024.06.04** [1.6.6版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.6) 和 [1.6.5版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.5)gpt-4o模型、钉钉流式卡片、讯飞语音识别/合成
>**2024.04.26** [1.6.0版本](https://github.com/zhayujie/chatgpt-on-wechat/releases/tag/1.6.0),新增 Kimi 接入、gpt-4-turbo版本升级、文件总结和语音识别问题修复
@@ -57,11 +66,13 @@ SaaS服务、私有化部署、稳定托管接入 等多种模式。
更早更新日志查看: [归档日志](/docs/version/old-version.md)
# 快速开始
<br>
快速开始文档:[项目搭建文档](https://docs.link-ai.tech/cow/quick-start)
# 🚀 快速开始
## 准备
快速开始详细文档:[项目搭建文档](https://docs.link-ai.tech/cow/quick-start)
## 一、准备
### 1. 账号注册
@@ -100,7 +111,7 @@ pip3 install -r requirements-optional.txt
```
> 如果某项依赖安装失败可注释掉对应的行再继续
## 配置
## 二、配置
配置文件的模板在根目录的`config-template.json`中,需复制该模板创建最终生效的 `config.json` 文件:
@@ -108,14 +119,13 @@ pip3 install -r requirements-optional.txt
cp config-template.json config.json
```
然后在`config.json`中填入配置,以下是对默认配置的说明,可根据需要进行自定义修改(请去掉注释
然后在`config.json`中填入配置,以下是对默认配置的说明,可根据需要进行自定义修改(注意实际使用时请去掉注释保证JSON格式的完整
```bash
# config.json文件内容示例
{
"open_ai_api_key": "YOUR API KEY", # 填入上面创建的 OpenAI API KEY
"model": "gpt-3.5-turbo", # 模型名称, 支持 gpt-3.5-turbo, gpt-3.5-turbo-16k, gpt-4, wenxin, xunfei, claude-3-opus-20240229
"claude_api_key":"YOUR API KEY" # 如果选用claude3模型的话配置这个key同时如想使用生图语音等功能仍需配置open_ai_api_key
"model": "gpt-3.5-turbo", # 模型名称, 支持 gpt-3.5-turbo, gpt-4, gpt-4-turbo, wenxin, xunfei, glm-4, claude-3-haiku, moonshot
"open_ai_api_key": "YOUR API KEY", # 如果使用openAI模型则填入上面创建的 OpenAI API KEY
"proxy": "", # 代理客户端的ip和端口国内环境开启代理的需要填写该项如 "127.0.0.1:7890"
"single_chat_prefix": ["bot", "@bot"], # 私聊时文本需要包含该前缀才能触发机器人回复
"single_chat_reply_prefix": "[bot] ", # 私聊时自动回复的前缀,用于区分真人
@@ -126,15 +136,13 @@ pip3 install -r requirements-optional.txt
"conversation_max_tokens": 1000, # 支持上下文记忆的最多字符数
"speech_recognition": false, # 是否开启语音识别
"group_speech_recognition": false, # 是否开启群组语音识别
"use_azure_chatgpt": false, # 是否使用Azure ChatGPT service代替openai ChatGPT service. 当设置为true时需要设置 open_ai_api_base如 https://xxx.openai.azure.com/
"azure_deployment_id": "", # 采用Azure ChatGPT时模型部署名称
"azure_api_version": "", # 采用Azure ChatGPT时API版本
"character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。", # 人格描述
"voice_reply_voice": false, # 是否使用语音回复语音
"character_desc": "你是基于大语言模型的AI智能助手旨在回答并解决人们的任何问题并且可以使用多种语言与人交流。", # 人格描述
# 订阅消息公众号和企业微信channel中请填写当被订阅时会自动回复可使用特殊占位符。目前支持的占位符有{trigger_prefix}在程序中它会自动替换成bot的触发词。
"subscribe_msg": "感谢您的关注!\n这里是ChatGPT可以自由对话。\n支持语音对话。\n支持图片输出画字开头的消息将按要求创作图片。\n支持角色扮演和文字冒险等丰富插件。\n输入{trigger_prefix}#help 查看详细指令。",
"use_linkai": false, # 是否使用LinkAI接口默认关闭开启后可国内访问使用知识库和MJ
"linkai_api_key": "", # LinkAI Api Key
"linkai_app_code": "" # LinkAI 应用code
"linkai_app_code": "" # LinkAI 应用或工作流code
}
```
**配置说明:**
@@ -155,11 +163,11 @@ pip3 install -r requirements-optional.txt
+ 添加 `"speech_recognition": true` 将开启语音识别默认使用openai的whisper模型识别为文字同时以文字回复该参数仅支持私聊 (注意由于语音消息无法匹配前缀,一旦开启将对所有语音自动回复,支持语音触发画图)
+ 添加 `"group_speech_recognition": true` 将开启群组语音识别默认使用openai的whisper模型识别为文字同时以文字回复参数仅支持群聊 (会匹配group_chat_prefix和group_chat_keyword, 支持语音触发画图)
+ 添加 `"voice_reply_voice": true` 将开启语音回复语音(同时作用于私聊和群聊)但是需要配置对应语音合成平台的key由于itchat协议的限制只能发送语音mp3文件若使用wechaty则回复的是微信语音。
+ 添加 `"voice_reply_voice": true` 将开启语音回复语音(同时作用于私聊和群聊)
**4.其他配置**
+ `model`: 模型名称,目前支持 `gpt-3.5-turbo`, `text-davinci-003`, `gpt-4`, `gpt-4-32k`, `wenxin` , `claude` , `xunfei`(其中gpt-4 api暂未完全开放申请通过后可使用)
+ `model`: 模型名称,目前支持 `gpt-3.5-turbo`, `gpt-4o`, `gpt-4-turbo`, `gpt-4`, `wenxin` , `claude` , `gemini`, `glm-4`, `xunfei`, `moonshot`
+ `temperature`,`frequency_penalty`,`presence_penalty`: Chat API接口参数详情参考[OpenAI官方文档。](https://platform.openai.com/docs/api-reference/chat)
+ `proxy`:由于目前 `openai` 接口国内无法访问,需配置代理客户端的地址,详情参考 [#351](https://github.com/zhayujie/chatgpt-on-wechat/issues/351)
+ 对于图像生成,在满足个人或群组触发条件外,还需要额外的关键词前缀来触发,对应配置 `image_create_prefix `
@@ -167,7 +175,7 @@ pip3 install -r requirements-optional.txt
+ `conversation_max_tokens`:表示能够记忆的上下文最大字数(一问一答为一组对话,如果累积的对话字数超出限制,就会优先移除最早的一组对话)
+ `rate_limit_chatgpt``rate_limit_dalle`:每分钟最高问答速率、画图速率,超速后排队按序处理。
+ `clear_memory_commands`: 对话内指令,主动清空前文记忆,字符串数组可自定义指令别名。
+ `hot_reload`: 程序退出后,暂存微信扫码状态,默认关闭。
+ `hot_reload`: 程序退出后,暂存等于状态,默认关闭。
+ `character_desc` 配置中保存着你对机器人说的一段话,他会记住这段话并作为他的设定,你可以为他定制任何人格 (关于会话上下文的更多内容参考该 [issue](https://github.com/zhayujie/chatgpt-on-wechat/issues/43))
+ `subscribe_msg`订阅消息公众号和企业微信channel中请填写当被订阅时会自动回复 可使用特殊占位符。目前支持的占位符有{trigger_prefix}在程序中它会自动替换成bot的触发词。
@@ -175,11 +183,11 @@ pip3 install -r requirements-optional.txt
+ `use_linkai`: 是否使用LinkAI接口开启后可国内访问使用知识库和 `Midjourney` 绘画, 参考 [文档](https://link-ai.tech/platform/link-app/wechat)
+ `linkai_api_key`: LinkAI Api Key可在 [控制台](https://link-ai.tech/console/interface) 创建
+ `linkai_app_code`: LinkAI 应用code选填
+ `linkai_app_code`: LinkAI 应用或工作流的code选填
**本说明文档可能会未及时更新,当前所有可选的配置项均在该[`config.py`](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/config.py)中列出。**
## 运行
## 三、运行
### 1.本地运行
@@ -189,7 +197,7 @@ pip3 install -r requirements-optional.txt
python3 app.py # windows环境下该命令通常为 python app.py
```
终端输出二维码后,使用微信进行扫码,当输出 "Start auto replying" 时表示自动回复程序已经成功运行了(注意:用于登录的微信需要在支付处已完成实名认证)。扫码登录后你的账号就成为机器人了,可以在手机端通过配置的关键词触发自动回复 (任意好友发送消息给你,或是自己发消息给好友),参考[#142](https://github.com/zhayujie/chatgpt-on-wechat/issues/142)。
终端输出二维码后,进行扫码登录,当输出 "Start auto replying" 时表示自动回复程序已经成功运行了(注意:用于登录的账号需要在支付处已完成实名认证)。扫码登录后你的账号就成为机器人了,可以在手机端通过配置的关键词触发自动回复 (任意好友发送消息给你,或是自己发消息给好友),参考[#142](https://github.com/zhayujie/chatgpt-on-wechat/issues/142)。
### 2.服务器部署
@@ -211,7 +219,7 @@ nohup python3 app.py & tail -f nohup.out # 在后台运行程序并通
> 前提是需要安装好 `docker` 及 `docker-compose`,安装成功的表现是执行 `docker -v` 和 `docker-compose version` (或 docker compose version) 可以查看到版本号,可前往 [docker官网](https://docs.docker.com/engine/install/) 进行下载。
#### (1) 下载 docker-compose.yml 文件
**(1) 下载 docker-compose.yml 文件**
```bash
wget https://open-1317903499.cos.ap-guangzhou.myqcloud.com/docker-compose.yml
@@ -219,7 +227,7 @@ wget https://open-1317903499.cos.ap-guangzhou.myqcloud.com/docker-compose.yml
下载完成后打开 `docker-compose.yml` 修改所需配置,如 `OPEN_AI_API_KEY``GROUP_NAME_WHITE_LIST` 等。
#### (2) 启动容器
**(2) 启动容器**
`docker-compose.yml` 所在目录下执行以下命令启动容器:
@@ -240,7 +248,7 @@ sudo docker compose up -d
sudo docker logs -f chatgpt-on-wechat
```
#### (3) 插件使用
**(3) 插件使用**
如果需要在docker容器中修改插件配置可通过挂载的方式完成将 [插件配置文件](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/plugins/config.json.template)
重命名为 `config.json`,放置于 `docker-compose.yml` 相同目录下,并在 `docker-compose.yml` 中的 `chatgpt-on-wechat` 部分下添加 `volumes` 映射:
@@ -262,16 +270,22 @@ volumes:
[![Deploy on Railway](https://railway.app/button.svg)](https://railway.app/template/qApznZ?referralCode=RC3znh)
## 常见问题
<br>
# 🔎 常见问题
FAQs <https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs>
或直接在线咨询 [项目小助手](https://link-ai.tech/app/Kv2fXJcH) (beta版本语料完善中,回复仅供参考)
或直接在线咨询 [项目小助手](https://link-ai.tech/app/Kv2fXJcH) (语料持续完善中,回复仅供参考)
## 开发
# 🛠️ 开发
欢迎接入更多应用,参考 [Terminal代码](https://github.com/zhayujie/chatgpt-on-wechat/blob/master/channel/terminal/terminal_channel.py) 实现接收和发送消息逻辑即可接入。 同时欢迎增加新的插件,参考 [插件说明文档](https://github.com/zhayujie/chatgpt-on-wechat/tree/master/plugins)。
## 联系
# ✉ 联系
欢迎提交PR、Issues以及Star支持一下。程序运行遇到问题可以查看 [常见问题列表](https://github.com/zhayujie/chatgpt-on-wechat/wiki/FAQs) ,其次前往 [Issues](https://github.com/zhayujie/chatgpt-on-wechat/issues) 中搜索。个人开发者可加入开源交流群参与更多讨论,企业用户可联系[产品顾问](https://img-1317903499.cos.ap-guangzhou.myqcloud.com/docs/product-manager-qrcode.jpg)咨询。
# 🌟 贡献者
![cow contributors](https://contrib.rocks/image?repo=zhayujie/chatgpt-on-wechat&max=1000)

View File

@@ -94,7 +94,7 @@ class BaiduWenxinBot(Bot):
logger.warn("[BAIDU] Exception: {}".format(e))
need_retry = False
self.sessions.clear_session(session.session_id)
result = {"completion_tokens": 0, "content": "出错了: {}".format(e)}
result = {"total_tokens": 0, "completion_tokens": 0, "content": "出错了: {}".format(e)}
return result
def get_access_token(self):

View File

@@ -171,24 +171,48 @@ class AzureChatGPTBot(ChatGPTBot):
self.args["deployment_id"] = conf().get("azure_deployment_id")
def create_img(self, query, retry_count=0, api_key=None):
api_version = "2022-08-03-preview"
url = "{}dalle/text-to-image?api-version={}".format(openai.api_base, api_version)
api_key = api_key or openai.api_key
headers = {"api-key": api_key, "Content-Type": "application/json"}
try:
body = {"caption": query, "resolution": conf().get("image_create_size", "256x256")}
submission = requests.post(url, headers=headers, json=body)
operation_location = submission.headers["Operation-Location"]
retry_after = submission.headers["Retry-after"]
status = ""
image_url = ""
while status != "Succeeded":
logger.info("waiting for image create..., " + status + ",retry after " + retry_after + " seconds")
time.sleep(int(retry_after))
response = requests.get(operation_location, headers=headers)
status = response.json()["status"]
image_url = response.json()["result"]["contentUrl"]
return True, image_url
except Exception as e:
logger.error("create image error: {}".format(e))
return False, "图片生成失败"
text_to_image_model = conf().get("text_to_image")
if text_to_image_model == "dall-e-2":
api_version = "2023-06-01-preview"
endpoint = conf().get("azure_openai_dalle_api_base","open_ai_api_base")
# 检查endpoint是否以/结尾
if not endpoint.endswith("/"):
endpoint = endpoint + "/"
url = "{}openai/images/generations:submit?api-version={}".format(endpoint, api_version)
api_key = conf().get("azure_openai_dalle_api_key","open_ai_api_key")
headers = {"api-key": api_key, "Content-Type": "application/json"}
try:
body = {"prompt": query, "size": conf().get("image_create_size", "256x256"),"n": 1}
submission = requests.post(url, headers=headers, json=body)
operation_location = submission.headers['operation-location']
status = ""
while (status != "succeeded"):
if retry_count > 3:
return False, "图片生成失败"
response = requests.get(operation_location, headers=headers)
status = response.json()['status']
retry_count += 1
image_url = response.json()['result']['data'][0]['url']
return True, image_url
except Exception as e:
logger.error("create image error: {}".format(e))
return False, "图片生成失败"
elif text_to_image_model == "dall-e-3":
api_version = conf().get("azure_api_version", "2024-02-15-preview")
endpoint = conf().get("azure_openai_dalle_api_base","open_ai_api_base")
# 检查endpoint是否以/结尾
if not endpoint.endswith("/"):
endpoint = endpoint + "/"
url = "{}openai/deployments/{}/images/generations?api-version={}".format(endpoint, conf().get("azure_openai_dalle_deployment_id","text_to_image"),api_version)
api_key = conf().get("azure_openai_dalle_api_key","open_ai_api_key")
headers = {"api-key": api_key, "Content-Type": "application/json"}
try:
body = {"prompt": query, "size": conf().get("image_create_size", "1024x1024"), "quality": conf().get("dalle3_image_quality", "standard")}
submission = requests.post(url, headers=headers, json=body)
image_url = submission.json()['data'][0]['url']
return True, image_url
except Exception as e:
logger.error("create image error: {}".format(e))
return False, "图片生成失败"
else:
return False, "图片生成失败未配置text_to_image参数"

View File

@@ -62,11 +62,12 @@ def num_tokens_from_messages(messages, model):
import tiktoken
if model in ["gpt-3.5-turbo-0301", "gpt-35-turbo", "gpt-3.5-turbo-1106", "moonshot"]:
if model in ["gpt-3.5-turbo-0301", "gpt-35-turbo", "gpt-3.5-turbo-1106", "moonshot", const.LINKAI_35]:
return num_tokens_from_messages(messages, model="gpt-3.5-turbo")
elif model in ["gpt-4-0314", "gpt-4-0613", "gpt-4-32k", "gpt-4-32k-0613", "gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k", "gpt-3.5-turbo-16k-0613", "gpt-35-turbo-16k", "gpt-4-turbo-preview",
"gpt-4-1106-preview", const.GPT4_TURBO_PREVIEW, const.GPT4_VISION_PREVIEW, const.GPT4_TURBO_01_25]:
"gpt-4-1106-preview", const.GPT4_TURBO_PREVIEW, const.GPT4_VISION_PREVIEW, const.GPT4_TURBO_01_25,
const.GPT_4o, const.LINKAI_4o, const.LINKAI_4_TURBO]:
return num_tokens_from_messages(messages, model="gpt-4")
elif model.startswith("claude-3"):
return num_tokens_from_messages(messages, model="gpt-3.5-turbo")

View File

@@ -122,7 +122,7 @@ class LinkAIBot(Bot):
headers = {"Authorization": "Bearer " + linkai_api_key}
# do http request
base_url = conf().get("linkai_api_base", "https://api.link-ai.chat")
base_url = conf().get("linkai_api_base", "https://api.link-ai.tech")
res = requests.post(url=base_url + "/v1/chat/completions", json=body, headers=headers,
timeout=conf().get("request_timeout", 180))
if res.status_code == 200:
@@ -261,7 +261,7 @@ class LinkAIBot(Bot):
headers = {"Authorization": "Bearer " + conf().get("linkai_api_key")}
# do http request
base_url = conf().get("linkai_api_base", "https://api.link-ai.chat")
base_url = conf().get("linkai_api_base", "https://api.link-ai.tech")
res = requests.post(url=base_url + "/v1/chat/completions", json=body, headers=headers,
timeout=conf().get("request_timeout", 180))
if res.status_code == 200:
@@ -304,7 +304,7 @@ class LinkAIBot(Bot):
def _fetch_app_info(self, app_code: str):
headers = {"Authorization": "Bearer " + conf().get("linkai_api_key")}
# do http request
base_url = conf().get("linkai_api_base", "https://api.link-ai.chat")
base_url = conf().get("linkai_api_base", "https://api.link-ai.tech")
params = {"app_code": app_code}
res = requests.get(url=base_url + "/v1/app/info", params=params, headers=headers, timeout=(5, 10))
if res.status_code == 200:
@@ -326,7 +326,7 @@ class LinkAIBot(Bot):
"response_format": "url",
"img_proxy": conf().get("image_proxy")
}
url = conf().get("linkai_api_base", "https://api.link-ai.chat") + "/v1/images/generations"
url = conf().get("linkai_api_base", "https://api.link-ai.tech") + "/v1/images/generations"
res = requests.post(url, headers=headers, json=data, timeout=(5, 90))
t2 = time.time()
image_url = res.json()["data"][0]["url"]

View File

@@ -39,20 +39,22 @@ class Bridge(object):
if model_type and model_type.startswith("claude-3"):
self.btype["chat"] = const.CLAUDEAPI
if conf().get("use_linkai") and conf().get("linkai_api_key"):
self.btype["chat"] = const.LINKAI
if not conf().get("voice_to_text") or conf().get("voice_to_text") in ["openai"]:
self.btype["voice_to_text"] = const.LINKAI
if not conf().get("text_to_voice") or conf().get("text_to_voice") in ["openai", const.TTS_1, const.TTS_1_HD]:
self.btype["text_to_voice"] = const.LINKAI
if model_type in ["claude"]:
self.btype["chat"] = const.CLAUDEAI
if model_type in ["moonshot-v1-8k", "moonshot-v1-32k", "moonshot-v1-128k"]:
self.btype["chat"] = const.MOONSHOT
if conf().get("use_linkai") and conf().get("linkai_api_key"):
self.btype["chat"] = const.LINKAI
if not conf().get("voice_to_text") or conf().get("voice_to_text") in ["openai"]:
self.btype["voice_to_text"] = const.LINKAI
if not conf().get("text_to_voice") or conf().get("text_to_voice") in ["openai", const.TTS_1, const.TTS_1_HD]:
self.btype["text_to_voice"] = const.LINKAI
self.bots = {}
self.chat_bots = {}
# 模型对应的接口
def get_bot(self, typename):
if self.bots.get(typename) is None:

View File

@@ -86,14 +86,14 @@ class ChatChannel(Channel):
if e_context.is_pass() or context is None:
return context
if cmsg.from_user_id == self.user_id and not config.get("trigger_by_self", True):
logger.debug("[WX]self message skipped")
logger.debug("[chat_channel]self message skipped")
return None
# 消息内容匹配过程并处理content
if ctype == ContextType.TEXT:
if first_in and "\n- - - - - - -" in content: # 初次匹配 过滤引用消息
logger.debug(content)
logger.debug("[WX]reference query skipped")
logger.debug("[chat_channel]reference query skipped")
return None
nick_name_black_list = conf().get("nick_name_black_list", [])
@@ -111,10 +111,10 @@ class ChatChannel(Channel):
nick_name = context["msg"].actual_user_nickname
if nick_name and nick_name in nick_name_black_list:
# 黑名单过滤
logger.warning(f"[WX] Nickname {nick_name} in In BlackList, ignore")
logger.warning(f"[chat_channel] Nickname {nick_name} in In BlackList, ignore")
return None
logger.info("[WX]receive group at")
logger.info("[chat_channel]receive group at")
if not conf().get("group_at_off", False):
flag = True
pattern = f"@{re.escape(self.name)}(\u2005|\u0020)"
@@ -130,13 +130,13 @@ class ChatChannel(Channel):
content = subtract_res
if not flag:
if context["origin_ctype"] == ContextType.VOICE:
logger.info("[WX]receive group voice, but checkprefix didn't match")
logger.info("[chat_channel]receive group voice, but checkprefix didn't match")
return None
else: # 单聊
nick_name = context["msg"].from_user_nickname
if nick_name and nick_name in nick_name_black_list:
# 黑名单过滤
logger.warning(f"[WX] Nickname '{nick_name}' in In BlackList, ignore")
logger.warning(f"[chat_channel] Nickname '{nick_name}' in In BlackList, ignore")
return None
match_prefix = check_prefix(content, conf().get("single_chat_prefix", [""]))
@@ -147,7 +147,7 @@ class ChatChannel(Channel):
else:
return None
content = content.strip()
img_match_prefix = check_prefix(content, conf().get("image_create_prefix"))
img_match_prefix = check_prefix(content, conf().get("image_create_prefix",[""]))
if img_match_prefix:
content = content.replace(img_match_prefix, "", 1)
context.type = ContextType.IMAGE_CREATE
@@ -159,17 +159,16 @@ class ChatChannel(Channel):
elif context.type == ContextType.VOICE:
if "desire_rtype" not in context and conf().get("voice_reply_voice") and ReplyType.VOICE not in self.NOT_SUPPORT_REPLYTYPE:
context["desire_rtype"] = ReplyType.VOICE
return context
def _handle(self, context: Context):
if context is None or not context.content:
return
logger.debug("[WX] ready to handle context: {}".format(context))
logger.debug("[chat_channel] ready to handle context: {}".format(context))
# reply的构建步骤
reply = self._generate_reply(context)
logger.debug("[WX] ready to decorate reply: {}".format(reply))
logger.debug("[chat_channel] ready to decorate reply: {}".format(reply))
# reply的包装步骤
if reply and reply.content:
@@ -187,7 +186,7 @@ class ChatChannel(Channel):
)
reply = e_context["reply"]
if not e_context.is_pass():
logger.debug("[WX] ready to handle context: type={}, content={}".format(context.type, context.content))
logger.debug("[chat_channel] ready to handle context: type={}, content={}".format(context.type, context.content))
if context.type == ContextType.TEXT or context.type == ContextType.IMAGE_CREATE: # 文字和图片消息
context["channel"] = e_context["channel"]
reply = super().build_reply_content(context.content, context)
@@ -199,7 +198,7 @@ class ChatChannel(Channel):
try:
any_to_wav(file_path, wav_path)
except Exception as e: # 转换失败直接使用mp3对于某些apimp3也可以识别
logger.warning("[WX]any to wav error, use raw path. " + str(e))
logger.warning("[chat_channel]any to wav error, use raw path. " + str(e))
wav_path = file_path
# 语音识别
reply = super().build_voice_to_text(wav_path)
@@ -210,7 +209,7 @@ class ChatChannel(Channel):
os.remove(wav_path)
except Exception as e:
pass
# logger.warning("[WX]delete temp file error: " + str(e))
# logger.warning("[chat_channel]delete temp file error: " + str(e))
if reply.type == ReplyType.TEXT:
new_context = self._compose_context(ContextType.TEXT, reply.content, **context.kwargs)
@@ -228,7 +227,7 @@ class ChatChannel(Channel):
elif context.type == ContextType.FUNCTION or context.type == ContextType.FILE: # 文件消息及函数调用等,当前无默认逻辑
pass
else:
logger.warning("[WX] unknown context type: {}".format(context.type))
logger.warning("[chat_channel] unknown context type: {}".format(context.type))
return
return reply
@@ -244,7 +243,7 @@ class ChatChannel(Channel):
desire_rtype = context.get("desire_rtype")
if not e_context.is_pass() and reply and reply.type:
if reply.type in self.NOT_SUPPORT_REPLYTYPE:
logger.error("[WX]reply type not support: " + str(reply.type))
logger.error("[chat_channel]reply type not support: " + str(reply.type))
reply.type = ReplyType.ERROR
reply.content = "不支持发送的消息类型: " + str(reply.type)
@@ -265,10 +264,10 @@ class ChatChannel(Channel):
elif reply.type == ReplyType.IMAGE_URL or reply.type == ReplyType.VOICE or reply.type == ReplyType.IMAGE or reply.type == ReplyType.FILE or reply.type == ReplyType.VIDEO or reply.type == ReplyType.VIDEO_URL:
pass
else:
logger.error("[WX] unknown reply type: {}".format(reply.type))
logger.error("[chat_channel] unknown reply type: {}".format(reply.type))
return
if desire_rtype and desire_rtype != reply.type and reply.type not in [ReplyType.ERROR, ReplyType.INFO]:
logger.warning("[WX] desire_rtype: {}, but reply type: {}".format(context.get("desire_rtype"), reply.type))
logger.warning("[chat_channel] desire_rtype: {}, but reply type: {}".format(context.get("desire_rtype"), reply.type))
return reply
def _send_reply(self, context: Context, reply: Reply):
@@ -281,14 +280,14 @@ class ChatChannel(Channel):
)
reply = e_context["reply"]
if not e_context.is_pass() and reply and reply.type:
logger.debug("[WX] ready to send reply: {}, context: {}".format(reply, context))
logger.debug("[chat_channel] ready to send reply: {}, context: {}".format(reply, context))
self._send(reply, context)
def _send(self, reply: Reply, context: Context, retry_cnt=0):
try:
self.send(reply, context)
except Exception as e:
logger.error("[WX] sendMsg error: {}".format(str(e)))
logger.error("[chat_channel] sendMsg error: {}".format(str(e)))
if isinstance(e, NotImplementedError):
return
logger.exception(e)
@@ -342,7 +341,7 @@ class ChatChannel(Channel):
if semaphore.acquire(blocking=False): # 等线程处理完毕才能删除
if not context_queue.empty():
context = context_queue.get()
logger.debug("[WX] consume context: {}".format(context))
logger.debug("[chat_channel] consume context: {}".format(context))
future: Future = handler_pool.submit(self._handle, context)
future.add_done_callback(self._thread_pool_callback(session_id, context=context))
if session_id not in self.futures:

View File

@@ -4,20 +4,81 @@
@author huiwen
@Date 2023/11/28
"""
import copy
import json
# -*- coding=utf-8 -*-
import logging
import time
import dingtalk_stream
from dingtalk_stream import AckMessage
from dingtalk_stream.card_replier import AICardReplier
from dingtalk_stream.card_replier import AICardStatus
from dingtalk_stream.card_replier import CardReplier
from bridge.context import Context, ContextType
from bridge.reply import Reply, ReplyType
from channel.chat_channel import ChatChannel
from channel.dingtalk.dingtalk_message import DingTalkMessage
from bridge.context import Context
from bridge.reply import Reply
from common.expired_dict import ExpiredDict
from common.log import logger
from common.singleton import singleton
from common.time_check import time_checker
from config import conf
from common.expired_dict import ExpiredDict
from bridge.context import ContextType
from channel.chat_channel import ChatChannel
import logging
from dingtalk_stream import AckMessage
import dingtalk_stream
class CustomAICardReplier(CardReplier):
def __init__(self, dingtalk_client, incoming_message):
super(AICardReplier, self).__init__(dingtalk_client, incoming_message)
def start(
self,
card_template_id: str,
card_data: dict,
recipients: list = None,
support_forward: bool = True,
) -> str:
"""
AI卡片的创建接口
:param support_forward:
:param recipients:
:param card_template_id:
:param card_data:
:return:
"""
card_data_with_status = copy.deepcopy(card_data)
card_data_with_status["flowStatus"] = AICardStatus.PROCESSING
return self.create_and_send_card(
card_template_id,
card_data_with_status,
at_sender=True,
at_all=False,
recipients=recipients,
support_forward=support_forward,
)
# 对 AICardReplier 进行猴子补丁
AICardReplier.start = CustomAICardReplier.start
def _check(func):
def wrapper(self, cmsg: DingTalkMessage):
msgId = cmsg.msg_id
if msgId in self.receivedMsgs:
logger.info("DingTalk message {} already received, ignore".format(msgId))
return
self.receivedMsgs[msgId] = True
create_time = cmsg.create_time # 消息时间戳
if conf().get("hot_reload") == True and int(create_time) < int(time.time()) - 60: # 跳过1分钟前的历史消息
logger.debug("[DingTalk] History message {} skipped".format(msgId))
return
if cmsg.my_msg and not cmsg.is_group:
logger.debug("[DingTalk] My message {} skipped".format(msgId))
return
return func(self, cmsg)
return wrapper
@singleton
@@ -39,11 +100,13 @@ class DingTalkChanel(ChatChannel, dingtalk_stream.ChatbotHandler):
super(dingtalk_stream.ChatbotHandler, self).__init__()
self.logger = self.setup_logger()
# 历史消息id暂存用于幂等控制
self.receivedMsgs = ExpiredDict(60 * 60 * 7.1)
logger.info("[dingtalk] client_id={}, client_secret={} ".format(
self.receivedMsgs = ExpiredDict(conf().get("expires_in_seconds"))
logger.info("[DingTalk] client_id={}, client_secret={} ".format(
self.dingtalk_client_id, self.dingtalk_client_secret))
# 无需群校验和前缀
conf()["group_name_white_list"] = ["ALL_GROUP"]
# 单聊无需前缀
conf()["single_chat_prefix"] = [""]
def startup(self):
credential = dingtalk_stream.Credential(self.dingtalk_client_id, self.dingtalk_client_secret)
@@ -51,50 +114,112 @@ class DingTalkChanel(ChatChannel, dingtalk_stream.ChatbotHandler):
client.register_callback_handler(dingtalk_stream.chatbot.ChatbotMessage.TOPIC, self)
client.start_forever()
async def process(self, callback: dingtalk_stream.CallbackMessage):
try:
incoming_message = dingtalk_stream.ChatbotMessage.from_dict(callback.data)
image_download_handler = self # 传入方法所在的类实例
dingtalk_msg = DingTalkMessage(incoming_message, image_download_handler)
if dingtalk_msg.is_group:
self.handle_group(dingtalk_msg)
else:
self.handle_single(dingtalk_msg)
return AckMessage.STATUS_OK, 'OK'
except Exception as e:
logger.error(f"dingtalk process error={e}")
return AckMessage.STATUS_SYSTEM_EXCEPTION, 'ERROR'
@time_checker
@_check
def handle_single(self, cmsg: DingTalkMessage):
# 处理单聊消息
if cmsg.ctype == ContextType.VOICE:
logger.debug("[dingtalk]receive voice msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive voice msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.IMAGE:
logger.debug("[dingtalk]receive image msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive image msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.IMAGE_CREATE:
logger.debug("[DingTalk]receive image create msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.PATPAT:
logger.debug("[dingtalk]receive patpat msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.TEXT:
expression = cmsg.my_msg
cmsg.content = conf()["single_chat_prefix"][0] + cmsg.content
logger.debug("[DingTalk]receive text msg: {}".format(cmsg.content))
else:
logger.debug("[DingTalk]receive other msg: {}".format(cmsg.content))
context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=False, msg=cmsg)
if context:
self.produce(context)
@time_checker
@_check
def handle_group(self, cmsg: DingTalkMessage):
# 处理群聊消息
if cmsg.ctype == ContextType.VOICE:
logger.debug("[dingtalk]receive voice msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive voice msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.IMAGE:
logger.debug("[dingtalk]receive image msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive image msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.IMAGE_CREATE:
logger.debug("[DingTalk]receive image create msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.PATPAT:
logger.debug("[dingtalk]receive patpat msg: {}".format(cmsg.content))
logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
elif cmsg.ctype == ContextType.TEXT:
expression = cmsg.my_msg
cmsg.content = conf()["group_chat_prefix"][0] + cmsg.content
logger.debug("[DingTalk]receive patpat msg: {}".format(cmsg.content))
else:
logger.debug("[DingTalk]receive other msg: {}".format(cmsg.content))
context = self._compose_context(cmsg.ctype, cmsg.content, isgroup=True, msg=cmsg)
context['no_need_at'] = True
if context:
self.produce(context)
async def process(self, callback: dingtalk_stream.CallbackMessage):
try:
incoming_message = dingtalk_stream.ChatbotMessage.from_dict(callback.data)
dingtalk_msg = DingTalkMessage(incoming_message)
if incoming_message.conversation_type == '1':
self.handle_single(dingtalk_msg)
else:
self.handle_group(dingtalk_msg)
return AckMessage.STATUS_OK, 'OK'
except Exception as e:
logger.error(e)
return self.FAILED_MSG
def send(self, reply: Reply, context: Context):
receiver = context["receiver"]
isgroup = context.kwargs['msg'].is_group
incoming_message = context.kwargs['msg'].incoming_message
self.reply_text(reply.content, incoming_message)
if conf().get("dingtalk_card_enabled"):
logger.info("[Dingtalk] sendMsg={}, receiver={}".format(reply, receiver))
def reply_with_text():
self.reply_text(reply.content, incoming_message)
def reply_with_at_text():
self.reply_text("📢 您有一条新的消息,请查看。", incoming_message)
def reply_with_ai_markdown():
button_list, markdown_content = self.generate_button_markdown_content(context, reply)
self.reply_ai_markdown_button(incoming_message, markdown_content, button_list, "", "📌 内容由AI生成", "",[incoming_message.sender_staff_id])
if reply.type in [ReplyType.IMAGE_URL, ReplyType.IMAGE, ReplyType.TEXT]:
if isgroup:
reply_with_ai_markdown()
reply_with_at_text()
else:
reply_with_ai_markdown()
else:
# 暂不支持其它类型消息回复
reply_with_text()
else:
self.reply_text(reply.content, incoming_message)
def generate_button_markdown_content(self, context, reply):
image_url = context.kwargs.get("image_url")
promptEn = context.kwargs.get("promptEn")
reply_text = reply.content
button_list = []
markdown_content = f"""
{reply.content}
"""
if image_url is not None and promptEn is not None:
button_list = [
{"text": "查看原图", "url": image_url, "iosUrl": image_url, "color": "blue"}
]
markdown_content = f"""
{promptEn}
!["图片"]({image_url})
{reply_text}
"""
logger.debug(f"[Dingtalk] generate_button_markdown_content, button_list={button_list} , markdown_content={markdown_content}")
return button_list, markdown_content

View File

@@ -1,44 +1,83 @@
from bridge.context import ContextType
from channel.chat_message import ChatMessage
import json
import os
import requests
from common.log import logger
from common.tmp_dir import TmpDir
from common import utils
from dingtalk_stream import ChatbotMessage
from bridge.context import ContextType
from channel.chat_message import ChatMessage
# -*- coding=utf-8 -*-
from common.log import logger
from common.tmp_dir import TmpDir
class DingTalkMessage(ChatMessage):
def __init__(self, event: ChatbotMessage):
def __init__(self, event: ChatbotMessage, image_download_handler):
super().__init__(event)
self.image_download_handler = image_download_handler
self.msg_id = event.message_id
msg_type = event.message_type
self.incoming_message =event
self.message_type = event.message_type
self.incoming_message = event
self.sender_staff_id = event.sender_staff_id
self.other_user_id = event.conversation_id
self.create_time = event.create_at
if event.conversation_type=="1":
self.image_content = event.image_content
self.rich_text_content = event.rich_text_content
if event.conversation_type == "1":
self.is_group = False
else:
self.is_group = True
if msg_type == "text":
if self.message_type == "text":
self.ctype = ContextType.TEXT
self.content = event.text.content.strip()
elif msg_type == "audio":
elif self.message_type == "audio":
# 钉钉支持直接识别语音,所以此处将直接提取文字,当文字处理
self.content = event.extensions['content']['recognition'].strip()
self.ctype = ContextType.TEXT
elif (self.message_type == 'picture') or (self.message_type == 'richText'):
self.ctype = ContextType.IMAGE
# 钉钉图片类型或富文本类型消息处理
image_list = event.get_image_list()
if len(image_list) > 0:
download_code = image_list[0]
download_url = image_download_handler.get_image_download_url(download_code)
self.content = download_image_file(download_url, TmpDir().path())
else:
logger.debug(f"[Dingtalk] messageType :{self.message_type} , imageList isEmpty")
if self.is_group:
self.from_user_id = event.conversation_id
self.actual_user_id = event.sender_id
else:
self.from_user_id = event.sender_id
self.actual_user_id = event.sender_id
self.to_user_id = event.chatbot_user_id
self.other_user_nickname = event.conversation_title
user_id = event.sender_id
nickname =event.sender_nick
def download_image_file(image_url, temp_dir):
headers = {
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36'
}
# 设置代理
# self.proxies
# , proxies=self.proxies
response = requests.get(image_url, headers=headers, stream=True, timeout=60 * 5)
if response.status_code == 200:
# 生成文件名
file_name = image_url.split("/")[-1].split("?")[0]
# 检查临时目录是否存在,如果不存在则创建
if not os.path.exists(temp_dir):
os.makedirs(temp_dir)
# 将文件保存到临时目录
file_path = os.path.join(temp_dir, file_name)
with open(file_path, 'wb') as file:
file.write(response.content)
return file_path
else:
logger.info(f"[Dingtalk] Failed to download image file, {response.content}")
return None

View File

@@ -40,7 +40,7 @@ class FeiShuChanel(ChatChannel):
self.feishu_app_id, self.feishu_app_secret, self.feishu_token))
# 无需群校验和前缀
conf()["group_name_white_list"] = ["ALL_GROUP"]
conf()["single_chat_prefix"] = []
conf()["single_chat_prefix"] = [""]
def startup(self):
urls = (

View File

@@ -109,7 +109,7 @@ class WechatChannel(ChatChannel):
def __init__(self):
super().__init__()
self.receivedMsgs = ExpiredDict(60 * 60)
self.receivedMsgs = ExpiredDict(conf().get("expires_in_seconds"))
self.auto_login_times = 0
def startup(self):
@@ -132,7 +132,7 @@ class WechatChannel(ChatChannel):
# start message listener
itchat.run()
except Exception as e:
logger.error(e)
logger.exception(e)
def exitCallback(self):
try:

View File

@@ -140,6 +140,42 @@ class WechatMPChannel(ChatChannel):
media_id = response["media_id"]
logger.info("[wechatmp] image uploaded, receiver {}, media_id {}".format(receiver, media_id))
self.cache_dict[receiver].append(("image", media_id))
elif reply.type == ReplyType.VIDEO_URL: # 从网络下载视频
video_url = reply.content
video_res = requests.get(video_url, stream=True)
video_storage = io.BytesIO()
for block in video_res.iter_content(1024):
video_storage.write(block)
video_storage.seek(0)
video_type = 'mp4'
filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
content_type = "video/" + video_type
try:
response = self.client.material.add("video", (filename, video_storage, content_type))
logger.debug("[wechatmp] upload video response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatmp] upload video failed: {}".format(e))
return
media_id = response["media_id"]
logger.info("[wechatmp] video uploaded, receiver {}, media_id {}".format(receiver, media_id))
self.cache_dict[receiver].append(("video", media_id))
elif reply.type == ReplyType.VIDEO: # 从文件读取视频
video_storage = reply.content
video_storage.seek(0)
video_type = 'mp4'
filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
content_type = "video/" + video_type
try:
response = self.client.material.add("video", (filename, video_storage, content_type))
logger.debug("[wechatmp] upload video response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatmp] upload video failed: {}".format(e))
return
media_id = response["media_id"]
logger.info("[wechatmp] video uploaded, receiver {}, media_id {}".format(receiver, media_id))
self.cache_dict[receiver].append(("video", media_id))
else:
if reply.type == ReplyType.TEXT or reply.type == ReplyType.INFO or reply.type == ReplyType.ERROR:
reply_text = reply.content
@@ -222,6 +258,38 @@ class WechatMPChannel(ChatChannel):
return
self.client.message.send_image(receiver, response["media_id"])
logger.info("[wechatmp] Do send image to {}".format(receiver))
elif reply.type == ReplyType.VIDEO_URL: # 从网络下载视频
video_url = reply.content
video_res = requests.get(video_url, stream=True)
video_storage = io.BytesIO()
for block in video_res.iter_content(1024):
video_storage.write(block)
video_storage.seek(0)
video_type = 'mp4'
filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
content_type = "video/" + video_type
try:
response = self.client.media.upload("video", (filename, video_storage, content_type))
logger.debug("[wechatmp] upload video response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatmp] upload video failed: {}".format(e))
return
self.client.message.send_video(receiver, response["media_id"])
logger.info("[wechatmp] Do send video to {}".format(receiver))
elif reply.type == ReplyType.VIDEO: # 从文件读取视频
video_storage = reply.content
video_storage.seek(0)
video_type = 'mp4'
filename = receiver + "-" + str(context["msg"].msg_id) + "." + video_type
content_type = "video/" + video_type
try:
response = self.client.media.upload("video", (filename, video_storage, content_type))
logger.debug("[wechatmp] upload video response: {}".format(response))
except WeChatClientException as e:
logger.error("[wechatmp] upload video failed: {}".format(e))
return
self.client.message.send_video(receiver, response["media_id"])
logger.info("[wechatmp] Do send video to {}".format(receiver))
return
def _success_callback(self, session_id, context, **kwargs): # 线程异常结束时的回调函数

View File

@@ -23,6 +23,10 @@ MOONSHOT = "moonshot"
CLAUDE3 = "claude-3-opus-20240229"
GPT35 = "gpt-3.5-turbo"
GPT4 = "gpt-4"
GPT_4o = "gpt-4o"
LINKAI_35 = "linkai-3.5"
LINKAI_4_TURBO = "linkai-4-turbo"
LINKAI_4o = "linkai-4o"
GPT4_TURBO_PREVIEW = "gpt-4-turbo-2024-04-09"
GPT4_TURBO_04_09 = "gpt-4-turbo-2024-04-09"
GPT4_TURBO_01_25 = "gpt-4-0125-preview"
@@ -32,8 +36,8 @@ TTS_1 = "tts-1"
TTS_1_HD = "tts-1-hd"
MODEL_LIST = ["gpt-3.5-turbo", "gpt-3.5-turbo-16k", "gpt-4", "wenxin", "wenxin-4", "xunfei", "claude", "claude-3-opus-20240229", "gpt-4-turbo",
"gpt-4-turbo-preview", "gpt-4-1106-preview", GPT4_TURBO_PREVIEW, GPT4_TURBO_01_25, QWEN, GEMINI, ZHIPU_AI, MOONSHOT,
QWEN_TURBO, QWEN_PLUS, QWEN_MAX]
"gpt-4-turbo-preview", "gpt-4-1106-preview", GPT4_TURBO_PREVIEW, GPT4_TURBO_01_25, GPT_4o, QWEN, GEMINI, ZHIPU_AI, MOONSHOT,
QWEN_TURBO, QWEN_PLUS, QWEN_MAX, LINKAI_35, LINKAI_4_TURBO, LINKAI_4o]
# channel
FEISHU = "feishu"

View File

@@ -45,7 +45,7 @@ class ChatClient(LinkAIClient):
elif reply_voice_mode == "always_reply_voice":
local_config["always_reply_voice"] = True
if config.get("admin_password") and plugin_config["Godcmd"]:
if config.get("admin_password") and plugin_config.get("Godcmd"):
plugin_config["Godcmd"]["password"] = config.get("admin_password")
PluginManager().instances["GODCMD"].reload()
@@ -56,11 +56,18 @@ class ChatClient(LinkAIClient):
pconf("linkai")["group_app_map"] = local_group_map
PluginManager().instances["LINKAI"].reload()
if config.get("text_to_image") and config.get("text_to_image") == "midjourney" and pconf("linkai"):
if pconf("linkai")["midjourney"]:
pconf("linkai")["midjourney"]["enabled"] = True
pconf("linkai")["midjourney"]["use_image_create_prefix"] = True
elif config.get("text_to_image") and config.get("text_to_image") in ["dall-e-2", "dall-e-3"]:
if pconf("linkai")["midjourney"]:
pconf("linkai")["midjourney"]["use_image_create_prefix"] = False
def start(channel):
global chat_client
chat_client = ChatClient(api_key=conf().get("linkai_api_key"),
host="link-ai.chat", channel=channel)
chat_client = ChatClient(api_key=conf().get("linkai_api_key"), host="", channel=channel)
chat_client.config = _build_config()
chat_client.start()
time.sleep(1.5)

View File

@@ -1,7 +1,5 @@
import hashlib
import re
import time
import config
from common.log import logger
@@ -10,31 +8,33 @@ def time_checker(f):
def _time_checker(self, *args, **kwargs):
_config = config.conf()
chat_time_module = _config.get("chat_time_module", False)
if chat_time_module:
chat_start_time = _config.get("chat_start_time", "00:00")
chat_stopt_time = _config.get("chat_stop_time", "24:00")
time_regex = re.compile(r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$") # 时间匹配包含24:00
chat_stop_time = _config.get("chat_stop_time", "24:00")
starttime_format_check = time_regex.match(chat_start_time) # 检查停止时间格式
stoptime_format_check = time_regex.match(chat_stopt_time) # 检查停止时间格式
chat_time_check = chat_start_time < chat_stopt_time # 确定启动时间<停止时间
time_regex = re.compile(r"^([01]?[0-9]|2[0-4])(:)([0-5][0-9])$")
# 时间格式检查
if not (starttime_format_check and stoptime_format_check and chat_time_check):
logger.warn("时间格式不正确,请在config.json中修改您的CHAT_START_TIME/CHAT_STOP_TIME,否则可能会影响您正常使用,开始({})-结束({})".format(starttime_format_check, stoptime_format_check))
if chat_start_time > "23:59":
logger.error("启动时间可能存在问题,请修改!")
# 服务时间检查
now_time = time.strftime("%H:%M", time.localtime())
if chat_start_time <= now_time <= chat_stopt_time: # 服务时间内,正常返回回答
f(self, *args, **kwargs)
if not (time_regex.match(chat_start_time) and time_regex.match(chat_stop_time)):
logger.warning("时间格式不正确请在config.json中修改CHAT_START_TIME/CHAT_STOP_TIME。")
return None
now_time = time.strptime(time.strftime("%H:%M"), "%H:%M")
chat_start_time = time.strptime(chat_start_time, "%H:%M")
chat_stop_time = time.strptime(chat_stop_time, "%H:%M")
# 结束时间小于开始时间,跨天了
if chat_stop_time < chat_start_time and (chat_start_time <= now_time or now_time <= chat_stop_time):
f(self, *args, **kwargs)
# 结束大于开始时间代表,没有跨天
elif chat_start_time < chat_stop_time and chat_start_time <= now_time <= chat_stop_time:
f(self, *args, **kwargs)
else:
if args[0]["Content"] == "#更新配置": # 不在服务时间内也可以更新配置
# 定义匹配规则,如果以 #reconf 或者 #更新配置 结尾, 非服务时间可以修改开始/结束时间并重载配置
pattern = re.compile(r"^.*#(?:reconf|更新配置)$")
if args and pattern.match(args[0].content):
f(self, *args, **kwargs)
else:
logger.info("非服务时间内,不接受访问")
logger.info("非服务时间内不接受访问")
return None
else:
f(self, *args, **kwargs) # 未开启时间模块则直接回答

View File

@@ -28,7 +28,7 @@
"voice_reply_voice": false,
"conversation_max_tokens": 2500,
"expires_in_seconds": 3600,
"character_desc": "你是ChatGPT, 一个由OpenAI训练的大型语言模型, 你旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。",
"character_desc": "你是基于大语言模型的AI智能助手旨在回答并解决人们的任何问题,并且可以使用多种语言与人交流。",
"temperature": 0.7,
"subscribe_msg": "感谢您的关注!\n这里是AI智能助手可以自由对话。\n支持语音对话。\n支持图片输入。\n支持图片输出画字开头的消息将按要求创作图片。\n支持tool、角色扮演和文字冒险等丰富的插件。\n输入{trigger_prefix}#help 查看详细指令。",
"use_linkai": false,

View File

@@ -4,6 +4,7 @@ import json
import logging
import os
import pickle
import copy
from common.log import logger
@@ -36,6 +37,13 @@ available_setting = {
"group_welcome_msg": "", # 配置新人进群固定欢迎语,不配置则使用随机风格欢迎
"trigger_by_self": False, # 是否允许机器人触发
"text_to_image": "dall-e-2", # 图片生成模型,可选 dall-e-2, dall-e-3
# Azure OpenAI dall-e-3 配置
"dalle3_image_style": "vivid", # 图片生成dalle3的风格可选有 vivid, natural
"dalle3_image_quality": "hd", # 图片生成dalle3的质量可选有 standard, hd
# Azure OpenAI DALL-E API 配置, 当use_azure_chatgpt为true时,用于将文字回复的资源和Dall-E的资源分开.
"azure_openai_dalle_api_base": "", # [可选] azure openai 用于回复图片的资源 endpoint默认使用 open_ai_api_base
"azure_openai_dalle_api_key": "", # [可选] azure openai 用于回复图片的资源 key默认使用 open_ai_api_key
"azure_openai_dalle_deployment_id":"", # [可选] azure openai 用于回复图片的资源 deployment id默认使用 text_to_image
"image_proxy": True, # 是否需要图片代理国内访问LinkAI时需要
"image_create_prefix": ["", "", ""], # 开启图片回复的前缀
"concurrency_in_session": 1, # 同一会话最多有多少条消息在处理中大于1可能乱序
@@ -95,7 +103,7 @@ available_setting = {
"baidu_api_key": "",
"baidu_secret_key": "",
# 1536普通话(支持简单的英文识别) 1737英语 1637粤语 1837四川话 1936普通话远场
"baidu_dev_pid": "1536",
"baidu_dev_pid": 1536,
# azure 语音api配置 使用azure语音识别和语音合成时需要
"azure_voice_api_key": "",
"azure_voice_region": "japaneast",
@@ -139,12 +147,13 @@ available_setting = {
# 钉钉配置
"dingtalk_client_id": "", # 钉钉机器人Client ID
"dingtalk_client_secret": "", # 钉钉机器人Client Secret
"dingtalk_client_secret": "", # 钉钉机器人Client Secret
"dingtalk_card_enabled": False,
# chatgpt指令自定义触发词
"clear_memory_commands": ["#清除记忆"], # 重置会话指令,必须以#开头
# channel配置
"channel_type": "wx", # 通道类型,支持:{wx,wxy,terminal,wechatmp,wechatmp_service,wechatcom_app}
"channel_type": "", # 通道类型,支持:{wx,wxy,terminal,wechatmp,wechatmp_service,wechatcom_app,dingtalk}
"subscribe_msg": "", # 订阅消息, 支持: wechatmp, wechatmp_service, wechatcom_app
"debug": False, # 是否开启debug模式开启后会打印更多日志
"appdata_dir": "", # 数据目录
@@ -163,7 +172,7 @@ available_setting = {
"use_linkai": False,
"linkai_api_key": "",
"linkai_app_code": "",
"linkai_api_base": "https://api.link-ai.chat", # linkAI服务地址,若国内无法访问或延迟较高可改为 https://api.link-ai.tech
"linkai_api_base": "https://api.link-ai.tech", # linkAI服务地址
}
@@ -224,6 +233,30 @@ class Config(dict):
config = Config()
def drag_sensitive(config):
try:
if isinstance(config, str):
conf_dict: dict = json.loads(config)
conf_dict_copy = copy.deepcopy(conf_dict)
for key in conf_dict_copy:
if "key" in key or "secret" in key:
if isinstance(key, str):
conf_dict_copy[key] = conf_dict_copy[key][0:3] + "*" * 5 + conf_dict_copy[key][-3:]
return json.dumps(conf_dict_copy, indent=4)
elif isinstance(config, dict):
config_copy = copy.deepcopy(config)
for key in config:
if "key" in key or "secret" in key:
if isinstance(key, str):
config_copy[key] = config_copy[key][0:3] + "*" * 5 + config_copy[key][-3:]
return config_copy
except Exception as e:
logger.exception(e)
return config
return config
def load_config():
global config
config_path = "./config.json"
@@ -232,7 +265,7 @@ def load_config():
config_path = "./config-template.json"
config_str = read_file(config_path)
logger.debug("[INIT] config str: {}".format(config_str))
logger.debug("[INIT] config str: {}".format(drag_sensitive(config_str)))
# 将json字符串反序列化为dict类型
config = Config(json.loads(config_str))
@@ -257,13 +290,11 @@ def load_config():
logger.setLevel(logging.DEBUG)
logger.debug("[INIT] set log level to DEBUG")
logger.info("[INIT] load config: {}".format(config))
logger.info("[INIT] load config: {}".format(drag_sensitive(config)))
config.load_user_datas()
def get_root():
return os.path.dirname(os.path.abspath(__file__))

Binary file not shown.

Before

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 326 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 180 KiB

View File

@@ -8,6 +8,6 @@
2023.03.25 支持插件化开发,目前已实现 多角色切换、文字冒险游戏、管理员指令、Stable Diffusion等插件使用参考 #578。(contributed by @lanvent in #565)
2023.03.09 基于 whisper API(后续已接入更多的语音API服务) 实现对微信语音消息的解析和回复,添加配置项 "speech_recognition":true 即可启用,使用参考 #415。(contributed by wanggang1987 in #385)
2023.03.09 基于 whisper API(后续已接入更多的语音API服务) 实现对语音消息的解析和回复,添加配置项 "speech_recognition":true 即可启用,使用参考 #415。(contributed by wanggang1987 in #385)
2023.02.09 扫码登录存在账号限制风险,请谨慎使用,参考#58

View File

@@ -9,6 +9,7 @@ from common.expired_dict import ExpiredDict
from common import const
import os
from .utils import Util
from config import plugin_config
@plugins.register(
@@ -69,7 +70,7 @@ class LinkAI(Plugin):
return
if (context.type == ContextType.SHARING and self._is_summary_open(context)) or \
(context.type == ContextType.TEXT and LinkSummary().check_url(context.content)):
(context.type == ContextType.TEXT and self._is_summary_open(context) and LinkSummary().check_url(context.content)):
if not LinkSummary().check_url(context.content):
return
_send_info(e_context, "正在为你加速生成摘要,请稍后")
@@ -196,7 +197,7 @@ class LinkAI(Plugin):
if context.kwargs.get("isgroup") and not self.sum_config.get("group_enabled"):
return False
support_type = self.sum_config.get("type") or ["FILE", "SHARING"]
if context.type.name not in support_type:
if context.type.name not in support_type and context.type.name != "TEXT":
return False
return True
@@ -253,6 +254,7 @@ class LinkAI(Plugin):
plugin_conf = json.load(f)
plugin_conf["midjourney"]["enabled"] = False
plugin_conf["summary"]["enabled"] = False
plugin_config["linkai"] = plugin_conf
return plugin_conf
except Exception as e:
logger.exception(e)

View File

@@ -68,7 +68,7 @@ class MJTask:
# midjourney bot
class MJBot:
def __init__(self, config):
self.base_url = conf().get("linkai_api_base", "https://api.link-ai.chat") + "/v1/img/midjourney"
self.base_url = conf().get("linkai_api_base", "https://api.link-ai.tech") + "/v1/img/midjourney"
self.headers = {"Authorization": "Bearer " + conf().get("linkai_api_key")}
self.config = config
self.tasks = {}

View File

@@ -61,7 +61,7 @@ class LinkSummary:
return None
def base_url(self):
return conf().get("linkai_api_base", "https://api.link-ai.chat")
return conf().get("linkai_api_base", "https://api.link-ai.tech")
def headers(self):
return {"Authorization": "Bearer " + conf().get("linkai_api_key")}

View File

@@ -27,6 +27,10 @@
"Blackroom": {
"url": "https://github.com/dividduang/blackroom.git",
"desc": "小黑屋插件,被拉进小黑屋的人将不能使用@bot的功能的插件"
},
"midjourney": {
"url": "https://github.com/baojingyu/midjourney.git",
"desc": "利用midjourney实现ai绘图的的插件"
}
}
}

View File

@@ -7,4 +7,4 @@ chardet>=5.1.0
Pillow
pre-commit
web.py
linkai>=0.0.5.0
linkai>=0.0.6.0

View File

@@ -62,7 +62,7 @@ class BaiduVoice(Voice):
# 识别本地文件
logger.debug("[Baidu] voice file name={}".format(voice_file))
pcm = get_pcm_from_wav(voice_file)
res = self.client.asr(pcm, "pcm", 8000, {"dev_pid": self.dev_id})
res = self.client.asr(pcm, "pcm", 16000, {"dev_pid": self.dev_id})
if res["err_no"] == 0:
logger.info("百度语音识别到了:{}".format(res["result"]))
text = "".join(res["result"])

View File

@@ -46,4 +46,8 @@ def create_voice(voice_type):
from voice.edge.edge_voice import EdgeVoice
return EdgeVoice()
elif voice_type == "xunfei":
from voice.xunfei.xunfei_voice import XunfeiVoice
return XunfeiVoice()
raise RuntimeError

View File

@@ -19,7 +19,7 @@ class LinkAIVoice(Voice):
def voiceToText(self, voice_file):
logger.debug("[LinkVoice] voice file name={}".format(voice_file))
try:
url = conf().get("linkai_api_base", "https://api.link-ai.chat") + "/v1/audio/transcriptions"
url = conf().get("linkai_api_base", "https://api.link-ai.tech") + "/v1/audio/transcriptions"
headers = {"Authorization": "Bearer " + conf().get("linkai_api_key")}
model = None
if not conf().get("text_to_voice") or conf().get("voice_to_text") == "openai":
@@ -54,7 +54,7 @@ class LinkAIVoice(Voice):
def textToVoice(self, text):
try:
url = conf().get("linkai_api_base", "https://api.link-ai.chat") + "/v1/audio/speech"
url = conf().get("linkai_api_base", "https://api.link-ai.tech") + "/v1/audio/speech"
headers = {"Authorization": "Bearer " + conf().get("linkai_api_key")}
model = const.TTS_1
if not conf().get("text_to_voice") or conf().get("text_to_voice") in ["openai", const.TTS_1, const.TTS_1_HD]:

View File

@@ -0,0 +1,7 @@
{
"APPID":"xxx71xxx",
"APIKey":"xxxx69058exxxxxx",
"APISecret":"xxxx697f0xxxxxx",
"BusinessArgsTTS":{"aue": "lame", "sfl": 1, "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"},
"BusinessArgsASR":{"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vad_eos":10000, "dwa": "wpgs"}
}

209
voice/xunfei/xunfei_asr.py Normal file
View File

@@ -0,0 +1,209 @@
# -*- coding:utf-8 -*-
#
# Author: njnuko
# Email: njnuko@163.com
#
# 这个文档是基于官方的demo来改的固体官方demo文档请参考官网
#
# 语音听写流式 WebAPI 接口调用示例 接口文档必看https://doc.xfyun.cn/rest_api/语音听写(流式版).html
# webapi 听写服务参考帖子必看http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=38947&extra=
# 语音听写流式WebAPI 服务热词使用方式登陆开放平台https://www.xfyun.cn/后,找到控制台--我的应用---语音听写(流式)---服务管理--个性化热词,
# 设置热词
# 注意:热词只能在识别的时候会增加热词的识别权重,需要注意的是增加相应词条的识别率,但并不是绝对的,具体效果以您测试为准。
# 语音听写流式WebAPI 服务方言试用方法登陆开放平台https://www.xfyun.cn/后,找到控制台--我的应用---语音听写(流式)---服务管理--识别语种列表
# 可添加语种或方言,添加后会显示该方言的参数值
# 错误码链接https://www.xfyun.cn/document/error-code code返回错误码时必看
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
import websocket
import datetime
import hashlib
import base64
import hmac
import json
from urllib.parse import urlencode
import time
import ssl
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
import _thread as thread
import os
import wave
STATUS_FIRST_FRAME = 0 # 第一帧的标识
STATUS_CONTINUE_FRAME = 1 # 中间帧标识
STATUS_LAST_FRAME = 2 # 最后一帧的标识
#############
#whole_dict 是用来存储返回值的由于带语音修正所以用dict来存储有更新的化pop之前的值最后再合并
global whole_dict
#这个文档是官方文档改的,这个参数是用来做函数调用时用的
global wsParam
##############
class Ws_Param(object):
# 初始化
def __init__(self, APPID, APIKey, APISecret,BusinessArgs, AudioFile):
self.APPID = APPID
self.APIKey = APIKey
self.APISecret = APISecret
self.AudioFile = AudioFile
self.BusinessArgs = BusinessArgs
# 公共参数(common)
self.CommonArgs = {"app_id": self.APPID}
# 业务参数(business),更多个性化参数可在官网查看
#self.BusinessArgs = {"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vinfo":1,"vad_eos":10000}
# 生成url
def create_url(self):
url = 'wss://ws-api.xfyun.cn/v2/iat'
# 生成RFC1123格式的时间戳
now = datetime.now()
date = format_date_time(mktime(now.timetuple()))
# 拼接字符串
signature_origin = "host: " + "ws-api.xfyun.cn" + "\n"
signature_origin += "date: " + date + "\n"
signature_origin += "GET " + "/v2/iat " + "HTTP/1.1"
# 进行hmac-sha256进行加密
signature_sha = hmac.new(self.APISecret.encode('utf-8'), signature_origin.encode('utf-8'),
digestmod=hashlib.sha256).digest()
signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8')
authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % (
self.APIKey, "hmac-sha256", "host date request-line", signature_sha)
authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8')
# 将请求的鉴权参数组合为字典
v = {
"authorization": authorization,
"date": date,
"host": "ws-api.xfyun.cn"
}
# 拼接鉴权参数生成url
url = url + '?' + urlencode(v)
#print("date: ",date)
#print("v: ",v)
# 此处打印出建立连接时候的url,参考本demo的时候可取消上方打印的注释比对相同参数时生成的url与自己代码生成的url是否一致
#print('websocket url :', url)
return url
# 收到websocket消息的处理
def on_message(ws, message):
global whole_dict
try:
code = json.loads(message)["code"]
sid = json.loads(message)["sid"]
if code != 0:
errMsg = json.loads(message)["message"]
print("sid:%s call error:%s code is:%s" % (sid, errMsg, code))
else:
temp1 = json.loads(message)["data"]["result"]
data = json.loads(message)["data"]["result"]["ws"]
sn = temp1["sn"]
if "rg" in temp1.keys():
rep = temp1["rg"]
rep_start = rep[0]
rep_end = rep[1]
for sn in range(rep_start,rep_end+1):
#print("before pop",whole_dict)
#print("sn",sn)
whole_dict.pop(sn,None)
#print("after pop",whole_dict)
results = ""
for i in data:
for w in i["cw"]:
results += w["w"]
whole_dict[sn]=results
#print("after add",whole_dict)
else:
results = ""
for i in data:
for w in i["cw"]:
results += w["w"]
whole_dict[sn]=results
#print("sid:%s call success!,data is:%s" % (sid, json.dumps(data, ensure_ascii=False)))
except Exception as e:
print("receive msg,but parse exception:", e)
# 收到websocket错误的处理
def on_error(ws, error):
print("### error:", error)
# 收到websocket关闭的处理
def on_close(ws,a,b):
print("### closed ###")
# 收到websocket连接建立的处理
def on_open(ws):
global wsParam
def run(*args):
frameSize = 8000 # 每一帧的音频大小
intervel = 0.04 # 发送音频间隔(单位:s)
status = STATUS_FIRST_FRAME # 音频的状态信息,标识音频是第一帧,还是中间帧、最后一帧
with wave.open(wsParam.AudioFile, "rb") as fp:
while True:
buf = fp.readframes(frameSize)
# 文件结束
if not buf:
status = STATUS_LAST_FRAME
# 第一帧处理
# 发送第一帧音频带business 参数
# appid 必须带上,只需第一帧发送
if status == STATUS_FIRST_FRAME:
d = {"common": wsParam.CommonArgs,
"business": wsParam.BusinessArgs,
"data": {"status": 0, "format": "audio/L16;rate=16000","audio": str(base64.b64encode(buf), 'utf-8'), "encoding": "raw"}}
d = json.dumps(d)
ws.send(d)
status = STATUS_CONTINUE_FRAME
# 中间帧处理
elif status == STATUS_CONTINUE_FRAME:
d = {"data": {"status": 1, "format": "audio/L16;rate=16000",
"audio": str(base64.b64encode(buf), 'utf-8'),
"encoding": "raw"}}
ws.send(json.dumps(d))
# 最后一帧处理
elif status == STATUS_LAST_FRAME:
d = {"data": {"status": 2, "format": "audio/L16;rate=16000",
"audio": str(base64.b64encode(buf), 'utf-8'),
"encoding": "raw"}}
ws.send(json.dumps(d))
time.sleep(1)
break
# 模拟音频采样间隔
time.sleep(intervel)
ws.close()
thread.start_new_thread(run, ())
#提供给xunfei_voice调用的函数
def xunfei_asr(APPID,APISecret,APIKey,BusinessArgsASR,AudioFile):
global whole_dict
global wsParam
whole_dict = {}
wsParam1 = Ws_Param(APPID=APPID, APISecret=APISecret,
APIKey=APIKey,BusinessArgs=BusinessArgsASR,
AudioFile=AudioFile)
#wsParam是global变量给上面on_open函数调用使用的
wsParam = wsParam1
websocket.enableTrace(False)
wsUrl = wsParam.create_url()
ws = websocket.WebSocketApp(wsUrl, on_message=on_message, on_error=on_error, on_close=on_close)
ws.on_open = on_open
ws.run_forever(sslopt={"cert_reqs": ssl.CERT_NONE})
#把字典的值合并起来做最后识别的输出
whole_words = ""
for i in sorted(whole_dict.keys()):
whole_words += whole_dict[i]
return whole_words

163
voice/xunfei/xunfei_tts.py Normal file
View File

@@ -0,0 +1,163 @@
# -*- coding:utf-8 -*-
#
# Author: njnuko
# Email: njnuko@163.com
#
# 这个文档是基于官方的demo来改的固体官方demo文档请参考官网
#
# 语音听写流式 WebAPI 接口调用示例 接口文档必看https://doc.xfyun.cn/rest_api/语音听写(流式版).html
# webapi 听写服务参考帖子必看http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=38947&extra=
# 语音听写流式WebAPI 服务热词使用方式登陆开放平台https://www.xfyun.cn/后,找到控制台--我的应用---语音听写(流式)---服务管理--个性化热词,
# 设置热词
# 注意:热词只能在识别的时候会增加热词的识别权重,需要注意的是增加相应词条的识别率,但并不是绝对的,具体效果以您测试为准。
# 语音听写流式WebAPI 服务方言试用方法登陆开放平台https://www.xfyun.cn/后,找到控制台--我的应用---语音听写(流式)---服务管理--识别语种列表
# 可添加语种或方言,添加后会显示该方言的参数值
# 错误码链接https://www.xfyun.cn/document/error-code code返回错误码时必看
# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
import websocket
import datetime
import hashlib
import base64
import hmac
import json
from urllib.parse import urlencode
import time
import ssl
from wsgiref.handlers import format_date_time
from datetime import datetime
from time import mktime
import _thread as thread
import os
STATUS_FIRST_FRAME = 0 # 第一帧的标识
STATUS_CONTINUE_FRAME = 1 # 中间帧标识
STATUS_LAST_FRAME = 2 # 最后一帧的标识
#############
#这个参数是用来做输出文件路径的
global outfile
#这个文档是官方文档改的,这个参数是用来做函数调用时用的
global wsParam
##############
class Ws_Param(object):
# 初始化
def __init__(self, APPID, APIKey, APISecret,BusinessArgs,Text):
self.APPID = APPID
self.APIKey = APIKey
self.APISecret = APISecret
self.BusinessArgs = BusinessArgs
self.Text = Text
# 公共参数(common)
self.CommonArgs = {"app_id": self.APPID}
# 业务参数(business),更多个性化参数可在官网查看
#self.BusinessArgs = {"aue": "raw", "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"}
self.Data = {"status": 2, "text": str(base64.b64encode(self.Text.encode('utf-8')), "UTF8")}
#使用小语种须使用以下方式此处的unicode指的是 utf16小端的编码方式即"UTF-16LE"”
#self.Data = {"status": 2, "text": str(base64.b64encode(self.Text.encode('utf-16')), "UTF8")}
# 生成url
def create_url(self):
url = 'wss://tts-api.xfyun.cn/v2/tts'
# 生成RFC1123格式的时间戳
now = datetime.now()
date = format_date_time(mktime(now.timetuple()))
# 拼接字符串
signature_origin = "host: " + "ws-api.xfyun.cn" + "\n"
signature_origin += "date: " + date + "\n"
signature_origin += "GET " + "/v2/tts " + "HTTP/1.1"
# 进行hmac-sha256进行加密
signature_sha = hmac.new(self.APISecret.encode('utf-8'), signature_origin.encode('utf-8'),
digestmod=hashlib.sha256).digest()
signature_sha = base64.b64encode(signature_sha).decode(encoding='utf-8')
authorization_origin = "api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"" % (
self.APIKey, "hmac-sha256", "host date request-line", signature_sha)
authorization = base64.b64encode(authorization_origin.encode('utf-8')).decode(encoding='utf-8')
# 将请求的鉴权参数组合为字典
v = {
"authorization": authorization,
"date": date,
"host": "ws-api.xfyun.cn"
}
# 拼接鉴权参数生成url
url = url + '?' + urlencode(v)
# print("date: ",date)
# print("v: ",v)
# 此处打印出建立连接时候的url,参考本demo的时候可取消上方打印的注释比对相同参数时生成的url与自己代码生成的url是否一致
# print('websocket url :', url)
return url
def on_message(ws, message):
#输出文件
global outfile
try:
message =json.loads(message)
code = message["code"]
sid = message["sid"]
audio = message["data"]["audio"]
audio = base64.b64decode(audio)
status = message["data"]["status"]
if status == 2:
print("ws is closed")
ws.close()
if code != 0:
errMsg = message["message"]
print("sid:%s call error:%s code is:%s" % (sid, errMsg, code))
else:
with open(outfile, 'ab') as f:
f.write(audio)
except Exception as e:
print("receive msg,but parse exception:", e)
# 收到websocket连接建立的处理
def on_open(ws):
global outfile
global wsParam
def run(*args):
d = {"common": wsParam.CommonArgs,
"business": wsParam.BusinessArgs,
"data": wsParam.Data,
}
d = json.dumps(d)
# print("------>开始发送文本数据")
ws.send(d)
if os.path.exists(outfile):
os.remove(outfile)
thread.start_new_thread(run, ())
# 收到websocket错误的处理
def on_error(ws, error):
print("### error:", error)
# 收到websocket关闭的处理
def on_close(ws):
print("### closed ###")
def xunfei_tts(APPID, APIKey, APISecret,BusinessArgsTTS, Text, OutFile):
global outfile
global wsParam
outfile = OutFile
wsParam1 = Ws_Param(APPID,APIKey,APISecret,BusinessArgsTTS,Text)
wsParam = wsParam1
websocket.enableTrace(False)
wsUrl = wsParam.create_url()
ws = websocket.WebSocketApp(wsUrl, on_message=on_message, on_error=on_error, on_close=on_close)
ws.on_open = on_open
ws.run_forever(sslopt={"cert_reqs": ssl.CERT_NONE})
return outfile

View File

@@ -0,0 +1,86 @@
#####################################################################
# xunfei voice service
# Auth: njnuko
# Email: njnuko@163.com
#
# 要使用本模块, 首先到 xfyun.cn 注册一个开发者账号,
# 之后创建一个新应用, 然后在应用管理的语音识别或者语音合同右边可以查看APPID API Key 和 Secret Key
# 然后在 config.json 中填入这三个值
#
# 配置说明:
# {
# "APPID":"xxx71xxx",
# "APIKey":"xxxx69058exxxxxx", #讯飞xfyun.cn控制台语音合成或者听写界面的APIKey
# "APISecret":"xxxx697f0xxxxxx", #讯飞xfyun.cn控制台语音合成或者听写界面的APIKey
# "BusinessArgsTTS":{"aue": "lame", "sfl": 1, "auf": "audio/L16;rate=16000", "vcn": "xiaoyan", "tte": "utf8"}, #语音合成的参数具体可以参考xfyun.cn的文档
# "BusinessArgsASR":{"domain": "iat", "language": "zh_cn", "accent": "mandarin", "vad_eos":10000, "dwa": "wpgs"} #语音听写的参数具体可以参考xfyun.cn的文档
# }
#####################################################################
import json
import os
import time
from bridge.reply import Reply, ReplyType
from common.log import logger
from common.tmp_dir import TmpDir
from config import conf
from voice.voice import Voice
from .xunfei_asr import xunfei_asr
from .xunfei_tts import xunfei_tts
from voice.audio_convert import any_to_mp3
import shutil
from pydub import AudioSegment
class XunfeiVoice(Voice):
def __init__(self):
try:
curdir = os.path.dirname(__file__)
config_path = os.path.join(curdir, "config.json")
conf = None
with open(config_path, "r") as fr:
conf = json.load(fr)
print(conf)
self.APPID = str(conf.get("APPID"))
self.APIKey = str(conf.get("APIKey"))
self.APISecret = str(conf.get("APISecret"))
self.BusinessArgsTTS = conf.get("BusinessArgsTTS")
self.BusinessArgsASR= conf.get("BusinessArgsASR")
except Exception as e:
logger.warn("XunfeiVoice init failed: %s, ignore " % e)
def voiceToText(self, voice_file):
# 识别本地文件
try:
logger.debug("[Xunfei] voice file name={}".format(voice_file))
#print("voice_file===========",voice_file)
#print("voice_file_type===========",type(voice_file))
#mp3_name, file_extension = os.path.splitext(voice_file)
#mp3_file = mp3_name + ".mp3"
#pcm_data=get_pcm_from_wav(voice_file)
#mp3_name, file_extension = os.path.splitext(voice_file)
#AudioSegment.from_wav(voice_file).export(mp3_file, format="mp3")
#shutil.copy2(voice_file, 'tmp/test1.wav')
#shutil.copy2(mp3_file, 'tmp/test1.mp3')
#print("voice and mp3 file",voice_file,mp3_file)
text = xunfei_asr(self.APPID,self.APISecret,self.APIKey,self.BusinessArgsASR,voice_file)
logger.info("讯飞语音识别到了: {}".format(text))
reply = Reply(ReplyType.TEXT, text)
except Exception as e:
logger.warn("XunfeiVoice init failed: %s, ignore " % e)
reply = Reply(ReplyType.ERROR, "讯飞语音识别出错了;{0}")
return reply
def textToVoice(self, text):
try:
# Avoid the same filename under multithreading
fileName = TmpDir().path() + "reply-" + str(int(time.time())) + "-" + str(hash(text) & 0x7FFFFFFF) + ".mp3"
return_file = xunfei_tts(self.APPID,self.APIKey,self.APISecret,self.BusinessArgsTTS,text,fileName)
logger.info("[Xunfei] textToVoice text={} voice file name={}".format(text, fileName))
reply = Reply(ReplyType.VOICE, fileName)
except Exception as e:
logger.error("[Xunfei] textToVoice error={}".format(fileName))
reply = Reply(ReplyType.ERROR, "抱歉,讯飞语音合成失败")
return reply