kaiyuan/chatgpt-on-wechat
1
0
Fork 0
mirror of https://github.com/zhayujie/chatgpt-on-wechat.git synced 2026-07-21 06:07:13 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: release 2.0.9

Browse Source
This commit is contained in:
zhayujie
2026-05-22 12:25:22 +08:00
parent cc10d230b0
commit 069bffa3e8
44 changed files with 1756 additions and 744 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
docs/en/tools/vision.mdx
View File

@@ -1,5 +1,5 @@
---
title: vision - Image Analysis
title: vision - Image Understanding
description: Analyze image content (recognition, description, OCR, etc.)
---
@@ -9,33 +9,49 @@ Analyze local images or image URLs using Vision API. Supports content descriptio
The vision tool uses a multi-level auto-selection strategy with automatic fallback — no manual configuration required:
1. **Main model** — uses the currently configured main model for image recognition (zero extra cost)
2. **Other configured models** — auto-discovers other models with configured API keys as alternatives
3. **OpenAI** — uses `open_ai_api_key` to call gpt-4.1-mini
4. **LinkAI** — uses `linkai_api_key` to call LinkAI vision service
When `use_linkai=true`, LinkAI is promoted to the highest priority.
1. **Main model** — uses the currently configured main model for image recognition (must be a multimodal model)
2. **Other configured models** — auto-discovers other multimodal models with configured API keys as alternatives
If the current provider fails, the tool automatically tries the next one until it succeeds or all fail.
### Supported Models
| Vendor | Vision Model | Notes |
| Provider | Vision Model | Notes |
| --- | --- | --- |
| OpenAI / Compatible | Main model | All OpenAI-compatible multimodal models |
| Baidu Qianfan | Main model | Multimodal main models (e.g. `ernie-5.1`) handle images directly; falls back to `ernie-4.5-turbo-vl` for text-only main models |
| Qwen (DashScope) | Main model | Via MultiModalConversation API |
| OpenAI / Compatible | Main model | All OpenAI-protocol-compatible multimodal models |
| Qwen (DashScope) | Main model | e.g. qwen3.6-plus, etc. |
| Claude | Main model | Anthropic native image format |
| Gemini | Main model | inlineData format |
| Doubao | Main model | doubao-seed-2-0 series natively supported |
| Kimi (Moonshot) | Main model | kimi-k2.6, kimi-k2.5 natively supported |
| ZhipuAI | glm-5v-turbo | Always uses dedicated vision model |
| MiniMax | MiniMax-Text-01 | Always uses dedicated vision model |
| Baidu Qianfan | Main model | Defaults to the multimodal main model (e.g. `ernie-5.1`); falls back to `ernie-4.5-turbo-vl` when the main model is not multimodal |
| ZhipuAI | glm-5v-turbo | Always uses the dedicated vision model |
| MiniMax | MiniMax-Text-01 | Always uses the dedicated vision model |
<Note>
ZhipuAI and MiniMax text models do not support image understanding, so their dedicated vision models are always used automatically.
</Note>
> When `use_linkai=true`, LinkAI's multimodal model is used by default.
## Custom Configuration
To specify the model used by Vision, configure it in `config.json`, for example:
```json
{
"tools": {
"vision": {
"model": "gpt-4.1"
}
}
}
```
The specified model is **used first**, and the tool automatically routes to the corresponding provider based on the model name; on failure, it falls back to other configured providers.
In most cases no configuration is needed — the tool works automatically as long as the main model supports multimodal input or any vision-capable API key is configured.
## Parameters
| Parameter | Type | Required | Description |
@@ -45,21 +61,7 @@ If the current provider fails, the tool automatically tries the next one until i
Supported image formats: jpg, jpeg, png, gif, webp
## Custom Configuration
To specify a particular model for the vision tool, add to `config.json`:
```json
{
"tools": {
"vision": {
"model": "ernie-4.5-turbo-vl"
}
}
}
```
In most cases no configuration is needed. The tool works automatically as long as the main model supports multimodal input or any vision-capable API key is configured.
## Use Cases
@@ -69,5 +71,5 @@ In most cases no configuration is needed. The tool works automatically as long a
- Analyze screenshots and scanned documents
<Note>
Images larger than 1MB are automatically compressed (max edge 1536px). All images (including remote URLs) are converted to base64 for transmission to ensure compatibility with all model backends.
Images larger than 1MB are automatically compressed before upload. All images (including remote URLs) are converted to base64 for transmission to ensure compatibility with all model backends.
</Note>