Commit Graph

23 Commits

Author SHA1 Message Date
HnBigVolibear
8ddfcbb125 feat(web): add custom provider support for embedding & vision models, and fix memory_get Windows path bug!
1. Embedding model: support custom provider
   - Add "custom" entry to EMBEDDING_VENDORS with supports_dim_param=False
   - Parse custom:<id> credentials and model fallback in agent_initializer
   - Expand custom_providers as custom:<id> entries in Web UI dropdown

2. Vision model: support custom provider
   - Add custom:<id> routing in _route_by_provider_id
   - Add _build_custom_provider reading credentials from custom_providers
   - Expand custom_providers in Web UI dropdown, add validation in _set_vision

3. Fix memory_get Windows path validation bug!
   - str.startswith(path+'/') always False on Windows due to backslashes, So All Users can not use "memory_get" tool in Windows.
   - Use os.path.realpath + os.sep, consistent with MemoryService

4. Fix historical needsModel:false bug preventing embedding provider switch
   - Change embedding needsModel to true in console.js
   - Support custom:<id> resolution in cow_cli /memory status, also for adding custom provider support

Closes #2908, Closes #2880
2026-06-19 18:35:44 +08:00
christop
ea47f3097e fix(web_fetch): add SSRF guard for model-supplied URLs
web_fetch fetched any http/https URL a model emitted, checking only the
scheme. It performed requests.get(..., allow_redirects=True) with no
hostname resolution check and no private/loopback/link-local/cloud-metadata
filtering, and never re-validated redirect targets. A model (including one
under prompt injection) could make CowAgent fetch 127.0.0.1, RFC1918,
169.254.169.254 or other internal endpoints and return their bodies into
the conversation; a public URL could also 302-bounce into a private target.

The repo already shipped an SSRF validator for the vision tool
(Vision._validate_url_safe). Extract that logic into a shared helper
(agent/tools/utils/url_safety.py) and reuse it:

- execute() now validates the URL before dispatching to either fetch path.
- A new _safe_get() helper disables auto-redirect and follows redirects
  manually, re-validating every hop so a public URL cannot bounce into an
  internal address. Both the webpage and document fetch paths use it.
- Vision._validate_url_safe now delegates to the shared helper (public API
  unchanged), so both URL-consuming tools share one guard.

Stdlib only (ipaddress, socket, urllib.parse); no new dependency. Adds
tests/test_security_ssrf_web_fetch.py covering loopback, cloud-metadata,
RFC1918 and a public->loopback redirect.

Sink: agent/tools/web_fetch/web_fetch.py (_fetch_webpage / _fetch_document).
Signed-off-by: christop <825583681@qq.com>
2026-06-17 17:38:35 +08:00
kirs-hi
e85290cddc fix(security): SSRF protection for vision tool + path traversal guard for skill install
1. Vision SSRF (#2878, #2872):
   Add _validate_url_safe() that resolves the target hostname via DNS and
   rejects any IP in private (RFC1918), loopback, link-local, or reserved
   ranges before requests.get() is called. This blocks attacks that use
   attacker-controlled image URLs to probe internal services or cloud
   metadata endpoints (169.254.169.254).

2. Skill install path traversal (#2873):
   Add _safe_skill_dir() that validates the skill name cannot escape the
   skills/ root directory. Rejects names containing '..', absolute paths,
   and any resolved path that falls outside the custom_dir boundary.
   Applied to _add_url(), _add_package(), and delete().

Both fixes include comprehensive unit tests (19 test cases) covering
blocked patterns, edge cases, and allowed legitimate usage.

Closes #2878
Closes #2873
Ref: #2872
2026-06-11 17:40:41 +08:00
zhayujie
08e23e5bd8 fix(vision): bump vision timeout from 60s to 180s to avoid premature failures 2026-06-09 16:36:01 +08:00
zhayujie
e812c7d29a feat(vision): increase vision tool max_tokens 2026-06-09 16:08:17 +08:00
zhayujie
92ec9653e5 feat(models): support qwen3.7-plus multi-modal model 2026-06-02 16:38:17 +08:00
zhayujie
bccce2d7cb feat(models): support xiaomi mimo 2026-05-28 10:49:52 +08:00
zhayujie
bd85fee7d7 fix(models): persist explicit provider for vision and image capabilities 2026-05-23 20:43:25 +08:00
zhayujie
571897e2fd fix: modify default model in vision tool 2026-05-22 18:18:16 +08:00
zhayujie
2517f2add8 feat(models): support gpt-5.5 2026-05-22 11:04:55 +08:00
zhayujie
8c25395805 feat(models): support gemini-3.5-flash 2026-05-22 10:39:04 +08:00
zhayujie
b8333e351c feat(voice): rework TTS/ASR stack and unify tool/skill config schema 2026-05-21 16:00:54 +08:00
zhayujie
a5790d82f6 feat(qianfan): scope vision support to multimodal models 2026-05-06 16:11:10 +08:00
jimmyzhuu
fb7962c7f2 fix: use available qianfan vision model 2026-05-06 13:34:39 +08:00
jimmyzhuu
fccb7ff9ed feat: route qianfan vision provider 2026-05-06 13:25:59 +08:00
zhayujie
80e9062041 fix(vision): respect tool.vision.model and add automatic fallback #2792 2026-05-03 22:28:32 +08:00
zhayujie
2c13e1b923 feat(models): support kimi-k2.6 2026-04-22 12:01:40 +08:00
zhayujie
26693acc3f feat(vision): prioritize main model for image recognition with multi-provider fallback
- Add call_vision method to all bot implementations (DashScope, Claude,
  Gemini, ZhipuAI, MiniMax, Doubao, Moonshot, OpenAICompatibleBot)
  using each vendor's native multimodal API format
- Remove call_with_tools/call_vision from Bot base class to fix MRO
  shadowing issue with OpenAICompatibleBot mixin
- Refactor vision tool provider resolution: MainModel → other configured
  models (auto-discovered) → OpenAI → LinkAI, with automatic fallback
- Return actual model name used in call_vision responses
- Sync config.json API keys to .env bidirectionally on startup
- Fix bot instance cache to detect bot_type/use_linkai config changes
- Add SSE reconnection support for web console
- Preserve image path hints in Gemini text for correct vision tool calls
- Update docs/tools/vision.mdx
2026-04-11 19:46:11 +08:00
zhayujie
ad86deb014 fix: prioritize using a custom master model for vision 2026-04-08 15:16:59 +08:00
zhayujie
22b8ca0095 feat: optimize vision image compression 2026-03-23 21:18:04 +08:00
zhayujie
b4e711f411 feat: add request header 2026-03-19 17:06:05 +08:00
zhayujie
b8437032e9 fix: optimize image recognition prompts 2026-03-18 10:10:23 +08:00
zhayujie
a50fafaca2 refactor: convert image vision from skill to native tool 2026-03-09 16:01:56 +08:00