fix(web_fetch): add SSRF guard for model-supplied URLs

web_fetch fetched any http/https URL a model emitted, checking only the
scheme. It performed requests.get(..., allow_redirects=True) with no
hostname resolution check and no private/loopback/link-local/cloud-metadata
filtering, and never re-validated redirect targets. A model (including one
under prompt injection) could make CowAgent fetch 127.0.0.1, RFC1918,
169.254.169.254 or other internal endpoints and return their bodies into
the conversation; a public URL could also 302-bounce into a private target.

The repo already shipped an SSRF validator for the vision tool
(Vision._validate_url_safe). Extract that logic into a shared helper
(agent/tools/utils/url_safety.py) and reuse it:

- execute() now validates the URL before dispatching to either fetch path.
- A new _safe_get() helper disables auto-redirect and follows redirects
  manually, re-validating every hop so a public URL cannot bounce into an
  internal address. Both the webpage and document fetch paths use it.
- Vision._validate_url_safe now delegates to the shared helper (public API
  unchanged), so both URL-consuming tools share one guard.

Stdlib only (ipaddress, socket, urllib.parse); no new dependency. Adds
tests/test_security_ssrf_web_fetch.py covering loopback, cloud-metadata,
RFC1918 and a public->loopback redirect.

Sink: agent/tools/web_fetch/web_fetch.py (_fetch_webpage / _fetch_document).
Signed-off-by: christop <825583681@qq.com>
This commit is contained in:
christop
2026-06-17 00:59:53 +08:00
parent e3dce45b2a
commit ea47f3097e
5 changed files with 325 additions and 39 deletions

View File

@@ -17,18 +17,16 @@ Provider resolution:
"""
import base64
import ipaddress
import os
import socket
import subprocess
import tempfile
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
from urllib.parse import urlparse
import requests
from agent.tools.base_tool import BaseTool, ToolResult
from agent.tools.utils.url_safety import validate_url_safe
from common import const
from common.log import logger
from config import conf
@@ -665,31 +663,13 @@ class Vision(BaseTool):
into non-public ranges. Also rejects URLs with no host, non-HTTP(S)
schemes, or hosts that fail DNS resolution.
Delegates to the shared ``agent.tools.utils.url_safety`` helper so the
same guard protects every tool that fetches model-supplied URLs.
Raises:
ValueError: if the URL targets a disallowed address.
"""
parsed = urlparse(url)
if parsed.scheme not in ("http", "https"):
raise ValueError(f"Unsupported URL scheme: {parsed.scheme}")
hostname = parsed.hostname
if not hostname:
raise ValueError("URL has no hostname")
try:
# Resolve all addresses for the hostname.
addr_infos = socket.getaddrinfo(hostname, None, socket.AF_UNSPEC, socket.SOCK_STREAM)
except socket.gaierror:
raise ValueError(f"Cannot resolve hostname: {hostname}")
for family, _, _, _, sockaddr in addr_infos:
ip_str = sockaddr[0]
ip = ipaddress.ip_address(ip_str)
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
raise ValueError(
f"URL resolves to a non-public address ({ip_str}), "
f"request blocked for security"
)
validate_url_safe(url)
def _build_image_content(self, image: str) -> dict:
"""