refactor: convert web-fetch from skill to native tool

This commit is contained in:
zhayujie
2026-03-09 10:13:48 +08:00
parent 8623287ac1
commit ccb9030d3c
6 changed files with 110 additions and 111 deletions

View File

@@ -95,7 +95,7 @@ Do NOT create auxiliary documentation files:
## Installing a Skill from URL
1. Fetch the URL content (curl or web-fetch skill)
1. Fetch the URL content (curl or web_fetch tool)
2. Extract `name` from YAML frontmatter
3. Create directory `<workspace>/skills/<name>/` and save content as `SKILL.md`
4. Check the saved SKILL.md for an installation/setup section — if it defines additional steps (e.g., downloading scripts, installing dependencies), execute them; otherwise installation is complete

View File

@@ -1,56 +0,0 @@
---
name: web-fetch
description: Fetch and extract readable content from web pages. Use for lightweight page access without browser automation.
homepage: https://github.com/zhayujie/chatgpt-on-wechat
metadata:
emoji: 🌐
requires:
bins: ["curl"]
always: true
---
# Web Fetch
Fetch and extract readable content from web pages using curl and basic text processing.
## Usage
**Important**: Scripts are located relative to this skill's base directory.
When you see this skill in `<available_skills>`, note the `<base_dir>` path.
```bash
# General pattern:
bash "<base_dir>/scripts/fetch.sh" <url> [output_file]
# Example (replace <base_dir> with actual path from skill listing):
bash "~/chatgpt-on-wechat/skills/web-fetch/scripts/fetch.sh" "https://example.com"
```
**Parameters:**
- `url`: The HTTP/HTTPS URL to fetch (required)
- `output_file`: Optional file to save the output (default: stdout)
**Returns:**
- Extracted page content with title and text
## Examples
### Fetch a web page
```bash
bash "<base_dir>/scripts/fetch.sh" "https://example.com"
```
### Save to file
```bash
bash "<base_dir>/scripts/fetch.sh" "https://example.com" output.txt
cat output.txt
```
## Notes
- Uses curl for HTTP requests (timeout: 10s)
- Extracts title and basic text content
- Removes HTML tags and scripts
- Works with any standard web page
- No external dependencies beyond curl

View File

@@ -1,54 +0,0 @@
#!/usr/bin/env bash
# Fetch and extract readable content from a web page
set -euo pipefail
url="${1:-}"
output_file="${2:-}"
if [ -z "$url" ]; then
echo "Error: URL is required"
echo "Usage: bash fetch.sh <url> [output_file]"
exit 1
fi
# Validate URL
if [[ ! "$url" =~ ^https?:// ]]; then
echo "Error: Invalid URL (must start with http:// or https://)"
exit 1
fi
# Fetch the page with curl
html=$(curl -sS -L --max-time 10 \
-H "User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36" \
-H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8" \
"$url" 2>&1) || {
echo "Error: Failed to fetch URL: $url"
exit 1
}
# Extract title
title=$(echo "$html" | grep -oP '(?<=<title>).*?(?=</title>)' | head -1 || echo "Untitled")
# Remove script and style tags
text=$(echo "$html" | sed 's/<script[^>]*>.*<\/script>//gI' | sed 's/<style[^>]*>.*<\/style>//gI')
# Remove HTML tags
text=$(echo "$text" | sed 's/<[^>]*>//g')
# Clean up whitespace
text=$(echo "$text" | tr -s ' ' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
# Format output
result="Title: $title
Content:
$text"
# Output to file or stdout
if [ -n "$output_file" ]; then
echo "$result" > "$output_file"
echo "Content saved to: $output_file"
else
echo "$result"
fi