The web_search tool was not respecting HTTP_PROXY/HTTPS_PROXY environment
variables, causing 'fetch failed' errors when running behind a proxy.
This fix adds ProxyAgent support for the Brave Search API, similar to how
other tools in OpenClaw handle proxy configuration.
Fixes#27405
* feat: add Gemini (Google Search grounding) as web_search provider
Add Gemini as a fourth web search provider alongside Brave, Perplexity,
and Grok. Uses Gemini's built-in Google Search grounding tool to return
search results with citations.
- Add runGeminiSearch() with Google Search grounding via tools API
- Resolve Gemini's grounding redirect URLs to direct URLs via parallel
HEAD requests (5s timeout, graceful fallback)
- Add Gemini config block (apiKey, model) with env var fallback
- Default model: gemini-2.5-flash (fast, cheap, grounding-capable)
- Strip API key from error messages for security
- Add config validation tests for Gemini provider
- Update docs/tools/web.md with Gemini provider documentation
Closes#13074
* feat: auto-detect search provider from available API keys
When no explicit provider is configured, resolveSearchProvider now
checks for available API keys in priority order (Brave → Gemini →
Perplexity → Grok) and selects the first provider with a valid key.
- Add auto-detection logic using existing resolve*ApiKey functions
- Export resolveSearchProvider via __testing_provider for tests
- Add 8 tests covering auto-detection, priority order, and explicit override
- Update docs/tools/web.md with auto-detection documentation
* fix: merge __testing exports, downgrade auto-detect log to debug
* fix: use defaultRuntime.log instead of .debug (not in RuntimeEnv type)
* fix: mark gemini apiKey as sensitive in zod schema
* fix: address Greptile review — add externalContent to Gemini payload, add Gemini/Grok entries to schema labels/help, remove dead schema-fields.ts
* fix(web-search): add JSON parse guard for Gemini API responses
Addresses Greptile review comment: add try/catch to handle non-JSON
responses from Gemini API gracefully, preventing runtime errors on
malformed responses.
Note: FIELD_HELP entries for gemini.apiKey and gemini.model were
already present in schema.help.ts, and gemini.apiKey was already
marked as sensitive in zod-schema.agent-runtime.ts (both fixed in
earlier commits).
* fix: use structured readResponseText result in Gemini error path
readResponseText returns { text, truncated, bytesRead }, not a string.
The Gemini error handler was using the result object directly, which
would always be truthy and never fall through to res.statusText.
Align with Perplexity/xAI/Brave error patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* style: fix import order and formatting after rebase onto main
* Web search: send Gemini API key via header
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
Add optional urlAllowlist config at tools.web level that restricts which
URLs can be accessed by web tools:
- Config types (types.tools.ts): Add urlAllowlist?: string[] to tools.web
- Zod schema: Add urlAllowlist field to ToolsWebSchema
- Schema help: Add help text for the new config fields
- web_search: Filter Brave search results by allowlist (provider=brave)
- web_fetch: Block URLs not matching allowlist before fetching
- ssrf.ts: Export normalizeHostnameAllowlist and matchesHostnameAllowlist
URL matching supports:
- Exact domain match (example.com)
- Wildcard patterns (*.github.com)
When urlAllowlist is not configured, all URLs are allowed (backwards compatible).
Tests: Add web-tools.url-allowlist.test.ts with 23 tests covering:
- URL allowlist resolution from config
- Wildcard pattern matching
- web_fetch error response format
- Brave search result filtering
The xAI /v1/responses API returns content in a structured format with
typed output blocks (type: 'message') containing typed content blocks
(type: 'output_text') and url_citation annotations. The previous code
only checked output[0].content[0].text without filtering by type,
which could miss content in responses with multiple output entries.
Changes:
- Update GrokSearchResponse type to include annotations on content blocks
- Filter output blocks by type='message' and content by type='output_text'
- Extract url_citation annotations as fallback citations when top-level
citations array is empty
- Deduplicate annotation-derived citation URLs
- Update tests for the new structured return type
Closes#13520
xAI's /v1/responses endpoint does not support the 'include' parameter,
returning 400 'Argument not supported: include'. Inline citations are
returned automatically when available — no explicit request needed.
Closes#12910
Co-authored-by: Luna AI <luna@coredirection.ai>
* fix(tools): correct Grok response parsing for xAI Responses API
The xAI Responses API returns content in output[0].content[0].text,
not in output_text field. Updated GrokSearchResponse type and
runGrokSearch to extract content from the correct path.
Fixes the 'No response' issue when using Grok web search.
* fix(tools): harden Grok web_search parsing (#13049) (thanks @ereid7)
---------
Co-authored-by: erai <erai@erais-Mac-mini.local>
Co-authored-by: Tak Hoffman <781889+Takhoffman@users.noreply.github.com>
* fix(web_search): Fix invalid model name sent to Perplexity
* chore: Only apply fix to direct Perplexity calls
* fix(web_search): normalize direct Perplexity model IDs
* fix: add changelog note for perplexity model normalization (#12795) (thanks @cdorsey)
* fix: align tests and fetch type for gate stability (#12795) (thanks @cdorsey)
* chore: keep #12795 scoped to web_search changes
---------
Co-authored-by: Sebastian <19554889+sebslight@users.noreply.github.com>
Add xAI's Grok as a new web_search provider alongside Brave and Perplexity.
Uses the xAI /v1/responses API with tools: [{type: "web_search"}].
Configuration:
- tools.web.search.provider: "grok"
- tools.web.search.grok.apiKey or XAI_API_KEY env var
- tools.web.search.grok.model (default: grok-4-1-fast)
- tools.web.search.grok.inlineCitations (optional, embeds markdown links)
Returns AI-synthesized answers with citations similar to Perplexity.
Adds support for Brave Search API's freshness parameter to filter results
by discovery time:
- 'pd' - Past 24 hours
- 'pw' - Past week
- 'pm' - Past month
- 'py' - Past year
- 'YYYY-MM-DDtoYYYY-MM-DD' - Custom date range
Useful for cron jobs and monitoring tasks that need recent results.
Note: Perplexity provider ignores this parameter (Brave only).
---
🤖 AI-assisted: This PR was created with Claude (Opus). Lightly tested via
build script. The change follows existing patterns for optional parameters
(country, search_lang, ui_lang).
