[Yeachan-Heo]

Fixes the bug reported in issue #3364 where DuckDuckGo challenge pages (HTTP 202 or similar without proper search result markup) would return junk results pointing back to DuckDuckGo itself.

Changes

• Add HTTP status code check to execute_web_search: return explicit error if response is not 2xx (e.g., 202 Accepted, 429 Too Many Requests, etc.) instead of proceeding with potentially invalid results
• Filter out links pointing back to the search engine host in the generic link fallback to avoid returning search engine self-links
• Keep CLAWD_WEB_SEARCH_BASE_URL behavior working unchanged
• Add regression tests for both scenarios

What this fixes

• When the search provider returns a non-success HTTP status, web search now fails with a clear error message instead of fabricating junk results
• When the search provider returns HTML without proper search result markup, the generic link fallback now filters out links back to the search engine itself

Refs #3364

Note: The optional provider proposal from #3364 (CLAWD_WEB_SEARCH_PROVIDER=keenable) is left for maintainer review as a separate product decision.

[XinwangTao]

Thanks for the fix — rejecting non-2xx responses instead of extracting junk from DuckDuckGo challenge pages, plus dropping self-referencing search-engine links, matches the symptom we hit (every connection returning HTTP 202). One note: the opt-in keyless API backend proposal from the triage issue is still unaddressed — worth tracking it as a separate issue so the 202 handling and the API backend stay decoupled.

[XinwangTao]

Thanks for the fix — rejecting non-2xx responses instead of extracting junk from DuckDuckGo challenge pages, plus dropping self-referencing search-engine links, matches the symptom we hit (every connection returning HTTP 202). One note: the opt-in keyless API backend proposal from the triage issue is still unaddressed — worth tracking it as a separate issue so the 202 handling and the API backend stay decoupled.