feat(tools): add an always-visible built-in fetch tool #20

Merged
sam merged 1 commit from feat/builtin-fetch-tool into main 2026-10-03 13:00:47 +00:00
Collaborator

An agent could only reach the web through an MCP server, so a session with no web-fetching server could not read a URL at all. This adds a fetch tool to the internal/tools package next to the built-in file tools, registered the same way: always visible, so its schema rides in every request with no tools discover/load round trip.

Modelled after the reference MCP fetch server (https://github.com/modelcontextprotocol/servers/tree/main/src/fetch).

What the tool does

fetch(url, maxLength?, startIndex?, raw?, timeout?):

  • Retrieves an http/https URL. Anything else is rejected before a request goes out.
  • HTML is converted to markdown with JohannesKaufmann/html-to-markdown (base, commonmark and table plugins): scripts, styles and other non-content elements are dropped; headings, lists, links, code and tables keep their shape; relative links are made absolute against the fetched page, so a URL that appears in the output can be fetched directly. raw: true returns the body as fetched, and a non-HTML content type is always returned raw behind a prefix saying so.
  • The result is windowed: maxLength defaults to 5000 characters, a shortened result ends by naming the startIndex to continue from, and a start at or past the end is reported as such. Indices count characters, not bytes, so a window never splits a multi-byte character. A body is read up to 8 MiB.
  • The fetch is bounded in time by timeout seconds (default 30, at most 300), applied as one deadline over the robots.txt check and the page together. The HTTP client itself has no timeout, so an earlier caller deadline and a canceled turn still win.
  • The origin's robots.txt is fetched and evaluated first, per RFC 9309 (longest matching pattern wins, Allow wins a tie), so a path the site disallows is refused. Following the reference server: a robots.txt answering 401/403 refuses the fetch, any other 4xx means there are no rules, and a robots.txt that cannot be read at all fails the fetch rather than fetching unchecked.
  • Each request identifies itself as brainless/0.0.1 (+https://git.norobo.chat/sam/brainless), follows redirects, and honors the proxy environment.

Notes for review

  • The shared tool output type is renamed fileResult → toolResult, since the fetch tool now shares it (a pure rename of an unexported type; verified identical after the rename).
  • go mod tidy also bumped the indirect mattn/go-isatty 0.0.20 → 0.0.22.
  • The servers-side readability extraction (readabilipy) is not implemented, so a page's navigation and boilerplate stay in the output. For an agent reading a page that is noise rather than harm, and it is documented in the README; adding go-shiori/go-readability before the conversion is the follow-up if the first window should be article text.

Verification

  • go test ./... passes; new tests cover conversion (incl. tables and absolute links), windowing and rune-safe slicing, content-type sniffing, raw mode, status errors, the timeout argument and a hanging server, robots allow/deny/failure modes, and RFC 9309 pattern matching.
  • racedetector test ./internal/tools/ ./internal/harness/ — no data races.
  • golangci-lint run ./... — 0 issues.
  • Manual smoke test against go.dev and en.wikipedia.org: robots checked, markdown returned, absolute links, truncation hint. That smoke test was a throwaway and is not part of the commit.

README

The new "Built-in fetch tool" section documents the arguments, the robots.txt rules, the time bound, and the readability gap.

An agent could only reach the web through an MCP server, so a session with no web-fetching server could not read a URL at all. This adds a `fetch` tool to the `internal/tools` package next to the built-in file tools, registered the same way: always visible, so its schema rides in every request with no `tools` discover/load round trip. Modelled after the reference MCP fetch server (https://github.com/modelcontextprotocol/servers/tree/main/src/fetch). ### What the tool does `fetch(url, maxLength?, startIndex?, raw?, timeout?)`: - Retrieves an `http`/`https` URL. Anything else is rejected before a request goes out. - HTML is converted to markdown with [JohannesKaufmann/html-to-markdown](https://github.com/JohannesKaufmann/html-to-markdown) (base, commonmark and table plugins): scripts, styles and other non-content elements are dropped; headings, lists, links, code and tables keep their shape; relative links are made absolute against the fetched page, so a URL that appears in the output can be fetched directly. `raw: true` returns the body as fetched, and a non-HTML content type is always returned raw behind a prefix saying so. - The result is windowed: `maxLength` defaults to 5000 characters, a shortened result ends by naming the `startIndex` to continue from, and a start at or past the end is reported as such. Indices count characters, not bytes, so a window never splits a multi-byte character. A body is read up to 8 MiB. - The fetch is bounded in time by `timeout` seconds (default 30, at most 300), applied as one deadline over the robots.txt check and the page together. The HTTP client itself has no timeout, so an earlier caller deadline and a canceled turn still win. - The origin's `robots.txt` is fetched and evaluated first, per RFC 9309 (longest matching pattern wins, `Allow` wins a tie), so a path the site disallows is refused. Following the reference server: a robots.txt answering 401/403 refuses the fetch, any other 4xx means there are no rules, and a robots.txt that cannot be read at all fails the fetch rather than fetching unchecked. - Each request identifies itself as `brainless/0.0.1 (+https://git.norobo.chat/sam/brainless)`, follows redirects, and honors the proxy environment. ### Notes for review - The shared tool output type is renamed `fileResult` → `toolResult`, since the fetch tool now shares it (a pure rename of an unexported type; verified identical after the rename). - `go mod tidy` also bumped the indirect `mattn/go-isatty` 0.0.20 → 0.0.22. - The servers-side readability extraction (readabilipy) is *not* implemented, so a page's navigation and boilerplate stay in the output. For an agent reading a page that is noise rather than harm, and it is documented in the README; adding `go-shiori/go-readability` before the conversion is the follow-up if the first window should be article text. ### Verification - `go test ./...` passes; new tests cover conversion (incl. tables and absolute links), windowing and rune-safe slicing, content-type sniffing, raw mode, status errors, the timeout argument and a hanging server, robots allow/deny/failure modes, and RFC 9309 pattern matching. - `racedetector test ./internal/tools/ ./internal/harness/` — no data races. - `golangci-lint run ./...` — 0 issues. - Manual smoke test against `go.dev` and `en.wikipedia.org`: robots checked, markdown returned, absolute links, truncation hint. That smoke test was a throwaway and is not part of the commit. ### README The new "Built-in fetch tool" section documents the arguments, the robots.txt rules, the time bound, and the readability gap.
feat(tools): add an always-visible built-in fetch tool
All checks were successful
code-review / Agent review (pull_request) Successful in 16s
a0c0465b69
An agent could only reach the web through an MCP server, so a session with
no web-fetching server could not read a URL at all. Add a `fetch` tool to
the internal/tools package, next to the built-in file tools, and register it
the same way: always visible, so its schema rides in every request with no
`tools` discover/load round trip.

Modelled after the reference MCP fetch server
(https://github.com/modelcontextprotocol/servers/tree/main/src/fetch):

- `fetch(url, maxLength?, startIndex?, raw?, timeout?)` retrieves an http/https
  URL.
- HTML is converted to markdown with github.com/JohannesKaufmann/html-to-markdown/v2
  (base, commonmark and table plugins), which drops scripts, styles and other
  non-content elements, keeps headings, lists, links, code and tables, and
  makes relative links absolute against the fetched page. `raw: true` returns
  the body as fetched; a non-HTML content type is always returned raw behind a
  prefix saying so.
- The result is windowed like the reference tool: `maxLength` defaults to 5000
  characters, a short result names the `startIndex` to continue from, and a
  start at or past the end is reported as such. Indices count characters, so a
  window never splits a multi-byte character. A body is read up to 8 MiB.
- The fetch is bounded in time by `timeout` seconds (default 30, at most 300),
  applied as one deadline over the robots.txt check and the page together. The
  HTTP client itself has no timeout, so the caller's deadline and a canceled
  turn still win.
- The origin's robots.txt is fetched and evaluated first (RFC 9309: longest
  matching pattern wins, Allow wins a tie), so a disallowed path is refused.
  Following the reference, a robots.txt answering 401/403 refuses the fetch, any
  other 4xx means no rules, and a robots.txt that cannot be read at all fails
  the fetch rather than fetching unchecked.

Also rename the shared tool output type from fileResult to toolResult, since
the fetch tool now shares it, and extend the harness tests: the end-to-end
round trip asserts the built-in tools are advertised in every model request,
and the tool-kind table pins fetch to ACP's Fetch kind.
pi-reviewer left a comment

Review of feat(tools): add an always-visible built-in fetch tool

Reviewed with the samber/cc-skills-golang pass order (style/naming/docs → correctness/safety/errors/concurrency → security/deps → tests/perf/observability/modernize). The diff was read from the PR head a0c0465 via the Forgejo API; every claim below was verified in this session against the repo's own toolchain (go1.27.1) in a scratch clone at the head commit.

Baseline (all green)

  • go build ./..., go vet ./..., go test ./... — pass (run at head).
  • gofmt -l clean; golangci-lint run ./... — 0 issues.
  • govulncheck ./... — 0 vulnerabilities in code paths; the 3 module-level findings are all in pre-existing golang.org/x/crypto@v0.55.0 (untouched by this PR). No replace directives; html-to-markdown/v2 v2.5.2 and dom v0.3.1 are current, require Go 1.25 (module is 1.27.1), not deprecated.
  • APIs asserted in the diff verified: acp.ToolKindFetch exists in coder/acp-go-sdk (go doc), slices/strings.SplitSeq/min/max exist in this toolchain, omitzero and t.Context() are fine on go 1.27.

Findings

One blocking, one important, four suggestions — all with reproduced evidence. The two redirect findings were reproduced with temporary tests run against the real code at head (test files removed afterwards; command outputs quoted inline). RFC 9309 pattern semantics (/private prefix-matching /privateer, longest-match/Allow-tie rules, §2.3.1.2 redirect handling of robots.txt) were checked against the RFC text and are implemented correctly; the user-agent matching nit (comment #4) and the 5xx robots.txt handling (comment #3) deviate from §2.2.1 and §2.3.1.4 respectively.

The rename fileResult → toolResult is a clean, complete rename (no leftover references). Tests are thorough, table-driven where it matters, use t.Helper()/t.Parallel()/t.Context(). Concurrency is clean: Fetch is stateless, *http.Client is documented safe for concurrent use, context is propagated end to end, and the per-call converter in htmlToMarkdown is a documented, deliberate choice. Error handling wraps with %w and never swallows. Naming, doc comments, and README are excellent.

## Review of feat(tools): add an always-visible built-in fetch tool Reviewed with the samber/cc-skills-golang pass order (style/naming/docs → correctness/safety/errors/concurrency → security/deps → tests/perf/observability/modernize). The diff was read from the PR head `a0c0465` via the Forgejo API; every claim below was verified in this session against the repo's own toolchain (go1.27.1) in a scratch clone at the head commit. ### Baseline (all green) - `go build ./...`, `go vet ./...`, `go test ./...` — pass (run at head). - `gofmt -l` clean; `golangci-lint run ./...` — 0 issues. - `govulncheck ./...` — 0 vulnerabilities in code paths; the 3 module-level findings are all in pre-existing `golang.org/x/crypto@v0.55.0` (untouched by this PR). No `replace` directives; `html-to-markdown/v2 v2.5.2` and `dom v0.3.1` are current, require Go 1.25 (module is 1.27.1), not deprecated. - APIs asserted in the diff verified: `acp.ToolKindFetch` exists in coder/acp-go-sdk (`go doc`), `slices`/`strings.SplitSeq`/`min`/`max` exist in this toolchain, `omitzero` and `t.Context()` are fine on go 1.27. ### Findings One blocking, one important, four suggestions — all with reproduced evidence. The two redirect findings were reproduced with temporary tests run against the real code at head (test files removed afterwards; command outputs quoted inline). RFC 9309 pattern semantics (`/private` prefix-matching `/privateer`, longest-match/Allow-tie rules, §2.3.1.2 redirect handling of robots.txt) were checked against the RFC text and are implemented correctly; the user-agent matching nit (comment #4) and the 5xx robots.txt handling (comment #3) deviate from §2.2.1 and §2.3.1.4 respectively. The rename `fileResult` → `toolResult` is a clean, complete rename (no leftover references). Tests are thorough, table-driven where it matters, use `t.Helper()`/`t.Parallel()`/`t.Context()`. Concurrency is clean: `Fetch` is stateless, `*http.Client` is documented safe for concurrent use, context is propagated end to end, and the per-call converter in `htmlToMarkdown` is a documented, deliberate choice. Error handling wraps with `%w` and never swallows. Naming, doc comments, and README are excellent.
README.md Outdated
@ -165,0 +198,4 @@
of holding the turn open, and a canceled turn still stops it immediately.
Like the file tools it takes the caller's context, so a cancelled turn stops the
fetch, and it runs without a permission prompt: the ACP client is trusted, the
Collaborator

🟡 SUGGESTION — this paragraph documents the trusted-ACP-client posture, but the fetch tool is the first outbound network channel exposed to the model, and it can reach internal addresses (localhost, RFC1918, cloud metadata 169.254.169.254) while page content — an untrusted prompt-injection vector — is fed straight back to the model. If brainless can ever run in an environment where metadata or internal services matter, consider documenting this exposure in the security section or blocking loopback/private/metadata ranges. This is a design consideration, not a claim that the current trust model is wrong.

🟡 SUGGESTION — this paragraph documents the trusted-ACP-client posture, but the fetch tool is the first outbound network channel exposed to the model, and it can reach internal addresses (localhost, RFC1918, cloud metadata 169.254.169.254) while page content — an untrusted prompt-injection vector — is fed straight back to the model. If brainless can ever run in an environment where metadata or internal services matter, consider documenting this exposure in the security section or blocking loopback/private/metadata ranges. This is a design consideration, not a claim that the current trust model is wrong.
@ -0,0 +140,4 @@
return toolResult{}, err
}
maxLength, err := fetchMaxLength(args.MaxLength)
Collaborator

🟡 SUGGESTION — maxLength and startIndex are validated only after checkRobots has already made a network request (lines 139-150). An invalid window argument therefore costs a robots.txt round-trip before the tool fails. Move fetchMaxLength and the startIndex check above checkRobots so all argument validation happens before any I/O.

🟡 SUGGESTION — maxLength and startIndex are validated only after `checkRobots` has already made a network request (lines 139-150). An invalid window argument therefore costs a robots.txt round-trip before the tool fails. Move `fetchMaxLength` and the startIndex check above `checkRobots` so all argument validation happens before any I/O.
@ -0,0 +163,4 @@
switch {
case isPageHTML && !args.Raw:
content, err = htmlToMarkdown(page.body, target)
Collaborator

🔴 BLOCKING — relative links are resolved against the original URL, not the URL the page actually came from after redirects. get follows redirects (default http.Client: up to 10, cross-host), so the body in page is the final response, but target is still the original URL when it is passed to htmlToMarkdown here and to the "Contents of" prefix at line 178. Any redirected page — http→https, www→apex, moved paths — gets wrong absolute links, breaking the tool's documented promise ("relative links are made absolute against the fetched page").

Reproduced with a temporary test at head (removed after): /old/page → 302 → http://127.0.0.1:44225/new/page serving <a href="/real-target"> produced output [link](http://127.0.0.1:46805/real-target) — resolved against the original host, not the final host. (Also verified go doc net/http.Response: "Request is the request that was sent to obtain this Response", i.e. the final request after redirects.)

Fix: have get return resp.Request.URL and pass that to htmlToMarkdown and the output prefix instead of the original target.

🔴 BLOCKING — relative links are resolved against the original URL, not the URL the page actually came from after redirects. `get` follows redirects (default `http.Client`: up to 10, cross-host), so the body in `page` is the final response, but `target` is still the original URL when it is passed to `htmlToMarkdown` here and to the "Contents of" prefix at line 178. Any redirected page — http→https, www→apex, moved paths — gets wrong absolute links, breaking the tool's documented promise ("relative links are made absolute against the fetched page"). Reproduced with a temporary test at head (removed after): `/old/page` → 302 → `http://127.0.0.1:44225/new/page` serving `<a href="/real-target">` produced output `[link](http://127.0.0.1:46805/real-target)` — resolved against the original host, not the final host. (Also verified `go doc net/http.Response`: "Request is the request that was sent to obtain this Response", i.e. the final request after redirects.) Fix: have `get` return `resp.Request.URL` and pass that to `htmlToMarkdown` and the output prefix instead of the original `target`.
@ -0,0 +297,4 @@
// at all is an error, because the tool does not fetch what it cannot check; 401
// and 403 mean autonomous fetching is not allowed; any other 4xx means there is
// no robots.txt to obey; and otherwise the rules decide.
func (f *Fetch) checkRobots(ctx context.Context, target *url.URL) error {
Collaborator

🟠 IMPORTANT — robots.txt is evaluated only for the original URL, so a redirect bypasses it. checkRobots builds the robots URL from target and evaluates rules against target.RequestURI(), but the page fetch (f.get, line 274) then follows up to 10 redirects (default http.Client policy, cross-host included). A site that 302s /public → /private/secret is fetched even when its robots.txt says Disallow: /private.

Reproduced with a temporary test at head (removed after): robots.txt User-agent: * Disallow: /private, /public → 302 → /private/secret; fetch(/public) returned "Contents of http://127.0.0.1:46583/public: SECRET DISALLOWED CONTENT" with no error. The reference server (httpx follow_redirects=True, single robots check) shares this gap, but the README and the tool description promise "a path the site disallows is refused".

Fix: set http.Client.CheckRedirect to re-run checkRobots against each redirect target (or follow redirects manually), or document the limitation.

🟠 IMPORTANT — robots.txt is evaluated only for the original URL, so a redirect bypasses it. `checkRobots` builds the robots URL from `target` and evaluates rules against `target.RequestURI()`, but the page fetch (`f.get`, line 274) then follows up to 10 redirects (default `http.Client` policy, cross-host included). A site that 302s `/public` → `/private/secret` is fetched even when its robots.txt says `Disallow: /private`. Reproduced with a temporary test at head (removed after): robots.txt `User-agent: * Disallow: /private`, `/public` → 302 → `/private/secret`; `fetch(/public)` returned `"Contents of http://127.0.0.1:46583/public: SECRET DISALLOWED CONTENT"` with no error. The reference server (httpx `follow_redirects=True`, single robots check) shares this gap, but the README and the tool description promise "a path the site disallows is refused". Fix: set `http.Client.CheckRedirect` to re-run `checkRobots` against each redirect target (or follow redirects manually), or document the limitation.
@ -0,0 +308,4 @@
switch {
case page.status == http.StatusUnauthorized || page.status == http.StatusForbidden:
return fmt.Errorf("fetch %s: %s returned status %d, so autonomous fetching is not allowed", target, robots, page.status)
case page.status >= http.StatusBadRequest:
Collaborator

🟡 SUGGESTION — a robots.txt answering 500-599 is treated as "no rules" (page.status >= http.StatusBadRequest covers 5xx), but RFC 9309 §2.3.1.4 says a server error means robots.txt is undefined and the crawler "MUST assume complete disallow". This also contradicts the doc comment above ("a robots.txt that cannot be read at all fails the fetch rather than fetching unchecked"). Condition: the origin's robots.txt returns a transient 5xx — the tool then fetches pages whose policy is unknown. The reference server fails open here too (it parses the 5xx body as robots.txt), so this is a fidelity note, not a parity break.

Fix: return an error for status >= 500 (fail closed per RFC), or at minimum fix the comment to say "any other 4xx or 5xx".

🟡 SUGGESTION — a robots.txt answering 500-599 is treated as "no rules" (`page.status >= http.StatusBadRequest` covers 5xx), but RFC 9309 §2.3.1.4 says a server error means robots.txt is undefined and the crawler "MUST assume complete disallow". This also contradicts the doc comment above ("a robots.txt that cannot be read at all fails the fetch rather than fetching unchecked"). Condition: the origin's robots.txt returns a transient 5xx — the tool then fetches pages whose policy is unknown. The reference server fails open here too (it parses the 5xx body as robots.txt), so this is a fidelity note, not a parity break. Fix: return an error for `status >= 500` (fail closed per RFC), or at minimum fix the comment to say "any other 4xx or 5xx".
@ -0,0 +144,4 @@
}
product, _, _ = strings.Cut(strings.ToLower(product), "/")
if product == agent || strings.HasPrefix(product, agent) {
Collaborator

🟡 SUGGESTION — strings.HasPrefix(product, agent) matches prefix tokens: a User-agent: brain group would be applied to the brainless product token. RFC 9309 §2.2.1 requires case-insensitive matching of the product token against the group's identifier — an exact identifier match, not a prefix. Practical impact is low while fetchProductToken is the constant "brainless", but the file claims RFC 9309 compliance and TestRobotsAllow does not cover this.

Fix: drop the || strings.HasPrefix(product, agent) arm and match product == agent only.

🟡 SUGGESTION — `strings.HasPrefix(product, agent)` matches prefix tokens: a `User-agent: brain` group would be applied to the `brainless` product token. RFC 9309 §2.2.1 requires case-insensitive matching of the product token against the group's identifier — an exact identifier match, not a prefix. Practical impact is low while `fetchProductToken` is the constant `"brainless"`, but the file claims RFC 9309 compliance and `TestRobotsAllow` does not cover this. Fix: drop the `|| strings.HasPrefix(product, agent)` arm and match `product == agent` only.
pi-agent force-pushed feat/builtin-fetch-tool from a0c0465b69
All checks were successful
code-review / Agent review (pull_request) Successful in 16s
to edb6a12cb3 2026-10-03 12:46:09 +00:00
Compare
Author
Collaborator

All six review findings are resolved on head edb6a12 (amended commit, force-pushed).

  1. #729 — BLOCKING: relative links after redirects → get now returns the effective URL (resp.Request.URL, the final request after redirects) and fetch uses it for both the markdown domain (WithDomain) and the Contents of …: prefix. Added TestFetchResolvesLinksAgainstFinalURL (two servers: /old/page 302 → dest/new/page with <a href="/real-target">; output resolves against dest, prefix names dest).

  2. #730 — IMPORTANT: redirects bypass robots.txt → the page client now has CheckRedirect that re-runs checkRobots against every redirect target, aborting the fetch when a hop is disallowed. A separate redirect-free client fetches robots.txt itself, so a redirecting robots.txt cannot loop back into another check. Added TestFetchRedirectCannotBypassRobots (/public 302 → /private/secret with Disallow: /private; fetch now fails with the robots refusal). The tool description now says "at every redirect the page takes".

  3. #731 — SUGGESTION: robots.txt 5xx → a 5xx robots.txt now fails the fetch closed (RFC 9309 §2.3.1.4, "assume complete disallow"), alongside 401/403, with the reason in the error; the doc comment and README updated. Covered by a new case in TestFetchRobotsTxtFailures.

  4. #733 — SUGGESTION: arg validation after I/O → fetchMaxLength, the startIndex check, and the timeout check now all run before any request, so a bad window or timeout costs no robots.txt round-trip.

  5. #732 — SUGGESTION: prefix user-agent matching → robotAgentMatch now matches the product token exactly (RFC 9309 §2.2.1), dropping the HasPrefix arm; a User-agent: brain group no longer applies to brainless. Covered in TestRobotsAllow.

  6. #734 — SUGGESTION: outbound exposure / prompt injection → documented in the README's fetch section ("Exposure."), stating the tool can reach internal addresses (localhost, RFC 1918, cloud metadata) and that page content is an untrusted prompt-injection vector handed to the model, safe only under the trusted-ACP-client posture; blocking ranges is left as a future restriction if the deployment requires it.

Verification: go test ./..., racedetector test ./internal/tools/ ./internal/harness/ (no races), golangci-lint run ./... — all green.

All six review findings are resolved on head `edb6a12` (amended commit, force-pushed). 1. **#729 — BLOCKING: relative links after redirects** → `get` now returns the effective URL (`resp.Request.URL`, the final request after redirects) and `fetch` uses it for both the markdown domain (`WithDomain`) and the `Contents of …:` prefix. Added `TestFetchResolvesLinksAgainstFinalURL` (two servers: `/old/page` 302 → `dest/new/page` with `<a href="/real-target">`; output resolves against `dest`, prefix names `dest`). 2. **#730 — IMPORTANT: redirects bypass robots.txt** → the page client now has `CheckRedirect` that re-runs `checkRobots` against every redirect target, aborting the fetch when a hop is disallowed. A separate redirect-free client fetches robots.txt itself, so a redirecting robots.txt cannot loop back into another check. Added `TestFetchRedirectCannotBypassRobots` (`/public` 302 → `/private/secret` with `Disallow: /private`; fetch now fails with the robots refusal). The tool description now says "at every redirect the page takes". 3. **#731 — SUGGESTION: robots.txt 5xx** → a 5xx robots.txt now fails the fetch closed (RFC 9309 §2.3.1.4, "assume complete disallow"), alongside 401/403, with the reason in the error; the doc comment and README updated. Covered by a new case in `TestFetchRobotsTxtFailures`. 4. **#733 — SUGGESTION: arg validation after I/O** → `fetchMaxLength`, the `startIndex` check, and the timeout check now all run before any request, so a bad window or timeout costs no robots.txt round-trip. 5. **#732 — SUGGESTION: prefix user-agent matching** → `robotAgentMatch` now matches the product token exactly (RFC 9309 §2.2.1), dropping the `HasPrefix` arm; a `User-agent: brain` group no longer applies to `brainless`. Covered in `TestRobotsAllow`. 6. **#734 — SUGGESTION: outbound exposure / prompt injection** → documented in the README's fetch section ("**Exposure.**"), stating the tool can reach internal addresses (localhost, RFC 1918, cloud metadata) and that page content is an untrusted prompt-injection vector handed to the model, safe only under the trusted-ACP-client posture; blocking ranges is left as a future restriction if the deployment requires it. Verification: `go test ./...`, `racedetector test ./internal/tools/ ./internal/harness/` (no races), `golangci-lint run ./...` — all green.
sam merged commit caf70455ed into main 2026-10-03 13:00:47 +00:00
sam deleted branch feat/builtin-fetch-tool 2026-10-03 13:00:47 +00:00
Sign in to join this conversation.
No reviewers
No labels
code-review
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
sam/brainless!20
No description provided.