Skip to content

feat(api): proxyUrl for residential/custom proxy egress on the API block#5867

Merged
waleedlatif1 merged 4 commits into
stagingfrom
feat/api-proxy-url-docs
Jul 22, 2026
Merged

feat(api): proxyUrl for residential/custom proxy egress on the API block#5867
waleedlatif1 merged 4 commits into
stagingfrom
feat/api-proxy-url-docs

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Supersedes #5865 (thanks @mzxchandra — your commit is carried here with attribution intact). Adds an optional Proxy URL field to the API block so a request can egress through a residential/custom http:// proxy, plus the docs the original PR was missing.

  • Optional Proxy URL under the API block's Advanced fields; when set, the request egresses from the proxy's IP instead of the app runtime's datacenter IPs (useful for Cloudflare/WAF-protected targets that 403/429 datacenter ranges)
  • Security model: validateAndPinProxyUrl enforces http:// scheme, resolves the proxy host's DNS and blocks private/reserved/loopback/metadata IPs, then pins the connection by rewriting the host to the resolved IP (creds/port preserved) — closing the DNS-rebinding window. Target URLs are still validated with validateUrlWithDNS; only target-IP pinning is bypassed when a proxy is active (the proxy resolves the target). Redirects carry the pinned proxy URL through and every hop's target is still re-validated
  • secureFetchWithPinnedIP swaps its pinned agent for HttpsProxyAgent/HttpProxyAgent keyed off the target protocol (CONNECT tunnel for https targets, absolute-URI for http); target SNI/cert validation is unaffected
  • LLMs cannot set the param (visibility: 'user-only'); the internal-route branch ignores it; proxy validation re-runs on every retry attempt
  • Docs: adds the missing Proxy URL bullet to the API block's Advanced section and steers proxy credentials to {{PROXY_URL}} env-var references (field placeholder/description updated to match)
  • No new supply-chain surface: http-proxy-agent@7.0.2 / https-proxy-agent@7.0.6 were already resolved in bun.lock as transitive deps; only manifest references are added

Type of Change

  • New feature

Testing

  • 574 tests passing across the affected suites (input-validation.test.ts, tools/index.test.ts, tools/utils.test.ts), including new coverage: scheme rejection (https/socks5), malformed URLs, private/metadata proxy hosts, IPv6 bracket pinning, creds/port preservation, pinned-URL passthrough to secureFetchWithPinnedIP, and trim/omit semantics in formatRequestParams
  • Typecheck, Biome, and check:api-validation:strict clean
  • Manual E2E from feat(api): add proxyUrl for residential/custom proxy egress on the API block #5865 against a local Basic-auth logging proxy: CONNECT tunnel for https targets, absolute-URI for http, direct egress when the field is cleared, scheme/blocked-IP/407 errors surfaced correctly

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

mzxchandra and others added 2 commits July 22, 2026 15:40
…I block

The HTTP/API block egresses from the app runtime's fixed datacenter IPs via
secureFetchWithPinnedIP, so targets behind Cloudflare/WAF that block datacenter
IPs (e.g. state .gov license portals) return 403/429 even when the identical
request works from a browser. There was no way to route a request through a
residential/custom proxy.

Add an optional `proxyUrl` field (Advanced) to the API block. When set, the
request routes through the given http:// proxy so it egresses from that proxy's
IP.

Security:
- validateAndPinProxyUrl resolves the proxy host's DNS and blocks
  private/reserved/loopback IPs (same SSRF guard as target URLs), then pins the
  connection by rewriting the host to the resolved IP (creds/port preserved),
  closing the DNS-rebinding window.
- Restricted to the http: proxy scheme (https/socks rejected) so host pinning is
  safe without breaking TLS-to-proxy SNI.
- Target-IP pinning is intentionally bypassed when a proxy is active (the proxy
  resolves the target); target URL validation still runs.

Threaded block field -> http tool param -> formatRequestParams ->
executeToolRequest (validate + pin) -> secureFetchWithPinnedIP, which swaps its
pinned Node agent for HttpsProxyAgent/HttpProxyAgent (keyed off target protocol)
when proxyUrl is set.
@waleedlatif1
waleedlatif1 requested a review from a team as a code owner July 22, 2026 23:03
@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 22, 2026 11:18pm

Request Review

@cursor

cursor Bot commented Jul 22, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Changes outbound HTTP security and egress: proxy mode bypasses target IP pinning and expands the SSRF surface to user-chosen proxy hosts, though mitigated by scheme limits, DNS pinning, and private-IP blocks.

Overview
Adds an optional Proxy URL on the API block (Advanced) so workflow HTTP calls can egress through a user-supplied http:// proxy (e.g. residential IPs) instead of the runtime datacenter.

The value flows through the http_request tool (user-only param), formatRequestParams, and external executeTool paths: when set, validateAndPinProxyUrl enforces http://, DNS-checks the proxy host (blocking private/loopback/metadata even where target validation would allow loopback), rewrites the host to the resolved IP to pin the proxy connection, then secureFetchWithPinnedIP uses HttpProxyAgent / HttpsProxyAgent instead of target IP pinning (the proxy resolves the destination). Target URLs are still validated with validateUrlWithDNS; invalid proxies fail before fetch.

Docs cover the new Advanced field and recommend {{PROXY_URL}} for credentials. Direct dependencies on http-proxy-agent and https-proxy-agent are added to the sim app manifest; tests cover validation, tool wiring, and param trim/omit behavior.

Reviewed by Cursor Bugbot for commit ec633dc. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds custom HTTP proxy egress to the API block. The main changes are:

  • Adds a user-only Proxy URL field and request parameter.
  • Validates proxy hosts, blocks private addresses, and pins resolved IPs.
  • Routes HTTP and HTTPS targets through protocol-specific proxy agents.
  • Carries proxy settings through retries and redirects.
  • Adds tests, dependency declarations, mocks, and user documentation.

Confidence Score: 5/5

This looks safe to merge.

  • The latest fixes close the private and loopback proxy-host path.
  • Proxy routing remains limited to validated external requests.
  • No qualifying unresolved issue was found in the updated code.

Important Files Changed

Filename Overview
apps/sim/lib/core/security/input-validation.server.ts Adds proxy URL validation, IP pinning, and proxy-agent routing for secure requests.
apps/sim/tools/index.ts Validates and forwards proxy settings for external tool requests.
apps/sim/tools/utils.ts Normalizes the optional proxy URL alongside other request parameters.
apps/sim/blocks/blocks/api.ts Exposes the optional Proxy URL field in the API block configuration.
apps/sim/lib/core/security/input-validation.test.ts Covers malformed schemes, blocked addresses, credential preservation, and IPv6 pinning.

Reviews (3): Last reviewed commit: "chore(api): tighten proxy-path inline co..." | Re-trigger Greptile

Comment thread apps/sim/lib/core/security/input-validation.server.ts
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit d9b9440. Configure here.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ec633dc. Configure here.

Comment thread apps/sim/lib/core/security/input-validation.server.ts
@waleedlatif1
waleedlatif1 merged commit 8cce661 into staging Jul 22, 2026
20 checks passed
@waleedlatif1
waleedlatif1 deleted the feat/api-proxy-url-docs branch July 22, 2026 23:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants