Appearance

Let EdgeComet through your firewall, WAF or bot protection ​

EdgeComet fetches your pages through your CDN like any other client, and your security rules can block it. Allow requests whose X-Render-Key header equals your site's render key, and they pass your WAF, rate limits and bot protection. EdgeComet publishes no list of IP addresses: the key identifies its requests.

Without the rule, renders fail or come back as challenge pages, and Test connection in the dashboard guide fails. Bot Control and bot management products often flag EdgeComet first, because it renders in headless Chrome from hosting providers.

What EdgeComet's requests look like ​

  • X-Render-Key: your render key, 32 lowercase hexadecimal characters, on every page request. Match it exactly and case-sensitively.
  • X-Edge-Render: on every request, including the scripts and API calls a render loads. Your routing rule sends these requests to your origin.
  • User agent: a mobile Chrome browser by default, without "EdgeComet" or a bot name in it.
  • No cookies.

Render requests send X-Render-Key only to your page's own domain, its subdomains and its parent domain. For the full list, see What EdgeComet sends to your origin.

Sitemaps and Merchant Center feeds are fetched without the render key, by EdgeCometSitemapFetcher/1.0 and EdgeCometFeedFetcher/1.0. If your firewall blocks unknown user agents, allow these two on your sitemap and feed URLs.

Treat the render key like a password, because the rule trusts it. See The render key.

Cloudflare WAF ​

The automatic Cloudflare install adds this rule for you. To add it by hand, open Security > WAF > Custom rules, create a rule with this expression in the expression editor, and set the action to Skip:

text
(any(http.request.headers["x-render-key"][*] eq "YOUR_RENDER_KEY"))

Under the Skip action, select:

  • All remaining custom rules
  • All rate limiting rules
  • All managed rules
  • All Super Bot Fight Mode rules
  • The legacy products your plan offers: Zone Lockdown, User Agent Blocking, Browser Integrity Check, Hotlink Protection, Security Level, Rate Limiting (previous version) and WAF managed rules (previous version)

Place the rule first. Cloudflare runs custom rules before Super Bot Fight Mode, and a custom rule can skip it.

Bot Fight Mode ​

Bot Fight Mode runs outside Cloudflare's rules engine, and no rule can skip it. Turn it off, or use a plan with Super Bot Fight Mode. See Bot Fight Mode.

AWS WAF ​

On CloudFront, AWS WAF reads the request before any Lambda@Edge function runs, as EdgeComet sent it. Add an allow rule at the top of the web ACL:

  1. Open the web ACL attached to the distribution and click Add rules > Add my own rules and rule groups.
  2. Choose Rule builder, name the rule AllowEdgeComet, type Regular rule.
  3. Inspect: Single header, header name x-render-key. Match type: Exactly matches string, String to match: your render key, no text transformation.
  4. Action: Allow.
  5. Under Set rule priority, move the rule to the top and save.

Allow ends rule processing for the request. Without the rule, and if your web ACL uses these rule groups, Bot Control blocks unverified bots and automated browsers, and the anonymous IP list blocks hosting providers.

Bot Control's CategoryAI rule blocks AI crawlers whether or not they are verified. AI crawlers it blocks never reach EdgeComet, and EdgeComet does not record them.

For a narrower exception, AWS documents a scope-down statement on the Bot Control rule group that excludes requests carrying a shared secret header. Use NOT (x-render-key exactly matches YOUR_RENDER_KEY) as the scope-down on Bot Control and any IP reputation group. Your other rules then still apply to EdgeComet.

An AWS WAF on an Application Load Balancer behind CloudFront sees the key only when CloudFront forwards X-Render-Key to the origin. Add the header to the origin request policy, not the cache policy.

Fastly ​

In VCL, wrap your own blocking rules in a condition on the exact key. See Let EdgeComet through Fastly security.

In Next-Gen WAF, add a request rule with the Allow action:

  1. Open Security > Next-Gen WAF > Rules and select the workspace for your site.
  2. Click Add workspace rule, type Request.
  3. Condition: Request header, Exists where, Name Equals X-Render-Key and Value (string) Equals your render key.
  4. Action: Allow. Click Create workspace rule.

An Allow rule takes precedence over block rules. A signal exclusion rule is the narrower option: it removes only the signal that blocks EdgeComet.

Akamai ​

Add exceptions on the request header X-Render-Key with your exact key, case-sensitive, without wildcards:

  • App & API Protector attack groups: an exception with a request header match condition.
  • Individual WAF rules, with Kona Site Defender or Advanced Security Management: the same condition on the rule's exceptions.
  • Rate policies: a condition on the request header that keeps EdgeComet's requests out of the count.
  • Bot Manager: allow EdgeComet's requests, for example with a custom-defined bot that matches the header and an allow or monitor action, where your Bot Manager setup supports a header condition. Bot Manager is likely to flag headless Chrome, which makes this one important.

Salesforce Commerce Cloud eCDN ​

On SFCC, EdgeComet's requests pass your Cloudflare zone and then the eCDN. Add an eCDN custom rule that allows requests whose X-Render-Key equals your key. See Let EdgeComet through the eCDN.

Nginx ​

If Nginx blocks user agents or rate-limits, exempt the exact key with a map. See Let EdgeComet through Nginx rules.

Other firewalls ​

Any WAF that can match a request header can allow EdgeComet:

  1. Match the header X-Render-Key against your exact key, case-sensitively.
  2. Apply the product's allow or skip action, or exclude the match from bot and rate rules.
  3. Run the rule before the rules that block bots.

If your CDN forwards requests to an origin with its own firewall, check that the CDN passes X-Render-Key through. Some CDNs drop unknown headers unless an origin request policy or a forwarding rule keeps them. If your WAF cannot match request headers, contact [email protected].

Check the rule ​

In the dashboard guide, click Test connection under Step 3: Verify connection. EdgeComet fetches your homepage with your render key and passes on HTTP 200. For the render check and its messages, see Check the render check.

Update the rule after a key rotation ​

The rule holds your render key. After a render key rotation, update every rule that matches it. The automatic Cloudflare install updates its rule when you click Re-run checks.