EdgeComet FAQ: setup, safety, bots and data
Short answers to the questions people ask most about EdgeComet, each with a link to the page that explains it in full.
Getting started
EdgeComet works at your CDN, needs no code changes and starts showing data within a day of the install.
What does EdgeComet do?
EdgeComet sits behind your CDN and handles requests from search engines, AI bots and link preview bots. It serves them pages from a cache it keeps current, renders JavaScript pages in headless Chrome when your site needs it, and records every bot request for its reports, site audit and alerts. See How EdgeComet serves search and AI bots.
Does EdgeComet work with my framework, CMS or CDN?
Yes, when your bot traffic passes through a CDN or proxy that can route by user agent. Single-page apps built with React, Vue or Angular use Cache + Render, and server-rendered sites such as WordPress, Next.js or Nuxt with server rendering, and static sites use Cache-only. See Cache only or cache and render.
Cloudflare installs automatically. Fastly, CloudFront, Akamai, Nginx, Apache, HAProxy and any other proxy take one routing rule, with a setup page each. Shopify and Salesforce Commerce Cloud stores put their own Cloudflare zone in front of the store and then use the Cloudflare install. See Choose your platform.
Do I need JavaScript rendering if my site is server-rendered?
Not if the HTML your server sends already holds your content, links and tags. Cache-only mode caches that HTML for bots without rendering. Choose Cache + Render when content appears only after JavaScript runs, such as prices loaded from an API. See Check what your server sends.
Does EdgeComet change my site or code?
No. You add a routing rule at your CDN, and your server, code and CMS stay as they are. Human visitors keep loading your site directly, with their pages, cookies and sessions unchanged. See How EdgeComet routes bot traffic.
When will I see data?
Search reports work as soon as Search Console imports, with 180 days of history during a trial and 360 days on a paid plan. Bot reports start at the first verified bot visit, usually within 24 hours of the install, and the first site audit and Action Board follow a few days later. See Your first days.
Do I need to configure EdgeComet before reports are useful?
No. Bots are served with default settings from the first request. In the first weeks, EdgeComet sets up branded keywords, page segments, custom extractions and crawl budget rules from your own data. See What EdgeComet sets up on its own.
How do I know EdgeComet is working?
Request a page with a bot user agent and look for an EC-Source response header, then request it as a browser and check that no EC- header comes back. The dashboard marks the site live at the first response to a verified bot. See Check the headers bots receive.
Speed, reliability and SEO safety
EdgeComet adds no step for your visitors, falls back to your origin when it cannot answer, and serves bots the same content people see.
Does EdgeComet slow down my site or change Core Web Vitals?
No. Human visitors never pass through EdgeComet, and Core Web Vitals come from those visitors. With bots served from EdgeComet's cache, your server has more capacity for visitors. See Does EdgeComet slow down your site?
What happens if EdgeComet is down?
Bots get your regular pages from your origin. The Cloudflare integration falls back to your origin on its own, and other proxies need the same fallback in their rule. When a render fails, EdgeComet serves the last cached copy or your server's HTML. See What happens if EdgeComet or your origin is down.
Is prerendering for bots cloaking?
No, when bots get the same content people see. Google says "Googlebot generally doesn't consider dynamic rendering as cloaking." EdgeComet renders the same URL a visitor loads and serves every bot the copy for its device type. See Is prerendering for bots cloaking?
How fresh are the pages bots get?
It depends on what signals the change. With Smart Re-cache on, pages bots request refresh as often as they change, product changes in your Merchant Center feed reach bots after the next feed check, and Edge SEO changes reach them right after you publish. See How soon bots get a change.
Does EdgeComet notify Google or Bing when a page changes?
No. EdgeComet keeps the cached copy fresh before a bot asks for it, and search engines still decide when they crawl. It does not send IndexNow or Indexing API notifications. See What freshness does not cover.
Do ChatGPT, Claude and Perplexity see content loaded by JavaScript?
The AI bots EdgeComet tested read only the HTML document: ChatGPT-User and Gemini's fetcher never ran JavaScript, and GPTBot ran it once in hundreds of requests. Plan for every AI bot to read only the HTML it receives. See Do GPTBot, ClaudeBot and other AI bots run JavaScript?
Bots and crawling
EdgeComet serves and records the bots your CDN routes to it, and it runs no crawler of its own.
Which bots does EdgeComet serve and record?
Every request your CDN routes to it: search engine crawlers, AI crawlers such as GPTBot and ClaudeBot, AI assistants fetching a page for a user, and link preview bots. On Cache + Render sites, search engines and AI bots get the rendered copy. See AI crawler user agents and Search crawler user agents.
How does EdgeComet tell a real Googlebot from a fake one?
It checks each request's IP address against the ranges the bot's operator publishes and stores the verdict with the request: Verified, Fake, Unverifiable or Error. Most reports leave fake requests out. See How EdgeComet verifies bots.
Can EdgeComet block bad bots?
Yes, by user agent. A URL rule with Match User-Agent answers a bot with 403 Forbidden or another status on all or some of your URLs. A fake Googlebot sends Googlebot's user agent, which makes an IP-based firewall rule the way to stop it. See Block a bad bot.
Does EdgeComet crawl my site?
Not with a crawler of its own. EdgeComet fetches a page from your server when it has no fresh copy for a bot or a pre-cache rule refreshes it, and it fetches the sitemaps and product feeds you list. Its site audit, Evergreen Crawl, is built from those fetches. See Does EdgeComet crawl my site?
Does EdgeComet respect robots.txt?
Your robots.txt stays on your server, and EdgeComet does not change or enforce it. Each bot reads your file and decides what to request, and EdgeComet records the requests bots make. Its own fetches are the pages bots asked for and the URLs your pre-cache rules list. See robots.txt and AI bots.
Does EdgeComet generate llms.txt?
Requests for /llms.txt go straight to your server, like robots.txt, and bots get the file you publish. Text files never pass through EdgeComet. See Which requests go to EdgeComet.
Does EdgeComet serve Markdown to AI bots?
Bots get the HTML page, rendered or as your server sends it. The Markdown view in Page Analysis shows you the text a bot reads in the cached copy. See Markdown: what text does the copy hold?
Can Edge SEO add structured data?
Yes. Edge SEO adds JSON-LD markup such as Product, Article and BreadcrumbList to the pages bots receive, built from what each page shows. Rendering also keeps the JSON-LD your pages already carry. See What Edge SEO adds.
Data, security and privacy
EdgeComet records bot requests, not visitors, and keeps bot data for as long as your subscription is active.
Do I need to upload server log files?
No. EdgeComet records each bot request as it serves it, with the page the bot received. There are no log files to export or parse, and it works the same on any web server. See Do I need to upload my server log files?
Does EdgeComet store data about my visitors?
EdgeComet records the bot requests your CDN routes to it, and human visitors' page requests never reach it. For each bot request it stores the bot, its IP address and verification result, the URL, the headers of each step and an SEO snapshot of the page. See What EdgeComet stores.
Where is my data stored, and how long is it kept?
EdgeComet runs on AWS, Hetzner and OVH in the United States, Europe and Australia, and can keep all your data in European data centers on request. Bot data is kept for as long as your subscription is active. See Data protection and compliance and How long data is kept.
What access does EdgeComet need to Search Console?
Read-only access to your property. EdgeComet changes nothing in Search Console, except to create the subfolder properties you ask for, and never removes properties, sitemaps or users. It imports up to 360 days of history when you connect, 180 during a trial, then one new day every day. See What EdgeComet imports.
Can I connect Search Console subfolder properties?
Yes. Each root property includes 10 subfolder properties, such as https://example.com/blog/, at no extra cost, and [email protected] can enable more. Each one adds its own daily Search Console export of pages and keywords, and EdgeComet merges them with the root property and counts every page once. See Subfolder properties.
Can AI change my site without my approval?
No. The Action Board, Ask and AI clients connected over MCP save drafts, and a person reviews and deploys every Edge SEO change. See Can AI deploy Edge SEO changes on its own?
What does my firewall need to allow?
Requests whose X-Render-Key header carries your site's render key. EdgeComet publishes no IP list: the key identifies its requests. On Cloudflare, the automatic install adds the allow rule, and Bot Fight Mode needs to be off. See Let EdgeComet through your firewall, Bot Fight Mode and What EdgeComet sends to your origin.
Account and usage
Usage counts Googlebot requests, and your role and account decide what you see in the dashboard.
How is usage counted?
Usage counts Googlebot requests. Every other bot, including Bingbot and every AI bot, is free, and fake Googlebot requests do not count. See Do AI bot requests count toward EdgeComet usage?
Why is a section missing from my sidebar?
Some sections appear only when your account includes them, and some settings only for Admins and Owners. Ask your account owner, or write to [email protected]. See What each role sees in the dashboard.
Can agencies manage client websites?
Yes. An agency account creates a sub account for each client, adds their websites and manages them, while the client owns the account and sees their own data. See How agency accounts work.
Is there an API or an MCP server?
Yes. The HTTP API recaches pages and queries or exports your data with an API token. The MCP server connects Claude, ChatGPT, Cursor, Copilot and other AI clients to your EdgeComet data. See Connect an AI client.
Is the EdgeComet engine open source?
Yes. The core engine is open source under the Apache 2.0 license at github.com/EdgeComet/engine, and EdgeComet's cloud runs on it, adding the dashboard, analytics, Edge SEO and AI features. To run the engine on your own servers, see The engine and EdgeComet cloud in the self-hosted engine docs, and for how the cloud uses it, see EdgeComet for developers.
How do I pause or remove EdgeComet?
Remove the routing. With the Cloudflare install, open Configuration > Connections > CDN integration, click Manage connection, then Disconnect. On other setups, delete the routing rule. Bots then get your origin again. See How to pause or remove EdgeComet.