For Developers

Rendering isn't just a marketing problem. EdgeComet was built by engineers for engineers.

System Overview

EdgeComet renders JavaScript pages in headless Chrome for search engine bots and AI crawlers. Caches locally, scales via Redis, fails gracefully to origin.
Deploys behind your existing infrastructure.

All
Traffic
Load Balancer
CDN/LB
Only Bots
Traffic
EdgeComet
Gateway 1
Gateway 2
Gateway N
Cache
Render 1
Render 2
Render 3
Render N
Your Website

Three Services, Clear Responsibilities

Each component does one thing well

Edge Gateway

Entry point that intercepts bot requests, manages caching, and coordinates rendering.

  • Under 50MB RAM, minimal CPU
  • Filesystem cache, Redis metadata
  • Graceful fallback to bypass mode
  • Automatic peer discovery
View docs

Render Service

Manages headless Chrome pool to render JavaScript-heavy pages.

  • Multiple Chrome instances, isolated tabs
  • Auto-restarts to prevent memory leaks
  • CPU-intensive, pair with EG
  • 3-second failover on crash
View docs
Optional

Cache Daemon

Background service that keeps cache fresh without impacting live traffic.

  • Three queue priorities
  • Rate-limits to preserve capacity
  • Recaches only visited pages
  • Exposes invalidation API
View docs

Chrome Without the Pain

EdgeComet manages Chrome instances proactively.

Event What Happens Your Involvement
Memory bloat Scheduled restart None
Instance hang Auto-detected, replaced None
Chrome crash New instance spawned None
Security update Update binary ~30 min, few times/year
No 2am pages. No memory leak firefighting. Chrome lifecycle is handled.

High Availability & Cache Replication

For high-traffic websites, hardware or software failures can mean lost crawl budget. EdgeComet prevents this with distributed caching.

No Single Point of Failure

  • Instances discover each other via Redis heartbeats
  • Failed instances detected within 3 seconds
  • Remaining instances continue serving without interruption

Automatic Replication

  • Configurable replication factor (default: 2 copies)
  • Push on render: immediate availability across cluster
  • Pull on access: lazy replication as content is requested

Zero Downtime

  • Instance failures invisible to crawlers
  • Scale your cluster without config changes
  • Bots always get content

Built-in Observability

Prometheus Metrics

Dedicated metrics port, ready for Grafana

Structured Logs

JSON format via Zap, ELK-ready

Diagnostic Headers

X-Render-Source, X-Cache-Age, X-Request-ID

Health Endpoints

Load balancer ready, liveness and readiness probes

View Metrics Reference

Minimal Attack Surface

Network Isolation

  • Runs behind your reverse proxy (NGINX, HAProxy, CDN)
  • Never directly exposed to the internet
  • Internal services communicate via private network

Limited Scope

  • Serves only bot traffic - not in the user request path
  • No user sessions, no cookies, no PII processed
  • No database connections to your application data

Chrome Sandboxing

  • Headless Chrome runs with standard sandbox protections
  • Isolated tab contexts per render (no cross-request leakage)
  • Automatic instance recycling prevents state accumulation

Authentication

  • X-Render-Key authenticates requests from reverse proxy
  • X-Internal-Auth secures service-to-service calls

Documentation Written for Engineers

Human-written technical docs covering deployment, maintenance, and troubleshooting.

Browse docs
  • Step-by-step troubleshooting guides
  • JavaScript rendering diagnosis with HAR debug
  • Complete log reference
  • NGINX and HAProxy integration examples
  • Prometheus metrics reference

Ready to Deploy?

Get EdgeComet running in your infrastructure.

$ curl -X POST https://api.edgecomet.com/render
{"url": "https://your-spa.com/page"}
Get a Quote How It Works