Troubleshooting Guide

BotRender Troubleshooting

Comprehensive guide to diagnose and fix common issues with BotRender integration. Get your prerendering back on track quickly.

How BotRender Works

Understanding the workflow helps identify where issues occur

Bot Request

Search engine bot requests your page

Detection

Your middleware detects bot and forwards to BotRender

Rendering

BotRender fetches and renders your JavaScript page

Delivery

Pre-rendered HTML sent back to bot

Common Issues & Solutions

Bots Blocked by Your Server

Your server may reject bot requests, preventing BotRender from intercepting them

high priority

Symptoms

No prerendered content served to search engines
Missing BotRender headers in responses
403 Forbidden errors in logs

Solutions

Ensure bots like Googlebot aren't blocked by firewall or .htaccess rules
Allow user agents listed in BotRender's comprehensive bot list
Check your CDN settings for bot blocking rules
Verify Apache mod_rewrite or Nginx proxy modules are enabled

Middleware Misconfiguration

Middleware attached in wrong order can prevent BotRender from working properly

high priority

Symptoms

Bot traffic bypasses BotRender service
Regular users get prerendered content
Inconsistent behavior across routes

Solutions

Attach BotRender middleware before others that serve JavaScript or APIs
Verify configuration matches BotRender's integration guidelines
Check middleware order in your server configuration
Test with different user agents to confirm routing

Geo or IP-Based Blocking

Geo-based filters or IP restrictions may prevent BotRender servers from accessing your site

medium priority

Symptoms

Timeouts when accessing from certain regions
Empty or error responses in prerendered content
Regional crawling inconsistencies

Solutions

Allow BotRender's IP ranges in your firewall settings
Test access by simulating requests using BotRender's user agent
Configure CDN to allow BotRender service requests
Check for geo-blocking rules in your hosting provider

Private Staging Sites Blocked

Private staging sites may restrict BotRender's access

low priority

Symptoms

Development/staging environments not rendering
Authentication errors on non-production sites
Missing content in staging prerendered pages

Solutions

Enable access for BotRender user agents on staging
Use smart firewall to whitelist BotRender's IP addresses
Configure basic auth bypass for BotRender service
Test staging access with curl and BotRender user agent

CDN Blocking BotRender User-Agent

CDNs may block requests from non-standard user agents, including BotRender

medium priority

Symptoms

403 Forbidden responses from CDN
Empty responses when using BotRender user agent
JavaScript errors in prerendered content

Solutions

Allow BotRender's user agents in your CDN settings
Simulate requests using BotRender user agent to test
Check CDN WAF rules for bot blocking
Create exceptions for BotRender service in security rules

Empty or Raw Content Returned

BotRender returns incomplete or unrendered content

high priority

Symptoms

Missing dynamic content in prerendered pages
JavaScript not fully executed
API calls not completed before rendering

Solutions

Use window.prerenderReady flag to control rendering timing
Ensure API calls complete before marking page ready
Check for JavaScript errors that prevent rendering
Verify async content loads properly

How to Test Your BotRender Integration

Verify that your pages are being prerendered correctly

curl Command Testing

Test your site with BotRender user agent using curl

curl -H "User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://your-site.com

Browser User Agent Override

Use Chrome DevTools to test with different user agents

Open Chrome DevTools (F12)
Click three dots menu → More tools → Network conditions
Uncheck 'Use browser default' under User agent
Select 'Googlebot' or add custom bot user agent
Reload page to see prerendered content

Header Verification

Check for BotRender headers in responses

Look for these headers:

X-Botrender-Token: [your-token]

X-Prerender-Status: success

X-Prerender-Timestamp: [render-time]

Quick Diagnostic Checklist

✅ Integration Checklist

BotRender middleware installed and configured
Organization UID and API token properly set
Server allows bot user agents (no blocking)
Firewall/CDN configured to allow BotRender

🔍 Testing Results

curl with bot user agent returns prerendered HTML
Response includes BotRender headers
Dynamic content appears in prerendered version
Regular users still get interactive JavaScript

Still Having Issues?

Our support team is here to help. Get personalized assistance with your BotRender integration.

Contact Support View Setup Examples

Common Issues & Solutions

Bots Blocked by Your Server

Your server may reject bot requests, preventing BotRender from intercepting them

high priority

Symptoms

No prerendered content served to search engines
Missing BotRender headers in responses
403 Forbidden errors in logs

Solutions

Ensure bots like Googlebot aren't blocked by firewall or .htaccess rules
Allow user agents listed in BotRender's comprehensive bot list
Check your CDN settings for bot blocking rules
Verify Apache mod_rewrite or Nginx proxy modules are enabled

Middleware Misconfiguration

Middleware attached in wrong order can prevent BotRender from working properly

high priority

Symptoms

Bot traffic bypasses BotRender service
Regular users get prerendered content
Inconsistent behavior across routes

Solutions

Attach BotRender middleware before others that serve JavaScript or APIs
Verify configuration matches BotRender's integration guidelines
Check middleware order in your server configuration
Test with different user agents to confirm routing

Geo or IP-Based Blocking

Geo-based filters or IP restrictions may prevent BotRender servers from accessing your site

medium priority

Symptoms

Timeouts when accessing from certain regions
Empty or error responses in prerendered content
Regional crawling inconsistencies

Solutions

Allow BotRender's IP ranges in your firewall settings
Test access by simulating requests using BotRender's user agent
Configure CDN to allow BotRender service requests
Check for geo-blocking rules in your hosting provider

Private Staging Sites Blocked

Private staging sites may restrict BotRender's access

low priority

Symptoms

Development/staging environments not rendering
Authentication errors on non-production sites
Missing content in staging prerendered pages

Solutions

Enable access for BotRender user agents on staging
Use smart firewall to whitelist BotRender's IP addresses
Configure basic auth bypass for BotRender service
Test staging access with curl and BotRender user agent

CDN Blocking BotRender User-Agent

CDNs may block requests from non-standard user agents, including BotRender

medium priority

Symptoms

403 Forbidden responses from CDN
Empty responses when using BotRender user agent
JavaScript errors in prerendered content

Solutions

Allow BotRender's user agents in your CDN settings
Simulate requests using BotRender user agent to test
Check CDN WAF rules for bot blocking
Create exceptions for BotRender service in security rules

Empty or Raw Content Returned

BotRender returns incomplete or unrendered content

high priority

Symptoms

Missing dynamic content in prerendered pages
JavaScript not fully executed
API calls not completed before rendering

Solutions

Use window.prerenderReady flag to control rendering timing
Ensure API calls complete before marking page ready
Check for JavaScript errors that prevent rendering
Verify async content loads properly