Troubleshooting Guide
BotRender Troubleshooting
Comprehensive guide to diagnose and fix common issues with BotRender integration. Get your prerendering back on track quickly.
How BotRender Works
Understanding the workflow helps identify where issues occur
1
Bot Request
Search engine bot requests your page
2
Detection
Your middleware detects bot and forwards to BotRender
3
Rendering
BotRender fetches and renders your JavaScript page
4
Delivery
Pre-rendered HTML sent back to bot
Common Issues & Solutions
Bots Blocked by Your Server
Your server may reject bot requests, preventing BotRender from intercepting them
high priority
Symptoms
- No prerendered content served to search engines
- Missing BotRender headers in responses
- 403 Forbidden errors in logs
Solutions
- Ensure bots like Googlebot aren't blocked by firewall or .htaccess rules
- Allow user agents listed in BotRender's comprehensive bot list
- Check your CDN settings for bot blocking rules
- Verify Apache mod_rewrite or Nginx proxy modules are enabled
Middleware Misconfiguration
Middleware attached in wrong order can prevent BotRender from working properly
high priority
Symptoms
- Bot traffic bypasses BotRender service
- Regular users get prerendered content
- Inconsistent behavior across routes
Solutions
- Attach BotRender middleware before others that serve JavaScript or APIs
- Verify configuration matches BotRender's integration guidelines
- Check middleware order in your server configuration
- Test with different user agents to confirm routing
Geo or IP-Based Blocking
Geo-based filters or IP restrictions may prevent BotRender servers from accessing your site
medium priority
Symptoms
- Timeouts when accessing from certain regions
- Empty or error responses in prerendered content
- Regional crawling inconsistencies
Solutions
- Allow BotRender's IP ranges in your firewall settings
- Test access by simulating requests using BotRender's user agent
- Configure CDN to allow BotRender service requests
- Check for geo-blocking rules in your hosting provider
Private Staging Sites Blocked
Private staging sites may restrict BotRender's access
low priority
Symptoms
- Development/staging environments not rendering
- Authentication errors on non-production sites
- Missing content in staging prerendered pages
Solutions
- Enable access for BotRender user agents on staging
- Use smart firewall to whitelist BotRender's IP addresses
- Configure basic auth bypass for BotRender service
- Test staging access with curl and BotRender user agent
CDN Blocking BotRender User-Agent
CDNs may block requests from non-standard user agents, including BotRender
medium priority
Symptoms
- 403 Forbidden responses from CDN
- Empty responses when using BotRender user agent
- JavaScript errors in prerendered content
Solutions
- Allow BotRender's user agents in your CDN settings
- Simulate requests using BotRender user agent to test
- Check CDN WAF rules for bot blocking
- Create exceptions for BotRender service in security rules
Empty or Raw Content Returned
BotRender returns incomplete or unrendered content
high priority
Symptoms
- Missing dynamic content in prerendered pages
- JavaScript not fully executed
- API calls not completed before rendering
Solutions
- Use window.prerenderReady flag to control rendering timing
- Ensure API calls complete before marking page ready
- Check for JavaScript errors that prevent rendering
- Verify async content loads properly
How to Test Your BotRender Integration
Verify that your pages are being prerendered correctly
curl Command Testing
Test your site with BotRender user agent using curl
curl -H "User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://your-site.com
Browser User Agent Override
Use Chrome DevTools to test with different user agents
- Open Chrome DevTools (F12)
- Click three dots menu → More tools → Network conditions
- Uncheck 'Use browser default' under User agent
- Select 'Googlebot' or add custom bot user agent
- Reload page to see prerendered content
Header Verification
Check for BotRender headers in responses
Look for these headers:
X-Botrender-Token: [your-token]
X-Prerender-Status: success
X-Prerender-Timestamp: [render-time]
Quick Diagnostic Checklist
✅ Integration Checklist
- BotRender middleware installed and configured
- Organization UID and API token properly set
- Server allows bot user agents (no blocking)
- Firewall/CDN configured to allow BotRender
🔍 Testing Results
- curl with bot user agent returns prerendered HTML
- Response includes BotRender headers
- Dynamic content appears in prerendered version
- Regular users still get interactive JavaScript
Still Having Issues?
Our support team is here to help. Get personalized assistance with your BotRender integration.