Next.js

Integrate xseek with Next.js using our middleware solution. Track and optimize your Next.js website for AI search engines and improve visibility.

Overview

Our Next.js middleware integration allows you to add AI SEO tracking capabilities to your Next.js applications with minimal configuration. By leveraging the built-in middleware capabilities of Next.js, you can track AI interactions across your entire site without modifying individual pages. This solution works with both app router and pages router applications and provides complete coverage of your Next.js site.

Features

Seamless integration with Next.js middleware
Automatic tracking of all routes
Real-time AI visibility analytics
Easy installation with npm/yarn/pnpm
Detailed performance insights for Next.js applications
Compatible with app router and pages router
Zero impact on site performance
Tracks all major AI systems including ChatGPT, Claude, and Perplexity

Requirements

Next.js 12.0.0 or higher
API key from xseek
Website ID from your xseek dashboard

Setup Process

1
Create an AI detection module in your Next.js project
2
Add the detection function to your middleware.ts file
3
Deploy your Next.js application to start tracking AI visits

Integration Setup

Important Information

Make sure to replace YOUR_API_KEY and YOUR_WEBSITE_ID with your actual values in the integration code. Your API key can be found in your account settings.

1Add detection code to your Next.js project

Create a new file (e.g., lib/ai-detection.js) with the following code:

// AI crawler detection for all major companies
const AI_BOTS = [
    { name: 'anthropic-ai', pattern: /anthropic-ai/i },
    { name: 'claudebot', pattern: /ClaudeBot/i },
    { name: 'claude-web', pattern: /claude-web/i },

    { name: 'perplexitybot', pattern: /PerplexityBot/i },
    { name: 'perplexity-user', pattern: /Perplexity-User/i },

    { name: 'grokbot', pattern: /GrokBot(?!.*DeepSearch)/i },
    { name: 'grok-search', pattern: /xAI-Grok/i },
    { name: 'grok-deepsearch', pattern: /Grok-DeepSearch/i },

    { name: 'deepseekbot', pattern: /DeepSeekBot/i },

    { name: 'GPTBot', pattern: /GPTBot/i },
    { name: 'chatgpt-user', pattern: /ChatGPT-User/i },
    { name: 'oai-searchbot', pattern: /OAI-SearchBot/i },

    { name: 'google-extended', pattern: /Google-Extended/i },

    { name: 'applebot', pattern: /Applebot(?!-Extended)/i },
    { name: 'applebot-extended', pattern: /Applebot-Extended/i },

    { name: 'meta-external', pattern: /meta-externalagent/i },
    { name: 'meta-externalfetcher', pattern: /meta-externalfetcher/i },

    { name: 'bingbot', pattern: /Bingbot(?!.*AI)/i },
    { name: 'bingpreview', pattern: /bingbot.*Chrome/i },
    { name: 'microsoftpreview', pattern: /MicrosoftPreview/i },

    { name: 'cohere-ai', pattern: /cohere-ai/i },
    { name: 'cohere-training-data-crawler', pattern: /cohere-training-data-crawler/i },

    { name: 'youbot', pattern: /YouBot/i },

    { name: 'duckassistbot', pattern: /DuckAssistBot/i },

    { name: 'semanticscholarbot', pattern: /SemanticScholarBot/i },

    { name: 'ccbot', pattern: /CCBot/i },

    { name: 'ai2bot', pattern: /AI2Bot/i },
    { name: 'ai2bot-dolma', pattern: /AI2Bot-Dolma/i },

    { name: 'aihitbot', pattern: /aiHitBot/i },

    { name: 'amazonbot', pattern: /Amazonbot/i },
    { name: 'novaact', pattern: /NovaAct/i },

    { name: 'brightbot', pattern: /Brightbot/i },

    { name: 'bytespider', pattern: /Bytespider/i },
    { name: 'tiktokspider', pattern: /TikTokSpider/i },

    { name: 'cotoyogi', pattern: /Cotoyogi/i },

    { name: 'crawlspace', pattern: /Crawlspace/i },

    { name: 'pangubot', pattern: /PanguBot/i },
    { name: 'petalbot', pattern: /PetalBot/i },

    { name: 'semrushbot-ocob', pattern: /SemrushBot-OCOB/i },
    { name: 'semrushbot-swa', pattern: /SemrushBot-SWA/i },

    { name: 'sidetrade-indexer', pattern: /Sidetrade indexer bot/i },

    { name: 'timpibot', pattern: /Timpibot/i },

    { name: 'velenpublicwebcrawler', pattern: /VelenPublicWebCrawler/i },

    { name: 'omgili', pattern: /omgili/i },
    { name: 'omgilibot', pattern: /omgilibot/i },
    { name: 'webzio-extended', pattern: /Webzio-Extended/i }
];

// Function to detect and log AI crawler visits
export function detectAndLogAIBot(req) {
    const userAgent = req.headers.get('user-agent') || '';
    
    for (const bot of AI_BOTS) {
        if (bot.pattern.test(userAgent)) {
            // Send data to xseek API
            fetch(`https://api.aiseotracker.com/track-ai-bot`, {
                method: 'POST',
                headers: {
                    'Content-Type': 'application/json',
                    'x-api-key': 'YOUR_API_KEY', // Replace with your actual API key
                },
                body: JSON.stringify({
                    botName: bot.name,
                    userAgent: userAgent,
                    url: req.nextUrl.toString(),
                    ip: req.ip || req.headers.get('x-forwarded-for') || undefined,
                    referer: req.headers.get('referer') || undefined,
                    websiteId: 'YOUR_WEBSITE_ID', // Add your website ID to track which website is being crawled
                }),
            }).catch(error => {
                console.error('Error calling track-ai-bot API:', error);
            });
            
            console.log(`AI crawler detected: ${bot.name}`);
            break;
        }
    }
}

2Call the function in your middleware

Add this code to your middleware.ts file:

// Add this to your middleware.ts
import { NextResponse } from 'next/server';

export function middleware(req) {
    // Detect and log AI bots
    detectAndLogAIBot(req);

    // Your other middleware logic...
    return NextResponse.next();
}

That's it!

Your Next.js app is now set up to track AI crawler visits. Come back to the xseek dashboard to see your data.

Need more help?

Check our comprehensive documentation for detailed instructions.

View documentation