ChatGPT is greater than only a prompting and response platform. You possibly can ship prompts to ask for assist with search engine marketing, nevertheless it turns into extra highly effective the second that you just make your individual agent.
I conduct many search engine marketing audits – it’s a necessity for an enterprise web site – so I used to be in search of a approach to streamline a few of these processes.
How did I do it? By making a ChatGPT agent that I’m going to share with you so as to customise and alter it to satisfy your wants.
I’ll hold issues as “untechnical” as doable, however simply observe the directions, and every thing ought to work.
I’m going to clarify the next steps”
- Configuration of your individual ChatGPT.
- Creating your individual Cloudflare code to fetch a web page’s HTML knowledge.
- Placing your search engine marketing audit brokers to work.
On the finish, you’ll have a bot that gives you with info, similar to:
You’ll additionally obtain a listing of actionable steps to take to enhance your search engine marketing primarily based on the agent’s findings.
Creating A Cloudflare Pages Employee For Your Agent
Cloudflare Pages staff assist your agent collect info from the web site you’re attempting to parse and examine its present state of search engine marketing.
You should use a free account to get began, and you’ll register by doing the next:
- Going to http://pages.dev/
- Creating an account
I used Google to enroll as a result of it’s simpler, however select the strategy you’re most comfy with. You’ll find yourself on a display that appears one thing like this:
Cloudflare Dashboard (Screenshot from Cloudfare, Might 2025)Navigate to Add > Staff.
Add a Cloudflare Employee (Screenshot from Cloudfare, Might 2025)You possibly can then choose a template, import a repository, or begin with Hiya World! I selected the Hiya World possibility, because it’s the best one to make use of.
Deciding on Cloudflare Employee (Screenshot from Cloudfare, Might 2025)Undergo the following display and hit “Deploy.” You’ll find yourself on a display that claims, “Success! Your venture is deployed to Area: Earth.”
Don’t click on off this web page.
As a substitute, click on on “Edit code,” take away all the current code, and enter the next code into the editor:
addEventListener('fetch', occasion => {
occasion.respondWith(handleRequest(occasion.request));
});
async operate handleRequest(request) {
const { searchParams } = new URL(request.url);
const targetUrl = searchParams.get('url');
const userAgentName = searchParams.get('user-agent');
if (!targetUrl) {
return new Response(
JSON.stringify({ error: "Lacking 'url' parameter" }),
{ standing: 400, headers: { 'Content material-Kind': 'utility/json' } }
);
}
const userAgents = {
googlebot: 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Construct/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.184 Cellular Safari/537.36 (appropriate; Googlebot/2.1; +http://www.google.com/bot.html)',
samsung5g: 'Mozilla/5.0 (Linux; Android 13; SM-S901B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Cellular Safari/537.36',
iphone13pmax: 'Mozilla/5.0 (iPhone14,3; U; CPU iPhone OS 15_0 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Model/10.0 Cellular/19A346 Safari/602.1',
msedge: 'Mozilla/5.0 (Home windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246',
safari: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_2) AppleWebKit/601.3.9 (KHTML, like Gecko) Model/9.0.2 Safari/601.3.9',
bingbot: 'Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; appropriate; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/',
chrome: 'Mozilla/5.0 (Home windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36',
};
const userAgent = userAgents[userAgentName] || userAgents.chrome;
const headers = {
'Person-Agent': userAgent,
'Settle for': 'textual content/html,utility/xhtml+xml,utility/xml;q=0.9,*/*;q=0.8',
'Settle for-Encoding': 'gzip',
'Cache-Management': 'no-cache',
'Pragma': 'no-cache',
};
strive {
let redirectChain = [];
let currentUrl = targetUrl;
let finalResponse;
// Comply with redirects
whereas (true) {
const response = await fetch(currentUrl, { headers, redirect: 'handbook' });
// Add the present URL and standing to the redirect chain provided that it isn't already added
if (!redirectChain.size || redirectChain[redirectChain.length - 1].url !== currentUrl) {
redirectChain.push({ url: currentUrl, standing: response.standing });
}
// Examine if the response is a redirect
if (response.standing >= 300 && response.standing < 400 && response.headers.get('location')) { const redirectUrl = new URL(response.headers.get('location'), currentUrl).href; currentUrl = redirectUrl; // Comply with the redirect } else { // No extra redirects; seize the ultimate response finalResponse = response; break; } } if (!finalResponse.okay) { throw new Error(`Request to ${targetUrl} failed with standing code: ${finalResponse.standing}`); } const html = await finalResponse.textual content(); // Robots.txt const area = new URL(targetUrl).origin; const robotsTxtResponse = await fetch(`${area}/robots.txt`, { headers }); const robotsTxt = robotsTxtResponse.okay ? await robotsTxtResponse.textual content() : 'robots.txt not discovered'; const sitemapMatches = robotsTxt.match(/Sitemap:s*(https?://[^s]+)/gi) || []; const sitemaps = sitemapMatches.map(sitemap => sitemap.change('Sitemap: ', '').trim());
// Metadata
const titleMatch = html.match(/]*>s*(.*?)s*</title>/i);
const title = titleMatch ? titleMatch[1] : 'No Title Discovered';
const metaDescriptionMatch = html.match(/<meta>/i);
const metaDescription = metaDescriptionMatch ? metaDescriptionMatch[1] : 'No Meta Description Discovered';
const canonicalMatch = html.match(/<link>/i);
const canonical = canonicalMatch ? canonicalMatch[1] : 'No Canonical Tag Discovered';
// Open Graph and Twitter Data
const ogTags = [])[1] ;
const twitterTags = ;
// Headings
const headings = {
h1: [...html.matchAll(/ You have two things to do at this point:
- Copy the URL to your worker.
- Deploy your worker.
This is the URL you’ll need in the next section. You can find it here:
Cloudflare Worker Preview (Screenshot from Cloudfare, May 2025)Be sure to hit “Deploy” before exiting the screen. If you want to see the basic output at this stage, you can.
Paste your URL into your browser and add the following after the /?url=https://www.searchenginejournal.com.
Your URL will look something like this: https://YOURURL.workers.dev/?url=https://searchenginejournal.com.
Swap out the URL to one of your choosing to test this out. It’s not the “prettiest” at this stage, so it’s now time to move on to the fun part of configuring your own GPT.
Note: This worker is not working with JavaScript-rendered sites. But for everyone else using this agent, it should work fine. Feel free to improve it to work with JavaScript rendering.
Configuring Your Own GPT To Mimic My Agent
You need to first configure your GPT, and you can do this by opening ChatGPT and going to “Explore GPTs,” or you can simply follow this link.
Explore GPTs (Screenshot from ChatGPT, May 2025)You’ll then go to “+ Create“:
Create GPT (Screenshot from ChatGPT, May 2025)Now, it will say “Create” and “Configure.” Go to Configure and start plugging in your information.
You can feel free to change things up a bit, but I recommend following everything I’m adding below to build the foundation of your auditor.
You’ll add what I’m going to list in this section:
Configure GPT (Screenshot from ChatGPT, May 2025)Name: OnPage SEO Audit
Description: Analyze SEO performance of any webpage using custom user-agents. Get detailed insights into metadata, redirect chains, Open Graph tags, Twitter Cards, sitemaps, and more. Perfect for SEO professionals and developers.
Instructions:
Trigger: When a user submits a URL (required) and an optional user-agent:
Instruction: Use the provided inputs to make an API request to retrieve SEO data. Default to the chrome user-agent if not provided.
Trigger: When the API returns valid data:
Instruction: Analyze the data and provide:
A summary of the page's SEO performance.
Actionable suggestions for improvement, categorized into metadata, technical SEO, and content.
Follow-up questions to clarify user priorities or goals, such as:
"Do you have specific goals for this page, such as improving search visibility, click-through rates, or user engagement?"
"Would you like me to focus on technical SEO or content-related improvements first?"
Example Response:
"The page's meta description is missing, which can impact click-through rates. Would you like me to suggest a draft description?"
Trigger: When the API returns HTTP 403:
Instruction:
Retry the request using the chrome user-agent.
If the issue persists:
Notify the user of the problem.
Suggest verifying the URL or user-agent compatibility.
Trigger: When the API returns a 400 error:
Instruction:
Clearly explain the error and provide actionable steps to resolve it (e.g., verify the URL format or ensure required parameters are provided).
Trigger: When data is incomplete or missing:
Instruction:
Request additional information from the user or permission to explore fallback data sources.
Example Follow-Up:
"The API response is missing a meta description for this page. Can you confirm if this was intentional, or should we explore other sources?"
Additional Guidelines:
Include:
A categorized summary of the page's SEO performance (e.g., metadata, technical SEO, content).
A prioritized list of recommended actions.
Visual examples or detailed explanations, when applicable.
Proactively address multiple detected issues with follow-up questions:
"The page has several critical issues, including missing Open Graph tags and a non-canonical URL. Would you like me to prioritize recommendations for social media or canonicalization first?"
Conversation starters
User-Agent: Googlebot, URL: https://example.com
Analyze the SEO details for https://example.com using Googlebot.
Analyze the page using the Samsung Galaxy S22 user-agent.
What metadata is available for https://example.com with Chrome?
Capabilities
Web Search
Code Interpreter & Data Analysis
Your configuration at this point should look something like this:
GPT Configurations Populated (Screenshot from ChatGPT, May 2025)Review all of these fields and see if you have populated each properly before moving on to Create new action.
Navigate to Authentication and choose None.
GPT Authentication (Screenshot from ChatGPT, May 2025)Add the following ChatGPT action coding to the Schema field, but be sure to change the servers > url field to your own URL. I’ll name it “https://CHANGETOYOURURL.com/” so that it’s easy for you to find.
{
"openapi": "3.1.0",
"info": {
"title": "Enhanced SEO Analysis and Audit API",
"description": "Fetch SEO data for analysis. Use the returned data to generate actionable SEO recommendations using AI or experts.",
"version": "1.2.0"
},
"servers": [
{
"url": "https://CHANGETOYOURURL.com/",
"description": "Base URL for Enhanced SEO Analysis API"
}
],
"paths": {
"/": {
"get": {
"operationId": "fetchAndAuditSEOData",
"abstract": "Fetch and Audit search engine marketing Information",
"description": "Retrieve search engine marketing evaluation knowledge utilizing a user-agent and URL and carry out a primary search engine marketing audit.",
"parameters": [
{
"name": "user-agent",
"in": "query",
"description": "The user-agent for the request.",
"required": true,
"schema": {
"type": "string",
"enum": ["chrome", "googlebot", "iphone13pmax", "samsung5g"]
}
},
{
"identify": "url",
"in": "question",
"description": "The URL of the webpage to research.",
"required": true,
"schema": {
"kind": "string",
"format": "uri"
}
}
],
"responses": {
"200": {
"description": "Profitable response with audit outcomes",
"content material": {
"utility/json": {
"schema": {
"kind": "object",
"properties": {
"metadata": {
"kind": "object",
"properties": {
"title": { "kind": "string" },
"metaDescription": { "kind": "string" },
"canonical": { "kind": "string" }
}
},
"redirectChain": {
"kind": "array",
"gadgets": {
"kind": "object",
"properties": {
"url": { "kind": "string" },
"standing": { "kind": "integer" }
}
}
},
"openGraph": {
"kind": "object",
"properties": {
"ogTitle": { "kind": "string" },
"ogDescription": { "kind": "string" },
"ogImage": { "kind": "string" }
}
},
"twitterCards": {
"kind": "object",
"properties": {
"twitterTitle": { "kind": "string" },
"twitterDescription": { "kind": "string" },
"twitterImage": { "kind": "string" }
}
},
"sitemaps": {
"kind": "array",
"gadgets": { "kind": "string" }
},
"robotsTxt": {
"kind": "string"
},
"audit": {
"kind": "object",
"properties": {
"points": {
"kind": "array",
"gadgets": { "kind": "string" }
},
"suggestions": {
"kind": "array",
"gadgets": { "kind": "string" }
}
}
},
"auditSummary": {
"kind": "array",
"gadgets": {
"kind": "string"
}
},
"nextSteps": {
"kind": "array",
"gadgets": {
"kind": "string"
}
}
}
}
}
}
},
"400": {
"description": "Unhealthy Request",
"content material": {
"utility/json": {
"schema": {
"kind": "object",
"properties": {
"error": { "kind": "string" }
}
}
}
}
}
}
}
}
}
}
It’s best to see it beneath “Out there actions: fetchAndAuditSEOData.”
Below Privateness Coverage, add a hyperlink to your individual privateness coverage.
Lastly, faucet “Create” on the prime proper and observe the prompts.
Now you can view your GPT; will probably be just like this On-Web page search engine marketing Audit GPT.
Testing Your GPT And Getting Acquainted With Your Choices
You’ve come a great distance with your individual GPT, and it’s time to check issues out.
Faucet on the second tile, “Analyze the search engine marketing particulars.” It’ll default to instance.com, however you’ll be able to immediate it to check a URL you want.
Let’s strive: netflix.com by prompting, “Use netflix.com because the URL.”
Screenshot from ChatGPT, Might 2025Now you can experiment with any of the GPT choices obtainable to see how every thing works collectively.
Customizing Your GPT Additional
You might wish to customise your GPT additional by going again to the place you created it and updating a couple of issues.
Replace your dialog starters to regulate:
- Person-agents.
- Edit directions to raised meet your wants by including triggers and responses.
If you happen to like, go into the coding in your employee and add to “const userAgents” so as to add extra consumer brokers to the checklist.
Now you can go to your customized GPT agent and easily immediate it on what web page to scan subsequent.
It’s straightforward to do that by prompting one thing like this: “Change the URL to THEDESIREDURL,” and your agent will get to be just right for you.
In Abstract
This tradition GPT agent is only one instance of how combining ChatGPT with Cloudflare Staff can streamline core search engine marketing duties.
Experiment with the Agent and alter it in your wants.
AI has the capability to assist with many duties and it’s right here to remain. So, embracing this know-how to be an assistant and an efficient device has prospects to assist SEOs be extra environment friendly at scale.
Extra Sources:
Featured Picture: ImageFlow/Shutterstock
