In brief
Use specific URLs and evidence at handover. What matters is the response actually served, not merely the existence of a setting.
First decide which pages belong in search results
Make a short URL list of valuable pages: services, about, contact and useful articles. Treat customer interfaces, carts and test pages separately. Set indexing rules according to each page’s role rather than relying on a single global switch.
At handover, inspect both the live HTML robots meta tag and the HTTP X-Robots-Tag header. Resolve any conflicts. A robots.txt crawling rule and noindex serve different purposes; do not assume blocking a URL makes a search engine automatically forget the page.
One consistent URL for each piece of content
Canonicals, sitemaps and internal navigation should point to the same preferred URL. Check the behavior of www and non-www, HTTP and HTTPS, and any legacy routes. Avoid redirect chains that add unnecessary steps to a straightforward page request.
Search and sorting parameters need a separate review. A blog’s search results page usually does not offer the same value as an individual article. Pagination should still let both visitors and search engines reach older articles.
Status codes are part of the page too
A nonexistent URL should return a genuine 404, not error text with HTTP 200. Removed content and changed routes are different cases: redirect to a relevant replacement when one exists, but do not send every broken URL to the homepage. This also makes operational problems easier to detect.
Prepare specific test URLs: an existing page, a nonexistent page, a legacy URL and a parameterized version. The developer handover should include response codes and final destinations. Automated checks can repeat this after each significant release.
Internal links and sitemaps
Connect individual pages with real, navigable links. The sitemap should contain only appropriate canonical, indexable URLs. Change lastmod when content changes substantively, not on every request. A sitemap can help uncover an article, but logical categories and related articles help visitors too.
Checking a sitemap does not end with opening its XML successfully. Sample several URLs and verify their responses and canonicals. Check that test pages, internal search or customer interfaces were not included accidentally.
Handover and follow-up
Request a list of checked pages, fixed issues and outstanding tasks. The authorized business should retain access to Search Console. Use URL Inspection and indexing reports to track actual indexing; submitting a sitemap does not mean immediate inclusion.
Assign a content owner as well as completing the technical handover. If a service ends or an article becomes outdated, maintain the site structure and links too. Good technical foundations help search engines, but do not promise traffic or first place on their own.
Example: a staging rule remains in production
A preview may correctly use noindex, but this must be reviewed at launch. Changing a source-code setting is not enough if the live environment serves an older version or an HTTP header still blocks indexing. A page that opens in a browser can therefore still be excluded from indexing.
After deployment, request fresh responses for an important service page and an article. Check the meta element, response headers and sitemap. If caching is involved, verify the update on the live URL. The plan should specify who performs this final check and when.
Priority: what gets in the visitor’s way?
Do not judge a technical audit by the number of minor warnings. Prioritize unavailable important pages, indexing blocks, broken redirects and faulty purchasing journeys. A missing optional meta element is not necessarily more important than an unusable contact form.
For every issue, specify the affected URL, impact and verifiable completion criteria. For example, an old service URL should redirect in one step to its relevant replacement, and the menu should already use the new URL. The fix then becomes a concrete, testable change rather than an increase in a vague SEO score.
- Check the live URL: status code, indexability and canonical URL.
- Record a baseline: date, affected page and a reproducible test.
- Measure again after the change: also test important user actions.
