Skip to content
Start a project

SEO

Technical SEO before a site launches: the checks that decide whether Google indexes it

· Head of Design — UI/UX & Web9 min readMore in SEO
Two people at a worktable in a Dubai studio at dusk, one at a laptop and one holding a ticked checklist, with two phones on the table showing the same page layout, one of them mirrored

Many new sites that are missing from Google have an indexing problem, not a ranking one. The checks we run before a new or rebuilt site goes live, from robots.txt and redirects to hreflang for Arabic, sourced from Google's own documentation.

On this page

A new website can be well built and still be missing from Google weeks after launch. The cause is rarely ranking. It is indexing: Google was told not to crawl the site, told not to index it, or sent to the wrong version of it.

This is our pre-launch checklist for a new or rebuilt site, each point from Google's own documentation. Google's SEO starter guide says there is "no guarantee that any particular site will be added to Google's index", so none of this promises a ranking; it removes the reasons not to index.

robots.txt and noindex do different jobs

  • robots.txt controls crawling. Google's robots.txt guide calls it "not a mechanism for keeping a web page out of Google"; a blocked URL can still appear in results, without a description.
  • noindex controls indexing, as a robots meta tag or X-Robots-Tag header, but only where Google can crawl. Block the page in robots.txt and "the crawler will never see the noindex rule".

The staging block that goes live

Staging is often hidden with a site-wide "Disallow: /" or noindex, and on launch day the block ships with the code. Google's site-move guide warns: "Don't forget to remove any noindex or robots.txt blocks that were only needed for the migration". Nor is the fix instant: Google generally caches robots.txt for up to 24 hours.

So we password-protect staging, as the robots.txt guide suggests, and production has nothing to remove. We check production's response headers too, since a noindex header never shows in the page source. And robots.txt must return a file or a 404: on a server error, Google stops crawling for the first 12 hours.

A sitemap, submitted

Google's sitemap rules:

  • Absolute URLs of the canonical pages you want in results.
  • At most 50,000 URLs or 50 MB uncompressed per file.
  • An accurate lastmod date. Google ignores priority and changefreq.
  • A Sitemap line in robots.txt, and a submission in Search Console's Sitemaps report.

Google calls submitting one "merely a hint". We verify Search Console as a Domain property, which covers every subdomain and protocol.

Redirects from the old site

This is where rebuilds lose traffic. The site-move guide's rules:

  • Map every old URL to its new equivalent, using the old sitemap, analytics and Search Console's links report.
  • Use server-side permanent (301) redirects, straight to the final URL rather than through a chain. Google says they "don't cause a loss in PageRank".
  • Don't send old URLs to the home page by default; Google says that "might be treated as a soft 404". Removed content should return a 404 or 410.
  • Keep the redirects "for as long as possible, generally at least 1 year".
  • Use the Change of Address tool only when the domain or subdomain changes.

Even done well, expect temporary ranking fluctuation: Google says a small to medium site "can take a few weeks for most pages to move".

A printed redirect map on a desk: two columns of abstract lines joined row by row with pencil arrows, beside a highlighter and a laptop
Every old URL gets its own new home; nothing goes to the home page by default.

Canonical tags

When a page is reachable at several URLs, with or without www or with tracking parameters, Google picks one to show. Its canonical guide rates redirects and rel="canonical" as strong signals and sitemaps as a weak one, and they should agree:

  • A self-referencing, absolute canonical in every page's head.
  • The same URL in canonical, sitemap and internal links.
  • No noindex or robots.txt to choose a canonical.
  • On a bilingual site, a canonical in the same language: an Arabic page canonicalised to the English one asks Google to show the English.

hreflang for the Arabic and English versions

Google's multilingual guidance asks for a separate URL per language and no automatic redirects by language: Googlebot usually crawls from the USA without an Accept-Language header, so it may never see the Arabic. Then hreflang:

  • Each version lists itself and every other version. If two pages don't point to each other, "the tags will be ignored".
  • Fully qualified https URLs.
  • The language first, a region optionally: en, ar or ar-AE. A country code alone is not allowed, and "ae" is not the UAE: it is the ISO 639-1 code for Avestan.
  • An x-default for a language selector or a home page that redirects.

Google detects a page's language from its visible content, not hreflang or the lang attribute, so the Arabic page needs Arabic content, not just Arabic menus. The design side is in what goes wrong with bilingual Arabic and English websites.

HTTPS, with no exceptions

The canonical guide says Google prefers the HTTPS version unless something contradicts it: an invalid certificate, insecure dependencies, a redirect through HTTP or a canonical pointing to HTTP. So every HTTP URL redirects to HTTPS; sitemaps, canonicals and hreflang list only HTTPS URLs; and the certificate covers both www and the bare domain.

Do it for security and the right canonical, not ranking. Google's page experience guidance counts secure serving as one aspect, but says those beyond Core Web Vitals "don't directly help your website rank higher".

Mobile-first indexing

Google indexes and ranks "the mobile version of a site's content, crawled with the smartphone agent". The mobile page needs the same content, headings, structured data, alt text and robots meta tags as desktop. Accordions and tabs are fine; content that loads only when someone taps, swipes or types is not seen. Responsive design, which Google recommends, removes most of the risk.

JavaScript rendering

Google renders JavaScript from a queue where a page may wait "a few seconds, but it can take longer than that", and still calls server-side or pre-rendering "a great idea". What we check:

  • Content, title, canonical and hreflang in the HTML the server sends.
  • Real links with an href, and History API routes rather than # fragments.
  • No noindex in the initial HTML of a page meant to be indexed: Google may skip rendering, so a script that removes it may not work.
  • Scripts and styles not blocked in robots.txt.
  • Missing pages returning a real 404 status.

Structured data, and what it really does

Structured data can make a page eligible for rich results, but Google "does not guarantee that your structured data will show up in search results", and a structured data manual action costs a page its rich results, not its ranking. Features also disappear: FAQ rich results stopped showing in Google Search on 7 May 2026.

What we add, where it is true and visible on the page:

  • Organization, plus LocalBusiness with address and hours for a business customers visit.
  • BreadcrumbList for the site structure.
  • Product for a shop, Article for the blog.

Not star ratings for your own business: Google supports LocalBusiness and Organization review snippets only on sites that review other businesses.

Core Web Vitals, in proportion

Google's targets are LCP (loading) within 2.5 seconds, INP (responsiveness) under 200 milliseconds and CLS (visual stability) under 0.1. The page experience guidance says "Core Web Vitals are used by our ranking systems", and also that Search "always seeks to show the most relevant content, even if the page experience is sub-par". Fix a slow site before launch, but don't expect speed to rescue thin content.

On a new site the Core Web Vitals report may show no data until enough real visits build up. More in Core Web Vitals are a business metric.

The first weeks in Search Console

On launch day, inspect the home page and one page per template in both languages, request indexing for the few that matter most, and submit the sitemap. On a rebuild, keep the old sitemap submitted too, to watch indexing move across. Then, weekly:

  • Page indexing. Pages you want that are blocked by robots.txt or noindex are a launch bug. "Duplicate, Google chose different canonical than user" means Google preferred another URL to your canonical. Google says not to expect every URL to be indexed, only the canonical ones.
  • Performance. Impressions and clicks moving to the new URLs.

Be sceptical of shortcuts. Google says crawling "can take anywhere from a few days to a few weeks", and that asking repeatedly "won't get it crawled any faster".

Almost all of this is decided before launch. A site that passes these checks has told Google what exists, which version to show and in which language; whether it ranks then depends on content and competition, which nobody can promise. We run this list on every site we build or rebuild.

Sources

  • technical SEO
  • Google Search Console
  • site migration
  • hreflang
  • Dubai

Start a project

Tell us what you need.

A website or an online shop, the photography and film for it, or the customers to it. Or all of them, planned as one.

Brands we have worked with

ToneaFlipStop&GoLobosMaven BlendSamimi DCSBingooGCCOMahboob EstateRoza General TradingMARC HomePartMate