Google analyst Gary Illyes has recently underscored two key issues that affect web crawling and SEO: soft 404 errors and related “crypto” mistakes. If not addressed, these can substantially hinder SEO strategies.
Soft 404 errors are essentially the misdirection of nonexistent web pages to a ‘200 OK’ status page. This confuses search engines and may affect inappropriate indexing. Similarly, ‘crypto’ errors refer to misconfigured security settings that may block web crawlers from necessary data, which directly disrupts efficient indexing.
These challenges, if not promptly rectified, can cause significant visibility issues for websites. Illyes emphasises the importance of regularly auditing websites to identify and correct these problems, ensuring sites are easily crawlable and can meet their SEO objectives.
Soft 404 errors are comparable to a customer expecting service in a café, only to find that nothing on the menu is available. This misleads not just the Google algorithm, but also the user. A major cause of soft 404 errors is incorrect redirects, where a page that should return a ‘404 Not Found’ error instead produces a ‘200 OK’ status. To maintain a strong online presence, website owners should regularly check for and fix such errors, focusing mainly on eliminating broken links and setting appropriate redirects.
Illyes describes soft errors as ‘resource wastage’.
Understanding and mitigating SEO obstacles through soft 404 errors
Crawlers use codes to identify successful fetches, but soft 404 errors result in them revisiting the same faulty pages, unable to differentiate between valuable and redundant resources. Correcting these errors can significantly increase crawler efficiency, thereby reducing resource wastage. Implementing correct redirects and custom error pages are some of the recommended actions in addressing these issues, preventing the crawler from wasting time on nonviable pages.
Illyes further elucidates the broader implications of soft 404 errors, suggesting that they can exclude web pages from search results. He advises using appropriate status codes while encountering errors, and highlights the importance of a robust site structure and solving any 404 errors to avoid impacting a website’s search visibility. Regular inspection of site performance can aid early detection and resolution of such issues, optimising the website for a better user experience.
A well-maintained website aids not only SEO efficacy but also provides a seamless user experience that can attract and retain more visitors. Illyes recommends several steps to identify and rectify soft 404 errors: regular website inspections, accurate handling of errors with corresponding HTTP codes, and using tools like Google Search Console for tracking site coverage.
He also suggests updating website content regularly, making sure pages don’t contain outdated or irrelevant information that may trigger soft 404 errors. Proper use of 404 pages, by offering alternatives and redirects to accessible pages, can enhance user experience and website traffic. Also, configuring servers correctly is crucial, as misconfigured servers may send incorrect response codes. Frequent checks on Google Search Console are necessary for monitoring soft 404 error reports and for maintaining active communication with the website’s admin and developers to ensure immediate detection and resolution of emerging soft 404 errors.
