email: ebobnar at gmail.com

Affordable SEO Consulting for the Results-Oriented Online Business.

Welcome to my blog. Here's a list of my best posts, as well as a complete archive of everything I've ever blogged about.

« bCentral is Done, Man | Main | Score 195 Dollars in Free adCenter Clicks for Americans »

November 22, 2006

How Indexable XML Files Can Kill Your Website

This site's been having some pretty serious trouble with supplemental results lately. Since it doesn't make any money I haven't been able to find the motivation to fix it, but I'm going to take a crack at it today.

A few months ago many of the pages on this blog started going supplemental. One unusual thing I noticed was that I had a few RSS feeds indexed, and these XML files created a sort of ceiling on my supplemental results. They would be indexed and ranked, placed in the supplemental index, and then any pages which ranked lower than them would also go supplemental.

So the number of supplemental pages was entirely dependent on where these XML files ranked. If they ranked low, I'd have few supplemental pages. If they ranked high, I'd have many.

Eventually I got rid of most of them by updating my robots.txt file to tell search engines not to crawl my RSS feeds. But one still has been hanging around, ranking higher and higher all the time. Right now it's up to position 6, meaning almost my entire site is in the supplemental index.

The page in question isn't even an RSS feed (it is an XML file, though). It's a little experiment with Google Homepage widgets to allow searches from various Google datacenters. And the page doesn't actually exist, I deleted it several weeks ago.

So here's how I plan to fix it. I'm going to 301 redirect the bastard to my homepage. Then link to it from this post so hopefully Google will crawl it and find that redirect. So here's the link, now I wait for Google to purge the page.

Like this article? Get lots more by subscribing to my feed.