Webmasters in online forums reported an unusual Google behavior. New web pages of some sites are indexed with a longer delay. Is it a new Google penalty? Or is it just a bug that affects only certain websites?

What is the exact problem?

The websites that reported the problem were used to immediate inclusion of their pages in Google’s search results.

If your website has a certain number of inbound links and if it has been indexed by Google, Google usually indexes new pages very quickly. That does not mean that the new pages get high rankings for any keyword but that the new pages can be found in Google’s results if you search for the exact web page title or other unique elements that appear on the new web page.

These websites that were used to immediate inclusion now observed a 7 day delay for new website content:

“My blog has been afloat for almost 1.5 years, was always indexed by Google fast and smooth, and was ranked pretty high by main keywords in its niche. However, it’s been several weeks that Google seems to have stopped indexing my new posts. [...]

It seems as if [Google is] applying an algorithm like ‘posting day + 7 days’ or something when indexing my new posts.”

Is this really a new Google penalty? What can you do against it?

Many of the web pages that have the 7 day indexing problem are Wordpress blogs. There are several theories why these blogs aren’t indexed as quickly as before:

1. The affected websites all seem to use the same SEO plugin for Wordpress that screws up the code and prevents Google from indexing the website pages correctly.

   If you use an SEO plugin, consider if you really need it. These plugins don’t do real SEO. They hide certain pages on your website from search engine spiders and they allow you to edit the header information of your web pages. This can also be done through the regular Wordpress interface.

2. Although the new blog post wasn’t listed for 7 days on Google, the index page of the blog was listed in Google’s search results.

   As the index page of a blog often shows the latest blog posts, it is likely that Google uses the 7 day delay for the blog page because the post can only be found on the blog index page during that time.

3. Some of the affected blogs publish press releases on their websites. As the same press release can also be found on many other websites, Google might return the first publisher of the press release in the search results and discard all other pages with the same content.
4. Other affected websites seem to be AdSense scraper websites. These websites have very little unique content and they have been built to attract visitors that then click on the AdSense ads on the website.
5. Some of the affected websites installed user agent sniffers on their server. These programs return different pages to different user agents (human web surfers, Googlebot and other bots).

   Using such a script on your server can easily prevent Google’s indexing robot from indexing your website and it can be misinterpreted as a spamming (cloaking) attempt. This is very similar to the SEO plugin problem above.

It looks as if the 7 day delay might be related to two different website types: websites that use server scripts that return different pages to different user agents and websites that contain little or no unique content.

The 7 day delay for blogs that display the content both on the blog index page and the blog posting page is not a penalty but a Google feature that prevents duplicate results.

If your website is a Made-For-AdSense scraper website then it’s probably time to think about a new business model. Google might give these websites lower rankings in the future.

If you use bot blocking scripts or other server scripts that change the content that is delivered to visitors of your website then you should make sure that you’re not accidentally blocking Google. It’s better not to use these scripts at all.

Tweet This Post

Google founders originally believed search should be non-profit

“All the recent debate about the Google index and newspapers, and Google’s interest (and problems) in indexing out-of-print books, would all be a much different discussion if the Google index were run as a non-profit. And it is rather ironic to note that Larry Page and Sergey Brin, the founders of Google, strongly believed that search engines should not be commercial enterprises. [...] The problem with the Google search engine at the time, Lent recalls, is that Larry and Sergey didn’t want to commercialize it, and Lent was anxious to become an entrepreneur. Their mantra at the time was more socialistic than entrepreneurial. Originally, ‘Don’t be evil’ was ‘Don’t go commercial’.”

Google founders originally believed search should be non-profit

Tweet This Post

“While Google remains the dominant player among small business advertisers, Yahoo! and Bing (formerly MSN) have gained significant market share since the third quarter of 2008, as much as 5 percent. [...]

Additionally, the report shows click-through rates (CTR) improved year over-year across all the engines, with the biggest improvement on Yahoo!, where CTR improved 123 percent from Q3 2008 to Q3 2009.”

Related: Microsoft’s Bing gets ’stickier,’ grabs more local dollars.

Tweet This Post

“Google usually shows a green web address, or URL, at the bottom of each search result to let you know where you’re headed. Today we’re rolling out an improvement that replaces the URL in some search results with a hierarchy showing the precise location of the page on the website. The new display provides valuable context and new navigation options. The changes are rolling out now and should be available globally in the next few days.”

http://googleblog.blogspot.com/2009/11/new-site-hierarchies-display-in-search.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+blogspot/MKuf+(Official+Google+Blog)

Tweet This Post

In July, Google upgraded AdWords to allow advertisers to add the full address to their search ads through location extensions. It appears that they are now testing a new ad layout similar to the one developed for Local Listing Ads.

http://blumenthals.com/blog/2009/11/20/is-google-adwords-testing-a-new-local-adwords-display-type/

Tweet This Post