

News Archive
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
New "Bingbot" Will Crawl Non-optimized Sites More Easily
June 28, 2010, 1:05 pmMicrosoft has announced that it will be bringing the Bing web crawler out of beta on October 1st. It will be rebranded as "the Bingbot" and replace the existing msnbot. "It will still honor robots.txt directives written for msnbot, so no change is required to robots.txt file(s)," a Bing representative tells WebProNews.
"Improvements to the bot enable more efficient crawling, and increase the ability to crawl content on sites not optimized for search," he says.
Rick DeJarnette has more about the change on the Bing Webmaster Blog:
Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)
The HTTP header From field will also change as shown below:
From: msnbot(at)microsoft.com
will become
From: bingbot(at)microsoft.com
If Bing finds separate sets of directives for Bingbot and for other crawlers, directives for bingbot will take precedence, the company says.
I find the part about increasing the ability to crawl content on sites not optimized for search to be particularly interesting. I wouldn't exactly call this an invitation to ignore SEO. Obviously Google is still the biggest search engine anyway, but even as far as Bing is concerned, good SEO practices will likely still help your rankings.
Also keep in mind that optimizing for Bing is becoming increasingly important. Not only is Facebook giving more reason for people to search (where Bing provides the web results), but the Yahoo/Bing integration will be here (likely) before the holidays.




