Our algorithm employs rules of Arabic spelling and grammar along with signals from historical search data to decide when to leave out spaces between words or when to remove unnecessarily repeated letters. Now, when you type a query leaving out spaces or repeating a letter, we’ll return better results based not only on what you typed, but also on what our algorithm understands is the “correct” query.

There’s a few nice things about this post besides the direct improvement on Arabic language searches. For one, this post joins other recent posts that pull back a little bit of the curtain on the 400+ ranking changes that we make every year. I hope that we keep doing these posts.

Another nice thing is that the post talks about the impact of the improvement (10% of Arabic language queries are affected by this change). For the recent blog post about how Google uses synonyms in ranking, Steven Baker mentioned that “synonyms affect 70 percent of user searches across the more than 100 languages Google supports.” I like giving a rough idea of a change’s impact. The vast majority of Google’s 400+ annual ranking changes affect a much smaller percentage of queries, so don’t get the wrong idea that every improvement to our ranking algorithms affects a large percentage of searches. One last nice thing is that this change again shows the value of historical search data to improve search quality. I know that few users care about that, but it’s good to point out.

Anyway, I like to point out when Google blogs about these internal changes to our scoring algorithms, because people always want to know more about how Google works.

What’s even more interesting is that BusinessWeek put up transcripts of some of the interviews. You can read interviews with:

• Eric Schmidt, Google’s CEO
• Udi Manber, vice-president of engineering and head of the search quality group
• Amit Singhal, head of Google’s core ranking team in the search quality group
• Scott Huffman, head of the group that evaluates quality in the search quality group
• me (Matt Cutts). I’m the head of the webspam team in the search quality group

Org-chart-wise, it looks like this:

Eric Schmidt would be at the top of the cloud, Udi would be the “Search Quality” box, I’d be in the webspam box, and Amit and Scott lead teams within the “Other groups” part.

The two interviews I liked the most were Amit’s and Scott’s. Amit sums up Google’s philosophy toward real-time, he discusses our pragmatic (yet algorithmic) approach to search, and our attitude toward our users:

Q: I think the criticism is: Where’s the money in those [non-search/ads parts of Google]?

A: The right way to look at it is not the money. Is there value to the users? If you bring value to the users, I think we will succeed in the long run. Some things make more money than others, but as long as we keep bringing value to the world, we will be successful.

I liked Scott’s interview because he goes into more detail of how we evaluate search quality than I’ve seen in the past. Evaluating search quality is really hard to get right. I also liked this quote:

But the other thing we always do is we go in and look in more detail at what are some of the individual positive and negative things that we’re getting out of this. Are the positive things really that positive, will they really make a difference to our users? And maybe more important, for the negative things, how important are they, can we live with them?

At the entrance to Google’s main cafe, there’s three doors. Two are normal doors that you pull to open, and they always work. The other door is a spiffy automatic door that slides open for you–except that the automatic door seems to be broken about 5-10% of the time. When the automatic door works, it’s very cool and you’d definitely prefer to use it. But when the door is broken, you’re left standing in front of a glass door and you feel like a dork as you wave your hands, move around, and generally try to get the “automatic” door to open for you. I’ve noticed that many people stopped using the sometimes-broken automatic door and instead always go straight to the reliable doors.

Search can be kind of like that door in a lot of ways. Spiffy features are great, but if they’re wrong or don’t trigger in some reasonable way that your mind can predict, the failure is worse somehow. The same holds true with the organic search results: a catastrophic search failure can stick in your mind much more than the 200 searches that worked well. Search quality evaluation is tricky because you need to take that factor plus hundreds more into account. It’s taken years for Google to really evaluate our quality well, and we still continue to learn important new things.

If you really want to understand more about how Google thinks, I highly recommend Amit’s and Scott’s interviews. They’re a great reminder to me that we have a very deep bench of smart, well-spoken people in the search quality group and in Google in general. I would love to see more Googlers talking about their work.

And finally, on the subject of Googlers talking about their work, a whole bunch of Googlers will be at the Search Marketing Expo East in New York this week. Joachim Kupke will talk about duplicate content, Ari Bezman will talk about maps, Jack Menzel will talk about what’s next in search and universal search, Jeremy Hylton will talk about real-time search, Maile Ohye will talk about best practices for search, Matthew Liu will talk about YouTube, and Frederick Vallaeys will answer questions about AdWords.

Also, don’t miss Bruce Johnson and Kathrin Probst from Google. They’ll be on the “CSS, AJAX, Web 2.0 & SEO” panel. If you’re at SMX East, I think you’ll enjoy that panel.

In an election year, everybody gets a little more sensitive about politics, so I wanted a write a pre-emptive post in case anyone accuses Google of political bias in our search results sometime this year.

This is my personal opinion, but in my way of looking at the world, search quality > politics. That is, preserving the quality and accuracy of our search results is the best way we can help our users, while skewing our search algorithms to espouse a particular political party’s viewpoint would be anathema. This month I finish my eighth year at Google and begin my ninth (geez, I’m old), and in that entire time I can’t remember even the tiniest suggestion to bias Google’s search results toward any political party. The trust of our users is important, and in my opinion it would be an abuse of that trust to skew our search results toward any particular political view. I suspect that if you checked with old-timers at other search engines, they’d say similar things.

Here are some questions and answers:

Q: How do I check out the Caffeine update?
A: If you search on http://www2.sandbox.google.com you can get a preview of how the search results will change over the next few weeks and months.

Q: It doesn’t look any different to me?
A: The Caffeine update isn’t about making some UI changes here or there. Currently, even power users won’t notice much of a difference at all. This update is primarily under the hood: we’re rewriting the foundation of some of our infrastructure. But some of the search results do change, so we wanted to open up a preview so that power searchers and web developers could give us feedback.

Q: Is this Caffeine Update because of Company X or Y is doing Z?
A: Nope. I love competition in search and want lots of it, but this change has been in the works for months. I think the best way for Google to do well in search is to continue what we’ve done for the last decade or so: focus relentlessly on pushing our search quality forward. Nobody cares more about search than Google, and I don’t think we’ll ever stop trying to improve.

Q: The url http://www2.sandbox.google.com doesn’t seem to work for mobile phones? I can only test on google.com, not google.co.uk?
A: That’s right. For now this is a only a preview, so we didn’t hook up a mobile version or an international version at this point. You’ll have to search on google.com to see the results right now.

Q: How do I give Google feedback?
A: If you want to give us feedback on how the search results are different, look on the search results page for a link at the bottom of the page that says “Dissatisfied? Help us improve.” Click on that link and type your feedback in the text box. Make sure to include the word caffeine somewhere in the feedback.

Q: Is there a way to give feedback in person?
A: Yes! If you want to give me feedback in person, I’ll be at Search Engine Strategies San Jose this week. I’m doing a site review panel on Thursday, or just walk up and say hello!

You can also read more about this change on Techmeme if you’re interested.