Building a search engine is a very complex task. I often find myself trying to justify to people why it is that search engines can’t understand their site. They seem fixated on believing that a search engine should understand it if a human can understand it. The short answer is that with an infinite amount of time the search engine could, but the scale of the Internet makes it oh so VERY hard.
The infographic below tries to give you some sense of the scale of the problem. Please note that a few numbers are hard to truly pin down, but I pulled them from the best sources I could. For example, no one really knows how many pages there are on the web, though Majestic SEO is aware of 3.7 trillion (the number I used) or the average web page size.
Regardless, the message is the same either way. The web is a really complex place!
See the Search Complexity Infographic at Full size
