Welcome to WebmasterWorld Guest from 54.80.185.137
Forum Moderators: mademetop
Search engines consist of five discrete software components:
Spider:
A spider is a robotic program that downloads webpages. It works just as your browser does when you connect to a web site and download a page. The spider just doesn't have any visual components. You can see the same thing by viewing any webpage, and then selecting "view source" in your browser.
Crawler:
As a spider downloads pages, it can strip apart the page and look for "links". It is the crawlers job to then decide where the spider should go to next based on the links, or based upon a preprogrammed list of urls.
Indexer:
An indexer rips apart a page into it's various components and analyze them. Entities such as, titles, headings, links, text, constructs, bold, italic, and other style portions of a page are ripped apart and analyzed.
Database:
The database is the storage medium for all the data a search engine downloads and analyzes. This can require huge amounts of storage space.
Search Engine Results Engine:
Ah, the heart of the beast. It is the results engine's job to decide what pages matches a users search. This is the portion of a search engine you interact with when you perform a search. It is also the one part we are concerned with here.
When a user types in a keyword and does a search, the search engine decides what to match for results under varying criteria. The means with which it decides is called an algorithm. You may hear search engine optimization (SEO) professionals discuss "algos" from time to time and this is what they are referring too.
Although search engines have changed a great deal, most still match results to searches similar to the following:
As you can see, a search engine will have to make many judgement calls based upon the entire page downloaded.
That's the abbreviated version of how a search engine works.
Thank you very much!
i appreciate it a lot.
For not native english speaking countries this is an excellent terms explication.
Most of the technical definitions about the language of computers, internet etc. are so new that you can't find them in a dictionary.
Angiolo
It's funny you had posted this information. I was thinking yesterday where I could find dictionary like type definitions about se's and seo.
Do you have anymore up your sleeve? I would love this type of format on information such as doorway pages, gateway pages, redirects, dynamic pages, subdomain pages and the sort.
If you have anymore oomph to give explanations such as the ones you have before I would greatly appreciate it. If not do you know of such a resource that may list these in dictionary format?
Crazy question but its so much easier refer to.
Thanks again for your post!
Brett - This term is asked about often enough that it should probably be put in the glossary... one more thing to do. I don't know how you manage it all.
I'm amazed at the quality and scope of this site, and I appreciate all the information you've shared with us, including your post above. Thanks.
You guys like stuff like the above? I didn't know if that would fly here or not. We are pretty top heavy with experienced members and I thought the above would be old hat to most. I guess it is time to get serious about a "new to se's" forum.
Maybe also link to this information (your first post above) right off the home page? Anyways, this sort of thing can be helpful for those of us who get questions like this all the time ;)