Can anyone imagine/knows what Googlebot's Interface is like?

Just wondering

[sidyadav]

Googlebot was updating my site every 2 days, so I started re-searching on it, I found out its made in c++ by Stanford when I came to the thought what Googlebot's Interface is actually like. Is it a GUI (Graphic User Interface, similar to Grub),Is it a Pure DOS based Interface which has no images or anything it just shows what site/s it is indexing etc. or is it a Web based Interface, which you can control by the web. Anyone knows what it is like or any thougts?

Sid

[dmorison]

Hi Sid,

This is pure speculation as it would be from anybody not closely involved with Google and the "googlebot".

"googlebot" is probably a collection of programs, distributed across many machines, all working together under some kind of command and control mechanism - probably file based; so one process will write commands to a file which are picked up by another processes and acted upon.

The individual programs are probably primarily written in C++, although knowing Google there will be plenty of Internet Duct Tape (PERL) in the mix; particularly if the command and control mechanism is primarily file based.

The actual program that makes a HTTP request to your server, with the user-agent "googlebot" is probably just a small, relatively simple part of the whole process; reading commands from a command file as to what to retrieve next; and sending the recovered content to another process for interpretation and handling.

So as far as a GUI is concerned; there is probably not one single GUI as such; instead, each of the processes that make up "googlebot" will probably have various configuration files, and there may be GUI's to manage these configuration files.

Basically; I would view googlebot as a "system" rather than a single program; and its GUI therefore would look more like the control room of a power station than a single program running on a desktop computer.

Hope this helps!

[sidyadav]

Thanks for the information, before all I knew was Googlebot is a program ,now I know a lot more.

But one more question ,are all other search engine robots based like this? for example as Matt Wells explains , there are only 8 computers the "Gigabot" is running on and he looks to it like a "Program" running on a computer.I mean Scooter, FAST web crawler , Slurp etc. AFAIK they are Big companies with more than a 100 staff members so they're probably a bit like Googlebot, Just a thought.

Sid

[volatilegx]

I would imagine that the spidering/indexing processes for the various search engines are as different from each other as their search engine results pages are.

I'd love to get some input on this from Googleguy or one of the other SE representatives that lurk here.