search engine spider software

index content of webpages in database

[seo4life]

hi guys,

i am searching a SE spider for spidering pages and dissociating html tags from the real content (head & body). as result i should get a database full of this real content. which software do you recommend for this? nedd not to be free stuff.

[takagi]

You might want to read:

Google like search engine .... / Making a Google like search engine .... [webmasterworld.com]

Running a search engine (and spider too) [webmasterworld.com]

why are spiders bad? [webmasterworld.com]