Forum Moderators: phranque
Can someone please help me find the right words to describe what I mean?
I'm playing around with spider/search engine scripts hoping to build a small vertical.
I'm assuming that search engine algorithms are able to "ignore" certain page elements for example my navigation and footer and not include these elements in SERPs.
I am looking for some words/search terms to describe this process.
I imagine that this process has psuedo-code like:
- crawl all pages
- identify repititous blocks/elements common to most/all pages
- exclude repititous blocks/elements from index
Is there a "scientific" term that describes this process that I can use to find out more?
Thanks
I hope I have posted in the right forum.