Welcome to WebmasterWorld Guest from 54.160.177.33

Forum Moderators: bakedjake

Message Too Old, No Replies

New Full Spidering Search Engine - Giga Blast

GigaBlast.com

     
12:38 am on Mar 16, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38048
votes: 12


I'd heard about this one awhile back and it appears they are up and running:

[gigablast.com...]

Real time spidering on the AddUrl page.

[gigablast.com...]

Feels like: Infoseek
Tastes like: Peppermint (minty fresh ;)
Looks like: squeaky clean.
Relavance like: who cares. it's a new se with instant addurl - done.

7:18 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 16, 2002
posts:65
votes: 0


it will spider dynamic urls but i currently have that option disabled. i will probably enable it shortly.
matt
7:25 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2001
posts:1238
votes: 0


I am seeing some good returns on various searches. In fact, I am suprised at the depth I see. This one will be interesting to watch... I'm gonna adopt it as a pet and see if it's housebroken! ;)

Ove

7:27 pm on Mar 16, 2002 (gmt 0)

Senior Member from SE 

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 24, 2001
posts:786
votes: 0


Thanks for the info Matt
this will be interessted to follow.

/Ove

7:28 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2001
posts:1238
votes: 0


Hey Matt! How about "Here Giggy, Giggy!" Wait a second... Giggy? Yeah!
7:28 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


More power to you Matt. I've been testing your engine for a few weeks now and I must say I am impressed. In many ways I found it to be more relevent than Google (before the attention you recently started receiving). I know you'll get all that fixed in due time so it doesn't change my opinion any.

I have a few questions:

1. Have you figured out a user agent for your spider and if so, what will it be?

2. Robots.txt- will your spider obey it?

3. (Just for my amusement) What is your policy on cloaking?

Key_Master

7:29 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Apr 3, 2001
posts:1609
votes: 0


Hi Matt!

Sorry for jumping to conclusions!

Unfortunately on the web..I've used the "if it looks like a rat and smells like a rat" acid test for some time.

Obviously I was mistaken, please accept my apology.

I wish you the very best!

holographic

7:32 pm on Mar 16, 2002 (gmt 0)

Inactive Member
Account Expired

 
 


It would be nice to see a SE publish exactly where they stand with what they do and don't accept.
7:33 pm on Mar 16, 2002 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38048
votes: 12


Thanks Matt, good luck with it!
7:34 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 31, 2001
posts:1238
votes: 0


Seriously Matt, it appears you are off to a nice start. The traffic will come...

Best of luck! And please, keep us informed - we love this stuff!

7:37 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 16, 2002
posts:65
votes: 0


1. Have you figured out a user agent for your spider and if so, what will it be?

it will be gigabot.

2. Robots.txt- will your spider obey it?
yes, except top level pages.
it also obeys the meta disallow tags, too.

3. (Just for my amusement) What is your policy on cloaking?
if it's used to abuse the search engine and mislead searchers, then my policy is to ban that site. I index the meta keyword/description tags so use those instead.

7:41 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 7, 2001
posts:661
votes: 0


I hope everyone realizes that the "last 5 queries" feature is exposing everyone's top keyword targets!

Be careful searching unless you want everyone at WmW to know what your pet keywords are!

7:42 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2001
posts:1472
votes: 0


>>>it also obeys the meta disallow tags, too.

Is this true for top level pages also?

7:46 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 16, 2002
posts:65
votes: 0


>>>it also obeys the meta disallow tags, too.
>>Is this true for top level pages also?

no, i always index top level pages regardless of robots.txt or meta disallow tags.

7:51 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 16, 2002
posts:65
votes: 0


NOTE: i've just added the option to set the number of summary lines in your search results on my advanced search page.

do you guys think this would be useful? or just confusing to users?

matt

8:36 pm on Mar 16, 2002 (gmt 0)

Full Member

10+ Year Member

joined:Feb 24, 2002
posts:225
votes: 0


Wow!

I added one of my urls and 3-4 seconds later it was indexed and ranked number one under targeted kw's! :-) One of my other sites were already indexed.

This is going to be a good one...

Alby

8:46 pm on Mar 16, 2002 (gmt 0)

Full Member

10+ Year Member

joined:June 8, 2001
posts:309
votes: 0


has anyone tried refreshing the page
indexes 200 to 350 pages per refresh. gigabot spider is busy.
9:04 pm on Mar 16, 2002 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 30, 2002
posts:4842
votes: 1


The last 5 queries thing has turned into a forum-like tool

Such is the diversity of the web :)

9:05 pm on Mar 16, 2002 (gmt 0)

Senior Member

joined:Sept 1, 2000
posts:1120
votes: 0


Welcome to Webmaster World Matt.

Thanks for the thrill. I agree, it reminds me of the old Infoseek rush. Those were the days.

If you wanted to stir up a bunch of web'sters, you've certainly done that. Good luck on your venture.

9:08 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member jeremy_goodrich is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 4, 2000
posts:3468
votes: 0


nicely done...good luck! (off to try and remember how to auto submit he he he)
9:10 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 7, 2001
posts:69
votes: 0


Nice, ... Matt will you make more little size your logo, its hard for me in Indonesia
9:26 pm on Mar 16, 2002 (gmt 0)

Senior Member

joined:Nov 20, 2000
posts:1336
votes: 0


Welcome Matt. A very nice system you have there as well. It deserves success.
9:26 pm on Mar 16, 2002 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 30, 2002
posts:4842
votes: 1


Was playing around with the last 5 queries thing on the home page

I searched for

place link <here> for googlebot

and it sorta went a bit off track. Check it out

[gigablast.com...]

9:28 pm on Mar 16, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 7, 2001
posts:69
votes: 0


I was test by search "java furniture", but i think java furniture is nothing relevant with sun java.
9:31 pm on Mar 16, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 10, 2001
posts:748
votes: 0


Ok, I'm lost. I admit it. A lot of unanswered questions.

First, littleman's UA and ip - I can't find it anywhere in the last week.

Compare this to the link in Brett's first post. Who is 'Guest from 208.254.87.133'? if not the spider and if that's really a cached page.

I can't tell if these pages are cached or not. I submitted a site about 8:00 am this morning, and the cached page shows an update 2 1/2 hours later. No sign of any spider in my logs, AT ALL.

I submitted 5 sites, they're all in, all saying today's date as having been spidered, but NO evidence that anything actually came around? The only spider that's been around lately is whizbang labs.

I'm glad the sites are in, and the SE looks promising. If I look at the data I have, it looks as if the sites were pre-spidered, and the cache (mirror) is showing the current page. I guess I could change a page and look at the cached copy to verify this. Has anybody seen evidence of a crawler? Or or my logs wrong?



continued [webmasterworld.com...]
This 84 message thread spans 3 pages: 84