Welcome to WebmasterWorld Guest from 23.20.18.183

Forum Moderators: lawman

Message Too Old, No Replies

Alpha Build - New Search Engine!

     
7:43 pm on Apr 11, 2010 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38061
votes: 13


You are the first to see our new site search engine. This represents a couple months work and we are very proud of it:

[webmasterworld.com...]

Notes:
- we are not sold on the serp display yet, but we have a couple other features we want to add (thread mode vs message mode), so we are still working with templates.

Known issues:
- about 15minutes delayed on indexing new content

Anyway - we like it thus far and it is providing great results.

Thanks!

...Feedback, comments, wishes, and praise ;-) welcome and encouraged.
5:31 pm on Apr 13, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member trillianjedi is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 15, 2003
posts:7242
votes: 0


Brett - great job. I'm impressed with the quality of results. That's the important thing. It's pretty fast too.

The cosmetics is the easy bit to fix up. I see lots of feedback in here already which is probably all you need to give it some tweaks. I'd throw it out to a wider audience before doing too much on that front though.
6:21 pm on Apr 13, 2010 (gmt 0)

Moderator from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7564
votes: 4


I think if we look beyond the layout what we have is a great site search. Good results and very fast. The layout issues are potentially the simplest thing to alter, and the feedback on here will certainly help to refine that factor.

Mack.
9:59 am on Apr 14, 2010 (gmt 0)

Preferred Member

5+ Year Member

joined:Mar 29, 2007
posts:592
votes: 0


Its amazing....
4:23 pm on Apr 14, 2010 (gmt 0)

Administrator

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month

joined:Jan 14, 2004
posts:859
votes: 3


This is going to make my life a bit easier when doing research for User-Agents Histories and the like.

Good job.
8:40 am on Apr 15, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:6964
votes: 385


I do like the advanced search very much. I'm also curious if a basic search at the top of every forum is also possible (with sub link to advanced to the side/below). This would make the search function more immediate and intimate with WW.
3:39 pm on Apr 15, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 9, 2005
posts:1094
votes: 2


Great tool, thanks!
One thing I'd change is put the thread date closer to its title. Something like:
Alpha Build - New Search Engine! - Apr 11, 2010
Would be just perfect.
10:54 am on Apr 16, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 9, 2004
posts:1435
votes: 0


Well done Brett & team.

Getting such a search system to work well is not a trivial matter.

There are a few things that I would like to see change over time:

Like most, the presentation of the results needs attention but that's just some CSS really.

Addition of some further attributes such as total number of posts in the thread. These along with the presentation options should probably be user selectable from an options page - choose once and then always see the results as you wish. Other attributes could include, thread start date & last post date (however that probably involves keeping a separate index for the last post date and total post numbers as adding a post to a thread would then change attributes on many posts, so you would want to do a small second pass on those data items - which would also mean that some of the data would not be easy to incorporate into the ranking algo options - although being able to rank by post count in a thread may be useful occasionally).

Have a "member match?" option on the form so that you can choose between posts or member profiles that you are searching - or don't have the option and separate the 2 distinct result sets in some way (columns or stacked with internal anchor link to lower set)

It's a great start though.
12:02 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:May 6, 2008
posts:2011
votes: 0


Any chance you'll share how the search used to work and how it works now? I'm always up for learning how to optimize queries.
12:13 pm on Apr 19, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:May 8, 2002
posts:440
votes: 0


Brett, echoing StoutFiles, you often said this was impossible given the nature of how the site is built (static files etc) - care to let us in on the technology you are using in the end to acheive this?
12:25 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 21, 2005
posts:2259
votes: 0


Great job, I love it. Now I won'thave to go to Google to find a WW thread! :)

Would be good if you could format the output more like the SEs.
1:21 pm on Apr 19, 2010 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38061
votes: 13


StoutFIles - core db is modified Sphinx. There is another meg of our Perl code for a spider and ranking algo. Sphinx is awesome in that it doesn't require an sql db. We just feed it raw xml. However, there is a major ton of tweaking to get the level of relevance that we want.

> format the output more like the SEs.

Agreed - however, there are significant differences in the way we have built this SE vs the way big G/Y/Bing work. We are indexing down to the message level and yet, we need to return SERPS at the page/thread level. That leaves a wide array of interpretations on how the display should look. We also want to be able to toggle between two modes (thread/page mode vs message mode). Lastly, we are going to throw in a 3rd option and just allow people to open the message right on the SERP via ajax.

So, we are going to monkey with it a bit more this week.
1:50 pm on Apr 19, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member themadscientist is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 14, 2008
posts:2910
votes: 62


That leaves a wide array of interpretations on how the display should look.

I was just going to post about the color and layout being all wrong...
It's not green and blue with lots of ads. What were you thinking? LOL

Actually, I think you all did a great job with it.
It had to be a huge (but hopefully fun) challenge. :)
2:21 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 8, 2003
posts:1141
votes: 0


Wow - and it only took you how many years?

If webmasterworld can implement a site search - who nows what happens next. Maybe "Duke Nukem Forever" will be released after all.
2:38 pm on Apr 19, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 14, 2005
posts:475
votes: 0


Superb!
3:04 pm on Apr 19, 2010 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14650
votes: 94


Works real nice.

Being able to search the supporters area is the best part!


I'd like to see date options to search for older documents and exclude newer ones such as

Posts > 6 Months
Posts > 1 Year
Posts > 2 Years

That way I could find older posts with sorting thru the current stuff, or as someone else suggested, allow sorting results by date.
3:35 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


I like it. The css for "member match" results could add an underline under those words to make it easier to see but the engine itself works great!
3:44 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 20, 2002
posts:1716
votes: 0


I really dig the member match - when I got my results and saw the member match, I thought, "Oh, cool, THAT's the guy to read."
5:45 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Dec 13, 2009
posts:945
votes: 0


There is one thing I dislike about this:

I can't search just by username (i.e. no other search criteria), and I feel that's an unecessary limitation. Other than that: awesome job guys :)
8:40 pm on Apr 19, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:6964
votes: 385


I'd like a member search tied to a date range. Oft times I'll remember who said something and about when, but not exactly what they said. :)
10:23 pm on Apr 19, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 4, 2002
posts:1793
votes: 2


I'm glad to see this Brett as it was annoying to have to go search elsewhere.

I'd like to see some color separations, maybe for title, like on the other pages.
5:47 am on Apr 21, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 26, 2002
posts:812
votes: 1


Wow, something we didn't dream will be available is here! Great job.
7:28 am on Apr 21, 2010 (gmt 0)

Full Member

5+ Year Member

joined:Jan 4, 2007
posts:221
votes: 0


Very similar spacing between "paras" of a single result and between one result and another (for me anyway (at work - Chrome 3, XP)). I'd prefer that the different parts of a result were closed up a bit to make this clearer.

Put together that which belongs together.
10:00 pm on Apr 26, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 18, 2003
posts:629
votes: 0


Searching for "Search Engine" gives pretty bad results.
12:10 am on May 1, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member themadscientist is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 14, 2008
posts:2910
votes: 62


Thanks a bunch for NOT eliminating 'stop words' from the searches... I just tried a search for information about a specific tld none of the other search engines I've tried with will display well in their results, but it was returned and highlighted here. YES!

None of the other search engines I've tried the search on even do very well with 'tld ".in" domain' as the search because they totally discount the '.' and the 'in' from the query, even if it's in quotes, so it's been difficult to find info about some tlds before.

Anyway, good thinkin' and THANKS!
5:06 pm on May 2, 2010 (gmt 0)

Junior Member

5+ Year Member

joined:Dec 23, 2008
posts:145
votes: 0


"Membernames are case insensitive"

Doing a [check] on "Jonesy" == Member name not found

Doing a [check] on "jonesy" == Member name exists

Jonesy
1:16 am on May 3, 2010 (gmt 0)

Preferred Member

5+ Year Member

joined:May 29, 2007
posts: 578
votes: 0


...Feedback, comments, wishes, and praise ;-) welcome and encouraged.
One wish:
Pass the keyword string (e.g. "Blue Widgets") into the resultant URI of the SERP

Doing so would make the new site search engine a useful tool when replying to posts that ask questions that have been comprehensively answered before

At the moment, a search for "Blue Widgets" generates a page with a (meaningless) URI of
http://www.webmasterworld.com/search.cgi


CF Google:
http://www.google.com/search?&q=site:www.webmasterworld.com+%22blue+widgets%22
12:31 am on May 6, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member penders is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2006
posts: 3123
votes: 0


bill: I don't know if it's just me, but the images in the search form take forever to load. It's been like this since I was privy to the earlier release. Today it took 20 seconds for them all to load from my office. On my home fiber connection it takes close to 10 seconds.


About 3 seconds for me - but the images are certainly the slow bit. Do we need images here at all?

In the SERPs, sometimes there is no text snippet for some results and often the phrase I'm searching for does not appear in that text snippet (because it appears much futher down the post). I think I would prefer to see the text surrounding the search phrase in the SERPs.

>> Time range...
May be (to fit in with your current method), have an "excluding" drop down next to the current "Time Range" drop down. Both drop downs are essentially the same. So, you could pick "Time Range 'Last 5 Years' excluding the 'Last 2 Years'" ?

Great stuff - well done!
3:46 am on May 18, 2010 (gmt 0)

New User

5+ Year Member

joined:Apr 20, 2010
posts:2
votes: 0


Convenient,I like it very much.
10:45 pm on June 8, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 8, 2004
posts:563
votes: 1


[webmasterworld.com...] image is not showing up.
10:54 pm on June 8, 2010 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 8, 2004
posts:563
votes: 1


I did a check for under "User Name" for my user name.
I searched for my name with a capital "P" like it shows up on the screen display to the left.
<<<<<<<<<<<<<<<<<<<<<<<<<<<< To the left and up.

Well, it came back that there's no user by that name. I then tried the lower case "p" and I found myself again.
Whoo, I had an identity crisis there for a second.
This 61 message thread spans 3 pages: 61