Forum Moderators: open
I've got a brand new site appearing at MSN, AOL and HotBot, but with the URL of the site that previously had the IP number.
It's appearing with phrases emphasized on other site pages, but it's only the index page that's being listed - for all of them.
I'm considering suggesting that a paid submission be made - depending on the length of update time.
Using the free submission is the same as asking INK to perform last rights on your site (unless the search terms are very unique - like a company name)
While the jury is still out on the INK PAY TO PLAY PLAN, I would only pay for pages with very competitive search terms, those that are not competitive (like company names will naturally rise to the top)
Also, we have found that a lot of external links to a page will get it spidered by INK - and in some cases into the DB. Without those external links it looks like a free submit will cycle out of the DB in 3 to 6 weeks.
How I found this was by checking the stats. Not only had it been spidered, but I found searches had been done on an innumerable number of combinations of the words. I specifically remember making submissions the Thursday after Thanksgiving.
This page, with the exact title and description used, has three #1 and 6 top five MSN listings on search terms (not the url), plus combos I hadn't even thought of - plus similar Hotbot listings (with a date of Dec. 10), although the traffic was from MSN.
What I noticed when I checked is that while my page was listed, title description and all, the url was that of the realty company. The web host made the correction this weekend, so it's now pulling up 404s - goodbye traffic. I submitted the URL of the realty company for removal and re-submitted the correct URL.
I wouldn't think of changing a thing, because it's getting similar results with AV, but yes it would be worth $20 if it'll take forever for the change to be reflected at MSN.
That's why I'd like to know how often updates are done.
If it's by domain name and not IP, I have a feeling they "visited" because of the prior site, and I will soon be history.
Sorry I can't be more help
Steve
BTW - are your KW's for this page competitive???
ONLY that page. There are links to every other page on the site, plus outbound links, including one never indexed. None were listed - just that one page.
>are your KW's for this page competitive?
For home made kitchen gifts, MSN is 1 out of 63,1216 HotBot is 1 out of 177,400 (both totals changed since last week), and #4 out of 4,198,110 at AltaVista.
I don't consider them particularly competitive, but they're the kind of items that would have specific searches done.
What I find interesting is that only the pages actually submitted appear - none of the linked pages within the site.
Just as well - changes need to be made.
Also, in spite of the links page being indexed this week, the index page (with the incorrect domain name - and a date at HotBot of Dec. 10) remains unchanged in spite of the update.
>would the client receive any value by having their home page indexed?
I believe this site is so seasonally oriented that it's probably worth paying for one seasonal index page in a subdirectory and making the appropriate timely changes to it.
Thanks for the timetable, etc. I've got the info I need now (and links are being taken care of).
The Inktomi j6000 spider only dos it's thing on the page submitted. It doesn't behave the same as googlebot and follow all the links. It appears that the spider comes around for a visit after about 24-48 hours after submission. I used dumptruck to submit, so not sure which engine actually picks up the submitted page for Inktomi.
He answered by describing three Ink databases, which are combined to create the Inktomi results at any one time.
1. One is a temporary index of recent free submissions, where pages usually hang around for a few weeks at most and then are dropped.
2. A second is the general index, which some people call the permanent index
3. The third is the paid inclusion index.
Jim said:
...if your URL is in both the general web index and the inclusion index you will only see the URL as it is in the general index.
It sounds like you are seeing this phenomenon, right? Also, my experience so far bears this out.
The only pages I plan to pay for, going forward, are those which keep falling out of the index (type 1) or new pages that I create explicitly for Inktomi.
as i recall, there were (pre-paid inclusion) 3 layers of ink databases. local, main, and gen3...
local is a db stored on the client server, where popular search results are cached. my guess is that clients (like iWon, whose top X results stay static for weeks at a time) have a reduced fee option that lets them keep a cached version locally. in effect, they "buy" a portion of the database. the reduced fee thing would come from not having to pay ink on a per-search basis. instead the site just pays ink a flat fee for a part of the database, and then only pays extra when searchers go to the 2nd+ page of results (which pulls from the main and/or gen3 dbs).
the main db is the ~110 million page directory, which provides results for the bulk of searches. these pages are evidently qualitatively evaluated in some way to justify their inclusion in this limited-set database.
the gen3 db is the rest, approx 390 million more pages. gen3 isnt hit unless a browser digs really deep for search results, or searches on something specific/arcane where results in the main db are limited.
the local db thing may be a guess based on experience, but i seem to remember reading something about it a few months back (cant recall where). the main db is the "permanent" db, and i think gen3 is effectively "everything else", perhaps including the temporary pages. gen3 is the catchall. gen3 might also account for having pages buried in search results, even when using a page strategy that has worked well in the past (ie. it works well for competitive kws in the local or main db, but is buried if in gen3, because the gen3 db isnt accessed until the first two are exhausted).
my guess is that paid inclusion (ala positiontech) gets you into the main db. if your pages are already there, then they stay and probably have some sort of trump based on being older..? anyway, i doubt ink rolled out a whole new db just for paids, i'd guess they just fit in this architecture somehow.
a few helpful articles:
re: inktomi database layout in general
[onlineinc.com...]
re: gen3 vs other dbs
[searchengineshowdown.com...]
seoboy
<note> Marcia, where are you adding the free submissions? I'd like to try some tests on a site I am working on. TIA! </note>
No matter how unlikely it may seem, Jim Stob is certainly in a position to know about this -- it's his company's service after all.
The behavior I am seeing from my paid and not-paid pages lines up with his description to a tee. A page that was in Ink before payment does not usually include info from the most recent crawl -- the listing only updates when the main database updates.
But a page that was created specially for paid inclusion and was never in Ink before shows refreshed information much more frequently.
Looking at the multiple databases on an over-simplified level, these updates would conceptually work as "batch processing" procedures, if I'm not mistaken.
In the first assignment in Advanced COBOL, a multi-step program, we had to enter data into a file, first running it through a validation routine. We then had to sort the data records according to a key field - in this case alphabetically.
In the second assignment we had to enter data into a separate update file - same validation routine, same sorting procedure. The data division was different, with different file names, but the procedure division for this part of it was identical except for the different filename prefixes. We then had to perform a routine to compare the records in the two files and update the main file - the one first created.
Now, these were flat-field databases - we didn't get into three dimensional tables until later - which would not be the case with search engine databases.
Now ostensibly, again over-simplified to an elementary understanding, the Ink partner is applying their own algorithm when performing a search, and querying the different databases to come up with the results for the searcher. So they're searching the PT database, the permanent database, and the "temporary" index of free submits (plus whatever else is going into their results).
This is where I'm getting a little lost and confused - wondering how the date of last update, at HotBot for example, is relevant to the source of the data being queried. There's obviously updating being done on the source files they're using - PT done frequently, if not real-time, new submits comprising a separate, new file, and the permanent - with no clue as to when or how often the actual permanent database is updated, and what the criteria are for inclusion in it.
>a page that was created specially for paid inclusion and was never in Ink before shows refreshed information much more frequently
This is what prompted the above thought process, and if this is the case, it might possibly be worth doing on a frequently changed seasonal type index page.
>seoboy, thanks for the links!
It looks as though something went wrong with your Inktomi listing. I can't find kitchenelf.com .
I am also seeing more listings come up in higher ranks in the AOL listings for Inktomi. Looks like they cleared out a lot of the spam on AOL anyway. I hope this is a good sign!!
Jill
On Jan. 2 it seems like AOL and MSN had some good updates. I'm getting a lot of good hits from both engines since then that I didn't have before. They were all free submissions.
Exactly the problem, and it was getting a bit of traffic from MSN before the web host fixed the error. Now I can't seem to get that listing out and the correct page added, even though I've submitted the other URL, which is now a 404.
The page comes up under enough different search terms that it's worth paying - we'll pay.
Jill
