I like Incredibill's idea here. And I don't understand why it didn't happen 10 years ago already. It's so obvious that email needs to be authenticated.
|I like Incredibill's idea here. And I don't understand why it didn't happen 10 years ago already. It's so obvious that email needs to be authenticated. |
I designed the leading Windows email product of the early 90s (win 3.0) and I've been pitching this concept for over 10 years. The problem is adoption rate and you have to get the big hosts to sign up for fully authenticated email before it's truly viable which means Earthlink, AOL, Yahoo, MSN, Google, Comcast, etc. all have to get onboard.
That's where the trouble starts is none of them want to agree on a damn thing.
FWIW, back in the old days when 56K modems were still in heavy use, people complained that the additional overhead was too much traffic but now in the golden days of broadband that overhead pales in comparison with all the junk mail and the savings alone of such validation would put money back into all their pockets from bandwidth savings alone.
[edited by: incrediBILL at 11:42 pm (utc) on May 24, 2007]
I also like incrediBILL's idea, except
|b) when an SMTP connection is made, you make a new connection back to the originating SMTP server by domain name the get a list of EMAIL ID's being transmitted |
If I read that right, wouldn't it be incredibly easy to bring down mail servers with a type of DoS attack? Just get multiple machines to send out millions of SPAM messages supposedly from mail.example.com to crash the mail.example.com server from the millions of incoming requests for valid mail IDs.
And how is that different from bringing any machine down today by making a bunch of connections?
If it was spam, the validation process would fail quickly and disconnect.
Compare that simple scenario with actually accepting millions of spams.
[edited by: incrediBILL at 3:11 am (utc) on May 25, 2007]
Authentication is fine if it works and people use it, but as I said before (albeit with different words) if the mail stays on the originating server until it is collected, there's no need for authentication.
I've always believed in correcting the root cause of problems rather than bolting on patchcode.
|I've always believed in correcting the root cause of problems rather than bolting on patchcode. |
The root cause of the problem is that you simply don't know that the SMTP server delivering the email is the delivering a legitimate email.
The only solution is to ask the source, that's not patching, that's implementing a full duplex delivery validation to solve the problem at it's root.
|The only solution is to ask the source |
That's a solution. My solution eliminates the need for authentication altogether.
When I go to Google, I don't have to perform authentication to verify that it's the real Google. Similarly, if emails are downloaded directly from the originating server, no authentication is necessary.
Adding authentication is a patch, eliminating the need for authentication is truly removing the root cause of the problem. As I explained, there are also a number of other advantages such as reducing bandwidth requirements, eliminating the need for receipts and opening the door to a fully secure (encrypted) email system. It can also be made reasonably backwards compatible.
|The recipient then either requests the mail or sends an "unwanted" signal |
Not sure if this is what you meant, but "unwanted" might have variations:
"I don't want THIS message".
"I don't want ANY messages from this sender."
"I don't want ANY messages from this server."
It would be to the spammer's advantage to honor "unwanted" messages, and remove the target address from future mailings. This is because they know, definitively, that the mail will be rejected, and it is going to cost them money (if they are actually paying, and not using a stolen credit card, etc.) and slow down their sending of spam.
While individual spammers might not go out of their way to honor unwanted messages, the feature would quickly work it's way into spamming software and touted as a big advantage for the spammer. "Send spam more than twice as fast!"
|Adding authentication is a patch, eliminating the need for authentication is truly removing the root cause of the problem. As I explained, there are also a number of other advantages such as reducing bandwidth requirements, eliminating the need for receipts and opening the door to a fully secure (encrypted) email system. It can also be made reasonably backwards compatible. |
The problem you're neglecting is how email routing works in very large systems, especially shared services such as Earthlink. The domain you connect with is Earthlink, not the site they host such as UncleBobsExampleDomain.com. Since the server sending you the email isn't always the same as the domain the email originated from, that's why I suggested the method I did which is to simply ask the originating domain if the email is legit.
In the case of a big host such as Earthlink, they would respond on behalf of UncleBobsExampleDomain.com, since they host it.
This is no more different than a reverse of delivery notification, "DID YOU SEND IT?" and if the answer is NO, it bounces.
Trust me on this as I looked at all sorts of various angles knowing what sorts of large corporate email systems and routing exists and it's the only thing I could come up with that could be solved by a single point of presence email system that potentially represents thousands of domains.
Your push/pull concept still doesn't preclude a spammer from originating the push and still requires action on the part of the recipient.
My concept is to stop the spoofing and BOUNCE high volumes of spam in real-time, thus shutting down the mechanism being exploited. Even your push/pull scenario allows the spammer to signal the sender, which in itself is an email, therefore you've doubled the actual volume of email just to get a single email and placed bottlenecks on servers with backlogs of email waiting to be delivered until someone "pulls" it from the server.
Have you ever seen a server with a backlog of thousands of emails that can't be delivered slow down and choke the process to the point that new emails take hours to process?
That's why I opted for a clean system that bounces spoofed email in real time at point of delivery so that undeliverable mail is cleanly rejected, email delivery queues are purged, no backlog anywhere.
|The problem you're neglecting is how email routing works in very large systems |
No I'm not - when collecting, there would be no routing beyond that normal to http requests. You're imagining complications where there are none.
This would not stop spamming dead in its tracks, nor would any other suggestion but it would do everything I claimed of it, including reducing bandwidth requirements since only headers would be pushed out not whole emails. Now you could argue that spammers would just send out more emails, maybe they would, but the detection of spam would become much easier - as soon as an email is so-identified the mail could be deleted from the server - you could go further and send out delete requests so that all the pushed headers for a spam item are deleted. Can any other system come anywhere near to that?
Now if spammers use their own servers, my system does not help much, but nor does any authentication system. If spammers use hacked servers, my system does not help much either, but again, nor does any other authentication system.
NOT TRUE, most spam would never be collected.
|therefore you've doubled the actual volume of email just to get a single email |
That is just nonsensical...
|and placed bottlenecks on servers with backlogs of email waiting to be delivered until someone "pulls" it from the server |
Firstly (in the UK, not sure about elsewhere) I believe ISPs have to keep all mail sent for a year or two anyway, secondly, stored data does not constitute a bottleneck (unless you're running out of disk space) it is the transmission of data that creates bottlenecks.
If you mean that you flood the sender with rejections, that really could turn into a bandwidth hog.
|That's why I opted for a clean system that bounces spoofed email in real time |
|Firstly (in the UK, not sure about elsewhere) I believe ISPs have to keep all mail sent for a year or two anyway, secondly, stored data does not constitute a bottleneck (unless you're running out of disk space) it is the transmission of data that creates bottlenecks. |
I'm not talking about permanent storage, I'm talking about the pending EMAIL QUEUE.
Your methodology leaves them piled up in the queue pending delivery based on end user input which would choke most high volume SMTP servers in short order if email keeps piling up waiting to be pulled.
|I'm not talking about permanent storage, I'm talking about the pending EMAIL QUEUE. |
You're misunderstanding my suggestion - there would be no (additional) pending email queue.
When a mail is sent two processes would take place
1) Notification is sent out (using the existing mail system with a single line consisting of a click-here-to-read link for backward compatibility).
2) The mail is stored on the originating server for retrieval by the recipient - there is no queue any more than there is a queue to read a web page. Additionally, caching by intermediate servers could reduce the bandwidth occupied by spam.
When all the recipients have collected their mail, it can be deleted. It could also be deleted after, say, 60 days, or sooner if positively identified as spam. Stores that send out special offer notifications could delete these mails when the offer expires, etc.
I'm not familiar with internet history, but I believe the email system predates browsers - that is probably why we have the current system rather than something along the lines of the one I have suggested.
Recipient servers could retrieve mail immediately, and the system could appear to work like the present system (but spoofing the source is now impossible) or mail clients could retrieve all the mail at login - this would be a little slower for users but again it would not appear to work any differently, or users could opt to retrieve mails individually - anyone that receives a lot of spam would probably take this option.
Detecting spam as it is sent out is probably tricky - I've never considered this problem, but this system allows mail servers that innocently send out spam to detect it later and deal with it, can any other system do that?
| This 42 message thread spans 2 pages: < < 42 ( 1  ) |