Forum Moderators: phranque
Surely it's not that hard to have a system whereby the origin of an email can be accurately determined, and rejected if it's a fake?
Is it time for all the larger ISP's to get together and agree a standard to combat the growing spam and phishing problem?
TJ
So you use a single SPF'ed server as your through point for large amounts of spam, would probably work. Just keep switchin servers everytime they catch on.
It would probably block those DSL spammers from hijacked machines for domains that implement SPF trying to deliver to a server that supports SPF.
Did using RBLs help, not at all because we were doing business with china and it seems that every mail server there is on an RBL somewhere.
Some of the blacklists are too aggressive and a single incidence could get you into the blacklist for a long time. IMHO, it would be a safer bet to just drop incoming connection to your server from source ports lesser than 10K from troublesome ip blocks (or redirect it to another server, like google's supplemental results :) where mail there will go to the spam folder). I may be wrong but I would guess it blocks Windows machines (servers or workstations) with a direct connection to the Internet, essentially blocking alot of trojaned machines.
[edited by: kwngian at 5:59 pm (utc) on Mar. 21, 2006]
That also makes me question the effectiveness of SPF.
ditto [webmasterworld.com] ;)
I've also sent an invite to the SPF crew to see if someone can help clear some things up for us
I started a discussion on Sender Policy Framework (SPF) [webmasterworld.com] but it didn't grow as much as I had hoped. There are quite a few folks that remain greatly concerned over the accreditation and paid subscriber model.
Also, to the best of my knowledge, SPF would not thwart any phishing attacks.
I merely added this since it was the opening of the discussion.
SPF checks will find that mywebsite.foo is not the same as somedomain.foo and reject the email, despite the fact that it was the clear wish of user_a to send the email, they just used a different server.
You can actually add that different server in the allowed server list. If it is a one off or infrequent thing, better to log back into your main server and send from there (authenticate before sending).
It would help to lessen the impact of your domain being used as returned address and if it gets implemented more widely, the spammers will have lesser fake email addresses to use. I doubt spammers would like their operation 'interrupted' so they would rather use a domain known not to have an SPF record.
Let's assume that SPF kills off the art of spoofing the domain-related headers 100% stone dead - absolutely DEAD.
Will this stop spammers?
Answer - my opinion : NO
It's all well and good getting into deep explanations about how it works but will SPF actually make a significant difference? Spammers spoof domain-related headers because they can rather than because they absolutely have to.
I can think of one sure fire way to make a much bigger difference - spread a rumour that spammers are fundraising for terrorism. A little bit of fake (or maybe real) evidence and a news story about a spammer being sent to Guantanamo Bay and it would all be over (well, nearly).
Kaled.
No, it doesn't. But, it does minimize the chance of your @example.com domain being used in a spoofing capacity.
All a spammer has to do is to point the return-path back to a domain which is under his own control and setup an SPF record for that domain. Even if the message mentions ebay.com or some other domain in the From: line, the SPF record of the message will validate and the message will pass SPF checking mailservers. This is why the percentage of spamming domains with properly setup SPF records is about twice the percentage of regular domains. (14% to 7% if I remember correctly).
If you want to use the "From:" header as the subject of authentication with SPF, you need to be familiar with the following:mailing lists
/etc/aliases-style forwarding
MUA "resend this message to"
web-generated email
the Sender header
the Resent-Sender and Resent-From headers
There is a big hole in the SPF setup in that it doesn't check the domain in the From: header line of an email message
Mail clients only showing the From: header from the body is a problem that should be fixed in the mail client - or in content filtering happening at the MUA level, not at the MTA level.
The real live situation is that 95% or more of all current emails have the same sender and author and we got use to the idea that sender and author are the same. The times of a separate secretary handling in- and outgoing emails are gone in most situations, but the standard still has provisions for it. The Sender Policy Framework checks the sender with its narrow definition as in the RFC.
Checking From: records against SPF rules might block legitimate emails like president@ovaloffice.whitehouse.gov when the message is sent by secretary@otherroom.whitehouse.gov.
If a good security policy is present, the SPF record of ovaloffice.whitehouse.gov does NOT contain the IP address of otherrroom.whitehouse.gov, to prevent the secretary doing as if she were the president herself. She may only act on behalf.
The president-case is maybe not very realistic, but it is realistic for mailing lists. If I would post a message in a mailing list, I don't want the IP of the mailing list server to be checked against my SPF record, and I don't want to include the IP of that server in my SPF record because I have no control of most of the messages sent by that server.
Checking From: records against SPF rules might block legitimate emails like president@ovaloffice.whitehouse.gov when the message is sent by secretary@otherroom.whitehouse.gov.
I'm hoping that is not the case. An SPF Record would be required for each sub-domain to prevent the above from happening.
In that case the president can send messages directly from his desk and the secretary can send her own messages, and messages written by the president but sent from her account like press releases etc. In that case the headers would look like:
From: president@ovaloffice.whitehouse.gov
Sender: secretary@otherroom.whitehouse.gov
Return-path: secretary@otherroom.whitehouse.gov
Reply-to: optional
mailserver: mailserver.otherroom.whitehouse.gov
The basic SPF implementation is where the receiving email server only checks the SPF record of otherroom.whitehouse.gov, because this is the trust-level as to which the message can be physically traced. If you want to test on the From: address, the IP of mailserver.otherroom.whitehouse.gov has to be added to the SPF record of ovaloffice.whitehouse.gov. But, if the SPF record of ovaloffice.whitehouse.gov contains the IP address of this less trusted mail server, anyone with access to that less trusted mail server can act as if he was sending emails as the president himself. Only changing the From address in the MUA would be enough to do this.
The same for mailing lists. A mailing list server is for me a less trusted mail sender than my own computer. Therefore I would add my ADSL IP in the SPF record of my domain, but never add the IP address of the mailing list server that sends messages on behalf of me to others.
The problem with SPF is that it completely breaks when forwarding comes into play. Let's take this scenario...
1. I have an SPFed domain example.com. I only send from my own mail server (127.0.0.1) and have my SPF record setup to advertise that.
2. I send an email to my buddy bob@example.net who has a vanity domain. His mail server (127.0.0.2) checks my SPF record and all is well so it accepts the email.
BUT...
3. Bob's using a vanity domain that forwards his mail right to his aol.com account. Let's say AOL is checking SPF records. They'll see an incoming message from me@example.com... BUT... the mail server sending it to AOL will be bob's... 127.0.0.2... which is NOT in my SPF record. So, AOL will think it's a forgery and drop it.
Congratulations. You just caused your own legit email to be blocked by implementing SPF.
you can implement anytime, as long as you control your mail servers
That's not correct.
You can implement SPF, as long as you control your DNS servers.
Control over the mail server is irrelevant. There's nothing you have to do to your mail server(s) to implement SPF.
SPF records are just DNS TXT records that have a specific format.
The vast majority of all domains are probably parked, and have no reason to send email. It would speed adoption of the SPF system if there were an outreach program to the owners and operators of parked domains. Just set the SPF records to mean "no email originates from this domain." Spammers often choose parked domains for their fake return path.
Basically, the only real way to authenticate an email is to provide proof of your identity in the *real world*
to a trusted digital certificate authority such as Verisign, and pay them an annual fee to authenticate your
identity.
Then, digitally sign all of your emails with a digital signature.
Then, it is up to the recipient to know how to validate your key if they even want to bother.
Then, you have to tell everyone you want to ever receive email from to do the same.
Then, reject all emails without a verifiable digital signature.
Keep in mind your email address will then cease to be anonymous to anyone you send email to.
You would have a pair of public/private keys, similar to SSL. Anything you send would be encrypted with your private key, and could be decrypted only with your public key, and the public key would be of course publically available (perhaps even automatically download from somewhere specified in the message source itself so the process for the end-user would be almost seamless). It would use existing transport methods (POP3, SMTP, IMAP, etc would all remain intact).
The encryption/decryption would be done by the sender/recipient, so that you wouldn't rely upon a third-party. This isn't a digital signature, which could probably be easily copied and spoofed. Instead, it's a private key, that only you have, that encrypts the entire message. And the recipient is guaranteed* that it's from you because your public key would have been used to decrypt it. It doesn't ensure privacy of emails.. anyone else who intercepts it could also use your public key to decrypt it, but it does ensure the sender.
* - It ensures the sender to a point, of course. If john@email.com sends you an email once, and his public key is downloaded into your cache, you'll know if another email from john@email.com is really from the same sender because your email will warn you that the keys don't match. However, you wouldn't know if the original email is from john@email.com. This is similar to how SSH works.. it uses a fingerprint to verify that you're connecting to the same machine each time, but asks you to verify the fingerprint manually the first time you connect (which most people tend to ignore, and it's normally safe to do so).
Obviously this doesn't directly stop spam or phishing, as each spammer can have their own set of keys. But it makes it easier to identify what you want to receive and what you don't, because the email that you want is essentially signed with someone that can't (easily) be forged.
Not exactly a shiny outcome.
You can implement SPF, as long as you control your DNS servers.
If you use enom's free DNS service, you have the option to use SPF through their system. This is great if you're like me and don't want to spend the money for the upgraded mail server software.
You can implement SPF, as long as you control your DNS servers.
I guess I should clarify, before somebody jumps on me. :)
There are two sides to SPF - sender and receiver.
To implement SPF on the sending side, you need to add SPF records to your DNS.
To implement SPF on the receiving side - i.e. to reject mail from severs not authorized by the sending domain's SPF records and/or to reject mail whose sending domain has no SPF records - you need either an email server that implements SPF checking (and to configure it as such) or else client software that does the same.
This is best done at the server, so that you avoid needlessly downloading messages that you are going to throw away. (Actually, a client doesn't need to download messages - only headers - and can then delete messages it doesn't want.)
So, I guess both answsers are correct. (Though I think what was being discussed at the time was how to set-up the sending side of SPF.)
yep, I mistyped, or was thinking awry or something ;)
either way, good explanation jtara
<Public/private keys are good until some widespread worm appears designed to grab private keys and send them to the owner>
Being a Linux user, I keep forgetting about the existance of worms and viruses :-).
With the current situation with Linux on desktops, you are not a target worth the effort :)
SPF checks will find that mywebsite.foo is not the same as somedomain.foo and reject the email, despite the fact that it was the clear wish of user_a to send the email, they just used a different server.
You can actually add that different server in the allowed server list. If it is a one off or infrequent thing, better to log back into your main server and send from there (authenticate before sending).
In may cases it is not possible or undesirable to send your email from your main server. As for adding the server to the allowed server list, you would need to add thousands of e-card sites, community sites and social networking sites to your allow list for your users to be able to send and receive e-cards, emails and invitations sent through these services.
Can I implement now
You want something that works now? Install PGP at both ends.
best, a.
you would need to add thousands of e-card sites, community sites and social networking sites to your allow list
But I won't turn it off on sites that I manage yet because we have been hit on bounced mail issue.
Seems like it won't work for vanity domains too.
Why not?
You just need an SPF record in the DNS. If you don't control the DNS for your vanity domain, it's easy enough to do so. (Use one of the free DNS services.) Other than that, you just need to know what server(s) you use to send mail, so that you can create an appropriate SPF record.
I'm referring to SMTP StartTTLS [sendmail.org] and SMTP AUTH [sendmail.org].
These are pretty good solutions for roaming sendmail, qmail and postfix users, and they provide a good deal of protection from people spoofing your From: domain (which simply must be allowed to pass in the absence of any other type of check, otherwise, what good is having a mail server?).
Beyond that, we're looking for solutions to verify the source of any email/traffic, and that's got to happen at the packet level, before the higher-level protocols like mail transport become involved. This strategy that has been ongoing [ece.cmu.edu] for years.
verify the source of any email/traffic
Packet source validation will not be able to verify anything except that the machine the packet stream originated from remained uncorrupted during its lifespan. While this won't do anything for phishing, which requires content-based validation, this will (a) minimize the risk of a compromised server being used successfully for spamming/etc and (b) prevent any type of attack where the packets change source in mid-stream (i.e. stream injections and redirects).
Content-based validation (phishing) must still be left to the higher-level protocols, as analysing that sort of thing at the packet level would be way too resource-intensive.
<With the current situation with Linux on desktops, you are not a target worth the effort :)>
And I probably never will be...
I've encountered a few inaccuracies or misconceptions in this thread, which I'll try to clear up.
First, SPF entirely opt-in. If you don't want to use it, then don't. You won't be treated any different from before by those who do use it.
Second, SPF is not meant to prevent spam. It is meant to prevent sender address forgery. So if many spammers adopt SPF early and add SPF records to their throw-away domains, this is actually a big win because they no longer abuse others' domains. (Spam may be prevented indirectly through reputation databases, which can be reliably applied only when the sender address is known to be authentic, which SPF tries to provide. But spam prevention is not the primary goal of SPF.)
SPF has a philosophy that SMTP, as it is now, has certain flaws, or legacy features, that need to be abandoned in order to solve the forgery problem. Alias-style forwarding (i.e. forwarding mail with the sender address unchanged) is one of them. Greeting card sites etc. being allowed to simply claim "legitimate" use of someone's domain is another of those legacy "features". SPF's vision proposes concrete alternatives to those legacy features which everyone should be able to implement easily.
If you come to the conclusion that you prefer to retain those legacy features over the benefits of SPF, then this is entirely your choice. You won't be disadvantaged by others adopting SPF, so do not try to argue to others that they should not adopt SPF because it would (negatively) affect everyone. It doesn't.
Then, SPF is meant for domain owners. It gives domain owners control over which hosts are allowed to use their domain in the envelope sender (AKA return path) when sending mail. Receivers are needed to check domain owners' SPF records, though. SPF cannot work without receivers' cooperation.
If you want every site out there, including greeting card sites, to be able to use your domain(s) as the envelope sender for mail they send, then don't use SPF. But be prepared to receive misdirected bounces for mail that you didn't send, because "every site out there" will potentially abuse your domain. There is really no way to distinguish between benign forgery (one of your domain's users sending a greeting card and using your domain as the sender address) and malicious forgery (phishing, spam, etc.), so you have to accept either none or both. Any theoretical definition of "legitimacy" (like "I sent it, so it must be legitimate") that cannot be reflected in technical implementations may be a nice play of thought but is irrelevant in practice.
Now, about SPF's "forwarding problem":
It is true that there are many forwarding services out there that simply forward mail without changing the envelope sender to their own domain. That is called "alias-style forwarding" because that's what most MTAs do when you configure an "alias". The problem with that is very similar to that of greeting card sites: anyone can simply claim to be "forwarding" a mail from your domain, and there is almost no way for the receiver to tell the benign forwards apart from the malicious ones.
I said "almost" because there is one case where the receiver knows the forwarding to be guaranteed to be legitimate. That is when the receiver themselves has set up the forwarding, as is the case with vanity domains, for example. This means that if you set up a forwarding with yourself as the target, then you need to trust the forwarder and not perform SPF checks on mails received from them (i.e. whitelist them from SPF checks).
Greeting card sites, forwarders, etc. can of course always choose to take responsibility (in a reputation-wise sense) for the mail they send or forward (or claim to forward) by changing the envelope sender to their own domain, so bounces and complaints will go to them instead of someone else whose domain was potentially abused. That is called "sender rewriting" and is considered good citizenship by SPF's philosophy. (There are several different methods of sender rewriting. One of them is called SRS [libsrs2.org].)
One last thing, brundlefly76 said about digital signatures to authenticate mail:
Basically, the only real way to authenticate an email is to provide proof of your identity in the *real world* to a trusted digital certificate authority such as Verisign, and pay them an annual fee to authenticate your identity.Then, digitally sign all of your emails with a digital signature.
Then, it is up to the recipient to know how to validate your key if they even want to bother.
Why should I trust some anonymous 3rd-party organization to correctly certify people's identities? For one, even they are known to have made errors in the past (they once issued microsoft.com(!) certs to a malicious person, for instance). And even more importantly, how does it help me if the spammer who just sent me 8000 spams has been correctly certified to be Mr. Huan Shinshao from China?
What people really need is direct trust relationships to their correspondents (whether those are friends or corporations). A correctly verified digital signature is only useful to me if I know who it belongs to, not if Verisign knows who it belongs to (and often even they don't really know).
So what banks and eBay (and every person) really should do is to digitally sign their mail and directly tell (through a more secure and less anonymous channel than e-mail) their correspondents the key ID they use for all their signatures. Perhaps send them a letter, or have them call a well-known service hotline?
