homepage Welcome to WebmasterWorld Guest from 107.20.25.215
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque & physics

Webmaster General Forum

    
SPF? not just for sunburns?
Sender Policy Framework
httpwebwitch




msg:4362218
 2:45 am on Sep 14, 2011 (gmt 0)

I sent an email to a customer tonight, and it bounced back with this message:

SMTP error from remote mail server after DATA:
host smtp.****bleep****.net [*****IP address****]: 550 5.7.1 SPF unauthorized mail is prohibited.


I thought I took care of this a long time ago. I don't understand the syntax, but my best advisors told me to go into WHM and create a DNS zone.

The record looks like:

example.com. (yes there's a dot after the TLD. Is that normal?)
14400 (I think that's the TTL)
IN (what does that mean?)
TXT (as opposed to CNAME or MX or A)
"v=spf1 a mx -all"

Is this correct?

 

httpwebwitch




msg:4362219
 3:15 am on Sep 14, 2011 (gmt 0)

ok it only took a few minutes on wikipedia to get a handle on the "v=..." part.

That SPF record says that mail is allowed to be sent from the "A" and "MX" systems, and all others should be forbidden.

So does this mean that the IP of the "sender's address" doesn't match the IP in the A record? And that's why the message is bouncing?

I'm sending my mail through the SMTP defined as "mail.(mydomain).com", authenticated by password.

The DNS record for the "mail" subdomain is a CNAME to (mydomain).com.

The MX record for (mydomain).com is [0] (mydomain).com.

The A record for (mydomain).com is the IP address of my server.

As far as I can tell, everything's set up properly. And... it's worked fine until just a few days ago when I started getting occasional messages bouncing back.

httpwebwitch




msg:4362220
 3:26 am on Sep 14, 2011 (gmt 0)

A test

As suggested by the SPF tools page:
[openspf.org...]

I sent an email to spf-test@openspf.org

The result: it bounced back (as it's designed to do), but the news wasn't good


This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

spf-test@openspf.org
SMTP error from remote mail server after RCPT TO:<spf-test@openspf.org>:
host mailout02.controlledmail.com [72.81.252.18]:
550 5.7.1 <spf-test@openspf.org>: Recipient address rejected:
SPF Tests: Mail-From Result="
fail": Mail From="**bleep**@***bleep***.com" HELO name="vps.***myVPSdomain***.com" HELO Result="none" Remote IP="***bleep***"


What's the problem?

httpwebwitch




msg:4362223
 3:37 am on Sep 14, 2011 (gmt 0)

the problem is that the Remote IP - as echoed by the response from openspf - doesn't match the IP defined as the "A" record of my domain.

They look very similar. But the last octet is slightly different. Definitely something funky going on with my host.

lucy24




msg:4362224
 3:50 am on Sep 14, 2011 (gmt 0)

What's the problem?

"bleep" seems to sum it up ;)

The sender is not authorized to send to the destination. This can be the result of per-host or per-recipient filtering.

But I like the generic x.7.x explanation better:
The security or policy status codes report failures involving policies such as per-recipient or per-host filtering and cryptographic operations. Security and policy status issues are assumed to be under the control of either or both the sender and recipient. Both the sender and recipient must permit the exchange of messages and arrange the exchange of necessary keys and certificates for cryptographic operations.

In other words, you need to get together with the recipient-- by carrier pigeon, I guess-- and agree on, uhm, something. Seems a bit extreme if all you were trying to say was "Yes, we have that in stock".

httpwebwitch




msg:4362371
 1:26 pm on Sep 14, 2011 (gmt 0)

Actually the mail in question is mission-critical for the normal functioning of my app, and the SPF dysfunction has worse consequences - I noticed that some hosts have added my domain to their spam blacklist. So, this not a trivial issue.

Thankfully, I've solved the problem and it wasn't difficult to do.

I'm running this site on a VPS, which hosts several domains. The root IP for the VPS is A.B.C.X (for example)

this one site in question has a dedicated IP, A.B.C.Y (for example)

So, the "A" record for the domain is A.B.C.Y, but the mail service is all hosted on the VPS, which is A.B.C.X. They don't match, and that's why the SPF Test failed.

My SPF record states "v=spf1 a mx -all" , meaning the IP of the sender-host must match the IP in the "A" record. It didn't.

Rather than fuss with the IPs, the solution was far simpler. I changed the SPF record to:

"v=spf1 a mx ip4:A.B.C.X -all"

Thus allowing the root VPS mail service to send mail originating from my domain.

Wait an hour for the DNS to propagate... then I sent another test message to spf-test@openspf.org

the response:


spf-test@openspf.org
SMTP error from remote mail server after RCPT TO:<spf-test@openspf.org>:
host mailout02.controlledmail.com [72.81.252.18]:
550 5.7.1 <spf-test@openspf.org>: Recipient address rejected:
SPF Tests: Mail-From Result="
pass": Mail From="***bleep***@***bleep***.com" HELO name="vps.***myVPSdomain.com" HELO Result="none" Remote IP="***bleep***"


Now the SPF test passes.

SPF is an arcane art; I can appreciate how easy it is to ignore, or configure badly. But now I know the importance of doing it, and doing it right. The first time around I was merely pasting in an ignorant suggestion, and without knowing it my SPF test has been failing for a long time. It took me a couple of hours to read a little deeper, understand how SPF works, how to test if it is working, and figure out a solution.

I hope this thread helps someone overcome the same problems.

Hoople




msg:4362374
 1:47 pm on Sep 14, 2011 (gmt 0)

As an email admin by day and webmaster by night I vote that this thread be placed in the WW Library.

httpwebwitch




msg:4362388
 2:29 pm on Sep 14, 2011 (gmt 0)

btw, the tongue-in-cheek title of this thread alludes to a more common meaning for the acronym SPF, "Sun Protection Factor", which is a number they stamp on sunscreen and lotion products in North America. Readers unfamiliar with that other meaning of SPF might be puzzled by that.

Status_203




msg:4362744
 8:35 am on Sep 15, 2011 (gmt 0)

Actually the mail in question is mission-critical for the normal functioning of my app


Do you have DomainKeys set up?

I recently added a forum to one of my domains (mostly to handle user registrations for tools on the site rather than for the forum itself) but it soon became clear that despite correct SPF and reverse DNS having been setup, certain email providers (including Yahoo) were bouncing mails from the server. Investigation indicated that lack of DomainKeys was the issue.

After a few searches I decided life was too short to include setting up DomainKeys on my current server and went with a third party, self-service, cloud based emailer ($0.001 per email - at my volumes $20 will last all year, maybe more :( ). Note if you're using PHP then the PEAR Mail package allows sending through 3rd party SMTP servers.

httpwebwitch




msg:4362941
 5:16 pm on Sep 15, 2011 (gmt 0)

No I don't have DomainKeys set up.

From the looks of their site, I don't think I'll be setting it up in this decade, either.

Look in the docs, under the heading "Implementation":

The signer needs to add code in the appropriate agent, to perform signing, and they need to modify their DNS administrative tools to permit creation of DKIM key records.

A validator needs to add code to the appropriate agent and then feed the result into the portion of their system needing it, such as a filtering engine.

The mere existence of a valid signature does not imply that the mail is acceptable, such as for delivery. Acceptability requires an assessment phase. Hence the result of signature validation must be fed into a vetting mechanism that is part of the validator's filter.


Seriously, that's it. The whole chapter. That's all you get.

DKIM = Yet another pile of esoteric technology that no one is going to use

Status_203




msg:4363279
 8:48 am on Sep 16, 2011 (gmt 0)

DKIM = Yet another pile of esoteric technology that I had to use (in some way) if I wanted people with yahoo email address to be able to register to use my tools!

Actually, my sunconcious has obviously been mulling this over for the last day I've got vague memories of the initial bouncing being related to switching forum software.

I emailed all currently active users ( I think it was approx 30 at the time!) that I was changing software, that the tools would be down on such and such a date, and if they couldn't log in when they were back up to let me know.

With the way my server was set up at the time I think it took only 6(!) identical emails to Yahoo addresses without DomainKeys for them to blacklist my web server as an email sender!

httpwebwitch




msg:4363406
 3:00 pm on Sep 16, 2011 (gmt 0)

Take heed: this SPF stuff requires that you change the DNS records for your domain. If you're familiar with DNS and how it works and how to change it without breaking everything, then all is well. But if you're new to DNS shenanigans, it's good to do a little reading & refresher before fiddling with it.

The SPF record has a type "TXT". Some of my servers are running WHM - and with that I'm able to add a "TXT" records to the DNS, no problem.

But I also have a couple of Rackspace Cloud instances, and using their management tool you can add an "A", "CNAME", "MX", but... not a "TXT". To add an SPF record to a Rackspace VPS, you have to open a support ticket and get one of their support staff to do it for you.

httpwebwitch




msg:4363408
 3:09 pm on Sep 16, 2011 (gmt 0)

My app sends notification emails to buyers giving them instructions for completing a sale. So, that's mission-critical, and when many people buy the same product the emails are pretty much identical. This may be more crucial than I thought.

To set up DomainKeys it's another "TXT" record added to the DNS.

Here's what an online tool generated for me, I have no idea yet if it's correct:

/._domainkey.example.com IN TXT "v=DKIM1; p={public key goes here}; s=email; t=y"

J_RaD




msg:4363469
 5:08 pm on Sep 16, 2011 (gmt 0)


(yes there's a dot after the TLD. Is that normal?)


yes always

Hoople




msg:4363593
 12:30 am on Sep 17, 2011 (gmt 0)

Two point worth mentioning for those on cPanel hosts:

The sending email server *may* be bound to a different IP than the webserver. If so add an additional ip4 server to setup.

Adding the DK/Sender ID/SPF records is easy - all forms and buttons.

Seb7




msg:4363597
 12:40 am on Sep 17, 2011 (gmt 0)

I recently had the same issue, mails getting bounced a lot, and ended up learning about SPF, then applying a similar rule to my dns which I must admit has helped greatly.

SPF has been around for sometime, but it does appear it is starting to be used quite heavily now.

I still have a problem with aol though, they seem quite happy in blocking too much. I have quiet a few important emails lost in the aol black hole, even though I'm not on any public blacklists.

Hoople




msg:4363602
 12:56 am on Sep 17, 2011 (gmt 0)

AOL has a surprisingly good help page [postmaster.aol.com ] visit it and apply for whitelisting to fix your blocking issue.

incrediBILL




msg:4363931
 4:54 am on Sep 18, 2011 (gmt 0)

I usually use the SPF Wizard on the MS site, works real nice
[microsoft.com...]

httpwebwitch




msg:4366012
 7:55 pm on Sep 22, 2011 (gmt 0)

that address for checking the validity of SPF (spf-test@openspf.org) was really handy. Without it I wouldn't have known for sure that everything was OK.

Is there such a thing for testing a Domain Key?

httpwebwitch




msg:4366043
 9:01 pm on Sep 22, 2011 (gmt 0)

Um I went into CPANEL and there's a button that said "Enable Domain Keys" and I clicked it. then the page refreshed and now it says "Status: Enabled & Active (DNS Check Passed)"

That's dandy. I don't know if it's really working. How can I check?

And then I'll need to do the same thing to some servers that don't have CPANEL. It'll be good for my soul to learn how to configure this from scratch.

Any tips for getting Domain Keys working on a Fedora box running postfix?

onlineleben




msg:4366186
 6:29 am on Sep 23, 2011 (gmt 0)

I don't know if it's really working. How can I check?

Send yourself an email to a freemail account and check the raw headers. You should find a line starting with: DKIM-Signature

httpwebwitch




msg:4366510
 8:08 pm on Sep 23, 2011 (gmt 0)

an email from the account in question, to my hotmail address.

after bleeping out personal details:


Authentication-Results: hotmail.com; sender-id=temperror (sender IP is --.--.---.---) header.from=------@-------.com; dkim=none header.d=------.com; x-hmca=none

X-Message-Status: n:0:n

X-SID-PRA: ----- ----- <-----@-----.com>

X-DKIM-Result: None

X-AUTH-Result: NONE

X-Message-Delivery: Vj0xLjE7dXM9MDws5TAhYT01O0Q9MTtTQ0w9Mw==

X-Message-Info: JGTYoYF78jGzuoPRzxQ33HYff5bTZMxB31RtcDy/c5MVSw1+ufv+rbWCLt7lLv+uM4fTNhHx0vpvpta+
MNDYBtFGrcpxjP9W1IIDSsU4EfvDFZsOucGHRBtjyQKWBecU5xomFn8cxbI=

Received: from ---.-------.com ([--.--.---.---]) by snt0-mc4-f12.Snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675);

Fri, 23 Sep 2011 13:03:48 -0700

Received: from --------.--------.--------------.net ([--.---.--.--] helo=[10.10.10.217])

by ---.-------.com with esmtpa (Exim 4.69)

(envelope-from <-----@------.com>)

id 1R7ByI-0003gg-Sb

for -----------@hotmail.com; Fri, 23 Sep 2011 13:03:43 -0700

From: --- ---- <-----@-------.com>

Content-Type: text/plain; charset=us-ascii

Content-Transfer-Encoding: 7bit

Subject: testing DKIM

Date: Fri, 23 Sep 2011 16:03:32 -0400

Message-Id: <92E7E47D-E0AD-4145-9011-4AC22D448D5F@scubbly.com>

To: --- ---- <----------@hotmail.com>

Mime-Version: 1.0 (Apple Message framework v1082)

X-Mailer: Apple Mail (2.1082)

X-AntiAbuse: This header was added to track abuse, please include it with any abuse report

X-AntiAbuse: Primary Hostname - ---.-------.com

X-AntiAbuse: Original Domain - hotmail.com

X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]

X-AntiAbuse: Sender Address Domain - -------.com

Return-Path: -----@-------.com

X-OriginalArrivalTime: 23 Sep 2011 20:03:48.0470 (UTC) FILETIME=[E8326160:01CC7A2B]

testing!

[edited by: phranque at 12:54 pm (utc) on Sep 24, 2011]
[edit reason] fix thread width [/edit]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved