Email deliverability issues with Microsoft.

I can't send mail to outlook or hotmail

[NickMNS]

I'm unable to send emails to addresses using Microsoft products such as Outlook or Hotmail. I can send messages anywhere else without any issues including to gmail. All the email spam checking tools such as as Spamhaus show no problem with my IP or mail domain.

Based on the error message from Microsoft, I accessed their SNDS service which shows my domain/IP status as having received complaints, but otherwise there is no data or any evidence of any issues.

I recently changed mail servers, but I'm certain that everything is configured correctly including my SPF, DKIM records. The only potential issue is that the mail server is hosted on a popular shared VPS.

Any ideas on how to resolve this issue? I already have an open ticket with Microsoft but I'm doubtful that it will do anything.

I should point out that my mail server handles the mail from several of my domains and websites, and messages for password rests and the like are sent initiated from the webserver but routed through the mail server.

[not2easy]

Some mail server/configuration testing tools are discussed and linked in this discussion from last July: [webmasterworld.com...]

I haven't run into the issues you describe and I haven't used the tests discussed there, but it may offer insight.

[NickMNS]

@Not2easy
Thanks for that link. I should have searched on my own. I pass all those tests as well. However I think there may be an issue with DMARC record. I just fixed it, I'll see what happens.

Amazingly I got a response from MS, it wasn't really useful though, they just links to the same to help/faq pages. But the back and forth is not done.

[engine]

These annoyances can make of break campaigns, and the smaller problems can be harder to find.


Let us know how that goes.

[tangor]

What is the actual error message status code?

[NickMNS]

@engine

These annoyances can make of break campaigns,

The problem is I'm not running a campaign, I am simply trying to send individual messages. This is impacting the usability of my website, if users can't get password reset or signup messages it is really a big problem.

@tangor
This is the error message.

Diagnostic-Code: smtp; 550 5.7.1 Unfortunately, messages from [11.22.33.44] weren't sent. Please contact your Internet service provider since part of their network is on our block list

The problem is that my IP is on Microsoft's black list. they refuse to remove it and will not indicate what the problem is or what steps to be taken to have it removed.

My next step is ask my VPS provider for assistance. If that doesn't work I'll be forced to get a new IP and start from scratch resetting all the DNS records.

[graeme_p]

That is all fairly typical of MS.

I cannot send to hotmail addresses, but I have no problem with Offce 365 ones.

I get the same sort of response from them.

You might want to send outgoing email through an SMTP provider. its annoying and a bit of a waste of money but not expensive unless you are sending a lot of email and they fix the problems.

[NickMNS]

That is all fairly typical of MS.

Yup! My VPS provider has long community thread describing the issue, with many other people describing my exact problem. It would appear to be an issue of having bad neighbors (IP). I have opened a ticket, let's see if they can get the problem resolved.

[engine]

@NickMNS

The problem is I'm not running a campaign, I am simply trying to send individual messages. This is impacting the usability of my website, if users can't get password reset or signup messages it is really a big problem.

Understood.

This issue seems eternal.

Whilst Hotmail provides user protection from spam, it's as if it's turned up beyond mid-way.

In the end I gave up trying to resolve this and made it clear to recipients that they should check their spam folder, or if not received within 24-hours should make contact with the office. We even tried moving it to sms to get over the problem but many people didn't want to give out their phone number.

Good luck.

[tangor]

Whilst Hotmail provides user protection from spam, it's as if it's turned up beyond mid-way.

TRUE! I have also noted that certain keywords in subjects ALWAYS get the axe, even if they are not problematic words. Things like "just for you", or "make your day", etc.

These are very perplexing and there's not much you can do except make sure your emails have not hint of "bait and click" in even the narrowest of confines.

[SumGuy]

So if you are logged into an outlook account or hotmail account (using, say, a web interface aka browser) and you are doing so from the IP in question, and you compose and send an email to another outlook or hotmail account, the email is rejected by MS (with the above-mentioned 5.7.1 error) ?

[NickMNS]

@SumGuy unfortunately I can't do that.

[martinibuster]

mail server is hosted on a popular shared VPS

Some hosts and especially cloud services are associated with VPNs, which can be poison.

Email failure can happen to regular web hosting environments too. So if it's a business critical email then that's when an email service like Mail Poet comes in handy.

[mailpoet.com...]

[NickMNS]

@Martinibuster using a third party service is not a solution I'm considering right now, I view it as a last resort.

I solved the issue in the mean time, by opening a support ticket with my VPS provider, who was able to have the IP removed off of MS blacklist.

[graeme_p]

@NickMNS it is clear that what @martinibuster and I suggested is using a service as a smarthost/relay for outgoing mail?

Its worse privacy though, they can read outgoing email (unless encrypted, which is rare) and all addresses. Better than using a completely hosted mail service as they cannot see incoming mail.

[NickMNS]

Its worse privacy though, they can read outgoing email (unless encrypted, which is rare) and all addresses.

It may be a workable solution for sending promotional material or a news letter, but when using email for password resets, it not just a privacy issue but a security issue. Because, the "service" would have sufficient information to be able to take control of accounts.

[NickMNS]

As noted above I resolved the issue with Microsoft. For the sake of anyone else looking to resolve a similar issue here are the required steps:

Step1 Make sure your DNS records are in order. You must have valid SPF, DKIM and DMARC records.
Step2 Check the spam services website to be sure that your domain and IP or not on those lists (eg: spamhaus).
Step3 When a message bounces back go to the "troubleshooting" link, and file a support ticket with Microsoft requesting removal of your IP from the blacklist. The request will likely be rejected, but it worth trying.
Step4 File a support ticket with your VPS provider, they should in turn file a request Microsoft. Be sure to include in the request the error message, and confirmation that you have completed the 3 steps above. It will take a few days, but this should get your IP off the MS blacklist.

[graeme_p]

It may be a workable solution for sending promotional material or a news letter, but when using email for password resets, it not just a privacy issue but a security issue. Because, the "service" would have sufficient information to be able to take control of accounts.

If you are receiving password resets, then they do not handle those.

If you are sending password resets, they can, but so can the recipients provider. The window of opportunity only exists between your system sending out a reset link and the receipient using it, so its not a huge security issue. Like a lot of things, do you trust them? I think you said in a nother thread that you are using a VPS? Your VPS provider can see all of this and more.

Just use a reasonably trustworthy provider. I would consider using a separate transactional email provider for these.

[NickMNS]

The window of opportunity only exists between your system sending out a reset link and the receipient using it, so its not a huge security issue.

No that is incorrect, given that the "service" has knowledge of the user's username and email address as well as access to intercept emails, this allows a malicious actor at the service to make a password reset request at any time, intercept the email and take control of the account. This isn't "not huge" security issue, it is a gaping hole.

Your VPS provider can see all of this and more.

While I'm not certain that it is technically possible, it would be extremely unlikely. The VPS provider I use has data center around the globe and has hundreds of millions of dollars in revenue annually, they also has wide list security certifications. In contrast the email service recommended by Martinibuster has 1 paragraph about security, that starts off with:

While no online service is 100% secure, we work very hard to protect information...

Moreover, this isn't either or, the VPS is required no matter what, the email service is simply adding to the attack surface.

[graeme_p]

No that is incorrect, given that the "service" has knowledge of the user's username and email address as well as access to intercept emails, this allows a malicious actor at the service to make a password reset request at any time, intercept the email and take control of the account

Good point. On the other hand they are going to have to do it one account at a time. If they need to know the username it will narrow their opportunities further.

This is a good argument for 1. not allowing resets on any admin accounts on a system and 2. using two factor auth.

While I'm not certain that it is technically possible, it would be extremely unlikely. The VPS provider I use has data center around the globe and has hundreds of millions of dollars in revenue annually

It is definitely possible. The data is stored on their hardware.Its trivial. Encrypting the drive may help a bit.

I am not convinced that bigger is necessarilly more trustworthy. In any case individual admins do not have that sort of money, and it only takes one bad actor who has access to do it.

they also has wide list security certifications. In contrast the email service recommended by Martinibuster has 1 paragraph about security, that starts off with:

I agree about that There are plenty of other services. If you like big suppiers with known names, Send Grid is owned by Twilio. Postmark is owned by Wildbit who are not that big, but have been around a while and run a lot of critical services for a lot of people.

Moreover, this isn't either or, the VPS is required no matter what, the email service is simply adding to the attack surface

Of course, but security always means trade offs. For example, it would be more secure to replace the VPS with a dedicated server, safer still to buy a brand new dedicated server and set it up your self and then colo, and more secure to use a server you control physical access to. The last options in impractical, but low end dedicated servers only cost tens of dollars a month.

[NickMNS]

It is definitely possible. The data is stored on their hardware.Its trivial. Encrypting the drive may help a bit.

It may be possible but it certainly is not trivial. Access is limited on the database, they may be able to read the content but they certainly cannot write to it. This limits what can be achieved in an attack. To achieve anything would require a high level of sophistication and an ability to code. This pales in comparison to the email service attack, where all that is required is interception of an email that is sent in readable format through the service to which they have access.

Note that my email server, and web server are in separate geo-locations, in fact in different countries, and emails sent by the web server are not saved on the mail server, the mail server simply relays the messages.

Again I have my doubts about how much is really possible, by it certainly isn't trivial. And if I'm wrong, then I'm not doing things correctly.

For example, it would be more secure to replace the VPS with a dedicated server, safer still to buy a brand new dedicated server and set it up your self and then colo

I'm actual in this situation (setting up new email accounts) because my previous mail server was on a dedicated server in colo, managed by an acquaintance. Due to series of unfortunate events, the owner of the collocation site took possession of the box, hacked into it and tried to use it as leverage for some unpaid bills. Fortunately, I was not directly impacted, but I had to move to mitigate the risk.

[graeme_p]

Access is limited on the database, they may be able to read the content but they certainly cannot write to it.

Interesting. Where is the database hosted, how is access restricted (passwords or something else?), and can your website not write to it either? I would have expected it to in order to provide functionality like password resets there must be writes to the database.

Note that my email server, and web server are in separate geo-locations, in fact in different countries, and emails sent by the web server are not saved on the mail server, the mail server simply relays the messages.

Is the connection between the two encrypted? Otherwise it can be intercepted without touching either server simply by watching traffic on outgoing ports. The same is true for unencrypted email going out from the mail server.

If someone can read the database and the code that sends out a reset email they can presumably read enough to reconstruct the email reset link.

I am finding this a useful conversation, by the way. I am thinking of things to check regarding security while doing this. One is, on sites that use email links to reset passwords to not allow this for admin or any kind of privileged accounts (at app level, not at OS level, I mean).

[NickMNS]

Where is the database hosted, how is access restricted (passwords or something else?),

The website's database is on the web server and the mail database is on the mail server. The database is password protected, and the web application has access rights to the database for both read and writes.

Is the connection between the two encrypted?

Messages are sent using SSL/TLS.
See this for details: [mailinabox.email...]

If someone can read the database and the code that sends out a reset email they can presumably read enough to reconstruct the email reset link.

Yes but it isn't so straight forward. Because the reset code is only present in the database for a short time. The attacker would need to initiate a password reset then access the database, and then complete the process. Yes it's possible, but this far more complex than simply intercepting an email.

I am finding this a useful conversation, by the way.

I agree.

[graeme_p]

Messages are sent using SSL/TLS.

So then you have that covered except in the (hopefully very rare) case where the recipients server does not accept TLS connections. Actually I should have assumed this is usual practice so this was a red herring. They can probably see meta data in log files, but that is about it.

Mail in a Box is great. I use it myself.

The website's database is on the web server and the mail database is on the mail server. The database is password protected, and the web application has access rights to the database for both read and writes.

This is where you might have a weakness. If the web app has acess, the password is presumably in a file somewhere in plain text. That can be easily read. At that point it is at least as easy as intercepting an email. As they will have write access they could add the reset code to the database and then create the link, or just change the password in the database directly (as they can see your code they know the algorithm used, salt etc.).

It might be a bit harder if the database only accepts connections from localhost (AFAIK that depends on virtualisation software and configuration). Connections over Unix sockets are better still and slightly faster too. With Postgres (I do not know whether other DBMS's support this) running on the same server I use peer authentication, which allows access over a unix socket to a particular OS user (by default with a name matching the postgres user).

[NickMNS]

It's back! Yup, Microsoft banned my IP again, not for anything I did, but I assume for the whole block. Again Spamhaus and all other services show the IP as being clean so it is really only Microsoft.

But from a business prospective this is problematic. Ideally I would like to manage my own email without requiring any third party services, but thanks to F'n MS that may not be possible.

[tangor]

but thanks to F'n MS that may not be possible.

... and one can only expect more of the same in the future. "We" are not "big enough" to make a noise that can be heard...

At this point it is "grin and bear it"...