All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
IMAP mirror idea
I wonder if anyone thinks the following idea makes sense. I'll ping @Jarland since I imagine using it with MXroute.
Basically the idea is to implement an IMAP mirror to run on a VPS. It would contain both an IMAP server and an IMAP client. The client would connect to a remote IMAP server run by an email provider. It would download all the incoming messages from the remote server as they come, and make them viewable on the local server, optionally deleting them from the remote server after downloading them, if you trust your own backups enough.
Reasons I'd want this:
I'd rather have email delivered to an email provider than self-host an SMTP listener. If the mirror is down for some reason, mail keeps arriving at the provider and the mirror gets it later, or I can login to the provider directly. The provider probably has more serious monitoring and ops than I do, so they'll notice and fix outages faster. And I get to use their spam filtering, which is a significant self-hosting nuisance.
I also don't have to worry about sending email since I'd use the provider's SMTP for that. So they deal with all the IP reputation issues and the like. The mirror would only be for receiving.
On the other hand I'd rather not leave stored email on the provider's machine, for the usual security and privacy reasons. Self-hosting so the mail never touches another machine at all would be ideal, but moving and deleting it as soon as it arrives is a reasonable compromise. The mirror could encrypt everything on arrival. I have years worth of email on Fastmail now, and while I know they are good, I'd rather have the archives under my own control.
The mirror can make replicated backups in real time: each incoming message is pushed out (encrypted) to multiple locations as it arrives, which some mail hosts don't do.
The mirror can encrypt all the stored messages so they are only readable by properly authenticated connections (the auth credential contains the decryption key). So taking over the server wouldn't give access to the messages, except by installing an active attack that gets the key the next time the authorized user logs in. That's much harder than an after-the-fact takeover.
Depending on the software, the mirror could have oter features like better search, that the provider's server doesn't have.
Other stuff that I've forgotten or haven't thought of.
I'm imagining this as a fairly simple script that wraps some existing IMAP client and server code. In fact maybe replication could be done by just installing multiple instances, so the backup instances would mirror the main one without deleting the new messages from it.
Is this idea attractive at all? Am I missing significant disadvantages or obstacles?
Thanks for any thoughts/ideas.
Comments
Its great, but alot of work involved, are you willing to do it?
Because normally there are IMAP clients written in most languages, to make it work both ways and customize with such features will require alot of work.
You can just use an incoming MX relay + an outgoing SMTP relay and self host.
External providers take care of incoming/outgoing, but the data is ultimately stored on your machine.
Worth noting that this is already how SMTP is defined to work on a protocol level; with periodic re-deliveries when an e-mail server is not immediately reachable.
I don't understand? There would be an IMAP client built into the mirror, using (say) Python's imap client library. It wouldn't have to implement every corner of the IMAP RFC. It only has to retrieve messages from the provider. Similarly, the server would just be some existing server. There are lots of those that work well and have plenty of features. The mirror script would mostly just read messages using the client, and deposit them into the server.
How would the incoming MX side of that work? Is MX relaying a standard function offered by email providers? I see there is a company that does it (mx-relay.com) but it doesn't sound common, based on my web search.
I like the idea of using an arbitrary IMAP server as the remote, e.g. it could be a gmail account, and it could be used directly if the mirror had problems.
You point your MX records at an external provider. The email hits their servers, they generally do some spam filtering and then it's sent to your server via SMTP.
Try searching for "backup MX"
https://www.mailcheap.co/backupmx.html
Thanks, that was informative! The products I saw were too enterprisey for my tastes, but the approach is good to know about.
For the basic setup (no encryption), you could start with something like https://github.com/gtozzi/imapcp to copy messages from the remote IMAP server to the local IMAP server. It will skip over messages that are already present on destination/local server (based on message ID). Should be able to implement "delete from source" with one or two extra lines.
So you'd just set that up as a cron job.
(Multiple replicas: just imapcp from remote to local and delete from remote, then imapcp from local to every replica.)
I guess you were looking more for thought on idea than thought on implementing it. It seems reasonable to me if you'll use something like gmail or other big personal email provider, so that you get their robust spam filtering. I think some people use Thunderbird and delete messages from server, so this would be like that but enables multiple clients.
For mxroute I wouldn't do it since I'd just configure two MX for redundancy, chances that both of them go down at the same time before I fix one is low and anyway the sender will probably retry delivery. For antispam, Spamhaus SBL probably won't be too much worse than whatever they have. But if you don't want to maintain that then I suppose it could be useful. Still should check if they could support backup MX first since that'd be simpler.
Another thought... could you just use alias e-mail functionality? (poor person's backup MX)
Assumption is that mxroute reliably delivers messages to configured aliases, i.e., if the MX for alias is down then it will retry for at least a day before giving up. (I'm also assuming mxroute supports configuring aliases.)
Then, you point MX for domain.com to mxroute, and MX for me.domain.com to your own server. On mxroute, setup alias [email protected] -> [email protected].
You host SMTP server to receive e-mails for me.domain.com on your own server, and view the messages with [email protected] IMAP account.
Edit: I think they call this "mail forwarding" instead of "alias".
Right, in past I was interested to write custom IMAP client for CyberPanel using https://github.com/mjs/imapclient which uses https://docs.python.org/2/library/imaplib.html internally.
But I can see that you have already got an answer to your questions as you can do all these things by combining various different software, or someone above just suggested to move to G suite for Zoho (and they will take care of spam filtering for you).
Mailcheap offers Backup MX, at a really reasonable rate.
What you're describing, sounds like a second mail client running somewhere (like a VPS) and not deleting mail from the server (but saving it locally).
Set up a PC at a friends house with thunderbird or outlook and you have this exact functionality. Or a VPS with mutt.
You can also use http://isync.sourceforge.net/mbsync.html to synchronize IMAP mailboxes
Fetchmail to retrieve mail from remote(s), dovecot to make the mailbox accessible.
I did something like that ~20 years ago with a group users on a pay-per-minute dialup connection. 5 or 6 mail checks per workday (9-5). It was a bit more complicated as the local server also handle outgoing mail, which was queued until the next dialup occurred.
This is how I would go about it:
With a dedicated mailserver from Mailcheap, its possible to avoid the imapsync step and dsync push mails directly to your local IMAP server.
Pavin.
I'm curious who you would be trying to sell this type of service to? I am imagining that it would only be useful to single-user end-user's. One thing that you may want to keep in mind is that in some companies (at least the ones that I consult with and have worked for) - if an employee is found to be using a service like this - it would be automatic grounds for termination.
That said - I originally was thinking that using IMAP/POP3 or similar would be a odd solution since as others have mentioned SMTP does a lot of this. But there are some advantages and reasons why someone would prefer the use of POP3 and/IMAP. I have in the past seen solutions to do what you are suggesting but they were roll-your-own solutions and I don't recall seeing it as a service. The only time I see this as a service is as an aggregator service - meaning that if you are an end-user and you have lots of different email accounts at multiple ISPs and/or email providers - you could use IMAP or POP3 to aggregate different email accounts into a single account which can then be access via IMAP. If you add something like that - I think it would widen the appeal.
One advantage that I see with your idea over using SMTP is that it can be done on a per-account basis. Also - it would solve any potential issues with senders that implement strict DMARC - a forwarding method using SMTP could be problematic.
Just my 2 cents.
I think he was intending it for personal use. Sounded like he was asking here to see if there was a better solution, as opposed to gauging interest in a commercial service.
If you use it for your personal e-mail and not your work e-mail then why would they care? (OP was talking about using it for mxroute, so probably he is using for his own domain.)
Don't use imapsync, it used to be open source but now they made it proprietary and latest version have to pay for. And it's not worth 72 EUR, it does the same thing as the open source scripts. The one I mentioned (https://github.com/gtozzi/imapcp) is free, and the code is very straightforward so you can easily edit to add PGP encryption and other things if desired.
As I said above you could probably do this with the alias feature on mxroute already.
Sorry for bringing this old thread back. But I am thinking about the same scenario. I have mxroute and other providers.
Like OP I thought about using mxroute as a kind of 'passthrough' service with no stored mails on it but on my very own server. Tho it's useful to have like at least the last few months still on mxroute if there is some downtime with my very own vps.
First idea that came to my mind was using a second server dovecot with fetch/getmail. Then I read about using a secondary mx entry with alias forwarding. But this would require another mx record meaning my personal mail server on vps is somehow exposed to the public. Where when using dovecot to basically fetch mails the server stays somewhat hidden. Only mxroute will know I connected to their server.
Both got it's Pro's and Con's.
Furthermore I read about IMAP push/idle.
My goal is to have a live backup mail server. Another advantage other this is if I am ever forced to change provider everything on my side would still work. I would only have to change some addresses and credentials in dovecot.
You might also look at Dovecot replication...
https://wiki.dovecot.org/Replication