Archiving postfix mails to another postfix server

Timtimo13

Hey guys,

I have a flawlessly running mail server with SoYouStart within a virtual machine but lately I got some problems with the hardware itself resulting in a machine which shuts down here and there.

As I wanted to move this virtual machine to another datacenter anyway I won't bother with that specific hypervisor machine anymore. A few days ago I needed to access my mails to get entry to a lake but the hypervisor stopped working... That was the moment I realized that backups itself are not enough for me and I wanted to build some kind of an online available arching solution for my mail server.

I started setting up a second mail server which is working flawless as well and now I'm using postfix bcc functions (recipient & sender) to send a copy of every mail which comes in to my arching server (which works great)

I have just one problem: The mail server is configured, that it sends a copy of every send mail to the
archiving server as well which results in the following situation:
mail comes in -> copy gets send to the archive server -> another copy of the sent mail will be send to the archiving server

Which means, that I hold two copies of every mail which comes in.

The setup is pretty easy:
sender_bcc_maps = hash:/etc/postfix/sender_bcc_maps.cf
recipient_bcc_maps = hash:/etc/postfix/recipient_bcc_maps.cf

Do you have an idea on how to stop the server sending a copy of a copy of an email to the archiving server ? I'd love to keep a copy of the mails I send as well but just in case there is no option, I would remove the sender_bcc_maps from my configuration.

Thanks in advance

dfroe

Probably replicating at the backend mail storage level could be more efficient?

I am running a productive DBMail cluster for more than a decade meanwhile. It's just awesome and with some tweaks rock solid.

DBMail uses MySQL/MariaDB for mail storage, provides IMAP4/POP3 daemons for mail access and a LMTP daemon to be fed by Postfix.

Basically you just setup (proper) MySQL master-master replication on two servers and run DBMail with Postfix on both of them. That's it.

You can than access you mail via IMAP from any of the two servers and both servers are able to store new mails. The MySQL replication keeps them in sync. You can load balance or failover external clients between the instances on DNS or IP level.

Just to provide you some ideas which may inspire you.

noen

Maybe Imapsync could work for you:
https://imapsync.lamiral.info/

Erisa

I second imapsync, used it to migrate between providers with no loss of data and also served to create a backup on the old provider for as long as the service remained.
If I remember correctly it's a little fiddly to get started with but once you get the hang of it it's an incredibly powerful tool. Best of luck!

jbuggie

if your settings on the two are similar, I don't see why a simple rsync -az --delete wouldn't do

Timtimo13

@jbuggie said:
if your settings on the two are similar, I don't see why a simple rsync -az --delete wouldn't do

I liked your idea but today I realized, that it would sync all folders and delete actions as well which is
not, what I wanted to happen as I'd like to have an offsite backup of all mails, even if the user deleted it (maybe by accident)

Imapsync does the same.

I thought about another scenario as well today with a backup MX server, which sends a BCC mail to the arching server as well and a BCC mail to main MX as soon as it's back online. The problem with that idea is, that incoming mails would be archived twice, the first time from backup MX, the second time wenn backup MX is delivering the mail to the main MX which sends out a bcc mail as well.

Any ideas on how to stop that ?

mhosting_in

@Timtimo13 said:

@jbuggie said:
if your settings on the two are similar, I don't see why a simple rsync -az --delete wouldn't do

I liked your idea but today I realized, that it would sync all folders and delete actions as well which is
not, what I wanted to happen as I'd like to have an offsite backup of all mails, even if the user deleted it (maybe by accident)

Imapsync does the same.

I thought about another scenario as well today with a backup MX server, which sends a BCC mail to the arching server as well and a BCC mail to main MX as soon as it's back online. The problem with that idea is, that incoming mails would be archived twice, the first time from backup MX, the second time wenn backup MX is delivering the mail to the main MX which sends out a bcc mail as well.

Any ideas on how to stop that ?

you are using VPS then VPS control panel allow any remote backup.
maybe your provider using e.g virtualizor or solousvm
you can setup full backup

Timtimo13

Hi,

It's not about backups, I'm backing up all my virtual machines every hour to a remote server, thats not the problem. I wan't to have a higher availability. Receiving mails while the main server is offline and then replicating them to the main server when it's back online. Users are able to read their mails by accessing the archiving server.

Just an idea on how to improve the "service".

just for information: I'm not a reseller or service provider. Just "providing service to myself and friends of mine" and interested in such solutions

Timtimo13

As I don't got around receiving duplicates of mails, I thought about some
filtering module which checks for duplicates in the users inbox and deletes the
mail in case it's a duplicate.

I'd love to write such a filter but now I'm stuck on how to get the message to be
passed to the filter script.

In master.cf I have the following config:
smtp inet n - y - - smtpd -o content_filter=dupfilter:

dupfilter unix - n n - - pipe
flags=Rq user=mail argv=/dupfilter.py ${sender} -- ${recipient}

When the server is receiving a mail now, I just get three arguments passed
to the script, the recipient, the sender and "--"

How do I get the full message to the script ?

@Timtimo13 said: