New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Is it bad idea to have courier-imap vmail directory under squashfs + overlayfs ?
Its a container with 15G space, Ubuntu 16.04, postfix+courier and /var/vmail has almost 200k files and over 5G.
In theory it sounds good, 5G compressed to 1.5G squashfs file, overlayfs mounted (in rc.local because it was racing with squashfs in fstab) and works fine, no postfix nor courier complaints are logged, for some reason Thunderbird is re-downloading all emails, but emails are being received and handled
Is this a bad idea ?
Comments
Sounds like a bit of a weird setup. If you want to transparently compress the files, try using btrfs with zstd compression, or zfs with compression (although the compression ratio with zfs won't be as high, as it doesn't use zstd yet).
I tried BTRFS but compression was poor and with its own overhead there was less space left than leaving it uncompressed. Maybe I did something wrong but my result was underwhelming.
I worked with SquashFS quite a lot when I was setting up custom built OpenWRT routers, it is actually quite neat concept. Both SquashFS and OverlayFS are built in Linux kernel for many years. It's not that uncommon even in hosting industry, for example according to some sources Docker is using Squash+Overlay combo.
Unfortunately latest Ubuntu 16.04 uses kernel 4.4 which does not support Zstandard yet (from 4.14). I downloaded zstd encoder/decoder and
tar -I zstdcompressed my 5G in 1m25s into 1325MB vstar -ztook 3m36s and resulted in 1615MB. So this is very impressive and way to go on Ubuntu 18.x containers.btrfs uses the same compression algorithms so it should be similar. It does have some overhead for metadata though.
Your squashfs + overlayfs idea seems like it'll be fine, as long as it preserves permissions (which I think it would). I don't think the mail server would care about exactly how the files are stored on disk, as long as it can read and write them.
For zstd support in squashfs, you may be able to install a backported kernel that has zstd support. Alternatively, compile your own kernel, or upgrade to a newer Ubuntu version
I think the difference is that BTRFS treats every file as its own little stream, while SquashFS works like tar and combines files into one stream for the compressing encoder, so entire filesystem shares one encoding dictionary, which makes huge difference for large amount of small similar files.
Well, imapd did not complain since an hour ago when I switched it, but is doing something and increasing system load. I shutdown the Thunderbird on all the PCs that were connecting to mailboxes here and there is still 3 instances of imapd running with 33.2% CPU load each. Wonder what that is about.
UPDATE:
turns out that one time when I rebooted and OverlayFS got ahead of SquashFS in fstab resulted in courier needing to reindex the maildirs. Hence all the activity.
Went back in time by stopping services, unmounting /var/vmail, clearing vmail directory, remounting it back and starting email services at the point when mksquashfs snapshot was taken. No frantic activity this time. Now just manualy copy files from previous overlay's Maildir/cur to vmail's Maildir/new, and no email was lost.
This seems promising. I plan on resquashing the vmail directory nightly. The squashfs file can be used as full and overlay upperdir as differential backup.