Ask HN: Do you backup your Emails?
I wonder if its just me or other people feel the urge as well to backup their emails. I feel oddly attached to my older emails and have the fear of loosing something important which might hide in my mail archives.
Also I don't necessarily trust some of my email providers (nor my self hosted one) and soonish have to clear one out because I run out of storage.
So what tools are you using to backup your emails? Is there a service which takes care of this automatically?
I use offlineimap to sync several email accounts both to local storage, and to accounts on other free email providers. E.g., a gmail account dedicated to bug reporting is replicated by offlineimap to an outlook.com account.
Address books and calendars for random gmail accounts get synced by vdirsyncer to both local storage and also to other backup gmail accounts.
It works well, and I don't have to worry about e.g., Google permanently locking me out of one of these accounts, without recourse.
* Both offlineimap and vdirsyncer are in Debian's repos. But, I had to pin/install vdirsyncer from testing to get working oath support for gmail (it did not pull in any dependencies from testing that were concerning [I don't recall if it pulled in anything at all, but I would not have installed from testing if it pulled in anything concerning]).
No. Yolo. (Also, in the past 20 years I never needed to find any personal email that's older than a week, so I'm not terribly concerned about the effect of dataloss.)
I would have agreed with you until this year where a single email from 18 years ago, which I didn’t think anything about at the time, is saving me from being sued. It doesn’t matter until it does. I got lucky.
This is why I save documents like this too, but I've only had receipts for small bills come in handy.
If it's possible to give any detail without giving too much, I'm curious what the nature of the situation was and what the saving document was?
A company claimed my use of their software library was outside what I paid for, so they wanted 17 years back pay at a ridiculous rate. I had an email from back then from them saying my usage was within the license. It didn’t even occur to me they might claim the opposite almost two decades later.
This is exactly why I save all of my business email and I don’t mince words about that. I’m very up front with my customers that email is important and that both parties should save our correspondence. Whenever I’m asking for/proposing/communicating about changes to the terms of our relationship I always explicitly say “you should keep a copy of this email as evidence of [...]”
On the flip side, when a company refuses to acknowledge things like arbitration opt-outs that comply with the requirements articulated in the ToS (you know who you are), I email myself a copy of the Exchange trace report showing that the server never recorded a reply. I’m also usually open that I’m doing that, too.
Generally speaking records are an asset not a liability, unless you’re up to something shady. Generally.
My counter point is that in 20 years, I've needed to find old emails probably about 100 times.
I have almost every email of my life, so it's easy to find them.
I regular look at years old emails to check for things like who I was insured with for something or what SKU to reorder for a part. I like to backup with Google takeout for a "no brain" backup.
It's also easier if you already think of emails as "disposables", so if you have any important information in them, you already back it up on a different medium.
My email provider ought to have its own backups for their own service-continuity. (IMAP in this case, but the picture looks very similar for POP.)
Meanwhile, my desktop e-mail client (Thunderbird) has its own local files which are captured by a consumer online-backup solution (e.g. Backblaze) along with other documents, photos, etc. That means both local-copies of server mail, and also items which have been perma-downloaded into mailboxes and purged from the server.
In a way, my biggest worry is not backup-coverage of the bytes per se, but potential archeological issues when it comes to arcane mailbox formats. (My second-biggest worry is that my organization is terrible and Inbox keeps growing.)
I'm using fastmail, but mostly for the reputation. I have a cron script that fetches mail every 5 minutes locally with mbsync (I view them with notmuch and emacs). Backup is done alongside my other files.
I'm using lieer and mujmap to sync my Gmail and Fastmail accounts to a local notmuch mail storage on top of ZFS pool. That ZFS pool is in turn replicated off site.
I use neomutt to access my archive over SSH. And notmuch is very fast at searching all of my emails.
* https://github.com/gauteh/lieer
* https://github.com/elizagamedev/mujmap
* https://notmuchmail.org/
My mail provider does some kind of periodic backup which they do out of their own goodness of heart even though they don’t have to (that what they say).
I save important official docs from my mail — I even sometimes export eml files to those archive/backup folders. So all those get backed up.
The emails themselves? Not really — even though the mail folder (Thunderbird’s; not Mail.app’s — can’t trust an Apple made utility app to be part of even a half hearted backup process) is actually part my backup routine as part of one of those 3 backup tools running on my system (and 2 sync tools). But I seriously doubt I am going to find an email from a backup snapshot ever. But if I must I have that.
But I would want to — have wanted to. Maybe a dedicated tool that downloads every single mail and keeps it synced with the server in an email file format that’s most efficient for backup and querying that backup later. Or maybe tweak around Thunderbird that makes that happen.
Edit: Oh, I forgot to mention — I immediately delete those official mails (like statements, bills) immediately after saving those PDFs if I have to. I think I delete 99% of email I receive (okay maybe 80-90% - not sure; you get the idea). Only personal emails are what I never delete. So I never have too much email to begin with. I do the same with photos and videos and screenshots taken - if it can be detected, it will be deleted.
If you use Thunderbird and want to store your emails "in an email file format that's most efficient for backup and querying that backup later", have you considered switching from mbox (Thunderbird's default mail format) to maildir (each email stored in its own file)?
https://support.mozilla.org/en-US/kb/maildir-thunderbird
https://wiki.mozilla.org/Thunderbird/Maildir
Note: Thunderbird lacks full support for maildir. This is the reason why I switched from Thunderbird to mu4e (an Emacs-based mail client) years ago. Though, some Thunderbird users say they've been using Thunderbird with maildir for years without issues.
I had thought of doing it last time. Thunderbird documentation makes this so scary that I ended up just leaving it be.
No, I stopped even reading my email unless something prompts me to retrieve something from it.
I don't even keep my emails. I have nothing in my inbox at the moment.
If an email contains something I need to keep, it is transferred to my proper files.
Empty mail, happy life.
I have an "empty" email folder. Nothing ever goes into it. Once I'm done checking my dozen email inboxes, I switch back to the "empty" folder and it gives me the feeling of "nothing in my inbox at the moment".
Clicking “Archive” also has the same effect + you can easily finds things while not being at the computer.
It just removes the clutter from view. It’s not the same thing.
I’m using this system as well.
I don’t keep letters in my mailbox. I don’t see why i should keep emails either.
This analogy isn’t convincing for me. My letter box and my email inbox both have a finite amount of space, but I can (effectively) infinitely extend my email inbox either “online” or by archiving it someplace.
I cannot achieve this with my letter box.
That assumes large volumes of important mail. Even without that though, an email inbox is a natural and “first class” place to store an email, whereas a letter box is not designed to be effective at any sort of reliable storage.
I rather move all important stuff out of email and back it up with more relevant tool.
To be calm that if all email is lost - it’s not a big deal.
I find the idea of backing up whole email life silly.
You don’t back up your physical post from beginning of ages, right? You rather keep only small fraction of most valuable things while dumping once in a while all temporarily important stuff.
Email is not a goal for me, just a mean to an end.
No. There is really nothing useful in there. It's mostly just all receipts for online orders that I have never needed.
I view old emails as all of the browser tabs I have open right now: I feel attached to them and avoid closing them. But if they all disappear I'll be fine. I'm not wasting energy backing them up.
Do you backup your Emails?
Yes. I have a combination of self hosted email and a vendor email provider. I access both with IMAPS. I pull down the emails with Thunderbird and never leave them on the server. I back up the local Thunderbird data folder locally then back it up to multiple encrypted SSD/NVME.
Why don't you leave your mail on your own server? I self-host (on the server-under-the-stairs here on the farm) as well and always leave mail on the server so I can access it from any client anywhere in the world. Assuming you're actually self-hosting as in 'running your own services on your own hardware at your own location' pulling it from the server doesn't make sense since you're just moving data from one of your drives to another. If you're 'self-hosting' on a VPS I can see why you'd do that but I don't consider this to be 'total' self-hosting since you're using hardware outside of your own control.
My preference. I don't really go anywhere so around the world is not required for me. It also means that when either my self hosted solution or the commercial solution gets popped there is nothing to leak. My emails are not exposed to the internet 99.99% of the time.
Can you search through old emails easily?
If I import the folder into Thunderbird, yes. The search in Thunderbird is a subjective topic but it works for me. There are probably other tools to search through them but I have never tried to find one.
I wonder if there’s an app that can enable this kind of backup, which when restored or searched can search through all those snapshots and just one and can tell me where that certain email is so that I can just restore only that snapshot.
https://notmuchmail.org/ is an excellent way to search through gigabytes of email extremely quickly. It’s faster than Gmail, Outlook or any other webmail provider.
I can probably 'rgrep' my Archives folder (I'm using the maildir format, each email a text file).
I have email going back more than 25 years from various software; in most cases an export dump when I stopped using, say, Pegasus Mail or Eudora. Since I started with Thunderbird I've created a new profile each year, and have profiles going back to 2008.
Earlier this year I imported all this historical data into the Mail Server on our Synology NAS. I don't use this server for current mail, but I can now search these old messages easily from within Thunderbird.
One thing I notice is how much more personal email there was before we started using chat services for group conversations. Nonetheless it's great being able to go back and read correspondence with, say, my dad who died in 2010.
I’m right there with you.
This thread suggests that there are many who view email as something to curate in the same way you would a letter box or a physical office inbox.
I find it more freeing to disregard whether I’m saving dust. I can purge that dust en masse any time I want without having to stay on top of it. The other side of that is that I can also locate anything important from one place with just two pieces of information, one of them being “it was an email”.
On my Mac I accomplish email backups using the Mail app and Chronosync. I have filtering rules to automatically file messages into multiple IMAP mailboxes based on sender or subject. I periodically manually move messages older than a week to local mailboxes to keep my iCloud storage from getting large enough to require paying for additional storage. The Mail app can also export those local mailboxes to a folder which I then back up using Chronosync so I have local copies on multiple drives.
The whole process only takes about 5 minutes once every week or two. It gives me easy access to over 20 years of email which is nice to have even though I only search the old ones a few times a year.
Get locked out of your account? Hope you had a backup. I lost access to an account can’t get back in.
Can’t get into your mail cuz of internet issues. Hope you had a backup
Too much space used? Make a backup, cleanup and continue
Thunderbird and Mail service exports for me.
You're right not to trust cloud providers with your life's history. They are not trustworthy.
Thunderbird is a great way to back up emails. You can tell it to store everything locally. It stores everything in handy sqlite tables under the covers.
Seeing people in this thread say that they delete all their emails is super weird. I'd like to know their job titles, ages, etc. Do they not use email to converse with their lawyers, business associates, etc.?
Email is not secure and therefore not suitable for serious business use. Even if you encrypt your message in transit (which is hard when mailing third parties), your stored message is one missed security patch away from snooping eyes.
Email is for casual use only.
I’m a 40 years old programmer. I’ve never kept emails for more than a week. Everything is deleted once I have saved the attachments or replied to the mail.
Also I never had a lawyer or an associate, I only have friends.
I can’t delete or encrypt emails for the same reason. It’s a legal document, though I don’t know how it’s treated by the courts. Definitely it’s evidence.
A law/regulation that prevents you from even encrypting your backups? That sounds pretty weird.
No, I meant, the evidence is lost when using end to end encrypted email (like with PGP or proton): the server as a trusted third party doesn’t have access to the email in plaintext to confirm the message or document.
Absent specific archiving solutions, most providers don’t and wont care about third party legal disputes.
I’ve never heard this argument before. It’s routine in many regulated industries with de facto or legally required preservation mandates to encrypt email, often with S/MIME or sometimes something proprietary.
What's your favorite email search tool?
Ideally something that runs against a local archive on PC, and has a mobile app. With instant results.
I used to use Lookout (for Outlook) and have never managed to find something as fast, simple and reliable.
I use mblaze, I find it especially useful for large mailing lists like the linux kernel.
I can recommend https://github.com/joeyates/imap-backup for backing up gmail.
I've been using imap-backup for years on both my personal and work machines. IIRC I had to set up a Gmail app password to get it to work, but that landscape may have changed since then.
31 GB written and still going strong.
However, this does not function when a security key is activated.
I use https://github.com/GAM-team/got-your-back for gmail
I use fetchmail, I've used OfflineIMAP before and MailDrop locally to autorefile anything since I use my mails locally (notmuch-emacs), in the past I've used OfflineIMAP also to restore mails on a third party IMAPs.
Essentially they are just a set of maildirs on my homeserver backed up with the rest, muchsync over SSH allow to have messages on multiple computers (actually just desktop and laptop).
I use mpop with a local mailbox file ~/Mail/inbox and mutt as the mua (so no fancy archive hierarchy), cleared annually and copied to a file named like ~/Mail/inbox.2024, searchable with grep, backed up remotely in encrypted tar files on backblaze with passwords committed to memory, locally on a zfs file server, and annually on dvd for at least a decade in case all my hardware gets fried.
I use synology mail server to pull emails from cloud provider. I can use it as backup interface and if needed can switch my email domain to local without losing anything and without need to restore mails from backup.
I use Thunderbird and just back up my home directory.
I was doing this. But then I started noticing that some emails I had read were not being stored locally in the Thunderbird IMAP folder structure on my computer, even though Thunderbird had fetched them and shown them to me. I couldn't figure out any rhyme or reason to this, so I started using mbsync on the commandline to properly mirror my IMAP account to a local store that then gets backed up with my home directory.
I backup my mail server and with that all email every day, keeping archives for the last 7 days, the last 4 weeks and the last 3 months. I have an email archive going back to 1992, from 1997 it is directly available in MUAs, older messages are in offline archives.
I just use thunderbird with imap and copy the whole thunderbird folder to a new device when I want to use email there.
That being said, I started using email about twenty years ago and I probably have looked at past emails maybe a dozen times. The vast majority I never look at again.
Personal no. But professional, yes.
Whenever I change jobs I always take a copy of all my emails and keep it in offline storage.
Arguably not always legal/compliant. But both for CYA, learned history, relevant contacts etc. it has had use for me in multiple occasions in the past
If it's not legal, could it ever be used in a situation CYA is required?
"However, referring to the scope of work in your email dated 20-SEP-2021…"
I mean, in that case couldn't they say "there was no email discussing scope of work on that date"? I guess maybe it works anyways...
It's seriously taking it to next level and most people you interact with aren't malicious. Also such emails tend to have other people on CC.
I download all of my mail from POP3 accounts, then I backup my Thunderbird directory: one copy on my home server, one encrypted copy to a VPS in another country. I use duplicity for that. One full backup every 2 weeks, then incrementals.
Yes. In Gmail I set up a label for the year which is added to every email. Then I download them periodically because Takeout lets you choose just a label, so I only end up with a years worth max. I have emails going back to the 90s. Why not?
Yes. My devices retrieve email from a cheap VPS where cron periodically runs tar gz on $HOME/Maildir. scp then retrieves the backup. I use GMail for SMTP but all outgoing mail is copied to the IMAP server on the VPS.
I am using neomutt + mbsync that gets rsynced to my backup server once a week.
I have about 11 years archived as annual maildir tarballs. I archive using thunderbird and export them when I remember to. I don't know why I bother, I almost never need anything from them.
I've never viewed emails as important enough to keep in general, though I pay for my email sevice. Anything important I'll back up individually. But mostly I delete straight away.
I self-host my email, and I backup my server. My Gmail account, which I seldom use, gets redirected to an address at my own server, so it automatically gets backed up as well.
What is the best format for email back up, so that it can be read decades in the future?
I once had a four year overdue reminder to setup gmail backups, still haven't set it up.
yes, I run a local mail server on my nas/server, and sync from online accounts every few months, and also keep local backup via syncing with Thunderbird
I use Gmail and just download a Google takeout every 3 months
Um, I back up EVERYTHING, like any sane person? And, like any other sane person, I self-host (on a server that's obviously backed up), and keep at least one copy of my email on my main working computer (which is also obviously backed up), so I don't have to do anything email-specific to make that happen.
Yeah. Email server's whole filesystem gets synced to my backup server, and the mbox files or whatever are in there somewhere.
> what tools are you using to backup your emails?
rsync and btrfs snapshots
> Is there a service which takes care of this automatically?
Cron?