Skip to content
Migrating mailboxes with imapsync — a practical runbook
Migration

Migrating mailboxes with imapsync — a practical runbook

Mailbox migration is one of those jobs that is either uneventful or a disaster, with very little in between. The difference is almost entirely down to sequencing rather than tooling — the tooling has been solved for years.

imapsync is the tool. It is a Perl script that connects to two IMAP servers as a client, compares what is in each, and copies across whatever is missing, preserving folder structure, flags and internal dates. It is idempotent, which is the property that makes the whole approach viable: you can run it repeatedly and it only transfers what has changed.

This article is the runbook we use, including the flags that actually matter and the ones that quietly cause data loss.

What imapsync will and will not move

It only moves what IMAP exposes: folders, messages, flags and internal dates. Calendars, contacts, server-side filter rules, autoresponders, aliases and forwarders are not IMAP objects and will not come across. Those have to be migrated separately, and forgetting them is the most common cause of post-migration complaints.

What you gain over a control panel’s built-in importer is that it works between any two IMAP servers, it can be run repeatedly, and it produces a per-mailbox log you can point at when someone says a message is missing.

Installation

Debian and Ubuntu package it:

sudo apt install imapsync

The packaged version is often somewhat behind. For a large migration it is worth running a current release from the project directly, because IMAP server quirks — and the workarounds for them — get added regularly.

Run it from a machine with good connectivity to both ends. A small VPS is ideal; a laptop on a domestic connection that may sleep mid-transfer is not.

The basic invocation

imapsync \
  --host1 imap.oldprovider.com --port1 993 --ssl1 \
  --user1 alice@example.com --password1 'OldPassword' \
  --host2 mail.newprovider.net --port2 993 --ssl2 \
  --user2 alice@example.com --password2 'NewPassword' \
  --automap

--ssl1 and --ssl2 select implicit TLS, which pairs with port 993. If a server only offers STARTTLS on port 143, use --tls1 or --tls2 with --port 143 instead. Do not run without one or the other — you would be sending credentials and every message in plaintext.

--automap is the flag that does the most work. Special folders have different names on different servers: Sent, Sent Items, Sent Messages, INBOX.Sent. --automap recognises the standard special-use folders and maps them to their equivalents at the destination, rather than creating a duplicate. Without it, users end up with two sent-mail folders and no idea which is current.

Dry run first, always

imapsync ... --dry --justfolders

--dry performs the whole comparison and reports what it would transfer without writing anything. --justfolders restricts it to folder structure, which is the fastest way to see whether your mapping is right before you commit to moving several gigabytes.

Read the folder mapping table it prints. This is where you will spot a source server using a folder separator of . while the destination uses /, or a namespace prefix such as INBOX. that needs handling. imapsync detects both automatically in most cases, but a dry run is where you confirm it rather than assume it.

Then dry-run the full sync and look at the message counts per folder. If the destination is not empty, this tells you what is already there.

Flags that matter in practice

--exclude takes a regular expression matched against folder names. Two folders are usually worth excluding:

--exclude '^Trash$' --exclude '^Junk$'

There is rarely value in migrating deleted mail or a spam folder, and on a large mailbox they can account for a substantial share of the total. Note the regex is anchored — without the anchors you would also exclude anything containing "Trash" as a substring.

--maxage limits transfer to messages newer than a given number of days. Useful for a first pass on a very large mailbox where you want the recent mail available immediately and the archive backfilled afterwards.

--useuid changes the message-matching strategy to use UIDs rather than header-derived identifiers. Faster on large mailboxes, but only correct when the destination is a previous target of the same sync. Do not reach for it on a first run.

--addheader writes a header into transferred messages recording the sync. Occasionally handy for forensics; adds size.

--dry — worth repeating. Every non-trivial change to your invocation should be dry-run before it is run.

The flag to be careful with is --delete2, which removes messages at the destination that no longer exist at the source. It exists to make repeated syncs converge exactly. It is also the flag that destroys mail if you ever run a sync in the wrong direction. Our practice is not to use it at all during a migration, and to reconcile any duplicates manually afterwards.

Handling many mailboxes

imapsync ships with the ability to read credentials from a file, which is how you drive a migration of more than a handful of accounts. A simple CSV of user1;password1;user2;password2 per line, one mailbox per row, driven by a shell loop:

#!/usr/bin/env bash
set -euo pipefail

while IFS=';' read -r u1 p1 u2 p2; do
  echo "=== ${u1} -> ${u2} ==="
  imapsync \
    --host1 imap.oldprovider.com --port1 993 --ssl1 \
    --user1 "$u1" --password1 "$p1" \
    --host2 mail.newprovider.net --port2 993 --ssl2 \
    --user2 "$u2" --password2 "$p2" \
    --automap \
    --exclude '^Trash$' --exclude '^Junk$' \
    --logfile "logs/${u1}.log" \
    || echo "FAILED: ${u1}" >> logs/failures.txt
done < mailboxes.csv

set -euo pipefail with an explicit || echo on the imapsync line means one failing mailbox is recorded and the loop continues, rather than the whole run stopping at account seventeen of two hundred.

Two operational notes. That credentials file is extremely sensitive — chmod 600, keep it off shared storage, and delete it when the migration completes. And run mailboxes sequentially rather than in parallel unless you have confirmed the source server will tolerate concurrency; several providers rate-limit IMAP connections per account or per IP, and hitting that limit produces failures that look like data problems.

The admin-credential shortcut

If either provider supports IMAP master or admin authentication, use it. It lets you authenticate as an administrator and act on any mailbox, using --authuser1 (or --authuser2) alongside --user1. That removes the need to collect or reset every user’s password, which is usually the single most painful part of a migration. Ask both providers before you start; it is frequently available and rarely advertised.

Sequencing the cutover

This is the part that determines whether the migration is uneventful.

Days before. Create every mailbox at the destination. Run the first full sync. This is the long one — for a few hundred gigabytes it may run overnight, or across several nights. Nothing has changed for users yet; mail is still flowing to the old server.

Reduce the TTL. At least 24 hours before cutover, drop the TTL on the MX records to 300 seconds. This has to happen far enough in advance that the old TTL has expired everywhere, or the reduction itself will not have propagated when you need it.

Migrate the non-IMAP items. Aliases, forwarders, autoresponders, distribution lists and server-side filters. These are usually configured by hand at the destination, and this is the step most likely to be forgotten because nothing in the sync log mentions them.

Cutover. Change the MX records to the new provider. Mail begins arriving at the destination within the TTL window. Leave the old mailboxes running and reachable — do not close the account.

Delta sync. Run imapsync again for every mailbox. This picks up everything that arrived at the old server between the first sync and the MX change, plus anything the user has read, flagged or filed in the meantime. It is fast, because almost everything is already there.

Keep syncing. Run the delta again after 24 hours and again after 72. Some senders cache MX records well beyond the stated TTL, and a small trickle of mail will keep arriving at the old server for days.

Decommission. Only after a week of clean delta runs with nothing new to transfer should the old service be cancelled. This is also when the credentials file gets deleted.

What to check before declaring it done

Per mailbox, compare message counts folder by folder. imapsync prints these at the end of every run; the line to look for reports the total messages transferred and the total already present at the destination, and on a converged mailbox the transferred count should be zero.

Then check the things the counts do not cover: that flags and read/unread state look right, that the sent folder is the real sent folder rather than a duplicate, that replies actually thread, that autoresponders behave, and that a message sent to each alias arrives where it should.

Finally, confirm the new server’s outbound path is correct — PTR record, SPF, DKIM signing — before users start sending in volume. A migration that moves the mail perfectly but sends it from an IP with no reverse DNS has simply exchanged one problem for a worse one.


FXRM migrates mailboxes onto our UK infrastructure as part of onboarding, including aliases, forwarders and filters. Talk to us about a migration →

CategoriesMigration
Avatar photo

Thomas Maynard

Leave a comment

Your email address will not be published. Required fields are marked *