Loading ...
Sorry, an error occurred while loading the content.

multi pronged upgrade/migration

Expand Messages
  • Stan Hoeppner
    This question touches on far more than Postfix, so I hope it doesn t offend anyone being slightly OT. A little background may help: I built my current Postfix
    Message 1 of 2 , Jan 1, 2010
    • 0 Attachment
      This question touches on far more than Postfix, so I hope it doesn't offend
      anyone being slightly OT.

      A little background may help:

      I built my current Postfix host in late 2005 as a single purpose
      firewall/gateway to sit in front of an old Ms Exch server, on which I've
      archived email since 2000. The Postfix firewall has functioned in this mode for
      4+ years. It's running on a Debian system up to date at 5.0.3 with all patches
      installed and a custom kernel, 2.6.31.1. I'm running the Debian Lenny Postfix
      2.5.5-1-1 package which I upgraded within the past few of months from 2.3.8
      during the upgrade to Lenny. I never configured local mail delivery on this
      host as I installed Postfix on build day in '05 and set it to relay only. I
      have no experience with standard *nix mailbox or maildir formats, but I've
      recently read up a little bit on the basics of both at the Dovecot website.

      What I want to do is:

      1. Switch Postfix from relay mode to local delivery
      2. Convert entries in relay_recipients to aliases pointing to a local Unix user
      3. Select an appropriate mailbox format, location, and filessytem
      4. Install an appropriate IMAP/POP server daemon package
      5. Migrate current emails and folders to the new Postfix/IMAP combo
      6. What have I overlooked?

      I know basically "how" to perform change [1] but the details depend on [2/3],
      which I need advice on. From what I've read on this list I think I'd like to
      use Dovecot for IMAP. What is the best way to reconfigure Postfix to drop mail
      in the best format and location for Dovecot (mbox vs maildir)? My current
      Postfix spool is on / which has ~30GB free, ext2 FS. If preferable I can create
      a new partition formatted with any appropriate filesystem, mount that and have
      Postfix/Dovecot dump new mail there, along with storing the IMAP mail and
      folders. I've already migrated most of the mail and folders from the old IMAP
      server to local Thunderbird folders on my PC, so migrating those to the new IMAP
      server should be fairly straight forward once I have the new IMAP server up and
      running.

      Of note, I have quite a few current IMAP folders with a thousand emails in each,
      and one folder with over 10,000 emails. Would mbox or maildir format be better
      here, and does this effect Postfix in any way, or only the IMAP server?

      Thanks for any/all advice, and again, sorry if some of this if OT. I know many
      here run Dovecot and other IMAP servers alongside Postfix so I figured you might
      have some pointers.

      Happy new year to all.

      --
      Stan
    • Barney Desmond
      I should warn beforehand that I m probably not giving this as much consideration as would be ideal - that d mean breaking out a pen and paper to collect my
      Message 2 of 2 , Jan 1, 2010
      • 0 Attachment
        I should warn beforehand that I'm probably not giving this as much
        consideration as would be ideal - that'd mean breaking out a pen and
        paper to collect my thoughts, which is a tough effort at this time of
        year :)

        2010/1/1 Stan Hoeppner <stan@...>:
        > What I want to do is:
        >
        > 1.  Switch Postfix from relay mode to local delivery
        > 2.  Convert entries in relay_recipients to aliases pointing to a local Unix user
        > 3.  Select an appropriate mailbox format, location, and filessytem
        > 4.  Install an appropriate IMAP/POP server daemon package
        > 5.  Migrate current emails and folders to the new Postfix/IMAP combo
        > 6.  What have I overlooked?

        > I know basically "how" to perform change [1] but the details depend on [2/3],
        > which I need advice on.

        Your list of steps looks pretty solid, and they should be mostly
        independent of each other. I don't have `postconf -n` (I'm sure you've
        posted one to the list in recent history though), but I'll assume a
        fairly simple and sane config with relay domains and recipients all
        setup and ready to go.

        The way we do things at work is to leave $mydestination at its default
        value (only the machine's hostname and localhost are considered to be
        local domains). For any "real" mail we're interested in, we then put
        everything into virtual_alias_{domains,maps} - we're lazy and use bare
        system usernames on the RHS of the maps for ease of administration.
        This way, legitimate addresses get rewritten to something@$myorigin
        (which defaults to $myhostname anyway, and is thus a local domain),
        and handled as a local delivery. For your conversion, I would change
        all your relay_domains to virtual_alias_domains, then add
        virtual_alias_maps entries as you see fit.

        N.B.:
        1. I'm ignoring how you choose to go about creating the local system users
        2. Things are a little messier if you only want to switch *some*
        addresses in a domain to local delivery instead of relaying, but it's
        doable.

        > From what I've read on this list I think I'd like to use Dovecot for IMAP.
        > What is the best way to reconfigure Postfix to drop mail
        > in the best format and location for Dovecot (mbox vs maildir)?
        > My current
        > Postfix spool is on / which has ~30GB free, ext2 FS. If preferable I can create
        > a new partition formatted with any appropriate filesystem, mount that and have
        > Postfix/Dovecot dump new mail there, along with storing the IMAP mail and
        > folders.

        I reckon you're almost in Holy War territory here. Like all good
        answers, "it depends", and there's no shortage of people telling you
        why their setup is best. A quick list of thoughts on filesystems and
        formats (not all necessarily accurate)
        * ext2 is stable, mature, supported everywhere, pretty good performance
        * ext3 adds journals, yay for integrity! just as stable, mature and
        supported as ext2. If you have last millenium-era hardware it might
        perform worse. *sob*
        * reiserfs, tail-packing, great for small files like maildir!
        ** (but it'll murder your data...)
        * XFS has great performance, especially for deleting lots of files.
        BTW I hope you have battery-backed write caches
        * ZFS is new and shiny and will bring about world peace; pity it'll
        never be in mainline, but hey, you can always use it with FUSE...
        * butterfs isn't stable yet, it might land face-down with all your data
        * mbox is historically crap for scaled performance due to its monolithic nature
        * Maildir might solve world hunger, but thousands of small files could
        get inefficient
        * store mail in /var/mail/$username - this should be the default for
        mboxes anyway
        * store mail in ~/Maildir - a popular choice for maildirs
        * If these users are only being used for mail, maybe you want a
        separate mail partition after all; it could make
        administration/backups/policy/whatever easier for you. It's easy
        enough to specify your own delivery location

        I think you can see where I'm going with this. Go with what you're
        most comfortable with, then come back if you're actually having
        performance problems. I'm a sysadmin for a webhosting company, and we
        care mostly about stability (but not so much that we think it's okay
        that Redhat ships postfix 2.3). One way to answer this is to ask
        yourself, do YOU want to be the one who has to handle a 12hr repair
        job when something blows up on a Sunday because you went for a more
        "interesting" setup choice. Not saying it doesn't happen with stable
        choices either, it's about what you're confident with.

        As for mbox vs. maildir, it depends on usage. mbox is just fine for
        infrequently-used or infrequently-updated mailboxes. Note that postfix
        has a builtin mailbox_size_limit of 50MiB. If your mboxes are getting
        that big, it's a hint that you probably want to be using maildir.
        Maildir is great for busy mailboxes, but all those files have their
        own cost too.

        > Of note, I have quite a few current IMAP folders with a thousand emails in each,
        > and one folder with over 10,000 emails.  Would mbox or maildir format be better
        > here, and does this effect Postfix in any way, or only the IMAP server?

        That doesn't sound unreasonable for maildir, but it's too variable to
        give a firm answer either way. If you're unsure, I suspect most people
        will recommend maildir, whatever the shortcomings. I believe the scope
        of effect should be mostly on the IMAP server, but if the IMAP server
        has an mbox locked then postfix might block on the delivery (I'm not
        certain on this point though).

        One thing I'd consider is using Dovecot as your local delivery agent
        (it's imaginatively called "deliver"). It's flexible with respect to
        mailbox formats, and can update its own indexes while it makes the
        delivery. Dovecot is also a great IMAP server, you'll find plenty of
        supporters for it here.

        You mentioned migrating in emails and folders; this will be the most
        frustrating and time-consuming part of the whole plan, the earlier
        steps are probably only several hours of work. Seeing as you said this
        box has only been used for relaying before, I'm assuming you want to
        migrate everything from the Exchange server. Mail migrations are never
        fun, so I wish you good luck with that. Assuming you sort out the
        customer-facing side of things, you should be able to do it with a
        small downtime window. I'm thinking you could enable soft_bounce then
        flick your configuration from relay to local (once all the local
        accounts are created). imapsync can be used to shuffle mail to the
        postfix machine, then you fix up the networking once it's all done
        (DNS change, IP migration, or tell users to update their mail
        clients). Unless I've missed something, the worst case should be that
        users can't see new mail.
      Your message has been successfully submitted and would be delivered to recipients shortly.