Loading ...
Sorry, an error occurred while loading the content.
 

Re: postfix clustering

Expand Messages
  • Reinaldo de Carvalho
    ... Cyrus 2.3 already replication support. Join cyrus mailling list. This is off-topic here. -- Reinaldo de Carvalho http://korreio.sf.net
    Message 1 of 18 , Oct 31, 2010
      On Sun, Oct 31, 2010 at 2:13 AM, Stan Hoeppner <stan@...> wrote:
      Reinaldo de Carvalho put forth on 10/30/2010 3:39 PM:

      > From Cyrus mailling list:
      >
      > "Now that Cyrus 2.4 has been released with a lot of the groundwork for
      > bandwidth efficient replication in place, Max is going to be working
      > on improving the management tools and monitoring of the replication
      > process.  Our goal is to support master-master replication with safe
      > conflict resolution, and multiple replication topologies including
      > replication with more than two copies.  This will allow efficient
      > failover within a single datacentre as well as geographically
      > distant close-to-real-time disaster recovery."

      Interesting.  This is a _huge_ leap in capability for any IMAP server
      I'm aware of.  Yet, entering either "replication" or "cluster" in the
      Cyrus home page search box returns zero results.  When will these
      features be released as production ready?  Right now it appears they are
      vaporware.


      Cyrus 2.3 already replication support. Join cyrus mailling list. This is off-topic here.


      --
      Reinaldo de Carvalho
      http://korreio.sf.net
      http://python-cyrus.sf.net

      "While not fully understand a software, don't try to adapt this software to the way you work, but rather yourself to the way the software works" (myself)

    • Peter
      Hi Stan, ... You have a great suggestion assuming the data center functions well. the data center primary site failure means that the data center itself
      Message 2 of 18 , Nov 1, 2010
        Hi Stan,

        > 1.  What are your specific failure concerns with your
        > primary site?
        > Network failure?  Host failure?  Storage hardware
        > failure?

        You have a great suggestion assuming the data center functions well.

        the data center primary site failure means that the data center
        itself failed, meaning the host machine/storage/network are all in-accessible.






        > Maybe a good question for you to ask of the members of this
        > list at this
        > point is:
        >
        > How many OPs here run with a multi site IMAP cluster setup
        > with a
        > physically distributed mail store, either via replication
        > or a cluster
        > filesystem over a wide area network?
        >

        That is a good question. It is something I am looking for.

        however, if not going for expensive cluster filesystem over a wide area network,
        one simple way is to rsync every 5 minutes to copy over a backup serer in another data center and a quick
        dns change if the primary data center failed. The TTL in DNS settings can be 5 minutes.


        Peter
      • Stan Hoeppner
        ... You seem to be looking at this from a macro point of view. For an entire datacenter to fail you re looking at something like a natural disaster
        Message 3 of 18 , Nov 1, 2010
          Peter put forth on 11/1/2010 6:51 PM:
          > Hi Stan,
          >
          >> 1. What are your specific failure concerns with your
          >> primary site?
          >> Network failure? Host failure? Storage hardware
          >> failure?
          >
          > You have a great suggestion assuming the data center functions well.
          >
          > the data center primary site failure means that the data center
          > itself failed, meaning the host machine/storage/network are all in-accessible.

          You seem to be looking at this from a macro point of view. For an
          entire datacenter to "fail" you're looking at something like a natural
          disaster (hurricane, earthquake, tornado, flood, lightning) destroying
          the facility, or all power and comm lines into it. The probability of
          these things is very low, assuming the datacenter was located, designed,
          and constructed properly.

          I suggest you need to look at disaster avoidance and recovery from a
          micro point of view. Failures at the micro level are far more common,
          and less expensive to architect around and recover from.

          >> Maybe a good question for you to ask of the members of this
          >> list at this
          >> point is:
          >>
          >> How many OPs here run with a multi site IMAP cluster setup
          >> with a
          >> physically distributed mail store, either via replication
          >> or a cluster
          >> filesystem over a wide area network?
          >>
          >
          > That is a good question. It is something I am looking for.

          Then simply start a new thread and ask the question of the members of
          this list. The answers you get should be very instructive. My guess is
          that very very few OPs here are doing what you're attempting to do, and
          yet they have great reliability. I may be all wrong. Ask the list and
          get consensus on how others approach disaster avoidance and recover of
          their IMAP stores.

          > however, if not going for expensive cluster filesystem over a wide area network,
          > one simple way is to rsync every 5 minutes to copy over a backup serer in another data center and a quick
          > dns change if the primary data center failed. The TTL in DNS settings can be 5 minutes.

          This is not automatic fail over. If you're going to bother with a
          remote hot site, you should have automatic fail over of the mailbox server.

          Again, I ask you, is your primary site so prone to failure that you
          _need_ a remote site? Let me guess: You have already sold your
          superiors on the idea of a remote hot site, and now you're trying to
          figure out how to implement it?

          If this is the case I'm wasting my breath and you are wasting the lists'
          time. A technical engineer identifies a problem and then finds and
          implements the proper solution. He doesn't pick a solution from a
          magazine article or blog that sounds neat, to a problem that may or may
          not exist at his organization, and then drive to implement it due to
          personal desires instead of operational needs.

          A hot backup site is actually relatively rare. Few organizations
          implement this strategy. In the vast majority of cases the cost of such
          an architecture (hardware, comms links, testing, admin time, etc)
          outweighs the benefit due to reliability of the primary site and thus
          the fact the hot site is rarely if ever used.

          Ask the members of this list how many do IMAP store replication/fail
          over to a remote site.

          --
          Stan
        • Mark Scholten
          ... It sounds interesting, however from a system administrator point of view it shouldn t be really difficult to create with the current tools. With some
          Message 4 of 18 , Nov 2, 2010
            > -----Original Message-----
            > From: owner-postfix-users@... [mailto:owner-postfix-
            > users@...] On Behalf Of Stan Hoeppner
            > Sent: Sunday, October 31, 2010 6:14 AM
            > To: postfix-users@...
            > Subject: Re: postfix clustering
            >
            > Reinaldo de Carvalho put forth on 10/30/2010 3:39 PM:
            >
            > > From Cyrus mailling list:
            > >
            > > "Now that Cyrus 2.4 has been released with a lot of the groundwork
            > for
            > > bandwidth efficient replication in place, Max is going to be working
            > > on improving the management tools and monitoring of the replication
            > > process. Our goal is to support master-master replication with safe
            > > conflict resolution, and multiple replication topologies including
            > > replication with more than two copies. This will allow efficient
            > > failover within a single datacentre as well as geographically
            > > distant close-to-real-time disaster recovery."
            >
            > Interesting. This is a _huge_ leap in capability for any IMAP server
            > I'm aware of. Yet, entering either "replication" or "cluster" in the
            > Cyrus home page search box returns zero results. When will these
            > features be released as production ready? Right now it appears they
            > are
            > vaporware.

            It sounds interesting, however from a system administrator point of view it
            shouldn't be really difficult to create with the current tools. With some
            custom scripts I think it should be possible to do it with current tools,
            the most difficult part is messages that get deleted from the file system
            (pop3/imap) I guess.

            I can set it up later this year in a test environment and publish my
            findings about it. If this is interesting for others to know please mention
            it and I'll test it and publish it.

            Regards, Mark
          • Peter
            - ... I am a little surprised about your common sense for the data protection. obviously, you never heard of data center on fire that wipes out all servers or
            Message 5 of 18 , Nov 2, 2010
              -
              > > Hi Stan,
              > >
              > >> 1.  What are your specific failure concerns
              > with your
              > >> primary site?
              > >> Network failure?  Host failure?  Storage
              > hardware
              > >> failure?
              > >
              > > You have a great suggestion assuming the data center
              > functions well.
              > >
              > > the data center primary site failure means that the
              > data center
              > > itself failed, meaning the host
              > machine/storage/network are all in-accessible.
              >
              > You seem to be looking at this from a macro point of
              > view.  For an
              > entire datacenter to "fail" you're looking at something
              > like a natural
              > disaster (hurricane, earthquake, tornado, flood, lightning)
              > destroying
              > the facility, or all power and comm lines into it. 
              > The probability of
              > these things is very low, assuming the datacenter was
              > located, designed,
              > and constructed properly.

              I am a little surprised about your common sense for the data protection.

              obviously, you never heard of data center on fire that wipes out
              all servers or all servers were down for 3 or more days.

              Just a quick google search and there are so many data centers related
              issue reported in the past.

              http://www.thetechherald.com/article.php/200823/1124/The-Planet-Outage-%E2%80%93-H1-Datacenter-suffers-fire-and-explosion-Update-1

              http://www.techflash.com/seattle/2009/07/Why_the_Seattle_data_center_fire_caught_companies_unprepared49978502.html

              http://www.datacenterknowledge.com/archives/2008/03/31/fire-destroys-wisconsin-data-center/

              http://www.intology.com/computers-internet/us-data-center-catches-fire-9000-servers-down/

              http://www.prisontalk.com/forums/showthread.php?t=340505

              anyway, thanks for your other suggestions.

              Peter
            • Peter
              ... it is great if you can create a blog to share. Thanks, Peter
              Message 6 of 18 , Nov 2, 2010
                > > Interesting.  This is a _huge_ leap in capability
                > for any IMAP server
                > > I'm aware of.  Yet, entering either "replication"
                > or "cluster" in the
                > > Cyrus home page search box returns zero results. 
                > When will these
                > > features be released as production ready?  Right
                > now it appears they
                > > are
                > > vaporware.
                >
                > It sounds interesting, however from a system administrator
                > point of view it
                > shouldn't be really difficult to create with the current
                > tools. With some
                > custom scripts I think it should be possible to do it with
                > current tools,
                > the most difficult part is messages that get deleted from
                > the file system
                > (pop3/imap) I guess.
                >
                > I can set it up later this year in a test environment and
                > publish my
                > findings about it. If this is interesting for others to
                > know please mention
                > it and I'll test it and publish it.

                it is great if you can create a blog to share.

                Thanks,

                Peter
              Your message has been successfully submitted and would be delivered to recipients shortly.