Loading ...
Sorry, an error occurred while loading the content.

Re: Verify cache dump

Expand Messages
  • Nikolaos Milas
    ... Thanks Noel, It worked fine in the end. Interestingly, at first I tried it but it didn t seem to produce any output at all (I had to break with Ctrl-C): #
    Message 1 of 11 , Jan 23, 2013
    • 0 Attachment
      On 23/1/2013 2:24 μμ, Noel Jones wrote:

      > man postmap the -s and -q options may be of particular interest.

      Thanks Noel,

      It worked fine in the end.

      Interestingly, at first I tried it but it didn't seem to produce any
      output at all (I had to break with Ctrl-C):

      # postmap -s /var/lib/postfix/verify_cache
      ^C
      # postmap -s btree:/var/lib/postfix/verify_cache
      ^C
      # postmap -q user@... /var/lib/postfix/verify_cache
      ^C
      # postmap -q user@... btree:/var/lib/postfix/verify_cache
      ^C

      Finally it worked as follows:

      # postmap -s btree:/var/lib/postfix/verify_cache
      # postmap -q user@... btree:/var/lib/postfix/verify_cache

      (I don't know if it is related, but it worked after I once tried:
      # postmap -vs btree:/var/lib/postfix/verify_cache
      i.e. with the -v option)

      Thanks again,
      Nick
    • Wietse Venema
      ... It s almost guaranteed not to work. To prevent multiple writers from corrupting the database, verify(8) maintains an exclusive lock on the database. Your
      Message 2 of 11 , Jan 23, 2013
      • 0 Attachment
        Nikolaos Milas:
        > On 23/1/2013 2:24 ??, Noel Jones wrote:
        >
        > > man postmap the -s and -q options may be of particular interest.
        >
        > Thanks Noel,
        >
        > It worked fine in the end.

        It's almost guaranteed not to work. To prevent multiple writers from
        corrupting the database, verify(8) maintains an exclusive lock on
        the database. Your client will hang until verify(8) terminates.

        Instead you can try to query a copy of the database, or use a client
        that ignores locks, but neither of these have correctness guarantees.

        Wietse
      • Nikolaos Milas
        ... I continue to have this problem (no output from verify_cache querying). For example, it occurs on two servers with 2.9.4 (CentOS 6.3 x86_64). How can I
        Message 3 of 11 , Apr 4, 2013
        • 0 Attachment
          On 23/1/2013 3:43 μμ, Nikolaos Milas wrote:

          > Interestingly, at first I tried it but it didn't seem to produce any
          > output at all (I had to break with Ctrl-C):
          >
          > # postmap -s /var/lib/postfix/verify_cache
          > ^C
          > # postmap -s btree:/var/lib/postfix/verify_cache
          > ^C
          > # postmap -q user@... /var/lib/postfix/verify_cache
          > ^C
          > # postmap -q user@... btree:/var/lib/postfix/verify_cache
          > ^C
          >
          > Finally it worked as follows:
          >
          > # postmap -s btree:/var/lib/postfix/verify_cache
          > # postmap -q user@... btree:/var/lib/postfix/verify_cache

          I continue to have this problem (no output from verify_cache querying).

          For example, it occurs on two servers with 2.9.4 (CentOS 6.3 x86_64).

          How can I troubleshoot it to see why it happens?

          If I run verbose:

          # postmap -vs btree:/var/lib/postfix/verify_cache
          postmap: name_mask: ipv4
          postmap: name_mask: ipv6
          postmap: inet_addr_local: configured 2 IPv4 addresses
          postmap: inet_addr_local: configured 3 IPv6 addresses
          postmap: Compiled against Berkeley DB: 4.7.25?
          postmap: Run-time linked against Berkeley DB: 4.7.25?
          ^C

          (It hungs, so I Ctrl-C.)

          Thanks,
          Nick
        • Wietse Venema
          ... I have mentioned before that the verify(8) daemon locks the Berkeley DB file exclusively. Concurrent read/write access is not safe with these files. The
          Message 4 of 11 , Apr 4, 2013
          • 0 Attachment
            Nikolaos Milas:
            > On 23/1/2013 3:43 ??, Nikolaos Milas wrote:
            >
            > > Interestingly, at first I tried it but it didn't seem to produce any
            > > output at all (I had to break with Ctrl-C):
            > >
            > > # postmap -s /var/lib/postfix/verify_cache
            > > ^C
            > > # postmap -s btree:/var/lib/postfix/verify_cache
            > > ^C
            > > # postmap -q user@... /var/lib/postfix/verify_cache
            > > ^C
            > > # postmap -q user@... btree:/var/lib/postfix/verify_cache
            > > ^C

            I have mentioned before that the verify(8) daemon locks the Berkeley
            DB file exclusively. Concurrent read/write access is not safe with
            these files. The exclusive lock guarantees that verify(8) will not
            crash or do silly things when some other Postfix program changes
            the database. The lock also ensures that other Postfix programs
            will not crash or do silly things when verify(8) changes the database.

            You could do this:

            # cp /var/lib/postfix/verify_cache.db tempfile.db
            # postmap -s btree:tempfile.db

            Cross your fingers, and if it breaks, repeat the two commands.

            The new LMDB database type is safe for concurrent read/write access;
            however this currently has an open problem when creating a very
            large database file from scratch (as with the "postmap" command).
            That is not an issue with verify(8) which makes only incremental
            updates.

            Of course there is no guarantee of what you will get when you dump
            a large database while it is being modified, even when the database
            file is safe for concurrent read/write access. But at least the
            programs will not crash or do silly things.

            Wietse
          • Nikolaos Milas
            ... Thanks Wietse, it works fine this way: # postmap -s btree:verify_cache_copy By the way, is there a way/command to lengthen the validity of all current
            Message 5 of 11 , Apr 4, 2013
            • 0 Attachment
              On 4/4/2013 3:38 μμ, Wietse Venema wrote:

              > # cp /var/lib/postfix/verify_cache.db tempfile.db
              > # postmap -s btree:tempfile.db

              Thanks Wietse, it works fine this way:

              # postmap -s btree:verify_cache_copy

              By the way, is there a way/command to lengthen the validity of all
              current entries in the verify cache?

              As currently set, our gateway mail server(s) queries the final
              destination (internal) server to verify recipients using:

              relay_recipient_maps = <empty>

              In case of planned downtime of internal (final destination) mail server,
              I would like to avoid expiration of the gateway server's verify cache
              during the downtime.

              Thanks again,
              Nick
            • Wietse Venema
              ... Configure a larger address_verify_positive_expire_time? The default is 31 days. Note that Postfix does not wait until cached information expires. By
              Message 6 of 11 , Apr 4, 2013
              • 0 Attachment
                Nikolaos Milas:
                > On 4/4/2013 3:38 ??, Wietse Venema wrote:
                >
                > > # cp /var/lib/postfix/verify_cache.db tempfile.db
                > > # postmap -s btree:tempfile.db
                >
                > Thanks Wietse, it works fine this way:
                >
                > # postmap -s btree:verify_cache_copy
                >
                > By the way, is there a way/command to lengthen the validity of all
                > current entries in the verify cache?

                Configure a larger address_verify_positive_expire_time?
                The default is 31 days.

                Note that Postfix does not wait until cached information expires.
                By default it refreshes cached records after 7 days (the refresh
                happens after the user receives email).

                Therefore. cached information about active accounts will be valid
                for 24..31 days.

                > As currently set, our gateway mail server(s) queries the final
                > destination (internal) server to verify recipients using:
                >
                > relay_recipient_maps = <empty>
                >
                > In case of planned downtime of internal (final destination) mail server,
                > I would like to avoid expiration of the gateway server's verify cache
                > during the downtime.

                How long is your expected downtime? Even it someone receives only
                one email per week, they should expire in 24 days.

                Wietse
              • Nikolaos Milas
                ... It will be around 5-6 hours. I would like to avoid the gateway server bouncing mails due to not being able to verify the recipients. Ideally, mails should
                Message 7 of 11 , Apr 4, 2013
                • 0 Attachment
                  On 4/4/2013 5:42 μμ, Wietse Venema wrote:

                  > How long is your expected downtime? Even it someone receives only
                  > one email per week, they should expire in 24 days.

                  It will be around 5-6 hours. I would like to avoid the gateway server
                  bouncing mails due to not being able to verify the recipients.

                  Ideally, mails should be accepted by the gateway server based on its
                  verify cache, and then kept in the queue during the main server downtime
                  period.

                  One solution would be to delete the verify_cache.db file NOW and thus
                  force the creation of a new one, so that any positive cached address
                  will be added and be valid for at least 7 days (provided that the
                  downtime will occur in the next 7 days); the only drawback - I think -
                  is that some addresses may not be used until the downtime occurs, so
                  they will be missing from the verify cache when the downtime occurs.

                  One way to solve this would be to send one mail to all our users, so
                  that they will be added to the verify cache. However there are multiple
                  addresses per user (due to aliases, so this complicates things a bit).

                  If we use a group alias like all-staff@... as a recipient
                  address, does that cause all of the true recipient addresses (that the
                  alias resolves to) to be added to the verify cache as well?

                  Or, can we manually add addresses to the verify cache?

                  Alternatively, we could switch to ldap-based verification during this
                  period - but I would like to avoid doing serious changes under pressure.

                  Thanks,
                  Nick
                • Wietse Venema
                  ... With the existing automatic cache refresh after 7 days, - ALL recipients who received their last email less than 1 week ago will remain cached for 24-31
                  Message 8 of 11 , Apr 4, 2013
                  • 0 Attachment
                    Nikolaos Milas:
                    > On 4/4/2013 5:42 ??, Wietse Venema wrote:
                    >
                    > > How long is your expected downtime? Even it someone receives only
                    > > one email per week, they should expire in 24 days.
                    >
                    > It will be around 5-6 hours. I would like to avoid the gateway server
                    > bouncing mails due to not being able to verify the recipients.

                    With the existing automatic cache refresh after 7 days,

                    - ALL recipients who received their last email less than 1 week ago will
                    remain cached for 24-31 days.

                    - ALL recipients who received their last email 1-2 weeks ago will remain
                    cached for 17-24 days.

                    - ALL recipients who received their last email 2-3 weeks ago will remain
                    cached for 10-17 days.

                    > One solution would be to delete the verify_cache.db file NOW and thus

                    NO, THAT WOULD BE AN ENORMOUS MISTAKE.

                    You weren't paying attention when i explained automatic cache refresh.

                    Wietse
                  • Nikolaos Milas
                    ... I admit I was confused with the initial explanation. Your latest clarifications make things much clearer to me. I understand that the gateway server should
                    Message 9 of 11 , Apr 4, 2013
                    • 0 Attachment
                      On 4/4/2013 11:06 μμ, Wietse Venema wrote:

                      > You weren't paying attention when i explained automatic cache refresh.

                      I admit I was confused with the initial explanation.

                      Your latest clarifications make things much clearer to me.

                      I understand that the gateway server should work fine with the current
                      (default) verify cache settings during the scheduled downtime.

                      Thanks,
                      Nick
                    Your message has been successfully submitted and would be delivered to recipients shortly.