Loading ...
Sorry, an error occurred while loading the content.

Verify cache dump

Expand Messages
  • Nikolaos Milas
    Hello, Is there a way to dump/view the verify cache so as to check which addresses are currently included therein? (I am using Postfix 2.9.4 on Centos 6.3
    Message 1 of 11 , Jan 23, 2013
    • 0 Attachment
      Hello,

      Is there a way to dump/view the verify cache so as to check which
      addresses are currently included therein?

      (I am using Postfix 2.9.4 on Centos 6.3 x86_64)

      # postconf | grep address_verify_map
      address_verify_map = btree:$data_directory/verify_cache

      # ls -la /var/lib/postfix/verify_cache.db
      -rw-r--r-- 1 postfix postfix 8192 Jan 23 11:43
      /var/lib/postfix/verify_cache.db

      Thanks,
      Nick
    • Noel Jones
      ... man postmap the -s and -q options may be of particular interest. -- Noel Jones
      Message 2 of 11 , Jan 23, 2013
      • 0 Attachment
        On 1/23/2013 6:11 AM, Nikolaos Milas wrote:
        > Hello,
        >
        > Is there a way to dump/view the verify cache so as to check which
        > addresses are currently included therein?
        >
        > (I am using Postfix 2.9.4 on Centos 6.3 x86_64)
        >
        > # postconf | grep address_verify_map
        > address_verify_map = btree:$data_directory/verify_cache
        >
        > # ls -la /var/lib/postfix/verify_cache.db
        > -rw-r--r-- 1 postfix postfix 8192 Jan 23 11:43
        > /var/lib/postfix/verify_cache.db
        >
        > Thanks,
        > Nick


        man postmap

        the -s and -q options may be of particular interest.




        -- Noel Jones
      • Nikolaos Milas
        ... Thanks Noel, It worked fine in the end. Interestingly, at first I tried it but it didn t seem to produce any output at all (I had to break with Ctrl-C): #
        Message 3 of 11 , Jan 23, 2013
        • 0 Attachment
          On 23/1/2013 2:24 μμ, Noel Jones wrote:

          > man postmap the -s and -q options may be of particular interest.

          Thanks Noel,

          It worked fine in the end.

          Interestingly, at first I tried it but it didn't seem to produce any
          output at all (I had to break with Ctrl-C):

          # postmap -s /var/lib/postfix/verify_cache
          ^C
          # postmap -s btree:/var/lib/postfix/verify_cache
          ^C
          # postmap -q user@... /var/lib/postfix/verify_cache
          ^C
          # postmap -q user@... btree:/var/lib/postfix/verify_cache
          ^C

          Finally it worked as follows:

          # postmap -s btree:/var/lib/postfix/verify_cache
          # postmap -q user@... btree:/var/lib/postfix/verify_cache

          (I don't know if it is related, but it worked after I once tried:
          # postmap -vs btree:/var/lib/postfix/verify_cache
          i.e. with the -v option)

          Thanks again,
          Nick
        • Wietse Venema
          ... It s almost guaranteed not to work. To prevent multiple writers from corrupting the database, verify(8) maintains an exclusive lock on the database. Your
          Message 4 of 11 , Jan 23, 2013
          • 0 Attachment
            Nikolaos Milas:
            > On 23/1/2013 2:24 ??, Noel Jones wrote:
            >
            > > man postmap the -s and -q options may be of particular interest.
            >
            > Thanks Noel,
            >
            > It worked fine in the end.

            It's almost guaranteed not to work. To prevent multiple writers from
            corrupting the database, verify(8) maintains an exclusive lock on
            the database. Your client will hang until verify(8) terminates.

            Instead you can try to query a copy of the database, or use a client
            that ignores locks, but neither of these have correctness guarantees.

            Wietse
          • Nikolaos Milas
            ... I continue to have this problem (no output from verify_cache querying). For example, it occurs on two servers with 2.9.4 (CentOS 6.3 x86_64). How can I
            Message 5 of 11 , Apr 4, 2013
            • 0 Attachment
              On 23/1/2013 3:43 μμ, Nikolaos Milas wrote:

              > Interestingly, at first I tried it but it didn't seem to produce any
              > output at all (I had to break with Ctrl-C):
              >
              > # postmap -s /var/lib/postfix/verify_cache
              > ^C
              > # postmap -s btree:/var/lib/postfix/verify_cache
              > ^C
              > # postmap -q user@... /var/lib/postfix/verify_cache
              > ^C
              > # postmap -q user@... btree:/var/lib/postfix/verify_cache
              > ^C
              >
              > Finally it worked as follows:
              >
              > # postmap -s btree:/var/lib/postfix/verify_cache
              > # postmap -q user@... btree:/var/lib/postfix/verify_cache

              I continue to have this problem (no output from verify_cache querying).

              For example, it occurs on two servers with 2.9.4 (CentOS 6.3 x86_64).

              How can I troubleshoot it to see why it happens?

              If I run verbose:

              # postmap -vs btree:/var/lib/postfix/verify_cache
              postmap: name_mask: ipv4
              postmap: name_mask: ipv6
              postmap: inet_addr_local: configured 2 IPv4 addresses
              postmap: inet_addr_local: configured 3 IPv6 addresses
              postmap: Compiled against Berkeley DB: 4.7.25?
              postmap: Run-time linked against Berkeley DB: 4.7.25?
              ^C

              (It hungs, so I Ctrl-C.)

              Thanks,
              Nick
            • Wietse Venema
              ... I have mentioned before that the verify(8) daemon locks the Berkeley DB file exclusively. Concurrent read/write access is not safe with these files. The
              Message 6 of 11 , Apr 4, 2013
              • 0 Attachment
                Nikolaos Milas:
                > On 23/1/2013 3:43 ??, Nikolaos Milas wrote:
                >
                > > Interestingly, at first I tried it but it didn't seem to produce any
                > > output at all (I had to break with Ctrl-C):
                > >
                > > # postmap -s /var/lib/postfix/verify_cache
                > > ^C
                > > # postmap -s btree:/var/lib/postfix/verify_cache
                > > ^C
                > > # postmap -q user@... /var/lib/postfix/verify_cache
                > > ^C
                > > # postmap -q user@... btree:/var/lib/postfix/verify_cache
                > > ^C

                I have mentioned before that the verify(8) daemon locks the Berkeley
                DB file exclusively. Concurrent read/write access is not safe with
                these files. The exclusive lock guarantees that verify(8) will not
                crash or do silly things when some other Postfix program changes
                the database. The lock also ensures that other Postfix programs
                will not crash or do silly things when verify(8) changes the database.

                You could do this:

                # cp /var/lib/postfix/verify_cache.db tempfile.db
                # postmap -s btree:tempfile.db

                Cross your fingers, and if it breaks, repeat the two commands.

                The new LMDB database type is safe for concurrent read/write access;
                however this currently has an open problem when creating a very
                large database file from scratch (as with the "postmap" command).
                That is not an issue with verify(8) which makes only incremental
                updates.

                Of course there is no guarantee of what you will get when you dump
                a large database while it is being modified, even when the database
                file is safe for concurrent read/write access. But at least the
                programs will not crash or do silly things.

                Wietse
              • Nikolaos Milas
                ... Thanks Wietse, it works fine this way: # postmap -s btree:verify_cache_copy By the way, is there a way/command to lengthen the validity of all current
                Message 7 of 11 , Apr 4, 2013
                • 0 Attachment
                  On 4/4/2013 3:38 μμ, Wietse Venema wrote:

                  > # cp /var/lib/postfix/verify_cache.db tempfile.db
                  > # postmap -s btree:tempfile.db

                  Thanks Wietse, it works fine this way:

                  # postmap -s btree:verify_cache_copy

                  By the way, is there a way/command to lengthen the validity of all
                  current entries in the verify cache?

                  As currently set, our gateway mail server(s) queries the final
                  destination (internal) server to verify recipients using:

                  relay_recipient_maps = <empty>

                  In case of planned downtime of internal (final destination) mail server,
                  I would like to avoid expiration of the gateway server's verify cache
                  during the downtime.

                  Thanks again,
                  Nick
                • Wietse Venema
                  ... Configure a larger address_verify_positive_expire_time? The default is 31 days. Note that Postfix does not wait until cached information expires. By
                  Message 8 of 11 , Apr 4, 2013
                  • 0 Attachment
                    Nikolaos Milas:
                    > On 4/4/2013 3:38 ??, Wietse Venema wrote:
                    >
                    > > # cp /var/lib/postfix/verify_cache.db tempfile.db
                    > > # postmap -s btree:tempfile.db
                    >
                    > Thanks Wietse, it works fine this way:
                    >
                    > # postmap -s btree:verify_cache_copy
                    >
                    > By the way, is there a way/command to lengthen the validity of all
                    > current entries in the verify cache?

                    Configure a larger address_verify_positive_expire_time?
                    The default is 31 days.

                    Note that Postfix does not wait until cached information expires.
                    By default it refreshes cached records after 7 days (the refresh
                    happens after the user receives email).

                    Therefore. cached information about active accounts will be valid
                    for 24..31 days.

                    > As currently set, our gateway mail server(s) queries the final
                    > destination (internal) server to verify recipients using:
                    >
                    > relay_recipient_maps = <empty>
                    >
                    > In case of planned downtime of internal (final destination) mail server,
                    > I would like to avoid expiration of the gateway server's verify cache
                    > during the downtime.

                    How long is your expected downtime? Even it someone receives only
                    one email per week, they should expire in 24 days.

                    Wietse
                  • Nikolaos Milas
                    ... It will be around 5-6 hours. I would like to avoid the gateway server bouncing mails due to not being able to verify the recipients. Ideally, mails should
                    Message 9 of 11 , Apr 4, 2013
                    • 0 Attachment
                      On 4/4/2013 5:42 μμ, Wietse Venema wrote:

                      > How long is your expected downtime? Even it someone receives only
                      > one email per week, they should expire in 24 days.

                      It will be around 5-6 hours. I would like to avoid the gateway server
                      bouncing mails due to not being able to verify the recipients.

                      Ideally, mails should be accepted by the gateway server based on its
                      verify cache, and then kept in the queue during the main server downtime
                      period.

                      One solution would be to delete the verify_cache.db file NOW and thus
                      force the creation of a new one, so that any positive cached address
                      will be added and be valid for at least 7 days (provided that the
                      downtime will occur in the next 7 days); the only drawback - I think -
                      is that some addresses may not be used until the downtime occurs, so
                      they will be missing from the verify cache when the downtime occurs.

                      One way to solve this would be to send one mail to all our users, so
                      that they will be added to the verify cache. However there are multiple
                      addresses per user (due to aliases, so this complicates things a bit).

                      If we use a group alias like all-staff@... as a recipient
                      address, does that cause all of the true recipient addresses (that the
                      alias resolves to) to be added to the verify cache as well?

                      Or, can we manually add addresses to the verify cache?

                      Alternatively, we could switch to ldap-based verification during this
                      period - but I would like to avoid doing serious changes under pressure.

                      Thanks,
                      Nick
                    • Wietse Venema
                      ... With the existing automatic cache refresh after 7 days, - ALL recipients who received their last email less than 1 week ago will remain cached for 24-31
                      Message 10 of 11 , Apr 4, 2013
                      • 0 Attachment
                        Nikolaos Milas:
                        > On 4/4/2013 5:42 ??, Wietse Venema wrote:
                        >
                        > > How long is your expected downtime? Even it someone receives only
                        > > one email per week, they should expire in 24 days.
                        >
                        > It will be around 5-6 hours. I would like to avoid the gateway server
                        > bouncing mails due to not being able to verify the recipients.

                        With the existing automatic cache refresh after 7 days,

                        - ALL recipients who received their last email less than 1 week ago will
                        remain cached for 24-31 days.

                        - ALL recipients who received their last email 1-2 weeks ago will remain
                        cached for 17-24 days.

                        - ALL recipients who received their last email 2-3 weeks ago will remain
                        cached for 10-17 days.

                        > One solution would be to delete the verify_cache.db file NOW and thus

                        NO, THAT WOULD BE AN ENORMOUS MISTAKE.

                        You weren't paying attention when i explained automatic cache refresh.

                        Wietse
                      • Nikolaos Milas
                        ... I admit I was confused with the initial explanation. Your latest clarifications make things much clearer to me. I understand that the gateway server should
                        Message 11 of 11 , Apr 4, 2013
                        • 0 Attachment
                          On 4/4/2013 11:06 μμ, Wietse Venema wrote:

                          > You weren't paying attention when i explained automatic cache refresh.

                          I admit I was confused with the initial explanation.

                          Your latest clarifications make things much clearer to me.

                          I understand that the gateway server should work fine with the current
                          (default) verify cache settings during the scheduled downtime.

                          Thanks,
                          Nick
                        Your message has been successfully submitted and would be delivered to recipients shortly.