Loading ...
Sorry, an error occurred while loading the content.

memcache adress verify cleanup

Expand Messages
  • Daniel Luttermann
    Hi, I ve changed my config to use memcache for postscreen and verify. Now, I assume that the cache cleanup for the verify db doesn t run anymore. For
    Message 1 of 7 , Nov 17, 2012
    • 0 Attachment
      Hi,

      I've changed my config to use memcache for postscreen and verify. Now,
      I assume that the cache cleanup for the verify db doesn't run anymore.

      For postscreen, there are entries like this in /var/log/mail.info:

      Nov 17 16:43:25 mail postfix/postscreen[863]: cache
      memcache:/etc/postfix/memcache-postscreen.cf full cleanup: retained=222
      dropped=11 entries

      Before memcache, there were similar entries like this in
      /var/log/mail.info for the verify.db:

      Nov 5 22:27:57 mail postfix/verify[11846]: cache
      btree:/var/lib/postfix/verify_cache full cleanup: retained=6 dropped=0
      entries

      Since a few days I don't see any cache cleanup entries for the verify
      database but the database in /var/lib/postfix gets updated regularly.

      Does the verify cleanup not run every 12 hours when using memcache?

      Config:

      address_verify_map = memcache:/etc/postfix/verify-memcache.cf
      address_verify_negative_expire_time = 3d
      address_verify_negative_refresh_time = 3h
      address_verify_positive_expire_time = 31d
      address_verify_positive_refresh_time = 7d
      postscreen_bare_newline_enable = yes
      postscreen_cache_map = memcache:/etc/postfix/memcache-postscreen.cf
      postscreen_non_smtp_command_enable = yes
      postscreen_pipelining_enable = yes
      proxy_read_maps = $local_recipient_maps $mydestination
      $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps
      $virtual_mailbox_domains $relay_recipient_maps $relay_domains
      $canonical_maps $sender_canonical_maps $recipient_canonical_maps
      $relocated_maps $transport_maps $mynetworks $sender_bcc_maps
      $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps $alias_maps
      proxy:btree:/var/lib/postfix/postscreen_cache_map
      proxy:btree:/var/lib/postfix/verify_cache_map
      proxy_write_maps = $smtp_sasl_auth_cache_name
      $lmtp_sasl_auth_cache_name $address_verify_map $postscreen_cache_map
      proxy:btree:/var/lib/postfix/postscreen_cache_map
      proxy:btree:/var/lib/postfix/verify_cache_map

      /etc/postfix/verify-memcache.cf:

      memcache = inet:127.0.0.1:11211
      backup = proxy:btree:/var/lib/postfix/verify_cache_map

      /etc/postfix/memcache-postscreen.cf:

      memcache = inet:127.0.0.1:11211
      backup = proxy:btree:/var/lib/postfix/postscreen_cache_map

      Thank you.

      Daniel
    • Noel Jones
      ... The cache cleanup will still run unless you ve turned it off. # postconf | grep cleanup_interval ... config unreadable. -- Noel Jones
      Message 2 of 7 , Nov 17, 2012
      • 0 Attachment
        On 11/17/2012 3:07 PM, Daniel Luttermann wrote:
        > Hi,
        > I've changed my config to use memcache for postscreen and verify.
        > Now, I assume that the cache cleanup for the verify db doesn't run
        > anymore.
        > For postscreen, there are entries like this in /var/log/mail.info:
        > Nov 17 16:43:25 mail postfix/postscreen[863]: cache
        > memcache:/etc/postfix/memcache-postscreen.cf full cleanup:
        > retained=222 dropped=11 entries
        > Before memcache, there were similar entries like this in
        > /var/log/mail.info for the verify.db:
        > Nov 5 22:27:57 mail postfix/verify[11846]: cache
        > btree:/var/lib/postfix/verify_cache full cleanup: retained=6
        > dropped=0 entries
        > Since a few days I don't see any cache cleanup entries for the
        > verify database but the database in /var/lib/postfix gets updated
        > regularly.
        > Does the verify cleanup not run every 12 hours when using memcache?


        The cache cleanup will still run unless you've turned it off.
        # postconf | grep cleanup_interval



        > Config:

        config unreadable.





        -- Noel Jones
      • Daniel Luttermann
        ... root@mail:~# postconf | grep cleanup_interval address_verify_cache_cleanup_interval = 12h postscreen_cache_cleanup_interval = 12h ... Sorry, hopefully
        Message 3 of 7 , Nov 17, 2012
        • 0 Attachment
          On 11/17/2012 Noel Jones wrote:

          > On 11/17/2012 3:07 PM, Daniel Luttermann wrote:
          >> Hi,
          >> I've changed my config to use memcache for postscreen and verify.
          >> Now, I assume that the cache cleanup for the verify db doesn't run
          >> anymore.
          >> For postscreen, there are entries like this in /var/log/mail.info:
          >> Nov 17 16:43:25 mail postfix/postscreen[863]: cache
          >> memcache:/etc/postfix/memcache-postscreen.cf full cleanup:
          >> retained=222 dropped=11 entries
          >> Before memcache, there were similar entries like this in
          >> /var/log/mail.info for the verify.db:
          >> Nov 5 22:27:57 mail postfix/verify[11846]: cache
          >> btree:/var/lib/postfix/verify_cache full cleanup: retained=6
          >> dropped=0 entries
          >> Since a few days I don't see any cache cleanup entries for the
          >> verify database but the database in /var/lib/postfix gets updated
          >> regularly.
          >> Does the verify cleanup not run every 12 hours when using memcache?


          > The cache cleanup will still run unless you've turned it off.
          > # postconf | grep cleanup_interval

          root@mail:~# postconf | grep cleanup_interval
          address_verify_cache_cleanup_interval = 12h
          postscreen_cache_cleanup_interval = 12h

          >> Config:

          > config unreadable.

          Sorry, hopefully better now...

          address_verify_map = memcache:/etc/postfix/verify-memcache.cf
          address_verify_negative_expire_time = 3d
          address_verify_negative_refresh_time = 3h
          address_verify_positive_expire_time = 31d
          address_verify_positive_refresh_time = 7d
          postscreen_bare_newline_enable = yes
          postscreen_cache_map = memcache:/etc/postfix/memcache-postscreen.cf
          postscreen_non_smtp_command_enable = yes
          postscreen_pipelining_enable = yes

          /etc/postfix/verify-memcache.cf:

          memcache = inet:127.0.0.1:11211
          backup = proxy:btree:/var/lib/postfix/verify_cache_map

          /etc/postfix/memcache-postscreen.cf:

          memcache = inet:127.0.0.1:11211
          backup = proxy:btree:/var/lib/postfix/postscreen_cache_map

          Thank you.

          Daniel
        • Wietse Venema
          ... And both postscreen and verify use the same code to clean the cache. ... As requested in the mailing list message, that is not how you report a Postfix
          Message 4 of 7 , Nov 17, 2012
          • 0 Attachment
            Daniel Luttermann:
            > > The cache cleanup will still run unless you've turned it off.
            > > # postconf | grep cleanup_interval
            >
            > root@mail:~# postconf | grep cleanup_interval
            > address_verify_cache_cleanup_interval = 12h
            > postscreen_cache_cleanup_interval = 12h

            And both postscreen and verify use the same code to clean the cache.
            > >> Config:
            >
            > > config unreadable.
            >
            > Sorry, hopefully better now...
            >
            > address_verify_map = memcache:/etc/postfix/verify-memcache.cf

            As requested in the mailing list message, that is not how you report
            a Postfix configuration. Follow the instructions and you will be
            helped better.

            Wietse

            TO REPORT A PROBLEM see http://www.postfix.org/DEBUG_README.html#mail

            TO (UN)SUBSCRIBE see http://www.postfix.org/lists.html

            Thank you for using Postfix.
          • Daniel Luttermann
            ... System: Postfix 2.9.4 on Debian Linux 6.0.6, 64-Bit For postscreen the cleanup process is OK. In the log there are entries like Nov 16 16:08:40 mail
            Message 5 of 7 , Nov 17, 2012
            • 0 Attachment
              On 11/17/2012 Wietse Venema wrote:

              > Daniel Luttermann:
              >> > The cache cleanup will still run unless you've turned it off.
              >> > # postconf | grep cleanup_interval
              >>
              >> root@mail:~# postconf | grep cleanup_interval
              >> address_verify_cache_cleanup_interval = 12h
              >> postscreen_cache_cleanup_interval = 12h

              > And both postscreen and verify use the same code to clean the cache.
              >> >> Config:
              >>
              >> > config unreadable.
              >>
              >> Sorry, hopefully better now...
              >>
              >> address_verify_map = memcache:/etc/postfix/verify-memcache.cf

              > As requested in the mailing list message, that is not how you report
              > a Postfix configuration. Follow the instructions and you will be
              > helped better.

              > Wietse

              > TO REPORT A PROBLEM see http://www.postfix.org/DEBUG_README.html#mail

              > TO (UN)SUBSCRIBE see http://www.postfix.org/lists.html

              > Thank you for using Postfix.


              System: Postfix 2.9.4 on Debian Linux 6.0.6, 64-Bit

              For postscreen the cleanup process is OK. In the log there are
              entries like

              Nov 16 16:08:40 mail postfix/postscreen[11233]: PASS OLD [63.236.76.20]:41453
              Nov 16 16:08:40 mail postfix/postscreen[11233]: cache memcache:/etc/postfix/memcache-postscreen.cf full cleanup: retained=174 dropped=2 entries

              But no line like

              Nov 4 18:43:04 mail postfix/verify[1580]: cache btree:/var/lib/postfix/verify_cache full cleanup: retained=6 dropped=0 entries

              for the verify database. Before I switched my config to postscreen
              with memcache, the cleanup of the verify database worked and the
              appropriate entries had been written to the log file.

              Now with memcache and postscreen - because there are no appropriate
              entries logged for the verifiy db cleanup - I'm not sure if the cleanup
              process gets started.

              Here's my complete config (postconf -n):

              address_verify_map = memcache:/etc/postfix/verify-memcache.cf
              address_verify_negative_expire_time = 3d
              address_verify_negative_refresh_time = 3h
              address_verify_positive_expire_time = 31d
              address_verify_positive_refresh_time = 7d
              alias_maps = hash:/etc/aliases
              bounce_queue_lifetime = 1d
              config_directory = /etc/postfix
              disable_vrfy_command = yes
              inet_protocols = all
              maximal_queue_lifetime = 1d
              mydestination = $myhostname, localhost.$mydomain, localhost
              mydomain = dlutt.de
              myhostname = mail.dlutt.de
              mynetworks_style = host
              myorigin = $myhostname
              postscreen_bare_newline_enable = yes
              postscreen_cache_map = memcache:/etc/postfix/memcache-postscreen.cf
              postscreen_non_smtp_command_enable = yes
              postscreen_pipelining_enable = yes
              proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps $alias_maps proxy:btree:/var/lib/postfix/postscreen_cache_map proxy:btree:/var/lib/postfix/verify_cache_map
              proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name $address_verify_map $postscreen_cache_map proxy:btree:/var/lib/postfix/postscreen_cache_map proxy:btree:/var/lib/postfix/verify_cache_map
              recipient_delimiter = +
              relay_domains = proxy:hash:/etc/postfix/relay_domains
              sendmail_path = /usr/sbin/sendmail
              show_user_unknown_table_name = no
              smtp_bind_address = 217.11.53.6
              smtp_bind_address6 = 2a00:1828:2000:206::2
              smtpd_client_connection_rate_limit = 25
              smtpd_discard_ehlo_keywords = silent-discard, dsn
              smtpd_helo_required = yes
              smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination
              smtpd_sasl_path = private/auth
              smtpd_sasl_type = dovecot
              smtpd_sender_login_maps = hash:/etc/postfix/smtpd_sender_login_maps
              smtpd_tls_cert_file = /etc/ssl/certs/postfix.pem
              smtpd_tls_key_file = /etc/ssl/private/postfix.key
              smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_tls_session_cache
              strict_rfc821_envelopes = yes
              transport_maps = proxy:hash:/etc/postfix/transport_maps
              unverified_recipient_reject_code = 550
              virtual_alias_maps = proxy:hash:/etc/postfix/virtual_alias_maps


              Thank you.

              Daniel
            • Wietse Venema
              ... Well there is your problem, According to memcache_table(5): When the same memcache database is used to cache information from multiple tables, you
              Message 6 of 7 , Nov 17, 2012
              • 0 Attachment
                Daniel Luttermann:
                >/etc/postfix/verify-memcache.cf:
                >
                >memcache = inet:127.0.0.1:11211
                >backup = proxy:btree:/var/lib/postfix/verify_cache_map
                >
                >/etc/postfix/memcache-postscreen.cf:
                >
                >memcache = inet:127.0.0.1:11211
                >backup = proxy:btree:/var/lib/postfix/postscreen_cache_map

                Well there is your problem,

                According to memcache_table(5):

                When the same memcache database is used to cache information
                from multiple tables, you can use the key_format feature to
                avoid name collisions by prepending a fixed string. Examples:

                key_format = aliases:%s
                key_format = access:%s

                The key_format parameter supports the following '%' expansions:
                [bunch of text omitted]

                You have collisions between the names of the cache entries that say
                when the last cleanup has happened.

                Wietse
              • Daniel Luttermann
                ... thank you. I will update the config and check the logfile if everything is OK afterwards. Thank you again. Daniel
                Message 7 of 7 , Nov 17, 2012
                • 0 Attachment
                  On 11/18/2012 Wietse Venema wrote:

                  > Daniel Luttermann:
                  >>/etc/postfix/verify-memcache.cf:
                  >>
                  >>memcache = inet:127.0.0.1:11211
                  >>backup = proxy:btree:/var/lib/postfix/verify_cache_map
                  >>
                  >>/etc/postfix/memcache-postscreen.cf:
                  >>
                  >>memcache = inet:127.0.0.1:11211
                  >>backup = proxy:btree:/var/lib/postfix/postscreen_cache_map

                  > Well there is your problem,

                  > According to memcache_table(5):

                  > When the same memcache database is used to cache information
                  > from multiple tables, you can use the key_format feature to
                  > avoid name collisions by prepending a fixed string. Examples:

                  > key_format = aliases:%s
                  > key_format = access:%s

                  > The key_format parameter supports the following '%' expansions:
                  > [bunch of text omitted]

                  > You have collisions between the names of the cache entries that say
                  > when the last cleanup has happened.

                  > Wietse

                  thank you. I will update the config and check the logfile if
                  everything is OK afterwards.

                  Thank you again.

                  Daniel
                Your message has been successfully submitted and would be delivered to recipients shortly.