Loading ...
Sorry, an error occurred while loading the content.

262302Re: smtpd processes congregating at the pub

Expand Messages
  • Stan Hoeppner
    Jan 30, 2010
    • 0 Attachment
      Wietse Venema put forth on 1/30/2010 7:14 PM:
      > Stan Hoeppner:
      >> AFAIK I don't use Berkeley DB tables, only hash (small,few) and cidr
      >> (very large, a handful).
      >
      > hash (and btree) == Berkeley DB.

      Ahh, good to know. I'd thought only btree used Berkeley DB and that hash tables
      used something else.

      > If you have big CIDR tables, you can save lots of memory by using
      > proxy:cidr: instead of cidr: (and running "postfix reload").
      > Effectively, this turns all that private memory into something that
      > can be shared via the proxy: protocol.

      I implemented proxymap but it doesn't appear to have changed the memory
      footprint of smtpd much at all, if any. I reloaded once, and restarted once
      just in case.

      PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
      4554 postfix 20 0 20828 17m 2268 S 0 4.5 0:00.46 smtpd
      4560 postfix 20 0 20036 16m 2268 S 0 4.3 0:00.47 smtpd
      4555 postfix 20 0 6812 3056 1416 S 0 0.8 0:00.10 proxymap

      > The current CIDR implementation is optimized to make it easy to
      > verify for correctness, and is optimized for speed when used with
      > limited lists of netblocks (mynetworks, unassigned address blocks,
      > reserved address blocks, etc.).

      Understood.

      > If you want to list large portions of Internet address space such
      > as entire countries the current implementation starts burning CPU
      > time (it examines all CIDR patterns in order; with a bit of extra
      > up-front work during initialization, address lookups could skip
      > over a lot of patterns, but the implementation would of course be
      > harder to verify for correctness), and it wastes 24 bytes per CIDR
      > rule when Postfix is compiled with IPv6 support (this roughly
      > doubles the amount memory that is used by CIDR tables).

      I don't really notice much CPU burn on any postfix processes with these largish
      CIDRs, never have. I've got 12,212 CIDRs in 3 files, 11,148 of them in just the
      "countries" file alone. After implementing proxymap, I'm not seeing much
      reduction in smtpd RES size, maybe 1MB if that. SHR is almost identical to
      before. If it's not the big tables bloating smtpd, I wonder what is? Or, have
      I not implemented proxymap correctly? Following are my postconf -n and main.cf
      relevant parts.

      alias_maps = hash:/etc/aliases
      append_dot_mydomain = no
      biff = no
      config_directory = /etc/postfix
      disable_vrfy_command = yes
      header_checks = pcre:/etc/postfix/header_checks
      inet_interfaces = all
      message_size_limit = 10240000
      mime_header_checks = pcre:/etc/postfix/mime_header_checks
      mydestination = hardwarefreak.com
      myhostname = greer.hardwarefreak.com
      mynetworks = 192.168.100.0/24
      myorigin = hardwarefreak.com
      parent_domain_matches_subdomains = debug_peer_list smtpd_access_maps
      proxy_interfaces = 65.41.216.221
      proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
      $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
      $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
      $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
      $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
      proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs
      readme_directory = /usr/share/doc/postfix
      recipient_bcc_maps = hash:/etc/postfix/recipient_bcc
      relay_domains =
      smtpd_banner = $myhostname ESMTP Postfix
      smtpd_helo_required = yes
      smtpd_recipient_restrictions = permit_mynetworks
      reject_unauth_destination check_recipient_access
      hash:/etc/postfix/whitelist check_sender_access hash:/etc/postfix/whitelist
      check_client_access hash:/etc/postfix/whitelist check_client_access
      hash:/etc/postfix/blacklist check_client_access
      regexp:/etc/postfix/fqrdns.regexp check_client_access
      pcre:/etc/postfix/ptr-tld.pcre check_client_access proxy:${cidr}/countries
      check_client_access proxy:${cidr}/spammer check_client_access
      proxy:${cidr}/misc-spam-srcs reject_unknown_reverse_client_hostname
      reject_non_fqdn_sender reject_non_fqdn_helo_hostname
      reject_invalid_helo_hostname reject_unknown_helo_hostname
      reject_unlisted_recipient reject_rbl_client zen.spamhaus.org
      check_policy_service inet:127.0.0.1:60000
      strict_rfc821_envelopes = yes
      virtual_alias_maps = hash:/etc/postfix/virtual

      /etc/postfix/main.cf snippet

      cidr=cidr:/etc/postfix/cidr_files

      proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps
      $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains
      $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps
      $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks
      $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps
      proxy:${cidr}/countries proxy:${cidr}/spammer proxy:${cidr}/misc-spam-srcs

      check_client_access proxy:${cidr}/countries
      check_client_access proxy:${cidr}/spammer
      check_client_access proxy:${cidr}/misc-spam-srcs

      --
      Stan
    • Show all 16 messages in this topic