Loading ...
Sorry, an error occurred while loading the content.

Problem with mysql postfix backend with amavisd

Expand Messages
  • rsmits-l
    Hello, We have an issue with our mailrelay system. Amavisd with spamassassin. Backend is Postfix with HASH db and mysql database. When the mail starts coming
    Message 1 of 6 , Jun 26, 2014
    • 0 Attachment
      Hello,

      We have an issue with our mailrelay system. Amavisd with spamassassin.
      Backend is Postfix with HASH db and mysql database. When the mail starts
      coming in, everything looks fine, but after a few minutes we see a lot
      of these errors.

      ----------------------------------
      Main.cf :
      Works fine :
      virtual_alias_maps = hash:/home/postfix/namen

      Errors :
      virtual_alias_maps = hash:/home/postfix/namen,
      mysql:/home/postfix/forwardsqlconnect.cf
      ----------------------------------

      Temporary lookup failure; from=<X.XXXXXXXX@...>
      to=<xxxxxxxx@...> proto=ESMTP helo=<localhost>

      Jun 26 14:25:27 mx4 amavis[8448]: (08448-02) smtp resp to RCPT (pip)
      (<xxxxxxxx@...>): 451 4.3.0 <xxxxxxxx@...>: Temporary lookup
      failure

      This happens only if the mysql backend is being used. If the mysql is
      being turned off in the config everything works as expected. Our setup
      is as followes :

      --> 25 / Postfix / smtpd
      --> 127.0.0.1:10026 / amavisd
      After spamcheck
      --> 127.0.0.1:10027 / Postfix smtpd
      Delivers to final recipient.

      We have noticed that recipient translations are being done twice. First
      before, and again after the spamcheck. We think the second translation
      is going wrong sometimes. Is there a way to disable recipeint check on
      second smtpd hop. (More of a question to postfix list)

      Can anyone help with this error ? Mysql database performance issue is
      not the case. After extensive monitoring we found that max connections
      and memory is fine.

      Greetings, Richard Smits.
    • lists@rhsoft.net
      ... get mysqltuner and/or look how many connections are allowed and make sure mysql don t close idle connections too soon wait_timeout = 604800
      Message 2 of 6 , Jun 26, 2014
      • 0 Attachment
        Am 26.06.2014 15:01, schrieb rsmits-l:
        > We have an issue with our mailrelay system. Amavisd with spamassassin. Backend is Postfix with HASH db and mysql
        > database. When the mail starts coming in, everything looks fine, but after a few minutes we see a lot of these errors.
        >
        > ----------------------------------
        > Main.cf :
        > Works fine :
        > virtual_alias_maps = hash:/home/postfix/namen
        >
        > Errors :
        > virtual_alias_maps = hash:/home/postfix/namen, mysql:/home/postfix/forwardsqlconnect.cf
        > ----------------------------------
        >
        > Temporary lookup failure; from=<X.XXXXXXXX@...> to=<xxxxxxxx@...> proto=ESMTP helo=<localhost>
        >
        > Jun 26 14:25:27 mx4 amavis[8448]: (08448-02) smtp resp to RCPT (pip) (<xxxxxxxx@...>): 451 4.3.0
        > <xxxxxxxx@...>: Temporary lookup failure
        >
        > This happens only if the mysql backend is being used. If the mysql is being turned off in the config everything

        get mysqltuner and/or look how many connections are allowed
        and make sure mysql don't close idle connections too soon

        wait_timeout = 604800
        interactive_timeout = 604800
      • Viktor Dukhovni
        ... In any case you really should consider: virtual_alias_maps = hash:/home/postfix/namen, proxy:mysql:/home/postfix/forwardsqlconnect.cf However, the real
        Message 3 of 6 , Jun 26, 2014
        • 0 Attachment
          On Thu, Jun 26, 2014 at 03:01:56PM +0200, rsmits-l wrote:

          > Errors :
          > virtual_alias_maps = hash:/home/postfix/namen,
          > mysql:/home/postfix/forwardsqlconnect.cf
          >
          > Can anyone help with this error? Mysql database performance issue is not
          > the case. After extensive monitoring we found that max connections and
          > memory is fine.

          In any case you really should consider:

          virtual_alias_maps = hash:/home/postfix/namen,
          proxy:mysql:/home/postfix/forwardsqlconnect.cf

          However, the real reason for the problem is in related log lines you
          have not reported.

          --
          Viktor.
        • rsmits-l
          ... Hello, I will send some logfile entry s but for starters I have read the proxymap explanation and have tried it on the mailserver. This seems to work !
          Message 4 of 6 , Jun 26, 2014
          • 0 Attachment
            On 06/26/2014 04:03 PM, Viktor Dukhovni wrote:
            > On Thu, Jun 26, 2014 at 03:01:56PM +0200, rsmits-l wrote:
            >
            >> Errors :
            >> virtual_alias_maps = hash:/home/postfix/namen,
            >> mysql:/home/postfix/forwardsqlconnect.cf
            >>
            >> Can anyone help with this error? Mysql database performance issue is not
            >> the case. After extensive monitoring we found that max connections and
            >> memory is fine.
            >
            > In any case you really should consider:
            >
            > virtual_alias_maps = hash:/home/postfix/namen,
            > proxy:mysql:/home/postfix/forwardsqlconnect.cf
            >
            > However, the real reason for the problem is in related log lines you
            > have not reported.
            >

            Hello,

            I will send some logfile entry's but for starters I have read the
            "proxymap" explanation and have tried it on the mailserver. This seems
            to work ! No more errors. I will read some more online, but if I
            understand corretly, this proxymap process will regulate the connections
            to mysql in a better way ?

            Greetings, Richard.
          • Viktor Dukhovni
            ... Turns out that perhaps the monitoring was misleading, and perhaps connection limits were exceeded. ... The most plausible solution to the rescue. ... With
            Message 5 of 6 , Jun 26, 2014
            • 0 Attachment
              On Thu, Jun 26, 2014 at 05:08:28PM +0200, rsmits-l wrote:

              > >>virtual_alias_maps = hash:/home/postfix/namen,
              > >> mysql:/home/postfix/forwardsqlconnect.cf
              > >>
              > >>Can anyone help with this error? Mysql database performance issue is not
              > >>the case. After extensive monitoring we found that max connections and
              > >>memory is fine.

              Turns out that perhaps the monitoring was misleading, and perhaps
              connection limits were exceeded.

              > >In any case you really should consider:
              > >
              > > virtual_alias_maps = hash:/home/postfix/namen,
              > > proxy:mysql:/home/postfix/forwardsqlconnect.cf
              > >
              > >However, the real reason for the problem is in related log lines you
              > >have not reported.
              >
              > I will send some logfile entry's but for starters I have read the "proxymap"
              > explanation and have tried it on the mailserver. This seems to work ! No
              > more errors.

              The most plausible solution to the rescue.

              > I will read some more online, but if I understand correctly,
              > this proxymap process will regulate the connections to mysql in a better way

              With proxyman(8) multiple processes query the database via a single
              proxy that uses a single connection to the database to make queries
              on behalf of multiple clients (other Postfix services).

              Database connection concurrency typically goes down by one to two
              orders of magnitude.

              --
              Viktor.
            • rsmits-l
              Hello, ... Well, it turns out it was the mysql idle time was set to short. (120 seconds) after we increased it to 1 hour there are no problems. I am not sure
              Message 6 of 6 , Jul 1, 2014
              • 0 Attachment
                Hello,

                On 06/26/2014 06:09 PM, Viktor Dukhovni wrote:
                > On Thu, Jun 26, 2014 at 05:08:28PM +0200, rsmits-l wrote:
                >
                >>>> virtual_alias_maps = hash:/home/postfix/namen,
                >>>> mysql:/home/postfix/forwardsqlconnect.cf
                >>>>
                >>>> Can anyone help with this error? Mysql database performance issue is not
                >>>> the case. After extensive monitoring we found that max connections and
                >>>> memory is fine.
                >
                > Turns out that perhaps the monitoring was misleading, and perhaps
                > connection limits were exceeded.

                Well, it turns out it was the mysql idle time was set to short. (120
                seconds) after we increased it to 1 hour there are no problems. I am not
                sure if i will use the "proxy:" parameter. If the mailserver is under a
                high load maybe the proxymap is a delay ? I will test this with more
                mailservers and see if there is a difference.

                wait_timeout = 120
                interactive_timeout = 120

                Greetings, Richard.
              Your message has been successfully submitted and would be delivered to recipients shortly.