Loading ...
Sorry, an error occurred while loading the content.
 

Postscreen & Google Apps

Expand Messages
  • Jon A.
    Today, a Google Apps user sent a message with two recipients to us, one with TO and other a CC internal mailing list. Naturally, Google treated each as an
    Message 1 of 4 , Jan 23, 2013
      Today, a Google Apps user sent a message with two recipients to us, one with TO and other a CC internal mailing list.  Naturally, Google treated each as an independent message.

      Over the course of an hour or so, because Google attempted to deliver the messages using different outgoing hosts, postscreen rejected the message(s) ~20 times, with a service unavailable, as we'd expect and normally want.

      Eventually, the TO recipient received the email where the distribution list recipients hadn't yet...  that message is still in some queue at Google, and continues to be tried with different outgoing addresses.  

      Unfortunately, the TO recipient has since replied to all recipients.

      Management(TM) saw the CC'ed reply, but hadn't gotten the original message.  This has caused some concern.

      How have others dealt with this type of situation?

      The only solution I can see would involve identifying the google MX IP range and white-listing those hosts.  This has two undesired side effects: 1st it's on me to find the hosts,and 2nd we should expect this for other services using a huge pool of boxes.  If I understand things correctly, this is too early in the process to permit based on sender's name, nor would that necessarily be good for stuff from "google" in general.

      The second thought I have is that the postscreen expiration should probably be made longer lest we go through this over and over again.

      Comments/Thoughts/Suggestions?
    • Noel Jones
      ... I think the usual way is to use postscreen in non-blocking mode for a couple weeks to build up the temporary whitelist. The default cache time for
      Message 2 of 4 , Jan 23, 2013
        On 1/23/2013 4:33 PM, Jon A. wrote:
        > Today, a Google Apps user sent a message with two recipients to us,
        > one with TO and other a CC internal mailing list. Naturally, Google
        > treated each as an independent message.
        >
        > Over the course of an hour or so, because Google attempted to
        > deliver the messages using different outgoing hosts, postscreen
        > rejected the message(s) ~20 times, with a service unavailable, as
        > we'd expect and normally want.
        >
        ...
        > Comments/Thoughts/Suggestions?


        I think the usual way is to use postscreen in non-blocking mode for
        a couple weeks to build up the temporary whitelist.

        The default cache time for successful after-220 tests is 30 days;
        that's probably sufficient for the majority. A very low volume
        server might need to cache longer. The DNS blocklist test will only
        cache for 1 hour, but that won't tempfail mail and shouldn't need to
        be changed.

        If you want to proactively whitelist google's servers, they publish
        SPF records so you don't have to spend much effort hunting them
        down. The postscreen access list is IP-only and can't use client or
        sender domain names. And you've already added a bunch of their
        servers to your cache.

        I don't bother with trying to whitelist big senders, and I don't
        think many other folks do either. The big senders usually end up in
        the the cache by themselves pretty quickly, and the
        once-every-30-days refresh isn't particularly intrusive. You just
        got caught in a situation where an important mail came through
        before the whitelist had a chance to populate.



        > Management(TM) saw the CC'ed reply, but hadn't gotten the original message. This has caused some concern.


        I probably repeat once a week to folks around here something like:
        "The mail protocol standards are heavily weighted towards not losing
        mail rather than instant delivery, and sometimes mail is unavoidably
        delayed. Much of this is outside our control. Either the delayed
        message will eventually arrive, or the sender will get a notice that
        it was not delivered."




        -- Noel Jones
      • Benny Pedersen
        ... To: and Cc: is not envelope recipient understanding smtp might help :)
        Message 3 of 4 , Jan 23, 2013
          Jon A. skrev den 2013-01-23 23:33:

          > Comments/Thoughts/Suggestions?

          To: and Cc: is not envelope recipient

          understanding smtp might help :)
        • Jon A.
          ... *smack* Thanks, that would do it. I initially ran my configuration in test mode on some boxes, then unified the configuration when I cut everything over
          Message 4 of 4 , Jan 24, 2013

            On Wed, Jan 23, 2013 at 6:24 PM, Noel Jones <njones@...> wrote:
            On 1/23/2013 4:33 PM, Jon A. wrote:
            > Today, a Google Apps user sent a message with two recipients to us,
            > one with TO and other a CC internal mailing list.  Naturally, Google
            > treated each as an independent message.
            >
            > Over the course of an hour or so, because Google attempted to
            > deliver the messages using different outgoing hosts, postscreen
            > rejected the message(s) ~20 times, with a service unavailable, as
            > we'd expect and normally want.
            >
            ...
            > Comments/Thoughts/Suggestions?


            I think the usual way is to use postscreen in non-blocking mode for
            a couple weeks to build up the temporary whitelist.


            *smack*  Thanks, that would do it.  I initially ran my configuration in test mode on some boxes, then unified the configuration when I cut everything over to production.  Which meant I left that whitelist data behind.  I've since moved back to building cache.

            Of course, as we'd expect, the original message eventually came in.
             
            The default cache time for successful after-220 tests is 30 days;
            that's probably sufficient for the majority.  A very low volume
            server might need to cache longer.  The DNS blocklist test will only
            cache for 1 hour, but that won't tempfail mail and shouldn't need to
            be changed.

            If you want to proactively whitelist google's servers, they publish
            SPF records so you don't have to spend much effort hunting them
            down.  The postscreen access list is IP-only and can't use client or
            sender domain names.  And you've already added a bunch of their
            servers to your cache.

            Indeed, after I posted I did grab the spf records for the biggie email providers and added them to the already-configured-in-case whitelist.  [Thanks Wietse for always building in exception mechanisms] However your email has convinced me this need was really a temporary measure.  The idea of chasing SPF changes from the laundry list of providers for the normal case just doesn't scale.

            I don't bother with trying to whitelist big senders, and I don't
            think many other folks do either. The big senders usually end up in
            the the cache by themselves pretty quickly, and the
            once-every-30-days refresh isn't particularly intrusive.  You just
            got caught in a situation where an important mail came through
            before the whitelist had a chance to populate.



            > Management(TM) saw the CC'ed reply, but hadn't gotten the original message.  This has caused some concern.


            I probably repeat once a week to folks around here something like:
            "The mail protocol standards are heavily weighted towards not losing
            mail rather than instant delivery, and sometimes mail is unavoidably
            delayed.  Much of this is outside our control.  Either the delayed
            message will eventually arrive, or the sender will get a notice that
            it was not delivered."


            If you don't mind, I may very well quote ya.   Thanks for a well thought out response Noel!  You gave me my first d'oh moment of the week.
             
          Your message has been successfully submitted and would be delivered to recipients shortly.