Loading ...
Sorry, an error occurred while loading the content.
 

Re: [aprsisce] APRSIS storing delayed duplicates?

Expand Messages
  • Keith VE7GDH
    Warren KC0U wrote... ... A KPC3+ per se isn t a problem with delayed packets... e.g. a stand-alone digipeater. It s when it s used in KISS mode with an APRS
    Message 1 of 16 , Dec 27, 2012
      Warren KC0U wrote...

      > None of the digipeaters used in my trip home are using a
      > KPC-3+, so I don't think that is the problem.

      A KPC3+ per se isn't a problem with delayed packets... e.g.
      a stand-alone digipeater. It's when it's used in KISS mode with
      an APRS client that it's a problem. After a period of time, it
      starts delaying the output on the serial port.

      > Perhaps I am using the wrong terminology, as the website aprs.fi
      > is obviously using stored information to reproduce my track. Is
      > aprs.fi not part of the APRSIS network?

      No, it isn't. The website aprs.fi gets data from the APRS-IS. The
      APRS-IS gets data from IGates and shares that data between all
      the servers that make up the APRS-IS, and clients such as UI-View,
      APRSISCE/32 etc. can if they wish get data from the APRS-IS.
      The site aprs.fi does store data, but it gets it from the APRS-IS.

      The APRS network consists of the stations themselves, the
      digipeaters, and the IGates. Whatever the IGates hear, they "gate"
      to whatever server they are connected to. All the servers share
      the data between themselves. Clients can connect to one of
      the servers and get some or all of the data from the APRS
      network. Sits like findu.com, aprs.fi, db0anf.de etc. can
      connect to the APRS-IS and get whatever data they want.
      The three I named would likely have a "full feed" from the
      APRS-IS... i.e. everything that was gated but not thrown
      out because it was a duplicate. The APRS-IS uses 30 second
      dupe checking, so packets delayed by less than 30 seconds
      won't show up. However, copies of a packet delayed by more
      than 30 seconds will show up on places like aprs.fi...
      or in your own APRS client if you were getting a feed
      from the APRS-IS.

      There are two likely causes of delayed packets.

      One would be a digi holding onto packets and later digipeating
      them... e.g. the digi could be in a location where the frequency is
      VERY busy, and it might have to wait a while before there was
      no activity before sending a digipeated packet... or the digi could
      be using squelched audio and the squelch could be on the hairy
      edge. It's better if the TNC sees unsquelched audio from the
      radio and it determines itself if it is receiving noise or data.

      The other common cause of delays are the IGates themselves.
      It could be one on a very under-powered computer with a
      slow CPU and not enough RAM, but I would think a more
      likely cause at an IGate would be a KPC3+ running in KISS
      mode. It's a known problem. UI-View is capable of restarting
      itself on a scheduled basis. This takes the TNC out of KISS
      mode and puts it back into KISS mode when it starts up
      again. There could be other reasons why an operator would
      like to restart it, but an added benefit of this is that it also
      restarts the TNC.

      There isn't anything you can do about delayed packets...
      except track down the cause(s) and notify the operators
      so they at least know about the problem. Some won't
      know what you are talking about and will tell you that
      "their station/digi or whatever has been operating for
      years without any problems" and others will look into it
      and hopefully take some kind of action. It's sometimes
      easier to track down the cause by watching / listening on
      RF, but viewing raw data at e.g. aprs.fi can sometimes
      lead to clues... e.g.

      2012-12-27 20:03:27 UTC:
      KC0U-8>APOTC1,WIDE1-1,qAR,KC0U-1
      :!/;5XK63m6> st

      KC0U-8 was heard direct and gated by your KCOU-1. No
      problems so far. Then about 9 minutes later...

      2012-12-27 20:12:25 UTC:
      KC0U-8>APOTC1,WX0U-1*,qAR,WX0U-2
      :!/;5XK63m6> st [Duplicate position packet]

      This appears to be an identical packet. I suppose it is possible
      that KC0U-8 had not moved and it just happened to have
      exactly the same location is the earlier beacon, but given
      that other beacons before and after make it look like it
      was moving, a more likely explanation is that either your
      own WXOU-1 spat out the digipeated copy, but with a delay,
      or... it was delayed by the nearby WX0U-2. That was just
      one example. In another I looked at, WX0U-1 was a
      possible culprit.

      I'm looking at the data at the following URL. WX0U-1 and
      WX0U-2 came up often as possible suspects. If it's a digi
      causing the delays, you might be able to spot it on RF if
      you watch long enough. If it's an IGate causing them,
      looking somewhere like aprs.fi can yield clues... e.g.
      a packet that went through a particular digi and was
      gated by two different IGates but with a delay, would
      point at the second digi as being the cause, as both would
      have heard the digi at the same time. When the delayed
      packet shows up but via a different digi path, it would
      take some more digging to prove if a digi caused the delay
      or if it was caused at the IGate level.

      http://aprs.fi/?c=raw&call=KC0U-8&limit=1000

      73 es cul - Keith VE7GDH
      www.ui-view.org
      --
      "I may be lost, but I know exactly where I am!"
    • Steve Daniels
      Is it possible for APRSIS to command a KPC-3+ to restart to clear the delay, am thinking here that if there is Lynn could perhaps code a restart system. That
      Message 2 of 16 , Dec 27, 2012

        Is it possible for APRSIS to command a KPC-3+ to restart to clear the delay, am thinking here that if there is Lynn could perhaps code a restart system. That cycles the KPC in the early hours of the morning each day/week.

        I realise this is slightly off topic

         

        Steve Daniels

        Amateur Radio Callsign G6UIM

        Torbay Freecycle  Owner

        http://uk.groups.yahoo.com/group/torbay_freecycle

        APRSISCE/32 Beta tester and WIKI editor http://aprsisce.wikidot.com

         


        From: aprsisce@yahoogroups.com [mailto: aprsisce@yahoogroups.com ] On Behalf Of Keith VE7GDH
        Sent: 27 December 2012 21:57
        To: aprsisce@yahoogroups.com
        Subject: Re: [aprsisce] APRSIS storing delayed duplicates?

         

         

        Warren KC0U wrote...

        > None of the digipeaters used in my trip home are using a
        > KPC-3+, so I don't think that is the problem.

        A KPC3+ per se isn't a problem with delayed packets... e.g.
        a stand-alone digipeater. It's when it's used in KISS mode with
        an APRS client that it's a problem. After a period of time, it
        starts delaying the output on the serial port.

        > Perhaps I am using the wrong terminology, as the website aprs.fi
        > is obviously using stored information to reproduce my track. Is
        > aprs.fi not part of the APRSIS network?

        No, it isn't. The website aprs.fi gets data from the APRS-IS. The
        APRS-IS gets data from IGates and shares that data between all
        the servers that make up the APRS-IS, and clients such as UI-View,
        APRSISCE/32 etc. can if they wish get data from the APRS-IS.
        The site aprs.fi does store data, but it gets it from the APRS-IS.

        The APRS network consists of the stations themselves, the
        digipeaters, and the IGates. Whatever the IGates hear, they "gate"
        to whatever server they are connected to. All the servers share
        the data between themselves. Clients can connect to one of
        the servers and get some or all of the data from the APRS
        network. Sits like findu.com, aprs.fi, db0anf.de etc. can
        connect to the APRS-IS and get whatever data they want.
        The three I named would likely have a "full feed" from the
        APRS-IS... i.e. everything that was gated but not thrown
        out because it was a duplicate. The APRS-IS uses 30 second
        dupe checking, so packets delayed by less than 30 seconds
        won't show up. However, copies of a packet delayed by more
        than 30 seconds will show up on places like aprs.fi...
        or in your own APRS client if you were getting a feed
        from the APRS-IS.

        There are two likely causes of delayed packets.

        One would be a digi holding onto packets and later digipeating
        them... e.g. the digi could be in a location where the frequency is
        VERY busy, and it might have to wait a while before there was
        no activity before sending a digipeated packet... or the digi could
        be using squelched audio and the squelch could be on the hairy
        edge. It's better if the TNC sees unsquelched audio from the
        radio and it determines itself if it is receiving noise or data.

        The other common cause of delays are the IGates themselves.
        It could be one on a very under-powered computer with a
        slow CPU and not enough RAM, but I would think a more
        likely cause at an IGate would be a KPC3+ running in KISS
        mode. It's a known problem. UI-View is capable of restarting
        itself on a scheduled basis. This takes the TNC out of KISS
        mode and puts it back into KISS mode when it starts up
        again. There could be other reasons why an operator would
        like to restart it, but an added benefit of this is that it also
        restarts the TNC.

        There isn't anything you can do about delayed packets...
        except track down the cause(s) and notify the operators
        so they at least know about the problem. Some won't
        know what you are talking about and will tell you that
        "their station/digi or whatever has been operating for
        years without any problems" and others will look into it
        and hopefully take some kind of action. It's sometimes
        easier to track down the cause by watching / listening on
        RF, but viewing raw data at e.g. aprs.fi can sometimes
        lead to clues... e.g.

        2012-12-27 20:03:27 UTC:
        KC0U-8>APOTC1,WIDE1-1,qAR,KC0U-1
        :!/;5XK63m6> st

        KC0U-8 was heard direct and gated by your KCOU-1. No
        problems so far. Then about 9 minutes later...

        2012-12-27 20:12:25 UTC:
        KC0U-8>APOTC1,WX0U-1*,qAR,WX0U-2
        :!/;5XK63m6> st [Duplicate position packet]

        This appears to be an identical packet. I suppose it is possible
        that KC0U-8 had not moved and it just happened to have
        exactly the same location is the earlier beacon, but given
        that other beacons before and after make it look like it
        was moving, a more likely explanation is that either your
        own WXOU-1 spat out the digipeated copy, but with a delay,
        or... it was delayed by the nearby WX0U-2. That was just
        one example. In another I looked at, WX0U-1 was a
        possible culprit.

        I'm looking at the data at the following URL. WX0U-1 and
        WX0U-2 came up often as possible suspects. If it's a digi
        causing the delays, you might be able to spot it on RF if
        you watch long enough. If it's an IGate causing them,
        looking somewhere like aprs.fi can yield clues... e.g.
        a packet that went through a particular digi and was
        gated by two different IGates but with a delay, would
        point at the second digi as being the cause, as both would
        have heard the digi at the same time. When the delayed
        packet shows up but via a different digi path, it would
        take some more digging to prove if a digi caused the delay
        or if it was caused at the IGate level.

        http://aprs.fi/?c=raw&call=KC0U-8&limit=1000

        73 es cul - Keith VE7GDH
        www.ui-view.org
        --
        "I may be lost, but I know exactly where I am!"

      • Lynn W Deffenbaugh (Mr)
        I plan to provide some sort of periodic execution of s in the upcoming port renovations, specifically for this purpose. Lynn (D) - KJ4ERJ -
        Message 3 of 16 , Dec 27, 2012
          I plan to provide some sort of periodic execution of <Open/CloseCmd>s in the upcoming port renovations, specifically for this purpose.

          Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32

          On 12/27/2012 5:34 PM, Steve Daniels wrote:

          Is it possible for APRSIS to command a KPC-3+ to restart to clear the delay, am thinking here that if there is Lynn could perhaps code a restart system. That cycles the KPC in the early hours of the morning each day/week.

          I realise this is slightly off topic

           

          Steve Daniels

          Amateur Radio Callsign G6UIM

          Torbay Freecycle  Owner

          http://uk.groups.yahoo.com/group/torbay_freecycle

          APRSISCE/32 Beta tester and WIKI editor http://aprsisce.wikidot.com

           


          From: aprsisce@yahoogroups.com [mailto:aprsisce@yahoogroups.com ] On Behalf Of Keith VE7GDH
          Sent: 27 December 2012 21:57
          To: aprsisce@yahoogroups.com
          Subject: Re: [aprsisce] APRSIS storing delayed duplicates?

           

           

          Warren KC0U wrote...

          > None of the digipeaters used in my trip home are using a
          > KPC-3+, so I don't think that is the problem.

          A KPC3+ per se isn't a problem with delayed packets... e.g.
          a stand-alone digipeater. It's when it's used in KISS mode with
          an APRS client that it's a problem. After a period of time, it
          starts delaying the output on the serial port.

          > Perhaps I am using the wrong terminology, as the website aprs.fi
          > is obviously using stored information to reproduce my track. Is
          > aprs.fi not part of the APRSIS network?

          No, it isn't. The website aprs.fi gets data from the APRS-IS. The
          APRS-IS gets data from IGates and shares that data between all
          the servers that make up the APRS-IS, and clients such as UI-View,
          APRSISCE/32 etc. can if they wish get data from the APRS-IS.
          The site aprs.fi does store data, but it gets it from the APRS-IS.

          The APRS network consists of the stations themselves, the
          digipeaters, and the IGates. Whatever the IGates hear, they "gate"
          to whatever server they are connected to. All the servers share
          the data between themselves. Clients can connect to one of
          the servers and get some or all of the data from the APRS
          network. Sits like findu.com, aprs.fi, db0anf.de etc. can
          connect to the APRS-IS and get whatever data they want.
          The three I named would likely have a "full feed" from the
          APRS-IS... i.e. everything that was gated but not thrown
          out because it was a duplicate. The APRS-IS uses 30 second
          dupe checking, so packets delayed by less than 30 seconds
          won't show up. However, copies of a packet delayed by more
          than 30 seconds will show up on places like aprs.fi...
          or in your own APRS client if you were getting a feed
          from the APRS-IS.

          There are two likely causes of delayed packets.

          One would be a digi holding onto packets and later digipeating
          them... e.g. the digi could be in a location where the frequency is
          VERY busy, and it might have to wait a while before there was
          no activity before sending a digipeated packet... or the digi could
          be using squelched audio and the squelch could be on the hairy
          edge. It's better if the TNC sees unsquelched audio from the
          radio and it determines itself if it is receiving noise or data.

          The other common cause of delays are the IGates themselves.
          It could be one on a very under-powered computer with a
          slow CPU and not enough RAM, but I would think a more
          likely cause at an IGate would be a KPC3+ running in KISS
          mode. It's a known problem. UI-View is capable of restarting
          itself on a scheduled basis. This takes the TNC out of KISS
          mode and puts it back into KISS mode when it starts up
          again. There could be other reasons why an operator would
          like to restart it, but an added benefit of this is that it also
          restarts the TNC.

          There isn't anything you can do about delayed packets...
          except track down the cause(s) and notify the operators
          so they at least know about the problem. Some won't
          know what you are talking about and will tell you that
          "their station/digi or whatever has been operating for
          years without any problems" and others will look into it
          and hopefully take some kind of action. It's sometimes
          easier to track down the cause by watching / listening on
          RF, but viewing raw data at e.g. aprs.fi can sometimes
          lead to clues... e.g.

          2012-12-27 20:03:27 UTC:
          KC0U-8>APOTC1,WIDE1-1,qAR,KC0U-1
          :!/;5XK63m6> st

          KC0U-8 was heard direct and gated by your KCOU-1. No
          problems so far. Then about 9 minutes later...

          2012-12-27 20:12:25 UTC:
          KC0U-8>APOTC1,WX0U-1*,qAR,WX0U-2
          :!/;5XK63m6> st [Duplicate position packet]

          This appears to be an identical packet. I suppose it is possible
          that KC0U-8 had not moved and it just happened to have
          exactly the same location is the earlier beacon, but given
          that other beacons before and after make it look like it
          was moving, a more likely explanation is that either your
          own WXOU-1 spat out the digipeated copy, but with a delay,
          or... it was delayed by the nearby WX0U-2. That was just
          one example. In another I looked at, WX0U-1 was a
          possible culprit.

          I'm looking at the data at the following URL. WX0U-1 and
          WX0U-2 came up often as possible suspects. If it's a digi
          causing the delays, you might be able to spot it on RF if
          you watch long enough. If it's an IGate causing them,
          looking somewhere like aprs.fi can yield clues... e.g.
          a packet that went through a particular digi and was
          gated by two different IGates but with a delay, would
          point at the second digi as being the cause, as both would
          have heard the digi at the same time. When the delayed
          packet shows up but via a different digi path, it would
          take some more digging to prove if a digi caused the delay
          or if it was caused at the IGate level.

          http://aprs.fi/?c=raw&call=KC0U-8&limit=1000

          73 es cul - Keith VE7GDH
          www.ui-view.org
          --
          "I may be lost, but I know exactly where I am!"


        • James Ewen
          Further analysis and evidence... *05:56:10 UTC: AUBURN APNW01,TCPIP*,qAC,CORE-2:@280556z* *05:56:13.132 AUBURN APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280556z*
          Message 4 of 16 , Dec 27, 2012
            Further analysis and evidence...

            05:56:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280556z
            05:56:13.132  AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280556z
            05:56:14.204  AUBURN>APNW01,MATFLD*,KC0WNY*,qAR,KC0U-1:@280556z
            05:59:37 UTC: AUBURN>APNW01,WIDE2-2,qAR,KC0BS:@280556z (delayed by 3:26)
            06:00:41 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280546z (delayed by 14:31)
            06:01:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280601z
            06:01:13.549  AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280601z
            06:01:14.513  AUBURN>APNW01,MATFLD*,KC0WNY*,qAR,KC0U-1:@280601z
            06:03:25 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280551z (delayed by 12:15)
            06:03:53 UTC: AUBURN>APNW01,WIDE2-1,qAR,KC0BS:@280601z (delayed by 2:43)
            06:06:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280606z
            06:06:13.664  AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280606z
            06:06:14.657  AUBURN>APNW01,MATFLD*,KC0WNY*,qAR,KC0U-1:@280606z
            06:08:19 UTC: AUBURN>APNW01,WIDE2-2,qAR,W0BZN:@280606z (delayed by 2:09)
            06:09:03 UTC: AUBURN>APNW01,W0BZN*,WA0D-1*,qAR,W0BZN:@280606z (delayed 2:53)
            06:09:44 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280556z (delayed by 13:34)
            06:11:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280611z
            06:11:13.865  AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280611z
            06:11:14.734  AUBURN>APNW01,MATFLD*,KC0WNY*,qAR,KC0U-1:@280611z
            06:11:49 UTC: AUBURN>APNW01,N0KTA*,qAR,W0BZN:@280611z (delayed by 39 seconds)
            06:12:47 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280601z (delayed by 11:37)
            06:15:25 UTC: AUBURN>APNW01,LAW*,WIDE2-1,qAR,KC0BS:@280611z (delayed by 4:15)
            06:16:11 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280616z
            06:16:13 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280606z (delayed by 10:02)
            06:17:56 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280611z (delayed by 6:45)

            The blue packets are injected directly into the APRS-IS stream CORE-2 server by AUBURN. This tells us exactly when the packet was generated. Within a second that packet would be on the air.

            The green packets are what KC0U-1 heard on RF. It can not hear AUBURN directly, but there are 2 digipeaters in range that are handling the packet. MATFLD handles the packet about 3 seconds after the packet enters the RF network (There may be a slight clock discrepancy between the APRS-IS servers and the timestamp from KC0U-1, but it is a consistent discrepancy) Almost 1 second later the KV0WNY digipeater handles the packet. These packets will not show up on the filtered APRS-IS feed because the duplicate filters drop the packets. These packets are being pulled from a non-filtered feed, and are therefore available for analysis.

            Any packets in red are packets that sat in a KPC-3 waiting to be delivered to the attached computer, and then on to the APRS-IS servers. Because the delays were longer than the 30 second dupe filter, these packets get accepted as valid packets even though the timestamp shows HUGE delays. 

            05:56:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280556z
            05:56:13.132  AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280556z
            06:09:44 UTC: AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,WX0U-2:@280556z (delayed by 13:34)

            We can see direct evidence here that MATFLD is not to blame as KC0U-1 heard MATFLD digipeat the packet on RF 3 seconds after it was injected into the APRS-IS stream by the source station. 13 minutes and 34 seconds later WX0U-2 finally gets around to sending the packet it heard MATFLD digipeat through to the APRS-IS servers.

            This is as blatant evidence as you can possibly find. We have the original packet sent to the APRS-IS stream, the packet heard nearly immediately on RF, and the delayed packet ultimately sent to the APRS-IS servers many minutes later. All timestamped to show the delivery times.

            James
            VE6SRV
          • Keith VE7GDH
            James VE6SRV wrote... ... Sorry, but I view all of my email i n plain yext format, but the timestamps work for me. However, why isn t the seconf copy, digi d
            Message 5 of 16 , Dec 28, 2012
              James VE6SRV wrote...

              > Further analysis and evidence...
              >
              > *05:56:10 UTC: AUBURN>APNW01,TCPIP*,qAC,CORE-2:@280556z*
              > *05:56:13.132 AUBURN>APNW01,MATFLD*,WIDE2-1,qAR,KC0U-1:@280556z*
              > *05:56:14.204 AUBURN>APNW01,MATFLD*,KC0WNY*,qAR,KC0U-1:@280556z*
              > *05:59:37 UTC: AUBURN>APNW01,WIDE2-2,qAR,KC0BS:@280556z
              > (delayed by 3:26)*

              > The blue packets are injected directly into the APRS-IS stream
              > CORE-2 server by AUBURN. This tells us exactly when the
              > packet was generated. Within a second that packet would be
              > on the air.

              Sorry, but I view all of my email i n plain yext format, but the
              timestamps work for me. However, why isn't the seconf copy,
              digi'd by MATFLD and gated just 3 seconds later, discaded as
              a dupe?

              > The green packets are what KC0U-1 heard on RF. It can not
              > hear AUBURN directly, but there are 2 digipeaters in range
              > that are handling the packet. MATFLD handles the packet
              > about 3 seconds after the packet enters the RF network (There
              > may be a slight clock discrepancy between the APRS-IS
              > servers and the timestamp from KC0U-1, but it is a consistent
              > discrepancy) Almost 1 second later the KV0WNY digipeater
              < handles the packet. These packets will not show up on the
              > filtered APRS-IS feed because the duplicate filters drop the
              > packets. These packets are being pulled from a non-filtered
              > feed, and are therefore available for analysis.

              Ah... I didn't know you could get an unfiltered feed.

              > Any packets in red are packets that sat in a KPC-3 waiting to be
              > delivered to the attached computer...

              One possible explanation, but a likely one.

              > and then on to the APRS-IS servers. Because the delays were longer
              > than the 30 second dupe filter, these packets get accepted as valid
              > packets even though the timestamp shows HUGE delays.

              The timestamps sure make it easy to spot delayed packets.

              73 es cul - Keith VE7GDH
              www.ui-view.org
              --
              "I may be lost, but I know exactly where I am!"
            Your message has been successfully submitted and would be delivered to recipients shortly.