Loading ...
Sorry, an error occurred while loading the content.

Re: [dxatlas] Skim Server and Master.dta?

Expand Messages
  • Alex, VE3NEA
    Hi Lee, I thought about this but could not find an acceptable solution. If Validation is set to Normal, almost every call is posted on the network when it
    Message 1 of 9 , Feb 1, 2011
    • 0 Attachment
      Hi Lee,

      I thought about this but could not find an acceptable solution. If
      Validation is set to Normal, almost every call is posted on the network when
      it first meets the Normal validation criteria. A few seconds later, when
      more data are received, the call may meet a higher validation level, but
      since it has already been posted, is will not be posted again. I have
      received tons of complaints about the same callsign posted multiple times,
      and had to enforce the 10-minute rule. The net result is, all spots, with a
      few exceptions, will have the Normal validation level.

      If your goal is to filter out possible errors after the aggregation of the
      spots from multiple Skimmers, then one simple solution is to publish only
      the callsigns reported by two or more Skimmers.

      73 Alex VE3NEA






      ----- Original Message -----

      Alex would it be possible to include a number in the Skimmer spots comment's
      field that lets users know what level of call verification is being used?
      It could be just before the CQ. For instance aggressive mode could be say
      "3" and normal "1".

      Lee VE7CC
    • Pete Smith
      Ah, I m glad to see this topic surfacing. Working with multiple Skimmers feeding the RBN, we ve encountered a rather different problem. Here s a summary from
      Message 2 of 9 , Feb 1, 2011
      • 0 Attachment
        Ah, I'm glad to see this topic surfacing. Working with multiple
        Skimmers feeding the RBN, we've encountered a rather different problem.

        Here's a summary from the N1MM developers group, in response to a
        complaint from N2IC about the plethora of bad spots he encountered over
        the weekend in the CQ 160 CW test:

        "[4:07:09 PM | Edited 4:07:49 PM] Pete Smith: I think I see the problem
        now, and as Steve says, it depends on your definition of error rate. If
        I watch the output of one skimmer, I see lots of spots and a very low
        error rate. But if you watch the whole RBN or even a sub-set (like the
        half-dozen Skimmers in PA, MD, VA, and WV) then the effective error rate
        is much higher, because the errors make it through while the good spots
        either come up gray (if already worked) or are duped out before they
        ever make it to the bandmap. This makes it a particularly bad problem
        late in a contest, when most of the legitimate stations have been worked
        but the op must still look at every spot to make sure it is a miscopy.

        [6:17:55 AM] Pete Smith: I will bring this to the people involved who
        actually know something (Felipe, Nick and Dave) and see if we can come
        up with a filter for the RBN that attempts to identify these situations
        and filter out the junk. It'll take some doing, but should be do-able.
        One simple-minded way would be to look at spots coming through within a
        short time window on the same frequency, and if there are spots that
        differ from the callsign that the majority are spotting correctly on
        that frequency, don't send them out."

        Kind of a voting system, I guess. Anyhow, the RBN solution aside. might
        it be possible to do the same thing in Skimmer so that folks like Lee
        who prefer to collect their Skimmer spots themselves. Or do you have a
        better idea, Alex?

        73, Pete N4ZR

        The World Contest Station Database, updated daily at www.conteststations.com
        The Reverse Beacon Network at http://reversebeacon.net, blog at reversebeacon.blogspot.com,
        spots at telnet.reversebeacon.net, port 7000



        On 2/1/2011 9:43 AM, Alex, VE3NEA wrote:
        > Hi Lee,
        >
        > I thought about this but could not find an acceptable solution. If
        > Validation is set to Normal, almost every call is posted on the network when
        > it first meets the Normal validation criteria. A few seconds later, when
        > more data are received, the call may meet a higher validation level, but
        > since it has already been posted, is will not be posted again. I have
        > received tons of complaints about the same callsign posted multiple times,
        > and had to enforce the 10-minute rule. The net result is, all spots, with a
        > few exceptions, will have the Normal validation level.
        >
        > If your goal is to filter out possible errors after the aggregation of the
        > spots from multiple Skimmers, then one simple solution is to publish only
        > the callsigns reported by two or more Skimmers.
        >
        > 73 Alex VE3NEA
        >
        >
        >
        >
        >
        >
        > ----- Original Message -----
        >
        > Alex would it be possible to include a number in the Skimmer spots comment's
        > field that lets users know what level of call verification is being used?
        > It could be just before the CQ. For instance aggressive mode could be say
        > "3" and normal "1".
        >
        > Lee VE7CC
        >
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
      • Alex, VE3NEA
        Hi Pete, As I have already written, a program that receives spots from multiple Skimmers can radically reduce the number of errors simply by showing only the
        Message 3 of 9 , Feb 1, 2011
        • 0 Attachment
          Hi Pete,

          As I have already written, a program that receives spots from multiple
          Skimmers can radically reduce the number of errors simply by showing only
          the spots received from two or more Skimmers. This is the same principle
          that is used for Aggressive filtering in CW Skimmer, except that instead of
          waiting for more copies of the callsign decoded by the same Skimmer, we
          would get those copies instantly from other Skimmers. In other words,
          instead of time diversity we would use spatial diversity.

          73 Alex VE3NEA





          ----- Original Message -----
          From: "Pete Smith" <n4zr@...>
          To: <dxatlas_group@yahoogroups.com>
          Cc: "Alex, VE3NEA" <alshovk@...>
          Sent: Tuesday, February 01, 2011 11:44 AM
          Subject: Re: [dxatlas] Skim Server and Master.dta?


          > Ah, I'm glad to see this topic surfacing. Working with multiple Skimmers
          > feeding the RBN, we've encountered a rather different problem.
          >
          > Here's a summary from the N1MM developers group, in response to a
          > complaint from N2IC about the plethora of bad spots he encountered over
          > the weekend in the CQ 160 CW test:
          >
          > "[4:07:09 PM | Edited 4:07:49 PM] Pete Smith: I think I see the problem
          > now, and as Steve says, it depends on your definition of error rate. If I
          > watch the output of one skimmer, I see lots of spots and a very low error
          > rate. But if you watch the whole RBN or even a sub-set (like the
          > half-dozen Skimmers in PA, MD, VA, and WV) then the effective error rate
          > is much higher, because the errors make it through while the good spots
          > either come up gray (if already worked) or are duped out before they ever
          > make it to the bandmap. This makes it a particularly bad problem late in
          > a contest, when most of the legitimate stations have been worked but the
          > op must still look at every spot to make sure it is a miscopy.
          >
          > [6:17:55 AM] Pete Smith: I will bring this to the people involved who
          > actually know something (Felipe, Nick and Dave) and see if we can come up
          > with a filter for the RBN that attempts to identify these situations and
          > filter out the junk. It'll take some doing, but should be do-able. One
          > simple-minded way would be to look at spots coming through within a short
          > time window on the same frequency, and if there are spots that differ
          > from the callsign that the majority are spotting correctly on that
          > frequency, don't send them out."
          >
          > Kind of a voting system, I guess. Anyhow, the RBN solution aside. might
          > it be possible to do the same thing in Skimmer so that folks like Lee who
          > prefer to collect their Skimmer spots themselves. Or do you have a better
          > idea, Alex?
          >
          > 73, Pete N4ZR
          >
          > The World Contest Station Database, updated daily at
          > www.conteststations.com
          > The Reverse Beacon Network at http://reversebeacon.net, blog at
          > reversebeacon.blogspot.com,
          > spots at telnet.reversebeacon.net, port 7000
          >
          >
          >
          > On 2/1/2011 9:43 AM, Alex, VE3NEA wrote:
          >> Hi Lee,
          >>
          >> I thought about this but could not find an acceptable solution. If
          >> Validation is set to Normal, almost every call is posted on the network
          >> when
          >> it first meets the Normal validation criteria. A few seconds later, when
          >> more data are received, the call may meet a higher validation level, but
          >> since it has already been posted, is will not be posted again. I have
          >> received tons of complaints about the same callsign posted multiple
          >> times,
          >> and had to enforce the 10-minute rule. The net result is, all spots, with
          >> a
          >> few exceptions, will have the Normal validation level.
          >>
          >> If your goal is to filter out possible errors after the aggregation of
          >> the
          >> spots from multiple Skimmers, then one simple solution is to publish only
          >> the callsigns reported by two or more Skimmers.
          >>
          >> 73 Alex VE3NEA
          >>
          >>
        • Rich Hallman - N7TR
          Im new to the skimmer group, but at least having the option to use a master.dta or some other verification file would be nice. Not all would be forced to
          Message 4 of 9 , Feb 1, 2011
          • 0 Attachment
            Im new to the skimmer group, but at least having the option to use a
            master.dta or some other verification file would be nice. Not all
            would be forced to using it, but for those like me, could turn that
            option on when needed (Manage Error Rates). I know part of this
            functionality is in CW Skimmer.....so would be nice to have the same
            option in server.



            I have just the N7TR skimmer spots showing up on my ARC4 cluster and do
            not propagate any further then to my node. Also feed the RBN Network.
            Users so far seem to like this approach....



            Telnet to dxc.n7tr.com Port 23



            Thanks....

            Rich







            From: dxatlas_group@yahoogroups.com
            [mailto:dxatlas_group@yahoogroups.com] On Behalf Of Pete Smith
            Sent: Tuesday, February 01, 2011 8:44 AM
            To: dxatlas_group@yahoogroups.com
            Cc: Alex, VE3NEA
            Subject: Re: [dxatlas] Skim Server and Master.dta?





            Ah, I'm glad to see this topic surfacing. Working with multiple
            Skimmers feeding the RBN, we've encountered a rather different problem.

            Here's a summary from the N1MM developers group, in response to a
            complaint from N2IC about the plethora of bad spots he encountered over
            the weekend in the CQ 160 CW test:

            "[4:07:09 PM | Edited 4:07:49 PM] Pete Smith: I think I see the problem
            now, and as Steve says, it depends on your definition of error rate. If
            I watch the output of one skimmer, I see lots of spots and a very low
            error rate. But if you watch the whole RBN or even a sub-set (like the
            half-dozen Skimmers in PA, MD, VA, and WV) then the effective error rate

            is much higher, because the errors make it through while the good spots
            either come up gray (if already worked) or are duped out before they
            ever make it to the bandmap. This makes it a particularly bad problem
            late in a contest, when most of the legitimate stations have been worked

            but the op must still look at every spot to make sure it is a miscopy.

            [6:17:55 AM] Pete Smith: I will bring this to the people involved who
            actually know something (Felipe, Nick and Dave) and see if we can come
            up with a filter for the RBN that attempts to identify these situations
            and filter out the junk. It'll take some doing, but should be do-able.
            One simple-minded way would be to look at spots coming through within a
            short time window on the same frequency, and if there are spots that
            differ from the callsign that the majority are spotting correctly on
            that frequency, don't send them out."

            Kind of a voting system, I guess. Anyhow, the RBN solution aside. might
            it be possible to do the same thing in Skimmer so that folks like Lee
            who prefer to collect their Skimmer spots themselves. Or do you have a
            better idea, Alex?

            73, Pete N4ZR

            The World Contest Station Database, updated daily at
            www.conteststations.com
            The Reverse Beacon Network at http://reversebeacon.net, blog at
            reversebeacon.blogspot.com,
            spots at telnet.reversebeacon.net, port 7000

            On 2/1/2011 9:43 AM, Alex, VE3NEA wrote:
            > Hi Lee,
            >
            > I thought about this but could not find an acceptable solution. If
            > Validation is set to Normal, almost every call is posted on the
            network when
            > it first meets the Normal validation criteria. A few seconds later,
            when
            > more data are received, the call may meet a higher validation level,
            but
            > since it has already been posted, is will not be posted again. I have
            > received tons of complaints about the same callsign posted multiple
            times,
            > and had to enforce the 10-minute rule. The net result is, all spots,
            with a
            > few exceptions, will have the Normal validation level.
            >
            > If your goal is to filter out possible errors after the aggregation of
            the
            > spots from multiple Skimmers, then one simple solution is to publish
            only
            > the callsigns reported by two or more Skimmers.
            >
            > 73 Alex VE3NEA
            >
            >
            >
            >
            >
            >
            > ----- Original Message -----
            >
            > Alex would it be possible to include a number in the Skimmer spots
            comment's
            > field that lets users know what level of call verification is being
            used?
            > It could be just before the CQ. For instance aggressive mode could be
            say
            > "3" and normal "1".
            >
            > Lee VE7CC
            >
            >
            >
            > ------------------------------------
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
            >





            [Non-text portions of this message have been removed]
          • Alex, VE3NEA
            Hi Rich, Skimmer Server is designed to skim 7 bands simultaneously, and its code is optimized for decoding as many concurrent CW signals as possible. I had to
            Message 5 of 9 , Feb 1, 2011
            • 0 Attachment
              Hi Rich,

              Skimmer Server is designed to skim 7 bands simultaneously, and its code is
              optimized for decoding as many concurrent CW signals as possible. I had to
              rewrite a number of procedures in Assembler, using the SSE instructions, in
              order to save some precious CPU cycles. It would be a shame to waste those
              hard-earned cycles on searching the master.dta file, something that is not
              part of skimming and that can be easily done elsewhere, e.g., at the cluster
              node or even in the client software.

              73 Alex VE3NEA





              ----- Original Message -----
              From: "Rich Hallman - N7TR" <rich@...>
              To: <dxatlas_group@yahoogroups.com>
              Cc: "Alex, VE3NEA" <alshovk@...>
              Sent: Tuesday, February 01, 2011 12:49 PM
              Subject: RE: [dxatlas] Skim Server and Master.dta?


              > Im new to the skimmer group, but at least having the option to use a
              > master.dta or some other verification file would be nice. Not all
              > would be forced to using it, but for those like me, could turn that
              > option on when needed (Manage Error Rates). I know part of this
              > functionality is in CW Skimmer.....so would be nice to have the same
              > option in server.
              >
              >
              >
              > I have just the N7TR skimmer spots showing up on my ARC4 cluster and do
              > not propagate any further then to my node. Also feed the RBN Network.
              > Users so far seem to like this approach....
              >
              >
              >
              > Telnet to dxc.n7tr.com Port 23
              >
              >
              >
              > Thanks....
              >
              > Rich
              >
              >
              >
              >
              >
              >
              >
              > From: dxatlas_group@yahoogroups.com
              > [mailto:dxatlas_group@yahoogroups.com] On Behalf Of Pete Smith
              > Sent: Tuesday, February 01, 2011 8:44 AM
              > To: dxatlas_group@yahoogroups.com
              > Cc: Alex, VE3NEA
              > Subject: Re: [dxatlas] Skim Server and Master.dta?
              >
              >
              >
              >
              >
              > Ah, I'm glad to see this topic surfacing. Working with multiple
              > Skimmers feeding the RBN, we've encountered a rather different problem.
              >
              > Here's a summary from the N1MM developers group, in response to a
              > complaint from N2IC about the plethora of bad spots he encountered over
              > the weekend in the CQ 160 CW test:
              >
              > "[4:07:09 PM | Edited 4:07:49 PM] Pete Smith: I think I see the problem
              > now, and as Steve says, it depends on your definition of error rate. If
              > I watch the output of one skimmer, I see lots of spots and a very low
              > error rate. But if you watch the whole RBN or even a sub-set (like the
              > half-dozen Skimmers in PA, MD, VA, and WV) then the effective error rate
              >
              > is much higher, because the errors make it through while the good spots
              > either come up gray (if already worked) or are duped out before they
              > ever make it to the bandmap. This makes it a particularly bad problem
              > late in a contest, when most of the legitimate stations have been worked
              >
              > but the op must still look at every spot to make sure it is a miscopy.
              >
              > [6:17:55 AM] Pete Smith: I will bring this to the people involved who
              > actually know something (Felipe, Nick and Dave) and see if we can come
              > up with a filter for the RBN that attempts to identify these situations
              > and filter out the junk. It'll take some doing, but should be do-able.
              > One simple-minded way would be to look at spots coming through within a
              > short time window on the same frequency, and if there are spots that
              > differ from the callsign that the majority are spotting correctly on
              > that frequency, don't send them out."
              >
              > Kind of a voting system, I guess. Anyhow, the RBN solution aside. might
              > it be possible to do the same thing in Skimmer so that folks like Lee
              > who prefer to collect their Skimmer spots themselves. Or do you have a
              > better idea, Alex?
              >
              > 73, Pete N4ZR
              >
              > The World Contest Station Database, updated daily at
              > www.conteststations.com
              > The Reverse Beacon Network at http://reversebeacon.net, blog at
              > reversebeacon.blogspot.com,
              > spots at telnet.reversebeacon.net, port 7000
              >
              > On 2/1/2011 9:43 AM, Alex, VE3NEA wrote:
              >> Hi Lee,
              >>
              >> I thought about this but could not find an acceptable solution. If
              >> Validation is set to Normal, almost every call is posted on the
              > network when
              >> it first meets the Normal validation criteria. A few seconds later,
              > when
              >> more data are received, the call may meet a higher validation level,
              > but
              >> since it has already been posted, is will not be posted again. I have
              >> received tons of complaints about the same callsign posted multiple
              > times,
              >> and had to enforce the 10-minute rule. The net result is, all spots,
              > with a
              >> few exceptions, will have the Normal validation level.
              >>
              >> If your goal is to filter out possible errors after the aggregation of
              > the
              >> spots from multiple Skimmers, then one simple solution is to publish
              > only
              >> the callsigns reported by two or more Skimmers.
              >>
              >> 73 Alex VE3NEA
              >>
              >>
              >>
              >>
              >>
              >>
              >> ----- Original Message -----
              >>
              >> Alex would it be possible to include a number in the Skimmer spots
              > comment's
              >> field that lets users know what level of call verification is being
              > used?
              >> It could be just before the CQ. For instance aggressive mode could be
              > say
              >> "3" and normal "1".
              >>
              >> Lee VE7CC
              >>
              >>
              >>
              >> ------------------------------------
              >>
              >> Yahoo! Groups Links
              >>
              >>
              >>
              >>
              >>
              >
              >
              >
              >
              >
              > [Non-text portions of this message have been removed]
              >
              >
            • Rich Hallman - N7TR
              Thanks for the update Alex and makes sense..... Rich From: dxatlas_group@yahoogroups.com [mailto:dxatlas_group@yahoogroups.com] On Behalf Of Alex, VE3NEA Sent:
              Message 6 of 9 , Feb 1, 2011
              • 0 Attachment
                Thanks for the update Alex and makes sense.....



                Rich



                From: dxatlas_group@yahoogroups.com
                [mailto:dxatlas_group@yahoogroups.com] On Behalf Of Alex, VE3NEA
                Sent: Tuesday, February 01, 2011 10:13 AM
                To: dxatlas_group@yahoogroups.com
                Subject: Re: [dxatlas] Skim Server and Master.dta?





                Hi Rich,

                Skimmer Server is designed to skim 7 bands simultaneously, and its code
                is
                optimized for decoding as many concurrent CW signals as possible. I had
                to
                rewrite a number of procedures in Assembler, using the SSE instructions,
                in
                order to save some precious CPU cycles. It would be a shame to waste
                those
                hard-earned cycles on searching the master.dta file, something that is
                not
                part of skimming and that can be easily done elsewhere, e.g., at the
                cluster
                node or even in the client software.

                73 Alex VE3NEA

                ----- Original Message -----
                From: "Rich Hallman - N7TR" <rich@... <mailto:rich%40n7tr.com> >
                To: <dxatlas_group@yahoogroups.com
                <mailto:dxatlas_group%40yahoogroups.com> >
                Cc: "Alex, VE3NEA" <alshovk@... <mailto:alshovk%40dxatlas.com> >
                Sent: Tuesday, February 01, 2011 12:49 PM
                Subject: RE: [dxatlas] Skim Server and Master.dta?

                > Im new to the skimmer group, but at least having the option to use a
                > master.dta or some other verification file would be nice. Not all
                > would be forced to using it, but for those like me, could turn that
                > option on when needed (Manage Error Rates). I know part of this
                > functionality is in CW Skimmer.....so would be nice to have the same
                > option in server.
                >
                >
                >
                > I have just the N7TR skimmer spots showing up on my ARC4 cluster and
                do
                > not propagate any further then to my node. Also feed the RBN Network.
                > Users so far seem to like this approach....
                >
                >
                >
                > Telnet to dxc.n7tr.com Port 23
                >
                >
                >
                > Thanks....
                >
                > Rich
                >
                >
                >
                >
                >
                >
                >
                > From: dxatlas_group@yahoogroups.com
                <mailto:dxatlas_group%40yahoogroups.com>
                > [mailto:dxatlas_group@yahoogroups.com
                <mailto:dxatlas_group%40yahoogroups.com> ] On Behalf Of Pete Smith
                > Sent: Tuesday, February 01, 2011 8:44 AM
                > To: dxatlas_group@yahoogroups.com
                <mailto:dxatlas_group%40yahoogroups.com>
                > Cc: Alex, VE3NEA
                > Subject: Re: [dxatlas] Skim Server and Master.dta?
                >
                >
                >
                >
                >
                > Ah, I'm glad to see this topic surfacing. Working with multiple
                > Skimmers feeding the RBN, we've encountered a rather different
                problem.
                >
                > Here's a summary from the N1MM developers group, in response to a
                > complaint from N2IC about the plethora of bad spots he encountered
                over
                > the weekend in the CQ 160 CW test:
                >
                > "[4:07:09 PM | Edited 4:07:49 PM] Pete Smith: I think I see the
                problem
                > now, and as Steve says, it depends on your definition of error rate.
                If
                > I watch the output of one skimmer, I see lots of spots and a very low
                > error rate. But if you watch the whole RBN or even a sub-set (like the
                > half-dozen Skimmers in PA, MD, VA, and WV) then the effective error
                rate
                >
                > is much higher, because the errors make it through while the good
                spots
                > either come up gray (if already worked) or are duped out before they
                > ever make it to the bandmap. This makes it a particularly bad problem
                > late in a contest, when most of the legitimate stations have been
                worked
                >
                > but the op must still look at every spot to make sure it is a miscopy.
                >
                > [6:17:55 AM] Pete Smith: I will bring this to the people involved who
                > actually know something (Felipe, Nick and Dave) and see if we can come
                > up with a filter for the RBN that attempts to identify these
                situations
                > and filter out the junk. It'll take some doing, but should be do-able.
                > One simple-minded way would be to look at spots coming through within
                a
                > short time window on the same frequency, and if there are spots that
                > differ from the callsign that the majority are spotting correctly on
                > that frequency, don't send them out."
                >
                > Kind of a voting system, I guess. Anyhow, the RBN solution aside.
                might
                > it be possible to do the same thing in Skimmer so that folks like Lee
                > who prefer to collect their Skimmer spots themselves. Or do you have a
                > better idea, Alex?
                >
                > 73, Pete N4ZR
                >
                > The World Contest Station Database, updated daily at
                > www.conteststations.com
                > The Reverse Beacon Network at http://reversebeacon.net, blog at
                > reversebeacon.blogspot.com,
                > spots at telnet.reversebeacon.net, port 7000
                >
                > On 2/1/2011 9:43 AM, Alex, VE3NEA wrote:
                >> Hi Lee,
                >>
                >> I thought about this but could not find an acceptable solution. If
                >> Validation is set to Normal, almost every call is posted on the
                > network when
                >> it first meets the Normal validation criteria. A few seconds later,
                > when
                >> more data are received, the call may meet a higher validation level,
                > but
                >> since it has already been posted, is will not be posted again. I have
                >> received tons of complaints about the same callsign posted multiple
                > times,
                >> and had to enforce the 10-minute rule. The net result is, all spots,
                > with a
                >> few exceptions, will have the Normal validation level.
                >>
                >> If your goal is to filter out possible errors after the aggregation
                of
                > the
                >> spots from multiple Skimmers, then one simple solution is to publish
                > only
                >> the callsigns reported by two or more Skimmers.
                >>
                >> 73 Alex VE3NEA
                >>
                >>
                >>
                >>
                >>
                >>
                >> ----- Original Message -----
                >>
                >> Alex would it be possible to include a number in the Skimmer spots
                > comment's
                >> field that lets users know what level of call verification is being
                > used?
                >> It could be just before the CQ. For instance aggressive mode could be
                > say
                >> "3" and normal "1".
                >>
                >> Lee VE7CC
                >>
                >>
                >>
                >> ------------------------------------
                >>
                >> Yahoo! Groups Links
                >>
                >>
                >>
                >>
                >>
                >
                >
                >
                >
                >
                > [Non-text portions of this message have been removed]
                >
                >





                [Non-text portions of this message have been removed]
              Your message has been successfully submitted and would be delivered to recipients shortly.