Re: [ARRL-LOTW] Duplicates
- I do know a little something about statistical analysis. That's why I asked. You don't have any usable data. First the 1700 hourly samples have no value because it is impossible to know what is going on between the hours. You get only 1 sample out of 3600 possible samples. Last I checked that's far to few to be of any value. You know 60 times 60 is 3600 absolute samples(seconds). Now that the system is for the most part processing data faster than it is being sent. There is no way it accurately tell how much raw data is being processed. From what I see, the LoTW system is only being loaded 5-10% of the time the numbers are showing a dupe rate of somewhere near 7% +/- 3 points.Thanks again,KeithFrom: "Joe Subich, W4TV" <lists@...>
Sent: Tuesday, February 19, 2013 3:37 PM
Subject: Re: [ARRL-LOTW] Duplicates
I am using average number of QSOs/per upload times the number of files
processed. Simple sampling based statistics familiar to anyone who
has had a basic course in statistical analysis.
There now are over 1700 hourly samples in the LotW queue status reports
on which to base the average number of QSOs per upload. The cumulative
number of files processed is available on the LotW home page. There
are more than enough samples to make any sampling error insignificant
in the analysis.
... Joe, W4TV
On 2/19/2013 4:08 PM, Pete Ennis wrote:
> Tell us where you are getting the exact number of raw QSO's that are being uploaded???? The ones that include the dupes and new/ modified QSOs???
> From: "Joe Subich, W4TV" mailto:lists%40subich.com>
> To: mailto:ARRL-LOTW%40yahoogroups.com
> Cc: VE6LB mailto:ve6lb%40telus.net>
> Sent: Tuesday, February 19, 2013 1:45 PM
> Subject: [ARRL-LOTW] Duplicates (was: Re: User Friendly LoTW)
> On 2/19/2013 12:04 PM, VE6LB wrote:
>> Now that processing is lightning fast, I wonder how many duplicate uploads happen.
>> Gerry VE6LB
> The duplicate rate is, if anything, greater than it was before the new
> hardware went on-line. In the last four weeks, the number of uploads
> per day is running about 58% ahead of the rate prior to the new hardware
> while the umber of new QSO records per day is only up 15%. That means
> the rate of previously processed QSOs has increased by 28% over what it
> was during the period immediately before the new hardware went on-line
> (as the average number of QSOs/log in the queue has remained nearly
> A 28% increase in the previously processed QSO rate - based on K1MK's
> statement that "half the QSOs being processed were duplicates" before
> the hardware change - would indicate that nearly 2/3 of all uploads
> are now duplicates. The exact mix will change from day to day - the
> proportion of "new QSO records" is consistently higher on Monday and
> Tuesday after a major contest weekend and is consistently highest on
> Monday/Tuesday after a bigger contest than after some of the less
> popular events.
> In any case, the indications are that many of those who do not use
> logging programs specifically designed to track LotW uploads and
> prevent duplicates continue to export and sign their entire log without
> using tQSL's date range feature.
> ... Joe, W4TV
- I run ACLog... When you hit the "ALL SINCE" button, change the date to
be something about a week prior to the LoTW failure...
I "believe", ACLog got very confused as a result of the fail mode of
LoTW. That corrected a very similar problem for me.
Thanks and 73's,
For equipment, and software setups and reviews see:
for MixW support see;
for Dopplergram information see:
for MM-SSTV see:
On Sun, 2014-08-24 at 09:05 -0700, reillyjf@... [ARRL-LOTW]
> Thanks for the suggestion. I did a complete download, and beat the
> number of duplicates down from 275 to 30. No exactly sure why the
> N3FJP ACL is missing this information.
> - 73, John, N0TA