Loading ...
Sorry, an error occurred while loading the content.

Report on 6M ISCAT-A vs. -B comparisons

Expand Messages
  • Jim and Hannelore Fisher
    VE1SKY and I were delighted to welcome WSJT9.3 and ISCAT-A to the suite. (I would also point out that Joe, K1JT kindly added elevation to the 9.3 Echo software
    Message 1 of 5 , Oct 8, 2012
    • 0 Attachment
      VE1SKY and I were delighted to welcome WSJT9.3 and ISCAT-A to the suite. (I would also point out that Joe, K1JT kindly added elevation to the 9.3 Echo software at my request--very handy for combining multiple days' EME echo runs for spreadsheet analysis of lobes under real conditions.) Joe indicates that ""ISCAT now has two sub-modes. ISCAT-B is the original ISCAT mode with total bandwidth 1809 Hz. ISCAT-A runs at half the rate, uses half the bandwidth, and (for average decodes on steady signals) is about 1 dB more sensitive. It has been found very effective for making aircraft scatter QSOs at 10 GHz. ISCAT-B probably remains the best mode for 6 meters, but experimentation is encouraged!"
       
      Happy to oblige, we decided to conduct some on-the-air comparisons with some other stations ranging from several hundred miles away to over 1000 to begin to get a feel for how the two compare under what conditions. When both of us were available, we alternated 30sec intervals and sent ISCAT-A solidly for minutes ending 1,2,3, and 4, and ISCAT-B for minutes 6,7,8,9. Minutes ending 0 and 5 were for both ends to switch between -A and -B. A similar approach was used for others to send to me later. Stations participating with us included N9OLT (significant participation on two different days at a distance of 1070 miles), W3NF, NY2NY, W9NHE, and KD9NH. Receiving stations were to set sync to -20, tolerance to 400, and keep hands off the decoding process so the records were automatic. VE1SKY and I exchanged mutual reports but because we were so close these were not included in the study, although they provided substance to a question about S/N reports discussed below. The original design was that in addition to concentrating exchanges into relatively short periods with rapid alternation between the two modes, I would put the results on Excel, sort by signal strength and see if there was a difference in decoding results by signal strength.
       
      The first thing to say is I am very impressed with the capabilities of both -A and -B. Thanks, Joe--another triumph! Many of these test sequences were above 1000 miles, and despite the fact that the times were not of high M/S activity, a high percentage of sequences on each decoded with relatively little garbage or garbled results. Watching the incoming signals from others, I noted that there were a number of decodes without visible or audible pings, and some decodes where there was no visible pattern on the screen or a very subdued visible ISCAT pattern that often went on for quite a number of seconds. Airplane scatter would also appear to be a possibility here, as the EU-NA flightpaths cross our horizon above the Bay of Fundy and Gulf of Maine. I would be interested in others' impressions of what propagation modes are likely involved.
       
      I applied some predetermined standards to which all.txt lines to include in the report (resulting in the selection of a total of 87 ISCAT-As and 100 ISCAT-Bs, partly due to when stations started or stopped transmitting or recording). Perfect decodes were valued at 1.0, single errors were valued at 0.5, and everything else was zero. Anyone wishing to receive the Excel summary or the individual all.txt files should email me and it is very likely that someone more advanced technically can derive additional or more accurate results and interpretations.
       
      Based on weighted decodes divided by numbers of sequences, -A scored 59.2% and -B scored 71.5%. This may be what K1JT would have predicted , but I believe he has also mentioned a potential 1dB sensitivity difference in favor of -A, so I was surprised.
       
      However, early on and consistently, it appeared that the ISCAT-A read-out was reporting S/N levels averaging several dBs below those of -B. This led me to question whether -A was fully driving the transmitted signal, but in fact my output meter was showing exactly the same on both. Subsequently, VE1SKY and I pointed antennas at each other and conducted a specific test of S-meter differences and the software's S/N reports and found no discernible shift in S-meters between -A and -B but a consistent 3 to 4 dB (mostly 4dB) difference in the ISCAT-reported S/N reports. This led me to wonder if the -A decoding software was performing correctly, as I had assumed -A would report a more favorable comparison to noise level than the broader -B digital passband. I wrote K1JT about this and he promptly replied:
       
      "Hi Jim,

      "Something may well be amiss with the reported S/N values in ISCAT-A relative to ISCAT-B. The "A" mode was cobbled together rather quickly when the Aussies were doing airplane scatter at 10 GHz, and I don't remember doing any exhaustive tests of the measured parameters. You'd better do your comparative analysis of decodes-vs-SNR treating SNR as consistent within each submode, but possibly not meaningful between modes.

      "The important thing, after all, is whether it decoded or not. Reported signal parameters are of secondary importance.

      "-- Joe, K1JT"
      I still couldn't resist looking at the decode scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the differences in reported S/N values, and as Joe commented, these comparisons between modes may not be valid.
       
      I am looking forward to using ISCAT in next summer's Transatlantic Es openings and may again ask other 6M users to participate in a study. Another potential use is in Transatlantic aurora periods--both VE1SKY and I have been copied in Finland on 6M ISCAT doing a bank shot off the aurora. In the meantime, I encourage others to conduct similar (or probably more technically sophisticated) A/B tests of their own and share their results with the community. If they care to and find it feasible, they would be welcome to combine the data we have collected (probably their own filtering of the raw data) with further collections..
       
      73,
       
      Jim, VE1JF
    • Bruce Brackin
      Jim, Roger, et.al. If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there s no statistical or practical difference.
      Message 2 of 5 , Oct 8, 2012
      • 0 Attachment
        Jim, Roger, et.al.  If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there's no statistical or practical difference.  Given the small sample size (30 and 37), you'd expect fairly wide error bars.  Probably the luck (or unlucky) of the draw.

        Keep up the good work.

        Bruce N5SIX

        "I still couldn't resist looking at the decode scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the differences in reported S/N values, and as Joe commented, these comparisons between modes may not be valid."
         
      • Barry Garratt
        Bruce et al, Actually I wouldn t expect wide errors at all Bruce even with the small sample size. Remember the testing was all being done basically with
        Message 3 of 5 , Oct 8, 2012
        • 0 Attachment

          Bruce et al,

           

          Actually I wouldn’t expect wide errors at all Bruce even with the small sample size. Remember the testing was all being done basically with meteors and the mode isn’t designed for that. Hence both modes performed equally poorly. Conversely FSK441 works very well for MS because that’s what it was designed for. If the tests were run against FSK441 I would expect to see a wide margin between the modes but not between the two Iscat modes.

           

          Interesting tests nonetheless and good documentation Jim.

           

          Barry KS7DX

           

           

          From: wsjtgroup@yahoogroups.com [mailto:wsjtgroup@yahoogroups.com] On Behalf Of Bruce Brackin
          Sent: Monday, October 08, 2012 5:41 PM
          To: Jim and Hannelore Fisher
          Cc: wsjtgroup@yahoogroups.com; Roger Sturtevant
          Subject: Re: [wsjtgroup] Report on 6M ISCAT-A vs. -B comparisons

           

           

          Jim, Roger, et.al.  If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there's no statistical or practical difference.  Given the small sample size (30 and 37), you'd expect fairly wide error bars.  Probably the luck (or unlucky) of the draw.

          Keep up the good work.

          Bruce N5SIX

          "I still couldn't resist looking at the decode scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the differences in reported S/N values, and as Joe commented, these comparisons between modes may not be valid."

           

        • Leigh Rainbird
          G day, I downloaded the new version over the weekend and did some testing with ISCAT B and JT65a on the 6m band. I should have spent some more time on it and
          Message 4 of 5 , Oct 8, 2012
          • 0 Attachment
            G'day,
             
            I downloaded the new version over the weekend and did some testing with ISCAT B and JT65a on the 6m band. I should have spent some more time on it and tried ISCAT A. I may do that still.
             
            But just briefly, I found that ISCAT B worked OK for meteors but not so good for direct, constant type signals like tropo.
            The meteors had to be quite long duration to obtain a full decode with no errors, short pings weren't good enough. The tropo signals needed to become quite strong before any real info could be obtained from the message being sent.
            These tests were with stations 770 and 400 km ranges.
             
            On the shorter 400 km paths, I tried for maybe an hour with VK3SMC and got nothing decoded despite being able to hear the signal by ear at times. This is mainly on tropo. We decided to try JT65a in comparison, and from the very first transmissions we had decoded right away and completed in about 5 mins.
             
            So from this test Ive currently concluded that on 6m paths via E or F layer, JT65a wins hands down over ISCAT.
             
            I will find another station and test ISCAT A and see how it goes for us.
             
            Leigh VK2KRR
             
             
            ----- Original Message -----
            Sent: Tuesday, October 09, 2012 12:52 PM
            Subject: RE: [wsjtgroup] Report on 6M ISCAT-A vs. -B comparisons

             

            Bruce et al,

            Actually I wouldn’t expect wide errors at all Bruce even with the small sample size. Remember the testing was all being done basically with meteors and the mode isn’t designed for that. Hence both modes performed equally poorly. Conversely FSK441 works very well for MS because that’s what it was designed for. If the tests were run against FSK441 I would expect to see a wide margin between the modes but not between the two Iscat modes.

            Interesting tests nonetheless and good documentation Jim.

            Barry KS7DX

            From: wsjtgroup@yahoogroups.com [mailto:wsjtgroup@yahoogroups.com] On Behalf Of Bruce Brackin
            Sent: Monday, October 08, 2012 5:41 PM
            To: Jim and Hannelore Fisher
            Cc: wsjtgroup@yahoogroups.com; Roger Sturtevant
            Subject: Re: [wsjtgroup] Report on 6M ISCAT-A vs. -B comparisons

             

            Jim, Roger, et.al.  If you compute the simple 95% confidence intervals for the proportion of A and B decodes, there's no statistical or practical difference.  Given the small sample size (30 and 37), you'd expect fairly wide error bars.  Probably the luck (or unlucky) of the draw.

            Keep up the good work.

            Bruce N5SIX

            "I still couldn't resist looking at the decode scores (1, 0.5 and 0) for the two modes individually at a reported -20 S/N. ISCAT-A scored 9/30 or 30% and ISCAT-B scored 9.5/37 or 25.7%. I'm still not sure how to interpret this given the differences in reported S/N values, and as Joe commented, these comparisons between modes may not be valid."

          • Jim and Hannelore Fisher
            Some additional thoughts on yesterday s report, prompted by several emails and a conversation with VE1SKY: o My personal focus is not on evaluating FSK441
            Message 5 of 5 , Oct 9, 2012
            • 0 Attachment
              Some additional thoughts on yesterday's report, prompted by several emails and a conversation with VE1SKY:
              o    My personal focus is not on evaluating FSK441 vs. ISCAT. I assume that if M/S pings are all that are going on FSK441 will outperform ISCAT for the distances feasible for M/S. (My impression is that more than one propagation mode was weakly active during these tests.) My 6M goal is DXCC (will complete my 10BDXCC since I just wrapped up my 9th band), and all DXCC entities within M/S reach for me are already worked or are easily worked by other modes. Therefore, having already used JT65A for EME and terrestrial DX and ISCAT-B to work EU, I am seeking to get a feel for how -A and -B perform under different circumstances and be ready for further studies and hopefully a bunch of new countries in EU, AF and maybe AS during the next transatlantic Es season.
              o    VE1SKY and I have both been heard via aurora in EU on ISCAT (-B at the time) and I am hopeful -A will be at least as good (perhaps in part due to greater tolerance for Doppler as well as the additional 1dB sensitivity?) and will result in some additional DX possibilities when the moon is down and Es and F2 are quiet. Using some of the near-real-time aurora images on Internet, I expect us to be able to refine our aiming techniques and understanding and use ON4KST 50-70 to alert potential QSO partners in EU when it may be useful to look for us. As a fellow HF contester used to say, you aren't loud if you aren't on.
              o    Others may wish to test FSK441 vs. ISCAT-A and/or B, but I am particularly hopeful that European stations will continue their gradual acceptance of ISCAT as an improvement over JT6M. Perhaps somebody who considers this still an unresolved issue would like to design and conduct a compressed test rapidly alternating JT6M with ISCAT-A and/or -B during transatlantic openings. I take K1JT's word that ISCAT is better than JT6M and am mainly interested in deciding when to use -A or -B or JT65A.
               
              I am hoping some additional members of our 6M digital community will either replicate or hopefully improve our test. For one thing, I am sure some others with deeper technical understanding can make better use of the complete raw data collected including all.txt data columns other than S/N. I still feel that the compressed alternations and various stations' all.txt records including segment-by-segment S/N figures provided me with a better feel than a series of individual QSOs spread over time.
               
              73,
               
              Jim VE1JF
            Your message has been successfully submitted and would be delivered to recipients shortly.