118Re: Bad upload data
- Jul 31, 2005Mike,
It appears that the data sent to the weather station and received
from the weather station does include a simple data integrity check.
The data frames contain what appears to be a single byte checksum.
Although this is not a very reliable data integrity check (CRC would
have been much better), it should catch transmission errors between
the weather station and the PC. My guess is that the serial data
between the weather station and the PC is fine and that the problem
is really with the station and it's data collection from the sensors.
Reading over the other forums, it appears that the data sent from the
sensors to the weather station does not include even a simple
checksum to validate the data.
When corrupt data is retrieved from the sensors, the weather station
is not performing data integrity checking, it is probably just
passing the info to the PC.
Heavyweather may or not be attempting to peform some checking
internally (before it sends to it's user interface or history.dat
file, but all indications are that it passes on the bogus data to
As consumers, we only have two options to reduce this noise.
1) Reduce the potential for noise.
2) Attempt to filter out bad data with range checking.
In the case of 1 as you mentioned, the sensors should not be mounted
to a metal mast (I have mine mounted on an aluminum rod). We can also
attempt to reduce the noise produced by the cheap wiring as mention
by other posters.
As for 2, as mentioned, I have done some basic data checking. It is
far from perfect though. I will continue to pursue improving the
If any user is detecting bad data in the uploads using version 129 or
later, please forward the information to me. Please capture the
entire line of data including temperature, wind speed, ect so that I
can see if there is a detectable pattern.
I want to be very careful on filtering the data as I do not want to
mask out rapidly changing conditions (such as wind gust data during
Thanks for your help.
--- In firstname.lastname@example.org, "Mike Frerkes"
> I was running an earlier version (maybe 127 or 126). I have justpipe
> installed the 129 version, so I'll see if anything changes with
> respect to this problem.
> If you are experiencing wind-related errors, consider using PVC
> for the mast. I was using metal pipe up until a few weeks ago. Onerrors
> days above 90, the metal would get hot and cause siginificant
> in the wind data, even in wireless mode. After switching to PVC, Iin
> was able to run the unit cabled, and now even on 90-degree days my
> data is error-free.
> Thanks for you help and suggestions with my problem.
> --- In email@example.com, "wuhu_software"
> <wuhu_software@y...> wrote:
> > Mike,
> > On my system, I seem to run for weeks without a single bad
> > Other times, I will have windspeed errors combined with other
> > in rapid succession. I have not been able to find a pattern to it.
> > The other possibility for these bad reading being written to
> > currdat.lst and not to the history file might possibly be a bug
> > Heavyweather itself. It is difficult to say.temperature
> > One thing that is for certain is that Heavyweather is writting
> > data.
> > Here is an example of a bad reading put out by Heavyweather:
> > [rain_24h]
> > mm = "15370.0"
> > inch = "605.11"
> > This was found in the currdat.lst file. As you can see,
> > is writing bogus data for both mm and inch measurements. The same
> > true for bad temps and windspeed data as well.
> > These bogus readings will be overwritten on the next update
> > by Heavyweather (which updates the currdat.lst file once per
> > As I mentioned before, I attempted to catch some of these errors.
> > instance, when reading temperature, if the current reading is not
> > within 10 degrees of the average temperature readings over the
> > ten minutes, then the software will use the last valid
> > reading (that passed the above test) as the current temperatureversion
> > reading.
> > These features were added in one of the later versions and is
> > definetly in version 129.
> > Which version are you running? If you are running version 129 and
> > still seeing these bogus temperature readings when running
> > 129 let me know. I can look over the code to see if there is athe
> > mistake.
> > Thanks.
> > --- In firstname.lastname@example.org, "Mike Frerkes"
> > <mfrerkes@m...> wrote:
> > > There are no sources of EM interference I'm aware of. I have
> > > learned that you should never use a metal pole as a mast for
> > > anemometer, however. A couple of weeks ago, I switched to PVCalways
> > > and the wind readings are now flawless, even with a cabled
> > > connection. Before the switch, I couldn't use the cabled
> > connection
> > > because the wind readings would be horribly inaccurate.
> > >
> > > I'm puzzled why the bad readings (i.e zero degree temp) DO NOT
> > > up on my HeavyWeather program under the min/max tabs for
> > temperature
> > > and dewpoint, or on the base unit's display. There is no rhyme
> > > reason to their occurence. Sometimes I may go an entire week
> > > without a false reading. There have been a few instances where
> > I've
> > > registered two in the same day. The only common thread is that
> > they
> > > occur within the first five minutes of each hour AND they
> > > take place between 12am and 5am.If
> > >
> > >
> > >
> > > --- In email@example.com, "wuhu_software"
> > > <wuhu_software@y...> wrote:
> > > >
> > > > Mike,
> > > >
> > > > What is interesting is that the Heavyweather application is
> > indeed
> > > > writing bogus data to the currdat.lst file but somehow this
> > > not
> > > > make in to the history.dat file.
> > > >
> > > > There are a couple of theories why this is the case. Firstly,
> > > > currdat.lst file is written once per minute by Heavyweather,
> > > once
> > > > per update (9 seconds in wired mode). Since it reading 5
> > > per
> > > > history file save, it might just be random. The other theory
> > > that
> > > > somehow it can detect bad data and rejects it internally.
> > > >
> > > > A way to catch the Heavyweather application writing bad data
> > to
> > > use
> > > > the alarms/events. You can set an alarm for low temperature
> > > > instance. When you hear the audio, open currdat.lst with
> > > within
> > > > 1 minute. You can see that the bogus data is being generated
> > > > Heavyweather.
> > > >
> > > > From what I gather from these forums, the best way to
> > > bogus
> > > > data is to build a shielded cable. I have not modified my
> > > but
> > > > it will be interesting to see if this corrects the problem.
> > > anyonedata,
> > > > has modified the cable, please post your results.
> > > >
> > > > I have attempted to detect and eliminate bogus wind readings
> > > > others. I am not sure if I had done the same on version 129
> > > is
> > > > available from the alternative download page. I believe I
> > a
> > > > message on the rules I used to attempt to detect bad data
> > > previously.
> > > >
> > > > If you have any idea or rules that can be applied to the
> > letmorning
> > > me
> > > > know. I am willing to modify the application if we can find
> > simple
> > > > rules to detect bogus data.
> > > >
> > > > P.S. Are there any source of EM radiation, power dips, or
> > anything
> > > else
> > > > that happens at that time around your home? Just wondering.
> > > >
> > > > --- In firstname.lastname@example.org, "Mike Frerkes"
> > > > <mfrerkes@m...> wrote:
> > > > > I have also experienced erroneous data uploads, almost
> always a
> > > > > temperature and dew point reading of 0. I've traced the
> > > defective
> > > > > data to the WUHU program itself, because those extreme
> > > are
> > > > > not registered on HeavyWeather or the La Crosse base unit.
> > > > >
> > > > > My false readings always seem to appear in the early
> > > hourshour.
> > > > > between 12am and 5am, and always within the first five
> > > of the
> > > > > hour (e.g. 1:03am or 4:05am). I have the upload interval
> > at
> > > five
> > > > > minutes, so it might be related to that in some way. There
> > > never
> > > > > been a bad reading after the first five minutes of each
> > > > >
> > > > > These incorrect readings are uploaded to both Weather
> > > Underground and
> > > > > CWOP, so I've ruled out any data glitches on their end.
> > > Hopefully
> > > > > this information may assist you in isolating the cause of
> > > problem.
- << Previous post in topic