Loading ...
Sorry, an error occurred while loading the content.

Re: Heyu Lock file issue

Expand Messages
  • Kris
    Hi ~ Just wanted you to see this script. It may or may not be what you need, but it offers some insight into job control with scripts. ######################
    Message 1 of 13 , Jul 19, 2013
    • 0 Attachment
      Hi ~

      Just wanted you to see this script. It may or may not be what you need, but it offers some insight into job control with scripts.

      ###################### Start Script
      #!/usr/bin/env bash
      set -f


      clear
      job1() {
      # We can source other scripts instead of typing them in by using the bash "source" builtin.
      # source /home/my_scripts/script1.sh
      for i in {0..200000}
      do
      echo "Running Job 1: $i">/dev/null
      done
      echo "

      Job 1 Finished."

      }


      job2() {
      # We can source other scripts instead of typing them in.
      # source /home/my_scripts/script2.sh
      for i in {0..100000}
      do
      echo "Running Job 2: $i">/dev/null
      done
      echo "

      Job 2 Finished."
      }

      # We can use the bash builtin "wait". This will wait for Job 1 to finish before running Job 2. If we don't wait then Job 2 will run at the same time Job 1 is running.
      # Try it. Remove the wait and running the script from terminal.
      time (job1) &
      wait
      time (job2) &

      ###################### End Script








      --- In heyu_users@yahoogroups.com, "Michael Stowe" <mstowe@...> wrote:
      >
      >
      > > Ok, I think I need to share more detail... I just didn't want to write a
      > > novel...
      > >
      > > I have a very complex X10 environment with over 40 devices, 6 sensors, and
      > > a CM11a, and firecracker for controls. I use both heyu and br in scripts.
      > > Sometimes I can have up to 10-15 heyu events happening at a time,
      > > depending on weather, lighting, movement, and state of switches, and or
      > > files. This is a 12 year old project. I am finally at a stage that logic
      > > in the scripts is needed due to collisions (yes I some how manged just to
      > > fire commands off and they worked.)
      >
      > Since only one X10 command can be operating on the power line at a time,
      > it does tend to be self-limiting, with a couple of exceptions:
      > 1) While a macro is executing, you can attempt to send more commands (heyu
      > wait is a crude but effective solution for this situation)
      > 2) More than one process can attempt to simultaneously send commands (a
      > semaphore is a general purpose solution for this issue.)
      >
      > > heyu wait has solved a lot of the trigger mishaps iniated from heyu config
      > > scripts, but cron does not have that functionality that I have found.
      >
      > You're mixing a couple of things (see above) -- causing a heyu process to
      > wait for a macro doesn't prevent another heyu process from attempting to
      > send commands. Also, the first process works by polling the interface,
      > so, logically, cron cannot possibly do the same thing.
      >
      > > I
      > > thought if I looked at the lock file to see if it was present, I could use
      > > that as a way to make the script wait for an opening in the power line.
      >
      > No, you can't, as I explained earlier, and this is what we refer to as a
      > REALLY bad idea. The semantics of the heyu lock file are not the same as
      > the procmail lockfile program, and mixing the two is unlikely to yield
      > desirable results, even if you didn't have process, fork, and race issues.
      >
      > > I probably have close to 60 scripts I would have to modify with the
      > > suggested my.lock code, and the reality is I only am annoyed by 2 cron
      > > jobs reporting via email. Everything else is working fine.
      >
      > You *could* always just alter the two scripts which annoy you, which may
      > be sufficient, or just write a script to change all 60 scripts at once
      > since the semantics of changing all 60 would be identical, and wouldn't
      > cause any harm to scripts invoked by heyu itself.
      >
      > > These 2 cron shell scripts... a mrtg initiated script that graphs my house
      > > temperature, and another that actually monitors the temperature, and
      > > controls the thermostat based on state file created by a web interface.
      > > It does not make since to combine the scripts because polling the RCS
      > > interface to update the state file for the web interface takes a lot
      > > longer than just getting the temp. I don't want to do it very often.
      > > Offsetting cron times will not work either because of other activity that
      > > may be on the line.
      >
      > > I have to figure out how to perform some kind of
      > > logic based on what the cm11a is seeing on the line, or what heyu is doing
      > > at the moment.
      >
      > Put aside the notion of trying to use the serial port lock file; it won't
      > do you any good other than determining if heyu is using the serial port.
      > It's absolutely possible for it not to exist and heyu to be in use.
      >
      > > I've thought about greping the heyu processes and doing a word count, but
      > > The lock file looked like a better solution?
      >
      > Just use a lock file in a sensible way. Grepping for named processes is
      > crude at best and unlikely to work well.
      >
      > > I have not messed with lock
      > > files much, so i must admit i do not know what the consequences of using
      > > process generated versis script generated lock files is about.
      >
      > I have told you twice now, but I will try to make this as clear as I can.
      > "Lock" files are interpreted by, and are meaningful ONLY to the process or
      > file that makes them. heyu and lockfile are both processes, so in neither
      > case has a script actually generated a lockfile. The heyu and lockfile
      > processes have DIFFERENT interpretations for lockfiles and are NOT
      > compatible and what you're doing is wrong, and morally reprehensible, like
      > eating a baby.
      >
      > > (Ugh, more
      > > reading.) The other problem I run into is "Unable to send command bytes"
      > > errors even if the lock file is clear, which if my memory serves me
      > > correctly is tell-tale of a collision, hence the code to restart the
      > > process if a non-number is returned.
      >
      > It may indicate a collision or noise on the power lines -- restarting the
      > process is unnecessary.
      >
      > > Even with all that, it still is failing 7-10 times a day. And I know some
      > > of you may think I am crazy at this point, but it really isn't a solution
      > > for me to say I maxed out the system, because I know with the right logic
      > > it will work. I actually had it working on an older machine with a few
      > > more devices, but i think the single core, and speed of the machine may
      > > have been why? The new machine is much faster, and simultaneously
      > > troubleshooting LED lighting anomolies for the past 2 months was
      > > discouraging. The limited time to spend on the project is the challenge
      > > right now.
      > >
      > > As I stated I am no scripting guru, most of my scripts are hacks from web
      > > tutorials and forums. This is a learning project, so any knowledge you
      > > could share... If I have some sort of revelation I will contribute myself.
      > > ;)
      >
    • klmcwhirter
      My mrtg script is called from mrtg, and I do not want to run my temperature check script running every time mrtg runs, so the whole parent/children script idea
      Message 2 of 13 , Jul 20, 2013
      • 0 Attachment
        My mrtg script is called from mrtg, and I do not want to run my temperature check script running every time mrtg runs, so the whole parent/children script idea isn't really a solution unfortunately. Thanks for the cool info though... Interesting script. Looks like I have more reading to do still on scripting. I can not use it for what I am trying to do, but I can see where it is useful.

        --- In heyu_users@yahoogroups.com, "Kris" <krisbeazley@...> wrote:
        >
        >
        >
        > Hi ~
        >
        > Just wanted you to see this script. It may or may not be what you need, but it offers some insight into job control with scripts.
        >
        > ###################### Start Script
        > #!/usr/bin/env bash
        > set -f
        >
        >
        > clear
        > job1() {
        > # We can source other scripts instead of typing them in by using the bash "source" builtin.
        > # source /home/my_scripts/script1.sh
        > for i in {0..200000}
        > do
        > echo "Running Job 1: $i">/dev/null
        > done
        > echo "
        >
        > Job 1 Finished."
        >
        > }
        >
        >
        > job2() {
        > # We can source other scripts instead of typing them in.
        > # source /home/my_scripts/script2.sh
        > for i in {0..100000}
        > do
        > echo "Running Job 2: $i">/dev/null
        > done
        > echo "
        >
        > Job 2 Finished."
        > }
        >
        > # We can use the bash builtin "wait". This will wait for Job 1 to finish before running Job 2. If we don't wait then Job 2 will run at the same time Job 1 is running.
        > # Try it. Remove the wait and running the script from terminal.
        > time (job1) &
        > wait
        > time (job2) &
        >
        > ###################### End Script
        >
        >
        >
        >
        >
        >
        >
        >
      • klmcwhirter
        OK, I stopped eating babies (I love your passionate analogies. ;) The code looks like this now... ... GetTemp () { while [ -e /var/lock/heyu.lock ] ; do
        Message 3 of 13 , Jul 20, 2013
        • 0 Attachment
          OK, I stopped eating babies (I love your passionate analogies. ;)

          The code looks like this now...
          ---
          GetTemp ()
          {
          while [ -e "/var/lock/heyu.lock" ] ;
          do
          /bin/sleep 2
          done
          /usr/bin/lockfile /var/lock/heyu.lock
          Temp=`/usr/local/bin/heyu temp_req preset j5 1 | /bin/grep "Temp" | /bin/cut --delimiter== -f2 | /bin/cut --delimiter=" " -f2`
          /usr/bin/printf '%g' "$Temp" &>/dev/null && /bin/echo "$Temp" || GetTemp;
          /bin/rm -f "/var/lock/heyu.lock"
          }
          ---

          It sort of works, But now the following error is back...

          "Unable to lock /var/lock/LCK..heyu.write.ttyS0
          Could not set up the heyu.write lock
          HEYU: Program exiting."

          I have to do some reading on lock files and try to understand why checking for the "LCK..heyu.write.ttyS0" file is "morally reprehensible" ;) , but the fact is that my previous code worked better than the code above. Instead of 2 different errors, I was only getting 1. I really just need to figure out how to get my scripts to loop if anything but a number is returned as a result. That is what I set out to do, but I can not seem to get it to work 100% of the time. :(

          If I find a solution I will post it. Keep the ideas coming, and I will try them when I have time to test.


          --- In heyu_users@yahoogroups.com, "Michael Stowe" <mstowe@...> wrote:
          >
          >
          > > Ok, I think I need to share more detail... I just didn't want to write a
          > > novel...
          > >
          > > I have a very complex X10 environment with over 40 devices, 6 sensors, and
          > > a CM11a, and firecracker for controls. I use both heyu and br in scripts.
          > > Sometimes I can have up to 10-15 heyu events happening at a time,
          > > depending on weather, lighting, movement, and state of switches, and or
          > > files. This is a 12 year old project. I am finally at a stage that logic
          > > in the scripts is needed due to collisions (yes I some how manged just to
          > > fire commands off and they worked.)
          >
          > Since only one X10 command can be operating on the power line at a time,
          > it does tend to be self-limiting, with a couple of exceptions:
          > 1) While a macro is executing, you can attempt to send more commands (heyu
          > wait is a crude but effective solution for this situation)
          > 2) More than one process can attempt to simultaneously send commands (a
          > semaphore is a general purpose solution for this issue.)
          >
          > > heyu wait has solved a lot of the trigger mishaps iniated from heyu config
          > > scripts, but cron does not have that functionality that I have found.
          >
          > You're mixing a couple of things (see above) -- causing a heyu process to
          > wait for a macro doesn't prevent another heyu process from attempting to
          > send commands. Also, the first process works by polling the interface,
          > so, logically, cron cannot possibly do the same thing.
          >
          > > I
          > > thought if I looked at the lock file to see if it was present, I could use
          > > that as a way to make the script wait for an opening in the power line.
          >
          > No, you can't, as I explained earlier, and this is what we refer to as a
          > REALLY bad idea. The semantics of the heyu lock file are not the same as
          > the procmail lockfile program, and mixing the two is unlikely to yield
          > desirable results, even if you didn't have process, fork, and race issues.
          >
          > > I probably have close to 60 scripts I would have to modify with the
          > > suggested my.lock code, and the reality is I only am annoyed by 2 cron
          > > jobs reporting via email. Everything else is working fine.
          >
          > You *could* always just alter the two scripts which annoy you, which may
          > be sufficient, or just write a script to change all 60 scripts at once
          > since the semantics of changing all 60 would be identical, and wouldn't
          > cause any harm to scripts invoked by heyu itself.
          >
          > > These 2 cron shell scripts... a mrtg initiated script that graphs my house
          > > temperature, and another that actually monitors the temperature, and
          > > controls the thermostat based on state file created by a web interface.
          > > It does not make since to combine the scripts because polling the RCS
          > > interface to update the state file for the web interface takes a lot
          > > longer than just getting the temp. I don't want to do it very often.
          > > Offsetting cron times will not work either because of other activity that
          > > may be on the line.
          >
          > > I have to figure out how to perform some kind of
          > > logic based on what the cm11a is seeing on the line, or what heyu is doing
          > > at the moment.
          >
          > Put aside the notion of trying to use the serial port lock file; it won't
          > do you any good other than determining if heyu is using the serial port.
          > It's absolutely possible for it not to exist and heyu to be in use.
          >
          > > I've thought about greping the heyu processes and doing a word count, but
          > > The lock file looked like a better solution?
          >
          > Just use a lock file in a sensible way. Grepping for named processes is
          > crude at best and unlikely to work well.
          >
          > > I have not messed with lock
          > > files much, so i must admit i do not know what the consequences of using
          > > process generated versis script generated lock files is about.
          >
          > I have told you twice now, but I will try to make this as clear as I can.
          > "Lock" files are interpreted by, and are meaningful ONLY to the process or
          > file that makes them. heyu and lockfile are both processes, so in neither
          > case has a script actually generated a lockfile. The heyu and lockfile
          > processes have DIFFERENT interpretations for lockfiles and are NOT
          > compatible and what you're doing is wrong, and morally reprehensible, like
          > eating a baby.
          >
          > > (Ugh, more
          > > reading.) The other problem I run into is "Unable to send command bytes"
          > > errors even if the lock file is clear, which if my memory serves me
          > > correctly is tell-tale of a collision, hence the code to restart the
          > > process if a non-number is returned.
          >
          > It may indicate a collision or noise on the power lines -- restarting the
          > process is unnecessary.
          >
          > > Even with all that, it still is failing 7-10 times a day. And I know some
          > > of you may think I am crazy at this point, but it really isn't a solution
          > > for me to say I maxed out the system, because I know with the right logic
          > > it will work. I actually had it working on an older machine with a few
          > > more devices, but i think the single core, and speed of the machine may
          > > have been why? The new machine is much faster, and simultaneously
          > > troubleshooting LED lighting anomolies for the past 2 months was
          > > discouraging. The limited time to spend on the project is the challenge
          > > right now.
          > >
          > > As I stated I am no scripting guru, most of my scripts are hacks from web
          > > tutorials and forums. This is a learning project, so any knowledge you
          > > could share... If I have some sort of revelation I will contribute myself.
          > > ;)
          >
        • Michael Stowe
          ... # This isn t doing anything at all, from here... ... # to here. The below line should look for a semaphore file, or if one # exists, just wait for it to
          Message 4 of 13 , Jul 20, 2013
          • 0 Attachment
            > OK, I stopped eating babies (I love your passionate analogies. ;)
            >
            > The code looks like this now...
            > ---
            > GetTemp ()
            > {
            # This isn't doing anything at all, from here...
            > while [ -e "/var/lock/heyu.lock" ] ;
            > do
            > /bin/sleep 2
            > done
            # to here. The below line should look for a semaphore file, or if one
            # exists, just wait for it to go away. Note that if something goes awry,
            # you could potentially have scripts just waiting forever
            > /usr/bin/lockfile /var/lock/heyu.lock
            > Temp=`/usr/local/bin/heyu temp_req preset j5 1 | /bin/grep "Temp" |
            > /bin/cut --delimiter== -f2 | /bin/cut --delimiter=" " -f2`
            > /usr/bin/printf '%g' "$Temp" &>/dev/null && /bin/echo "$Temp" || GetTemp;
            > /bin/rm -f "/var/lock/heyu.lock"
            > }
            > ---
            >
            > It sort of works, But now the following error is back...
            >
            > "Unable to lock /var/lock/LCK..heyu.write.ttyS0
            > Could not set up the heyu.write lock
            > HEYU: Program exiting."

            There are a couple of possible reasons:
            1) Another heyu process is using the serial port, and therefor has a lock
            file open
            2) Something is preventing heyu from creating its lock file (an old lock
            file, permissions, directory problems, etc.)

            > I have to do some reading on lock files and try to understand why checking
            > for the "LCK..heyu.write.ttyS0" file is "morally reprehensible" ;)

            Checking for it, in and of itself isn't a problem, but using procmail's
            lockfile (that's where the "lockfile" program generally hails from) is,
            since they don't understand lock files the same way. Also note that the
            heyu lock file doesn't mean "heyu is in use" it means "heyu has grabbed
            the serial port" so it still leaves open the possibility that more than
            one heyu will be running simultaneously -- for example:

            1) heyu launches
            2) you check for heyu's lock file, which does not exist, so you launch
            another heyu process (for clarity, we'll call this heyu2, and the
            previously launched process heyu1.)
            3) heyu1 opens the serial port
            4) heyu2 tries to open the serial port, and notices that heyu1 already has
            it open

            So step #2 isn't doing you a whole lot of good, really.

            > , but
            > the fact is that my previous code worked better than the code above.
            > Instead of 2 different errors, I was only getting 1. I really just need
            > to figure out how to get my scripts to loop if anything but a number is
            > returned as a result. That is what I set out to do, but I can not seem to
            > get it to work 100% of the time. :(

            Insofar as anything in bash is straightforward, that's at least not
            difficult:

            myval = "not a number"
            expr "$myval + 1" 2> /dev/null
            while [ $? -ne 0 ]
            do
            echo "myval is not numeric"
            # Loop and set value to what-have-you
            myval = 0
            expr "$myval + 1" 2> /dev/null
            done

            > If I find a solution I will post it. Keep the ideas coming, and I will
            > try them when I have time to test.
          • K W
            Hello, I also have a complicated setup, but to avoid cron v.s. heyu problems I just use entries in the x10sched file to turn on/off psuedo-devices that are
            Message 5 of 13 , Aug 5, 2013
            • 0 Attachment
              Hello,

              I also have a complicated setup, but to avoid cron v.s. heyu problems I just use entries in the "x10sched" file to turn on/off psuedo-devices that are meaningful to "x10config".   The psuedo-devices use a housecode I'm not using ("P" is an obvious choice) and it has the very great advantage that heyu knows when dawn/dusk occurs at my lat/long, as opposed to cron only knowing time.  So my x10sched turns on a P11 device at dawn+30 and dusk-30 that my x10config takes to mean "send every device the on/off/dim I think it has so the device matches the settings I think it has".  This fixes cases where the x10 signal was missed or somebody pressed a wall-switch that is also under automatic control.

              Best wishes,
              ~JW




              ________________________________
              From: Michael Stowe <mstowe@...>
              To: heyu_users@yahoogroups.com
              Sent: Saturday, July 20, 2013 1:46 PM
              Subject: Re: [heyu_users] Re: Heyu Lock file issue



               

              > OK, I stopped eating babies (I love your passionate analogies. ;)
              >
              > The code looks like this now...
              > ---
              > GetTemp ()
              > {
              # This isn't doing anything at all, from here...
              > while [ -e "/var/lock/heyu.lock" ] ;
              > do
              > /bin/sleep 2
              > done
              # to here. The below line should look for a semaphore file, or if one
              # exists, just wait for it to go away. Note that if something goes awry,
              # you could potentially have scripts just waiting forever
              > /usr/bin/lockfile /var/lock/heyu.lock
              > Temp=`/usr/local/bin/heyu temp_req preset j5 1 | /bin/grep "Temp" |
              > /bin/cut --delimiter== -f2 | /bin/cut --delimiter=" " -f2`
              > /usr/bin/printf '%g' "$Temp" &>/dev/null && /bin/echo "$Temp" || GetTemp;
              > /bin/rm -f "/var/lock/heyu.lock"
              > }
              > ---
              >
              > It sort of works, But now the following error is back...
              >
              > "Unable to lock /var/lock/LCK..heyu.write.ttyS0
              > Could not set up the heyu.write lock
              > HEYU: Program exiting."

              There are a couple of possible reasons:
              1) Another heyu process is using the serial port, and therefor has a lock
              file open
              2) Something is preventing heyu from creating its lock file (an old lock
              file, permissions, directory problems, etc.)

              > I have to do some reading on lock files and try to understand why checking
              > for the "LCK..heyu.write.ttyS0" file is "morally reprehensible" ;)

              Checking for it, in and of itself isn't a problem, but using procmail's
              lockfile (that's where the "lockfile" program generally hails from) is,
              since they don't understand lock files the same way. Also note that the
              heyu lock file doesn't mean "heyu is in use" it means "heyu has grabbed
              the serial port" so it still leaves open the possibility that more than
              one heyu will be running simultaneously -- for example:

              1) heyu launches
              2) you check for heyu's lock file, which does not exist, so you launch
              another heyu process (for clarity, we'll call this heyu2, and the
              previously launched process heyu1.)
              3) heyu1 opens the serial port
              4) heyu2 tries to open the serial port, and notices that heyu1 already has
              it open

              So step #2 isn't doing you a whole lot of good, really.

              > , but
              > the fact is that my previous code worked better than the code above.
              > Instead of 2 different errors, I was only getting 1. I really just need
              > to figure out how to get my scripts to loop if anything but a number is
              > returned as a result. That is what I set out to do, but I can not seem to
              > get it to work 100% of the time. :(

              Insofar as anything in bash is straightforward, that's at least not
              difficult:

              myval = "not a number"
              expr "$myval + 1" 2> /dev/null
              while [ $? -ne 0 ]
              do
              echo "myval is not numeric"
              # Loop and set value to what-have-you
              myval = 0
              expr "$myval + 1" 2> /dev/null
              done

              > If I find a solution I will post it. Keep the ideas coming, and I will
              > try them when I have time to test.




              [Non-text portions of this message have been removed]
            • klmcwhirter
              I started using cron because I only have 1.8% of interface memory left, and I noticed dawn and dusk timers eat up a lot of memory in the CM11a interface. I
              Message 6 of 13 , Aug 6, 2013
              • 0 Attachment
                I started using cron because I only have 1.8% of interface memory left, and I noticed dawn and dusk timers eat up a lot of memory in the CM11a interface. I make extensive use of dawn and dusk timers for my irrigation, and outdoor lighting. (That reminds me, I have been meaning to go thru my schedule file and combine and delete some items too. ;) Running mrtg as a heyu macro is probably not desirable in any case, hence the necessity for cron as well?


                ... I am back to checking for the write lock file. Doing so has eliminated the Lock error, now I just have "Unable to send command bytes" errors. There are far less errors in any case, and a couple different sources claim that checking for that lock file is what I should be doing with a hardware stream anyway. I did back away from using 'lockfile' as I originally found the program, and was using it out of curiosity, and it seemed to do what I needed. I eventually, will read more about it, but I just haven't had time to play with it.

                The piece I can not get to work consistently is checking to see if heyu returns a number, or kicks out an error... if it kicks out an error I want it to loop until it kicks out a number. I have googled extensively, and tried several printf, while, if..then, case, code variations (including a couple from this board), but for some reason whether I use /dev/null, or not, the errors get spit out, and cause the script to send me an email with the error instead of the value it changed. I believe there is some time out code within heyu that is the culprit (or possibly a built in fail safe?) that I don't understand yet. For now I have the errors down to about 10 a day, and I am living with it until I can dive into it more.

                Cheers.


                --- In heyu_users@yahoogroups.com, K W <jw037703@...> wrote:
                >
                > Hello,
                >
                > I also have a complicated setup, but to avoid cron v.s. heyu problems I just use entries in the "x10sched" file to turn on/off psuedo-devices that are meaningful to "x10config".   The psuedo-devices use a housecode I'm not using ("P" is an obvious choice) and it has the very great advantage that heyu knows when dawn/dusk occurs at my lat/long, as opposed to cron only knowing time.  So my x10sched turns on a P11 device at dawn+30 and dusk-30 that my x10config takes to mean "send every device the on/off/dim I think it has so the device matches the settings I think it has".  This fixes cases where the x10 signal was missed or somebody pressed a wall-switch that is also under automatic control.
                >
                > Best wishes,
                > ~JW
                >
                >
                >
                >
                > ________________________________
                >
              Your message has been successfully submitted and would be delivered to recipients shortly.