Loading ...
Sorry, an error occurred while loading the content.
 

Re: Cron appends junk to my wget downloads!

Expand Messages
  • bullfrog528
    Hi I noticed this recently too on one of my downloads, however i normally used files that have been tar ed. The tar xvf must ignore the junk at the end,
    Message 1 of 5 , Oct 10 5:28 AM
      Hi

      I noticed this recently too on one of my downloads, however i
      normally used files that have been tar'ed. The tar xvf must ignore
      the junk at the end, however when I downloaded a file directly it to
      had some junk on the end.

      There appear to be several versions of wget it's part of busybox
      which is installed as part of unslung but also can be installed
      seperately i believe which installs in /opt/bin if i remember
      correctly. I didn't look into it any further but it's possible the
      bug only relates to one of these wget versions. Might be worth trying
      the other one.

      If I discover anything else I'll let you know.

      regards

      Bullfrog


      --- In nslu2-general@yahoogroups.com, Airw0lf <aerwulfe@...> wrote:
      >
      > Hi all,
      >
      > Thanks to everyone's help here, I was able to get cron to run a
      script that
      > calls wget. However, I have another weird problem! I am using
      Unslung v6.8.
      >
      > Here is my /etc/crontab line:
      >
      > 0 2 * * * root /root/wget_script &>/dev/null
      >
      > My wget_script is as follows:
      >
      > cd /public
      > /opt/bin/wget -c -i /public/download.txt
      >
      > download.txt merely contains a list of URLs for wget to download.
      >
      > Now when I check the files downloaded by crontab, I always find
      junk
      > appended to the ends of the files! Here is an example of the bytes
      appended
      > to the end of one file:
      >
      > 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
      >
      > Using a hex editor, I can identify this as "100% 76.02 KB/s..".
      So it
      > seems that output from the shell is being put inside my downloads!
      >
      > If I just execute "./wget_script" I get no problems. If I download
      the SAME
      > file using the cron job, my file is hopelessly corrupted.
      >
      > Can anyone explain why this is happening?
      >
    • Thaths
      ... Maybe redirecting the stdout of the cron script might help: cd /public /opt/bin/wget -c -i /public/downliad.txt /dev/null 2 &1 Thaths -- Homer: He has
      Message 2 of 5 , Oct 10 6:29 AM
        On 10/10/06, Airw0lf <aerwulfe@...> wrote:
        > My wget_script is as follows:
        >
        > cd /public
        > /opt/bin/wget -c -i /public/download.txt
        >
        > download.txt merely contains a list of URLs for wget to download.
        >
        > Now when I check the files downloaded by crontab, I always find junk
        > appended to the ends of the files! Here is an example of the bytes appended
        > to the end of one file:
        >
        > 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
        >
        > Using a hex editor, I can identify this as "100% 76.02 KB/s..". So it
        > seems that output from the shell is being put inside my downloads!

        Maybe redirecting the stdout of the cron script might help:

        cd /public
        /opt/bin/wget -c -i /public/downliad.txt > /dev/null 2>&1

        Thaths
        --
        Homer: He has all the money in the world, but there's one thing he can't buy.
        Marge: What's that?
        Homer: (pause) A dinosaur.
        -- Homer J. Simpson
        Sudhakar Chandra Slacker Without Borders
      • Airw0lf
        I m not sure if it s a wget+cron issue, or just a cron issue. I don t see the problem for just calling wget. The version I m using is a separate install (from
        Message 3 of 5 , Oct 10 11:05 PM
          I'm not sure if it's a wget+cron issue, or just a cron issue. I don't see
          the problem for just calling wget.

          The version I'm using is a separate install (from ipkg install wget) in
          /opt/bin. I call /opt/bin/wget explicitly because I want to use this
          "full-featured" wget version.

          I managed to eliminate the problem by passing -q to wget. This stops all
          console output. However, I don't think it should be necessary to do
          this...

          "bullfrog528" <yahoo@...> wrote in
          news:egg3l8+10gjg@...:

          > Hi
          >
          > I noticed this recently too on one of my downloads, however i
          > normally used files that have been tar'ed. The tar xvf must ignore
          > the junk at the end, however when I downloaded a file directly it to
          > had some junk on the end.
          >
          > There appear to be several versions of wget it's part of busybox
          > which is installed as part of unslung but also can be installed
          > seperately i believe which installs in /opt/bin if i remember
          > correctly. I didn't look into it any further but it's possible the
          > bug only relates to one of these wget versions. Might be worth trying
          > the other one.
          >
          > If I discover anything else I'll let you know.
          >
          > regards
          >
          > Bullfrog
          >
          >
          > --- In nslu2-general@yahoogroups.com, Airw0lf
          > <aerwulfe@...> wrote:
          >>
          >> Hi all,
          >>
          >> Thanks to everyone's help here, I was able to get cron to run a
          > script that
          >> calls wget. However, I have another weird problem! I am using
          > Unslung v6.8.
          >>
          >> Here is my /etc/crontab line:
          >>
          >> 0 2 * * * root /root/wget_script &>/dev/null
          >>
          >> My wget_script is as follows:
          >>
          >> cd /public
          >> /opt/bin/wget -c -i /public/download.txt
          >>
          >> download.txt merely contains a list of URLs for wget to download.
          >>
          >> Now when I check the files downloaded by crontab, I always find
          > junk
          >> appended to the ends of the files! Here is an example of the bytes
          > appended
          >> to the end of one file:
          >>
          >> 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
          >>
          >> Using a hex editor, I can identify this as "100% 76.02 KB/s..".
          > So it
          >> seems that output from the shell is being put inside my downloads!
          >>
          >> If I just execute "./wget_script" I get no problems. If I download
          > the SAME
          >> file using the cron job, my file is hopelessly corrupted.
          >>
          >> Can anyone explain why this is happening?
          >>
          >
          >
          >
          >
          >
          >
          >
          >
          > Yahoo! Groups Links
          >
          >
          >
          >
          > ne.org
          >
          >
          >
          >
          >
          >
        • Airw0lf
          Thaths wrote in ... Thanks for that idea. I will look into it when I feel like learning some more. But for now I think I can manage by
          Message 4 of 5 , Oct 10 11:06 PM
            Thaths <thaths@...> wrote in
            news:1bc234630610100629p347df0by4891a99eefe19801@...:

            > On 10/10/06, Airw0lf
            > <aerwulfe@...> wrote:
            >> My wget_script is as follows:
            >>
            >> cd /public
            >> /opt/bin/wget -c -i /public/download.txt
            >>
            >> download.txt merely contains a list of URLs for wget to download.
            >>
            >> Now when I check the files downloaded by crontab, I always find junk
            >> appended to the ends of the files! Here is an example of the bytes
            >> appended to the end of one file:
            >>
            >> 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
            >>
            >> Using a hex editor, I can identify this as "100% 76.02 KB/s..". So
            >> it seems that output from the shell is being put inside my downloads!
            >
            > Maybe redirecting the stdout of the cron script might help:
            >
            > cd /public
            > /opt/bin/wget -c -i /public/downliad.txt > /dev/null 2>&1
            >
            > Thaths

            Thanks for that idea. I will look into it when I feel like learning some
            more.

            But for now I think I can manage by calling wget -q. This stops wget from
            writing to stdout.
          Your message has been successfully submitted and would be delivered to recipients shortly.