Loading ...
Sorry, an error occurred while loading the content.
 

Cron appends junk to my wget downloads!

Expand Messages
  • Airw0lf
    Hi all, Thanks to everyone s help here, I was able to get cron to run a script that calls wget. However, I have another weird problem! I am using Unslung v6.8.
    Message 1 of 5 , Oct 10, 2006
      Hi all,

      Thanks to everyone's help here, I was able to get cron to run a script that
      calls wget. However, I have another weird problem! I am using Unslung v6.8.

      Here is my /etc/crontab line:

      0 2 * * * root /root/wget_script &>/dev/null

      My wget_script is as follows:

      cd /public
      /opt/bin/wget -c -i /public/download.txt

      download.txt merely contains a list of URLs for wget to download.

      Now when I check the files downloaded by crontab, I always find junk
      appended to the ends of the files! Here is an example of the bytes appended
      to the end of one file:

      31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A

      Using a hex editor, I can identify this as "100% 76.02 KB/s..". So it
      seems that output from the shell is being put inside my downloads!

      If I just execute "./wget_script" I get no problems. If I download the SAME
      file using the cron job, my file is hopelessly corrupted.

      Can anyone explain why this is happening?
    • bullfrog528
      Hi I noticed this recently too on one of my downloads, however i normally used files that have been tar ed. The tar xvf must ignore the junk at the end,
      Message 2 of 5 , Oct 10, 2006
        Hi

        I noticed this recently too on one of my downloads, however i
        normally used files that have been tar'ed. The tar xvf must ignore
        the junk at the end, however when I downloaded a file directly it to
        had some junk on the end.

        There appear to be several versions of wget it's part of busybox
        which is installed as part of unslung but also can be installed
        seperately i believe which installs in /opt/bin if i remember
        correctly. I didn't look into it any further but it's possible the
        bug only relates to one of these wget versions. Might be worth trying
        the other one.

        If I discover anything else I'll let you know.

        regards

        Bullfrog


        --- In nslu2-general@yahoogroups.com, Airw0lf <aerwulfe@...> wrote:
        >
        > Hi all,
        >
        > Thanks to everyone's help here, I was able to get cron to run a
        script that
        > calls wget. However, I have another weird problem! I am using
        Unslung v6.8.
        >
        > Here is my /etc/crontab line:
        >
        > 0 2 * * * root /root/wget_script &>/dev/null
        >
        > My wget_script is as follows:
        >
        > cd /public
        > /opt/bin/wget -c -i /public/download.txt
        >
        > download.txt merely contains a list of URLs for wget to download.
        >
        > Now when I check the files downloaded by crontab, I always find
        junk
        > appended to the ends of the files! Here is an example of the bytes
        appended
        > to the end of one file:
        >
        > 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
        >
        > Using a hex editor, I can identify this as "100% 76.02 KB/s..".
        So it
        > seems that output from the shell is being put inside my downloads!
        >
        > If I just execute "./wget_script" I get no problems. If I download
        the SAME
        > file using the cron job, my file is hopelessly corrupted.
        >
        > Can anyone explain why this is happening?
        >
      • Thaths
        ... Maybe redirecting the stdout of the cron script might help: cd /public /opt/bin/wget -c -i /public/downliad.txt /dev/null 2 &1 Thaths -- Homer: He has
        Message 3 of 5 , Oct 10, 2006
          On 10/10/06, Airw0lf <aerwulfe@...> wrote:
          > My wget_script is as follows:
          >
          > cd /public
          > /opt/bin/wget -c -i /public/download.txt
          >
          > download.txt merely contains a list of URLs for wget to download.
          >
          > Now when I check the files downloaded by crontab, I always find junk
          > appended to the ends of the files! Here is an example of the bytes appended
          > to the end of one file:
          >
          > 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
          >
          > Using a hex editor, I can identify this as "100% 76.02 KB/s..". So it
          > seems that output from the shell is being put inside my downloads!

          Maybe redirecting the stdout of the cron script might help:

          cd /public
          /opt/bin/wget -c -i /public/downliad.txt > /dev/null 2>&1

          Thaths
          --
          Homer: He has all the money in the world, but there's one thing he can't buy.
          Marge: What's that?
          Homer: (pause) A dinosaur.
          -- Homer J. Simpson
          Sudhakar Chandra Slacker Without Borders
        • Airw0lf
          I m not sure if it s a wget+cron issue, or just a cron issue. I don t see the problem for just calling wget. The version I m using is a separate install (from
          Message 4 of 5 , Oct 10, 2006
            I'm not sure if it's a wget+cron issue, or just a cron issue. I don't see
            the problem for just calling wget.

            The version I'm using is a separate install (from ipkg install wget) in
            /opt/bin. I call /opt/bin/wget explicitly because I want to use this
            "full-featured" wget version.

            I managed to eliminate the problem by passing -q to wget. This stops all
            console output. However, I don't think it should be necessary to do
            this...

            "bullfrog528" <yahoo@...> wrote in
            news:egg3l8+10gjg@...:

            > Hi
            >
            > I noticed this recently too on one of my downloads, however i
            > normally used files that have been tar'ed. The tar xvf must ignore
            > the junk at the end, however when I downloaded a file directly it to
            > had some junk on the end.
            >
            > There appear to be several versions of wget it's part of busybox
            > which is installed as part of unslung but also can be installed
            > seperately i believe which installs in /opt/bin if i remember
            > correctly. I didn't look into it any further but it's possible the
            > bug only relates to one of these wget versions. Might be worth trying
            > the other one.
            >
            > If I discover anything else I'll let you know.
            >
            > regards
            >
            > Bullfrog
            >
            >
            > --- In nslu2-general@yahoogroups.com, Airw0lf
            > <aerwulfe@...> wrote:
            >>
            >> Hi all,
            >>
            >> Thanks to everyone's help here, I was able to get cron to run a
            > script that
            >> calls wget. However, I have another weird problem! I am using
            > Unslung v6.8.
            >>
            >> Here is my /etc/crontab line:
            >>
            >> 0 2 * * * root /root/wget_script &>/dev/null
            >>
            >> My wget_script is as follows:
            >>
            >> cd /public
            >> /opt/bin/wget -c -i /public/download.txt
            >>
            >> download.txt merely contains a list of URLs for wget to download.
            >>
            >> Now when I check the files downloaded by crontab, I always find
            > junk
            >> appended to the ends of the files! Here is an example of the bytes
            > appended
            >> to the end of one file:
            >>
            >> 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
            >>
            >> Using a hex editor, I can identify this as "100% 76.02 KB/s..".
            > So it
            >> seems that output from the shell is being put inside my downloads!
            >>
            >> If I just execute "./wget_script" I get no problems. If I download
            > the SAME
            >> file using the cron job, my file is hopelessly corrupted.
            >>
            >> Can anyone explain why this is happening?
            >>
            >
            >
            >
            >
            >
            >
            >
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
            > ne.org
            >
            >
            >
            >
            >
            >
          • Airw0lf
            Thaths wrote in ... Thanks for that idea. I will look into it when I feel like learning some more. But for now I think I can manage by
            Message 5 of 5 , Oct 10, 2006
              Thaths <thaths@...> wrote in
              news:1bc234630610100629p347df0by4891a99eefe19801@...:

              > On 10/10/06, Airw0lf
              > <aerwulfe@...> wrote:
              >> My wget_script is as follows:
              >>
              >> cd /public
              >> /opt/bin/wget -c -i /public/download.txt
              >>
              >> download.txt merely contains a list of URLs for wget to download.
              >>
              >> Now when I check the files downloaded by crontab, I always find junk
              >> appended to the ends of the files! Here is an example of the bytes
              >> appended to the end of one file:
              >>
              >> 31 30 30 25 20 20 20 37 36 2E 30 32 20 4B 42 2F 73 0A 0A
              >>
              >> Using a hex editor, I can identify this as "100% 76.02 KB/s..". So
              >> it seems that output from the shell is being put inside my downloads!
              >
              > Maybe redirecting the stdout of the cron script might help:
              >
              > cd /public
              > /opt/bin/wget -c -i /public/downliad.txt > /dev/null 2>&1
              >
              > Thaths

              Thanks for that idea. I will look into it when I feel like learning some
              more.

              But for now I think I can manage by calling wget -q. This stops wget from
              writing to stdout.
            Your message has been successfully submitted and would be delivered to recipients shortly.