Loading ...
Sorry, an error occurred while loading the content.

newbie threads question

Expand Messages
  • Santosh Sugur
    Hi Folks, My aim is to perform simultaneous operations (namely write) on files in a filesystem and so am using threads to accomplish this. I have never used
    Message 1 of 7 , Nov 23, 2007
    • 0 Attachment
      Hi Folks,

      My aim is to perform simultaneous operations (namely write) on files
      in a filesystem and so am using threads to accomplish this. I have
      never used threads before trying to write this test code, so I may be
      doing something drastically wrong!

      I have written the following code,

      #!/usr/bin/perl

      use strict;
      use warnings;
      use Fcntl;
      use threads;
      use Thread::Queue;


      my $ofs = $ARGV[0];
      my $dset = $ARGV[1];
      my $data_loc = "$ofs/$dset";
      $data_loc =~ s/\/\//\//;

      my (@threads);

      print "Opening (opendir) dir $data_loc \n";
      my $rc = opendir DIRHANDLE, "$data_loc";
      if (!$rc)
      {
      print "$! \n";
      exit;
      }

      my @allfiles = readdir DIRHANDLE;
      print "\nThis is \@allfiles @allfiles\n";

      foreach my $file (@allfiles)
      {
      push(@threads, threads->new(\&write_truncate,$file));
      }

      foreach my $myThread(@threads)
      {
      my @ReturnData = $myThread->join ;
      print "Thread returned: @ReturnData\n";
      }
      sub write_truncate
      {
      my $log;
      my $files = @_;
      if (-f "$data_loc/files")
      {
      print "Opening $data_loc/$files\n";
      sysopen(FILE, "$data_loc/$files", O_RDWR) or $fail++, print
      "Can't open $data_loc/$files: $!";
      print ("Writing to $files\n");
      $log = undef;
      $log = `dd if=/dev/urandom of=\'$data_loc/$files\' bs=1024 count=2 2>&1`;
      unless ($log)
      {
      print "Couldn't write to $data_loc/$files\n";
      }
      print $log;
      close FILE;
      }
      else
      {
      return;
      }
      }



      When I run this I get the following output

      # ./functional.pl /mnt/fuse testdata/jpeg
      Opening (opendir) dir /mnt/fuse/testdata/jpeg

      This is @allfiles . newdir Winter.jpg new2dir Water lilies.jpg ..
      Sunset.jpg Blue hills.jpg
      Thread returned:
      Thread returned:
      Thread returned:
      Thread returned:
      Thread returned:
      Thread returned:
      Thread returned:
      Thread returned:
      *** glibc detected *** double free or corruption (!prev): 0x0815b668 ***
      Aborted

      What does this output mean? Why am I getting it?
      The jpeg folder contains whats displayed by @allfiles in the output..
      /mnt/fuse is the normal linux filesystem area and nothing is mounted
      into it, the testdata folder is just hand copied there. So it could
      have been just any other location.
      Linux version is
      uname -a
      Linux localhost.localdomain 2.6.9-22.EL #1 Mon Sep 19 18:20:28 EDT
      2005 i686 i686 i386 GNU/Linux


      Thanks in anticipation,
      Santosh
    • merlyn@stonehenge.com
      Why Threads? Is this Windows? On Unix, it s better to fork. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
      Message 2 of 7 , Nov 23, 2007
      • 0 Attachment
        Why Threads? Is this Windows? On Unix, it's better to fork.
        --
        Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
        <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
        Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
        See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
      • Santosh Sugur
        This is actually to see how our filesystem would respond to multiple thread access and it should behave like linux s fs, so I am checking my script on linux
        Message 3 of 7 , Nov 23, 2007
        • 0 Attachment
          This is actually to see how our filesystem would respond to multiple
          thread access and it should behave like linux's fs, so I am checking
          my script on linux first. So its not that I want a task to be done but
          rather check how this would work.

          Regards,
          Santosh


          On Nov 23, 2007 9:22 PM, Randal L. Schwartz <merlyn@...> wrote:
          > Why Threads? Is this Windows? On Unix, it's better to fork.
          > --
          > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
          > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
          >
        • merlyn@stonehenge.com
          ... Santosh This is actually to see how our filesystem would respond to multiple Santosh thread access and it should behave like linux s fs, so I am checking
          Message 4 of 7 , Nov 23, 2007
          • 0 Attachment
            >>>>> "Santosh" == Santosh Sugur <santosh.sugur@...> writes:

            Santosh> This is actually to see how our filesystem would respond to multiple
            Santosh> thread access and it should behave like linux's fs, so I am checking
            Santosh> my script on linux first. So its not that I want a task to be done but
            Santosh> rather check how this would work.

            In general, "threads" in Perl cannot take advantage of multiple
            cores in modern multiprocessor systems, so the action on the filesystem
            won't be any different than a single "thread" would have generated.

            To give your filesystem a workout, fork. Only then can you get more activity.

            --
            Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
            <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
            Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
            See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
          • Santosh Sugur
            So we always use multiple processes as opposed to multiple threads while using perl on unix is it? That means threads in Perl is only for windows and is a
            Message 5 of 7 , Nov 23, 2007
            • 0 Attachment
              So we always use multiple processes as opposed to multiple threads
              while using perl on unix is it? That means threads in Perl is only for
              windows and is a "dummy" thing for unix?

              Santosh

              On Nov 23, 2007 9:42 PM, Randal L. Schwartz <merlyn@...> wrote:
              > >>>>> "Santosh" == Santosh Sugur <santosh.sugur@...> writes:
              >
              > Santosh> This is actually to see how our filesystem would respond to multiple
              > Santosh> thread access and it should behave like linux's fs, so I am checking
              > Santosh> my script on linux first. So its not that I want a task to be done but
              > Santosh> rather check how this would work.
              >
              > In general, "threads" in Perl cannot take advantage of multiple
              > cores in modern multiprocessor systems, so the action on the filesystem
              > won't be any different than a single "thread" would have generated.
              >
              > To give your filesystem a workout, fork. Only then can you get more activity.
              >
              > --
              >
              > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
              > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
              > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
              > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
              >
            • merlyn@stonehenge.com
              ... Santosh So we always use multiple processes as opposed to multiple threads Santosh while using perl on unix is it? That means threads in Perl is only for
              Message 6 of 7 , Nov 23, 2007
              • 0 Attachment
                >>>>> "Santosh" == Santosh Sugur <santosh.sugur@...> writes:

                Santosh> So we always use multiple processes as opposed to multiple threads
                Santosh> while using perl on unix is it? That means threads in Perl is only for
                Santosh> windows and is a "dummy" thing for unix?

                It's mostly for compatability, yes. In fact, "fork" works on both,
                but uses threads in windows, and natural forking in Unix.

                --
                Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
                <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
              • Santosh Sugur
                Thanks. I shall check this code on windows (for educational reasons) and come back of I have problems. regards, Santosh
                Message 7 of 7 , Nov 23, 2007
                • 0 Attachment
                  Thanks. I shall check this code on windows (for educational reasons)
                  and come back of I have problems.

                  regards,
                  Santosh

                  On Nov 23, 2007 9:52 PM, Randal L. Schwartz <merlyn@...> wrote:
                  > >>>>> "Santosh" == Santosh Sugur <santosh.sugur@...> writes:
                  >
                  > Santosh> So we always use multiple processes as opposed to multiple threads
                  > Santosh> while using perl on unix is it? That means threads in Perl is only for
                  > Santosh> windows and is a "dummy" thing for unix?
                  >
                  > It's mostly for compatability, yes. In fact, "fork" works on both,
                  > but uses threads in windows, and natural forking in Unix.
                  >
                  > --
                  >
                  > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
                  > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                  > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                  > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
                  >
                Your message has been successfully submitted and would be delivered to recipients shortly.