Loading ...
Sorry, an error occurred while loading the content.

Destroying anonymous arrays

Expand Messages
  • Ken Shail
    How can I make sure that anonymous arrays do not get left around. It is difficult to isolate the code but it goes like this file is opened regularly for
    Message 1 of 3 , Jul 5, 2005
    • 0 Attachment
      How can I make sure that anonymous arrays do not get left around.
      It is difficult to isolate the code but it goes like this

      file is opened regularly for reading
      foreach my $temp(<FILEHANDLE>)
      {
      @buffer=split /,/,$temp; # each line of the file is a comma-separated
      list
      # the first item in the list is the name by which it gets referred to later
      $bufRef{$buffer[0]}=[@buffer];
      }

      later on I want to find a specific list by the $name

      my $reff=$bufRef{$name};
      @recoveredList=@$reff;

      If I continue to do this will I not eventually run out of memory or does
      Perl "know" to destroy the old anonymous arrays each time I run the foreach
      loop again?

      Ken
    • Brad Lhotsky
      perl does automatic garbage collection, but the structures have to go out of scope. This is why you ll usually hear people recommend that you declare a
      Message 2 of 3 , Jul 6, 2005
      • 0 Attachment
        perl does automatic garbage collection, but the structures have to go
        out of scope. This is why you'll usually hear people recommend that you
        declare a variable in the narrowest scope possible.

        If there's no references to the anonymous array, it will go away.. so if
        you undef your $reff and %bufRef=(); you should be able to put things
        back in there without a problem.

        I'm not entirely sure what you're trying to do by your example though.
        Are you opening multiple files? are you reopening the same file? I don't
        usually see the foreach() for reading from files.. There are CSV
        Modules on CPAN that might simplify your process some..

        lets say we've got multiple files and we're doing stuff with the files'
        data.

        my @files = qw(a.txt b.txt);
        my %sums = ();

        foreach my $file (@files) {
        addUp($file);
        }

        sub addUp {
        my $fn = shift;
        my %_sums = ();
        open FILE, "<$fn" or die "unable to open $fn: $!\n";
        while ( <FILE> ) {
        my ($k,$v) = split ',', $_;
        $_sums{$k} += $v;
        }
        close FILE;

        #
        # maybe do something else here..

        # then cumulate data:
        $sums{$_} += $_sums{$_}
        for keys %_sums;
        #
        # as we return, %_sums is freed.. so is $fn
        return;
        }

        Its a basic example, but I don't know what you're doing. When the
        subroutine addUp() returns it frees all the data.. any anonymous
        structures that were contained in its data structure are also freed
        _UNLESS_ there is still a reference to them still in scope. This is why
        you're discouraged from using global variables and hastily predeclaring
        like your comp sci teachers taught you in 101 and 201 ;).

        if you leave the %bufRef hash filled with references and its global,
        those anonymous arrays will still be in scope, and they will not be
        collected.

        Sure this sounds terrible, but in OO Perl, we can (ab)use this to do
        private variables. Consider:

        #
        # Use a Block to set a local scope
        {
        my %props = map { $_ => 1 } qw(name age gender);

        my %_data = ();

        #
        # BOTH %props and %_data are IN SCOPE here.
        sub set {
        my ($k,$v) = @_;
        return unless exists %props{lc $k};
        return $_data{lc $k} = $v;
        }

        sub get {
        my $k = shift;
        return unless exists %props{lc $k};
        return $_data{lc $k};
        }

        };

        #
        # %props and %_data are not in scope, but because our get/set methods
        # reference them, they are NOT garbage collected.

        # Subroutines are global entries in the package namespace, so they never
        # "go away" and so we can do this like this:

        set('age', 15);

        print 'I am : ', get('age'), "years old\n";

        There, everything you wanted to know and more that you don't even care
        about..

        On Tue, Jul 05, 2005 at 10:17:59PM +0100, Ken Shail wrote:
        > How can I make sure that anonymous arrays do not get left around.
        > It is difficult to isolate the code but it goes like this
        >
        > file is opened regularly for reading
        > foreach my $temp(<FILEHANDLE>)
        > {
        > @buffer=split /,/,$temp; # each line of the file is a comma-separated
        > list
        > # the first item in the list is the name by which it gets referred to later
        > $bufRef{$buffer[0]}=[@buffer];
        > }
        >
        > later on I want to find a specific list by the $name
        >
        > my $reff=$bufRef{$name};
        > @recoveredList=@$reff;
        >
        > If I continue to do this will I not eventually run out of memory or does
        > Perl "know" to destroy the old anonymous arrays each time I run the foreach
        > loop again?
        >
        > Ken
        >
        >
        >
        > Unsubscribing info is here: http://help.yahoo.com/help/us/groups/groups-32.html
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
        >

        --
        Brad Lhotsky
      • Ken at Home
        Hi Brad Thanks for such a comprehensive answer and for spending so much time on my humble question. I was opening multiple files. Each file has an undefined
        Message 3 of 3 , Jul 6, 2005
        • 0 Attachment
          Hi Brad
          Thanks for such a comprehensive answer and for spending so much time on my
          humble question.

          I was opening multiple files. Each file has an undefined number of text
          lines. A line is a data record in a fixed, known format comma-separated.
          E.G. the first item is the mac address of a computer and therefore
          terrestrially exclusive. The rest of the line (list) is time-dependant
          status data from that computer.
          The files do not exist at any one time, they get generated sequentially and
          written to by another process.
          Each file does not necessarily have data from all mac addresses. The mac
          addresses can pre-exist or can be new.

          My first, foreach block got run whenever a new file is discovered. I admit
          that reading from a file with a foreach is a little unconventional and that
          was a result of me reducing the code for simplicity together with general
          incompetence.

          I also have to admit that my anonymous arrays were global and therefore ( I
          presume) always in scope. I wanted to use the hash to hold their references
          as an easy way to get random access to the appropriate anonymous array later
          by mac address. An anonymous array may have some new data concerning a
          computer that has already had a hash{ref } and therefore the same mac
          address key. Would perl be clever enough to see that the hash key now had a
          new value (reference) and therefore backtrack the old reference to destroy
          is anonymous array. That would be clever!

          Anyway, with lots of red-lights and warning bells at the back of my mind
          about memory-leaks I decided to simplify the method and just hold each line
          in the hash before splitting it and then split it only when accessed
          later-on. That way it will only be the hash that will grow, one element per
          new mac address.

          Thanks again
          Ken


          ----- Original Message -----
          From: Brad Lhotsky
          To: perl-beginner@yahoogroups.com
          Sent: Wednesday, July 06, 2005 4:19 PM
          Subject: Re: [PBML] Destroying anonymous arrays


          perl does automatic garbage collection, but the structures have to go
          out of scope. This is why you'll usually hear people recommend that you
          declare a variable in the narrowest scope possible.

          If there's no references to the anonymous array, it will go away.. so if
          you undef your $reff and %bufRef=(); you should be able to put things
          back in there without a problem.

          I'm not entirely sure what you're trying to do by your example though.
          Are you opening multiple files? are you reopening the same file? I don't
          usually see the foreach() for reading from files.. There are CSV
          Modules on CPAN that might simplify your process some..

          lets say we've got multiple files and we're doing stuff with the files'
          data.

          my @files = qw(a.txt b.txt);
          my %sums = ();

          foreach my $file (@files) {
          addUp($file);
          }

          sub addUp {
          my $fn = shift;
          my %_sums = ();
          open FILE, "<$fn" or die "unable to open $fn: $!\n";
          while ( <FILE> ) {
          my ($k,$v) = split ',', $_;
          $_sums{$k} += $v;
          }
          close FILE;

          #
          # maybe do something else here..

          # then cumulate data:
          $sums{$_} += $_sums{$_}
          for keys %_sums;
          #
          # as we return, %_sums is freed.. so is $fn
          return;
          }

          Its a basic example, but I don't know what you're doing. When the
          subroutine addUp() returns it frees all the data.. any anonymous
          structures that were contained in its data structure are also freed
          _UNLESS_ there is still a reference to them still in scope. This is why
          you're discouraged from using global variables and hastily predeclaring
          like your comp sci teachers taught you in 101 and 201 ;).

          if you leave the %bufRef hash filled with references and its global,
          those anonymous arrays will still be in scope, and they will not be
          collected.

          Sure this sounds terrible, but in OO Perl, we can (ab)use this to do
          private variables. Consider:

          #
          # Use a Block to set a local scope
          {
          my %props = map { $_ => 1 } qw(name age gender);

          my %_data = ();

          #
          # BOTH %props and %_data are IN SCOPE here.
          sub set {
          my ($k,$v) = @_;
          return unless exists %props{lc $k};
          return $_data{lc $k} = $v;
          }

          sub get {
          my $k = shift;
          return unless exists %props{lc $k};
          return $_data{lc $k};
          }

          };

          #
          # %props and %_data are not in scope, but because our get/set methods
          # reference them, they are NOT garbage collected.

          # Subroutines are global entries in the package namespace, so they never
          # "go away" and so we can do this like this:

          set('age', 15);

          print 'I am : ', get('age'), "years old\n";

          There, everything you wanted to know and more that you don't even care
          about..

          On Tue, Jul 05, 2005 at 10:17:59PM +0100, Ken Shail wrote:
          > How can I make sure that anonymous arrays do not get left around.
          > It is difficult to isolate the code but it goes like this
          >
          > file is opened regularly for reading
          > foreach my $temp(<FILEHANDLE>)
          > {
          > @buffer=split /,/,$temp; # each line of the file is a comma-separated
          > list
          > # the first item in the list is the name by which it gets referred to
          later
          > $bufRef{$buffer[0]}=[@buffer];
          > }
          >
          > later on I want to find a specific list by the $name
          >
          > my $reff=$bufRef{$name};
          > @recoveredList=@$reff;
          >
          > If I continue to do this will I not eventually run out of memory or does
          > Perl "know" to destroy the old anonymous arrays each time I run the
          foreach
          > loop again?
          >
          > Ken
          >
          >
          >
          > Unsubscribing info is here:
          http://help.yahoo.com/help/us/groups/groups-32.html
          > Yahoo! Groups Links
          >
          >
          >
          >
          >
          >

          --
          Brad Lhotsky


          Unsubscribing info is here:
          http://help.yahoo.com/help/us/groups/groups-32.html




          YAHOO! GROUPS LINKS

          Visit your group "perl-beginner" on the web.

          To unsubscribe from this group, send an email to:
          perl-beginner-unsubscribe@yahoogroups.com

          Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.
        Your message has been successfully submitted and would be delivered to recipients shortly.