Loading ...
Sorry, an error occurred while loading the content.
 

RE: [PBML] Memory release problem?

Expand Messages
  • michael watson (IAH-C)
    Hi Thanks for the response. The data is directed to a file, not to the screen, and there are plenty of new-lines in there so perl should be flushing it s
    Message 1 of 9 , Oct 3 1:22 AM
      Hi

      Thanks for the response. The data is directed to a file, not to the
      screen, and there are plenty of new-lines in there so perl should be
      flushing it's buffers enough times.

      Mick

      ________________________________

      From: perl-beginner@yahoogroups.com
      [mailto:perl-beginner@yahoogroups.com] On Behalf Of Ramesh Polepalli
      Sent: 03 October 2007 06:37
      To: perl-beginner@yahoogroups.com
      Subject: RE: [PBML] Memory release problem?



      I Guess,

      The problem is due to the limitation of screen buffer.

      I would suggest you to,

      Try redirecting the output to a file instead of screen and

      share your experience.

      ________________________________

      From: perl-beginner@yahoogroups.com
      <mailto:perl-beginner%40yahoogroups.com>
      [mailto:perl-beginner@yahoogroups.com
      <mailto:perl-beginner%40yahoogroups.com> ] On Behalf Of Mick Watson
      Sent: Tuesday, October 02, 2007 3:22 PM
      To: perl-beginner@yahoogroups.com
      <mailto:perl-beginner%40yahoogroups.com>
      Subject: [PBML] Memory release problem?

      Hi

      I am not a beginner to perl, but am a beginner to this problem!

      I create a perl object, read about 500,000 rows from a MySQL
      database, populate some structures and print out a table of results.
      I do this using DBI. I create one database handle, which I
      disconnect from, and about 6 statement handles, all of which I
      explicitly finish.

      After the results have printed, the script simply dies. However, the
      results finish printing after about 4 minutes from execution; the
      process then actually finishes some 20 minutes later - what could be
      going on?

      In my object's DESTROY method, I explicitly disconnect from the db
      handle (takes less than one second) and explicitly undef $self (takes
      less than one second). So I am guessing that the rest of the 20
      minutes is perl returning memory to the OS...? Why would it take so
      long? The server has 3Gb of RAM, so this should not be a problem.

      Thanks
      Mick

      I am running an old-ish version of SuSe Linux Enterprise Server 8.2.
      Could this be the problem?

      Ramesh Polepalli

      mFormation Technologies Inc.
      Direct Line: +91 80 6620 5939
      Switchboard: +91 80 6620 5900

      http://www.mformation.com <http://www.mformation.com>


      Visit WWW.MFORMATION.COM and see how the MFORMATION SERVICE MANAGER can
      help you to transform your business.

      [Non-text portions of this message have been removed]






      [Non-text portions of this message have been removed]
    • John Francini
      Ah, but Perl doesn t always flush output out to the terminal as it s generated unless you have the special variable $| (or $AUTOFLUSH if you used English) set
      Message 2 of 9 , Oct 3 6:07 AM
        Ah, but Perl doesn't always flush output out to the terminal as it's
        generated unless you have the special variable $| (or $AUTOFLUSH if
        you used English) set to 1. Otherwise, it may not flush until the
        output buffer is full or the program exits.

        John


        At 11:27 +0100 10/3/07, michael watson (IAH-C) wrote:
        >Hi
        >
        >I have "print STDERR..." statements throught my script, with a call to
        >`date` appended so I know when it happens.
        >I also have similar statements before and after disconnecting from MySQL
        >using DBI and undef-ing $self in my perl object.
        >
        >As far as I can tell, the script has completely finished executing, and
        >then there is about a 20 minute wait for the process to end.
        >
        >Mick
        >
        >________________________________
        >
        >From: <mailto:perl-beginner%40yahoogroups.com>perl-beginner@yahoogroups.com
        >[mailto:<mailto:perl-beginner%40yahoogroups.com>perl-beginner@yahoogroups.com]
        >On Behalf Of Jenda Krynicky
        >Sent: 03 October 2007 11:12
        >To: <mailto:perl-beginner%40yahoogroups.com>perl-beginner@yahoogroups.com
        >Subject: Re: [PBML] Memory release problem?
        >
        >On 2 Oct 2007 at 9:51, Mick Watson wrote:
        >> I am not a beginner to perl, but am a beginner to this problem!
        >>
        >> I create a perl object, read about 500,000 rows from a MySQL
        >> database, populate some structures and print out a table of results.
        >> I do this using DBI. I create one database handle, which I disconnect
        >> from, and about 6 statement handles, all of which I explicitly finish.
        >>
        >> After the results have printed, the script simply dies. However, the
        >> results finish printing after about 4 minutes from execution; the
        >> process then actually finishes some 20 minutes later - what could be
        >> going on?
        >>
        >> In my object's DESTROY method, I explicitly disconnect from the db
        >> handle (takes less than one second) and explicitly undef $self (takes
        >> less than one second). So I am guessing that the rest of the 20
        >> minutes is perl returning memory to the OS...? Why would it take so
        >> long? The server has 3Gb of RAM, so this should not be a problem.
        >
        >I don't think you are looking at the right place. Though it's hard to
        >
        >suggest where to look.
        >
        >What does it do if you add something like
        >
        >END {print "Exiting the script: 1"}
        >
        >before the use statements, after the use statements and at the end of
        >
        >the script Are these messages printed immedialtely after the normal
        >output of the script or at the end of those 20 minutes of waiting?
        >
        >Jenda
        >
        >===== <mailto:Jenda%40Krynicky.cz>Jenda@...
        ><mailto:Jenda%40Krynicky.cz> ===
        ><http://Jenda.Krynicky.cz>http://Jenda.Krynicky.cz
        ><<http://Jenda.Krynicky.cz>http://Jenda.Krynicky.cz> =====
        >When it comes to wine, women and song, wizards are allowed
        >to get drunk and croon as much as they like.
        >-- Terry Pratchett in Sourcery
        >
        >[Non-text portions of this message have been removed]
        >
        >

        --
        John Francini, francini@...

        "The journey is more important than the destination -- that's part of life.
        If you only live for getting to the end, you're almost always disappointed."
        -- Donald Knuth

        [Non-text portions of this message have been removed]
      • merlyn@stonehenge.com
        ... michael Hmmm, OK, I will try this, thank you, but as there are only 17000 rows michael of output, is it really possible perl takes 20 mins to write that
        Message 3 of 9 , Oct 4 6:31 AM
          >>>>> "michael" == michael watson \(IAH-C\) <michael> writes:

          michael> Hmmm, OK, I will try this, thank you, but as there are only 17000 rows
          michael> of output, is it really possible perl takes 20 mins to write that to a
          michael> file?!

          When you profiled your program using Devel::DProf or Devel::SmallProf,
          where did it show that it was spending its time? It would help to know
          that.

          --
          Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
          See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
        Your message has been successfully submitted and would be delivered to recipients shortly.