Loading ...
Sorry, an error occurred while loading the content.
 

RE: [PBML] Re: Eval and memory problems

Expand Messages
  • Boyle, Christopher
    http://faq.perl.org/ ________________________________ From: perl-beginner@yahoogroups.com [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
    Message 1 of 15 , Jan 8, 2008
      http://faq.perl.org/





      ________________________________

      From: perl-beginner@yahoogroups.com
      [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
      Sent: Tuesday, January 08, 2008 2:05 PM
      To: perl-beginner@yahoogroups.com
      Subject: [PBML] Re: Eval and memory problems





      which FAQ ?

      thanks by the way.

      --- In perl-beginner@yahoogroups.com
      <mailto:perl-beginner%40yahoogroups.com> , merlyn@... wrote:
      >
      > >>>>> "computerdribble" == computerdribble <computerdribble@...>
      writes:
      >
      > computerdribble> how do I macro expand ?
      >
      > Is the answer in the FAQ not clear enough?
      >
      > --
      > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
      777 0095
      > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/
      <http://www.stonehenge.com/merlyn/> >
      > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
      > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
      training!
      >


      ______________________________________________________________________
      This email has been scanned by the MessageLabs Email Security System.
      For more information please visit http://www.messagelabs.com/email
      ______________________________________________________________________



      NOTICE OF CONFIDENTIALITY: Information included in and/or attached to this electronic mail transmission may be confidential. This electronic mail transmission is intended for the addressee(s) only. Any unauthorized disclosure, reproduction, or distribution of, and/or any unauthorized action taken in reliance on the information in this electronic mail is prohibited. If you believe that you have received this electronic mail transmission in error, please notify the sender by reply transmission, or contact helpdesk@..., and delete the message without copying or disclosing it.

      ______________________________________________________________________
      This email has been scanned by the MessageLabs Email Security System.
      For more information please visit http://www.messagelabs.com/email
      ______________________________________________________________________

      [Non-text portions of this message have been removed]
    • computerdribble
      Nope , not in there. there is nothing in the FAQ concerning Macro expansion. Only Macros for Vi (ick) Went through that AGAIN and no luck. anywhere else? I
      Message 2 of 15 , Jan 8, 2008
        Nope , not in there. there is nothing in the FAQ concerning Macro
        expansion.

        Only Macros for Vi (ick)

        Went through that AGAIN and no luck.

        anywhere else?

        I really appreciate the pointers. I guess thats what the
        "perl-beginner" is all about.

        tia!




        --- In perl-beginner@yahoogroups.com, "Boyle, Christopher"
        <Christopher.Boyle@...> wrote:
        >
        > http://faq.perl.org/
        >
        >
        >
        >
        >
        > ________________________________
        >
        > From: perl-beginner@yahoogroups.com
        > [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
        > Sent: Tuesday, January 08, 2008 2:05 PM
        > To: perl-beginner@yahoogroups.com
        > Subject: [PBML] Re: Eval and memory problems
        >
        >
        >
        >
        >
        > which FAQ ?
        >
        > thanks by the way.
        >
        > --- In perl-beginner@yahoogroups.com
        > <mailto:perl-beginner%40yahoogroups.com> , merlyn@ wrote:
        > >
        > > >>>>> "computerdribble" == computerdribble <computerdribble@>
        > writes:
        > >
        > > computerdribble> how do I macro expand ?
        > >
        > > Is the answer in the FAQ not clear enough?
        > >
        > > --
        > > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
        > 777 0095
        > > <merlyn@> <URL:http://www.stonehenge.com/merlyn/
        > <http://www.stonehenge.com/merlyn/> >
        > > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
        > > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
        > training!
        > >
        >
        >
        > ______________________________________________________________________
        > This email has been scanned by the MessageLabs Email Security System.
        > For more information please visit http://www.messagelabs.com/email
        > ______________________________________________________________________
        >
        >
        >
        > NOTICE OF CONFIDENTIALITY: Information included in and/or attached
        to this electronic mail transmission may be confidential. This
        electronic mail transmission is intended for the addressee(s) only.
        Any unauthorized disclosure, reproduction, or distribution of, and/or
        any unauthorized action taken in reliance on the information in this
        electronic mail is prohibited. If you believe that you have received
        this electronic mail transmission in error, please notify the sender
        by reply transmission, or contact helpdesk@..., and delete the message
        without copying or disclosing it.
        >
        > ______________________________________________________________________
        > This email has been scanned by the MessageLabs Email Security System.
        > For more information please visit http://www.messagelabs.com/email
        > ______________________________________________________________________
        >
        > [Non-text portions of this message have been removed]
        >
      • merlyn@stonehenge.com
        ... computerdribble Nope , not in there. there is nothing in the FAQ concerning Macro computerdribble expansion. By the way, it s on *YOUR DISK*. No need
        Message 3 of 15 , Jan 8, 2008
          >>>>> "computerdribble" == computerdribble <computerdribble@...> writes:

          computerdribble> Nope , not in there. there is nothing in the FAQ concerning Macro
          computerdribble> expansion.

          By the way, it's on *YOUR DISK*. No need to go off to the net for the FAQ.
          Type "perldoc perlfaq". And you can search it too, which I will now do for
          you, to show you the FAQ *does* cover what you're looking for.

          I suggest you read the subject lines of "perldoc perlfaq" and "perldoc perl"
          weekly until you can recite them from memory. Perl comes with a HUGE amount
          of info, but it's pointless if you don't know what's there.

          $ perldoc -q expand

          Found in /usr/libdata/perl5/pod/perlfaq4.pod
          How do I expand function calls in a string?
          (contributed by brian d foy)

          This is documented in perlref, and although it's not the easiest thing
          to read, it does work. In each of these examples, we call the function
          inside the braces used to dereference a reference. If we have a more
          than one return value, we can construct and dereference an anonymous
          array. In this case, we call the function in list context.

          print "The time values are @{ [localtime] }.\n";

          If we want to call the function in scalar context, we have to do a bit
          more work. We can really have any code we like inside the braces, so we
          simply have to end with the scalar reference, although how you do that
          is up to you, and you can use code inside the braces.

          print "The time is ${\(scalar localtime)}.\n"

          print "The time is ${ my $x = localtime; \$x }.\n";

          If your function already returns a reference, you don't need to create
          the reference yourself.

          sub timestamp { my $t = localtime; \$t }

          print "The time is ${ timestamp() }.\n";

          The "Interpolation" module can also do a lot of magic for you. You can
          specify a variable name, in this case "E", to set up a tied hash that
          does the interpolation for you. It has several other methods to do this
          as well.

          use Interpolation E => 'eval';
          print "The time values are $E{localtime()}.\n";

          In most cases, it is probably easier to simply use string concatenation,
          which also forces scalar context.

          print "The time is " . localtime . ".\n";

          How do I expand tabs in a string?
          You can do it yourself:

          1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;

          Or you can just use the Text::Tabs module (part of the standard Perl
          distribution).

          use Text::Tabs;
          @expanded_lines = expand(@lines_with_tabs);

          How can I expand variables in text strings?
          Let's assume that you have a string that contains placeholder variables.

          $text = 'this has a $foo in it and a $bar';

          You can use a substitution with a double evaluation. The first /e turns
          $1 into $foo, and the second /e turns $foo into its value. You may want
          to wrap this in an "eval": if you try to get the value of an undeclared
          variable while running under "use strict", you get a fatal error.

          eval { $text =~ s/(\$\w+)/$1/eeg };
          die if $@;

          It's probably better in the general case to treat those variables as
          entries in some special hash. For example:

          %user_defs = (
          foo => 23,
          bar => 19,
          );
          $text =~ s/\$(\w+)/$user_defs{$1}/g;




          --
          Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
          See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
        • computerdribble
          Thank you. I went onto the faq.perl.org site and searched through all of that and nothing came back. This is one of the problems and reasons I have stayed
          Message 4 of 15 , Jan 8, 2008
            Thank you.

            I went onto the faq.perl.org site and searched through all of that
            and nothing came back.

            This is one of the problems and reasons I have stayed away from perl
            for so long. There's SO much to it and it HAS its OWN quirkiness that
            outpaces other languages.


            but again, thank you!


            Now, one other question....


            this works:
            $text =~ s/(\$\w+)/$1/eeg

            it takes;
            $aa="hello";
            $bb="goodbye";


            and inserts it into;


            $text = "( ( $aa =~ /lo/ ) and ($bb =~ /ye/ ) )";


            perfectly to create:


            ( ( hello =~ /lo/ ) and ( goodbye =~ /ye/ ) )



            notice the problem ? No quotes around the $aa expanded variable. So the
            if ( @{ [ $text ] } )

            always fails.



            I'm NOT a regex person and stumble my way through it so far.. but this
            is beyond me at the moment. (yes more reading is required)

            How do I get the expansion (or can it be done? ) to automatically put
            the quotes around the variables expanded?

            I can add the quotes into the database around the variables, but then
            its a cludge.

            thanks again and thanks in advance...


            joe















            --- In perl-beginner@yahoogroups.com, merlyn@... wrote:
            >
            > >>>>> "computerdribble" == computerdribble <computerdribble@...>
            writes:
            >
            > computerdribble> Nope , not in there. there is nothing in the FAQ
            concerning Macro
            > computerdribble> expansion.
            >
            > By the way, it's on *YOUR DISK*. No need to go off to the net for
            the FAQ.
            > Type "perldoc perlfaq". And you can search it too, which I will now
            do for
            > you, to show you the FAQ *does* cover what you're looking for.
            >
            > I suggest you read the subject lines of "perldoc perlfaq" and
            "perldoc perl"
            > weekly until you can recite them from memory. Perl comes with a
            HUGE amount
            > of info, but it's pointless if you don't know what's there.
            >
            > $ perldoc -q expand
            >
            > Found in /usr/libdata/perl5/pod/perlfaq4.pod
            > How do I expand function calls in a string?
            > (contributed by brian d foy)
            >
            > This is documented in perlref, and although it's not the easiest
            thing
            > to read, it does work. In each of these examples, we call the
            function
            > inside the braces used to dereference a reference. If we have a more
            > than one return value, we can construct and dereference an anonymous
            > array. In this case, we call the function in list context.
            >
            > print "The time values are @{ [localtime] }.\n";
            >
            > If we want to call the function in scalar context, we have to do
            a bit
            > more work. We can really have any code we like inside the
            braces, so we
            > simply have to end with the scalar reference, although how you
            do that
            > is up to you, and you can use code inside the braces.
            >
            > print "The time is ${\(scalar localtime)}.\n"
            >
            > print "The time is ${ my $x = localtime; \$x }.\n";
            >
            > If your function already returns a reference, you don't need to
            create
            > the reference yourself.
            >
            > sub timestamp { my $t = localtime; \$t }
            >
            > print "The time is ${ timestamp() }.\n";
            >
            > The "Interpolation" module can also do a lot of magic for you.
            You can
            > specify a variable name, in this case "E", to set up a tied hash
            that
            > does the interpolation for you. It has several other methods to
            do this
            > as well.
            >
            > use Interpolation E => 'eval';
            > print "The time values are $E{localtime()}.\n";
            >
            > In most cases, it is probably easier to simply use string
            concatenation,
            > which also forces scalar context.
            >
            > print "The time is " . localtime . ".\n";
            >
            > How do I expand tabs in a string?
            > You can do it yourself:
            >
            > 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`)
            % 8)/e;
            >
            > Or you can just use the Text::Tabs module (part of the standard Perl
            > distribution).
            >
            > use Text::Tabs;
            > @expanded_lines = expand(@lines_with_tabs);
            >
            > How can I expand variables in text strings?
            > Let's assume that you have a string that contains placeholder
            variables.
            >
            > $text = 'this has a $foo in it and a $bar';
            >
            > You can use a substitution with a double evaluation. The first
            /e turns
            > $1 into $foo, and the second /e turns $foo into its value. You
            may want
            > to wrap this in an "eval": if you try to get the value of an
            undeclared
            > variable while running under "use strict", you get a fatal error.
            >
            > eval { $text =~ s/(\$\w+)/$1/eeg };
            > die if $@;
            >
            > It's probably better in the general case to treat those variables as
            > entries in some special hash. For example:
            >
            > %user_defs = (
            > foo => 23,
            > bar => 19,
            > );
            > $text =~ s/\$(\w+)/$user_defs{$1}/g;
            >
            >
            >
            >
            > --
            > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
            777 0095
            > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
            > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
            > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
            training!
            >
          • Boyle, Christopher
            perldoc -quote ________________________________ From: perl-beginner@yahoogroups.com [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble Sent:
            Message 5 of 15 , Jan 8, 2008
              perldoc -quote



              ________________________________

              From: perl-beginner@yahoogroups.com
              [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
              Sent: Tuesday, January 08, 2008 3:38 PM
              To: perl-beginner@yahoogroups.com
              Subject: [PBML] Re: Eval and memory problems




              Thank you.

              I went onto the faq.perl.org site and searched through all of that
              and nothing came back.

              This is one of the problems and reasons I have stayed away from perl
              for so long. There's SO much to it and it HAS its OWN quirkiness that
              outpaces other languages.

              but again, thank you!

              Now, one other question....

              this works:
              $text =~ s/(\$\w+)/$1/eeg

              it takes;
              $aa="hello";
              $bb="goodbye";

              and inserts it into;

              $text = "( ( $aa =~ /lo/ ) and ($bb =~ /ye/ ) )";

              perfectly to create:

              ( ( hello =~ /lo/ ) and ( goodbye =~ /ye/ ) )

              notice the problem ? No quotes around the $aa expanded variable. So the
              if ( @{ [ $text ] } )

              always fails.

              I'm NOT a regex person and stumble my way through it so far.. but this
              is beyond me at the moment. (yes more reading is required)

              How do I get the expansion (or can it be done? ) to automatically put
              the quotes around the variables expanded?

              I can add the quotes into the database around the variables, but then
              its a cludge.

              thanks again and thanks in advance...

              joe

              --- In perl-beginner@yahoogroups.com
              <mailto:perl-beginner%40yahoogroups.com> , merlyn@... wrote:
              >
              > >>>>> "computerdribble" == computerdribble <computerdribble@...>
              writes:
              >
              > computerdribble> Nope , not in there. there is nothing in the FAQ
              concerning Macro
              > computerdribble> expansion.
              >
              > By the way, it's on *YOUR DISK*. No need to go off to the net for
              the FAQ.
              > Type "perldoc perlfaq". And you can search it too, which I will now
              do for
              > you, to show you the FAQ *does* cover what you're looking for.
              >
              > I suggest you read the subject lines of "perldoc perlfaq" and
              "perldoc perl"
              > weekly until you can recite them from memory. Perl comes with a
              HUGE amount
              > of info, but it's pointless if you don't know what's there.
              >
              > $ perldoc -q expand
              >
              > Found in /usr/libdata/perl5/pod/perlfaq4.pod
              > How do I expand function calls in a string?
              > (contributed by brian d foy)
              >
              > This is documented in perlref, and although it's not the easiest
              thing
              > to read, it does work. In each of these examples, we call the
              function
              > inside the braces used to dereference a reference. If we have a more
              > than one return value, we can construct and dereference an anonymous
              > array. In this case, we call the function in list context.
              >
              > print "The time values are @{ [localtime] }.\n";
              >
              > If we want to call the function in scalar context, we have to do
              a bit
              > more work. We can really have any code we like inside the
              braces, so we
              > simply have to end with the scalar reference, although how you
              do that
              > is up to you, and you can use code inside the braces.
              >
              > print "The time is ${\(scalar localtime)}.\n"
              >
              > print "The time is ${ my $x = localtime; \$x }.\n";
              >
              > If your function already returns a reference, you don't need to
              create
              > the reference yourself.
              >
              > sub timestamp { my $t = localtime; \$t }
              >
              > print "The time is ${ timestamp() }.\n";
              >
              > The "Interpolation" module can also do a lot of magic for you.
              You can
              > specify a variable name, in this case "E", to set up a tied hash
              that
              > does the interpolation for you. It has several other methods to
              do this
              > as well.
              >
              > use Interpolation E => 'eval';
              > print "The time values are $E{localtime()}.\n";
              >
              > In most cases, it is probably easier to simply use string
              concatenation,
              > which also forces scalar context.
              >
              > print "The time is " . localtime . ".\n";
              >
              > How do I expand tabs in a string?
              > You can do it yourself:
              >
              > 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`)
              % 8)/e;
              >
              > Or you can just use the Text::Tabs module (part of the standard Perl
              > distribution).
              >
              > use Text::Tabs;
              > @expanded_lines = expand(@lines_with_tabs);
              >
              > How can I expand variables in text strings?
              > Let's assume that you have a string that contains placeholder
              variables.
              >
              > $text = 'this has a $foo in it and a $bar';
              >
              > You can use a substitution with a double evaluation. The first
              /e turns
              > $1 into $foo, and the second /e turns $foo into its value. You
              may want
              > to wrap this in an "eval": if you try to get the value of an
              undeclared
              > variable while running under "use strict", you get a fatal error.
              >
              > eval { $text =~ s/(\$\w+)/$1/eeg };
              > die if $@;
              >
              > It's probably better in the general case to treat those variables as
              > entries in some special hash. For example:
              >
              > %user_defs = (
              > foo => 23,
              > bar => 19,
              > );
              > $text =~ s/\$(\w+)/$user_defs{$1}/g;
              >
              >
              >
              >
              > --
              > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
              777 0095
              > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/
              <http://www.stonehenge.com/merlyn/> >
              > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
              > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
              training!
              >


              ______________________________________________________________________
              This email has been scanned by the MessageLabs Email Security System.
              For more information please visit http://www.messagelabs.com/email
              ______________________________________________________________________



              NOTICE OF CONFIDENTIALITY: Information included in and/or attached to this electronic mail transmission may be confidential. This electronic mail transmission is intended for the addressee(s) only. Any unauthorized disclosure, reproduction, or distribution of, and/or any unauthorized action taken in reliance on the information in this electronic mail is prohibited. If you believe that you have received this electronic mail transmission in error, please notify the sender by reply transmission, or contact helpdesk@..., and delete the message without copying or disclosing it.

              ______________________________________________________________________
              This email has been scanned by the MessageLabs Email Security System.
              For more information please visit http://www.messagelabs.com/email
              ______________________________________________________________________

              [Non-text portions of this message have been removed]
            • Jenda Krynicky
              From: computerdribble ... Compile each test just once and run it repeatedly. Before opening the log file, but after reading the
              Message 6 of 15 , Jan 8, 2008
                From: "computerdribble" <computerdribble@...>
                > The "specific things" are stored in a mysql database as a regex expression
                > Example:
                >
                > ( ( $field =~ /blah/) and ($field =~ /that/) )
                >
                > I open the MySql database, no problem, load the contents into arrays,
                > no problem.
                >
                > Open the log file (approx 1.48megs in size) for reading only.
                >
                > I have a logic loop
                >
                > grab a line from the log file
                >
                > run all regex expressions against the log line grabbed. This could be
                > from 1 to 500 different tests.
                >
                >
                > its done in a EVAL statement.
                >
                > similar to:
                >
                > if ( eval $array[$vartoarraycontents] )

                Compile each test just once and run it repeatedly.
                Before opening the log file, but after reading the expressions do
                something like

                for my $test (@array) {
                $test = eval 'sub { ' . $test . '}'
                or die "Failed to compile '$test' : $@\n";
                }

                and then while looping through the logfile use

                if ($array[$vartoarraycontents]->()) {

                There is a possible catch though! The variables you plan to make
                available to the tests must be declared before the tests are compiled
                by those eval('sub {'...) statements!

                Jenda
                ===== Jenda@... === http://Jenda.Krynicky.cz =====
                When it comes to wine, women and song, wizards are allowed
                to get drunk and croon as much as they like.
                -- Terry Pratchett in Sourcery
              • Computer-Dribble
                Thank you! One problem I ran into was the EVAL was killing 4 gigs of memeory on large log files. At least I traced it back to that. This is why I wanted to
                Message 7 of 15 , Jan 9, 2008
                  Thank you!


                  One problem I ran into was the EVAL was killing 4 gigs
                  of memeory on large log files. At least I traced it
                  back to that.

                  This is why I wanted to avoid EVAL completely and do
                  some macro expansion.

                  thanks again!!

                  Joe.

                  --- Jenda Krynicky <Jenda@...> wrote:

                  > From: "computerdribble" <computerdribble@...>
                  > > The "specific things" are stored in a mysql
                  > database as a regex expression
                  > > Example:
                  > >
                  > > ( ( $field =~ /blah/) and ($field =~ /that/) )
                  > >
                  > > I open the MySql database, no problem, load the
                  > contents into arrays,
                  > > no problem.
                  > >
                  > > Open the log file (approx 1.48megs in size) for
                  > reading only.
                  > >
                  > > I have a logic loop
                  > >
                  > > grab a line from the log file
                  > >
                  > > run all regex expressions against the log line
                  > grabbed. This could be
                  > > from 1 to 500 different tests.
                  > >
                  > >
                  > > its done in a EVAL statement.
                  > >
                  > > similar to:
                  > >
                  > > if ( eval $array[$vartoarraycontents] )
                  >
                  > Compile each test just once and run it repeatedly.
                  > Before opening the log file, but after reading the
                  > expressions do
                  > something like
                  >
                  > for my $test (@array) {
                  > $test = eval 'sub { ' . $test . '}'
                  > or die "Failed to compile '$test' : $@\n";
                  > }
                  >
                  > and then while looping through the logfile use
                  >
                  > if ($array[$vartoarraycontents]->()) {
                  >
                  > There is a possible catch though! The variables you
                  > plan to make
                  > available to the tests must be declared before the
                  > tests are compiled
                  > by those eval('sub {'...) statements!
                  >
                  > Jenda
                  > ===== Jenda@... === http://Jenda.Krynicky.cz
                  > =====
                  > When it comes to wine, women and song, wizards are
                  > allowed
                  > to get drunk and croon as much as they like.
                  > -- Terry Pratchett in Sourcery
                  >
                  >
                  >
                  > Unsubscribing info is here:
                  > http://help.yahoo.com/help/us/groups/groups-32.html
                  > Yahoo! Groups Links
                  >
                  >
                  > mailto:perl-beginner-fullfeatured@yahoogroups.com
                  >
                  >
                  >



                  ____________________________________________________________________________________
                  Never miss a thing. Make Yahoo your home page.
                  http://www.yahoo.com/r/hs
                • Jenda Krynicky
                  - No I don t. And others don t either. - But you do not need to read what I wrote originaly and what you replied last time, you remember it, don t you? -
                  Message 8 of 15 , Jan 9, 2008
                    - No I don't. And others don't either.
                    - But you do not need to read what I wrote originaly and what you
                    replied last time, you remember it, don't you?
                    - Because it's all backwards.
                    - Why is that?
                    - Because it's hard to read.
                    - Why?
                    - Please do not top-post!

                    Since the proposed change decreases the number of eval""s from
                    (number of tests)*(number of lines in logfile)
                    to just
                    (number of tests)
                    any leaks in eval"" should be irrelevant.

                    Jenda

                    From: Computer-Dribble <computerdribble@...>
                    > One problem I ran into was the EVAL was killing 4 gigs
                    > of memeory on large log files. At least I traced it
                    > back to that.
                    >
                    > This is why I wanted to avoid EVAL completely and do
                    > some macro expansion.
                    >
                    > thanks again!!
                    >
                    > Joe.
                    >
                    > --- Jenda Krynicky <Jenda@...> wrote:
                    >
                    > > From: "computerdribble" <computerdribble@...>
                    > > > The "specific things" are stored in a mysql
                    > > database as a regex expression
                    > > > Example:
                    > > >
                    > > > ( ( $field =~ /blah/) and ($field =~ /that/) )
                    > > >
                    > > > I open the MySql database, no problem, load the
                    > > contents into arrays,
                    > > > no problem.
                    > > >
                    > > > Open the log file (approx 1.48megs in size) for
                    > > reading only.
                    > > >
                    > > > I have a logic loop
                    > > >
                    > > > grab a line from the log file
                    > > >
                    > > > run all regex expressions against the log line
                    > > grabbed. This could be
                    > > > from 1 to 500 different tests.
                    > > >
                    > > >
                    > > > its done in a EVAL statement.
                    > > >
                    > > > similar to:
                    > > >
                    > > > if ( eval $array[$vartoarraycontents] )
                    > >
                    > > Compile each test just once and run it repeatedly.
                    > > Before opening the log file, but after reading the
                    > > expressions do
                    > > something like
                    > >
                    > > for my $test (@array) {
                    > > $test = eval 'sub { ' . $test . '}'
                    > > or die "Failed to compile '$test' : $@\n";
                    > > }
                    > >
                    > > and then while looping through the logfile use
                    > >
                    > > if ($array[$vartoarraycontents]->()) {
                    > >
                    > > There is a possible catch though! The variables you
                    > > plan to make
                    > > available to the tests must be declared before the
                    > > tests are compiled
                    > > by those eval('sub {'...) statements!
                    > >
                    > > Jenda
                    > > ===== Jenda@... === http://Jenda.Krynicky.cz
                    > > =====
                    > > When it comes to wine, women and song, wizards are
                    > > allowed
                    > > to get drunk and croon as much as they like.
                    > > -- Terry Pratchett in Sourcery
                    > >
                    > >
                    > >
                    > > Unsubscribing info is here:
                    > > http://help.yahoo.com/help/us/groups/groups-32.html
                    > > Yahoo! Groups Links
                    > >
                    > >
                    > > mailto:perl-beginner-fullfeatured@yahoogroups.com
                    > >
                    > >
                    > >
                    >
                    >
                    >
                    > ____________________________________________________________________________________
                    > Never miss a thing. Make Yahoo your home page.
                    > http://www.yahoo.com/r/hs
                    >
                    >
                    > Unsubscribing info is here: http://help.yahoo.com/help/us/groups/groups-32.html
                    > Yahoo! Groups Links
                    >
                    >
                    >
                    >


                    ===== Jenda@... === http://Jenda.Krynicky.cz =====
                    When it comes to wine, women and song, wizards are allowed
                    to get drunk and croon as much as they like.
                    -- Terry Pratchett in Sourcery
                  • computerdribble
                    I humbly thank ALL who have read or responded. Can you, with all of the examples to date... do away with the EVAL of the variable that contains the testing
                    Message 9 of 15 , Jan 9, 2008
                      I humbly thank ALL who have read or responded.

                      Can you, with all of the examples to date...
                      do away with the EVAL of the variable that contains the testing
                      information and just do a straight macro expansion ?

                      $this = "that"
                      $that ="this"

                      $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '

                      something like @{[ $$test ] }


                      I'm sorry to have ruffled so many feathers, but just because one has
                      20 years of experience in a language does not mean they should look
                      down on someone TRYING to learn it. Much less understand it andits
                      quirkiness.

                      Usually, a pre-madonna is a target in the computer field, remember that.

                      I find that perl is never as straight forward as one thinks. It takes
                      a great deal of trial and error to get it to do what you want it to do
                      .. for a beginner, its frustrating. For someone with 30 years
                      experince in other languages, its VERY frustrating. Unless I
                      completely dive into the language full time.. I'll never gain the
                      knowledge you guys have. But I don't need to. This is a 1 time
                      thing. I can go someone else to find the answers BUT... I thought
                      that this was a "perl-beginner" group/forum not a "lets pick on
                      someone trying to learn the language" area.
                      Is this what "knowledge sharing" has degraded to ?

                      Most of the "trial and error" is because of the REGEX. Each language,
                      has its own way of looking at REGEX and Perl is notorious for its own
                      quirks.

                      I bow before greater GODs than I in the Perl field and I humbly do so.
                      If I had my way, I would have written this in another language
                      altogether, but a client wants what a client wants.

                      As for "top loading or top responding" Not my fault, blame Yahoo for
                      doing that way.. hell blame the whole freaking internet for it,
                      because that's just how it is done. Talk to the hand.






                      Joe.
                    • Jenda Krynicky
                      From: computerdribble ... No. You can t. It s as simple as that. If you did not store the expression like this in the database,
                      Message 10 of 15 , Jan 9, 2008
                        From: "computerdribble" <computerdribble@...>
                        > I humbly thank ALL who have read or responded.
                        >
                        > Can you, with all of the examples to date...
                        > do away with the EVAL of the variable that contains the testing
                        > information and just do a straight macro expansion ?
                        >
                        > $this = "that"
                        > $that ="this"
                        >
                        > $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '
                        >
                        > something like @{[ $$test ] }

                        No. You can't. It's as simple as that.

                        If you did not store the expression like this in the database, but
                        instead stored the name of the variable and the regexp (and nothing
                        else) in separate columns you could do something like

                        if ($data{$varname} =~ /$regexp/) {

                        where the $varname and $regexp comes from the database. Whether that
                        would be enough for you, I don't know.

                        Even in this case though it would be better to preprocess the
                        regexps:

                        foreach my $test (@tests) {
                        # $test = [$varname, $regexp]
                        eval {
                        $test[1] = qr/$test[1]/
                        } or die qq{Invalid regexp /$test[1]/ : $@\n};
                        }

                        This way the regexps are parsed and compiled just once instead of
                        each time they are used.

                        > I'm sorry to have ruffled so many feathers, but just because one has
                        > 20 years of experience in a language does not mean they should look
                        > down on someone TRYING to learn it. Much less understand it andits
                        > quirkiness.
                        >
                        > ...

                        I think all this was unnecessary. Noone's picking on you.

                        > As for "top loading or top responding" Not my fault, blame Yahoo for
                        > doing that way.. hell blame the whole freaking internet for it,
                        > because that's just how it is done. Talk to the hand.

                        The fact that Yahoo (by default?) formats replies one way doesn't
                        mean you have to keep it that way. Top-posting is quite OK for
                        personal emails to a single recipient. For mailing lists its ... not
                        working too well. Always try to make it easy to the people you are
                        asking for help, even if it means you spend two more minutes
                        formatting the email.

                        Jenda
                        ===== Jenda@... === http://Jenda.Krynicky.cz =====
                        When it comes to wine, women and song, wizards are allowed
                        to get drunk and croon as much as they like.
                        -- Terry Pratchett in Sourcery
                      • computerdribble
                        Thank you for responding. ... dang it all! oh well. ... I would love to.. except there are certain instances that what needs to be checked is.. well let me
                        Message 11 of 15 , Jan 9, 2008
                          Thank you for responding.


                          --- In perl-beginner@yahoogroups.com, "Jenda Krynicky" <Jenda@...> wrote:
                          >
                          > From: "computerdribble" <computerdribble@...>
                          > > I humbly thank ALL who have read or responded.
                          > >
                          > > Can you, with all of the examples to date...
                          > > do away with the EVAL of the variable that contains the testing
                          > > information and just do a straight macro expansion ?
                          > >
                          > > $this = "that"
                          > > $that ="this"
                          > >
                          > > $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '
                          > >
                          > > something like @{[ $$test ] }
                          >
                          > No. You can't. It's as simple as that.
                          >


                          dang it all!
                          oh well.


                          > If you did not store the expression like this in the database, but
                          > instead stored the name of the variable and the regexp (and nothing
                          > else) in separate columns you could do something like
                          >
                          > if ($data{$varname} =~ /$regexp/) {
                          >
                          > where the $varname and $regexp comes from the database. Whether that
                          > would be enough for you, I don't know.
                          >
                          > Even in this case though it would be better to preprocess the
                          > regexps:
                          >
                          > foreach my $test (@tests) {
                          > # $test = [$varname, $regexp]
                          > eval {
                          > $test[1] = qr/$test[1]/
                          > } or die qq{Invalid regexp /$test[1]/ : $@\n};
                          > }
                          >


                          I would love to.. except there are certain instances that what needs
                          to be checked is.. well let me put it this way:


                          (( $field =~ /this/) and ( ($field =~ /that/) or ($field1 =~/a/) or
                          $field2 =~/xx/) or... or.... or.....) )

                          actually some of the rules/checks are quite extensive. I **wish**
                          they were easier.. but they are not and will probably be redefined to
                          something more tighter.. looking for VERY specific things.



                          I appreciate all of the help and guidance.


                          joe






                          > This way the regexps are parsed and compiled just once instead of
                          > each time they are used.
                          >
                          > > I'm sorry to have ruffled so many feathers, but just because one has
                          > > 20 years of experience in a language does not mean they should look
                          > > down on someone TRYING to learn it. Much less understand it andits
                          > > quirkiness.
                          > >
                          > > ...
                          >
                          > I think all this was unnecessary. Noone's picking on you.
                          >
                          > > As for "top loading or top responding" Not my fault, blame Yahoo for
                          > > doing that way.. hell blame the whole freaking internet for it,
                          > > because that's just how it is done. Talk to the hand.
                          >
                          > The fact that Yahoo (by default?) formats replies one way doesn't
                          > mean you have to keep it that way. Top-posting is quite OK for
                          > personal emails to a single recipient. For mailing lists its ... not
                          > working too well. Always try to make it easy to the people you are
                          > asking for help, even if it means you spend two more minutes
                          > formatting the email.
                          >
                          > Jenda
                          > ===== Jenda@... === http://Jenda.Krynicky.cz =====
                          > When it comes to wine, women and song, wizards are allowed
                          > to get drunk and croon as much as they like.
                          > -- Terry Pratchett in Sourcery
                          >
                        Your message has been successfully submitted and would be delivered to recipients shortly.