Loading ...
Sorry, an error occurred while loading the content.

Eval and memory problems

Expand Messages
  • computerdribble
    Greetings all... I used references found elsewhere to create a script to analyze log files for specific things. The specific things are stored in a mysql
    Message 1 of 15 , Jan 8, 2008
    • 0 Attachment
      Greetings all...

      I used references found elsewhere to create a script to analyze log
      files for specific things.

      The "specific things" are stored in a mysql database as a regex expression
      Example:

      ( ( $field =~ /blah/) and ($field =~ /that/) )

      I need the regex expressions in a database to tweaking and allowing
      others to maintain who do not know or who are as green as I am to Perl.

      I open the MySql database, no problem, load the contents into arrays,
      no problem.

      Open the log file (approx 1.48megs in size) for reading only.

      Ok, in the program ( I can;t post it as it stands)


      I have a logic loop

      grab a line from the log file

      run all regex expressions against the log line grabbed. This could be
      from 1 to 500 different tests.


      its done in a EVAL statement.

      similar to:

      if ( eval $array[$vartoarraycontents] )
      {
      ## if found.. do this

      ## do this: store information on line found and
      ## what found when compared to a rule/regex expression
      ## to be sent as an email to me

      }

      ## send email.



      that's about it.

      It runs perfectly for small log files (under 500k in size)
      BUT on the larger log files... EVAL blows out with a memory exhausted
      error. I am chaulking this up to the problem of memory leakage on the
      EVAL function.


      I am wondering IF there are any other ways to EVAL something prior to
      doing a comparision?

      I tried;

      if ( eval { $array[$pointer] } )


      and it throws back a hex code to me when I debug the EVAL contents.
      (stored to a variable for debugging)

      I am assuming that it throws me back a HASH of the test, but I can;t
      figure out how to use that information.

      I also tried to store the eval { $array[ $pointer] } to a variable
      and it ONLY shows the contents of the array based upon the pointer!
      <scratches head>



      So my main question, is is there an alternative to the EVAL


      I'm using perl 5.8.1


      thanks in advance!

      Joe.
    • computerdribble
      I think what I want is to be able to macro expand a variable that contains references to variables example: my $blah = ( ( $field =~ /blah/) and ($field =~
      Message 2 of 15 , Jan 8, 2008
      • 0 Attachment
        I think what I want is to be able to macro expand
        a variable that contains references to variables

        example:

        my $blah ="( ( $field =~ /blah/) and ($field =~ /that/) )"

        if ($blah)
        etc

        how do I macro expand ?

        thanks


        --- In perl-beginner@yahoogroups.com, "computerdribble"
        <computerdribble@...> wrote:
        >
        > Greetings all...
        >
        > I used references found elsewhere to create a script to analyze log
        > files for specific things.
        >
        > The "specific things" are stored in a mysql database as a regex
        expression
        > Example:
        >
        > ( ( $field =~ /blah/) and ($field =~ /that/) )
        >
        > I need the regex expressions in a database to tweaking and allowing
        > others to maintain who do not know or who are as green as I am to Perl.
        >
        > I open the MySql database, no problem, load the contents into arrays,
        > no problem.
        >
        > Open the log file (approx 1.48megs in size) for reading only.
        >
        > Ok, in the program ( I can;t post it as it stands)
        >
        >
        > I have a logic loop
        >
        > grab a line from the log file
        >
        > run all regex expressions against the log line grabbed. This could be
        > from 1 to 500 different tests.
        >
        >
        > its done in a EVAL statement.
        >
        > similar to:
        >
        > if ( eval $array[$vartoarraycontents] )
        > {
        > ## if found.. do this
        >
        > ## do this: store information on line found and
        > ## what found when compared to a rule/regex expression
        > ## to be sent as an email to me
        >
        > }
        >
        > ## send email.
        >
        >
        >
        > that's about it.
        >
        > It runs perfectly for small log files (under 500k in size)
        > BUT on the larger log files... EVAL blows out with a memory exhausted
        > error. I am chaulking this up to the problem of memory leakage on the
        > EVAL function.
        >
        >
        > I am wondering IF there are any other ways to EVAL something prior to
        > doing a comparision?
        >
        > I tried;
        >
        > if ( eval { $array[$pointer] } )
        >
        >
        > and it throws back a hex code to me when I debug the EVAL contents.
        > (stored to a variable for debugging)
        >
        > I am assuming that it throws me back a HASH of the test, but I can;t
        > figure out how to use that information.
        >
        > I also tried to store the eval { $array[ $pointer] } to a variable
        > and it ONLY shows the contents of the array based upon the pointer!
        > <scratches head>
        >
        >
        >
        > So my main question, is is there an alternative to the EVAL
        >
        >
        > I'm using perl 5.8.1
        >
        >
        > thanks in advance!
        >
        > Joe.
        >
      • merlyn@stonehenge.com
        ... computerdribble how do I macro expand ? Is the answer in the FAQ not clear enough? -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
        Message 3 of 15 , Jan 8, 2008
        • 0 Attachment
          >>>>> "computerdribble" == computerdribble <computerdribble@...> writes:

          computerdribble> how do I macro expand ?

          Is the answer in the FAQ not clear enough?

          --
          Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
          See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
        • computerdribble
          which FAQ ? thanks by the way. ... 777 0095 ... training!
          Message 4 of 15 , Jan 8, 2008
          • 0 Attachment
            which FAQ ?

            thanks by the way.



            --- In perl-beginner@yahoogroups.com, merlyn@... wrote:
            >
            > >>>>> "computerdribble" == computerdribble <computerdribble@...>
            writes:
            >
            > computerdribble> how do I macro expand ?
            >
            > Is the answer in the FAQ not clear enough?
            >
            > --
            > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
            777 0095
            > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
            > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
            > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
            training!
            >
          • Boyle, Christopher
            http://faq.perl.org/ ________________________________ From: perl-beginner@yahoogroups.com [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
            Message 5 of 15 , Jan 8, 2008
            • 0 Attachment
              http://faq.perl.org/





              ________________________________

              From: perl-beginner@yahoogroups.com
              [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
              Sent: Tuesday, January 08, 2008 2:05 PM
              To: perl-beginner@yahoogroups.com
              Subject: [PBML] Re: Eval and memory problems





              which FAQ ?

              thanks by the way.

              --- In perl-beginner@yahoogroups.com
              <mailto:perl-beginner%40yahoogroups.com> , merlyn@... wrote:
              >
              > >>>>> "computerdribble" == computerdribble <computerdribble@...>
              writes:
              >
              > computerdribble> how do I macro expand ?
              >
              > Is the answer in the FAQ not clear enough?
              >
              > --
              > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
              777 0095
              > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/
              <http://www.stonehenge.com/merlyn/> >
              > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
              > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
              training!
              >


              ______________________________________________________________________
              This email has been scanned by the MessageLabs Email Security System.
              For more information please visit http://www.messagelabs.com/email
              ______________________________________________________________________



              NOTICE OF CONFIDENTIALITY: Information included in and/or attached to this electronic mail transmission may be confidential. This electronic mail transmission is intended for the addressee(s) only. Any unauthorized disclosure, reproduction, or distribution of, and/or any unauthorized action taken in reliance on the information in this electronic mail is prohibited. If you believe that you have received this electronic mail transmission in error, please notify the sender by reply transmission, or contact helpdesk@..., and delete the message without copying or disclosing it.

              ______________________________________________________________________
              This email has been scanned by the MessageLabs Email Security System.
              For more information please visit http://www.messagelabs.com/email
              ______________________________________________________________________

              [Non-text portions of this message have been removed]
            • computerdribble
              Nope , not in there. there is nothing in the FAQ concerning Macro expansion. Only Macros for Vi (ick) Went through that AGAIN and no luck. anywhere else? I
              Message 6 of 15 , Jan 8, 2008
              • 0 Attachment
                Nope , not in there. there is nothing in the FAQ concerning Macro
                expansion.

                Only Macros for Vi (ick)

                Went through that AGAIN and no luck.

                anywhere else?

                I really appreciate the pointers. I guess thats what the
                "perl-beginner" is all about.

                tia!




                --- In perl-beginner@yahoogroups.com, "Boyle, Christopher"
                <Christopher.Boyle@...> wrote:
                >
                > http://faq.perl.org/
                >
                >
                >
                >
                >
                > ________________________________
                >
                > From: perl-beginner@yahoogroups.com
                > [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
                > Sent: Tuesday, January 08, 2008 2:05 PM
                > To: perl-beginner@yahoogroups.com
                > Subject: [PBML] Re: Eval and memory problems
                >
                >
                >
                >
                >
                > which FAQ ?
                >
                > thanks by the way.
                >
                > --- In perl-beginner@yahoogroups.com
                > <mailto:perl-beginner%40yahoogroups.com> , merlyn@ wrote:
                > >
                > > >>>>> "computerdribble" == computerdribble <computerdribble@>
                > writes:
                > >
                > > computerdribble> how do I macro expand ?
                > >
                > > Is the answer in the FAQ not clear enough?
                > >
                > > --
                > > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
                > 777 0095
                > > <merlyn@> <URL:http://www.stonehenge.com/merlyn/
                > <http://www.stonehenge.com/merlyn/> >
                > > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                > > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
                > training!
                > >
                >
                >
                > ______________________________________________________________________
                > This email has been scanned by the MessageLabs Email Security System.
                > For more information please visit http://www.messagelabs.com/email
                > ______________________________________________________________________
                >
                >
                >
                > NOTICE OF CONFIDENTIALITY: Information included in and/or attached
                to this electronic mail transmission may be confidential. This
                electronic mail transmission is intended for the addressee(s) only.
                Any unauthorized disclosure, reproduction, or distribution of, and/or
                any unauthorized action taken in reliance on the information in this
                electronic mail is prohibited. If you believe that you have received
                this electronic mail transmission in error, please notify the sender
                by reply transmission, or contact helpdesk@..., and delete the message
                without copying or disclosing it.
                >
                > ______________________________________________________________________
                > This email has been scanned by the MessageLabs Email Security System.
                > For more information please visit http://www.messagelabs.com/email
                > ______________________________________________________________________
                >
                > [Non-text portions of this message have been removed]
                >
              • merlyn@stonehenge.com
                ... computerdribble Nope , not in there. there is nothing in the FAQ concerning Macro computerdribble expansion. By the way, it s on *YOUR DISK*. No need
                Message 7 of 15 , Jan 8, 2008
                • 0 Attachment
                  >>>>> "computerdribble" == computerdribble <computerdribble@...> writes:

                  computerdribble> Nope , not in there. there is nothing in the FAQ concerning Macro
                  computerdribble> expansion.

                  By the way, it's on *YOUR DISK*. No need to go off to the net for the FAQ.
                  Type "perldoc perlfaq". And you can search it too, which I will now do for
                  you, to show you the FAQ *does* cover what you're looking for.

                  I suggest you read the subject lines of "perldoc perlfaq" and "perldoc perl"
                  weekly until you can recite them from memory. Perl comes with a HUGE amount
                  of info, but it's pointless if you don't know what's there.

                  $ perldoc -q expand

                  Found in /usr/libdata/perl5/pod/perlfaq4.pod
                  How do I expand function calls in a string?
                  (contributed by brian d foy)

                  This is documented in perlref, and although it's not the easiest thing
                  to read, it does work. In each of these examples, we call the function
                  inside the braces used to dereference a reference. If we have a more
                  than one return value, we can construct and dereference an anonymous
                  array. In this case, we call the function in list context.

                  print "The time values are @{ [localtime] }.\n";

                  If we want to call the function in scalar context, we have to do a bit
                  more work. We can really have any code we like inside the braces, so we
                  simply have to end with the scalar reference, although how you do that
                  is up to you, and you can use code inside the braces.

                  print "The time is ${\(scalar localtime)}.\n"

                  print "The time is ${ my $x = localtime; \$x }.\n";

                  If your function already returns a reference, you don't need to create
                  the reference yourself.

                  sub timestamp { my $t = localtime; \$t }

                  print "The time is ${ timestamp() }.\n";

                  The "Interpolation" module can also do a lot of magic for you. You can
                  specify a variable name, in this case "E", to set up a tied hash that
                  does the interpolation for you. It has several other methods to do this
                  as well.

                  use Interpolation E => 'eval';
                  print "The time values are $E{localtime()}.\n";

                  In most cases, it is probably easier to simply use string concatenation,
                  which also forces scalar context.

                  print "The time is " . localtime . ".\n";

                  How do I expand tabs in a string?
                  You can do it yourself:

                  1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;

                  Or you can just use the Text::Tabs module (part of the standard Perl
                  distribution).

                  use Text::Tabs;
                  @expanded_lines = expand(@lines_with_tabs);

                  How can I expand variables in text strings?
                  Let's assume that you have a string that contains placeholder variables.

                  $text = 'this has a $foo in it and a $bar';

                  You can use a substitution with a double evaluation. The first /e turns
                  $1 into $foo, and the second /e turns $foo into its value. You may want
                  to wrap this in an "eval": if you try to get the value of an undeclared
                  variable while running under "use strict", you get a fatal error.

                  eval { $text =~ s/(\$\w+)/$1/eeg };
                  die if $@;

                  It's probably better in the general case to treat those variables as
                  entries in some special hash. For example:

                  %user_defs = (
                  foo => 23,
                  bar => 19,
                  );
                  $text =~ s/\$(\w+)/$user_defs{$1}/g;




                  --
                  Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
                  <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                  Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                  See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
                • computerdribble
                  Thank you. I went onto the faq.perl.org site and searched through all of that and nothing came back. This is one of the problems and reasons I have stayed
                  Message 8 of 15 , Jan 8, 2008
                  • 0 Attachment
                    Thank you.

                    I went onto the faq.perl.org site and searched through all of that
                    and nothing came back.

                    This is one of the problems and reasons I have stayed away from perl
                    for so long. There's SO much to it and it HAS its OWN quirkiness that
                    outpaces other languages.


                    but again, thank you!


                    Now, one other question....


                    this works:
                    $text =~ s/(\$\w+)/$1/eeg

                    it takes;
                    $aa="hello";
                    $bb="goodbye";


                    and inserts it into;


                    $text = "( ( $aa =~ /lo/ ) and ($bb =~ /ye/ ) )";


                    perfectly to create:


                    ( ( hello =~ /lo/ ) and ( goodbye =~ /ye/ ) )



                    notice the problem ? No quotes around the $aa expanded variable. So the
                    if ( @{ [ $text ] } )

                    always fails.



                    I'm NOT a regex person and stumble my way through it so far.. but this
                    is beyond me at the moment. (yes more reading is required)

                    How do I get the expansion (or can it be done? ) to automatically put
                    the quotes around the variables expanded?

                    I can add the quotes into the database around the variables, but then
                    its a cludge.

                    thanks again and thanks in advance...


                    joe















                    --- In perl-beginner@yahoogroups.com, merlyn@... wrote:
                    >
                    > >>>>> "computerdribble" == computerdribble <computerdribble@...>
                    writes:
                    >
                    > computerdribble> Nope , not in there. there is nothing in the FAQ
                    concerning Macro
                    > computerdribble> expansion.
                    >
                    > By the way, it's on *YOUR DISK*. No need to go off to the net for
                    the FAQ.
                    > Type "perldoc perlfaq". And you can search it too, which I will now
                    do for
                    > you, to show you the FAQ *does* cover what you're looking for.
                    >
                    > I suggest you read the subject lines of "perldoc perlfaq" and
                    "perldoc perl"
                    > weekly until you can recite them from memory. Perl comes with a
                    HUGE amount
                    > of info, but it's pointless if you don't know what's there.
                    >
                    > $ perldoc -q expand
                    >
                    > Found in /usr/libdata/perl5/pod/perlfaq4.pod
                    > How do I expand function calls in a string?
                    > (contributed by brian d foy)
                    >
                    > This is documented in perlref, and although it's not the easiest
                    thing
                    > to read, it does work. In each of these examples, we call the
                    function
                    > inside the braces used to dereference a reference. If we have a more
                    > than one return value, we can construct and dereference an anonymous
                    > array. In this case, we call the function in list context.
                    >
                    > print "The time values are @{ [localtime] }.\n";
                    >
                    > If we want to call the function in scalar context, we have to do
                    a bit
                    > more work. We can really have any code we like inside the
                    braces, so we
                    > simply have to end with the scalar reference, although how you
                    do that
                    > is up to you, and you can use code inside the braces.
                    >
                    > print "The time is ${\(scalar localtime)}.\n"
                    >
                    > print "The time is ${ my $x = localtime; \$x }.\n";
                    >
                    > If your function already returns a reference, you don't need to
                    create
                    > the reference yourself.
                    >
                    > sub timestamp { my $t = localtime; \$t }
                    >
                    > print "The time is ${ timestamp() }.\n";
                    >
                    > The "Interpolation" module can also do a lot of magic for you.
                    You can
                    > specify a variable name, in this case "E", to set up a tied hash
                    that
                    > does the interpolation for you. It has several other methods to
                    do this
                    > as well.
                    >
                    > use Interpolation E => 'eval';
                    > print "The time values are $E{localtime()}.\n";
                    >
                    > In most cases, it is probably easier to simply use string
                    concatenation,
                    > which also forces scalar context.
                    >
                    > print "The time is " . localtime . ".\n";
                    >
                    > How do I expand tabs in a string?
                    > You can do it yourself:
                    >
                    > 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`)
                    % 8)/e;
                    >
                    > Or you can just use the Text::Tabs module (part of the standard Perl
                    > distribution).
                    >
                    > use Text::Tabs;
                    > @expanded_lines = expand(@lines_with_tabs);
                    >
                    > How can I expand variables in text strings?
                    > Let's assume that you have a string that contains placeholder
                    variables.
                    >
                    > $text = 'this has a $foo in it and a $bar';
                    >
                    > You can use a substitution with a double evaluation. The first
                    /e turns
                    > $1 into $foo, and the second /e turns $foo into its value. You
                    may want
                    > to wrap this in an "eval": if you try to get the value of an
                    undeclared
                    > variable while running under "use strict", you get a fatal error.
                    >
                    > eval { $text =~ s/(\$\w+)/$1/eeg };
                    > die if $@;
                    >
                    > It's probably better in the general case to treat those variables as
                    > entries in some special hash. For example:
                    >
                    > %user_defs = (
                    > foo => 23,
                    > bar => 19,
                    > );
                    > $text =~ s/\$(\w+)/$user_defs{$1}/g;
                    >
                    >
                    >
                    >
                    > --
                    > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
                    777 0095
                    > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                    > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                    > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
                    training!
                    >
                  • Boyle, Christopher
                    perldoc -quote ________________________________ From: perl-beginner@yahoogroups.com [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble Sent:
                    Message 9 of 15 , Jan 8, 2008
                    • 0 Attachment
                      perldoc -quote



                      ________________________________

                      From: perl-beginner@yahoogroups.com
                      [mailto:perl-beginner@yahoogroups.com] On Behalf Of computerdribble
                      Sent: Tuesday, January 08, 2008 3:38 PM
                      To: perl-beginner@yahoogroups.com
                      Subject: [PBML] Re: Eval and memory problems




                      Thank you.

                      I went onto the faq.perl.org site and searched through all of that
                      and nothing came back.

                      This is one of the problems and reasons I have stayed away from perl
                      for so long. There's SO much to it and it HAS its OWN quirkiness that
                      outpaces other languages.

                      but again, thank you!

                      Now, one other question....

                      this works:
                      $text =~ s/(\$\w+)/$1/eeg

                      it takes;
                      $aa="hello";
                      $bb="goodbye";

                      and inserts it into;

                      $text = "( ( $aa =~ /lo/ ) and ($bb =~ /ye/ ) )";

                      perfectly to create:

                      ( ( hello =~ /lo/ ) and ( goodbye =~ /ye/ ) )

                      notice the problem ? No quotes around the $aa expanded variable. So the
                      if ( @{ [ $text ] } )

                      always fails.

                      I'm NOT a regex person and stumble my way through it so far.. but this
                      is beyond me at the moment. (yes more reading is required)

                      How do I get the expansion (or can it be done? ) to automatically put
                      the quotes around the variables expanded?

                      I can add the quotes into the database around the variables, but then
                      its a cludge.

                      thanks again and thanks in advance...

                      joe

                      --- In perl-beginner@yahoogroups.com
                      <mailto:perl-beginner%40yahoogroups.com> , merlyn@... wrote:
                      >
                      > >>>>> "computerdribble" == computerdribble <computerdribble@...>
                      writes:
                      >
                      > computerdribble> Nope , not in there. there is nothing in the FAQ
                      concerning Macro
                      > computerdribble> expansion.
                      >
                      > By the way, it's on *YOUR DISK*. No need to go off to the net for
                      the FAQ.
                      > Type "perldoc perlfaq". And you can search it too, which I will now
                      do for
                      > you, to show you the FAQ *does* cover what you're looking for.
                      >
                      > I suggest you read the subject lines of "perldoc perlfaq" and
                      "perldoc perl"
                      > weekly until you can recite them from memory. Perl comes with a
                      HUGE amount
                      > of info, but it's pointless if you don't know what's there.
                      >
                      > $ perldoc -q expand
                      >
                      > Found in /usr/libdata/perl5/pod/perlfaq4.pod
                      > How do I expand function calls in a string?
                      > (contributed by brian d foy)
                      >
                      > This is documented in perlref, and although it's not the easiest
                      thing
                      > to read, it does work. In each of these examples, we call the
                      function
                      > inside the braces used to dereference a reference. If we have a more
                      > than one return value, we can construct and dereference an anonymous
                      > array. In this case, we call the function in list context.
                      >
                      > print "The time values are @{ [localtime] }.\n";
                      >
                      > If we want to call the function in scalar context, we have to do
                      a bit
                      > more work. We can really have any code we like inside the
                      braces, so we
                      > simply have to end with the scalar reference, although how you
                      do that
                      > is up to you, and you can use code inside the braces.
                      >
                      > print "The time is ${\(scalar localtime)}.\n"
                      >
                      > print "The time is ${ my $x = localtime; \$x }.\n";
                      >
                      > If your function already returns a reference, you don't need to
                      create
                      > the reference yourself.
                      >
                      > sub timestamp { my $t = localtime; \$t }
                      >
                      > print "The time is ${ timestamp() }.\n";
                      >
                      > The "Interpolation" module can also do a lot of magic for you.
                      You can
                      > specify a variable name, in this case "E", to set up a tied hash
                      that
                      > does the interpolation for you. It has several other methods to
                      do this
                      > as well.
                      >
                      > use Interpolation E => 'eval';
                      > print "The time values are $E{localtime()}.\n";
                      >
                      > In most cases, it is probably easier to simply use string
                      concatenation,
                      > which also forces scalar context.
                      >
                      > print "The time is " . localtime . ".\n";
                      >
                      > How do I expand tabs in a string?
                      > You can do it yourself:
                      >
                      > 1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`)
                      % 8)/e;
                      >
                      > Or you can just use the Text::Tabs module (part of the standard Perl
                      > distribution).
                      >
                      > use Text::Tabs;
                      > @expanded_lines = expand(@lines_with_tabs);
                      >
                      > How can I expand variables in text strings?
                      > Let's assume that you have a string that contains placeholder
                      variables.
                      >
                      > $text = 'this has a $foo in it and a $bar';
                      >
                      > You can use a substitution with a double evaluation. The first
                      /e turns
                      > $1 into $foo, and the second /e turns $foo into its value. You
                      may want
                      > to wrap this in an "eval": if you try to get the value of an
                      undeclared
                      > variable while running under "use strict", you get a fatal error.
                      >
                      > eval { $text =~ s/(\$\w+)/$1/eeg };
                      > die if $@;
                      >
                      > It's probably better in the general case to treat those variables as
                      > entries in some special hash. For example:
                      >
                      > %user_defs = (
                      > foo => 23,
                      > bar => 19,
                      > );
                      > $text =~ s/\$(\w+)/$user_defs{$1}/g;
                      >
                      >
                      >
                      >
                      > --
                      > Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503
                      777 0095
                      > <merlyn@...> <URL:http://www.stonehenge.com/merlyn/
                      <http://www.stonehenge.com/merlyn/> >
                      > Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
                      > See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl
                      training!
                      >


                      ______________________________________________________________________
                      This email has been scanned by the MessageLabs Email Security System.
                      For more information please visit http://www.messagelabs.com/email
                      ______________________________________________________________________



                      NOTICE OF CONFIDENTIALITY: Information included in and/or attached to this electronic mail transmission may be confidential. This electronic mail transmission is intended for the addressee(s) only. Any unauthorized disclosure, reproduction, or distribution of, and/or any unauthorized action taken in reliance on the information in this electronic mail is prohibited. If you believe that you have received this electronic mail transmission in error, please notify the sender by reply transmission, or contact helpdesk@..., and delete the message without copying or disclosing it.

                      ______________________________________________________________________
                      This email has been scanned by the MessageLabs Email Security System.
                      For more information please visit http://www.messagelabs.com/email
                      ______________________________________________________________________

                      [Non-text portions of this message have been removed]
                    • Jenda Krynicky
                      From: computerdribble ... Compile each test just once and run it repeatedly. Before opening the log file, but after reading the
                      Message 10 of 15 , Jan 8, 2008
                      • 0 Attachment
                        From: "computerdribble" <computerdribble@...>
                        > The "specific things" are stored in a mysql database as a regex expression
                        > Example:
                        >
                        > ( ( $field =~ /blah/) and ($field =~ /that/) )
                        >
                        > I open the MySql database, no problem, load the contents into arrays,
                        > no problem.
                        >
                        > Open the log file (approx 1.48megs in size) for reading only.
                        >
                        > I have a logic loop
                        >
                        > grab a line from the log file
                        >
                        > run all regex expressions against the log line grabbed. This could be
                        > from 1 to 500 different tests.
                        >
                        >
                        > its done in a EVAL statement.
                        >
                        > similar to:
                        >
                        > if ( eval $array[$vartoarraycontents] )

                        Compile each test just once and run it repeatedly.
                        Before opening the log file, but after reading the expressions do
                        something like

                        for my $test (@array) {
                        $test = eval 'sub { ' . $test . '}'
                        or die "Failed to compile '$test' : $@\n";
                        }

                        and then while looping through the logfile use

                        if ($array[$vartoarraycontents]->()) {

                        There is a possible catch though! The variables you plan to make
                        available to the tests must be declared before the tests are compiled
                        by those eval('sub {'...) statements!

                        Jenda
                        ===== Jenda@... === http://Jenda.Krynicky.cz =====
                        When it comes to wine, women and song, wizards are allowed
                        to get drunk and croon as much as they like.
                        -- Terry Pratchett in Sourcery
                      • Computer-Dribble
                        Thank you! One problem I ran into was the EVAL was killing 4 gigs of memeory on large log files. At least I traced it back to that. This is why I wanted to
                        Message 11 of 15 , Jan 9, 2008
                        • 0 Attachment
                          Thank you!


                          One problem I ran into was the EVAL was killing 4 gigs
                          of memeory on large log files. At least I traced it
                          back to that.

                          This is why I wanted to avoid EVAL completely and do
                          some macro expansion.

                          thanks again!!

                          Joe.

                          --- Jenda Krynicky <Jenda@...> wrote:

                          > From: "computerdribble" <computerdribble@...>
                          > > The "specific things" are stored in a mysql
                          > database as a regex expression
                          > > Example:
                          > >
                          > > ( ( $field =~ /blah/) and ($field =~ /that/) )
                          > >
                          > > I open the MySql database, no problem, load the
                          > contents into arrays,
                          > > no problem.
                          > >
                          > > Open the log file (approx 1.48megs in size) for
                          > reading only.
                          > >
                          > > I have a logic loop
                          > >
                          > > grab a line from the log file
                          > >
                          > > run all regex expressions against the log line
                          > grabbed. This could be
                          > > from 1 to 500 different tests.
                          > >
                          > >
                          > > its done in a EVAL statement.
                          > >
                          > > similar to:
                          > >
                          > > if ( eval $array[$vartoarraycontents] )
                          >
                          > Compile each test just once and run it repeatedly.
                          > Before opening the log file, but after reading the
                          > expressions do
                          > something like
                          >
                          > for my $test (@array) {
                          > $test = eval 'sub { ' . $test . '}'
                          > or die "Failed to compile '$test' : $@\n";
                          > }
                          >
                          > and then while looping through the logfile use
                          >
                          > if ($array[$vartoarraycontents]->()) {
                          >
                          > There is a possible catch though! The variables you
                          > plan to make
                          > available to the tests must be declared before the
                          > tests are compiled
                          > by those eval('sub {'...) statements!
                          >
                          > Jenda
                          > ===== Jenda@... === http://Jenda.Krynicky.cz
                          > =====
                          > When it comes to wine, women and song, wizards are
                          > allowed
                          > to get drunk and croon as much as they like.
                          > -- Terry Pratchett in Sourcery
                          >
                          >
                          >
                          > Unsubscribing info is here:
                          > http://help.yahoo.com/help/us/groups/groups-32.html
                          > Yahoo! Groups Links
                          >
                          >
                          > mailto:perl-beginner-fullfeatured@yahoogroups.com
                          >
                          >
                          >



                          ____________________________________________________________________________________
                          Never miss a thing. Make Yahoo your home page.
                          http://www.yahoo.com/r/hs
                        • Jenda Krynicky
                          - No I don t. And others don t either. - But you do not need to read what I wrote originaly and what you replied last time, you remember it, don t you? -
                          Message 12 of 15 , Jan 9, 2008
                          • 0 Attachment
                            - No I don't. And others don't either.
                            - But you do not need to read what I wrote originaly and what you
                            replied last time, you remember it, don't you?
                            - Because it's all backwards.
                            - Why is that?
                            - Because it's hard to read.
                            - Why?
                            - Please do not top-post!

                            Since the proposed change decreases the number of eval""s from
                            (number of tests)*(number of lines in logfile)
                            to just
                            (number of tests)
                            any leaks in eval"" should be irrelevant.

                            Jenda

                            From: Computer-Dribble <computerdribble@...>
                            > One problem I ran into was the EVAL was killing 4 gigs
                            > of memeory on large log files. At least I traced it
                            > back to that.
                            >
                            > This is why I wanted to avoid EVAL completely and do
                            > some macro expansion.
                            >
                            > thanks again!!
                            >
                            > Joe.
                            >
                            > --- Jenda Krynicky <Jenda@...> wrote:
                            >
                            > > From: "computerdribble" <computerdribble@...>
                            > > > The "specific things" are stored in a mysql
                            > > database as a regex expression
                            > > > Example:
                            > > >
                            > > > ( ( $field =~ /blah/) and ($field =~ /that/) )
                            > > >
                            > > > I open the MySql database, no problem, load the
                            > > contents into arrays,
                            > > > no problem.
                            > > >
                            > > > Open the log file (approx 1.48megs in size) for
                            > > reading only.
                            > > >
                            > > > I have a logic loop
                            > > >
                            > > > grab a line from the log file
                            > > >
                            > > > run all regex expressions against the log line
                            > > grabbed. This could be
                            > > > from 1 to 500 different tests.
                            > > >
                            > > >
                            > > > its done in a EVAL statement.
                            > > >
                            > > > similar to:
                            > > >
                            > > > if ( eval $array[$vartoarraycontents] )
                            > >
                            > > Compile each test just once and run it repeatedly.
                            > > Before opening the log file, but after reading the
                            > > expressions do
                            > > something like
                            > >
                            > > for my $test (@array) {
                            > > $test = eval 'sub { ' . $test . '}'
                            > > or die "Failed to compile '$test' : $@\n";
                            > > }
                            > >
                            > > and then while looping through the logfile use
                            > >
                            > > if ($array[$vartoarraycontents]->()) {
                            > >
                            > > There is a possible catch though! The variables you
                            > > plan to make
                            > > available to the tests must be declared before the
                            > > tests are compiled
                            > > by those eval('sub {'...) statements!
                            > >
                            > > Jenda
                            > > ===== Jenda@... === http://Jenda.Krynicky.cz
                            > > =====
                            > > When it comes to wine, women and song, wizards are
                            > > allowed
                            > > to get drunk and croon as much as they like.
                            > > -- Terry Pratchett in Sourcery
                            > >
                            > >
                            > >
                            > > Unsubscribing info is here:
                            > > http://help.yahoo.com/help/us/groups/groups-32.html
                            > > Yahoo! Groups Links
                            > >
                            > >
                            > > mailto:perl-beginner-fullfeatured@yahoogroups.com
                            > >
                            > >
                            > >
                            >
                            >
                            >
                            > ____________________________________________________________________________________
                            > Never miss a thing. Make Yahoo your home page.
                            > http://www.yahoo.com/r/hs
                            >
                            >
                            > Unsubscribing info is here: http://help.yahoo.com/help/us/groups/groups-32.html
                            > Yahoo! Groups Links
                            >
                            >
                            >
                            >


                            ===== Jenda@... === http://Jenda.Krynicky.cz =====
                            When it comes to wine, women and song, wizards are allowed
                            to get drunk and croon as much as they like.
                            -- Terry Pratchett in Sourcery
                          • computerdribble
                            I humbly thank ALL who have read or responded. Can you, with all of the examples to date... do away with the EVAL of the variable that contains the testing
                            Message 13 of 15 , Jan 9, 2008
                            • 0 Attachment
                              I humbly thank ALL who have read or responded.

                              Can you, with all of the examples to date...
                              do away with the EVAL of the variable that contains the testing
                              information and just do a straight macro expansion ?

                              $this = "that"
                              $that ="this"

                              $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '

                              something like @{[ $$test ] }


                              I'm sorry to have ruffled so many feathers, but just because one has
                              20 years of experience in a language does not mean they should look
                              down on someone TRYING to learn it. Much less understand it andits
                              quirkiness.

                              Usually, a pre-madonna is a target in the computer field, remember that.

                              I find that perl is never as straight forward as one thinks. It takes
                              a great deal of trial and error to get it to do what you want it to do
                              .. for a beginner, its frustrating. For someone with 30 years
                              experince in other languages, its VERY frustrating. Unless I
                              completely dive into the language full time.. I'll never gain the
                              knowledge you guys have. But I don't need to. This is a 1 time
                              thing. I can go someone else to find the answers BUT... I thought
                              that this was a "perl-beginner" group/forum not a "lets pick on
                              someone trying to learn the language" area.
                              Is this what "knowledge sharing" has degraded to ?

                              Most of the "trial and error" is because of the REGEX. Each language,
                              has its own way of looking at REGEX and Perl is notorious for its own
                              quirks.

                              I bow before greater GODs than I in the Perl field and I humbly do so.
                              If I had my way, I would have written this in another language
                              altogether, but a client wants what a client wants.

                              As for "top loading or top responding" Not my fault, blame Yahoo for
                              doing that way.. hell blame the whole freaking internet for it,
                              because that's just how it is done. Talk to the hand.






                              Joe.
                            • Jenda Krynicky
                              From: computerdribble ... No. You can t. It s as simple as that. If you did not store the expression like this in the database,
                              Message 14 of 15 , Jan 9, 2008
                              • 0 Attachment
                                From: "computerdribble" <computerdribble@...>
                                > I humbly thank ALL who have read or responded.
                                >
                                > Can you, with all of the examples to date...
                                > do away with the EVAL of the variable that contains the testing
                                > information and just do a straight macro expansion ?
                                >
                                > $this = "that"
                                > $that ="this"
                                >
                                > $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '
                                >
                                > something like @{[ $$test ] }

                                No. You can't. It's as simple as that.

                                If you did not store the expression like this in the database, but
                                instead stored the name of the variable and the regexp (and nothing
                                else) in separate columns you could do something like

                                if ($data{$varname} =~ /$regexp/) {

                                where the $varname and $regexp comes from the database. Whether that
                                would be enough for you, I don't know.

                                Even in this case though it would be better to preprocess the
                                regexps:

                                foreach my $test (@tests) {
                                # $test = [$varname, $regexp]
                                eval {
                                $test[1] = qr/$test[1]/
                                } or die qq{Invalid regexp /$test[1]/ : $@\n};
                                }

                                This way the regexps are parsed and compiled just once instead of
                                each time they are used.

                                > I'm sorry to have ruffled so many feathers, but just because one has
                                > 20 years of experience in a language does not mean they should look
                                > down on someone TRYING to learn it. Much less understand it andits
                                > quirkiness.
                                >
                                > ...

                                I think all this was unnecessary. Noone's picking on you.

                                > As for "top loading or top responding" Not my fault, blame Yahoo for
                                > doing that way.. hell blame the whole freaking internet for it,
                                > because that's just how it is done. Talk to the hand.

                                The fact that Yahoo (by default?) formats replies one way doesn't
                                mean you have to keep it that way. Top-posting is quite OK for
                                personal emails to a single recipient. For mailing lists its ... not
                                working too well. Always try to make it easy to the people you are
                                asking for help, even if it means you spend two more minutes
                                formatting the email.

                                Jenda
                                ===== Jenda@... === http://Jenda.Krynicky.cz =====
                                When it comes to wine, women and song, wizards are allowed
                                to get drunk and croon as much as they like.
                                -- Terry Pratchett in Sourcery
                              • computerdribble
                                Thank you for responding. ... dang it all! oh well. ... I would love to.. except there are certain instances that what needs to be checked is.. well let me
                                Message 15 of 15 , Jan 9, 2008
                                • 0 Attachment
                                  Thank you for responding.


                                  --- In perl-beginner@yahoogroups.com, "Jenda Krynicky" <Jenda@...> wrote:
                                  >
                                  > From: "computerdribble" <computerdribble@...>
                                  > > I humbly thank ALL who have read or responded.
                                  > >
                                  > > Can you, with all of the examples to date...
                                  > > do away with the EVAL of the variable that contains the testing
                                  > > information and just do a straight macro expansion ?
                                  > >
                                  > > $this = "that"
                                  > > $that ="this"
                                  > >
                                  > > $test = ' ( ( "that" =~ /at/) and ( "this" =~ /at/ ) ) '
                                  > >
                                  > > something like @{[ $$test ] }
                                  >
                                  > No. You can't. It's as simple as that.
                                  >


                                  dang it all!
                                  oh well.


                                  > If you did not store the expression like this in the database, but
                                  > instead stored the name of the variable and the regexp (and nothing
                                  > else) in separate columns you could do something like
                                  >
                                  > if ($data{$varname} =~ /$regexp/) {
                                  >
                                  > where the $varname and $regexp comes from the database. Whether that
                                  > would be enough for you, I don't know.
                                  >
                                  > Even in this case though it would be better to preprocess the
                                  > regexps:
                                  >
                                  > foreach my $test (@tests) {
                                  > # $test = [$varname, $regexp]
                                  > eval {
                                  > $test[1] = qr/$test[1]/
                                  > } or die qq{Invalid regexp /$test[1]/ : $@\n};
                                  > }
                                  >


                                  I would love to.. except there are certain instances that what needs
                                  to be checked is.. well let me put it this way:


                                  (( $field =~ /this/) and ( ($field =~ /that/) or ($field1 =~/a/) or
                                  $field2 =~/xx/) or... or.... or.....) )

                                  actually some of the rules/checks are quite extensive. I **wish**
                                  they were easier.. but they are not and will probably be redefined to
                                  something more tighter.. looking for VERY specific things.



                                  I appreciate all of the help and guidance.


                                  joe






                                  > This way the regexps are parsed and compiled just once instead of
                                  > each time they are used.
                                  >
                                  > > I'm sorry to have ruffled so many feathers, but just because one has
                                  > > 20 years of experience in a language does not mean they should look
                                  > > down on someone TRYING to learn it. Much less understand it andits
                                  > > quirkiness.
                                  > >
                                  > > ...
                                  >
                                  > I think all this was unnecessary. Noone's picking on you.
                                  >
                                  > > As for "top loading or top responding" Not my fault, blame Yahoo for
                                  > > doing that way.. hell blame the whole freaking internet for it,
                                  > > because that's just how it is done. Talk to the hand.
                                  >
                                  > The fact that Yahoo (by default?) formats replies one way doesn't
                                  > mean you have to keep it that way. Top-posting is quite OK for
                                  > personal emails to a single recipient. For mailing lists its ... not
                                  > working too well. Always try to make it easy to the people you are
                                  > asking for help, even if it means you spend two more minutes
                                  > formatting the email.
                                  >
                                  > Jenda
                                  > ===== Jenda@... === http://Jenda.Krynicky.cz =====
                                  > When it comes to wine, women and song, wizards are allowed
                                  > to get drunk and croon as much as they like.
                                  > -- Terry Pratchett in Sourcery
                                  >
                                Your message has been successfully submitted and would be delivered to recipients shortly.