Loading ...
Sorry, an error occurred while loading the content.

Re: [PBML] Regarding flat file access vs. database

Expand Messages
  • Paul Archer
    I just got into a similar project (parsing NFS log files), and had the same decision to make. In my case, I know my data set is going to get large enough to
    Message 1 of 4 , Apr 5, 2006
    • 0 Attachment
      I just got into a similar project (parsing NFS log files), and had the same
      decision to make. In my case, I know my data set is going to get large
      enough to warrant using a real database. In your case, it sounds like it'd
      be easier to simply stay with the format you've already got.
      Just write (or find) a parser for your tab delimited file. Read it in at the
      start of your program and store the data in whatever format you use
      internally (hash of arrays, for example). Then gather your data, merge the
      sets, and write it all back out.
      To make things simpler, you could use the 'Storable' module to dump your
      data to a file and read back from that file. And you might want to look at
      'tie' as well.

      Paul


      4:03pm, Roman Daszczyszak wrote:

      > Hello all,
      >
      > I am trying to figure out a way to do something in Perl, and would
      > appreciate y'alls opinion.
      >
      > I have a Perl script that I run each night to gather computer
      > information from the workstations on my network. It collects the
      > Windows NetBIOS name, the IP address and the MAC address, then dumps
      > this info into a tab-delimited file, one record on each line.
      >
      > I would like to improve upon this functionality by having the script
      > pull the previous night's information in, run the collection, then
      > compare the two sets of results and update any changes to the DB.
      > Reason being, right now if a machine is offline for the night, it gets
      > "wiped out" from the inventory when the script is run. Whereas it
      > should remain, unless the MAC address or computer name actually moves
      > to a different IP.
      >
      > Now, given the relatively small amount of information gathered right
      > now, I think a flat file is fine. However, if someone can point me at
      > a simple database that is easily implemented in Perl, that would be
      > great too.
      >
      > What do you all think? Flat file or database?
      >
      > Do you know of a couple good modules that can help me with this? I
      > have searched a bit on CPAN, and ActiveState (the Perl script is run
      > on a Windows machine) and cannot seem to find anything that will
      > manipulate a flat file DB.
      >
      > Any recommendations or advice is certainly welcome.
      >
      >
      > Unsubscribing info is here: http://help.yahoo.com/help/us/groups/groups-32.html
      > Yahoo! Groups Links
      >
      >
      >
      >
      >
      >



      ---------------------------------------------------------
      To mess up a Linux box, you need to work at it;
      to mess up your Windows box, you just need to work on it.
      I know which one I'll trust. How about you?
      --Scott Granneman
      ---------------------------------------------------------
    • merlyn@stonehenge.com
      ... Roman Now, given the relatively small amount of information gathered right Roman now, I think a flat file is fine. However, if someone can point me at
      Message 2 of 4 , Apr 5, 2006
      • 0 Attachment
        >>>>> "Roman" == Roman Daszczyszak <romandas@...> writes:

        Roman> Now, given the relatively small amount of information gathered right
        Roman> now, I think a flat file is fine. However, if someone can point me at
        Roman> a simple database that is easily implemented in Perl, that would be
        Roman> great too.

        See the articles I've written on data persistence by googling
        for "site:stonehenge.com persistence".

        Roman> Do you know of a couple good modules that can help me with this? I
        Roman> have searched a bit on CPAN, and ActiveState (the Perl script is run
        Roman> on a Windows machine) and cannot seem to find anything that will
        Roman> manipulate a flat file DB.

        Well, the quick-and-dirty solution I can easily recommend is DBM::Deep. It
        basically makes a "flat file" that has an arbitrarily deep array/hash
        structure, which you can structure as you wish. The nice thing is that it
        *doesn't* depend on any DBM already installed, or XS, and the installation is
        a single .pm file, making it perfect for the "shared host" solution.

        --
        Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
        <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
        Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
        See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
      • Mr. Shawn H. Corey
        ... I think they don t teach enough analysis in school. How are you using this data? Do you, most of the time, look up single records? Or do you use it all to
        Message 3 of 4 , Apr 5, 2006
        • 0 Attachment
          On Wed, 2006-05-04 at 16:03 +0200, Roman Daszczyszak wrote:
          > Hello all,
          >
          > I am trying to figure out a way to do something in Perl, and would
          > appreciate y'alls opinion.
          >
          > I have a Perl script that I run each night to gather computer
          > information from the workstations on my network. It collects the
          > Windows NetBIOS name, the IP address and the MAC address, then dumps
          > this info into a tab-delimited file, one record on each line.
          >
          > I would like to improve upon this functionality by having the script
          > pull the previous night's information in, run the collection, then
          > compare the two sets of results and update any changes to the DB.
          > Reason being, right now if a machine is offline for the night, it gets
          > "wiped out" from the inventory when the script is run. Whereas it
          > should remain, unless the MAC address or computer name actually moves
          > to a different IP.
          >
          > Now, given the relatively small amount of information gathered right
          > now, I think a flat file is fine. However, if someone can point me at
          > a simple database that is easily implemented in Perl, that would be
          > great too.
          >
          > What do you all think? Flat file or database?

          I think they don't teach enough analysis in school. How are you using
          this data? Do you, most of the time, look up single records? Or do you
          use it all to create a report or a statistical summary? Databases work
          best when most of the work is looking up a single (or a small set) of
          records. If you use all the data, a flat file takes up less disk space
          and is faster.

          I would have one array in my program that contained all the records and
          use tie (see `perldoc -f tie`) to give it persistency. That way, if a
          machine cannot be contacted, its old record remains.


          --
          __END__

          Just my 0.00000002 million dollars worth,
          --- Shawn

          "For the things we have to learn before we can do them, we learn by doing them."
          Aristotle

          * Perl tutorials at http://perlmonks.org/?node=Tutorials
          * A searchable perldoc is at http://perldoc.perl.org/
        Your message has been successfully submitted and would be delivered to recipients shortly.