Loading ...
Sorry, an error occurred while loading the content.
 

Regarding flat file access vs. database

Expand Messages
  • Roman Daszczyszak
    Hello all, I am trying to figure out a way to do something in Perl, and would appreciate y alls opinion. I have a Perl script that I run each night to gather
    Message 1 of 4 , Apr 5, 2006
      Hello all,

      I am trying to figure out a way to do something in Perl, and would
      appreciate y'alls opinion.

      I have a Perl script that I run each night to gather computer
      information from the workstations on my network. It collects the
      Windows NetBIOS name, the IP address and the MAC address, then dumps
      this info into a tab-delimited file, one record on each line.

      I would like to improve upon this functionality by having the script
      pull the previous night's information in, run the collection, then
      compare the two sets of results and update any changes to the DB.
      Reason being, right now if a machine is offline for the night, it gets
      "wiped out" from the inventory when the script is run. Whereas it
      should remain, unless the MAC address or computer name actually moves
      to a different IP.

      Now, given the relatively small amount of information gathered right
      now, I think a flat file is fine. However, if someone can point me at
      a simple database that is easily implemented in Perl, that would be
      great too.

      What do you all think? Flat file or database?

      Do you know of a couple good modules that can help me with this? I
      have searched a bit on CPAN, and ActiveState (the Perl script is run
      on a Windows machine) and cannot seem to find anything that will
      manipulate a flat file DB.

      Any recommendations or advice is certainly welcome.
    • Paul Archer
      I just got into a similar project (parsing NFS log files), and had the same decision to make. In my case, I know my data set is going to get large enough to
      Message 2 of 4 , Apr 5, 2006
        I just got into a similar project (parsing NFS log files), and had the same
        decision to make. In my case, I know my data set is going to get large
        enough to warrant using a real database. In your case, it sounds like it'd
        be easier to simply stay with the format you've already got.
        Just write (or find) a parser for your tab delimited file. Read it in at the
        start of your program and store the data in whatever format you use
        internally (hash of arrays, for example). Then gather your data, merge the
        sets, and write it all back out.
        To make things simpler, you could use the 'Storable' module to dump your
        data to a file and read back from that file. And you might want to look at
        'tie' as well.

        Paul


        4:03pm, Roman Daszczyszak wrote:

        > Hello all,
        >
        > I am trying to figure out a way to do something in Perl, and would
        > appreciate y'alls opinion.
        >
        > I have a Perl script that I run each night to gather computer
        > information from the workstations on my network. It collects the
        > Windows NetBIOS name, the IP address and the MAC address, then dumps
        > this info into a tab-delimited file, one record on each line.
        >
        > I would like to improve upon this functionality by having the script
        > pull the previous night's information in, run the collection, then
        > compare the two sets of results and update any changes to the DB.
        > Reason being, right now if a machine is offline for the night, it gets
        > "wiped out" from the inventory when the script is run. Whereas it
        > should remain, unless the MAC address or computer name actually moves
        > to a different IP.
        >
        > Now, given the relatively small amount of information gathered right
        > now, I think a flat file is fine. However, if someone can point me at
        > a simple database that is easily implemented in Perl, that would be
        > great too.
        >
        > What do you all think? Flat file or database?
        >
        > Do you know of a couple good modules that can help me with this? I
        > have searched a bit on CPAN, and ActiveState (the Perl script is run
        > on a Windows machine) and cannot seem to find anything that will
        > manipulate a flat file DB.
        >
        > Any recommendations or advice is certainly welcome.
        >
        >
        > Unsubscribing info is here: http://help.yahoo.com/help/us/groups/groups-32.html
        > Yahoo! Groups Links
        >
        >
        >
        >
        >
        >



        ---------------------------------------------------------
        To mess up a Linux box, you need to work at it;
        to mess up your Windows box, you just need to work on it.
        I know which one I'll trust. How about you?
        --Scott Granneman
        ---------------------------------------------------------
      • merlyn@stonehenge.com
        ... Roman Now, given the relatively small amount of information gathered right Roman now, I think a flat file is fine. However, if someone can point me at
        Message 3 of 4 , Apr 5, 2006
          >>>>> "Roman" == Roman Daszczyszak <romandas@...> writes:

          Roman> Now, given the relatively small amount of information gathered right
          Roman> now, I think a flat file is fine. However, if someone can point me at
          Roman> a simple database that is easily implemented in Perl, that would be
          Roman> great too.

          See the articles I've written on data persistence by googling
          for "site:stonehenge.com persistence".

          Roman> Do you know of a couple good modules that can help me with this? I
          Roman> have searched a bit on CPAN, and ActiveState (the Perl script is run
          Roman> on a Windows machine) and cannot seem to find anything that will
          Roman> manipulate a flat file DB.

          Well, the quick-and-dirty solution I can easily recommend is DBM::Deep. It
          basically makes a "flat file" that has an arbitrarily deep array/hash
          structure, which you can structure as you wish. The nice thing is that it
          *doesn't* depend on any DBM already installed, or XS, and the installation is
          a single .pm file, making it perfect for the "shared host" solution.

          --
          Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
          See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
        • Mr. Shawn H. Corey
          ... I think they don t teach enough analysis in school. How are you using this data? Do you, most of the time, look up single records? Or do you use it all to
          Message 4 of 4 , Apr 5, 2006
            On Wed, 2006-05-04 at 16:03 +0200, Roman Daszczyszak wrote:
            > Hello all,
            >
            > I am trying to figure out a way to do something in Perl, and would
            > appreciate y'alls opinion.
            >
            > I have a Perl script that I run each night to gather computer
            > information from the workstations on my network. It collects the
            > Windows NetBIOS name, the IP address and the MAC address, then dumps
            > this info into a tab-delimited file, one record on each line.
            >
            > I would like to improve upon this functionality by having the script
            > pull the previous night's information in, run the collection, then
            > compare the two sets of results and update any changes to the DB.
            > Reason being, right now if a machine is offline for the night, it gets
            > "wiped out" from the inventory when the script is run. Whereas it
            > should remain, unless the MAC address or computer name actually moves
            > to a different IP.
            >
            > Now, given the relatively small amount of information gathered right
            > now, I think a flat file is fine. However, if someone can point me at
            > a simple database that is easily implemented in Perl, that would be
            > great too.
            >
            > What do you all think? Flat file or database?

            I think they don't teach enough analysis in school. How are you using
            this data? Do you, most of the time, look up single records? Or do you
            use it all to create a report or a statistical summary? Databases work
            best when most of the work is looking up a single (or a small set) of
            records. If you use all the data, a flat file takes up less disk space
            and is faster.

            I would have one array in my program that contained all the records and
            use tie (see `perldoc -f tie`) to give it persistency. That way, if a
            machine cannot be contacted, its old record remains.


            --
            __END__

            Just my 0.00000002 million dollars worth,
            --- Shawn

            "For the things we have to learn before we can do them, we learn by doing them."
            Aristotle

            * Perl tutorials at http://perlmonks.org/?node=Tutorials
            * A searchable perldoc is at http://perldoc.perl.org/
          Your message has been successfully submitted and would be delivered to recipients shortly.