Loading ...
Sorry, an error occurred while loading the content.

Re: [pcgen-xml] XML plan summary - XML dependacies

Expand Messages
  • Keith Davies
    ... I plan to have a key value that may be used to indicate that two files are to be considered the same . Then you just load the one you want to override
    Message 1 of 4 , Mar 4, 2003
    • 0 Attachment
      On Tue, Mar 04, 2003 at 11:45:32AM -0500, CC Americas 1 Carstensen James wrote:
      > Keith wrote:
      > > > > > 3. All files will identify their dependencies.
      > > I don't think there are actually *too* many dependencies between
      > files.
      > > There are some, but for the most part they should be pretty obvious.
      > > Actually, there are more than there could be because of how
      > associations
      > > are made between things, but I don't want to risk changing that right
      > > now.
      >
      > Keith, a couple of quick questions about this.
      >
      > 1. Will this be MOD-able? To use your example with race files having a
      > dependency on class files for favored class, if a publisher uses
      > standard SRD races but their own classes, and the favored classes change
      > we don't want to load the SRD/PHB classes as a dependency since it won't
      > be referenced.

      I plan to have a key value that may be used to indicate that two files
      are to be considered 'the same'. Then you just load the one you want to
      override the other first.

      An example would be the PHB, and the PHB with errata (assuming you
      didn't just load PHB, then errata, of course). You might have:

      <pcgen>
      <!-- mycampaign.pcc -->
      <depends href="data/saf.pcc" />
      <depends href="data/tab.pcc" />
      <depends href="data/sas.pcc" />

      <data>
      <!-- stuff -->
      </data>
      </pcgen>

      Now, normally the three splat books (abbreviated titles above) would
      indicate that they need the PHB. It is not necessary to load the PHB
      directly. However, they will want to use the core PHB. If we do the
      following:

      <pcgen>
      <!-- mycampaign.pcc -->
      <depends href="data/phb-errataed.pcc" />
      <depends href="data/saf.pcc" />
      <depends href="data/tab.pcc" />
      <depends href="data/sas.pcc" />

      <data>
      <!-- stuff -->
      </data>
      </pcgen>

      Then the errataed version of the PHB will be loaded. There is a key
      attribute in the phb-errataed file that indicates that it is to be
      considered the same as the PHB. When the program tries to load phb.pcc,
      it will find its key value already present and skip the contents.

      With the second splat book the file will have been marked as 'already
      loaded' by the href being present in a 'loaded' list, so it won't even
      open it.

      I haven't figured out a more elegant method of doing this.

      > 2. This may end up fragmenting files some - if part of A requires B,
      > and a different part of B requires A, it's cleaner to break up either A
      > (so A -> B -> C) or B (same) instead of having cross dependencies that
      > we don't know the load order to satisfy.

      Probably a good idea to avoid circular references, but these should be
      fairly easy to handle. When a file is opened it gets marked as 'loaded
      or will be loaded'. When another file requests it, it will be known
      that the data will be available at some time. As a result, we don't
      need to worry too much about.

      I agree that it's cleaner to break the files up so that dependencies are
      hierarchical rather than linear but at this point I don't want to
      disturb things any more than we need to.

      > 3. A solution for both might be instead of putting in /file/
      > dependencies in the files, put in /XML ID/ dependencies and have PCGen
      > (eventually, not phase 1) determine what it's missing and look though an
      > (automatically generated) index of the LST files to find which it needs
      > to include. (I'd suggest a default index in the DL and a tool option
      > instead of reindexing on each load just for speed reasons. Maybe using
      > the LST editors would auto-kick off a reindex.)

      I'd considered putting IDs on the individual files and just specifying
      in each file which file IDs need to be satisfied, but I figured it
      wouldn't fly right now. What you're describing is different, but an
      interesting idea. I'll need to think about it.

      The index *should* be pretty static. The only times it would change is
      when things are added or removed, so there would be no real need to
      regen it each time. However, one downside is that it might load *only*
      what is needed, not necessarily what is desired, or that it might
      require the load of entire files when only a small part is needed.

      I have considered predeclaration of data -- a file that contains the IDs
      and *bare minimal* information about things being used (name,
      basically). Later files would modify these declarations for actual use.
      This would relax the dependencies between data because they could be
      formally linked when used.

      That is, if 'races.xc' needs class IDs, it might load 'classes.xh'. All
      IDs are satisfied, so processing can continue. If someone uses 'elf',
      it may need information about 'wizard' and so will trigger loading
      'classes.xc'.

      I'm not sure if this is a simplification or not. I do know that
      maintaining the information, even minimal information like this, in two
      files will not be popular.


      Keith
      --
      Keith Davies
      keith.davies@...

      PCGen: <reaper/>, smartass
      "You just can't argue with a moron. It's like handling Nuclear
      waste. It's not good, it's not evil, but for Christ's sake, don't
      get any on you!!" -- Chuck, PCGen mailing list
    • CC Americas 1 Carstensen James
      ... PCGen ... an ... needs ... using ... You still have your sources, this would be only to bring in parts of additional sources. For example, say that
      Message 2 of 4 , Mar 4, 2003
      • 0 Attachment
        Keith wrote:
        >> 3. A solution for both might be instead of putting in /file/
        >> dependencies in the files, put in /XML ID/ dependencies and have
        PCGen
        >> (eventually, not phase 1) determine what it's missing and look though
        an
        >> (automatically generated) index of the LST files to find which it
        needs
        >> to include. (I'd suggest a default index in the DL and a tool option
        >> instead of reindexing on each load just for speed reasons. Maybe
        using
        >> the LST editors would auto-kick off a reindex.)
        >
        >I'd considered putting IDs on the individual files and just specifying
        >in each file which file IDs need to be satisfied, but I figured it
        >wouldn't fly right now. What you're describing is different, but an
        >interesting idea. I'll need to think about it.
        >
        >The index *should* be pretty static. The only times it would change is
        >when things are added or removed, so there would be no real need to
        >regen it each time. However, one downside is that it might load *only*
        >what is needed, not necessarily what is desired, or that it might
        >require the load of entire files when only a small part is needed.

        You still have your sources, this would be only to bring in parts of
        additional sources.

        For example, say that "Monte's Ranger" was in "Monte's Web Stuff" which
        also contained other things. A DM modified the favored class for elves
        to Monte's Ranger, but aren't using anything else from Monte's Stuff.

        The DM loads SRD, Splatbooks (from CMP), and Traps & Treachery. The SRD
        sees that as a favored class for Elf it needs "class.ranger-monte-cook".
        It finds in the index to know which file, and adds it to the
        too-be-loaded list (or the "you need this source too should I add it"
        list). (And of course since you just added a source you need to check
        for additional dependencies, but you would need to do that anyway.)

        Yes, this functionality can currently be done with the PCC file - it
        would be expanding on it just like XML file dependencies would expand it
        with two critical differences.

        1. Finer granularity - only load exact dependencies, not whole file for
        one dependency.
        2. Doesn't need to be maintained.

        >I have considered predeclaration of data -- a file that contains the
        IDs
        >and *bare minimal* information about things being used (name,
        >basically). Later files would modify these declarations for actual
        use.
        >This would relax the dependencies between data because they could be
        >formally linked when used.

        This could be automatically generated, and be the index we were talking
        about above if we also included the file to load from. This would also
        stop the need for modifying it from later files, keeping it cleaner. In
        my book the more places we automate things people shouldn't need to
        think of, the better.

        >That is, if 'races.xc' needs class IDs, it might load 'classes.xh'.
        All
        >IDs are satisfied, so processing can continue. If someone uses 'elf',
        >it may need information about 'wizard' and so will trigger loading
        >'classes.xc'.

        Ah, this would cut down on the load time of the index. However, an
        index that doesn't need complex parsing (all the same format: XML ID =
        this filename), only needs to be gone through once, and can be discarded
        after source load doesn't sound like it should be too resource
        intensive, either in terms of time or long term memory usage.

        >I'm not sure if this is a simplification or not. I do know that
        >maintaining the information, even minimal information like this, in two
        >files will not be popular.

        That's why I'm suggesting that we automate it. it's really along the
        lines of a recursive grep through the data directory for the id tag, and
        generating a list of id tags to the file they exist in.

        Bad perlish pseudopodia

        my %foundin;
        my @files=get_filenames; #(recursive opendir to get list of filename)

        foreach my $file (@files) { # Go through files, load into $file
        next if /^\./; # Skip . & .. on UNIX platforms
        open (IN,$file); # Open file
        while <IN> { # Read each line, put in $_
        next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; # Skip
        unless has a id tag, and put contents of tag in $1
        $foundin{$1}=$file; # load hash %foundin with filename
        keyed by XML ID
        }
        close IN;
        }

        Sorry, very message and that's not the nicest regex. (For everyone else
        reading, the line next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; should
        be read as:

        Skip to the next line unless the current line contains: less than,
        optional whitespace, the string "id", optional whitespace, an equals
        sign, more optional whitespace, double quotes, (start remembering), one
        or more of word characters, periods, or dashes, (stop remembering),
        double quotes, yet more optional whitespace, greater than.

        Not sure if I properly escaped everything, but you should get the idea.
        Read through the index file once, now for every dependency XML ID you
        know where to find it. This only works with the unique ids, but as long
        as we have them we might as well push all of this off onto the computer
        to leave people free to do the hard stuff.

        Cheers,
        Blue
      • Keith Davies
        On Tue, Mar 04, 2003 at 12:55:06PM -0500, CC Americas 1 Carstensen James wrote: ... I like the idea. Let me think about it for a bit; I think I
        Message 3 of 4 , Mar 4, 2003
        • 0 Attachment
          On Tue, Mar 04, 2003 at 12:55:06PM -0500, CC Americas 1 Carstensen James wrote:

          <killer snip>

          > Bad perlish pseudopodia
          >
          > my %foundin;
          > my @files=get_filenames; #(recursive opendir to get list of filename)
          >
          > foreach my $file (@files) { # Go through files, load into $file
          > next if /^\./; # Skip . & .. on UNIX platforms
          > open (IN,$file); # Open file
          > while <IN> { # Read each line, put in $_
          > next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; # Skip
          > unless has a id tag, and put contents of tag in $1
          > $foundin{$1}=$file; # load hash %foundin with filename
          > keyed by XML ID
          > }
          > close IN;
          > }
          >
          > Sorry, very message and that's not the nicest regex. (For everyone else
          > reading, the line next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; should
          > be read as:
          >
          > Skip to the next line unless the current line contains: less than,
          > optional whitespace, the string "id", optional whitespace, an equals
          > sign, more optional whitespace, double quotes, (start remembering), one
          > or more of word characters, periods, or dashes, (stop remembering),
          > double quotes, yet more optional whitespace, greater than.
          >
          > Not sure if I properly escaped everything, but you should get the idea.
          > Read through the index file once, now for every dependency XML ID you
          > know where to find it. This only works with the unique ids, but as long
          > as we have them we might as well push all of this off onto the computer
          > to leave people free to do the hard stuff.

          I like the idea. Let me think about it for a bit; I think I see a
          couple things that might cause problems and I don't have time to
          concentrate on this at the moment.


          Keith
          --
          Keith Davies
          keith.davies@...

          PCGen: <reaper/>, smartass
          "You just can't argue with a moron. It's like handling Nuclear
          waste. It's not good, it's not evil, but for Christ's sake, don't
          get any on you!!" -- Chuck, PCGen mailing list
        Your message has been successfully submitted and would be delivered to recipients shortly.