Loading ...
Sorry, an error occurred while loading the content.

RE: [pcgen-xml] XML plan summary - XML dependacies

Expand Messages
  • CC Americas 1 Carstensen James
    ... files. ... associations ... Keith, a couple of quick questions about this. 1. Will this be MOD-able? To use your example with race files having a
    Message 1 of 4 , Mar 4, 2003
    • 0 Attachment
      Keith wrote:
      > > > > 3. All files will identify their dependencies.
      > I don't think there are actually *too* many dependencies between
      files.
      > There are some, but for the most part they should be pretty obvious.
      > Actually, there are more than there could be because of how
      associations
      > are made between things, but I don't want to risk changing that right
      > now.

      Keith, a couple of quick questions about this.

      1. Will this be MOD-able? To use your example with race files having a
      dependency on class files for favored class, if a publisher uses
      standard SRD races but their own classes, and the favored classes change
      we don't want to load the SRD/PHB classes as a dependency since it won't
      be referenced.

      2. This may end up fragmenting files some - if part of A requires B,
      and a different part of B requires A, it's cleaner to break up either A
      (so A -> B -> C) or B (same) instead of having cross dependencies that
      we don't know the load order to satisfy.

      3. A solution for both might be instead of putting in /file/
      dependencies in the files, put in /XML ID/ dependencies and have PCGen
      (eventually, not phase 1) determine what it's missing and look though an
      (automatically generated) index of the LST files to find which it needs
      to include. (I'd suggest a default index in the DL and a tool option
      instead of reindexing on each load just for speed reasons. Maybe using
      the LST editors would auto-kick off a reindex.)

      Cheers,
      Blue

      "...you might be eaten by a Grue..."
    • Keith Davies
      ... I plan to have a key value that may be used to indicate that two files are to be considered the same . Then you just load the one you want to override
      Message 2 of 4 , Mar 4, 2003
      • 0 Attachment
        On Tue, Mar 04, 2003 at 11:45:32AM -0500, CC Americas 1 Carstensen James wrote:
        > Keith wrote:
        > > > > > 3. All files will identify their dependencies.
        > > I don't think there are actually *too* many dependencies between
        > files.
        > > There are some, but for the most part they should be pretty obvious.
        > > Actually, there are more than there could be because of how
        > associations
        > > are made between things, but I don't want to risk changing that right
        > > now.
        >
        > Keith, a couple of quick questions about this.
        >
        > 1. Will this be MOD-able? To use your example with race files having a
        > dependency on class files for favored class, if a publisher uses
        > standard SRD races but their own classes, and the favored classes change
        > we don't want to load the SRD/PHB classes as a dependency since it won't
        > be referenced.

        I plan to have a key value that may be used to indicate that two files
        are to be considered 'the same'. Then you just load the one you want to
        override the other first.

        An example would be the PHB, and the PHB with errata (assuming you
        didn't just load PHB, then errata, of course). You might have:

        <pcgen>
        <!-- mycampaign.pcc -->
        <depends href="data/saf.pcc" />
        <depends href="data/tab.pcc" />
        <depends href="data/sas.pcc" />

        <data>
        <!-- stuff -->
        </data>
        </pcgen>

        Now, normally the three splat books (abbreviated titles above) would
        indicate that they need the PHB. It is not necessary to load the PHB
        directly. However, they will want to use the core PHB. If we do the
        following:

        <pcgen>
        <!-- mycampaign.pcc -->
        <depends href="data/phb-errataed.pcc" />
        <depends href="data/saf.pcc" />
        <depends href="data/tab.pcc" />
        <depends href="data/sas.pcc" />

        <data>
        <!-- stuff -->
        </data>
        </pcgen>

        Then the errataed version of the PHB will be loaded. There is a key
        attribute in the phb-errataed file that indicates that it is to be
        considered the same as the PHB. When the program tries to load phb.pcc,
        it will find its key value already present and skip the contents.

        With the second splat book the file will have been marked as 'already
        loaded' by the href being present in a 'loaded' list, so it won't even
        open it.

        I haven't figured out a more elegant method of doing this.

        > 2. This may end up fragmenting files some - if part of A requires B,
        > and a different part of B requires A, it's cleaner to break up either A
        > (so A -> B -> C) or B (same) instead of having cross dependencies that
        > we don't know the load order to satisfy.

        Probably a good idea to avoid circular references, but these should be
        fairly easy to handle. When a file is opened it gets marked as 'loaded
        or will be loaded'. When another file requests it, it will be known
        that the data will be available at some time. As a result, we don't
        need to worry too much about.

        I agree that it's cleaner to break the files up so that dependencies are
        hierarchical rather than linear but at this point I don't want to
        disturb things any more than we need to.

        > 3. A solution for both might be instead of putting in /file/
        > dependencies in the files, put in /XML ID/ dependencies and have PCGen
        > (eventually, not phase 1) determine what it's missing and look though an
        > (automatically generated) index of the LST files to find which it needs
        > to include. (I'd suggest a default index in the DL and a tool option
        > instead of reindexing on each load just for speed reasons. Maybe using
        > the LST editors would auto-kick off a reindex.)

        I'd considered putting IDs on the individual files and just specifying
        in each file which file IDs need to be satisfied, but I figured it
        wouldn't fly right now. What you're describing is different, but an
        interesting idea. I'll need to think about it.

        The index *should* be pretty static. The only times it would change is
        when things are added or removed, so there would be no real need to
        regen it each time. However, one downside is that it might load *only*
        what is needed, not necessarily what is desired, or that it might
        require the load of entire files when only a small part is needed.

        I have considered predeclaration of data -- a file that contains the IDs
        and *bare minimal* information about things being used (name,
        basically). Later files would modify these declarations for actual use.
        This would relax the dependencies between data because they could be
        formally linked when used.

        That is, if 'races.xc' needs class IDs, it might load 'classes.xh'. All
        IDs are satisfied, so processing can continue. If someone uses 'elf',
        it may need information about 'wizard' and so will trigger loading
        'classes.xc'.

        I'm not sure if this is a simplification or not. I do know that
        maintaining the information, even minimal information like this, in two
        files will not be popular.


        Keith
        --
        Keith Davies
        keith.davies@...

        PCGen: <reaper/>, smartass
        "You just can't argue with a moron. It's like handling Nuclear
        waste. It's not good, it's not evil, but for Christ's sake, don't
        get any on you!!" -- Chuck, PCGen mailing list
      • CC Americas 1 Carstensen James
        ... PCGen ... an ... needs ... using ... You still have your sources, this would be only to bring in parts of additional sources. For example, say that
        Message 3 of 4 , Mar 4, 2003
        • 0 Attachment
          Keith wrote:
          >> 3. A solution for both might be instead of putting in /file/
          >> dependencies in the files, put in /XML ID/ dependencies and have
          PCGen
          >> (eventually, not phase 1) determine what it's missing and look though
          an
          >> (automatically generated) index of the LST files to find which it
          needs
          >> to include. (I'd suggest a default index in the DL and a tool option
          >> instead of reindexing on each load just for speed reasons. Maybe
          using
          >> the LST editors would auto-kick off a reindex.)
          >
          >I'd considered putting IDs on the individual files and just specifying
          >in each file which file IDs need to be satisfied, but I figured it
          >wouldn't fly right now. What you're describing is different, but an
          >interesting idea. I'll need to think about it.
          >
          >The index *should* be pretty static. The only times it would change is
          >when things are added or removed, so there would be no real need to
          >regen it each time. However, one downside is that it might load *only*
          >what is needed, not necessarily what is desired, or that it might
          >require the load of entire files when only a small part is needed.

          You still have your sources, this would be only to bring in parts of
          additional sources.

          For example, say that "Monte's Ranger" was in "Monte's Web Stuff" which
          also contained other things. A DM modified the favored class for elves
          to Monte's Ranger, but aren't using anything else from Monte's Stuff.

          The DM loads SRD, Splatbooks (from CMP), and Traps & Treachery. The SRD
          sees that as a favored class for Elf it needs "class.ranger-monte-cook".
          It finds in the index to know which file, and adds it to the
          too-be-loaded list (or the "you need this source too should I add it"
          list). (And of course since you just added a source you need to check
          for additional dependencies, but you would need to do that anyway.)

          Yes, this functionality can currently be done with the PCC file - it
          would be expanding on it just like XML file dependencies would expand it
          with two critical differences.

          1. Finer granularity - only load exact dependencies, not whole file for
          one dependency.
          2. Doesn't need to be maintained.

          >I have considered predeclaration of data -- a file that contains the
          IDs
          >and *bare minimal* information about things being used (name,
          >basically). Later files would modify these declarations for actual
          use.
          >This would relax the dependencies between data because they could be
          >formally linked when used.

          This could be automatically generated, and be the index we were talking
          about above if we also included the file to load from. This would also
          stop the need for modifying it from later files, keeping it cleaner. In
          my book the more places we automate things people shouldn't need to
          think of, the better.

          >That is, if 'races.xc' needs class IDs, it might load 'classes.xh'.
          All
          >IDs are satisfied, so processing can continue. If someone uses 'elf',
          >it may need information about 'wizard' and so will trigger loading
          >'classes.xc'.

          Ah, this would cut down on the load time of the index. However, an
          index that doesn't need complex parsing (all the same format: XML ID =
          this filename), only needs to be gone through once, and can be discarded
          after source load doesn't sound like it should be too resource
          intensive, either in terms of time or long term memory usage.

          >I'm not sure if this is a simplification or not. I do know that
          >maintaining the information, even minimal information like this, in two
          >files will not be popular.

          That's why I'm suggesting that we automate it. it's really along the
          lines of a recursive grep through the data directory for the id tag, and
          generating a list of id tags to the file they exist in.

          Bad perlish pseudopodia

          my %foundin;
          my @files=get_filenames; #(recursive opendir to get list of filename)

          foreach my $file (@files) { # Go through files, load into $file
          next if /^\./; # Skip . & .. on UNIX platforms
          open (IN,$file); # Open file
          while <IN> { # Read each line, put in $_
          next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; # Skip
          unless has a id tag, and put contents of tag in $1
          $foundin{$1}=$file; # load hash %foundin with filename
          keyed by XML ID
          }
          close IN;
          }

          Sorry, very message and that's not the nicest regex. (For everyone else
          reading, the line next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; should
          be read as:

          Skip to the next line unless the current line contains: less than,
          optional whitespace, the string "id", optional whitespace, an equals
          sign, more optional whitespace, double quotes, (start remembering), one
          or more of word characters, periods, or dashes, (stop remembering),
          double quotes, yet more optional whitespace, greater than.

          Not sure if I properly escaped everything, but you should get the idea.
          Read through the index file once, now for every dependency XML ID you
          know where to find it. This only works with the unique ids, but as long
          as we have them we might as well push all of this off onto the computer
          to leave people free to do the hard stuff.

          Cheers,
          Blue
        • Keith Davies
          On Tue, Mar 04, 2003 at 12:55:06PM -0500, CC Americas 1 Carstensen James wrote: ... I like the idea. Let me think about it for a bit; I think I
          Message 4 of 4 , Mar 4, 2003
          • 0 Attachment
            On Tue, Mar 04, 2003 at 12:55:06PM -0500, CC Americas 1 Carstensen James wrote:

            <killer snip>

            > Bad perlish pseudopodia
            >
            > my %foundin;
            > my @files=get_filenames; #(recursive opendir to get list of filename)
            >
            > foreach my $file (@files) { # Go through files, load into $file
            > next if /^\./; # Skip . & .. on UNIX platforms
            > open (IN,$file); # Open file
            > while <IN> { # Read each line, put in $_
            > next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; # Skip
            > unless has a id tag, and put contents of tag in $1
            > $foundin{$1}=$file; # load hash %foundin with filename
            > keyed by XML ID
            > }
            > close IN;
            > }
            >
            > Sorry, very message and that's not the nicest regex. (For everyone else
            > reading, the line next unless /<\s*id\s*=\s*"([\w\.\-]+)"\s*>/; should
            > be read as:
            >
            > Skip to the next line unless the current line contains: less than,
            > optional whitespace, the string "id", optional whitespace, an equals
            > sign, more optional whitespace, double quotes, (start remembering), one
            > or more of word characters, periods, or dashes, (stop remembering),
            > double quotes, yet more optional whitespace, greater than.
            >
            > Not sure if I properly escaped everything, but you should get the idea.
            > Read through the index file once, now for every dependency XML ID you
            > know where to find it. This only works with the unique ids, but as long
            > as we have them we might as well push all of this off onto the computer
            > to leave people free to do the hard stuff.

            I like the idea. Let me think about it for a bit; I think I see a
            couple things that might cause problems and I don't have time to
            concentrate on this at the moment.


            Keith
            --
            Keith Davies
            keith.davies@...

            PCGen: <reaper/>, smartass
            "You just can't argue with a moron. It's like handling Nuclear
            waste. It's not good, it's not evil, but for Christ's sake, don't
            get any on you!!" -- Chuck, PCGen mailing list
          Your message has been successfully submitted and would be delivered to recipients shortly.