Loading ...
Sorry, an error occurred while loading the content.

RE: [pcgen] Duplicate bonuses from duplicate sources

Expand Messages
  • Barak
    AFAIK, the KEY tag only works in an equipment modifier file. Barak ~PCGen BoD ~OS Silverback
    Message 1 of 19 , Nov 3, 2003
    • 0 Attachment
      AFAIK, the KEY tag only works in an equipment modifier file.

      Barak
      ~PCGen BoD
      ~OS Silverback


      > -----Original Message-----
      > From: Brass Tilde [mailto:brasstilde@...]
      >
      > The duplicate item problem seems to be a fairly common one.
      > Is there any way to have PCGen warn of duplicate items? Not
      > necessarily as they occur, but one window at the end of
      > source load that said something like "These items, from these
      > source files, are duplicates" with a list.
      >
      > Alternatively, is the KEY tag functionality sufficient to
      > differentiate between items loaded from the SRD (or any other
      > data set) and the CMP ones? For example, if the MONK from the
      > SRD was loaded at the same time as the MONK from the CMP
      > sets, but with a different KEY tag, would PCGen be able to
      > keep the two separate? Even if there were two MONK entries
      > in the class tab, the source info below would show the
      > difference to the user.
    • Frugal
      ... Is the problem that there are duplicate items in different LST files, or is the problem that the 2 different PCC files are both
      Message 2 of 19 , Nov 3, 2003
      • 0 Attachment
        <quote who="Brass Tilde">
        > The duplicate item problem seems to be a fairly common one. Is there any
        > way to have PCGen warn of duplicate items? Not necessarily as they occur,
        > but one window at the end of source load that said something like "These
        > items, from these source files, are duplicates" with a list.

        Is the problem that there are duplicate items in different LST files, or
        is the problem that the 2 different PCC files are both loading the same
        LST file ?

        --
        regards,
        Frugal
        -OS Chimp
      • Brass Tilde
        ... any ... occur, ... As far as I ve been able to determine, the former.
        Message 3 of 19 , Nov 3, 2003
        • 0 Attachment
          > > The duplicate item problem seems to be a fairly common one. Is there
          any
          > > way to have PCGen warn of duplicate items? Not necessarily as they
          occur,


          > Is the problem that there are duplicate items in different LST files, or
          > is the problem that the 2 different PCC files are both loading the same
          > LST file ?

          As far as I've been able to determine, the former.
        • taluroniscandar
          ... there ... files, or ... same ... Having duplicate names in different lst files has been a BADIdea (TM) since somewhere around version 2.7. Some lst file
          Message 4 of 19 , Nov 3, 2003
          • 0 Attachment
            --- In pcgen@yahoogroups.com, "Brass Tilde" <brasstilde@i...> wrote:
            > > > The duplicate item problem seems to be a fairly common one. Is
            there
            > any
            > > > way to have PCGen warn of duplicate items? Not necessarily as they
            > occur,
            >
            >
            > > Is the problem that there are duplicate items in different LST
            files, or
            > > is the problem that the 2 different PCC files are both loading the
            same
            > > LST file ?
            >
            > As far as I've been able to determine, the former.

            Having duplicate names in different lst files has been a BADIdea (TM)
            since somewhere around version 2.7. Some lst file types react worse to
            it than others is all. Classes and feats are the worst. Spells and
            equipment are the most forgiving. Hmph, having the same name for a
            feat AND a class is bad (which WoTC does see Foe Hunter).

            I believe the reason it has been coming up more lately is the rsrd
            includes classes and feat that exist in other, older lst files. To
            load srd and rsrd together is the worst. But if you have certain older
            lst files and have maintaining them by hand you have to remember that
            the rsrd has certain items not found in the srd.
          • Brass Tilde
            ... *I* think the reason for it is the advent of the CMP data sets, which people are inadvertently loading along with the SRD. That s why I asked about the
            Message 5 of 19 , Nov 3, 2003
            • 0 Attachment
              > I believe the reason it has been coming up more lately is the rsrd
              > includes classes and feat that exist in other, older lst files. To
              > load srd and rsrd together is the worst. But if you have certain older
              > lst files and have maintaining them by hand you have to remember that
              > the rsrd has certain items not found in the srd.

              *I* think the reason for it is the advent of the CMP data sets, which people
              are inadvertently loading along with the SRD. That's why I asked about the
              KEY tag. I wasn't aware that it's functionality was limited to just the
              EQMOD stuff; I thought that the KEY tag had been implemented for all
              entries.
            • Brass Tilde
              ... That s kind of a bummer. I thought it had been implemented for everything. I see no evidence for it in the docs though, and I don t have access to the
              Message 6 of 19 , Nov 3, 2003
              • 0 Attachment
                > > The duplicate item problem seems to be a fairly common one.
                > > Is there any way to have PCGen warn of duplicate items? Not
                > > necessarily as they occur, but one window at the end of
                > > source load that said something like "These items, from these
                > > source files, are duplicates" with a list.

                > > Alternatively, is the KEY tag functionality sufficient to
                > > differentiate between items loaded from the SRD (or any other


                > AFAIK, the KEY tag only works in an equipment modifier file.

                That's kind of a bummer. I thought it had been implemented for everything.
                I see no evidence for it in the docs though, and I don't have access to the
                code from here.

                Ah well. What about the other option, warning the user of duplicate items,
                whether they be classes, races or whatever? Is that feasible? Is that
                functionality currently displayed in debug mode? If so, it might be
                reasonable to move it out of there and post a message dialog about it.
              • Eddy Anthony
                This sounds similar to an experience I had a few months back, might be related so I ll pass on the story. One of my players had some stats that were just
                Message 7 of 19 , Nov 3, 2003
                • 0 Attachment
                  This sounds similar to an experience I had a few months back, might be
                  related so I'll pass on the story. One of my players had some stats that
                  were just wrong, to high. I started investigating and found that the problem
                  was one I had created. We have many DM's in our group and we play multiple
                  games. I had created a GAMEMODE for one of them as it had fairly complex
                  homebrew rules that needed changes in the gamemode. What messed me up was
                  that I duplicated the SRD files and changed the GAMEMODE tag. Now I know
                  that they should not have loaded or be seen in the normal gamemode (and you
                  could not see them) but somehow some of it was getting double loaded as the
                  problem disappeared when I removed the files. I have some of suggestions as
                  to dealing with this situation.


                  1) Have the Source Material Selected panel show all the sources.
                  Since a .pcc file can load other .pcc files from any other gamemode it would
                  be good if they show up in this panel even if they don't appear in the
                  Sources available panel.

                  2) Some sort of warning when duplicate objects are loaded. Now I would not
                  want a window to pop up every time this happened (but that might be a good
                  idea) but something like the console which could give me a list of these
                  things would be nice.

                  3) Do not allow duplicate objects. PCGen loads sources by rank right? What
                  if you make it so that if it runs into a duplicate object in another .pcc
                  file it just ignores it and it never loads. I know this may cause problems
                  for the source that has an object not loaded especially considering that it
                  now my have other objects referencing something for a completely different
                  source, but is this problem greater than the existing situation.

                  4) Allow objects from each dataset (.pcc file) to be unique and exist on its
                  own. That way if there are 2 versions of something they will both be on the
                  list and you could tell them apart by selecting them and seeing the source.
                  I'm no programmer but I do see that this would be most difficult (if not
                  impossible) to implement as it cuts deep into how the program functions. I
                  put it out here because this would be ideal.


                  This problem will only increase as the number of game books does the
                  same. I know that some publishers have the bad habit of using the same names
                  for feats, classes etc. in their books as WotC and, in addition, sometimes
                  reprint these classes and feats in multiple books, and we know what happens
                  when the same feat is loaded from 2 different sources.

                  --
                  Eddy
                  PCGen Doc Gibbon
                • Michael Tucker
                  I have a question: how will the software be able to tell the difference between: a) an erroneous duplication of an object (e.g., a second feat with the same
                  Message 8 of 19 , Nov 3, 2003
                  • 0 Attachment
                    I have a question: how will the software be able to tell the difference
                    between:

                    a) an erroneous duplication of an object (e.g., a second feat with the
                    same name as one that's already been loaded), and

                    b) a legitimate extension, modification or redefinition of an object?

                    Without tediously extensive logic to see whether a record falls into
                    the latter category, I don't see any way to do this. (Check for ".MOD",
                    ".COPY", etc.; if none of those, then check to see whether a new tag is
                    being added to an existing object, thus extending its definition; if
                    the answer is "apparently not", then throw an exception.)

                    To my thinking, the only solution is to use some sort of unique
                    identifier on all objects (something like the KEY tag), so PCGen could
                    tell whether this is supposed to be a new object (with a new and
                    different KEY), or refers to an existing one (with the same KEY).
                    Either that, or carefully edit all of the data files to insure that all
                    object names are unique, if we're going to be keying off of the object
                    names.

                    Another approach would be to "compile" the data files. That way, you
                    could parse and validate each file *once* after modifying it (not every
                    #$@! time you load it), and all such conflicts and inconsistencies
                    could be resolved. Every object could have an internal unique
                    identifier, assigned by the "compiler". As an extra benefit, PCGen
                    would run a *lot* faster.

                    But that's opening a completely different can of worms. :-P

                    Yours,
                    Java Kensai
                  • Brass Tilde
                    ... would ... I like this one. Maybe have the loaded panel be a tree like the left pane, with the sources loaded by the main entry be child nodes. ... Yes,
                    Message 9 of 19 , Nov 3, 2003
                    • 0 Attachment
                      > 1) Have the Source Material Selected panel show all the sources.
                      > Since a .pcc file can load other .pcc files from any other gamemode it
                      would
                      > be good if they show up in this panel even if they don't appear in the
                      > Sources available panel.

                      I like this one. Maybe have the loaded panel be a tree like the left pane,
                      with the sources loaded by the "main" entry be child nodes.

                      > 2) Some sort of warning when duplicate objects are loaded. Now I would not
                      > want a window to pop up every time this happened (but that might be a good
                      > idea) but something like the console which could give me a list of these
                      > things would be nice.

                      Yes, one dialog at the end of source loading saying something like "The
                      following items are duplicated" with a list of items and their source.

                      >
                      > 3) Do not allow duplicate objects. PCGen loads sources by rank right? What
                      > if you make it so that if it runs into a duplicate object in another .pcc
                      > file it just ignores it and it never loads. I know this may cause problems
                      > for the source that has an object not loaded especially considering that
                      it
                      > now my have other objects referencing something for a completely different
                      > source, but is this problem greater than the existing situation.

                      The problem with this one is that the item, even though duplicately named,
                      may not actually be duplicated. It might actually be a different item,
                      however slightly different. I believe the general practice in the past has
                      been to use the publisher's name in the item to make sure they were unique,
                      with the first to be published getting the "honor" of having the plain name.
                      This would imply that either the SRD item names would have to be appended
                      with "(SRD)" or the WotC files created by CMP would need their names
                      changed.

                      > 4) Allow objects from each dataset (.pcc file) to be unique and exist on
                      its
                      > own. That way if there are 2 versions of something they will both be on
                      the
                      > list and you could tell them apart by selecting them and seeing the
                      source.
                      > I'm no programmer but I do see that this would be most difficult (if not
                      > impossible) to implement as it cuts deep into how the program functions. I
                      > put it out here because this would be ideal.

                      It would require that the program add a unique key to each loaded item *and*
                      be able to properly associate that item with all the other items which refer
                      to it by name. So the program would be required to know that they "Monk" to
                      which the "TYPE:Monk" in the Kama entry refers to the WotC Monk, and not the
                      Billy Joe Bob's campaign Monk.

                      Adding a KEY tag and demanding that it be unique (with the aforementioned
                      denial of loading for duplicate items), and including it at LST design time
                      rather than program run time, would actually be better. Though, frankly, it
                      would likely be time consuming as well.
                    • Frugal
                      ... Funnily enough the XML data project was going to do exactly this, have a unique ID for each object (class, feat, skill etc) So
                      Message 10 of 19 , Nov 3, 2003
                      • 0 Attachment
                        <quote who="Michael Tucker">
                        > To my thinking, the only solution is to use some sort of unique
                        > identifier on all objects (something like the KEY tag), so PCGen could
                        > tell whether this is supposed to be a new object (with a new and
                        > different KEY), or refers to an existing one (with the same KEY).
                        > Either that, or carefully edit all of the data files to insure that all
                        > object names are unique, if we're going to be keying off of the object
                        > names.

                        Funnily enough the XML data project was going to do exactly this, have a
                        unique ID for each object (class, feat, skill etc) So duplicates would not
                        be possible

                        > Another approach would be to "compile" the data files. That way, you
                        > could parse and validate each file *once* after modifying it (not every
                        > #$@! time you load it), and all such conflicts and inconsistencies
                        > could be resolved. Every object could have an internal unique
                        > identifier, assigned by the "compiler". As an extra benefit, PCGen
                        > would run a *lot* faster.

                        You are going to need to justify this statement. I do not see that there
                        is any great advantage to loading the data and parsing the text and
                        loading the data and parsing the code.

                        the major time constraint is searching for an existing instance of that
                        data and updating the internal representation. Both of which yuo will
                        still have to do.

                        --
                        regards,
                        Frugal
                        -OS Chimp
                      • Ed Beckwith
                        how hard would it be to config the system to append the full path of the .lst file from a given point to any name sources... for example, instead of
                        Message 11 of 19 , Nov 3, 2003
                        • 0 Attachment
                          how hard would it be to config the system to append the full path of the
                          .lst file from a given point to any name sources... for example, instead of
                          CLASS:Cleric, the computer reads CLASS:d20ogl\srd\srdclassesbase.lst\Cleric

                          this wouldbe of course different from the
                          d20ogl\rsrd\rsrdclassesbase.lst\Cleric

                          my 2 copper

                          Ed
                          ----- Original Message -----
                          From: "Eddy Anthony" <eddyba@...>
                          To: "PCGen NewsGroup" <pcgen@yahoogroups.com>
                          Sent: Monday, November 03, 2003 10:36 AM
                          Subject: [pcgen] Re: Duplicate bonuses from duplicate sources


                          > This sounds similar to an experience I had a few months back, might be
                          > related so I'll pass on the story. One of my players had some stats that
                          > were just wrong, to high. I started investigating and found that the
                          problem
                          > was one I had created. We have many DM's in our group and we play multiple
                          > games. I had created a GAMEMODE for one of them as it had fairly complex
                          > homebrew rules that needed changes in the gamemode. What messed me up was
                          > that I duplicated the SRD files and changed the GAMEMODE tag. Now I know
                          > that they should not have loaded or be seen in the normal gamemode (and
                          you
                          > could not see them) but somehow some of it was getting double loaded as
                          the
                          > problem disappeared when I removed the files. I have some of suggestions
                          as
                          > to dealing with this situation.
                          >
                          >
                          > 1) Have the Source Material Selected panel show all the sources.
                          > Since a .pcc file can load other .pcc files from any other gamemode it
                          would
                          > be good if they show up in this panel even if they don't appear in the
                          > Sources available panel.
                          >
                          > 2) Some sort of warning when duplicate objects are loaded. Now I would not
                          > want a window to pop up every time this happened (but that might be a good
                          > idea) but something like the console which could give me a list of these
                          > things would be nice.
                          >
                          > 3) Do not allow duplicate objects. PCGen loads sources by rank right? What
                          > if you make it so that if it runs into a duplicate object in another .pcc
                          > file it just ignores it and it never loads. I know this may cause problems
                          > for the source that has an object not loaded especially considering that
                          it
                          > now my have other objects referencing something for a completely different
                          > source, but is this problem greater than the existing situation.
                          >
                          > 4) Allow objects from each dataset (.pcc file) to be unique and exist on
                          its
                          > own. That way if there are 2 versions of something they will both be on
                          the
                          > list and you could tell them apart by selecting them and seeing the
                          source.
                          > I'm no programmer but I do see that this would be most difficult (if not
                          > impossible) to implement as it cuts deep into how the program functions. I
                          > put it out here because this would be ideal.
                          >
                          >
                          > This problem will only increase as the number of game books does the
                          > same. I know that some publishers have the bad habit of using the same
                          names
                          > for feats, classes etc. in their books as WotC and, in addition, sometimes
                          > reprint these classes and feats in multiple books, and we know what
                          happens
                          > when the same feat is loaded from 2 different sources.
                          >
                          > --
                          > Eddy
                          > PCGen Doc Gibbon
                          >
                          >
                          >
                          > PCGen's release site: http://pcgen.sourceforge.net
                          > PCGen's alpha build: http://rpg.plambert.net/pcgen
                          > PCGen's FAQ:
                          > http://rpg.plambert.net/pcgen/current/_docs/
                          > To unsubscribe from this group, send an email to:
                          > pcgen-unsubscribe@egroups.com
                          >
                          > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                          >
                          >
                        • Brass Tilde
                          ... Done correctly, there wouldn t need to be any of the relatively slow text parsing that currently happens when loading the LST files. Instead, something
                          Message 12 of 19 , Nov 3, 2003
                          • 0 Attachment
                            > > Another approach would be to "compile" the data files. That way, you
                            > > could parse and validate each file *once* after modifying it (not every
                            > > #$@! time you load it), and all such conflicts and inconsistencies
                            > > could be resolved. Every object could have an internal unique
                            > > identifier, assigned by the "compiler". As an extra benefit, PCGen
                            > > would run a *lot* faster.
                            >
                            > You are going to need to justify this statement. I do not see that there
                            > is any great advantage to loading the data and parsing the text and
                            > loading the data and parsing the code.

                            Done correctly, there wouldn't need to be any of the relatively slow text
                            parsing that currently happens when loading the LST files. Instead,
                            something much closer to the representation of the internal objects would be
                            saved in the "compiled" files. This should have the effect of speeding up
                            loading, though I agree that I can't see that it would make the rest of
                            PCGen run much faster.

                            > the major time constraint is searching for an existing instance of that
                            > data and updating the internal representation. Both of which yuo will
                            > still have to do.

                            No. The *result* of all the file loads, searches and updates is what is
                            compiled and saved. Thus the search and update process is bypassed when
                            loading the compiled versions.

                            I think in the past it was deemed not worth the effort, since loading is the
                            only thing that would be sped up.
                          • Michael Tucker
                            ... Was going to do this? Was that plan scrapped for some reason, or is that still in the works? ... *boggle* You don t see any performance or logic
                            Message 13 of 19 , Nov 3, 2003
                            • 0 Attachment
                              On Monday, November 3, 2003, at 09:58 AM, Frugal wrote:

                              > <quote who="Michael Tucker">
                              >> To my thinking, the only solution is to use some sort of unique
                              >> identifier on all objects ...
                              >>
                              > Funnily enough the XML data project was going to do exactly this, have
                              > a
                              > unique ID for each object (class, feat, skill etc) So duplicates would
                              > not
                              > be possible
                              >

                              "Was" going to do this? Was that plan scrapped for some reason, or is
                              that still in the works?

                              >> Another approach would be to "compile" the data files. That way, you
                              >> could parse and validate each file *once* after modifying it (not
                              >> every
                              >> #$@! time you load it), and all such conflicts and inconsistencies
                              >> could be resolved. Every object could have an internal unique
                              >> identifier, assigned by the "compiler". As an extra benefit, PCGen
                              >> would run a *lot* faster.
                              >
                              > You are going to need to justify this statement. I do not see that
                              > there
                              > is any great advantage to loading the data and parsing the text and
                              > loading the data and parsing the code.
                              >

                              *boggle* You don't see any performance or logic advantage to
                              pre-validating the data once, instead of every #$@! time you launch
                              PCGen and load the file? You don't see any advantage to loading
                              pre-compiled binary data, as opposed to text that has to be parsed and
                              checked for errors? You don't see any advantage to having to parse a
                              string, figure out what the heck kind of object that string is supposed
                              to be describing, check for any errors, and store the result; versus
                              reading a binary object that's already done all of that?

                              Wow. I don't know where to begin with "justifying that statement", if
                              that's the case. :-/

                              > the major time constraint is searching for an existing instance of that
                              > data and updating the internal representation. Both of which yuo will
                              > still have to do.
                              >
                              > --
                              > regards,
                              > Frugal
                              > -OS Chimp
                              >

                              Yes. But a "compiler" could pay that price, once and only once after
                              modifying a particular data file; as opposed to having PCGen do that
                              every #$@! time that every #$@! file is loaded, whether it's ever been
                              modified or not.

                              But, as I said, that opens a can of worms which departs radically from
                              the current thread (and isn't likely to be implemented in PCGen, given
                              the current direction of development). So, there's probably not much
                              point in discussing this "red herring" much, except to note it as an
                              alternative to consider which would solve the "duplicate sources"
                              problem.

                              Java Kensai
                            • Michael Tucker
                              p.s. Frugal, I apologize for the inflammatory attack style tone of my previous post. It wasn t intended to come across that way, but once I read it again I
                              Message 14 of 19 , Nov 3, 2003
                              • 0 Attachment
                                p.s. Frugal, I apologize for the inflammatory "attack style" tone of my
                                previous post. It wasn't intended to come across that way, but once I
                                read it again I realized that it did.

                                Java Kensai
                              • Michael Tucker
                                ... Which is exactly my point. If you typically launch PCGen once, load the files once, and leave it up and running on your PC for weeks at a time; then the
                                Message 15 of 19 , Nov 3, 2003
                                • 0 Attachment
                                  On Monday, November 3, 2003, at 10:07 AM, Brass Tilde wrote:
                                  > Done correctly, there wouldn't need to be any of the relatively slow
                                  > text
                                  > parsing that currently happens when loading the LST files. Instead,
                                  > something much closer to the representation of the internal objects
                                  > would be
                                  > saved in the "compiled" files. This should have the effect of
                                  > speeding up
                                  > loading, though I agree that I can't see that it would make the rest of
                                  > PCGen run much faster.
                                  >
                                  >> the major time constraint is searching for an existing instance of
                                  >> that
                                  >> data and updating the internal representation. Both of which yuo will
                                  >> still have to do.
                                  >
                                  > No. The *result* of all the file loads, searches and updates is what
                                  > is
                                  > compiled and saved. Thus the search and update process is bypassed
                                  > when
                                  > loading the compiled versions.
                                  >
                                  > I think in the past it was deemed not worth the effort, since loading
                                  > is the
                                  > only thing that would be sped up.
                                  >

                                  Which is exactly my point. If you typically launch PCGen once, load the
                                  files once, and leave it up and running on your PC for weeks at a time;
                                  then the time spent loading the files is inconsequential.

                                  I suspect that many (if not most) of the folks closest to the code use
                                  PCGen in that manner, and thus don't consider it to be a big deal,
                                  worth spending much effort on trying to fix.

                                  However, if you typically launch PCGen once every week or two, load the
                                  files, use it long enough to update a character or two, then exit; then
                                  the time spent loading the files is very significant.

                                  Running PCGen 5.1.6 on my Mac PowerBook (400 MHz G3, 512 MB RAM), it
                                  typically takes *several minutes* to load the basic data sets (core
                                  rules + splat books); *much* longer than it takes to actually use the
                                  program to do what I want.

                                  It's like if it took 5 minutes to start your car every time you wanted
                                  to drive it. If your typical trip is several hours long, then that's no
                                  big deal. However, if your typical trip is 10 minutes down the street
                                  to the store, then that startup delay is a painful and irritating
                                  experience, especially since you have to endure it every #$@! time.

                                  If those 5 minutes could be reduced significantly, I think that a lot
                                  of users would be very happy to see it. If the fix involved a
                                  pre-compiler that parsed and validated the data, thus eliminating any
                                  data-related problems in PCGen (along with a raft of on-the-fly
                                  error-checking code), then so much the better. :-)

                                  Java Kensai
                                • Frugal
                                  ... Well, the original plan was to move to XML for the data as soon as we hit 5.0... There was a flurry of activity in about April
                                  Message 16 of 19 , Nov 4, 2003
                                  • 0 Attachment
                                    <quote who="Michael Tucker">
                                    > On Monday, November 3, 2003, at 09:58 AM, Frugal wrote:
                                    >> Funnily enough the XML data project was going to do exactly this, have
                                    >> a
                                    >> unique ID for each object (class, feat, skill etc) So duplicates would
                                    >> not
                                    >> be possible
                                    >
                                    > "Was" going to do this? Was that plan scrapped for some reason, or is
                                    > that still in the works?

                                    Well, the original plan was to move to XML for the data as soon as we hit
                                    5.0... There was a flurry of activity in about April in pcgen-xml, but
                                    there has been nothing since then. the mailing list has about 5 emails a
                                    month and there has been no mention of implementing any of the
                                    specification.

                                    > *boggle* You don't see any performance or logic advantage to
                                    > pre-validating the data once, instead of every #$@! time you launch
                                    > PCGen and load the file? You don't see any advantage to loading
                                    > pre-compiled binary data, as opposed to text that has to be parsed and
                                    > checked for errors? You don't see any advantage to having to parse a
                                    > string, figure out what the heck kind of object that string is supposed
                                    > to be describing, check for any errors, and store the result; versus
                                    > reading a binary object that's already done all of that?
                                    >
                                    > Wow. I don't know where to begin with "justifying that statement", if
                                    > that's the case. :-/

                                    Okay, I guess I did not make myself clear about how I interpreted a
                                    "binary format".

                                    You have 2 real options.
                                    1 - serialize the data to disk
                                    2 - create a custom binary format

                                    Problems with 1: As the data can be loaded in from different files you
                                    will only be able to serailize the data after you have loaded _all_ of the
                                    LST files that go to make up your campaign. If you do not then you will
                                    still have to merge the serialized objects together to make the actual
                                    object. If the data is serialized after it is loaded in then you will have
                                    to have a separate set of compiled files for each different combination of
                                    source files loaded.

                                    Problems with 2: You have not gained anything other than reading binary as
                                    opposed to reading text, you still have to do the same level of validation
                                    (if not more so). If you create a binary format of each LST file then you
                                    have simply changed the encoding on disk. If you create a dump of the
                                    loaded sources then you have the same problems as with (1) with regard to
                                    the binary dump only being valid for a particular combination of files.

                                    Compilers work really well because each file is self contained. Take the
                                    example of Java, each java source file is self contained and complete. Yes
                                    they do 'import' data from other files, but they import complete objects.
                                    PCGen has a much more complicated problem because we can modify what is
                                    read in one file with the contents of another. In the java example it
                                    would be like defining a method in one file and compiling it, then
                                    compiling another file that altered the internals of the method. To do
                                    this quickly, safely and accurately is extremely difficult.

                                    > Yes. But a "compiler" could pay that price, once and only once after
                                    > modifying a particular data file; as opposed to having PCGen do that
                                    > every #$@! time that every #$@! file is loaded, whether it's ever been
                                    > modified or not.

                                    It depends entirely on wether your compiler is compiling a single LST
                                    file, or a complete set of LST files that represent a certain
                                    configuration that the use is currently using...

                                    Also given the vastly different technicaly ability levels of the PCGen
                                    userbase I would be very reluctant to add an extra point of failure. How
                                    will we know wether a user has a problem because the LST syntax was wrong,
                                    or because the compiler made an error, or because they are using a
                                    different set of compiled files than they thought, or because they forgot
                                    to compile a file they had changed...

                                    For a project of this size I think that the complexity is too high for the
                                    reward returned (200+ man hours to gain a data load time decrease from 8
                                    to 4 seconds (roughly)). I would much rather see that time spent
                                    refactoring some of the 12,000 line class files ;O)

                                    --
                                    regards,
                                    Frugal
                                    -OS Chimp
                                  • Frugal
                                    ... Strange, it did not come across as inflamatory to me. I guess because I realised that I had not stated where my opinion was
                                    Message 17 of 19 , Nov 4, 2003
                                    • 0 Attachment
                                      <quote who="Michael Tucker">
                                      > p.s. Frugal, I apologize for the inflammatory "attack style" tone of my
                                      > previous post. It wasn't intended to come across that way, but once I
                                      > read it again I realized that it did.

                                      Strange, it did not come across as inflamatory to me. I guess because I
                                      realised that I had not stated where my opinion was coming from. I have
                                      not done much practical compiler design work, but I have spent a large
                                      lump of time reading and writing binary formats ;O)

                                      --
                                      regards,
                                      Frugal
                                      -OS Chimp
                                    • Keith Davies
                                      ... I ll cop some of the blame for that. A hellish work schedule combined with not actually having time to play the game has driven the priority down a bit.
                                      Message 18 of 19 , Nov 4, 2003
                                      • 0 Attachment
                                        On Tue, Nov 04, 2003 at 09:10:58AM +0000, Frugal wrote:
                                        >
                                        > <quote who="Michael Tucker">
                                        > > On Monday, November 3, 2003, at 09:58 AM, Frugal wrote:
                                        > >> Funnily enough the XML data project was going to do exactly this, have
                                        > >> a
                                        > >> unique ID for each object (class, feat, skill etc) So duplicates would
                                        > >> not
                                        > >> be possible
                                        > >
                                        > > "Was" going to do this? Was that plan scrapped for some reason, or is
                                        > > that still in the works?
                                        >
                                        > Well, the original plan was to move to XML for the data as soon as we
                                        > hit 5.0... There was a flurry of activity in about April in pcgen-xml,
                                        > but there has been nothing since then. the mailing list has about 5
                                        > emails a month and there has been no mention of implementing any of
                                        > the specification.

                                        I'll cop some of the blame for that. A hellish work schedule combined
                                        with not actually having time to play the game has driven the priority
                                        down a bit. The last version of PCGen I have installed on my
                                        workstation is 4.3.3. Add in an almost complete lack of input from
                                        anyone -- even after posting documents for review -- I have little
                                        incentive to work on this.

                                        So, while the XML project doesn't have much of a leader right now, it
                                        also doesn't much to lead. I got tired of carrying the whole damn thing
                                        on my own. I'd love to see it done, but right now 'what PCGen needs'
                                        and 'what Keith wants' are two different things.

                                        > Problems with 2: You have not gained anything other than reading binary as
                                        > opposed to reading text, you still have to do the same level of validation
                                        > (if not more so). If you create a binary format of each LST file then you
                                        > have simply changed the encoding on disk.

                                        More or less true, though a *compiled* version (regardless of format)
                                        can be markedly more reliable because the hard work of validation will
                                        already have been done. Making such files use an obfuscated binary
                                        format makes it harder for someone to go in manually and mess with it
                                        (good for keeping users out, bad for maintaining data). Overall,
                                        probably not really worth it. I probably wouldn't bother.

                                        OTOH, a compiled 'campaign file' would be very useful. Take 'keith.pcc'
                                        and compile it, building a single file containing everything that would
                                        load when 'keith.pcc' is loaded. Locks down the versions of all
                                        subsidiary files and gives me a single file to move around and give to
                                        my players. Different versions of PCGen distribution -- assuming no
                                        changes to data format -- wouldn't break things when object definitions
                                        in the distro get changed. This would probably be worthwhile. Also, it
                                        would make a useful debugging tool because it's then possible to go
                                        through the data as seen by the program and figure out exactly what's
                                        there.

                                        > Compilers work really well because each file is self contained. Take the
                                        > example of Java, each java source file is self contained and complete. Yes
                                        > they do 'import' data from other files, but they import complete objects.
                                        > PCGen has a much more complicated problem because we can modify what is
                                        > read in one file with the contents of another. In the java example it
                                        > would be like defining a method in one file and compiling it, then
                                        > compiling another file that altered the internals of the method. To do
                                        > this quickly, safely and accurately is extremely difficult.

                                        This makes it a pain in the ass. From what I can see, the files being
                                        loaded are less 'data files' than 'transaction files' (or 'data files
                                        containing transactions', if that makes it easier). The default
                                        transaction is to add the object defined. Other transactions including
                                        modifying an object, copying an object modifying the copy, dropping an
                                        object, and so on. This realization made working on the XML much
                                        easier.


                                        Keith
                                        --
                                        Keith Davies "Your ability to bang your head against
                                        keith.davies@... reality in the hope that reality will
                                        crack first is impressive, but futile"
                                        -- Geoffrey Brent, rec.games.frp.dnd
                                      Your message has been successfully submitted and would be delivered to recipients shortly.