Re: [pcgen] Re: Duplicate bonuses from duplicate sources
> 1) Have the Source Material Selected panel show all the sources.would
> Since a .pcc file can load other .pcc files from any other gamemode it
> be good if they show up in this panel even if they don't appear in theI like this one. Maybe have the loaded panel be a tree like the left pane,
> Sources available panel.
with the sources loaded by the "main" entry be child nodes.
> 2) Some sort of warning when duplicate objects are loaded. Now I would notYes, one dialog at the end of source loading saying something like "The
> want a window to pop up every time this happened (but that might be a good
> idea) but something like the console which could give me a list of these
> things would be nice.
following items are duplicated" with a list of items and their source.
> 3) Do not allow duplicate objects. PCGen loads sources by rank right? What
> if you make it so that if it runs into a duplicate object in another .pcc
> file it just ignores it and it never loads. I know this may cause problems
> for the source that has an object not loaded especially considering that
> now my have other objects referencing something for a completely differentThe problem with this one is that the item, even though duplicately named,
> source, but is this problem greater than the existing situation.
may not actually be duplicated. It might actually be a different item,
however slightly different. I believe the general practice in the past has
been to use the publisher's name in the item to make sure they were unique,
with the first to be published getting the "honor" of having the plain name.
This would imply that either the SRD item names would have to be appended
with "(SRD)" or the WotC files created by CMP would need their names
> 4) Allow objects from each dataset (.pcc file) to be unique and exist onits
> own. That way if there are 2 versions of something they will both be onthe
> list and you could tell them apart by selecting them and seeing thesource.
> I'm no programmer but I do see that this would be most difficult (if notIt would require that the program add a unique key to each loaded item *and*
> impossible) to implement as it cuts deep into how the program functions. I
> put it out here because this would be ideal.
be able to properly associate that item with all the other items which refer
to it by name. So the program would be required to know that they "Monk" to
which the "TYPE:Monk" in the Kama entry refers to the WotC Monk, and not the
Billy Joe Bob's campaign Monk.
Adding a KEY tag and demanding that it be unique (with the aforementioned
denial of loading for duplicate items), and including it at LST design time
rather than program run time, would actually be better. Though, frankly, it
would likely be time consuming as well.
> > Another approach would be to "compile" the data files. That way, youDone correctly, there wouldn't need to be any of the relatively slow text
> > could parse and validate each file *once* after modifying it (not every
> > #$@! time you load it), and all such conflicts and inconsistencies
> > could be resolved. Every object could have an internal unique
> > identifier, assigned by the "compiler". As an extra benefit, PCGen
> > would run a *lot* faster.
> You are going to need to justify this statement. I do not see that there
> is any great advantage to loading the data and parsing the text and
> loading the data and parsing the code.
parsing that currently happens when loading the LST files. Instead,
something much closer to the representation of the internal objects would be
saved in the "compiled" files. This should have the effect of speeding up
loading, though I agree that I can't see that it would make the rest of
PCGen run much faster.
> the major time constraint is searching for an existing instance of thatNo. The *result* of all the file loads, searches and updates is what is
> data and updating the internal representation. Both of which yuo will
> still have to do.
compiled and saved. Thus the search and update process is bypassed when
loading the compiled versions.
I think in the past it was deemed not worth the effort, since loading is the
only thing that would be sped up.
- On Monday, November 3, 2003, at 10:07 AM, Brass Tilde wrote:
> Done correctly, there wouldn't need to be any of the relatively slowWhich is exactly my point. If you typically launch PCGen once, load the
> parsing that currently happens when loading the LST files. Instead,
> something much closer to the representation of the internal objects
> would be
> saved in the "compiled" files. This should have the effect of
> speeding up
> loading, though I agree that I can't see that it would make the rest of
> PCGen run much faster.
>> the major time constraint is searching for an existing instance of
>> data and updating the internal representation. Both of which yuo will
>> still have to do.
> No. The *result* of all the file loads, searches and updates is what
> compiled and saved. Thus the search and update process is bypassed
> loading the compiled versions.
> I think in the past it was deemed not worth the effort, since loading
> is the
> only thing that would be sped up.
files once, and leave it up and running on your PC for weeks at a time;
then the time spent loading the files is inconsequential.
I suspect that many (if not most) of the folks closest to the code use
PCGen in that manner, and thus don't consider it to be a big deal,
worth spending much effort on trying to fix.
However, if you typically launch PCGen once every week or two, load the
files, use it long enough to update a character or two, then exit; then
the time spent loading the files is very significant.
Running PCGen 5.1.6 on my Mac PowerBook (400 MHz G3, 512 MB RAM), it
typically takes *several minutes* to load the basic data sets (core
rules + splat books); *much* longer than it takes to actually use the
program to do what I want.
It's like if it took 5 minutes to start your car every time you wanted
to drive it. If your typical trip is several hours long, then that's no
big deal. However, if your typical trip is 10 minutes down the street
to the store, then that startup delay is a painful and irritating
experience, especially since you have to endure it every #$@! time.
If those 5 minutes could be reduced significantly, I think that a lot
of users would be very happy to see it. If the fix involved a
pre-compiler that parsed and validated the data, thus eliminating any
data-related problems in PCGen (along with a raft of on-the-fly
error-checking code), then so much the better. :-)
- <quote who="Michael Tucker">
> On Monday, November 3, 2003, at 09:58 AM, Frugal wrote:Well, the original plan was to move to XML for the data as soon as we hit
>> Funnily enough the XML data project was going to do exactly this, have
>> unique ID for each object (class, feat, skill etc) So duplicates would
>> be possible
> "Was" going to do this? Was that plan scrapped for some reason, or is
> that still in the works?
5.0... There was a flurry of activity in about April in pcgen-xml, but
there has been nothing since then. the mailing list has about 5 emails a
month and there has been no mention of implementing any of the
> *boggle* You don't see any performance or logic advantage toOkay, I guess I did not make myself clear about how I interpreted a
> pre-validating the data once, instead of every #$@! time you launch
> PCGen and load the file? You don't see any advantage to loading
> pre-compiled binary data, as opposed to text that has to be parsed and
> checked for errors? You don't see any advantage to having to parse a
> string, figure out what the heck kind of object that string is supposed
> to be describing, check for any errors, and store the result; versus
> reading a binary object that's already done all of that?
> Wow. I don't know where to begin with "justifying that statement", if
> that's the case. :-/
You have 2 real options.
1 - serialize the data to disk
2 - create a custom binary format
Problems with 1: As the data can be loaded in from different files you
will only be able to serailize the data after you have loaded _all_ of the
LST files that go to make up your campaign. If you do not then you will
still have to merge the serialized objects together to make the actual
object. If the data is serialized after it is loaded in then you will have
to have a separate set of compiled files for each different combination of
source files loaded.
Problems with 2: You have not gained anything other than reading binary as
opposed to reading text, you still have to do the same level of validation
(if not more so). If you create a binary format of each LST file then you
have simply changed the encoding on disk. If you create a dump of the
loaded sources then you have the same problems as with (1) with regard to
the binary dump only being valid for a particular combination of files.
Compilers work really well because each file is self contained. Take the
example of Java, each java source file is self contained and complete. Yes
they do 'import' data from other files, but they import complete objects.
PCGen has a much more complicated problem because we can modify what is
read in one file with the contents of another. In the java example it
would be like defining a method in one file and compiling it, then
compiling another file that altered the internals of the method. To do
this quickly, safely and accurately is extremely difficult.
> Yes. But a "compiler" could pay that price, once and only once afterIt depends entirely on wether your compiler is compiling a single LST
> modifying a particular data file; as opposed to having PCGen do that
> every #$@! time that every #$@! file is loaded, whether it's ever been
> modified or not.
file, or a complete set of LST files that represent a certain
configuration that the use is currently using...
Also given the vastly different technicaly ability levels of the PCGen
userbase I would be very reluctant to add an extra point of failure. How
will we know wether a user has a problem because the LST syntax was wrong,
or because the compiler made an error, or because they are using a
different set of compiled files than they thought, or because they forgot
to compile a file they had changed...
For a project of this size I think that the complexity is too high for the
reward returned (200+ man hours to gain a data load time decrease from 8
to 4 seconds (roughly)). I would much rather see that time spent
refactoring some of the 12,000 line class files ;O)
- <quote who="Michael Tucker">
> p.s. Frugal, I apologize for the inflammatory "attack style" tone of myStrange, it did not come across as inflamatory to me. I guess because I
> previous post. It wasn't intended to come across that way, but once I
> read it again I realized that it did.
realised that I had not stated where my opinion was coming from. I have
not done much practical compiler design work, but I have spent a large
lump of time reading and writing binary formats ;O)
- On Tue, Nov 04, 2003 at 09:10:58AM +0000, Frugal wrote:
>I'll cop some of the blame for that. A hellish work schedule combined
> <quote who="Michael Tucker">
> > On Monday, November 3, 2003, at 09:58 AM, Frugal wrote:
> >> Funnily enough the XML data project was going to do exactly this, have
> >> a
> >> unique ID for each object (class, feat, skill etc) So duplicates would
> >> not
> >> be possible
> > "Was" going to do this? Was that plan scrapped for some reason, or is
> > that still in the works?
> Well, the original plan was to move to XML for the data as soon as we
> hit 5.0... There was a flurry of activity in about April in pcgen-xml,
> but there has been nothing since then. the mailing list has about 5
> emails a month and there has been no mention of implementing any of
> the specification.
with not actually having time to play the game has driven the priority
down a bit. The last version of PCGen I have installed on my
workstation is 4.3.3. Add in an almost complete lack of input from
anyone -- even after posting documents for review -- I have little
incentive to work on this.
So, while the XML project doesn't have much of a leader right now, it
also doesn't much to lead. I got tired of carrying the whole damn thing
on my own. I'd love to see it done, but right now 'what PCGen needs'
and 'what Keith wants' are two different things.
> Problems with 2: You have not gained anything other than reading binary asMore or less true, though a *compiled* version (regardless of format)
> opposed to reading text, you still have to do the same level of validation
> (if not more so). If you create a binary format of each LST file then you
> have simply changed the encoding on disk.
can be markedly more reliable because the hard work of validation will
already have been done. Making such files use an obfuscated binary
format makes it harder for someone to go in manually and mess with it
(good for keeping users out, bad for maintaining data). Overall,
probably not really worth it. I probably wouldn't bother.
OTOH, a compiled 'campaign file' would be very useful. Take 'keith.pcc'
and compile it, building a single file containing everything that would
load when 'keith.pcc' is loaded. Locks down the versions of all
subsidiary files and gives me a single file to move around and give to
my players. Different versions of PCGen distribution -- assuming no
changes to data format -- wouldn't break things when object definitions
in the distro get changed. This would probably be worthwhile. Also, it
would make a useful debugging tool because it's then possible to go
through the data as seen by the program and figure out exactly what's
> Compilers work really well because each file is self contained. Take theThis makes it a pain in the ass. From what I can see, the files being
> example of Java, each java source file is self contained and complete. Yes
> they do 'import' data from other files, but they import complete objects.
> PCGen has a much more complicated problem because we can modify what is
> read in one file with the contents of another. In the java example it
> would be like defining a method in one file and compiling it, then
> compiling another file that altered the internals of the method. To do
> this quickly, safely and accurately is extremely difficult.
loaded are less 'data files' than 'transaction files' (or 'data files
containing transactions', if that makes it easier). The default
transaction is to add the object defined. Other transactions including
modifying an object, copying an object modifying the copy, dropping an
object, and so on. This realization made working on the XML much
Keith Davies "Your ability to bang your head against
keith.davies@... reality in the hope that reality will
crack first is impressive, but futile"
-- Geoffrey Brent, rec.games.frp.dnd