Loading ...
Sorry, an error occurred while loading the content.
 

[Frugal] Converting the PCGen Data Files (WAS Re: (unknown))

Expand Messages
  • andargor
    ... handling the ... Yes, it s only a demo as I m experimenting with the various options Frugal described for the logic and rules portion of the problem. I
    Message 1 of 28 , Jul 4 4:33 PM
      --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
      >
      > <quote who="David Finch">
      > > Frugal wrote:
      > >> Have a look at the work andargor had been doing:
      > >> http://www.andargor.com/files/panther-demo.zip
      > >
      > > Very intresting, did this go any further or was it just a tech demo?
      >
      > It is curently a tech demo in that this is as far as he has developed it
      > over a couple of weeks.
      >
      > Like all of the tech demos there is a huge difference between
      handling the
      > basics and handling the more complicated stuff ( I know that I keep
      > hitting problems doing some of the more esoteric things).
      >
      > --
      > regards,
      > Frugal
      > -OS Chimp

      Yes, it's only a demo as I'm experimenting with the various options
      Frugal described for the "logic" and "rules" portion of the problem.

      I tend to favor "option 3":

      - use a neutral version of the base data which isn't egine specific
      (enter data once, use many times)
      - Transform data into an engine specific format (Frugal's, mine,
      Engine XYZ) using XSLT
      - Have scriptlets that hook in to provide the rules aspect (I use the
      term "scriptlets" to denote small bits of code in the engine's
      preferred language: javascript, python, perl, whatever)

      The reason that I like this approach is that you don't have to agree
      on a very specific format for the neutral base data. It can be very
      loose, like I described. As well, option 2 is still too close to
      engine specific code, since everyone would have to agree on a language
      and an API.

      On the maintenance side, as publisher data is released you don't need
      developer-grade people to code in loose XML. There are a lot of people
      out there that are willing to help but that don't know LST,
      javascript, etc. So their task would be to enter the base data using
      guidelines. Sure a schema or DTD could be used, but we would have to
      agree on it, which is no mean task.

      The rules or logic portion would be then the responsibility of the
      developer-grade people for each engine. You distribute the workload,
      and you remove the engine dependency from the base data if an engine
      gets updated.

      My general philosophy is that we should minimize the requirements for
      standards, since that is what has stopped XML projects in the past.
      Everyone has his/her own view of how the XML data should be
      structured, and arguments. Let's just get something that can be
      machine readable and reused out there. The "small steps" theory.

      Frugal, is it possible for you to do a "neutral" dump as I described
      if it's not too much trouble? It would allow me to demonstrate what I
      mean. I could even craft an XSLT that would format the neutral data
      into one compatible with your engine (as much as possible). It would
      be much appreciated :)

      If you don't have the time, could you upload your LST parser so I can
      fiddle with it?

      Andargor
    • Paul M. Lambert
      ... I ve been reading this discussion and I d like to make sure I understand what you mean, andargor. Tell me what below matches your thoughts and what
      Message 2 of 28 , Jul 4 6:07 PM
        On Sun, 04 Jul 2004, andargor wrote:

        > Nice wrench. :)
        >
        > I don't want ot discuss semantics, but the example I gave is not
        > specific to any engine. For example, I do not plan to use the
        > fighter_bonus tag in my program and I doubt Frugal or PCGen will ever
        > use it.
        >
        > The point is to use XML for what it is intended for: tagging data,
        > nothing more, nothing less. It is a machine readable version of
        > publisher data, that's all.
        >
        > fighter_bonus gets transformed into engine specific tags:
        > <bonus><type>Fighter</type></bonus> or <bonus id="Alterness"
        > type="fighter.bonus"> or even TYPE:General.Fighter if you want to
        > regenerate LST. You use XSLT to do that.
        >
        > The marvelous thing is that you only coded the publisher data once.
        > All you have to do is maintain a transform (XSLT). Ain't that sweet?
        >
        > Andargor

        I've been reading this discussion and I'd like to make sure I understand
        what you mean, andargor.

        Tell me what below matches your thoughts and what doesn't, so we can get
        on the same page. Then I'll be able to tell if I actually agree with
        you. :-)

        As I see it your "data neutral" format idea could be re-expressed as a
        high level pseudo language. This language could then be converted into
        a given engine's native format (LST, PCGEN-XML--whatever that is,
        "plambertgen", whatever) if someone writes such a converter. And then the
        engine can handle that data as it sees fit.

        I see this to mean that the data neutral format would allow tag
        (re-)definitions, so, for example, <figher_bonus> could be a tag
        used in one data file, and it might be definied in coarser game
        terms in another. This would allow "libraries" of definitions for
        a given game system or rule set that would allow easy mapping to
        specific engines in the converters.

        I'm trying to think of a concrete example. Perhaps something like:

        <str>7</str>

        in a character definition for a 3.5 SRD character. And in the
        library data file for 3.5 SRD, you might have:

        <character>
        ...
        <attribute>
        <name>Strength</name>
        <abbr>STR</abbr>
        <tag>str</tag>
        <description>Physical strength and prowess</description>
        ...
        </attribute>
        ...
        </character>

        These are extremely rough non-scaling examples, of course, and not
        necessarily how I actually suggest doing attributes, but anyway...

        A converter then might take that information and turn it into LST data.
        The converter would know the limits of its target format. For example,
        if it's not possible to actually change/rename/add attributes in LST
        format, it would basically know to ignore the attribute definitions and
        use the ones inherent to the engine. If another were used, it could
        generate a warning or error.

        A converter might even have full internal knowledge of a given "library"
        as well, if it was a good choice for the coder/engine/etc. For example,
        an engine extremely specific to a certain game rule set (maybe an SRD3.5
        only web character repository, or whatever) could ignore the library
        entirely, since the author would know what was and wasn't defined in
        there.

        This would mean the libraries and the converters could be maintained by
        people with strong such skills, while the casual user who just wants to
        say "I'd like to add a Bow of Tumultuous Angst to my equipment lists;
        it's a Long Bow that gives +2/+4 only to Chaotic Neutral wielders."
        could do it with dirt-simple, straightforward tags. Still valid XML, but
        really simple tags whose inner workings are defined elsewhere, out of
        sight.

        However, if they want it to give the wielder +1 to hit for each pound of
        bat guano they're carrying, they'll need to dig deeper. But simple
        things will be easy and complex things will be possible. Right?

        Please let me know where I misunderstand.

        --plambert
      • andargor
        Answers inline. ... (snip) ... then the ... You sure think ahead :) Actually, what I propose is more basic: just convert publisher data into a machine readable
        Message 3 of 28 , Jul 4 6:49 PM
          Answers inline.

          --- In pcgen-xml@yahoogroups.com, "Paul M. Lambert" <plambert@p...> wrote:
          (snip)
          > As I see it your "data neutral" format idea could be re-expressed as a
          > high level pseudo language. This language could then be converted into
          > a given engine's native format (LST, PCGEN-XML--whatever that is,
          > "plambertgen", whatever) if someone writes such a converter. And
          then the
          > engine can handle that data as it sees fit.
          >
          > I see this to mean that the data neutral format would allow tag
          > (re-)definitions, so, for example, <figher_bonus> could be a tag
          > used in one data file, and it might be definied in coarser game
          > terms in another. This would allow "libraries" of definitions for
          > a given game system or rule set that would allow easy mapping to
          > specific engines in the converters.


          You sure think ahead :)

          Actually, what I propose is more basic: just convert publisher data
          into a machine readable format with a minimum of context (i.e. XML
          tags) so that it can be manipulated. Your library concept could then
          be implemented, if you wish to do so. First things first :)

          (snip)
          > This would mean the libraries and the converters could be maintained by
          > people with strong such skills, while the casual user who just wants to
          > say "I'd like to add a Bow of Tumultuous Angst to my equipment lists;
          > it's a Long Bow that gives +2/+4 only to Chaotic Neutral wielders."
          > could do it with dirt-simple, straightforward tags. Still valid
          XML, but
          > really simple tags whose inner workings are defined elsewhere, out of
          > sight.


          Yes, the general idea is to leverage the large quantity of people that
          have the willingess to help but not necessarily the coding skills.
          They could describe that bow very simply, and fill in the logic
          portion as a description. Someone with coding skills could then craft
          an engine specific XSLT and translate the logic into engine-specific
          "scriptlets" for a particular tool.

          Andargor
        • Frugal
          ... The LST parser is already up on the groups file section: http://groups.yahoo.com/group/pcgen-xml/files/sample_source-20031216.tgz As
          Message 4 of 28 , Jul 5 1:08 AM
            <quote who="andargor">

            > Frugal, is it possible for you to do a "neutral" dump as I described
            > if it's not too much trouble? It would allow me to demonstrate what I
            > mean. I could even craft an XSLT that would format the neutral data
            > into one compatible with your engine (as much as possible). It would
            > be much appreciated :)
            >
            > If you don't have the time, could you upload your LST parser so I can
            > fiddle with it?

            The LST parser is already up on the groups file section:

            http://groups.yahoo.com/group/pcgen-xml/files/sample_source-20031216.tgz

            As the date indicates this has not been worked on since the middle of
            december. Be warned: "here be kludges".

            Feel free to do with it as you will, There should be a GPL header at the
            top of each file ;O)

            --
            regards,
            Frugal
            -OS Chimp
          • David Finch
            ... A first good step would be that the data had been input. If the logic has been typed in english then at least when the conversion to PCgen format is done
            Message 5 of 28 , Jul 5 3:03 AM
              andargor wrote:
              > Yes, the general idea is to leverage the large quantity of people that
              > have the willingess to help but not necessarily the coding skills.
              > They could describe that bow very simply, and fill in the logic
              > portion as a description. Someone with coding skills could then craft
              > an engine specific XSLT and translate the logic into engine-specific
              > "scriptlets" for a particular tool.

              A first good step would be that the data had been input. If the logic has
              been typed in english then at least when the conversion to PCgen format is
              done someone can go though at 'specialise it' to PCgen format. This has the
              added advatage that someone can check that the english logic matches the
              real programming logic.
            • andargor
              ... 20031216.tgz ... of ... at the ... Yikes! There be kludges :) Well, I had to craft an Ant build.xml, and it seems to compile properly... What version of
              Message 6 of 28 , Jul 6 5:36 AM
                --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                >
                > The LST parser is already up on the groups file section:
                >
                > http://groups.yahoo.com/group/pcgen-xml/files/sample_source-
                20031216.tgz
                >
                > As the date indicates this has not been worked on since the middle
                of
                > december. Be warned: "here be kludges".
                >
                > Feel free to do with it as you will, There should be a GPL header
                at the
                > top of each file ;O)
                >
                > --
                > regards,
                > Frugal
                > -OS Chimp

                Yikes! There be kludges :)

                Well, I had to craft an Ant build.xml, and it seems to compile
                properly...

                What version of the data files were you using? The parser is croaking
                on several of the 5.7.2 ones.

                Andargor
              • Frugal
                ... Oh, yeah, oops. I was using Eclipse and it managed all of the building for me ;O) ... I was using the CVS data as of the date of the
                Message 7 of 28 , Jul 6 5:43 AM
                  <quote who="andargor">
                  > --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                  >>
                  >> The LST parser is already up on the groups file section:
                  >>
                  >> http://groups.yahoo.com/group/pcgen-xml/files/sample_source-
                  > 20031216.tgz
                  >>
                  >> As the date indicates this has not been worked on since the middle
                  > of
                  >> december. Be warned: "here be kludges".
                  >>
                  >> Feel free to do with it as you will, There should be a GPL header
                  > at the
                  >> top of each file ;O)
                  >
                  > Yikes! There be kludges :)
                  >
                  > Well, I had to craft an Ant build.xml, and it seems to compile
                  > properly...

                  Oh, yeah, oops. I was using Eclipse and it managed all of the building for
                  me ;O)

                  > What version of the data files were you using? The parser is croaking
                  > on several of the 5.7.2 ones.

                  I was using the CVS data as of the date of the archive. So any new tags
                  will fail.

                  IT would be so nice if every time someone messed around with a data tag
                  they were forced to increment the LST version number. It would make data
                  management a lot easier if you could state which version of LST syntax
                  your LST files were.

                  --
                  regards,
                  Frugal
                  -OS Chimp
                • andargor
                  ... tag ... data ... syntax ... Or if the data were in XML and the rule logic described somewhere for PCGen, the LST could be regenerated with each new version
                  Message 8 of 28 , Jul 6 10:09 AM
                    --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                    >
                    > IT would be so nice if every time someone messed around with a data
                    tag
                    > they were forced to increment the LST version number. It would make
                    data
                    > management a lot easier if you could state which version of LST
                    syntax
                    > your LST files were.
                    >
                    > --
                    > regards,
                    > Frugal
                    > -OS Chimp

                    Or if the data were in XML and the rule logic described somewhere for
                    PCGen, the LST could be regenerated with each new version ;)

                    Andargor
                  • Mark Coletti
                    On Thu, 1 Jul 2004 08:55:51 +0100 (BST), Frugal ... It was put the rules in the data that did it for me. What you re talking about is really a knowledge
                    Message 9 of 28 , Jul 14 12:56 PM
                      On Thu, 1 Jul 2004 08:55:51 +0100 (BST), Frugal
                      <frugal@...> wrote:
                      >
                      > <quote who="Tir Gwaith">
                      > > There is no "neutral" version, just the perception of one. [...] there won't
                      > > ever be a stable version, because new rules will
                      > > always be written.

                      > I think that a 'neutral' version of the data can be farily easily created.
                      > The problem the D20 is not the data, but the manipulation of the data.
                      > Every time a company brings out a new rule the way the data is to be
                      > manipulated needs to be changed. As you stated the problem is not the
                      > data, but the extra rules that are always added.

                      > As I see it encoding the rules can be done in 3 ways:
                      > - The PCGen way: every thing is hard coded into the program, new rules
                      > mean new code
                      > - Put the rules in the data: As well as encoding the data, also encode a
                      > scripting language to manipulate the data.
                      > - Plugins: New rules get encoded as plugins that are loaded into the
                      > program as needed. So a dataset would contain data and a list of plugins
                      > that are required to manipulate the data.

                      It was "put the rules in the data" that did it for me. What you're
                      talking about is really a knowledge base; which makes sense in that
                      PCGen *is* essentially an expert system that relies on a
                      knowledge-base containing rules for various d20 based gaming systems.
                      The problem is that most of the knowledge is either hard-coded in Java
                      or in LST files, and most of the "inferencing" for the huge cloud of
                      rules is also hard-coded. (Think IF-THEN and SWITCH statements as a
                      crude sort of linear, data-driven 'inferencing'.)

                      JESS is a free, open-source, expert system implemented in Java. I can
                      easily see where most of the PCGen "knowledge-base" currently spread
                      among Java and LST files could be rolled into a knowledge-base
                      containing explicit rules. Specific systems would just add new
                      knowledge-bases that contain fresh sets of rules (including some that
                      may remove or modify existing ones).

                      A JESS (or similar system) based PCGen system would be inherently
                      smaller since the brunt of the inferencing will be left to the
                      inference engine, and the rules would be explicitly codified as, well,
                      rules. You could also set up the knowledge-base such that you can
                      generate explanations. One thing about the d20 systems is that
                      they're very complex once you consider all the different rules and
                      their respective interactions. If, say, a character has an 18 AC, it
                      might be nice to know *how* and *why* it's currently 18. All the
                      rules that trigger AC mods could contribute to explanatory text. "AC
                      base 10, +2 for Dexterity, +3 for ring of nimbleness, +3 to <mumble>
                      spell effect."

                      Of course this is all pie-in-the-sky ramblings implementing JESS in
                      PCGen would entail quite a bit of work. (Um, I think.) But, what the
                      hell, it makes for a pleasant gedankenexperiment. %-)

                      More info:

                      http://web.njit.edu/all_topics/Prog_Lang_Docs/html/jess51/

                      Cheers!

                      MAC
                      --
                      I'm taking reality in small doses to build immunity.
                    Your message has been successfully submitted and would be delivered to recipients shortly.