Loading ...
Sorry, an error occurred while loading the content.

[Frugal] Converting the PCGen Data Files (WAS Re: (unknown))

Expand Messages
  • andargor
    ... non-engine specific format, truely.... ... No in and of itself is engine specific... Types of feats that are choosable as
    Message 1 of 28 , Jul 4, 2004
    • 0 Attachment
      --- In pcgen-xml@yahoogroups.com, "Tir Gwaith"
      <thoron-tir-gwaith@l...> wrote:
      > Just to throw a wrench in, because I think there will NEVER be an
      'non-engine specific' format, truely....
      >
      > Your example IS engine specific....
      <fighter_bonus>No</fighter_bonus> in and of itself is engine
      specific... Types of feats that are choosable as class adjuncts are
      many, and not just "Fighter."
      >
      > There is no "neutral" version, just the perception of one. As a
      data monkey, I've learned to challenge what my assumptions are, since
      publishers will always find new and interesting ways of looking at the
      game. And there won't ever be a stable version, because new rules
      will always be written.
      >
      > I don't see the point in striving to make things separate from the
      interpretor. To me that is like designing a language without taking
      the listener into account.
      >
      > Tir Gwaith
      > LST Chimp

      Nice wrench. :)

      I don't want ot discuss semantics, but the example I gave is not
      specific to any engine. For example, I do not plan to use the
      fighter_bonus tag in my program and I doubt Frugal or PCGen will ever
      use it.

      The point is to use XML for what it is intended for: tagging data,
      nothing more, nothing less. It is a machine readable version of
      publisher data, that's all.

      fighter_bonus gets transformed into engine specific tags:
      <bonus><type>Fighter</type></bonus> or <bonus id="Alterness"
      type="fighter.bonus"> or even TYPE:General.Fighter if you want to
      regenerate LST. You use XSLT to do that.

      The marvelous thing is that you only coded the publisher data once.
      All you have to do is maintain a transform (XSLT). Ain't that sweet?

      Andargor
    • andargor
      ... handling the ... Yes, it s only a demo as I m experimenting with the various options Frugal described for the logic and rules portion of the problem. I
      Message 2 of 28 , Jul 4, 2004
      • 0 Attachment
        --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
        >
        > <quote who="David Finch">
        > > Frugal wrote:
        > >> Have a look at the work andargor had been doing:
        > >> http://www.andargor.com/files/panther-demo.zip
        > >
        > > Very intresting, did this go any further or was it just a tech demo?
        >
        > It is curently a tech demo in that this is as far as he has developed it
        > over a couple of weeks.
        >
        > Like all of the tech demos there is a huge difference between
        handling the
        > basics and handling the more complicated stuff ( I know that I keep
        > hitting problems doing some of the more esoteric things).
        >
        > --
        > regards,
        > Frugal
        > -OS Chimp

        Yes, it's only a demo as I'm experimenting with the various options
        Frugal described for the "logic" and "rules" portion of the problem.

        I tend to favor "option 3":

        - use a neutral version of the base data which isn't egine specific
        (enter data once, use many times)
        - Transform data into an engine specific format (Frugal's, mine,
        Engine XYZ) using XSLT
        - Have scriptlets that hook in to provide the rules aspect (I use the
        term "scriptlets" to denote small bits of code in the engine's
        preferred language: javascript, python, perl, whatever)

        The reason that I like this approach is that you don't have to agree
        on a very specific format for the neutral base data. It can be very
        loose, like I described. As well, option 2 is still too close to
        engine specific code, since everyone would have to agree on a language
        and an API.

        On the maintenance side, as publisher data is released you don't need
        developer-grade people to code in loose XML. There are a lot of people
        out there that are willing to help but that don't know LST,
        javascript, etc. So their task would be to enter the base data using
        guidelines. Sure a schema or DTD could be used, but we would have to
        agree on it, which is no mean task.

        The rules or logic portion would be then the responsibility of the
        developer-grade people for each engine. You distribute the workload,
        and you remove the engine dependency from the base data if an engine
        gets updated.

        My general philosophy is that we should minimize the requirements for
        standards, since that is what has stopped XML projects in the past.
        Everyone has his/her own view of how the XML data should be
        structured, and arguments. Let's just get something that can be
        machine readable and reused out there. The "small steps" theory.

        Frugal, is it possible for you to do a "neutral" dump as I described
        if it's not too much trouble? It would allow me to demonstrate what I
        mean. I could even craft an XSLT that would format the neutral data
        into one compatible with your engine (as much as possible). It would
        be much appreciated :)

        If you don't have the time, could you upload your LST parser so I can
        fiddle with it?

        Andargor
      • Paul M. Lambert
        ... I ve been reading this discussion and I d like to make sure I understand what you mean, andargor. Tell me what below matches your thoughts and what
        Message 3 of 28 , Jul 4, 2004
        • 0 Attachment
          On Sun, 04 Jul 2004, andargor wrote:

          > Nice wrench. :)
          >
          > I don't want ot discuss semantics, but the example I gave is not
          > specific to any engine. For example, I do not plan to use the
          > fighter_bonus tag in my program and I doubt Frugal or PCGen will ever
          > use it.
          >
          > The point is to use XML for what it is intended for: tagging data,
          > nothing more, nothing less. It is a machine readable version of
          > publisher data, that's all.
          >
          > fighter_bonus gets transformed into engine specific tags:
          > <bonus><type>Fighter</type></bonus> or <bonus id="Alterness"
          > type="fighter.bonus"> or even TYPE:General.Fighter if you want to
          > regenerate LST. You use XSLT to do that.
          >
          > The marvelous thing is that you only coded the publisher data once.
          > All you have to do is maintain a transform (XSLT). Ain't that sweet?
          >
          > Andargor

          I've been reading this discussion and I'd like to make sure I understand
          what you mean, andargor.

          Tell me what below matches your thoughts and what doesn't, so we can get
          on the same page. Then I'll be able to tell if I actually agree with
          you. :-)

          As I see it your "data neutral" format idea could be re-expressed as a
          high level pseudo language. This language could then be converted into
          a given engine's native format (LST, PCGEN-XML--whatever that is,
          "plambertgen", whatever) if someone writes such a converter. And then the
          engine can handle that data as it sees fit.

          I see this to mean that the data neutral format would allow tag
          (re-)definitions, so, for example, <figher_bonus> could be a tag
          used in one data file, and it might be definied in coarser game
          terms in another. This would allow "libraries" of definitions for
          a given game system or rule set that would allow easy mapping to
          specific engines in the converters.

          I'm trying to think of a concrete example. Perhaps something like:

          <str>7</str>

          in a character definition for a 3.5 SRD character. And in the
          library data file for 3.5 SRD, you might have:

          <character>
          ...
          <attribute>
          <name>Strength</name>
          <abbr>STR</abbr>
          <tag>str</tag>
          <description>Physical strength and prowess</description>
          ...
          </attribute>
          ...
          </character>

          These are extremely rough non-scaling examples, of course, and not
          necessarily how I actually suggest doing attributes, but anyway...

          A converter then might take that information and turn it into LST data.
          The converter would know the limits of its target format. For example,
          if it's not possible to actually change/rename/add attributes in LST
          format, it would basically know to ignore the attribute definitions and
          use the ones inherent to the engine. If another were used, it could
          generate a warning or error.

          A converter might even have full internal knowledge of a given "library"
          as well, if it was a good choice for the coder/engine/etc. For example,
          an engine extremely specific to a certain game rule set (maybe an SRD3.5
          only web character repository, or whatever) could ignore the library
          entirely, since the author would know what was and wasn't defined in
          there.

          This would mean the libraries and the converters could be maintained by
          people with strong such skills, while the casual user who just wants to
          say "I'd like to add a Bow of Tumultuous Angst to my equipment lists;
          it's a Long Bow that gives +2/+4 only to Chaotic Neutral wielders."
          could do it with dirt-simple, straightforward tags. Still valid XML, but
          really simple tags whose inner workings are defined elsewhere, out of
          sight.

          However, if they want it to give the wielder +1 to hit for each pound of
          bat guano they're carrying, they'll need to dig deeper. But simple
          things will be easy and complex things will be possible. Right?

          Please let me know where I misunderstand.

          --plambert
        • andargor
          Answers inline. ... (snip) ... then the ... You sure think ahead :) Actually, what I propose is more basic: just convert publisher data into a machine readable
          Message 4 of 28 , Jul 4, 2004
          • 0 Attachment
            Answers inline.

            --- In pcgen-xml@yahoogroups.com, "Paul M. Lambert" <plambert@p...> wrote:
            (snip)
            > As I see it your "data neutral" format idea could be re-expressed as a
            > high level pseudo language. This language could then be converted into
            > a given engine's native format (LST, PCGEN-XML--whatever that is,
            > "plambertgen", whatever) if someone writes such a converter. And
            then the
            > engine can handle that data as it sees fit.
            >
            > I see this to mean that the data neutral format would allow tag
            > (re-)definitions, so, for example, <figher_bonus> could be a tag
            > used in one data file, and it might be definied in coarser game
            > terms in another. This would allow "libraries" of definitions for
            > a given game system or rule set that would allow easy mapping to
            > specific engines in the converters.


            You sure think ahead :)

            Actually, what I propose is more basic: just convert publisher data
            into a machine readable format with a minimum of context (i.e. XML
            tags) so that it can be manipulated. Your library concept could then
            be implemented, if you wish to do so. First things first :)

            (snip)
            > This would mean the libraries and the converters could be maintained by
            > people with strong such skills, while the casual user who just wants to
            > say "I'd like to add a Bow of Tumultuous Angst to my equipment lists;
            > it's a Long Bow that gives +2/+4 only to Chaotic Neutral wielders."
            > could do it with dirt-simple, straightforward tags. Still valid
            XML, but
            > really simple tags whose inner workings are defined elsewhere, out of
            > sight.


            Yes, the general idea is to leverage the large quantity of people that
            have the willingess to help but not necessarily the coding skills.
            They could describe that bow very simply, and fill in the logic
            portion as a description. Someone with coding skills could then craft
            an engine specific XSLT and translate the logic into engine-specific
            "scriptlets" for a particular tool.

            Andargor
          • Frugal
            ... The LST parser is already up on the groups file section: http://groups.yahoo.com/group/pcgen-xml/files/sample_source-20031216.tgz As
            Message 5 of 28 , Jul 5, 2004
            • 0 Attachment
              <quote who="andargor">

              > Frugal, is it possible for you to do a "neutral" dump as I described
              > if it's not too much trouble? It would allow me to demonstrate what I
              > mean. I could even craft an XSLT that would format the neutral data
              > into one compatible with your engine (as much as possible). It would
              > be much appreciated :)
              >
              > If you don't have the time, could you upload your LST parser so I can
              > fiddle with it?

              The LST parser is already up on the groups file section:

              http://groups.yahoo.com/group/pcgen-xml/files/sample_source-20031216.tgz

              As the date indicates this has not been worked on since the middle of
              december. Be warned: "here be kludges".

              Feel free to do with it as you will, There should be a GPL header at the
              top of each file ;O)

              --
              regards,
              Frugal
              -OS Chimp
            • David Finch
              ... A first good step would be that the data had been input. If the logic has been typed in english then at least when the conversion to PCgen format is done
              Message 6 of 28 , Jul 5, 2004
              • 0 Attachment
                andargor wrote:
                > Yes, the general idea is to leverage the large quantity of people that
                > have the willingess to help but not necessarily the coding skills.
                > They could describe that bow very simply, and fill in the logic
                > portion as a description. Someone with coding skills could then craft
                > an engine specific XSLT and translate the logic into engine-specific
                > "scriptlets" for a particular tool.

                A first good step would be that the data had been input. If the logic has
                been typed in english then at least when the conversion to PCgen format is
                done someone can go though at 'specialise it' to PCgen format. This has the
                added advatage that someone can check that the english logic matches the
                real programming logic.
              • andargor
                ... 20031216.tgz ... of ... at the ... Yikes! There be kludges :) Well, I had to craft an Ant build.xml, and it seems to compile properly... What version of
                Message 7 of 28 , Jul 6, 2004
                • 0 Attachment
                  --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                  >
                  > The LST parser is already up on the groups file section:
                  >
                  > http://groups.yahoo.com/group/pcgen-xml/files/sample_source-
                  20031216.tgz
                  >
                  > As the date indicates this has not been worked on since the middle
                  of
                  > december. Be warned: "here be kludges".
                  >
                  > Feel free to do with it as you will, There should be a GPL header
                  at the
                  > top of each file ;O)
                  >
                  > --
                  > regards,
                  > Frugal
                  > -OS Chimp

                  Yikes! There be kludges :)

                  Well, I had to craft an Ant build.xml, and it seems to compile
                  properly...

                  What version of the data files were you using? The parser is croaking
                  on several of the 5.7.2 ones.

                  Andargor
                • Frugal
                  ... Oh, yeah, oops. I was using Eclipse and it managed all of the building for me ;O) ... I was using the CVS data as of the date of the
                  Message 8 of 28 , Jul 6, 2004
                  • 0 Attachment
                    <quote who="andargor">
                    > --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                    >>
                    >> The LST parser is already up on the groups file section:
                    >>
                    >> http://groups.yahoo.com/group/pcgen-xml/files/sample_source-
                    > 20031216.tgz
                    >>
                    >> As the date indicates this has not been worked on since the middle
                    > of
                    >> december. Be warned: "here be kludges".
                    >>
                    >> Feel free to do with it as you will, There should be a GPL header
                    > at the
                    >> top of each file ;O)
                    >
                    > Yikes! There be kludges :)
                    >
                    > Well, I had to craft an Ant build.xml, and it seems to compile
                    > properly...

                    Oh, yeah, oops. I was using Eclipse and it managed all of the building for
                    me ;O)

                    > What version of the data files were you using? The parser is croaking
                    > on several of the 5.7.2 ones.

                    I was using the CVS data as of the date of the archive. So any new tags
                    will fail.

                    IT would be so nice if every time someone messed around with a data tag
                    they were forced to increment the LST version number. It would make data
                    management a lot easier if you could state which version of LST syntax
                    your LST files were.

                    --
                    regards,
                    Frugal
                    -OS Chimp
                  • andargor
                    ... tag ... data ... syntax ... Or if the data were in XML and the rule logic described somewhere for PCGen, the LST could be regenerated with each new version
                    Message 9 of 28 , Jul 6, 2004
                    • 0 Attachment
                      --- In pcgen-xml@yahoogroups.com, "Frugal" <frugal@p...> wrote:
                      >
                      > IT would be so nice if every time someone messed around with a data
                      tag
                      > they were forced to increment the LST version number. It would make
                      data
                      > management a lot easier if you could state which version of LST
                      syntax
                      > your LST files were.
                      >
                      > --
                      > regards,
                      > Frugal
                      > -OS Chimp

                      Or if the data were in XML and the rule logic described somewhere for
                      PCGen, the LST could be regenerated with each new version ;)

                      Andargor
                    • Mark Coletti
                      On Thu, 1 Jul 2004 08:55:51 +0100 (BST), Frugal ... It was put the rules in the data that did it for me. What you re talking about is really a knowledge
                      Message 10 of 28 , Jul 14, 2004
                      • 0 Attachment
                        On Thu, 1 Jul 2004 08:55:51 +0100 (BST), Frugal
                        <frugal@...> wrote:
                        >
                        > <quote who="Tir Gwaith">
                        > > There is no "neutral" version, just the perception of one. [...] there won't
                        > > ever be a stable version, because new rules will
                        > > always be written.

                        > I think that a 'neutral' version of the data can be farily easily created.
                        > The problem the D20 is not the data, but the manipulation of the data.
                        > Every time a company brings out a new rule the way the data is to be
                        > manipulated needs to be changed. As you stated the problem is not the
                        > data, but the extra rules that are always added.

                        > As I see it encoding the rules can be done in 3 ways:
                        > - The PCGen way: every thing is hard coded into the program, new rules
                        > mean new code
                        > - Put the rules in the data: As well as encoding the data, also encode a
                        > scripting language to manipulate the data.
                        > - Plugins: New rules get encoded as plugins that are loaded into the
                        > program as needed. So a dataset would contain data and a list of plugins
                        > that are required to manipulate the data.

                        It was "put the rules in the data" that did it for me. What you're
                        talking about is really a knowledge base; which makes sense in that
                        PCGen *is* essentially an expert system that relies on a
                        knowledge-base containing rules for various d20 based gaming systems.
                        The problem is that most of the knowledge is either hard-coded in Java
                        or in LST files, and most of the "inferencing" for the huge cloud of
                        rules is also hard-coded. (Think IF-THEN and SWITCH statements as a
                        crude sort of linear, data-driven 'inferencing'.)

                        JESS is a free, open-source, expert system implemented in Java. I can
                        easily see where most of the PCGen "knowledge-base" currently spread
                        among Java and LST files could be rolled into a knowledge-base
                        containing explicit rules. Specific systems would just add new
                        knowledge-bases that contain fresh sets of rules (including some that
                        may remove or modify existing ones).

                        A JESS (or similar system) based PCGen system would be inherently
                        smaller since the brunt of the inferencing will be left to the
                        inference engine, and the rules would be explicitly codified as, well,
                        rules. You could also set up the knowledge-base such that you can
                        generate explanations. One thing about the d20 systems is that
                        they're very complex once you consider all the different rules and
                        their respective interactions. If, say, a character has an 18 AC, it
                        might be nice to know *how* and *why* it's currently 18. All the
                        rules that trigger AC mods could contribute to explanatory text. "AC
                        base 10, +2 for Dexterity, +3 for ring of nimbleness, +3 to <mumble>
                        spell effect."

                        Of course this is all pie-in-the-sky ramblings implementing JESS in
                        PCGen would entail quite a bit of work. (Um, I think.) But, what the
                        hell, it makes for a pleasant gedankenexperiment. %-)

                        More info:

                        http://web.njit.edu/all_topics/Prog_Lang_Docs/html/jess51/

                        Cheers!

                        MAC
                        --
                        I'm taking reality in small doses to build immunity.
                      Your message has been successfully submitted and would be delivered to recipients shortly.