Loading ...
Sorry, an error occurred while loading the content.

Re: [PCGD] Code Meeting April 2009 - Log

Expand Messages
  • Tom Parker
    ... My primary issue with the output tokens is that some require the core, and others are pure plugins.  100% pure plugins has to be the target - and no more
    Message 1 of 15 , Apr 29, 2009
    • 0 Attachment
      --- In pcgen_developers@yahoogroups.com, "Martijn Verburg" <martijnverburg@...> wrote:
      > Tom also had some very alpha thoughts on the OS Token clean up (IIRC),
      > so he'll hopefully comment here. At a guess the OS tokens either need a
      > clean up (ala the LST tokens). I'm assuming at this stage that an OS token
      > system is still the preferred mechanism.

      My primary issue with the output tokens is that some require the core, and others are pure plugins.  100% pure plugins has to be the target - and no more switch statements - they should have subtokens like the current LST token system.

      Having said that, the architecture needs to take into account the differences that a templating system like Velocity would introduce, and the thoughts above may not be complete in that respect.

      Andrew Wilson:
      > I'm
      intending to do some more work on formula processing,
      > JEP, output sheet stuff—which is all intermingled.

      Can you help on the scope of what you would be doing with formula processing and JEP? I would like to rip it out eventually, due to it being a license exception combined with concerns that the new version is closed source, leaving us with a stagnant code library.

      We're also in a position where we're still parsing "after the fact" (don't check at LST load) and there is significant work that could be done with a (better) formula system to accelerate formula processing.


      One of the root reasons for using the Formula interface as defined with the resolve() methods is to isolate JEP and thus allow swapping it out in the future without wild quantities of pain.

      TP.


    • Andrew Wilson
      ... At the moment, any formula can contain any OS token. The way that formulae get chopped up into btis we can evaluate is very convoluted and doesn t follow
      Message 2 of 15 , Apr 29, 2009
      • 0 Attachment
        2009/4/29 Tom Parker <thpr@...>:
        > --- In pcgen_developers@yahoogroups.com, "Martijn Verburg"
        > <martijnverburg@...> wrote:
        >> Tom also had some very alpha thoughts on the OS Token clean up (IIRC),
        >> so he'll hopefully comment here. At a guess the OS tokens either need a
        >> clean up (ala the LST tokens). I'm assuming at this stage that an OS token
        >> system is still the preferred mechanism.
        >
        > My primary issue with the output tokens is that some require the core, and
        > others are pure plugins.  100% pure plugins has to be the target - and no
        > more switch statements - they should have subtokens like the current LST
        > token system.
        >
        > Having said that, the architecture needs to take into account the
        > differences that a templating system like Velocity would introduce, and the
        > thoughts above may not be complete in that respect.

        At the moment, any formula can contain any OS token. The way that
        formulae get chopped up into btis we can evaluate is very convoluted
        and doesn't follow standard mathematical precedence.

        > Andrew Wilson:
        >> I'm intending to do some more work on formula processing,
        >> JEP, output sheet stuff—which is all intermingled.
        >
        > Can you help on the scope of what you would be doing with formula processing
        > and JEP? I would like to rip it out eventually, due to it being a license
        > exception combined with concerns that the new version is closed source,
        > leaving us with a stagnant code library.

        I'd like to rip it out too—because the version we're using is mostly
        abandoned and stagnating, and also because it has far more features
        than we need and they just get in our way. I was under the impression
        that, as a project, we wanted embrace more of JEP rather than rip it out.
        My plans involve replacing the "old parser" with something that can
        (a) recognise the bits that make up a PCGEN formula, (b) follow
        mathematical precedence for operators and (c) not involve tens
        (hundreds) of recursive calls every time we evaluate a formula.

        This would need to be able to Isolate the output tokens, the hardcoded
        variabes (current term objects), LST variables, JEP functions and
        operators. Everything except the operators then gets something
        resembling a term object. I'd envisaged ending up with each formula
        parsed once into a tree of operators and term objects that can be
        evaluated on demand.

        Since I thought the project wanted to keep JEP, I was planning to use
        it (or bits of it anyway) to derive the tree. Now I think I wouldn't bother to
        use it at all.

        > We're also in a position where we're still parsing "after the fact" (don't
        > check at LST load) and there is significant work that could be done with a
        > (better) formula system to accelerate formula processing.

        This could cope with that. I wasn't planning to do it right now, but once
        we have something that isolates the terms and operators, we can write
        validators for the terms. This could then be called on LST load and we'd
        know if we have valid formulae.

        > One of the root reasons for using the Formula interface as defined with the
        > resolve() methods is to isolate JEP and thus allow swapping it out in the
        > future without wild quantities of pain.

        I'm all for avoiding massive amounts of pain.

        To be honest this cycle I think I'd only have three new term objects one that
        know how to look up a LST variable, one that knows how to evaluate an OS
        token and one that can process a JEP function. I think we can eliminate
        the multiple recursive calls to the old parser and JEP on each formula
        evaluation. The overall formula would be parsed once. The OS tokens and
        poossibly JEP functions would be parsed every time they're evaluated.
        If we get that far we can eliminate all of JEP, except the bits we added to it.

        Does that sound compatible with your vision for the architecture?

        andrew
      • Tom Parker
        ... I was under the impression that, as a project, we wanted embrace more of JEP rather than rip it out. In my opinion, that was true until JEP went closed
        Message 3 of 15 , Apr 30, 2009
        • 0 Attachment

          --- On Wed, 4/29/09, Andrew Wilson <andrew@...> wrote:
          I was under the impression
          that, as a project, we wanted embrace more of JEP rather than rip it out.

          In my opinion, that was true until JEP went closed source.  To my recollection, no deep discussion took place on the consequences of that JEP change to our long-term formula strategy.  I think it's been a matter of priorities.  Thus I believe the issue is open and discussion on that aspect of our strategy is fair game.

          While I hate reinventing wheels without *really* good reasons and while there is certainly risk in moving to our own solution (since it introduces more code for us by eliminating a library), those may be outweighted by the risk in remaining on JEP (unsupported, now closed source, library).

          My plans involve replacing the "old parser" with something that can
          (a) recognise the bits that make up a PCGEN formula, (b) follow
          mathematical precedence for operators and (c) not involve tens
          (hundreds) of recursive calls every time we evaluate a formula.

          I would add:
          (d) and complain at LST load (vs at runtime) if a formula is invalid (you had addressed this later on in your note), which means it also has the ability to detect and warn at LST load for deprecated formula styles, e.g. 2MAXINT
          and
          (e) Consider whether we have a formula factory that can detect situations where a formula like 1+INT is used multiple places in the LST.  These should be reduced via a cache to a single Formula object (since a Formula is immutable) to save memory and hopefully reduce LST load time.

          I'd also note that (c) is difficult, since some of that recursiveness is necessary based on our BONUS structure.  So while I agree that's part of the vision, whether it's part of scope in 5.17 might need to be up for discussion (I defer to you on evaluating/defining that scope)

          Same for (e) - certainly doesn't need to be an immediate step in 5.17. 

          To be honest this cycle I think I'd only have three new term objects one
          that
          know how to look up a LST variable, one that knows how to evaluate an OS
          token and one that can process a JEP function.

          For clarity - how would:
          1+MYVAR

          be evaluated with the 3 terms you describe above?  Specifically, is the goal for 5.17 still to have JEP doing the evaluation of the +? 

          I think we can eliminate
          the multiple recursive calls to the old parser and JEP on each formula
          evaluation.

          Good, and I agree this is the right path.  Just make sure we have some unit tests for dependent items, e.g.:
          DEFINE:MYVAR|5
          BONUS:VAR|MYVAR|3
          DEFINE:OTHERVAR|MYVAR+2

          (there are some internal variables that are even worse)

           The overall formula would be parsed once.

          This is how I noticed the SCORE replacement (specifically the String replacement in StatList's getModForNumber(int, int) for those who want to look) which makes me wonder how often we can only parse once vs. the formula being string modified before it goes into the evaluator.  I know equipment (and equipment modifiers especially) are ugly for String replacement right before formula evaluation.

          I like your term system, but as I've gone through the core, there are places we need to expand what is a term vs. what is string replacement.  Not to be addressed as part of this conversation, but awareness to folks that faster runtime performance on formulas doesn't solve all of our problems in one shot.  Also need to be conscious with our own parser that we don't change the rules to the point where the replacement would break.

          If we get that far we can eliminate all of JEP, except the bits we added to it.

          This is where I get lost and what triggered the question on + above.

          Does that sound compatible with your vision for the architecture?

          Sounds right on, with the caveat that I'd like to comprehend what is doing the mathematical operator work to produce the tree in the first place (new code?).

          ...

          Having said that, under the assumption that we're going to be developing a parser for ourselves... and thus having to write something to do evaluation of the tree the parser produces as well as testing the parser and evaluator... then I have an idea I would like to float (or sink, as the case may be).

          So this is Tom as code contributor (not even Code Chimp, since James might demote me for something this reckless :D ), not Tom in Architecture Silverback mode:

          For various reasons, I wrote a formula parser some years ago (2004-2005).  It uses JJTree to produce a .jj file which is then compiled to .java files through javacc.  Connor recently did work to eliminate javacc, which is probably controversy #1 in reintroducing derived code and javacc as an ant task (Though I believe it's the most reliable way to produce a parser, so if we go our own way from JEP, this should be the path)

          A portion of the project at the time was learning the inner workings of compilers, Java and JavaCC, so I was also playing with ASM (potential controversy #2, adding yet another library)

          What I produced is code that can take a formula (e.g. "5+T") and compile it directly into Java bytecode (and then load that bytecode as a class).  This usually raises hair on the back of people's necks, because dynamic code is considered heresy under certain design rules (controversy #3).

          Hey, at least it's not self-modifying code.  I could have done that :P

          I dropped it as a project about the time I joined PCGen, but it does work for many of the basics (perhaps enough for PCGen?) and can even simplify constants like 1+3 so the resolve method is simply load 4 onto the stack and return.

          By compiling to bytecode it is extremely fast (some items - terms in Andrew's parlance - simply become a method call to another Formula, while variables can be referred to through a resolving object) at runtime.

          I haven't looked at it in a while, so it may be a bit of a pain to integrate with your Term system, but I'm happy to hand it over to you (and others) to work on, so it's an option we can evaluate if people don't think it's completely nuts.

          Note also for any non-JEP system, we produce the complication that the existing functions (e.g. count) we have would need to be rewritten into the new system vs the JEP functions we have today.

          TP.
          --
          Tom Parker
          thpr@... and tppublic@...


        • Andrew Wilson
          ... Yeah, I see those as separate formulae. What I really hate is the way we seem to need multiple calls to and fro between JEP and the old parser. ... I see
          Message 4 of 15 , May 2, 2009
          • 0 Attachment
            2009/4/30 Tom Parker <thpr@...>:
            > I would add:
            > (d) and complain at LST load (vs at runtime) if a formula is invalid (you
            > had addressed this later on in your note), which means it also has the
            > ability to detect and warn at LST load for deprecated formula styles, e.g.
            > 2MAXINT
            > and
            > (e) Consider whether we have a formula factory that can detect situations
            > where a formula like 1+INT is used multiple places in the LST.  These should
            > be reduced via a cache to a single Formula object (since a Formula is
            > immutable) to save memory and hopefully reduce LST load time.
            >
            > I'd also note that (c) is difficult, since some of that recursiveness is
            > necessary based on our BONUS structure.  So while I agree that's part of the
            > vision, whether it's part of scope in 5.17 might need to be up for
            > discussion (I defer to you on evaluating/defining that scope)

            Yeah, I see those as separate formulae. What I really hate is the way we
            seem to need multiple calls to and fro between JEP and the old parser.

            > To be honest this cycle I think I'd only have three new term objects one
            > that
            > know how to look up a LST variable, one that knows how to evaluate an OS
            > token and one that can process a JEP function.
            >
            > For clarity - how would:
            > 1+MYVAR
            >
            > be evaluated with the 3 terms you describe above?  Specifically, is the goal
            > for 5.17 still to have JEP doing the evaluation of the +?

            I see that as two terms the 1 is a constant that evaluates to itself, the MYVAR
            would use the variable lookup term. Then the formula evaluator would
            walk the tree,
            evaluating and combining as it goes.

            > I think we can eliminate
            > the multiple recursive calls to the old parser and JEP on each formula
            > evaluation.
            >
            > Good, and I agree this is the right path.  Just make sure we have some unit
            > tests for dependent items, e.g.:
            > DEFINE:MYVAR|5
            > BONUS:VAR|MYVAR|3
            > DEFINE:OTHERVAR|MYVAR+2
            >
            > (there are some internal variables that are even worse)

            noted.

            > The overall formula would be parsed once.
            >
            > This is how I noticed the SCORE replacement (specifically the String
            > replacement in StatList's getModForNumber(int, int) for those who want to
            > look) which makes me wonder how often we can only parse once vs. the formula
            > being string modified before it goes into the evaluator.  I know equipment
            > (and equipment modifiers especially) are ugly for String replacement right
            > before formula evaluation.
            >
            > I like your term system, but as I've gone through the core, there are places
            > we need to expand what is a term vs. what is string replacement.  Not to be
            > addressed as part of this conversation, but awareness to folks that faster
            > runtime performance on formulas doesn't solve all of our problems in one
            > shot.  Also need to be conscious with our own parser that we don't change
            > the rules to the point where the replacement would break.
            >
            > If we get that far we can eliminate all of JEP, except the bits we added to
            > it.
            >
            > This is where I get lost and what triggered the question on + above.

            When I formulated my original ideas I was planning to use JEP to
            do the parsing and evaluating. The caveat was that the formula
            was going to need heavily preprocessed before handing to JEP.
            I was planning to take something like:

            1+CL=monkey (code)/(CLASS.0.LEVEL*MYVAR)

            and turn that into

            1+VAR001/(VAR002*VAR003)

            then when JEP processed it, it'd find 3 variables

            VAR001 => CL=monkey (code)
            VAR002 => CLASS.0.LEVEL
            VAR003 => MYVAR

            I was then planning to hijack JEP's symbol lookup so that when it looked for a
            value for for VAR001 it found a term object that knows how to evaluate CL=
            VAR002 is a term object that can look up an output tag and VAR003 a term
            object that looks up a LST variable.

            Now I'm not so sure that's the best option if we are planning to ditch
            JEP anyway, then I don't need to bolt my system onto both sides of it.
            I think I'd like to look into whether we can carve up the current JEP
            system and maybe only pinch bits of the parser and get it to build
            a tree of term objects. Using bits of JEP will probably mean we can
            integrate it's function handling. One of the major obstacles is the
            use of things that should be reserved characters but aren't, () are
            nasty. classlevel("code-monkey") means you can't just split on
            operators, 20/(5*classlevel("monkey (code)")) actually has three
            different uses of parens. 20/(5*somefunction(")")) actually does
            have balanced parens. Parsing this is not a trivial problem,
            but we need to deal with it at some point.

            > Does that sound compatible with your vision for the architecture?
            >
            > Sounds right on, with the caveat that I'd like to comprehend what is doing
            > the mathematical operator work to produce the tree in the first place (new
            > code?).

            Probably new code, possibly heavily influenced by JEP's parser.

            > Having said that, under the assumption that we're going to be developing a
            > parser for ourselves... and thus having to write something to do evaluation
            > of the tree the parser produces as well as testing the parser and
            > evaluator... then I have an idea I would like to float (or sink, as the case
            > may be).
            >
            > So this is Tom as code contributor (not even Code Chimp, since James might
            > demote me for something this reckless :D ), not Tom in Architecture
            > Silverback mode:
            >
            > For various reasons, I wrote a formula parser some years ago (2004-2005).
            > It uses JJTree to produce a .jj file which is then compiled to .java files
            > through javacc.  Connor recently did work to eliminate javacc, which is
            > probably controversy #1 in reintroducing derived code and javacc as an ant
            > task (Though I believe it's the most reliable way to produce a parser, so if
            > we go our own way from JEP, this should be the path)
            >
            > A portion of the project at the time was learning the inner workings of
            > compilers, Java and JavaCC, so I was also playing with ASM (potential
            > controversy #2, adding yet another library)
            >
            > What I produced is code that can take a formula (e.g. "5+T") and compile it
            > directly into Java bytecode (and then load that bytecode as a class).  This
            > usually raises hair on the back of people's necks, because dynamic code is
            > considered heresy under certain design rules (controversy #3).
            >
            > Hey, at least it's not self-modifying code.  I could have done that :P
            >
            > I dropped it as a project about the time I joined PCGen, but it does work
            > for many of the basics (perhaps enough for PCGen?) and can even simplify
            > constants like 1+3 so the resolve method is simply load 4 onto the stack and
            > return.
            >
            > By compiling to bytecode it is extremely fast (some items - terms in
            > Andrew's parlance - simply become a method call to another Formula, while
            > variables can be referred to through a resolving object) at runtime.
            >
            > I haven't looked at it in a while, so it may be a bit of a pain to integrate
            > with your Term system, but I'm happy to hand it over to you (and others) to
            > work on, so it's an option we can evaluate if people don't think it's
            > completely nuts.

            I'd certainly like to take a look at it. It sounds horrendously complex and
            to be honest I think this problem is complex enough already, but as I said
            I'd be interested in seeing it.

            > Note also for any non-JEP system, we produce the complication that the
            > existing functions (e.g. count) we have would need to be rewritten into the
            > new system vs the JEP functions we have today.

            Yes, I know. I wrote the count function it only really works for Abilty
            objects at the moment. If we can get a system that understands our
            hardcoded stuff like COUNT[] then we can probably implement term objects
            to handle that and deprecate the count() function. There are other JEP
            functions that are useful. var() would become obsolete, since var("MYVAR")
            and MYVAR should do identical things (the former might be less efficient).

            andrew
          • Tom Parker
            ... One of the major obstacles is the use of things that should be reserved characters but aren t, () are nasty. I don t think this is as nasty as you think it
            Message 5 of 15 , May 2, 2009
            • 0 Attachment

              --- On Sat, 5/2/09, Andrew Wilson <affable@...> wrote:
              One of the major obstacles is the
              use of things that should be reserved characters but aren't, () are
              nasty.

              I don't think this is as nasty as you think it is... as long as they can be distinguished, it will work in a parser system like jjtree/javacc.  I've written a few compilers (not the depth of complexity of modern languages, but that type of thing), and it's not bad, and arguably easier than how we separate tokens in the LST system today.

              I'd certainly like to take a look at it.  It sounds horrendously
               complex
              and
              to be honest I think this problem is complex enough already, but as I said
              I'd be interested in seeing it.

              Complexity is relative.  I think it actually transfers a lot of complexity into javacc.  The main complexity is mostly in the optimization (making 1+3 into 4), which we could really tear out if it is too confusing or too complex.  With that removed, you may be surprised at how simple a parser/compiler really is.

              I'll dig it up and upload it into a sandbox.

              TP.
              --
              Tom Parker
              thpr@... and tppublic@...


            • Tom Parker
              ... This is now loaded into sandbox/FormulaCompiler Yes, it s a bit of a heavy user of System.err at the moment, it s still a bit broken, and the tests aren t
              Message 6 of 15 , May 2, 2009
              • 0 Attachment
                --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@...> wrote:
                > I'll dig it up and upload it into a sandbox.

                This is now loaded into sandbox/FormulaCompiler

                Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...

                TP.
              • Andrew Wilson
                Thanks, I ll have a look. andrew ...
                Message 7 of 15 , May 2, 2009
                • 0 Attachment
                  Thanks, I'll have a look.

                  andrew

                  2009/5/2 Tom Parker <thpr@...>:
                  > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@...> wrote:
                  >> I'll dig it up and upload it into a sandbox.
                  >
                  > This is now loaded into sandbox/FormulaCompiler
                  >
                  > Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...
                  >
                  > TP.
                  >
                  >
                  >
                  >
                  > ------------------------------------
                  >
                  > Yahoo! Groups Links
                  >
                  >
                  >
                  >

                  <div><br></div>
                • Martijn Verburg
                  Hi All, Searched for alternatives to JEP and also looked for Forks of that library, and what do you know! I found a discussion between myself and their main
                  Message 8 of 15 , May 3, 2009
                  • 0 Attachment
                    Hi All,

                    Searched for alternatives to JEP and also looked for Forks of that library, and what do you know! I found a discussion between myself and their main developer about forking the project.

                    It is a valid option if we want to do that as well.

                    Thoughts?

                    K

                    > Thanks, I'll have a look.
                    >
                    > andrew
                    >
                    > 2009/5/2 Tom Parker <thpr@...>:
                    > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@> wrote:
                    > >> I'll dig it up and upload it into a sandbox.
                    > >
                    > > This is now loaded into sandbox/FormulaCompiler
                    > >
                    > > Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...
                    > >
                    > > TP.
                    > >
                    > >
                    > >
                    > >
                    > > ------------------------------------
                    > >
                    > > Yahoo! Groups Links
                    > >
                    > >
                    > >
                    > >
                    >
                    > <div><br></div>
                    >
                  • Tom Parker
                    My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His
                    Message 9 of 15 , May 3, 2009
                    • 0 Attachment
                      My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His project is GPL2, which causes problems for us.  We are using Apache libraries that are not compatible with GPL2. 

                      I would strongly vote against any fork which was still operating under an exception - there is *way* to much risk involved since it starts to impact our ability to use code we would have to develop/modify.  The fork would have to be LGPL or Apache licensed (he would have to explicitly license it that way in a public forum, not just in an email to us) or it's a non-starter IMHO.

                      I also happen to agree with Andrew - what we're looking for is much simpler than JEP.  The effort spent taking out the complexity we don't want is better spent building up what we DO want.  Parsers really aren't hard.  In fact, we may need to take complexity OUT of the (much simpler) parser I wrote (though we need to add handling of some additional characters inside quotes)

                      I also think a fork is actually worse, unless we get extensive test code along with his code.  If we have to spend years developing a test library, then it's better & faster to start from scratch so we can know what we are dealing with and only have to test the subset of what we want & develop (not what people could use if they knew JEP inside and out)

                      TP.
                      --
                      Tom Parker
                      thpr@... and tppublic@...

                      --- On Sun, 5/3/09, Martijn Verburg <martijnverburg@...> wrote:
                      From: Martijn Verburg <martijnverburg@...>
                      Subject: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                      To: pcgen_developers@yahoogroups.com
                      Date: Sunday, May 3, 2009, 4:49 AM

                      Hi All,

                      Searched for alternatives to JEP and also looked for Forks of that library, and
                      what do you know! I found a discussion between myself and their main developer
                      about forking the project.

                      It is a valid option if we want to do that as well.

                      Thoughts?

                      K

                      > Thanks, I'll have a look.
                      >
                      > andrew
                      >
                      > 2009/5/2 Tom Parker <thpr@...>:
                      > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@>
                      wrote:
                      > >> I'll dig it up and upload it into a sandbox.
                      > >
                      > > This is now loaded into sandbox/FormulaCompiler
                      > >
                      > > Yes, it's a bit of a heavy user of System.err at the moment,
                      it's still a bit broken, and the tests aren't *really* tests in the
                      jUnit sense, but this really wasn't originally intended for public
                      consumption...
                      > >
                      > > TP.
                      > >
                      > >
                      > >
                      > >
                      > > ------------------------------------
                      > >
                      > > Yahoo! Groups Links
                      > >
                      > >
                      > >
                      > >
                      >
                      > <div><br></div>
                      >




                      ------------------------------------

                      Yahoo! Groups Links

                      <*> To visit your group on the web, go to:
                      http://groups.yahoo.com/group/pcgen_developers/

                      <*> Your email settings:
                      Individual Email | Traditional

                      <*> To change settings online go to:
                      http://groups.yahoo.com/group/pcgen_developers/join
                      (Yahoo! ID required)

                      <*> To change settings via email:
                      mailto:pcgen_developers-digest@yahoogroups.com
                      mailto:pcgen_developers-fullfeatured@yahoogroups.com

                      <*> To unsubscribe from this group, send an email to:
                      pcgen_developers-unsubscribe@yahoogroups.com

                      <*> Your use of Yahoo! Groups is subject to:
                      http://docs.yahoo.com/info/terms/


                    • Eric Jarman
                      Here s a BSD licensed expression parser: http://sourceforge.net/projects/expression-tree/
                      Message 10 of 15 , May 3, 2009
                      • 0 Attachment
                        Here's a BSD licensed expression parser:  http://sourceforge.net/projects/expression-tree/


                        On Sun, May 3, 2009 at 11:36 AM, Tom Parker <thpr@...> wrote:


                        My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His project is GPL2, which causes problems for us.  We are using Apache libraries that are not compatible with GPL2. 

                        I would strongly vote against any fork which was still operating under an exception - there is *way* to much risk involved since it starts to impact our ability to use code we would have to develop/modify.  The fork would have to be LGPL or Apache licensed (he would have to explicitly license it that way in a public forum, not just in an email to us) or it's a non-starter IMHO.

                        I also happen to agree with Andrew - what we're looking for is much simpler than JEP.  The effort spent taking out the complexity we don't want is better spent building up what we DO want.  Parsers really aren't hard.  In fact, we may need to take complexity OUT of the (much simpler) parser I wrote (though we need to add handling of some additional characters inside quotes)

                        I also think a fork is actually worse, unless we get extensive test code along with his code.  If we have to spend years developing a test library, then it's better & faster to start from scratch so we can know what we are dealing with and only have to test the subset of what we want & develop (not what people could use if they knew JEP inside and out)

                        TP.
                        --
                        Tom Parker
                        thpr@... and tppublic@...

                        --- On Sun, 5/3/09, Martijn Verburg <martijnverburg@...> wrote:
                        From: Martijn Verburg <martijnverburg@...>
                        Subject: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                        To: pcgen_developers@yahoogroups.com
                        Date: Sunday, May 3, 2009, 4:49 AM


                        Hi All,

                        Searched for alternatives to JEP and also looked for Forks of that library, and
                        what do you know! I found a discussion between myself and their main developer
                        about forking the project.

                        It is a valid option if we want to do that as well.

                        Thoughts?

                        K

                        > Thanks, I'll have a look.
                        >
                        > andrew
                        >
                        > 2009/5/2 Tom Parker <thpr@...>:
                        > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@>
                        wrote:
                        > >> I'll dig it up and upload it into a sandbox.
                        > >
                        > > This is now loaded into sandbox/FormulaCompiler
                        > >
                        > > Yes, it's a bit of a heavy user of System.err at the moment,
                        it's still a bit broken, and the tests aren't *really* tests in the
                        jUnit sense, but this really wasn't originally intended for public
                        consumption...
                        > >
                        > > TP.
                        > >
                        > >
                        > >
                        > >
                        > > ------------------------------------
                        > >
                        > > Yahoo! Groups Links
                        > >
                        > >
                        > >
                        > >
                        >
                        > <div><br></div>
                        >




                        ------------------------------------

                        Yahoo! Groups Links

                        <*> To visit your group on the web, go to:
                        http://groups.yahoo.com/group/pcgen_developers/

                        <*> Your email settings:
                        Individual Email | Traditional

                        <*> To change settings online go to:
                        http://groups.yahoo.com/group/pcgen_developers/join
                        (Yahoo! ID required)

                        <*> To change settings via email:
                        mailto:pcgen_developers-digest@yahoogroups.com
                        mailto:pcgen_developers-fullfeatured@yahoogroups.com

                        <*> To unsubscribe from this group, send an email to:
                        pcgen_developers-unsubscribe@yahoogroups.com

                        <*> Your use of Yahoo! Groups is subject to:
                        http://docs.yahoo.com/info/terms/





                      • Tom Parker
                        ... Eric, Interesting find, thanks. I think there may be some things we can learn from this even if we can t use it (such as writing expressions back out to
                        Message 11 of 15 , May 3, 2009
                        • 0 Attachment
                          --- In pcgen_developers@yahoogroups.com, Eric Jarman <ejarman@...> wrote:
                          >
                          > Here's a BSD licensed expression parser:
                          > http://sourceforge.net/projects/expression-tree/
                          >

                          Eric,

                          Interesting find, thanks. I think there may be some things we can learn from this even if we can't use it (such as writing expressions back out to Strings).

                          My biggest concern with this design is that it is one big method (lots of ifs, etc.) that does the work.

                          This is fragile (consider how many times specific operators like '+' appear in the method and how hard it thus is to change). It also seems variable names with numbers are not supported (potentially a problem for us, and also subject to the 'fragile method' problem).

                          I think the test cases are useful for us to use as a demonstration, but would be reluctant to recommend this vs. the use of a parser generated by JJTree/JavaCC.

                          I think Andrew should take a look at this too, however, as I think he'll be leading up the formula work.

                          TP.
                        • Andrew Wilson
                          ... I glanced at it but it hasn t had any work done for several years. I d assumed it has similar problems to JEP so I didn t pursue it. I ll have another
                          Message 12 of 15 , May 4, 2009
                          • 0 Attachment
                            2009/5/3 Tom Parker <thpr@...>:
                            > I think Andrew should take a look at this too, however, as I think
                            > he'll be leading up the formula work.

                            I glanced at it but it hasn't had any work done for several years.
                            I'd assumed it has similar problems to JEP so I didn't pursue
                            it. I'll have another look.

                            andrew
                          • Tom Parker
                            I share the time lag concern, but at least useful to look at the archictectural decisions since one of our options is starting fro scratch TP. -- Tom Parker
                            Message 13 of 15 , May 4, 2009
                            • 0 Attachment
                              I share the time lag concern, but at least useful to look at the archictectural decisions since one of our options is "starting fro scratch"

                              TP.
                              --
                              Tom Parker
                              thpr@... and tppublic@...

                              --- On Mon, 5/4/09, Andrew Wilson <affable@...> wrote:
                              From: Andrew Wilson <affable@...>
                              Subject: Re: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                              To: pcgen_developers@yahoogroups.com
                              Date: Monday, May 4, 2009, 6:31 AM

                              2009/5/3 Tom Parker <thpr@...>:
                              > I think Andrew should take a look at this too, however, as I think
                              > he'll be leading up the formula work.

                              I glanced at it but it hasn't had any work done for several years.
                              I'd assumed it has similar problems to JEP so I didn't pursue
                              it. I'll have another look.

                              andrew


                              ------------------------------------

                              Yahoo! Groups Links

                              <*> To visit your group on the web, go to:
                              http://groups.yahoo.com/group/pcgen_developers/

                              <*> Your email settings:
                              Individual Email | Traditional

                              <*> To change settings online go to:
                              http://groups.yahoo.com/group/pcgen_developers/join
                              (Yahoo! ID required)

                              <*> To change settings via email:
                              mailto:pcgen_developers-digest@yahoogroups.com
                              mailto:pcgen_developers-fullfeatured@yahoogroups.com

                              <*> To unsubscribe from this group, send an email to:
                              pcgen_developers-unsubscribe@yahoogroups.com

                              <*> Your use of Yahoo! Groups is subject to:
                              http://docs.yahoo.com/info/terms/


                            Your message has been successfully submitted and would be delivered to recipients shortly.