Loading ...
Sorry, an error occurred while loading the content.

Re: [PCGD] Code Meeting April 2009 - Log

Expand Messages
  • James Dempsey
    Hi Andrew, On 29/04/2009 10:52 PM Andrew Wilson wrote ... Oh that s no good! I d just assumed you d fallen asleep - it was well early for you by that stage. At
    Message 1 of 15 , Apr 29 6:18 AM
    View Source
    • 0 Attachment
      Hi Andrew,

      On 29/04/2009 10:52 PM Andrew Wilson wrote
      > 2009/4/29 James Dempsey <jdempsey@...>:
      >
      >> [13:32] [Code_SB]james: any others?
      >> [13:34] [Code_SB]james: I'm assuming that Tom is focussed on CDOM and
      >> Connor on UI. Andrew anything you want to work on there or otherwise?
      >> [13:38] [Code_SB]james: k, I'll take that as the main focus will be UI
      >> and CDOM, which admittedly is plenty to do. If anyone wants to add
      >> anything to that, just post to the dev list.
      >> [13:39] [Code_SB]james: 5. Anything else anyone wants to raise
      >> [13:40] [Code_SB]james: I'm assuming that everyone has raised what
      >> they want to at this stage
      >> [13:40] [Code_SB]james: So unless there are objections I'll declare
      >> the meeting closed
      >>
      >
      > Unhelpfully, my computer totally died at this point and everyone had left
      > the meeting by the time I got back in. This was the bit of the meeting
      > I was particularly interested in.
      >
      >
      Oh that's no good! I'd just assumed you'd fallen asleep - it was well
      early for you by that stage. At any rate I had to get back to work - was
      starting to get hectic again.
      > I'm intending to do some more work on formula processing, JEP, output
      > sheet stuff—which is all intermingled. I asked the question: what are the
      > plans for replacing the output system? Does anyone have strong ideas
      > and are we intending to make the current output tokens go away or will
      > we be using them with the new system? But I don't think it got as far as
      > the channel. Can anyojne spread some light on the long term plans
      > here?
      >
      >
      It's rather late here so I'll just give a quick overview now. My plan is
      to look at either the Velocity or Freemarker templating engines as a
      replacement for the overall output structure. At this stage I am leaning
      towards Freemarker as it is the more capable of the two. This would be a
      full replacement for our branching and looping code, which is rather
      complex. We would then need to create a hook via which to call the
      export tokens. The tokens themselves would most likely remain unchanged.

      PS: I've cross-posted to the new list - we should start migrating our
      discussions there now.

      Cheers,
      James.
    • Martijn Verburg
      Hi Andrew, Also x-posting from old list (I ll continue here in future) As a formerly active OS monkey (and now not so active), I d love to see an overhaul of
      Message 2 of 15 , Apr 29 6:33 AM
      View Source
      • 0 Attachment
        Hi Andrew,

        Also x-posting from old list (I'll continue here in future)

        As a formerly active OS monkey (and now not so active), I'd love to see an overhaul of the way we do OS.

        > I'm intending to do some more work on formula processing, JEP, output
        > sheet stuff—which is all intermingled. I asked the question: what are the
        > plans for replacing the output system? Does anyone have strong ideas
        > and are we intending to make the current output tokens go away or will
        > we be using them with the new system? But I don't think it got as far as
        > the channel. Can anyone spread some light on the long term plans
        > here?

        My take (technology agnostic at this stage). We have lots of different Output Sheets, it is very difficult to update them because:

        1.) There are many of them, and we have to update the common stuff in every single one of them
        1.b.) We have them in different technologies (e.g. TXT, HTML, PDF), so we can't do something in one place and have it automatically filter through.

        2.) We don't know what tokens/data is actually appropriate for each OS

        3.) We don't always have available to us the data we need as a OS token
        3.b) The state of the OS tokens are much like the LST tokens before the Tom lead clean up, in that they're inconsistent & not 100% documented

        So to resolve those points I'd like to see a unified templating technology that we can use to build OS's (important that we can have 'includes') and have that translated into the output technology of choice (HTML, PDF, TXT, whatever). I think that Velocity has been mentioned by James in the past, I'm sure that there are others out there.

        Tom also had some very alpha thoughts on the OS Token clean up (IIRC), so he'll hopefully comment here. At a guess the OS tokens either need a clean up (ala the LST tokens). I'm assuming at this stage that an OS token system is still the preferred mechanism.

        If that all sounds to grandiose, I'd settle for the for a consistency and documentation cleanup, there are a stack of freqs which cover this and I'm sure a formal proposal can be hashed out (ala the LST token arch notes that Tom had).

        Andrew, I'd be delighted if you could even look into the smaller aspects of this, so thanks from me already :)

        K
      • Tom Parker
        ... My primary issue with the output tokens is that some require the core, and others are pure plugins.  100% pure plugins has to be the target - and no more
        Message 3 of 15 , Apr 29 8:50 AM
        View Source
        • 0 Attachment
          --- In pcgen_developers@yahoogroups.com, "Martijn Verburg" <martijnverburg@...> wrote:
          > Tom also had some very alpha thoughts on the OS Token clean up (IIRC),
          > so he'll hopefully comment here. At a guess the OS tokens either need a
          > clean up (ala the LST tokens). I'm assuming at this stage that an OS token
          > system is still the preferred mechanism.

          My primary issue with the output tokens is that some require the core, and others are pure plugins.  100% pure plugins has to be the target - and no more switch statements - they should have subtokens like the current LST token system.

          Having said that, the architecture needs to take into account the differences that a templating system like Velocity would introduce, and the thoughts above may not be complete in that respect.

          Andrew Wilson:
          > I'm
          intending to do some more work on formula processing,
          > JEP, output sheet stuff—which is all intermingled.

          Can you help on the scope of what you would be doing with formula processing and JEP? I would like to rip it out eventually, due to it being a license exception combined with concerns that the new version is closed source, leaving us with a stagnant code library.

          We're also in a position where we're still parsing "after the fact" (don't check at LST load) and there is significant work that could be done with a (better) formula system to accelerate formula processing.


          One of the root reasons for using the Formula interface as defined with the resolve() methods is to isolate JEP and thus allow swapping it out in the future without wild quantities of pain.

          TP.


        • Andrew Wilson
          ... At the moment, any formula can contain any OS token. The way that formulae get chopped up into btis we can evaluate is very convoluted and doesn t follow
          Message 4 of 15 , Apr 29 4:36 PM
          View Source
          • 0 Attachment
            2009/4/29 Tom Parker <thpr@...>:
            > --- In pcgen_developers@yahoogroups.com, "Martijn Verburg"
            > <martijnverburg@...> wrote:
            >> Tom also had some very alpha thoughts on the OS Token clean up (IIRC),
            >> so he'll hopefully comment here. At a guess the OS tokens either need a
            >> clean up (ala the LST tokens). I'm assuming at this stage that an OS token
            >> system is still the preferred mechanism.
            >
            > My primary issue with the output tokens is that some require the core, and
            > others are pure plugins.  100% pure plugins has to be the target - and no
            > more switch statements - they should have subtokens like the current LST
            > token system.
            >
            > Having said that, the architecture needs to take into account the
            > differences that a templating system like Velocity would introduce, and the
            > thoughts above may not be complete in that respect.

            At the moment, any formula can contain any OS token. The way that
            formulae get chopped up into btis we can evaluate is very convoluted
            and doesn't follow standard mathematical precedence.

            > Andrew Wilson:
            >> I'm intending to do some more work on formula processing,
            >> JEP, output sheet stuff—which is all intermingled.
            >
            > Can you help on the scope of what you would be doing with formula processing
            > and JEP? I would like to rip it out eventually, due to it being a license
            > exception combined with concerns that the new version is closed source,
            > leaving us with a stagnant code library.

            I'd like to rip it out too—because the version we're using is mostly
            abandoned and stagnating, and also because it has far more features
            than we need and they just get in our way. I was under the impression
            that, as a project, we wanted embrace more of JEP rather than rip it out.
            My plans involve replacing the "old parser" with something that can
            (a) recognise the bits that make up a PCGEN formula, (b) follow
            mathematical precedence for operators and (c) not involve tens
            (hundreds) of recursive calls every time we evaluate a formula.

            This would need to be able to Isolate the output tokens, the hardcoded
            variabes (current term objects), LST variables, JEP functions and
            operators. Everything except the operators then gets something
            resembling a term object. I'd envisaged ending up with each formula
            parsed once into a tree of operators and term objects that can be
            evaluated on demand.

            Since I thought the project wanted to keep JEP, I was planning to use
            it (or bits of it anyway) to derive the tree. Now I think I wouldn't bother to
            use it at all.

            > We're also in a position where we're still parsing "after the fact" (don't
            > check at LST load) and there is significant work that could be done with a
            > (better) formula system to accelerate formula processing.

            This could cope with that. I wasn't planning to do it right now, but once
            we have something that isolates the terms and operators, we can write
            validators for the terms. This could then be called on LST load and we'd
            know if we have valid formulae.

            > One of the root reasons for using the Formula interface as defined with the
            > resolve() methods is to isolate JEP and thus allow swapping it out in the
            > future without wild quantities of pain.

            I'm all for avoiding massive amounts of pain.

            To be honest this cycle I think I'd only have three new term objects one that
            know how to look up a LST variable, one that knows how to evaluate an OS
            token and one that can process a JEP function. I think we can eliminate
            the multiple recursive calls to the old parser and JEP on each formula
            evaluation. The overall formula would be parsed once. The OS tokens and
            poossibly JEP functions would be parsed every time they're evaluated.
            If we get that far we can eliminate all of JEP, except the bits we added to it.

            Does that sound compatible with your vision for the architecture?

            andrew
          • Tom Parker
            ... I was under the impression that, as a project, we wanted embrace more of JEP rather than rip it out. In my opinion, that was true until JEP went closed
            Message 5 of 15 , Apr 30 1:46 PM
            View Source
            • 0 Attachment

              --- On Wed, 4/29/09, Andrew Wilson <andrew@...> wrote:
              I was under the impression
              that, as a project, we wanted embrace more of JEP rather than rip it out.

              In my opinion, that was true until JEP went closed source.  To my recollection, no deep discussion took place on the consequences of that JEP change to our long-term formula strategy.  I think it's been a matter of priorities.  Thus I believe the issue is open and discussion on that aspect of our strategy is fair game.

              While I hate reinventing wheels without *really* good reasons and while there is certainly risk in moving to our own solution (since it introduces more code for us by eliminating a library), those may be outweighted by the risk in remaining on JEP (unsupported, now closed source, library).

              My plans involve replacing the "old parser" with something that can
              (a) recognise the bits that make up a PCGEN formula, (b) follow
              mathematical precedence for operators and (c) not involve tens
              (hundreds) of recursive calls every time we evaluate a formula.

              I would add:
              (d) and complain at LST load (vs at runtime) if a formula is invalid (you had addressed this later on in your note), which means it also has the ability to detect and warn at LST load for deprecated formula styles, e.g. 2MAXINT
              and
              (e) Consider whether we have a formula factory that can detect situations where a formula like 1+INT is used multiple places in the LST.  These should be reduced via a cache to a single Formula object (since a Formula is immutable) to save memory and hopefully reduce LST load time.

              I'd also note that (c) is difficult, since some of that recursiveness is necessary based on our BONUS structure.  So while I agree that's part of the vision, whether it's part of scope in 5.17 might need to be up for discussion (I defer to you on evaluating/defining that scope)

              Same for (e) - certainly doesn't need to be an immediate step in 5.17. 

              To be honest this cycle I think I'd only have three new term objects one
              that
              know how to look up a LST variable, one that knows how to evaluate an OS
              token and one that can process a JEP function.

              For clarity - how would:
              1+MYVAR

              be evaluated with the 3 terms you describe above?  Specifically, is the goal for 5.17 still to have JEP doing the evaluation of the +? 

              I think we can eliminate
              the multiple recursive calls to the old parser and JEP on each formula
              evaluation.

              Good, and I agree this is the right path.  Just make sure we have some unit tests for dependent items, e.g.:
              DEFINE:MYVAR|5
              BONUS:VAR|MYVAR|3
              DEFINE:OTHERVAR|MYVAR+2

              (there are some internal variables that are even worse)

               The overall formula would be parsed once.

              This is how I noticed the SCORE replacement (specifically the String replacement in StatList's getModForNumber(int, int) for those who want to look) which makes me wonder how often we can only parse once vs. the formula being string modified before it goes into the evaluator.  I know equipment (and equipment modifiers especially) are ugly for String replacement right before formula evaluation.

              I like your term system, but as I've gone through the core, there are places we need to expand what is a term vs. what is string replacement.  Not to be addressed as part of this conversation, but awareness to folks that faster runtime performance on formulas doesn't solve all of our problems in one shot.  Also need to be conscious with our own parser that we don't change the rules to the point where the replacement would break.

              If we get that far we can eliminate all of JEP, except the bits we added to it.

              This is where I get lost and what triggered the question on + above.

              Does that sound compatible with your vision for the architecture?

              Sounds right on, with the caveat that I'd like to comprehend what is doing the mathematical operator work to produce the tree in the first place (new code?).

              ...

              Having said that, under the assumption that we're going to be developing a parser for ourselves... and thus having to write something to do evaluation of the tree the parser produces as well as testing the parser and evaluator... then I have an idea I would like to float (or sink, as the case may be).

              So this is Tom as code contributor (not even Code Chimp, since James might demote me for something this reckless :D ), not Tom in Architecture Silverback mode:

              For various reasons, I wrote a formula parser some years ago (2004-2005).  It uses JJTree to produce a .jj file which is then compiled to .java files through javacc.  Connor recently did work to eliminate javacc, which is probably controversy #1 in reintroducing derived code and javacc as an ant task (Though I believe it's the most reliable way to produce a parser, so if we go our own way from JEP, this should be the path)

              A portion of the project at the time was learning the inner workings of compilers, Java and JavaCC, so I was also playing with ASM (potential controversy #2, adding yet another library)

              What I produced is code that can take a formula (e.g. "5+T") and compile it directly into Java bytecode (and then load that bytecode as a class).  This usually raises hair on the back of people's necks, because dynamic code is considered heresy under certain design rules (controversy #3).

              Hey, at least it's not self-modifying code.  I could have done that :P

              I dropped it as a project about the time I joined PCGen, but it does work for many of the basics (perhaps enough for PCGen?) and can even simplify constants like 1+3 so the resolve method is simply load 4 onto the stack and return.

              By compiling to bytecode it is extremely fast (some items - terms in Andrew's parlance - simply become a method call to another Formula, while variables can be referred to through a resolving object) at runtime.

              I haven't looked at it in a while, so it may be a bit of a pain to integrate with your Term system, but I'm happy to hand it over to you (and others) to work on, so it's an option we can evaluate if people don't think it's completely nuts.

              Note also for any non-JEP system, we produce the complication that the existing functions (e.g. count) we have would need to be rewritten into the new system vs the JEP functions we have today.

              TP.
              --
              Tom Parker
              thpr@... and tppublic@...


            • Andrew Wilson
              ... Yeah, I see those as separate formulae. What I really hate is the way we seem to need multiple calls to and fro between JEP and the old parser. ... I see
              Message 6 of 15 , May 2, 2009
              View Source
              • 0 Attachment
                2009/4/30 Tom Parker <thpr@...>:
                > I would add:
                > (d) and complain at LST load (vs at runtime) if a formula is invalid (you
                > had addressed this later on in your note), which means it also has the
                > ability to detect and warn at LST load for deprecated formula styles, e.g.
                > 2MAXINT
                > and
                > (e) Consider whether we have a formula factory that can detect situations
                > where a formula like 1+INT is used multiple places in the LST.  These should
                > be reduced via a cache to a single Formula object (since a Formula is
                > immutable) to save memory and hopefully reduce LST load time.
                >
                > I'd also note that (c) is difficult, since some of that recursiveness is
                > necessary based on our BONUS structure.  So while I agree that's part of the
                > vision, whether it's part of scope in 5.17 might need to be up for
                > discussion (I defer to you on evaluating/defining that scope)

                Yeah, I see those as separate formulae. What I really hate is the way we
                seem to need multiple calls to and fro between JEP and the old parser.

                > To be honest this cycle I think I'd only have three new term objects one
                > that
                > know how to look up a LST variable, one that knows how to evaluate an OS
                > token and one that can process a JEP function.
                >
                > For clarity - how would:
                > 1+MYVAR
                >
                > be evaluated with the 3 terms you describe above?  Specifically, is the goal
                > for 5.17 still to have JEP doing the evaluation of the +?

                I see that as two terms the 1 is a constant that evaluates to itself, the MYVAR
                would use the variable lookup term. Then the formula evaluator would
                walk the tree,
                evaluating and combining as it goes.

                > I think we can eliminate
                > the multiple recursive calls to the old parser and JEP on each formula
                > evaluation.
                >
                > Good, and I agree this is the right path.  Just make sure we have some unit
                > tests for dependent items, e.g.:
                > DEFINE:MYVAR|5
                > BONUS:VAR|MYVAR|3
                > DEFINE:OTHERVAR|MYVAR+2
                >
                > (there are some internal variables that are even worse)

                noted.

                > The overall formula would be parsed once.
                >
                > This is how I noticed the SCORE replacement (specifically the String
                > replacement in StatList's getModForNumber(int, int) for those who want to
                > look) which makes me wonder how often we can only parse once vs. the formula
                > being string modified before it goes into the evaluator.  I know equipment
                > (and equipment modifiers especially) are ugly for String replacement right
                > before formula evaluation.
                >
                > I like your term system, but as I've gone through the core, there are places
                > we need to expand what is a term vs. what is string replacement.  Not to be
                > addressed as part of this conversation, but awareness to folks that faster
                > runtime performance on formulas doesn't solve all of our problems in one
                > shot.  Also need to be conscious with our own parser that we don't change
                > the rules to the point where the replacement would break.
                >
                > If we get that far we can eliminate all of JEP, except the bits we added to
                > it.
                >
                > This is where I get lost and what triggered the question on + above.

                When I formulated my original ideas I was planning to use JEP to
                do the parsing and evaluating. The caveat was that the formula
                was going to need heavily preprocessed before handing to JEP.
                I was planning to take something like:

                1+CL=monkey (code)/(CLASS.0.LEVEL*MYVAR)

                and turn that into

                1+VAR001/(VAR002*VAR003)

                then when JEP processed it, it'd find 3 variables

                VAR001 => CL=monkey (code)
                VAR002 => CLASS.0.LEVEL
                VAR003 => MYVAR

                I was then planning to hijack JEP's symbol lookup so that when it looked for a
                value for for VAR001 it found a term object that knows how to evaluate CL=
                VAR002 is a term object that can look up an output tag and VAR003 a term
                object that looks up a LST variable.

                Now I'm not so sure that's the best option if we are planning to ditch
                JEP anyway, then I don't need to bolt my system onto both sides of it.
                I think I'd like to look into whether we can carve up the current JEP
                system and maybe only pinch bits of the parser and get it to build
                a tree of term objects. Using bits of JEP will probably mean we can
                integrate it's function handling. One of the major obstacles is the
                use of things that should be reserved characters but aren't, () are
                nasty. classlevel("code-monkey") means you can't just split on
                operators, 20/(5*classlevel("monkey (code)")) actually has three
                different uses of parens. 20/(5*somefunction(")")) actually does
                have balanced parens. Parsing this is not a trivial problem,
                but we need to deal with it at some point.

                > Does that sound compatible with your vision for the architecture?
                >
                > Sounds right on, with the caveat that I'd like to comprehend what is doing
                > the mathematical operator work to produce the tree in the first place (new
                > code?).

                Probably new code, possibly heavily influenced by JEP's parser.

                > Having said that, under the assumption that we're going to be developing a
                > parser for ourselves... and thus having to write something to do evaluation
                > of the tree the parser produces as well as testing the parser and
                > evaluator... then I have an idea I would like to float (or sink, as the case
                > may be).
                >
                > So this is Tom as code contributor (not even Code Chimp, since James might
                > demote me for something this reckless :D ), not Tom in Architecture
                > Silverback mode:
                >
                > For various reasons, I wrote a formula parser some years ago (2004-2005).
                > It uses JJTree to produce a .jj file which is then compiled to .java files
                > through javacc.  Connor recently did work to eliminate javacc, which is
                > probably controversy #1 in reintroducing derived code and javacc as an ant
                > task (Though I believe it's the most reliable way to produce a parser, so if
                > we go our own way from JEP, this should be the path)
                >
                > A portion of the project at the time was learning the inner workings of
                > compilers, Java and JavaCC, so I was also playing with ASM (potential
                > controversy #2, adding yet another library)
                >
                > What I produced is code that can take a formula (e.g. "5+T") and compile it
                > directly into Java bytecode (and then load that bytecode as a class).  This
                > usually raises hair on the back of people's necks, because dynamic code is
                > considered heresy under certain design rules (controversy #3).
                >
                > Hey, at least it's not self-modifying code.  I could have done that :P
                >
                > I dropped it as a project about the time I joined PCGen, but it does work
                > for many of the basics (perhaps enough for PCGen?) and can even simplify
                > constants like 1+3 so the resolve method is simply load 4 onto the stack and
                > return.
                >
                > By compiling to bytecode it is extremely fast (some items - terms in
                > Andrew's parlance - simply become a method call to another Formula, while
                > variables can be referred to through a resolving object) at runtime.
                >
                > I haven't looked at it in a while, so it may be a bit of a pain to integrate
                > with your Term system, but I'm happy to hand it over to you (and others) to
                > work on, so it's an option we can evaluate if people don't think it's
                > completely nuts.

                I'd certainly like to take a look at it. It sounds horrendously complex and
                to be honest I think this problem is complex enough already, but as I said
                I'd be interested in seeing it.

                > Note also for any non-JEP system, we produce the complication that the
                > existing functions (e.g. count) we have would need to be rewritten into the
                > new system vs the JEP functions we have today.

                Yes, I know. I wrote the count function it only really works for Abilty
                objects at the moment. If we can get a system that understands our
                hardcoded stuff like COUNT[] then we can probably implement term objects
                to handle that and deprecate the count() function. There are other JEP
                functions that are useful. var() would become obsolete, since var("MYVAR")
                and MYVAR should do identical things (the former might be less efficient).

                andrew
              • Tom Parker
                ... One of the major obstacles is the use of things that should be reserved characters but aren t, () are nasty. I don t think this is as nasty as you think it
                Message 7 of 15 , May 2, 2009
                View Source
                • 0 Attachment

                  --- On Sat, 5/2/09, Andrew Wilson <affable@...> wrote:
                  One of the major obstacles is the
                  use of things that should be reserved characters but aren't, () are
                  nasty.

                  I don't think this is as nasty as you think it is... as long as they can be distinguished, it will work in a parser system like jjtree/javacc.  I've written a few compilers (not the depth of complexity of modern languages, but that type of thing), and it's not bad, and arguably easier than how we separate tokens in the LST system today.

                  I'd certainly like to take a look at it.  It sounds horrendously
                   complex
                  and
                  to be honest I think this problem is complex enough already, but as I said
                  I'd be interested in seeing it.

                  Complexity is relative.  I think it actually transfers a lot of complexity into javacc.  The main complexity is mostly in the optimization (making 1+3 into 4), which we could really tear out if it is too confusing or too complex.  With that removed, you may be surprised at how simple a parser/compiler really is.

                  I'll dig it up and upload it into a sandbox.

                  TP.
                  --
                  Tom Parker
                  thpr@... and tppublic@...


                • Tom Parker
                  ... This is now loaded into sandbox/FormulaCompiler Yes, it s a bit of a heavy user of System.err at the moment, it s still a bit broken, and the tests aren t
                  Message 8 of 15 , May 2, 2009
                  View Source
                  • 0 Attachment
                    --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@...> wrote:
                    > I'll dig it up and upload it into a sandbox.

                    This is now loaded into sandbox/FormulaCompiler

                    Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...

                    TP.
                  • Andrew Wilson
                    Thanks, I ll have a look. andrew ...
                    Message 9 of 15 , May 2, 2009
                    View Source
                    • 0 Attachment
                      Thanks, I'll have a look.

                      andrew

                      2009/5/2 Tom Parker <thpr@...>:
                      > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@...> wrote:
                      >> I'll dig it up and upload it into a sandbox.
                      >
                      > This is now loaded into sandbox/FormulaCompiler
                      >
                      > Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...
                      >
                      > TP.
                      >
                      >
                      >
                      >
                      > ------------------------------------
                      >
                      > Yahoo! Groups Links
                      >
                      >
                      >
                      >

                      <div><br></div>
                    • Martijn Verburg
                      Hi All, Searched for alternatives to JEP and also looked for Forks of that library, and what do you know! I found a discussion between myself and their main
                      Message 10 of 15 , May 3, 2009
                      View Source
                      • 0 Attachment
                        Hi All,

                        Searched for alternatives to JEP and also looked for Forks of that library, and what do you know! I found a discussion between myself and their main developer about forking the project.

                        It is a valid option if we want to do that as well.

                        Thoughts?

                        K

                        > Thanks, I'll have a look.
                        >
                        > andrew
                        >
                        > 2009/5/2 Tom Parker <thpr@...>:
                        > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@> wrote:
                        > >> I'll dig it up and upload it into a sandbox.
                        > >
                        > > This is now loaded into sandbox/FormulaCompiler
                        > >
                        > > Yes, it's a bit of a heavy user of System.err at the moment, it's still a bit broken, and the tests aren't *really* tests in the jUnit sense, but this really wasn't originally intended for public consumption...
                        > >
                        > > TP.
                        > >
                        > >
                        > >
                        > >
                        > > ------------------------------------
                        > >
                        > > Yahoo! Groups Links
                        > >
                        > >
                        > >
                        > >
                        >
                        > <div><br></div>
                        >
                      • Tom Parker
                        My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His
                        Message 11 of 15 , May 3, 2009
                        View Source
                        • 0 Attachment
                          My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His project is GPL2, which causes problems for us.  We are using Apache libraries that are not compatible with GPL2. 

                          I would strongly vote against any fork which was still operating under an exception - there is *way* to much risk involved since it starts to impact our ability to use code we would have to develop/modify.  The fork would have to be LGPL or Apache licensed (he would have to explicitly license it that way in a public forum, not just in an email to us) or it's a non-starter IMHO.

                          I also happen to agree with Andrew - what we're looking for is much simpler than JEP.  The effort spent taking out the complexity we don't want is better spent building up what we DO want.  Parsers really aren't hard.  In fact, we may need to take complexity OUT of the (much simpler) parser I wrote (though we need to add handling of some additional characters inside quotes)

                          I also think a fork is actually worse, unless we get extensive test code along with his code.  If we have to spend years developing a test library, then it's better & faster to start from scratch so we can know what we are dealing with and only have to test the subset of what we want & develop (not what people could use if they knew JEP inside and out)

                          TP.
                          --
                          Tom Parker
                          thpr@... and tppublic@...

                          --- On Sun, 5/3/09, Martijn Verburg <martijnverburg@...> wrote:
                          From: Martijn Verburg <martijnverburg@...>
                          Subject: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                          To: pcgen_developers@yahoogroups.com
                          Date: Sunday, May 3, 2009, 4:49 AM

                          Hi All,

                          Searched for alternatives to JEP and also looked for Forks of that library, and
                          what do you know! I found a discussion between myself and their main developer
                          about forking the project.

                          It is a valid option if we want to do that as well.

                          Thoughts?

                          K

                          > Thanks, I'll have a look.
                          >
                          > andrew
                          >
                          > 2009/5/2 Tom Parker <thpr@...>:
                          > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@>
                          wrote:
                          > >> I'll dig it up and upload it into a sandbox.
                          > >
                          > > This is now loaded into sandbox/FormulaCompiler
                          > >
                          > > Yes, it's a bit of a heavy user of System.err at the moment,
                          it's still a bit broken, and the tests aren't *really* tests in the
                          jUnit sense, but this really wasn't originally intended for public
                          consumption...
                          > >
                          > > TP.
                          > >
                          > >
                          > >
                          > >
                          > > ------------------------------------
                          > >
                          > > Yahoo! Groups Links
                          > >
                          > >
                          > >
                          > >
                          >
                          > <div><br></div>
                          >




                          ------------------------------------

                          Yahoo! Groups Links

                          <*> To visit your group on the web, go to:
                          http://groups.yahoo.com/group/pcgen_developers/

                          <*> Your email settings:
                          Individual Email | Traditional

                          <*> To change settings online go to:
                          http://groups.yahoo.com/group/pcgen_developers/join
                          (Yahoo! ID required)

                          <*> To change settings via email:
                          mailto:pcgen_developers-digest@yahoogroups.com
                          mailto:pcgen_developers-fullfeatured@yahoogroups.com

                          <*> To unsubscribe from this group, send an email to:
                          pcgen_developers-unsubscribe@yahoogroups.com

                          <*> Your use of Yahoo! Groups is subject to:
                          http://docs.yahoo.com/info/terms/


                        • Eric Jarman
                          Here s a BSD licensed expression parser: http://sourceforge.net/projects/expression-tree/
                          Message 12 of 15 , May 3, 2009
                          View Source
                          • 0 Attachment
                            Here's a BSD licensed expression parser:  http://sourceforge.net/projects/expression-tree/


                            On Sun, May 3, 2009 at 11:36 AM, Tom Parker <thpr@...> wrote:


                            My biggest concern with JEP is that we are operating under a license exception.  We would need to be *very* clear on how licensing would work in a fork - His project is GPL2, which causes problems for us.  We are using Apache libraries that are not compatible with GPL2. 

                            I would strongly vote against any fork which was still operating under an exception - there is *way* to much risk involved since it starts to impact our ability to use code we would have to develop/modify.  The fork would have to be LGPL or Apache licensed (he would have to explicitly license it that way in a public forum, not just in an email to us) or it's a non-starter IMHO.

                            I also happen to agree with Andrew - what we're looking for is much simpler than JEP.  The effort spent taking out the complexity we don't want is better spent building up what we DO want.  Parsers really aren't hard.  In fact, we may need to take complexity OUT of the (much simpler) parser I wrote (though we need to add handling of some additional characters inside quotes)

                            I also think a fork is actually worse, unless we get extensive test code along with his code.  If we have to spend years developing a test library, then it's better & faster to start from scratch so we can know what we are dealing with and only have to test the subset of what we want & develop (not what people could use if they knew JEP inside and out)

                            TP.
                            --
                            Tom Parker
                            thpr@... and tppublic@...

                            --- On Sun, 5/3/09, Martijn Verburg <martijnverburg@...> wrote:
                            From: Martijn Verburg <martijnverburg@...>
                            Subject: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                            To: pcgen_developers@yahoogroups.com
                            Date: Sunday, May 3, 2009, 4:49 AM


                            Hi All,

                            Searched for alternatives to JEP and also looked for Forks of that library, and
                            what do you know! I found a discussion between myself and their main developer
                            about forking the project.

                            It is a valid option if we want to do that as well.

                            Thoughts?

                            K

                            > Thanks, I'll have a look.
                            >
                            > andrew
                            >
                            > 2009/5/2 Tom Parker <thpr@...>:
                            > > --- In pcgen_developers@yahoogroups.com, Tom Parker <thpr@>
                            wrote:
                            > >> I'll dig it up and upload it into a sandbox.
                            > >
                            > > This is now loaded into sandbox/FormulaCompiler
                            > >
                            > > Yes, it's a bit of a heavy user of System.err at the moment,
                            it's still a bit broken, and the tests aren't *really* tests in the
                            jUnit sense, but this really wasn't originally intended for public
                            consumption...
                            > >
                            > > TP.
                            > >
                            > >
                            > >
                            > >
                            > > ------------------------------------
                            > >
                            > > Yahoo! Groups Links
                            > >
                            > >
                            > >
                            > >
                            >
                            > <div><br></div>
                            >




                            ------------------------------------

                            Yahoo! Groups Links

                            <*> To visit your group on the web, go to:
                            http://groups.yahoo.com/group/pcgen_developers/

                            <*> Your email settings:
                            Individual Email | Traditional

                            <*> To change settings online go to:
                            http://groups.yahoo.com/group/pcgen_developers/join
                            (Yahoo! ID required)

                            <*> To change settings via email:
                            mailto:pcgen_developers-digest@yahoogroups.com
                            mailto:pcgen_developers-fullfeatured@yahoogroups.com

                            <*> To unsubscribe from this group, send an email to:
                            pcgen_developers-unsubscribe@yahoogroups.com

                            <*> Your use of Yahoo! Groups is subject to:
                            http://docs.yahoo.com/info/terms/





                          • Tom Parker
                            ... Eric, Interesting find, thanks. I think there may be some things we can learn from this even if we can t use it (such as writing expressions back out to
                            Message 13 of 15 , May 3, 2009
                            View Source
                            • 0 Attachment
                              --- In pcgen_developers@yahoogroups.com, Eric Jarman <ejarman@...> wrote:
                              >
                              > Here's a BSD licensed expression parser:
                              > http://sourceforge.net/projects/expression-tree/
                              >

                              Eric,

                              Interesting find, thanks. I think there may be some things we can learn from this even if we can't use it (such as writing expressions back out to Strings).

                              My biggest concern with this design is that it is one big method (lots of ifs, etc.) that does the work.

                              This is fragile (consider how many times specific operators like '+' appear in the method and how hard it thus is to change). It also seems variable names with numbers are not supported (potentially a problem for us, and also subject to the 'fragile method' problem).

                              I think the test cases are useful for us to use as a demonstration, but would be reluctant to recommend this vs. the use of a parser generated by JJTree/JavaCC.

                              I think Andrew should take a look at this too, however, as I think he'll be leading up the formula work.

                              TP.
                            • Andrew Wilson
                              ... I glanced at it but it hasn t had any work done for several years. I d assumed it has similar problems to JEP so I didn t pursue it. I ll have another
                              Message 14 of 15 , May 4, 2009
                              View Source
                              • 0 Attachment
                                2009/5/3 Tom Parker <thpr@...>:
                                > I think Andrew should take a look at this too, however, as I think
                                > he'll be leading up the formula work.

                                I glanced at it but it hasn't had any work done for several years.
                                I'd assumed it has similar problems to JEP so I didn't pursue
                                it. I'll have another look.

                                andrew
                              • Tom Parker
                                I share the time lag concern, but at least useful to look at the archictectural decisions since one of our options is starting fro scratch TP. -- Tom Parker
                                Message 15 of 15 , May 4, 2009
                                View Source
                                • 0 Attachment
                                  I share the time lag concern, but at least useful to look at the archictectural decisions since one of our options is "starting fro scratch"

                                  TP.
                                  --
                                  Tom Parker
                                  thpr@... and tppublic@...

                                  --- On Mon, 5/4/09, Andrew Wilson <affable@...> wrote:
                                  From: Andrew Wilson <affable@...>
                                  Subject: Re: [pcgen_developers] Re: [PCGD] Code Meeting April 2009 - Log
                                  To: pcgen_developers@yahoogroups.com
                                  Date: Monday, May 4, 2009, 6:31 AM

                                  2009/5/3 Tom Parker <thpr@...>:
                                  > I think Andrew should take a look at this too, however, as I think
                                  > he'll be leading up the formula work.

                                  I glanced at it but it hasn't had any work done for several years.
                                  I'd assumed it has similar problems to JEP so I didn't pursue
                                  it. I'll have another look.

                                  andrew


                                  ------------------------------------

                                  Yahoo! Groups Links

                                  <*> To visit your group on the web, go to:
                                  http://groups.yahoo.com/group/pcgen_developers/

                                  <*> Your email settings:
                                  Individual Email | Traditional

                                  <*> To change settings online go to:
                                  http://groups.yahoo.com/group/pcgen_developers/join
                                  (Yahoo! ID required)

                                  <*> To change settings via email:
                                  mailto:pcgen_developers-digest@yahoogroups.com
                                  mailto:pcgen_developers-fullfeatured@yahoogroups.com

                                  <*> To unsubscribe from this group, send an email to:
                                  pcgen_developers-unsubscribe@yahoogroups.com

                                  <*> Your use of Yahoo! Groups is subject to:
                                  http://docs.yahoo.com/info/terms/


                                Your message has been successfully submitted and would be delivered to recipients shortly.