Loading ...
Sorry, an error occurred while loading the content.

Re: 2.7.2a4 ANTLR/C# - build ok, panic shutdown

Expand Messages
  • slightlynew
    Micheal, ... Sure. I m using NUnit 2.0. I write unit tests to exercise whatever production code I write. The unit tests are in a separate project, which I
    Message 1 of 14 , Dec 5, 2002
    • 0 Attachment
      Micheal,

      > Care to talk a little about these unit tests and the tools/harness
      > you employ for them. About your ANTLR-related testing in general in
      > fact.

      Sure. I'm using NUnit 2.0. I write unit tests to exercise whatever
      production code I write. The unit tests are in a separate project,
      which I find to be easiest for a number of reasons; for example,
      the production code doesn't have to ship with a dependency on the
      test framework.

      The unit tests' job is, of course, to catch errors and verify that
      the code is behaving correctly. But over time they grow into a
      comprehensive test suite that (usually) fails immediately when some
      aspect of the program is broken. Knowing right away whether
      something's been messed up is remarkably freeing to the developer. A
      good analogy is that of a safety net underneath a tightrope walker.
      You can go faster and try riskier things without fear. In
      particular, upgrading something from one version to the next becomes
      relatively low-risk and painless.

      In terms of ANTLR, the unit tests call my lexer and parser with
      various expressions, statements, and so on, verifying that the well-
      formed stuff is parsed as expected, while the invalid stuff generates
      the expected errors. I try to imagine weird things that might go
      wrong and then write tests that will fail in those cases. It's not
      complicated.

      Many people think that systematic unit testing means taking a
      productivity hit in exchange for greater quality, because of the time
      it takes to write the tests. But actually, in my experience, it
      *saves* time. That's because there's far less manually stepping
      through code, troubleshooting weird integration issues, etc.
      And, the time you're saving is largely in the drudgery aspects of
      programming. That means you spend a higher percentage of your time
      coding, which is the fun part of programming. So it turns out that
      systematic unit testing gives greater productivity, better quality,
      and more fun, all at the same time. I'm not sure I could claim that
      for any other programming practice I've tried.

      The picture is different with an already-existing large system with
      no unit tests. In that case, it may not make sense to go back and
      write, say, a unit test for every public method. But you can still
      use xUnit to write higher-level functional tests that give you good
      coverage, and many of the above benefits.

      - Daniel

      --- In antlr-interest@y..., "micheal_jor" <open.zone@v...> wrote:
      > Daniel,
      >
      > > Overriding panic() is certainly a workaround, and indeed was the
      > > first thing I did after discovering the problem. However, I
      don't
      > > really see that as the point. The important thing is whether a
      > > library should handle fatal errors by shutting down the calling
      > > application without warning!
      >
      > Not ideally but it works [ ;-) ], it's due for revision real soon
      > anyways.
      >
      > > A further problem with overriding panic() is that while you can
      do
      > > this for lexers, you cannot do it for parsers or tree parsers,
      > where
      > > it is a static method.
      > >
      > > By the way, Micheal, just for curiosity I also built the C#
      codegen
      > > with Antlr 2.7.2a6. I was able to do so quite easily, and my
      unit
      > > tests all passed without modification (unlike the upgrade from
      > > 2.7.2a2 to 2.7.2a4).
      >
      > I did mention that it should be possible. There are a few problems
      > with doing that (some of which you've already encountered but you
      can
      > certainly do useful work with it).
      >
      > Care to talk a little about these unit tests and the tools/harness
      > you employ for them. About your ANTLR-related testing in general in
      > fact.
      >
      > > In fact, the discrepancies between the C# and
      > > the main Antlr branch are now quite few, which bodes well for
      > merging
      > > the two streams.
      >
      > I'm not sure I follow you here. What discrepancies are you
      referring
      > to?
      >
      > > What I wasn't able to get working in Antlr 2.7.2a6 was that third
      > > parameter that Terence added to the AST constructor syntax: #
      > > [FOO, "Foo", "FooNode"] (or something like that!) I couldn't get
      > > that to work either with Java or C# (having ported Terence's
      > changes
      > > to the C# action.g). Antlr rejects the third parameter with a
      > syntax
      > > error saying "unexpected ','". Is there something else one needs
      > to
      > > do, to get Antlr to use the new action.g?
      >
      > Quite a bit more. ANTLR/C# 2.7.2a6 would debut as soon as it's all
      > done.
      >
      > Cheers,
      >
      > Micheal
    • Terence Parr
      Hi Daniel, Just tried this /tmp $ java antlr.Tool t.g ANTLR Parser Generator Version 2.7.2a6 (20021130-1) 1989-2002 jGuru.com /tmp $ cat t.g class P
      Message 2 of 14 , Dec 5, 2002
      • 0 Attachment
        Hi Daniel,

        Just tried this

        /tmp $ java antlr.Tool t.g
        ANTLR Parser Generator Version 2.7.2a6 (20021130-1) 1989-2002
        jGuru.com
        /tmp $ cat t.g
        class P extends Parser;

        a : A {#[a,b,c];}
        ;

        Perhaps you grabbed 11-29's version not the 30th :)

        Ter

        On Thursday, December 5, 2002, at 06:01 PM, slightlynew wrote:

        > Ter,
        >
        >> I updated this java/action.g rule [...]
        >> To allow the 3rd arg. Hmm....
        >
        > Oh no, that's definitely there. Let me try to express this more
        > clearly. I tried two separate things, one with Java Antlr and one
        > with C#. Neither worked.
        >
        > (1) For C#, I ported your changes to java/action.g to
        > csharp/action.g. In other words, I added your new AST_CONSTRUCTOR
        > rule to the C# version. This didn't work, but I didn't really expect
        > it to, because Micheal had already implied it wouldn't until the
        > 2.7.2a6 C# codegen comes out. And he has since confirmed that.
        >
        > (2) For Java, I didn't have to change action.g at all, because you'd
        > already done so. I was just trying to use it. I modified a grammar
        > to pass a third argument to an AST constructor, and then tried to run
        > java Antlr on it. It didn't work. In fact, I got the same syntax
        > error as with the C#: "unexpected ','". This was why I asked, "Is
        > there something else one needs to do, to get Antlr to use the new
        > action.g?" The answer is "yes" in the C# case because 2.7.2a6 isn't
        > out yet. But what about the Java case?
        >
        > Maybe there are some crossed wires and I wasn't really using the Java
        > 2.7.2a6 codegen... but I definitely changed my grammar to
        > say "Language = Java" instead of C#.
        >
        > - Daniel
        >
        > --- In antlr-interest@y..., Terence Parr <parrt@j...> wrote:
        >>
        >> On Wednesday, December 4, 2002, at 03:08 PM, slightlynew wrote:
        >>> What I wasn't able to get working in Antlr 2.7.2a6 was that third
        >>> parameter that Terence added to the AST constructor syntax: #
        >>> [FOO, "Foo", "FooNode"] (or something like that!) I couldn't get
        >>> that to work either with Java or C# (having ported Terence's
        > changes
        >>> to the C# action.g). Antlr rejects the third parameter with a
        > syntax
        >>> error saying "unexpected ','". Is there something else one needs
        > to
        >>> do, to get Antlr to use the new action.g?
        >>
        >> Hmm...the latest version should have that. I could have sworn I
        > tested
        >> that. I updated this java/action.g rule:
        >>
        >> protected
        >> AST_CONSTRUCTOR!
        >> : '[' (WS)? x:AST_CTOR_ELEMENT (WS)?
        >> (',' (WS)? y:AST_CTOR_ELEMENT (WS)? )?
        >> (',' (WS)? z:AST_CTOR_ELEMENT (WS)? )? ']'
        >> {
        >> String args = x.getText();
        >> if ( y!=null ) {
        >> args += ","+y.getText();
        >> }
        >> if ( z!=null ) {
        >> args += ","+z.getText();
        >> }
        >> $setText(generator.getASTCreateString(null,args));
        >> }
        >> ;
        >>
        >> To allow the 3rd arg. Hmm....
        >>
        >> Ter
        >> --
        >> Co-founder, http://www.jguru.com
        >> Creator, ANTLR Parser Generator: http://www.antlr.org
        >> Lecturer in Comp. Sci., University of San Francisco
        >
        >
        >
        >
        > Your use of Yahoo! Groups is subject to
        > http://docs.yahoo.com/info/terms/
        >
        >
        --
        Co-founder, http://www.jguru.com
        Creator, ANTLR Parser Generator: http://www.antlr.org
        Lecturer in Comp. Sci., University of San Francisco
      • slightlynew
        ... Yeah, or made a dumber error than that! If I do get it to work I ll let you know. I might just wait for the C# version since my work is in C# these days.
        Message 3 of 14 , Dec 5, 2002
        • 0 Attachment
          > Perhaps you grabbed 11-29's version not the 30th :)

          Yeah, or made a dumber error than that!

          If I do get it to work I'll let you know. I might just wait for the
          C# version since my work is in C# these days.

          By the way, I have a question about heterogeneous ASTs that relates
          to the "third arg". From recent posts I gather that the generated
          code used to do something like this (I'm simplifying here):

          ASTNode n = new MyTypeOfNode();

          and that this has been changed to something like this:

          ASTNode n = getASTFactory().createInstance("MyTypeOfNode");

          Internally, the factory appears to be using reflection to go from the
          string type name to a new object of that type. (I should qualify
          that I'm talking about the C# code, and assuming that the Java code
          does something similar.)

          Isn't creating an object in this way significantly more expensive?
          If you're creating, say, thousands of nodes of many different types,
          wouldn't one expect slower performance? Given that the type
          information *is* known at compile time, why pay the overhead of run-
          time object creation?

          This isn't an argument against using a factory. The factory could do
          a big switch statement (or the equivalent) on the name, like this:

          case "MyTypeOfNode":
          return new MyTypeOfNode();

          and then resort to the run-time way at the end:

          default:
          return createInstance("MyTypeOfNode");

          Although, come to think of it, it's not obvious to me exactly what
          the advantage of the factory approach is in this context?

          Any comments?

          Ciao,
          Daniel


          --- In antlr-interest@y..., Terence Parr <parrt@j...> wrote:
          > Hi Daniel,
          >
          > Just tried this
          >
          > /tmp $ java antlr.Tool t.g
          > ANTLR Parser Generator Version 2.7.2a6 (20021130-1) 1989-2002
          > jGuru.com
          > /tmp $ cat t.g
          > class P extends Parser;
          >
          > a : A {#[a,b,c];}
          > ;
          >
          > Perhaps you grabbed 11-29's version not the 30th :)
          >
          > Ter
          >
        • slightlynew
          ... Simply the fact that file diffs between the .java files in the C# distribution and their counterparts in the regular Antlr distribution reveal fewer
          Message 4 of 14 , Dec 5, 2002
          • 0 Attachment
            Micheal, I realized I forgot to answer this:

            > > In fact, the discrepancies between the C# and
            > > the main Antlr branch are now quite few
            >
            > I'm not sure I follow you here. What discrepancies
            > are you referring to?

            Simply the fact that file diffs between the .java files in the C#
            distribution and their counterparts in the regular Antlr distribution
            reveal fewer differences than there used to be.

            DG
          • micheal_jor
            ... distribution ... He...he...he. Didn t think anyone noticed ;-) Mostly down to coding style and bracket placement mon ami. I finally made some time to
            Message 5 of 14 , Dec 6, 2002
            • 0 Attachment
              > Micheal, I realized I forgot to answer this:
              >
              > > > In fact, the discrepancies between the C# and
              > > > the main Antlr branch are now quite few
              > >
              > > I'm not sure I follow you here. What discrepancies
              > > are you referring to?
              >
              > Simply the fact that file diffs between the .java files in the C#
              > distribution and their counterparts in the regular Antlr
              distribution
              > reveal fewer differences than there used to be.

              He...he...he. Didn't think anyone noticed ;-)

              Mostly down to coding style and bracket placement mon ami. I finally
              made some time to restore the ANTLR coding style in the CSharp*.java
              files.

              The coding style was changed - in private copies - while finding our
              way around ANTLR but, when the changes were all done, there were no
              volunteers to change it back.....until I got wiped out suspiciously
              fast in an Empire Earth tie-breaker so... ;-)

              Cheers!,

              Micheal
            • Terence Parr
              ... The 30th version has the C# integrated :) Try it out :) ... Yep. ... Correct. ... Yes. Oddly, the java version runs faster than before for homogeneous
              Message 6 of 14 , Dec 6, 2002
              • 0 Attachment
                On Thursday, December 5, 2002, at 09:30 PM, slightlynew wrote:

                >> Perhaps you grabbed 11-29's version not the 30th :)
                >
                > Yeah, or made a dumber error than that!
                >
                > If I do get it to work I'll let you know. I might just wait for the
                > C# version since my work is in C# these days.

                The 30th version has the C# integrated :) Try it out :)

                >
                > By the way, I have a question about heterogeneous ASTs that relates
                > to the "third arg". From recent posts I gather that the generated
                > code used to do something like this (I'm simplifying here):
                >
                > ASTNode n = new MyTypeOfNode();
                >
                > and that this has been changed to something like this:
                >
                > ASTNode n = getASTFactory().createInstance("MyTypeOfNode");
                >

                Yep.

                > Internally, the factory appears to be using reflection to go from the
                > string type name to a new object of that type. (I should qualify
                > that I'm talking about the C# code, and assuming that the Java code
                > does something similar.)

                Correct.

                > Isn't creating an object in this way significantly more expensive?
                > If you're creating, say, thousands of nodes of many different types,
                > wouldn't one expect slower performance?

                Yes. Oddly, the java version runs faster than before for homogeneous
                creation (the only real test I had), which I didn't bother to
                investigate. It uses the reflection thing too I believe.

                > Given that the type
                > information *is* known at compile time, why pay the overhead of run-
                > time object creation?

                Consistency for one. Flexibility for two: no point in a factory if you
                don't use ;)

                > This isn't an argument against using a factory. The factory could do
                > a big switch statement (or the equivalent) on the name, like this:
                >
                > case "MyTypeOfNode":
                > return new MyTypeOfNode();

                I'd have to generate it though. You are free to build your own that
                does this though. Literally I tried to add an IF to check for the
                CommonAST type and then do "new CommonAST" and it was slower!

                > and then resort to the run-time way at the end:
                >
                > default:
                > return createInstance("MyTypeOfNode");
                >
                > Although, come to think of it, it's not obvious to me exactly what
                > the advantage of the factory approach is in this context?

                Only that you can track or otherwise play with the node creation w/o
                changing the generated code.

                Ter
                --
                Co-founder, http://www.jguru.com
                Creator, ANTLR Parser Generator: http://www.antlr.org
                Lecturer in Comp. Sci., University of San Francisco
              • slightlynew <gackle@shaw.ca>
                Ter, Ok, I buy all of that. What you re saying is consistent with the famous 3 commandments of optimization anyway. The only way to know for sure if X is
                Message 7 of 14 , Dec 6, 2002
                • 0 Attachment
                  Ter,

                  Ok, I buy all of that.

                  What you're saying is consistent with the famous "3 commandments" of
                  optimization anyway. The only way to know for sure if X is slower or
                  faster than Y is to measure it. With that in mind, I will do some
                  measuring when I get to the appropriate stage, and in the meantime
                  use the factory.

                  Daniel

                  --- In antlr-interest@yahoogroups.com, Terence Parr <parrt@j...>
                  wrote:
                  >
                  > On Thursday, December 5, 2002, at 09:30 PM, slightlynew wrote:
                  >
                  > >> Perhaps you grabbed 11-29's version not the 30th :)
                  > >
                  > > Yeah, or made a dumber error than that!
                  > >
                  > > If I do get it to work I'll let you know. I might just wait for
                  the
                  > > C# version since my work is in C# these days.
                  >
                  > The 30th version has the C# integrated :) Try it out :)
                  >
                  > >
                  > > By the way, I have a question about heterogeneous ASTs that
                  relates
                  > > to the "third arg". From recent posts I gather that the generated
                  > > code used to do something like this (I'm simplifying here):
                  > >
                  > > ASTNode n = new MyTypeOfNode();
                  > >
                  > > and that this has been changed to something like this:
                  > >
                  > > ASTNode n = getASTFactory().createInstance("MyTypeOfNode");
                  > >
                  >
                  > Yep.
                  >
                  > > Internally, the factory appears to be using reflection to go from
                  the
                  > > string type name to a new object of that type. (I should qualify
                  > > that I'm talking about the C# code, and assuming that the Java
                  code
                  > > does something similar.)
                  >
                  > Correct.
                  >
                  > > Isn't creating an object in this way significantly more expensive?
                  > > If you're creating, say, thousands of nodes of many different
                  types,
                  > > wouldn't one expect slower performance?
                  >
                  > Yes. Oddly, the java version runs faster than before for
                  homogeneous
                  > creation (the only real test I had), which I didn't bother to
                  > investigate. It uses the reflection thing too I believe.
                  >
                  > > Given that the type
                  > > information *is* known at compile time, why pay the overhead of
                  run-
                  > > time object creation?
                  >
                  > Consistency for one. Flexibility for two: no point in a factory if
                  you
                  > don't use ;)
                  >
                  > > This isn't an argument against using a factory. The factory
                  could do
                  > > a big switch statement (or the equivalent) on the name, like this:
                  > >
                  > > case "MyTypeOfNode":
                  > > return new MyTypeOfNode();
                  >
                  > I'd have to generate it though. You are free to build your own
                  that
                  > does this though. Literally I tried to add an IF to check for the
                  > CommonAST type and then do "new CommonAST" and it was slower!
                  >
                  > > and then resort to the run-time way at the end:
                  > >
                  > > default:
                  > > return createInstance("MyTypeOfNode");
                  > >
                  > > Although, come to think of it, it's not obvious to me exactly what
                  > > the advantage of the factory approach is in this context?
                  >
                  > Only that you can track or otherwise play with the node creation
                  w/o
                  > changing the generated code.
                  >
                  > Ter
                  > --
                  > Co-founder, http://www.jguru.com
                  > Creator, ANTLR Parser Generator: http://www.antlr.org
                  > Lecturer in Comp. Sci., University of San Francisco
                Your message has been successfully submitted and would be delivered to recipients shortly.