Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] How to TFD a compiler?

Expand Messages
  • Michael Feathers
    ... From: kevinbsmith ... It depends on whether you think you will need it or not. I ve used parser generators in the past, but
    Message 1 of 3 , Apr 30, 2002
    • 0 Attachment
      ----- Original Message -----
      From: "kevinbsmith" <kevinxp@...>
      > Hypothetically, let's imagine that I plan to write a pseudo
      > compiler that would take a script language file as input
      > (perhaps Ruby), and would output "something else" (perhaps
      > C code, or some kind of bytecode).
      >
      > As my own Customer, I *know* I want a full langauge, which
      > would lead me towards using an existing YACC-like tool. But
      > that is not where I would start if I take a purely TestFirst
      > approach, one story at a time.
      >
      > If you were about to start this project, would you:
      > 1. Start right off with something like YACC?
      > 2. Start with a single test, and the simplest possible code
      > that could pass that test, essentially rolling your own
      > lexer and parser, and stick with it until it became painful
      > (and perhaps, maybe, possibly, it never would)?
      > 3. Start rolling your own, but "fairly soon" plan to switch
      > over to a more powerful and robust solution like YACC?

      It depends on whether you think you will need it or not. I've used parser
      generators in the past, but I've also done well writing hand parsers. I was
      surprised to see Bjarne Stroustrup mention in the Design and Evolution of
      C++ that he regretted using a parser generator for cfront, and that (at the
      time of that writing) there were a couple of production C++ compilers which
      used recursive descent.

      On the test first front, I've done a parser for a subset of HTML, and a
      little language that I'm writing for fun named Grot.


      Michael Feathers
      www.objectmentor.com
    • Pollice, Gary
      If the language is one that is well-known, there is probably a test suite already created for it. If not, I would start with a grammar and begin generating the
      Message 2 of 3 , May 1, 2002
      • 0 Attachment
        If the language is one that is well-known, there is probably a test suite
        already created for it. If not, I would start with a grammar and begin
        generating the tests that start with small, simple statements, and
        non-statements to make sure that it doesn't compile what it shouldn't. Then
        I'd continue to build up my test suite as I increased the capability and
        uncovered any defects. I used this technique several years ago when I did
        the Sun 386i compilers and it had terrific quality.

        As for using a parser generator, decide if you need one. I think using one
        with C++ was problematical because the C++ grammar is not LR(1). I happen to
        prefer writing recursive-descent parsers because I have more control over
        them and, I'm sure the XP folks would like this, the code is much more
        understandable.

        My $.02.

        --Gary

        -----Original Message-----
        From: kevinbsmith [mailto:kevinxp@...]
        Sent: Tuesday, April 30, 2002 10:42 PM
        To: extremeprogramming@yahoogroups.com
        Subject: [XP] How to TFD a compiler?


        Hypothetically, let's imagine that I plan to write a pseudo
        compiler that would take a script language file as input
        (perhaps Ruby), and would output "something else" (perhaps
        C code, or some kind of bytecode).

        As my own Customer, I *know* I want a full langauge, which
        would lead me towards using an existing YACC-like tool. But
        that is not where I would start if I take a purely TestFirst
        approach, one story at a time.

        If you were about to start this project, would you:
        1. Start right off with something like YACC?
        2. Start with a single test, and the simplest possible code
        that could pass that test, essentially rolling your own
        lexer and parser, and stick with it until it became painful
        (and perhaps, maybe, possibly, it never would)?
        3. Start rolling your own, but "fairly soon" plan to switch
        over to a more powerful and robust solution like YACC?

        Kevin



        To Post a message, send it to: extremeprogramming@...

        To Unsubscribe, send a blank message to:
        extremeprogramming-unsubscribe@...

        ad-free courtesy of objectmentor.com

        Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
      • Baker, Bram
        ... I ve done a few projects where we rolled our own compilers, and one project that used LEX/YACC. We did not use TFD, and we did not do much refactoring at
        Message 3 of 3 , May 2, 2002
        • 0 Attachment
          > -----Original Message-----
          > From: kevinbsmith [mailto:kevinxp@...]
          > Sent: Tuesday, April 30, 2002 10:42 PM
          > To: extremeprogramming@yahoogroups.com
          > Subject: [XP] How to TFD a compiler?
          >
          >
          > Hypothetically, let's imagine that I plan to write a pseudo
          > compiler that would take a script language file as input
          > (perhaps Ruby), and would output "something else" (perhaps
          > C code, or some kind of bytecode).
          >
          > As my own Customer, I *know* I want a full langauge, which
          > would lead me towards using an existing YACC-like tool. But
          > that is not where I would start if I take a purely TestFirst
          > approach, one story at a time.
          >
          > If you were about to start this project, would you:
          > 1. Start right off with something like YACC?
          > 2. Start with a single test, and the simplest possible code
          > that could pass that test, essentially rolling your own
          > lexer and parser, and stick with it until it became painful
          > (and perhaps, maybe, possibly, it never would)?
          > 3. Start rolling your own, but "fairly soon" plan to switch
          > over to a more powerful and robust solution like YACC?
          >
          > Kevin
          >

          I've done a few projects where we rolled our own compilers, and one project
          that used LEX/YACC.

          We did not use TFD, and we did not do much refactoring at the time. We ended
          up with a pile of hairy code, and we thought YACC was the answer. We have
          BNF grammars defined for our languages, so using LEX/YACC would be
          straight-forward. I started the port to YACC, but abandoned the effort due
          to inertia, and no real business need to do so. Plus, I hated the code YACC
          generates.

          Then I had to extend one of our compilers to support a new language
          statement. I decided to refactor the code a little bit before adding the new
          language construct. I found that a few days of code clean-up resulted in a
          very easy to understand code base. It is so nice that I no longer think YACC
          would help us.

          In the TFD approach, how/when would you decide you'd hit the wall, and
          moving to YACC would be TSTCCPW? I would be surprised to find that following
          the path of merciless refactoring, YAGNI, and OAOO that you would arrive at
          YACC. OTOH, if you have the grammar for your language defined, then YACC
          would be a good choice.

          I think it depends on what you're trying to do. TFD the grammar, or the
          compiler implementation.


          ----------------------------------------------------------------------------
          -----
          On a somewhat un-related topic, I recently had a simple compiler fall out of
          a test suite. One aspect of our product requires that we store object
          relationship graphs in a configuration file (currently the windows registry,
          but moving to XML). The graph contains constraints and conditions on the
          relationships in the graph, and the source file in which we defined the
          relationships was somewhat brittle, i.e. it was easy to make a mistake and
          screw up the coherence of the graph, which usually results in a few hours
          wasted in a debugging session.

          I decided that I would write some tests that would check the validity of the
          graph once it was stored in the registry. I built a bunch of tests that
          tested each constraint and condition, etc. Once I was done, I realised I was
          a few short hours from having a tool to generate the graph. I just added
          methods to each test case to write its test input to a file, and to be able
          to read its test input from a file, and a method to write the values to the
          registry in addition to reading them. Viola, I now have a "compiler" that
          reads our source file, sets up the object graph, plus enforces all of our
          constraints. Not bad for a test suite.


          Bram Baker



          -----------------------------------------------------------------------------------------------------------------------------------------------
          This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately.

          Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen.


          ==============================================================================


          [Non-text portions of this message have been removed]
        Your message has been successfully submitted and would be delivered to recipients shortly.