Loading ...
Sorry, an error occurred while loading the content.

Re: [parsexs] What I did on my summer vacation

Expand Messages
  • David Golden
    Sounds like a productive trip! I would qualify my thoughts about compatibility. Given that there is so little coverage today, I would like to see test coverage
    Message 1 of 8 , Sep 1, 2009
    • 0 Attachment

      Sounds like a productive trip!

      I would qualify my thoughts about compatibility. Given that there is so little coverage today, I would like to see test coverage of the existing ParseXS increased substantially, possibly to 100% of branches. If that finds and fixes bugs that happen to break existing XS on CPAN, so be it, as long as we're confident the new output is correct to the perlxs spec.

      Then, for the OO rewrite, I only think it needs to pass the enhanced test suite. It might be interesting to see what it breaks on CPAN vs the old version, but unless is breaks major portions of xs or breaks a Volatile 100 distro, I don't think it's a show stopper.

      David

      On Aug 31, 2009 9:45 PM, "James E Keenan" <jkeen@...> wrote:

       

      Last week I was on vacation from $job -- but not fully on vacation
      from Perl or Parrot. I would like to post a summary of what I did
      with regards to ExtUtils::ParseXS during that period.

      On Tuesday, August 25 I had dinner with Dan M and Mike C of the
      Buffalo Perlmongers group. I have known Dan for five years now, but
      only met Mike at YAPC::NA::2009 in Pittsburgh in June. Mike is very
      knowledgeable about git and, unlike many of the blowhard git
      advocates I've encountered, has extensive experience converting a
      department of 80 developers to (partiallly) use git in day-to-day
      development in a commercial context. He will be a resource (for me,
      at least) in converting version control for this project to git.

      I spoke with Dan and Mike about my work so far on ParseXS. What I
      tried to convey there and later in Toronto was:

      (a) Any refactoring of ParseXS must be able to demonstrate that it
      produces functionally the same output as the current version of
      ParseXS when any .xs file currently found on CPAN is fed into it.

      (b) To carry out (a), David Golden has suggested the construction of
      a minicpan and a script which traverses that minicpan, identifies .xs
      files and then does "before and after" -- or, more precisely,
      "current versus refactored" blackbox testing of versions of ParseXS.
      I'm hoping that Buffalo.pm will be able to contribute to this.

      (c) We will ultimately have to guarantee that every XS feature
      described in 'perldoc perlxs' is tested with dummy copy files in the
      test suite. Since there are many features to XS, we should, as a
      first step, write a "test specification" which prescribes the range
      of dummy copy files we will need to fully test ParseXS.

      (d) We will ultimately have to correct -- and, more likely, overhaul
      -- 'perldoc perlxstut'.

      Mike C also raised an interesting question: Would attempting to
      demonstrate that any revision of ParseXS does not break the XS that's
      already up on CPAN tie us into a requirement of eternal backwards
      compatibility? I responded that it was a step most needed now
      because (i) ParseXS has little test coverage; and (ii) my proposed OO-
      rewrite of ParseXS will touch almost every line of that module, which
      means that extra testing is needed at this stage to make sure I do no
      harm. Ultimately, the test suite which will be generated on the
      basis of the test specification described in (c) will be the proof.

      On Tuesday, August 27, I was a fill-in speaker at Toronto.pm. The
      last 25 minutes or so of my 60-minute talk was devoted to the
      Corehackers project and ParseXS in particular. (The first part was
      devoted to Parrot and Rakudo). I tried to convey points (a) through
      (d) above. I was pleased that veteran Toronto.pm member Tom L.
      volunteered to begin to work on the test specification. In a day or
      two I'll make a post specifically devoted to that topic.

      Thank you very much.
      Jim Keenan

    • James E Keenan
      ... David, that s going to be very difficult. The code in the current ParseXS.pm is so poorly encapsulated that it s very difficult to exercise the degree of
      Message 2 of 8 , Sep 1, 2009
      • 0 Attachment
        On Sep 1, 2009, at 5:33 AM, David Golden wrote:

        >
        >
        > Sounds like a productive trip!
        >
        > I would qualify my thoughts about compatibility. Given that there
        > is so little coverage today, I would like to see test coverage of
        > the existing ParseXS increased substantially, possibly to 100% of
        > branches. If that finds and fixes bugs that happen to break
        > existing XS on CPAN, so be it, as long as we're confident the new
        > output is correct to the perlxs spec.
        >
        David, that's going to be very difficult. The code in the current
        ParseXS.pm is so poorly encapsulated that it's very difficult to
        exercise the degree of control needed to get satisfactory coverage of
        branches and conditions. It's so poorly encapsulated that we don't
        yet have even 100% coverage of subroutines, which is the easiest
        aspect to get to 100%.
      • David Golden
        ... Well, I can dream. :-) I think it s a goal, not a requirement. Depending on how the testing specification turns out that might be even better. For all
        Message 3 of 8 , Sep 1, 2009
        • 0 Attachment
          On Tue, Sep 1, 2009 at 8:09 AM, James E Keenan<jkeen@...> wrote:
          >> I would qualify my thoughts about compatibility. Given that there
          >> is so little coverage today, I would like to see test coverage of
          >> the existing ParseXS increased substantially, possibly to 100% of
          >> branches. If that finds and fixes bugs that happen to break
          >> existing XS on CPAN, so be it, as long as we're confident the new
          >> output is correct to the perlxs spec.
          >>
          > David, that's going to be very difficult. The code in the current
          > ParseXS.pm is so poorly encapsulated that it's very difficult to
          > exercise the degree of control needed to get satisfactory coverage of
          > branches and conditions. It's so poorly encapsulated that we don't
          > yet have even 100% coverage of subroutines, which is the easiest
          > aspect to get to 100%.

          Well, I can dream. :-)

          I think it's a goal, not a requirement. Depending on how the testing
          specification turns out that might be even better. For all we know,
          there are unreachable branches in the existing codebase.

          David
        • James E Keenan
          ... So it turned out to be six days rather than one or two. A conversation with xdg at his Labor Day party has helped to nudging me to get back into work on
          Message 4 of 8 , Sep 6, 2009
          • 0 Attachment
            On Aug 31, 2009, at 9:45 PM, James E Keenan wrote:

            >
            > (c) We will ultimately have to guarantee that every XS feature
            > described in 'perldoc perlxs' is tested with dummy copy files in the
            > test suite. Since there are many features to XS, we should, as a
            > first step, write a "test specification" which prescribes the range
            > of dummy copy files we will need to fully test ParseXS.
            >
            ...
            > I was pleased that veteran Toronto.pm member Tom L.
            > volunteered to begin to work on the test specification. In a day or
            > two I'll make a post specifically devoted to that topic.
            >

            So it turned out to be six days rather than one or two. A
            conversation with xdg at his Labor Day party has helped to nudging me
            to get back into work on ParseXS.pm. Let me provide a bit more
            description as to what I mean by "test specification" and why I think
            it will be helpful to us.

            For better or worse, 'perlxs' is the formal specification of XS. Any
            XS-to-C (ecstasy ?) parser must accomplish all the things described
            therein. Every possible XS feature therefore ought to be represented
            in an .xs file which can be used as the input for compiling and whose
            compiling will indicate whether we have correctly implemented the
            feature or not.

            But which .xs files should we create? We won't know until we've
            conducted a close reading of 'perlxs' and have translated each
            paragraph therein into a list of the sample .xs files needed to test
            that paragraph's features. The test specification is the list we
            will compose of all sample .xs files needed to test all XS features.

            Here's an example. (Note that since I've never tried to do this
            before, I'm not necessarily going to get it any more correct than
            anybody else.)

            "The PACKAGE Keyword

            "When functions within an XS source file must be separated into
            packages the PACKAGE keyword should be used. This keyword is used
            with the MODULE keyword and must follow immediately after it when used.

            ...

            "The same package name can be used more than once, allowing for non-
            contiguous code. This is useful if you have a stronger ordering
            principle than package names."

            To me this suggests that we shall need sample.xs files reflecting the
            following features:

            1. sample.xs without the PACKAGE keyword.
            2. sample.xs with single PACKAGE keyword.
            3. sample.xs with multiple non-contiguous instances of a particular
            PACKAGE keyword -- which implies an instance of a different PACKAGE
            keyword in same sample.xs file.
            4. for testing of error detection, sample.xs file where PACKAGE
            keyword does not follow the MODULE keyword

            Here's another example from farther down in 'perlxs':

            "Initializing Function Parameters

            "C function parameters are normally initialized with their values
            from the argument stack (which in turn contains the parameters that
            were passed to the XSUB from Perl). The typemaps contain the code
            segments which are used to translate the Perl values to the C
            parameters. The programmer, however, is allowed to override the
            typemaps and supply alternate (or additional) initialization code.
            Initialization code starts with the first = , ; or + on a line in the
            INPUT: section. The only exception happens if this ; terminates the
            line, then this ; is quietly ignored."

            To me this suggests that we shall need sample.xs files reflecting the
            following features:

            1. sample.xs without any initialization code.
            2. sample.xs file with initialization code containing an '=' on a
            line in the INPUT: section.
            3. sample.xs file with initialization code containing a ';' on a
            line in the INPUT: section -- but not ending the line.
            4. sample.xs file with initialization code containing a '+' on a
            line in the INPUT: section.
            5. sample.xs file with initialization code with a ';' ending a line
            in the INPUT:section.

            Now, at this point, I don't have any idea what '=', ';', or '+' might
            be doing inside an INPUT: section. Indeed, I have only a vague idea
            of what the INPUT: section does. But that's not the point. I *will*
            have to understand those features when I go to write the sample.xs
            file. But I *don't* have to understand them fully when I'm just
            composing the test specification.

            So, to oversimplify, the test specification is simply a master plan
            for composing .xs files suitable as inputs for the ExtUtils::ParseXS
            test suite.

            Some Frequently Anticipated Questions:

            1. Do we have to write a test specification that completely covers
            'perlxs' before we can begin to write sample.xs files?

            No, we can build this up in pieces. We should just indicate which
            paragraph in 'perlxs' a particular fragment of the test spec is
            describing.

            2. Are the test specification and the sample.xs files going to be
            written by different people?

            Yes and no. They *can* be written by different people. Indeed, my
            hope is that people who as yet know nothing about XS can read
            'perlxs' and begin writing sections of the test spec. I also hope
            that by writing little parts of the test spec -- or reading parts of
            the test spec written by others -- a person's "XS fear factor" will
            be reduced and s/he will be coaxed into trying to write some
            sample.xs files themselves. (I fall into this category.) Finally,
            people who are already fluent in XS will be able to use the test spec
            to churn out sample.xs files for use in the test suite.

            3. Will the test specification be part of the CPAN distribution of
            ExtUtils::ParseXS and/or the Perl 5 tarball?

            It doesn't have to be, but that will be the ParseXS maintainers'
            call. But we should at least store the test spec in a uniform place
            in our repositories/mirrors. xdg: suggestions?

            4. My guess is that we could end up with hundreds of sample.xs files
            by the time we're done? Won't that be seen as excessive?

            We probably will write many dozens of sample.xs files as we work
            through the spec. Over time, we'll probably see our way clear to
            combine these files into larger ones without losing test coverage.
            Personally, I prefer many small test files -- which to me implies
            small sample.xs files -- each focused on a single piece of
            functionality. However, in the course of my similar testing work on
            Parrot Jerry Gay raised as an objection to this the higher cost of
            starting up a single process (such as running a .t file) on Windows
            rather than *nix. In the short run, however, I think we should be
            liberal about adding .t files to the t/ directories on our branches
            or github clones.

            5. The sample.xs files we eventually write -- we'll have to make
            sure that they are valid XS, correct?

            Yes. That means that the test suite will have to compile them into C
            object files and then have corresponding Perl functions exercise
            them. So the test suite will necessarily be doing a lot of C
            compilation. Since I have never written a module that contains XS,
            I'm unclear as to the best way to go about this; suggestions
            welcome. But I have a hunch that we should try to make the
            underlying C code as simple as possible -- e.g., calls to C built-ins
            like 'printf' as much as possible -- rather than calls to obscure or
            complex libraries (such as the 'rpcb_gettime' library used in
            'perlxs', which I do not have installed by default on my systems).

            Thank you very much.
            Jim Keenan
          • arocker@vex.net
            I ve just waded through perlxs. Although I can see the need to ensure that every one of the examples works, I don t think that would be a comprehensive test.
            Message 5 of 8 , Sep 7, 2009
            • 0 Attachment
              I've just waded through perlxs. Although I can see the need to ensure that
              every one of the examples works, I don't think that would be a
              comprehensive test.

              What we need is (at least) a table of possible variable type combinations,
              so we can ensure that the individual transmissions work, and some
              plausible combinations thereof.
            • James E Keenan
              ... 1. I haven t tried the examples in perlxs . (See earlier posts re my struggles with examples in perlxstut .) The fact that they all refer to a library
              Message 6 of 8 , Sep 7, 2009
              • 0 Attachment
                On Sep 7, 2009, at 9:43 AM, arocker@... wrote:

                >
                > I've just waded through perlxs. Although I can see the need to
                > ensure that
                > every one of the examples works, I don't think that would be a
                > comprehensive test.

                1. I haven't tried the examples in 'perlxs'. (See earlier posts re
                my struggles with examples in 'perlxstut'.) The fact that they all
                refer to a library which I don't have installed on either Darwin or
                Linux (because it's non-default) means that I've pushed that onto the
                tail of a long TODO list -- and I would imagine that's what others
                have done over the years as well.

                2. What I am suggesting instead is that we will really need two
                types of sample.xs files to use as inputs into tests:
                a. XS files already visible on CPAN, which are presumptively being
                handled correctly by the current version of ParseXS.pm. We will need
                these to demonstrate we're not doing anything wrong in any
                refactoring we do.
                b. XS files that we ourselves create based on our close reading of
                'perlxs'. With these we demonstrate that we understand the spec.
                With these we may find that there are corner cases in ParseXS.pm that
                don't work as expected but that have never been formally tested or
                tested by CPAN modules.

                >
                > What we need is (at least) a table of possible variable type
                > combinations,
                > so we can ensure that the individual transmissions work, and some
                > plausible combinations thereof.
                >

                I'm not exactly sure what you're proposing. It sounds to me like
                something we would do *after* 2(b) above -- i.e., an even bigger
                project than we currently have. Can you clarify?

                Thank you very much.
                JIm Keenan
              • David Golden
                ... I suspect that printf won t be enough. The nice thing about rpcb_gettime is that it took arguments, returned an argument and also changed a value via a
                Message 7 of 8 , Sep 7, 2009
                • 0 Attachment
                  On Sun, Sep 6, 2009 at 1:19 PM, James E Keenan<jkeen@...> wrote:
                  > Yes. That means that the test suite will have to compile them into C
                  > object files and then have corresponding Perl functions exercise
                  > them. So the test suite will necessarily be doing a lot of C
                  > compilation. Since I have never written a module that contains XS,
                  > I'm unclear as to the best way to go about this; suggestions
                  > welcome. But I have a hunch that we should try to make the
                  > underlying C code as simple as possible -- e.g., calls to C built-ins
                  > like 'printf' as much as possible -- rather than calls to obscure or
                  > complex libraries (such as the 'rpcb_gettime' library used in
                  > 'perlxs', which I do not have installed by default on my systems).

                  I suspect that "printf" won't be enough. The nice thing about
                  rpcb_gettime is that it took arguments, returned an argument and also
                  changed a value via a pointer. So I think we should look for
                  something in the standard library that has similar properties.

                  I'm tempted by something like modf(), which is:

                  double modf(x, double *ip)

                  However, testing doubles is a PITA due to representational/rounding
                  issues, so maybe strtol() instead.

                  -- David
                Your message has been successfully submitted and would be delivered to recipients shortly.