Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Developing a secure link

Expand Messages
  • William Wake
    To me, the question is what test would convince you that it s right? . I m using test in a broader sense than executable code. I can think of 4 answers off
    Message 1 of 35 , Apr 30, 2001
    • 0 Attachment
      To me, the question is "what test would convince you that it's right?". I'm
      using 'test' in a broader sense than executable code.

      I can think of 4 answers off the top of my head (so there are probably many
      more):
      * a "convincing" set of test cases (such as test-first might generate)
      * a design/code review with the "right" people
      * a formal proof or analysis of some sort
      * a combination of the above

      If nothing short of a proof will pass your 'test', then test-first may not
      make it. I have no trouble believing that test-first will not be convincing
      enough in high-risk domains.


      The other part is - there's an aspect of professionalism in knowing when
      you're out of your league. ("You" in the generic sense, as we haven't met:)
      For example, most people fresh out of college should not be in charge of
      designing and implementing a new security system responsible for keeping
      billions of dollars safe, or designing _my_ pacemaker, etc.

      Some problems are just subtle and hard. Look at what Java's been through
      with its security flaws, or how hard things like threading are to get just
      right, or published proofs that get withdrawn. By the time you know the "one
      more test case you should have written," it's too late. And as the existence
      of flawed proofs demonstrates, "proof is a social process". (See DeMillo,
      Lipton, and Perlis, "Social processes and proofs of theorems and programs,"
      Communications of the ACM, 22(5), May 1979.)


      Would there be stories, spikes, etc? I could see all those still being
      present, even if you decide to augment them with something else.


      Regards,
      Bill Wake
      william.wake@...
      www.xp123.com


      _________________________________________________________________
      Get your FREE download of MSN Explorer at http://explorer.msn.com
    • wecaputo@thoughtworks.com
      ... if_ ... assurance ... or ... small ... Nicely put Bill. IMHO *how* this mechanism allows us to treat the code as exactly right is though the
      Message 35 of 35 , May 7, 2001
      • 0 Attachment
        William Wake:

        >'Relentless testing' is more a mechanism that lets us treat the code _as
        if_
        >it is exactly right (knowing deep down that it might not be). Our
        assurance
        >is only as good as our ability to generate test cases. And when there are
        >cases where that assurance isn't enough (ala Cockburn's 'essential money'
        or
        >'life critical'), we may need other mechanisms. Even then, there is a
        small
        >voice of doubt (nothing is for sure, as the history of security mechanisms

        >shows).

        Nicely put Bill. IMHO *how* this mechanism allows us to treat the code as
        'exactly right' is though the precise,objective, a priori specification of
        these tests as an input into development (acceptance tests) so we can
        'know' when (if) the code works to the appropriate level of silencing
        doubt.

        As someone else in the thread pointed out, running tests for known exploits
        might go a long way toward verifying the system works. Having a security
        expert acting as the test definer couldn't hurt either. When a hole is
        found after release (and there will be some) writing a test to illuminate
        the defect ensures it never rears its ugly head again.
        It all comes down to tests as input into development (upstream) vs tests to
        verify that development was done (downstream). Same tests, same rigorous
        approach, different order.

        >We're back to Dijkstra's "Testing can show the presence of bugs, but never

        >their absence."

        That quote makes it clear why security can never be absolute. to rephrase:
        "Testing can show the presecne of security holes, but never their absence."
        -- or something like that ;-)

        Best,
        Bill
      Your message has been successfully submitted and would be delivered to recipients shortly.