Loading ...
Sorry, an error occurred while loading the content.
 

Stories and Algorithms

Expand Messages
  • mpkirby@frontiernet.net
    We ve got a new feature that we re trying to figure out the best way to break into stories. In a nut shell, we print out a bunch of color patches, and scan the
    Message 1 of 16 , Apr 30, 2006
      We've got a new feature that we're trying to figure out the best way to break into stories.

      In a nut shell, we print out a bunch of color patches, and scan the patches into a
      spectrophotmenter.

      The results are read, and are used for a fairly complex multi-step algorithm that produces
      control files for our device.

      Certainly the actual act of controlling and reading data from the spectrophotometer could be
      a story, but what about the algorithm itself? It's not likely that the entire thing can be done
      within a single iteration.

      Do we do a "step of the algorithm per story"?

      That doesn't seem very "customer observable". What about customer acceptance tests?
      You'll need a math degree to notice anything.

      The actual behavior on the final device itself isn't really something that can be immediately
      noticed either. The effect of the algorithm on the final product is something that takes place
      over 10's of hours of operation.

      Mike

      ---
      mpkirby@...
    • Ron Jeffries
      ... Who s asking for this algorithm? Why do they want it? What will they get when it s done that they don t have now? How will they know it s done? Ron
      Message 2 of 16 , Apr 30, 2006
        On Sunday, April 30, 2006, at 10:53:17 PM, mpkirby@... wrote:

        > The results are read, and are used for a fairly complex
        > multi-step algorithm that produces
        > control files for our device.

        Who's asking for this algorithm? Why do they want it? What will they
        get when it's done that they don't have now? How will they know it's
        done?

        Ron Jeffries
        www.XProgramming.com
        Reason is and ought only to be the slave of the passions. -- David Hume
      • Steven Gordon
        Is there a simple case (such as all the patches are the same color) where a simple path through the algorithm could be developed, tested and demonstrated in
        Message 3 of 16 , Apr 30, 2006
          Is there a simple case (such as all the patches are the same color)
          where a simple path through the algorithm could be developed, tested
          and demonstrated in one iteration? Then, cound you identify
          additional complexities to add, test, and demo in subsequent single
          iterations until the algorithm handles all the required cases?

          Steven Gordon

          On 4/30/06, mpkirby@... <mpkirby@...> wrote:
          > We've got a new feature that we're trying to figure out the best way to break into stories.
          >
          > In a nut shell, we print out a bunch of color patches, and scan the patches into a
          > spectrophotmenter.
          >
          > The results are read, and are used for a fairly complex multi-step algorithm that produces
          > control files for our device.
          >
          > Certainly the actual act of controlling and reading data from the spectrophotometer could be
          > a story, but what about the algorithm itself? It's not likely that the entire thing can be done
          > within a single iteration.
          >
          > Do we do a "step of the algorithm per story"?
          >
          > That doesn't seem very "customer observable". What about customer acceptance tests?
          > You'll need a math degree to notice anything.
          >
          > The actual behavior on the final device itself isn't really something that can be immediately
          > noticed either. The effect of the algorithm on the final product is something that takes place
          > over 10's of hours of operation.
          >
          > Mike
          >
          > ---
          > mpkirby@...
        • Steven Farlie
          What made agile successful was not that they magically found a silver bullet to solve all problems. Instead they discarded preconceived notions and dogma to
          Message 4 of 16 , Apr 30, 2006
            What made agile successful was not that they magically found a silver
            bullet to solve all problems. Instead they discarded preconceived notions
            and dogma to experiment with process and find out what works for them and
            what doesn't. It just so happened that a lot of other people had the same
            problems that they did.

            My suggestion is that you go the agile way and stop trying to crowbar this
            problem into iterations.
            --
            Steven Farlie
          • Steven Gordon
            Does that mean just do not solve this problem or any other problem that you cannot fit into iterations?
            Message 5 of 16 , Apr 30, 2006
              Does that mean just do not solve this problem or any other problem
              that you cannot fit into iterations?

              On 4/30/06, Steven Farlie <steven@...> wrote:
              > What made agile successful was not that they magically found a silver
              > bullet to solve all problems. Instead they discarded preconceived notions
              > and dogma to experiment with process and find out what works for them and
              > what doesn't. It just so happened that a lot of other people had the same
              > problems that they did.
              >
              > My suggestion is that you go the agile way and stop trying to crowbar this
              > problem into iterations.
              > --
              > Steven Farlie
            • Steven Farlie
              Not at all. I m saying that you should recognise when a particular process is inappropriate for the task at hand. It s the old when you have a hammer
              Message 6 of 16 , May 1, 2006
                Not at all. I'm saying that you should recognise when a particular process
                is inappropriate for the task at hand.

                It's the old "when you have a hammer everything looks like a nail"
                problem. Put down the hammer (iterations) and find the proper tool for the
                job. Mike may need to move the algorithm work into a subproject with a
                more appropriate methodology.

                There are plenty of other methodologies out there. A lot of them worked
                for someone at some point in time for a particular problem. Perhaps their
                problem was similar to Mike's.
                --
                Steven Farlie

                > Does that mean just do not solve this problem or any other problem
                > that you cannot fit into iterations?
                >
                > On 4/30/06, Steven Farlie <steven@...> wrote:
                >> What made agile successful was not that they magically found a silver
                >> bullet to solve all problems. Instead they discarded preconceived
                >> notions
                >> and dogma to experiment with process and find out what works for them
                >> and
                >> what doesn't. It just so happened that a lot of other people had the
                >> same
                >> problems that they did.
                >>
                >> My suggestion is that you go the agile way and stop trying to crowbar
                >> this
                >> problem into iterations.
                >> --
                >> Steven Farlie
                >
              • Steven Gordon
                Driving to find a way to slice a problem so it fits into iterative development often leads to good things like: - identifying concrete test cases before
                Message 7 of 16 , May 1, 2006
                  Driving to find a way to slice a problem so it fits into iterative
                  development often leads to good things like:
                  - identifying concrete test cases before implementation
                  - discovering sets of inputs that are of no use to the customer (which
                  can sometimes lead to a less complex solution)
                  - being able to get valuable feedback early
                  - all the other things we get from the agile approach (rhythm, more
                  realistic estimates, early discovery of issues, elimination of waste
                  ...)

                  Just passing the more complex problems to the non-agile approaches is
                  a cop-out which will make it too easy to just not try to tame our
                  complexities.



                  On 5/1/06, Steven Farlie <steven@...> wrote:
                  > Not at all. I'm saying that you should recognise when a particular process
                  > is inappropriate for the task at hand.
                  >
                  > It's the old "when you have a hammer everything looks like a nail"
                  > problem. Put down the hammer (iterations) and find the proper tool for the
                  > job. Mike may need to move the algorithm work into a subproject with a
                  > more appropriate methodology.
                  >
                  > There are plenty of other methodologies out there. A lot of them worked
                  > for someone at some point in time for a particular problem. Perhaps their
                  > problem was similar to Mike's.
                  > --
                  > Steven Farlie
                  >
                  > > Does that mean just do not solve this problem or any other problem
                  > > that you cannot fit into iterations?
                  > >
                  > > On 4/30/06, Steven Farlie <steven@...> wrote:
                  > >> What made agile successful was not that they magically found a silver
                  > >> bullet to solve all problems. Instead they discarded preconceived
                  > >> notions
                  > >> and dogma to experiment with process and find out what works for them
                  > >> and
                  > >> what doesn't. It just so happened that a lot of other people had the
                  > >> same
                  > >> problems that they did.
                  > >>
                  > >> My suggestion is that you go the agile way and stop trying to crowbar
                  > >> this
                  > >> problem into iterations.
                  > >> --
                  > >> Steven Farlie
                  > >
                • Steven Farlie
                  I agree with all the things that you just said and I would never advise people to just cop-out on the tough stuff. Iterative development has been proven to be
                  Message 8 of 16 , May 1, 2006
                    I agree with all the things that you just said and I would never advise
                    people to just cop-out on the tough stuff. Iterative development has been
                    proven to be extremely effective in many situations. But I would never
                    advise people to use it in *all* situations, just 99.9% of them.

                    I have worked on algorithms before and they are quite unlike "normal"
                    business software. They tend to have an innate complexity that appears
                    quite irreducible. They are usually easy to test, in that they give right
                    or wrong answers, but also difficult to test, in that the answer may take
                    weeks or months to come out. If you are lucky you will have one person in
                    your team that actually understands the problem and can work the solution.

                    So I would advise Mike (though I don't really know the problem) to isolate
                    the algorithm into it's own sub-project and give it to your resident
                    genius. Leave the algorithm as a black box in the main project. It may
                    seem unbelieveable to some, but many projects succeeded without
                    iterations.
                    --
                    Steven Farlie

                    > Driving to find a way to slice a problem so it fits into iterative
                    > development often leads to good things like:
                    > - identifying concrete test cases before implementation
                    > - discovering sets of inputs that are of no use to the customer (which
                    > can sometimes lead to a less complex solution)
                    > - being able to get valuable feedback early
                    > - all the other things we get from the agile approach (rhythm, more
                    > realistic estimates, early discovery of issues, elimination of waste
                    > ...)
                    >
                    > Just passing the more complex problems to the non-agile approaches is
                    > a cop-out which will make it too easy to just not try to tame our
                    > complexities.
                    >
                    >
                    >
                    > On 5/1/06, Steven Farlie <steven@...> wrote:
                    >> Not at all. I'm saying that you should recognise when a particular
                    >> process
                    >> is inappropriate for the task at hand.
                    >>
                    >> It's the old "when you have a hammer everything looks like a nail"
                    >> problem. Put down the hammer (iterations) and find the proper tool for
                    >> the
                    >> job. Mike may need to move the algorithm work into a subproject with a
                    >> more appropriate methodology.
                    >>
                    >> There are plenty of other methodologies out there. A lot of them worked
                    >> for someone at some point in time for a particular problem. Perhaps
                    >> their
                    >> problem was similar to Mike's.
                    >> --
                    >> Steven Farlie
                    >>
                    >> > Does that mean just do not solve this problem or any other problem
                    >> > that you cannot fit into iterations?
                    >> >
                    >> > On 4/30/06, Steven Farlie <steven@...> wrote:
                    >> >> What made agile successful was not that they magically found a silver
                    >> >> bullet to solve all problems. Instead they discarded preconceived
                    >> >> notions
                    >> >> and dogma to experiment with process and find out what works for them
                    >> >> and
                    >> >> what doesn't. It just so happened that a lot of other people had the
                    >> >> same
                    >> >> problems that they did.
                    >> >>
                    >> >> My suggestion is that you go the agile way and stop trying to crowbar
                    >> >> this
                    >> >> problem into iterations.
                    >> >> --
                    >> >> Steven Farlie
                    >> >
                  • Ron Jeffries
                    ... Well ... it s possible that some work cannot be done in iterations: I even have a couple of candidates in mind. However, a focus on finding a way to get
                    Message 9 of 16 , May 1, 2006
                      On Monday, May 1, 2006, at 3:08:10 AM, Steven Farlie wrote:

                      > Not at all. I'm saying that you should recognise when a particular process
                      > is inappropriate for the task at hand.

                      > It's the old "when you have a hammer everything looks like a nail"
                      > problem. Put down the hammer (iterations) and find the proper tool for the
                      > job. Mike may need to move the algorithm work into a subproject with a
                      > more appropriate methodology.

                      > There are plenty of other methodologies out there. A lot of them worked
                      > for someone at some point in time for a particular problem. Perhaps their
                      > problem was similar to Mike's.

                      Well ... it's possible that some work cannot be done in iterations:
                      I even have a couple of candidates in mind.

                      However, a focus on finding a way to get things done in small bites
                      is very valuable for a number of reasons including these few:

                      iterations make progress more visible, more steady, and more
                      predictable;

                      time-boxing our work helps us avoid over-engineering, and helps
                      discover problems sooner;

                      frequent integration due to short iterations keeps us ready to
                      ship and practiced at it;

                      short iterations make communications problems between PO and
                      developers more visible, and correct them sooner.

                      I would not lightly suggest that iterations be dropped.

                      With respect to the algorithm, I'd want to explore not just the
                      questions I asked before, which relate to who wants it and what
                      value they perceive in it. I'd also want to consider the algorithm
                      itself. Many, if not most, algorithms are modular by nature,
                      containing phases, approximations, refinements.

                      It's hard for me to imagine an algorithm that can't be usefully
                      attacked in a week, much less in two or a month. No doubt there are
                      some, but I'd not assume that going in.

                      Ron Jeffries
                      www.XProgramming.com
                      If it is more than you need, it is waste. -- Andy Seidl
                    • Ron Jeffries
                      ... +1 Ron Jeffries www.XProgramming.com Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back of his head. It is, as far as he knows, the
                      Message 10 of 16 , May 1, 2006
                        On Monday, May 1, 2006, at 3:18:59 AM, Steven Gordon wrote:

                        > Just passing the more complex problems to the non-agile approaches is
                        > a cop-out which will make it too easy to just not try to tame our
                        > complexities.

                        +1

                        Ron Jeffries
                        www.XProgramming.com
                        Here is Edward Bear, coming downstairs now, bump, bump, bump, on the back
                        of his head. It is, as far as he knows, the only way of coming downstairs,
                        but sometimes he feels that there really is another way, if only he could
                        stop bumping for a moment and think of it. And then he feels that perhaps
                        there isn't. -- A. A. Milne
                      • Ron Jeffries
                        ... Yes ... but if this is true, odds are that Mike s problem is not one that shouldn t be done in iterations. ... That s interesting. I ve never seen an
                        Message 11 of 16 , May 1, 2006
                          On Monday, May 1, 2006, at 5:24:19 AM, Steven Farlie wrote:

                          > I agree with all the things that you just said and I would never advise
                          > people to just cop-out on the tough stuff. Iterative development has been
                          > proven to be extremely effective in many situations. But I would never
                          > advise people to use it in *all* situations, just 99.9% of them.

                          Yes ... but if this is true, odds are that Mike's problem is not one
                          that shouldn't be done in iterations.

                          > I have worked on algorithms before and they are quite unlike "normal"
                          > business software. They tend to have an innate complexity that appears
                          > quite irreducible. They are usually easy to test, in that they give right
                          > or wrong answers, but also difficult to test, in that the answer may take
                          > weeks or months to come out. If you are lucky you will have one person in
                          > your team that actually understands the problem and can work the solution.

                          That's interesting. I've never seen an algorithm that was
                          irreducible in that way, at least at the scale of an iteration. In
                          order to be written up at all, algorithms seem to need section-,
                          paragraph=, sentence-equivalents, so that people can understand what
                          they are.

                          > So I would advise Mike (though I don't really know the problem) to isolate
                          > the algorithm into it's own sub-project and give it to your resident
                          > genius. Leave the algorithm as a black box in the main project. It may
                          > seem unbelieveable to some, but many projects succeeded without
                          > iterations.

                          Of course projects succeed without iterations -- at least without
                          formally recognizing that they have them. (In all the successful
                          non-iterative projects I've seen, there were a kind of development
                          "phases", where the team worked on this stuff for a while, then that
                          stuff. Selected work, iterative in that sense, but not sliced into
                          fixed time-boxes. There may be exceptions to that as well.)

                          I'd like to know more about the situation, and the algorithm, before
                          releasing the team from the time-boxes ... especially if they're on
                          one-month or two-week iterations.

                          Ron Jeffries
                          www.XProgramming.com
                          Adapt, improvise, overcome.
                          --Gunnery Sergeant Tom Highway (Heartbreak Ridge)
                        • mpkirby@frontiernet.net
                          ... It s a process control stability algorithm. Basically, the physics of the problem will cause setpoints to drift to a point where the closed-loop feedback
                          Message 12 of 16 , May 1, 2006
                            On 30 Apr 2006 at 23:02, Ron Jeffries wrote:

                            > > The results are read, and are used for a fairly complex
                            > > multi-step algorithm that produces
                            > > control files for our device.
                            >
                            > Who's asking for this algorithm? Why do they want it? What will they
                            > get when it's done that they don't have now? How will they know it's
                            > done?


                            It's a process control stability algorithm. Basically, the physics of the problem will cause
                            setpoints to drift to a point where the closed-loop feedback controls no longer work. This
                            does an off-line analysis of additional data such that the setpoints are adjusted and the
                            closed-loop feedback algorithm works again.

                            There are final tests (faults, etc) that are run on the device when the complete algorithm is
                            delivered (absence of faults will mean it works). Of course, these faults are useless for
                            partial deliveries.

                            I think I need to sit down with those who understand this problem in more detail then me
                            (customer surrogates in this case), and go through the algorithm in a bit of detail. I suspect
                            that will illuminate it a bit.

                            Mike


                            ---
                            mpkirby@...
                          • Steven Farlie
                            ... Once again I must concede to you Ron. (But one day, when you least expect it, I ll be there. And then... the list shall be mine!!) -- Steven Farlie
                            Message 13 of 16 , May 1, 2006
                              On Mon, May 1, 2006 8:12 pm, Ron Jeffries wrote:
                              > <snip!>
                              >
                              > Of course projects succeed without iterations -- at least without
                              > formally recognizing that they have them. (In all the successful
                              > non-iterative projects I've seen, there were a kind of development
                              > "phases", where the team worked on this stuff for a while, then that
                              > stuff. Selected work, iterative in that sense, but not sliced into
                              > fixed time-boxes. There may be exceptions to that as well.)
                              >
                              > I'd like to know more about the situation, and the algorithm, before
                              > releasing the team from the time-boxes ... especially if they're on
                              > one-month or two-week iterations.
                              >
                              > Ron Jeffries

                              Once again I must concede to you Ron.

                              (But one day, when you least expect it, I'll be there. And then... the
                              list shall be mine!!)
                              --
                              Steven Farlie
                            • Ron Jeffries
                              ... Cool. ... I want to know this to understand whether the customer for this is the regular PO, or whether they are proxying for someone else. This will
                              Message 14 of 16 , May 1, 2006
                                On Monday, May 1, 2006, at 6:54:20 AM, mpkirby@... wrote:

                                >> Who's asking for this algorithm? Why do they want it? What will they
                                >> get when it's done that they don't have now? How will they know it's
                                >> done?

                                > It's a process control stability algorithm. Basically, the physics
                                > of the problem will cause setpoints to drift to a point where the
                                > closed-loop feedback controls no longer work. This does an
                                > off-line analysis of additional data such that the setpoints are
                                > adjusted and the closed-loop feedback algorithm works again.

                                > There are final tests (faults, etc) that are run on the device
                                > when the complete algorithm is delivered (absence of faults will
                                > mean it works). Of course, these faults are useless for partial
                                > deliveries.

                                > I think I need to sit down with those who understand this problem
                                > in more detail then me (customer surrogates in this case), and go
                                > through the algorithm in a bit of detail. I suspect that will
                                > illuminate it a bit.

                                Cool.

                                I asked:

                                >> Who's asking for this algorithm?

                                I want to know this to understand whether the customer for this is
                                the regular PO, or whether they are proxying for someone else. This
                                will change my view of how to split the story.

                                >> Why do they want it?

                                I want to know this because there is some kind of benefit to the
                                story. I'd like to dig into that benefit to understand whether
                                partial benefit can be provided with partial work.

                                >> What will they get when it's done that they don't have now?

                                This goes to the question of benefit also.

                                >> How will they know it's done?

                                This goes to the question of stages of completion. Your paragraph
                                here starts to address that:

                                > There are final tests (faults, etc) that are run on the device
                                > when the complete algorithm is delivered (absence of faults will
                                > mean it works). Of course, these faults are useless for partial
                                > deliveries.

                                I would think that there is more than one possible algorithm for
                                eliminating faults, perhaps more than one possible fault type. My
                                question goes to whether those faults can be eliminated one or a few
                                at a time, staging the release of a partial algorithm. One without
                                Foontang-Muffit's correction to the Bachhs-Coutier approximation to
                                Ouiseau's description of the ideal luminance curve, or something
                                like that. Maybe not exactly like that. Anyway, I'm just probing for
                                places to split the algorithm.

                                Ron Jeffries
                                www.XProgramming.com
                                The model that really matters is the one that people have in
                                their minds. All other models and documentation exist only to
                                get the right model into the right mind at the right time.
                                -- Paul Oldfield
                              • Ron Jeffries
                                ... I m not trying to win, Steven, except in the sense that I feel like a winner when I manage to help. Not looking for concession, but for improved
                                Message 15 of 16 , May 1, 2006
                                  On Monday, May 1, 2006, at 7:04:49 AM, Steven Farlie wrote:

                                  >> I'd like to know more about the situation, and the algorithm, before
                                  >> releasing the team from the time-boxes ... especially if they're on
                                  >> one-month or two-week iterations.
                                  >>
                                  >> Ron Jeffries

                                  > Once again I must concede to you Ron.

                                  > (But one day, when you least expect it, I'll be there. And then... the
                                  > list shall be mine!!)

                                  I'm not trying to win, Steven, except in the sense that I feel like
                                  a winner when I manage to help. Not looking for concession, but for
                                  improved understanding all around.

                                  Ron Jeffries
                                  www.XProgramming.com
                                  There is no art without intention. -- Duke Ellington
                                • mike.dwyer1@comcast.net
                                  Ron Perhaps the reason I share your position on the value of a weeks worth of work is the notion that after a week we now have that much more information about
                                  Message 16 of 16 , May 1, 2006
                                    Ron
                                    Perhaps the reason I share your position on the value of a weeks worth of work is the notion that after a week we now have that much more information about what the problem isn't.
                                     
                                    --
                                    Perseverance is not a long race; it is many short races one after another. ~Walter Elliott, The Spiritual Life


                                    The greatest oak was once a little nut who held its ground. ~Author Unknown
                                     
                                    -------------- Original message --------------
                                    From: Ron Jeffries <ronjeffries@...>

                                    > On Monday, May 1, 2006, at 3:08:10 AM, Steven Farlie wrote:
                                    >
                                    > > Not at all. I'm saying that you should recognise when a particular process
                                    > > is inappropriate for the task at hand.
                                    >
                                    > > It's the old "when you have a hammer everything looks like a nail"
                                    > > problem. Put down the hammer (iterations) and find the proper tool for the
                                    > > job. Mike may need to move the algorithm work into a subproject with a
                                    > > more appropriate methodology.
                                    >
                                    > > There are plenty of other methodologies out there. A lot of them worked
                                    > > for someone at some point in time for a particular problem. Perhaps their
                                    > > problem was similar to Mike's.
                                    >
                                    > Well ... it's possible that some work cannot be done in iterations:
                                    > I even have a couple of candidates in mind.
                                    >
                                    > However, a focus on finding a way to get things done in small bites
                                    > is very valuable for a number of reasons including these few:
                                    >
                                    > iterations make progress more visible, more steady, and more
                                    > predictable;
                                    >
                                    > time-boxing our work helps us avoid over-engineering, and helps
                                    > discover problems sooner;
                                    >
                                    > frequent integration due to short iterations keeps us ready to
                                    > ship and practiced at it;
                                    >
                                    > short iterations make communications problems between PO and
                                    > developers more visible, and correct them sooner.
                                    >
                                    > I would not lightly suggest that iterations be dropped.
                                    >
                                    > With respect to the algorithm, I'd want to explore not just the
                                    > questions I asked before, which relate to who wants it and what
                                    > value they perceive in it. I'd also want to consider the algorithm
                                    > itself. Many, if not most, algorithms are modular by nature,
                                    > containing phases, approximations, refinements.
                                    >
                                    > It's hard for me to imagine an algorithm that can't be usefully
                                    > attacked in a week, much less in two or a month. No doubt there are
                                    > some, but I'd not assume that going in.
                                    >
                                    > Ron Jeffries
                                    > www.XProgramming.com
                                    > If it is more than you need, it is waste. -- Andy Seidl
                                    >
                                    >
                                    >
                                    > To Post a message, send it to: scrumdevelopment@...
                                    > To Unsubscribe, send a blank message to:
                                    > scrumdevelopment-unsubscribe@...
                                    > Yahoo! Groups Links
                                    >
                                    > <*> To visit your group on the web, go to:
                                    > http://groups.yahoo.com/group/scrumdevelopment/
                                    >
                                    > <*> To unsubscribe from this group, send an email to:
                                    > scrumdevelopment-unsubscribe@yahoogroups.com
                                    >
                                    > <*> Your use of Yahoo! Groups is subject to:
                                    > http://docs.yahoo.com/info/terms/
                                    >
                                    >
                                    >
                                  Your message has been successfully submitted and would be delivered to recipients shortly.