Re: [agile-usability] Real data
- I quite the approach to what is, essentially, the same problem: doctors determining what treatment or diagnosis to give a patient. There's a strong movement called "evidence based medicine" - which is, essentially, what Larry is proposing we do - get as much evidence as possible. However, its not all about evidence:
"Good doctors and health professionals use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. Evidence-based medicine is not restricted to randomised trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions." (from http://www.cebm.net/index.aspx?o=1914)
To answer Larry's question: if there was evidence that another development process would work better than agile in my environment, I'd switch. However, we'd need to define "better". It could be cost, time, satisfaction, quality, or other things - and what "better" means could differ between projects/organisations/people/etc.
TimOn Sun, Jan 31, 2010 at 10:05 AM, Ted Young <tedyoung@...> wrote:
If it were true, I'd switch to it immediately. But my difficulty in accepting it is that I don't believe there is a single "truth" because I fundamentally believe software development is far too dependent on people and context. If someone said that "for your team, company, and industry, we have found the perfect process", then I'd look very closely. But even then, it would only be appropriate for a single moment in time -- which is why I believe in a Development Strategy instead of a Process. The process must change over time because the team, company, and industry change over time.
Ted M. Young, Dev Manager
BillingCenter(r) Product Team
Guidewire Software, San Mateo, CAOn Sat, Jan 30, 2010 at 12:12 PM, Larry Constantine <lconstantine@...> wrote:
Great discussion, interesting contributions, which is, of course, what I was hoping for.
The data I referred to may indeed be flawed, the collection methods are rife with concerns, and the tables do compare apples and oranges (RUP versus CMM 5 versus agile? what the?), but that is not the issue I wanted to raise. I did not pass on the data not only because they belong to Capers but because I was interested in a different issue and wanted to pose a provocative question that I think is important for the community to consider.
I thoroughly expected people to attack and question the data and methods, which a number of people in the group did even though I said almost nothing about either. I asked simply: What if it were true?
Ron got insulted and William took it as snarky, but neither was intended. I think it’s a legitimate question we all should consider. Despite the protests about evolution and doing whatever works and adopting new practices and practicing many different variations, there seems to be another dynamic that is part of the reality of agile practice today.
If honestly answered, I think some of the responses to my question would be along the lines of: “It wouldn’t matter.” “I’d keep doing what I think/know works.” “The data are clearly wrong or the research was flawed or the interpretation was incorrect.” “Those academics/researchers/consultants don’t know what they are talking about.” However carefully I frame the question as “what if it were so,” I have actually gotten responses like these. Some of the reaction in this forum hints at something not entirely dissimilar.
Agile methods suffer from the same two maladies that have infected every other modern method. First, the unacknowledged truth is the extent to which agile has taken on the flavor of a religion (or at least a political party), not for everybody and not all to the same degree, but it’s a real phenomenon. (William referred to the community as a “broad church”; the “movement” is guided by a “manifesto.” I will get a storm of protest, I am sure, but such language is not entirely coincidental or irrelevant.) There are members of the community, and we all know some of them, for whom the truly honest answer would be that nothing would dissuade them from what they know is the right way to do things; no data, no argument, no logic would ever be sufficient, even while we loudly proclaim ourselves as rational pragmatists.
So the question for each of us and for all of us is the extent to which what we as professionals do and advocate is grounded in belief and how much it is based on evidence. For my part, I am involved in the SEMAT initiative in large part because I think we need to make software development more evidence-based, much more.
Second, all modern methods, agile included, are more or less ad hoc inventions or agglomerations of miscellany from many sources that somebody thought was a good idea or the right combination based on “experience.” But the truth is we do not know whether these are the right or the best or even just a pretty good combination and never will know if things continue with business as usual in the business of software.
Capers’ data, flawed though it may be, is one of the best data sets that we have, but it does not go very far in teasing out what actually makes a difference. Is it pair programming, test-first-development, or standup meetings that makes the most difference? Would a combination of BDUF and XP actually work better than waterfall or XP in purer form? Which of the details of level 5 actually matter and make a difference? (Parenthetically, limited though Capers’ data may be, it provides strong support for agile as an effective approach while also clearly showing that it scales but only so far. Those are the kind of findings that could be useful to know and understand as a community.)
Some of us who have pondered and puzzled over these matters have some guesses about what really works and is central, and almost everyone here has fairly strong opinions, but none of us really knows. For my part, I have come to be persuaded that there are some core pieces of the agile orthodoxy that may need to change if we are ever to take the next leap forward in the fast, cheap, and good game, but I would be the first to admit that my experiences do not constitute data and my arguments often fall, perhaps deservedly, on deaf ears.
One of the places I am coming from is as someone with a foot in two communities who has become aware that in the integration of usability into agile almost all the important accommodation has been and continues to be expected to be a matter of usability professionals fitting their practices into agile methods as the givens. I see usability professionals bending over backward to find a way of fragmenting field ethnography or cramming architectural design into sprint zero or accelerating to light speed to do user testing in 3-week cycles or… I rarely see agile programmers giving up any of what they “know” to be the “right” way of doing things, and I overhear way too many conversations in which the “extreme” programmers simply refuse to budge from the book.
Of course, none of this applies to anyone here, for we are all open-minded and reasonable creatures. :-)
--Larry Constantine, IDSA, ACM Fellow
Professor | University of Madeira | Funchal, Portugal
Institute Fellow | Madeira Interactive Technologies Institute | www.M-ITI.org
- Hi, Jon,
Jon Kern wrote:
> ok, maybe it is silly to debate the term...I've never found creating software to be a "one and only time" no matter
> George it's a free country, use method anytime you please :-)
> I personally will only use "method" when I want to describe some way
> that I achieve something over and over. Often in an abstract sense,
> often in a step-wise way. Often because the "something" is a desirous
> end goal, and one that I (or someone else) wants more than one time.
> I would not describe the /ad hoc/ "how" of the one and only time I will
> ever do something as a "method" if it is not.
how small the program. There's a lot of similarity between writing one
line of code and writing the next.
And I've observed, that people generally continue to do something
somewhat in the fashion they've done it before. Thoughtful people will
consider the result their achieving, and modify their actions to try to
improve some aspect.
I've never seen anyone continue to approach the work as if they'd never
done anything like it before, choosing some completely different way of
working. And I've never seen anyone carefully follow the recipe in a
process manual. At best, a process manual gives the worker some ideas.
It's the process people actually /do/ that has an effect. If you and
Scott and Glen want to reserve the word "method" for officially blessed
procedures, go right ahead. It won't change a thing.
* George Dinwiddie * http://blog.gdinwiddie.com
Software Development http://www.idiacomputing.com
Consultant and Coach http://www.agilemaryland.org