## XP and image processing

Expand Messages
• Hello all! From the first time I saw XP I thought it is a very good methodology. But I have a questions about unit testing. As you may already know, in image
Message 1 of 11 , Jun 6, 2008
Hello all!
From the first time I saw XP I thought it is a very good methodology.
But I have a questions about unit testing.
As you may already know, in image processing most of the things work
with a probability. Let's say you have a method that finds the
location of faces in a given image, and this algorithm works well 98%
of the time, and gives false trues (a location that doesn't contain
any face). How do you test such a method?
• You would have to find the reasons why it give the false trues and write test conditions for them. If you do not know why it is giving false trues, the unit
Message 2 of 11 , Jun 6, 2008
You would have to find the reasons why it give the false trues and write test conditions for them. If you do not know why it is giving false trues, the unit test well not know either.

--- On Fri, 6/6/08, Baris Evrim Demiröz <demirozba@...> wrote:

From: Baris Evrim Demiröz <demirozba@...>
Subject: [XP] XP and image processing
To: extremeprogramming@yahoogroups.com
Date: Friday, June 6, 2008, 1:50 PM

Hello all!
From the first time I saw XP I thought it is a very good methodology.
But I have a questions about unit testing.
As you may already know, in image processing most of the things work
with a probability. Let's say you have a method that finds the
location of faces in a given image, and this algorithm works well 98%
of the time, and gives false trues (a location that doesn't contain
any face). How do you test such a method?

[Non-text portions of this message have been removed]
• This sounds like something too complicated for a unit test. I imagine I d have UTs for smaller tasks, like finding dark contiguous regions, or an edge, or
Message 3 of 11 , Jun 6, 2008
This sounds like something too complicated for a unit test. I imagine I'd
have UTs for smaller tasks, like finding dark contiguous regions, or an
edge, or other more clearly defined small tasks. Then I'd have Acceptance
Tests with images that had been interpreted by a human, and upon running a
suite of tests against a series of different face recognition algorithms
you'd get feedback on how good that overall algorithm is.

On Fri, Jun 6, 2008 at 1:50 PM, Baris Evrim Demiröz <demirozba@...>
wrote:

> Hello all!
> From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?
>
>
>

--
D. André Dhondt
mobile: 267-283-8270
home: 267-286-6875

If you're a software developer in the area, join Agile Philly (
http://groups.yahoo.com/group/agilephilly/)!

[Non-text portions of this message have been removed]
• I think you could unit test it by using images that you know the information about. The ones that work should return the correct information, the ones that
Message 4 of 11 , Jun 6, 2008
I think you could unit test it by using images that you know the information about. The ones that work should return the correct information, the ones that knowenly don't work should return the correct information that says it cant find them.

--- On Fri, 6/6/08, D. André Dhondt <d.andre.dhondt@...> wrote:

From: D. André Dhondt <d.andre.dhondt@...>
Subject: Re: [XP] XP and image processing
To: extremeprogramming@yahoogroups.com
Date: Friday, June 6, 2008, 3:52 PM

This sounds like something too complicated for a unit test. I imagine I'd
have UTs for smaller tasks, like finding dark contiguous regions, or an
edge, or other more clearly defined small tasks. Then I'd have Acceptance
Tests with images that had been interpreted by a human, and upon running a
suite of tests against a series of different face recognition algorithms
you'd get feedback on how good that overall algorithm is.

On Fri, Jun 6, 2008 at 1:50 PM, Baris Evrim Demiröz <demirozba@yahoo. com>
wrote:

> Hello all!
> From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?
>
>
>

--
D. André Dhondt
mobile: 267-283-8270
home: 267-286-6875

If you're a software developer in the area, join Agile Philly (
http://groups. yahoo.com/ group/agilephill y/)!

[Non-text portions of this message have been removed]

[Non-text portions of this message have been removed]
• [CCd to testfirstuserinterfaces@yahoogroups.com ] ... Study the source to Cairo (part of Gtk+). It uses a system called golden master test . Each test case
Message 5 of 11 , Jun 6, 2008
[CCd to testfirstuserinterfaces@yahoogroups.com ]

Baris Evrim Demiröz wrote:

> From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?

Study the source to Cairo (part of Gtk+). It uses a system called "golden master
test". Each test case (at this level) draws a little graphic, such as one
diagonal line. Then the test case byte-compares the image to a png file.

If the test output is right and the png file is wrong, engineers replace the png
file with the new output. That's why the master files are "golden" - they are

This system won't prove you match faces. TDD cannot test that crypto makes your
app bulletproof, or that compression absolutely minimizes your files, or that a
Turing loop will never halt.

This system should stop you if you make a change too big for its tolerances.
This frees your edits up to make whatever change you like. Get a battery of
known faces, and test that they all get recognized to within their expected
variations.

When you actually TDD, you are simply matching tests to low-level code, to check
it manipulates its bits correctly. Most of your tests should run below the level
of the faces.

--
Phlip
• Never tried test in this area, but I see some article about this, may be you can take a look? http://www.xprogramming.com/xpmag/dbcShotgunGray.htm
Message 6 of 11 , Jun 6, 2008
be you can take a look?

http://www.xprogramming.com/xpmag/dbcShotgunGray.htm
http://www.xprogramming.com/xpmag/dbcShootingThePicture.htm
http://www.xprogramming.com/xpmag/dbcShootingForEffect.htm
http://www.xprogramming.com/xpmag/dbcImprovingDrawingCode.htm
http://www.xprogramming.com/xpmag/dbcShootingInCircles.htm

On Sat, Jun 7, 2008 at 1:50 AM, Baris Evrim Demiröz <demirozba@...> wrote:
> Hello all!
> From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?
>
>
> ------------------------------------
>
> To Post a message, send it to: extremeprogramming@...
>
> To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
>
>
>
>
>
• While image processing is difficult, what you re looking for isn t a unit test - it s an acceptance test. As Phlip says, it simply isn t low enough level for a
Message 7 of 11 , Jun 6, 2008
While image processing is difficult, what you're
looking for isn't a unit test - it's an acceptance
test. As Phlip says, it simply isn't low enough level
for a unit test.

Since it's an acceptance test with poorly defined
criteria, you throw a boatload of images at it, each of which
has a known output (the "Guru says" pattern).
Then you inspect the differences to see whether
they fall into known problem areas (shadows, for

The advantages of this approach are:

1. You've got an automated regression test suite
you can qualify algorithm improvements against.

2. You can add test files indefinitely, qualifying each
from last month's Vogue".

John Roth

----- Original Message -----
From: "Baris Evrim Demiröz" <demirozba@...>
To: <extremeprogramming@yahoogroups.com>
Sent: Friday, June 06, 2008 11:50 AM
Subject: [XP] XP and image processing

Hello all!
From the first time I saw XP I thought it is a very good methodology.
But I have a questions about unit testing.
As you may already know, in image processing most of the things work
with a probability. Let's say you have a method that finds the
location of faces in a given image, and this algorithm works well 98%
of the time, and gives false trues (a location that doesn't contain
any face). How do you test such a method?
• Thanks for all the replies. I will have a look at the things you all have pointed. The conclusion I have drawn is this task is too complicated for unit
Message 8 of 11 , Jun 7, 2008
Thanks for all the replies. I will have a look at the things you all
have pointed.
The conclusion I have drawn is this task is too complicated for unit
testing. Instead of unit testing face locating method, I should write
tests for smaller tasks like image resizing. And those test should
check for the image width/height, handling Null cases etc.
These are my opinions before looking to the links you people have
given, after reading they may change ;)
Thanks again!

--- In extremeprogramming@yahoogroups.com, John Roth <JohnRoth1@...>
wrote:
>
> While image processing is difficult, what you're
> looking for isn't a unit test - it's an acceptance
> test. As Phlip says, it simply isn't low enough level
> for a unit test.
>
> Since it's an acceptance test with poorly defined
> criteria, you throw a boatload of images at it, each of which
> has a known output (the "Guru says" pattern).
> Then you inspect the differences to see whether
> they fall into known problem areas (shadows, for
>
> The advantages of this approach are:
>
> 1. You've got an automated regression test suite
> you can qualify algorithm improvements against.
>
> 2. You can add test files indefinitely, qualifying each
> from last month's Vogue".
>
> John Roth
>
>
>
> ----- Original Message -----
> From: "Baris Evrim Demiröz" <demirozba@...>
> To: <extremeprogramming@yahoogroups.com>
> Sent: Friday, June 06, 2008 11:50 AM
> Subject: [XP] XP and image processing
>
>
> Hello all!
> From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?
>
• ... What do we want to know about the method? If we have a given saved pictures, do we want it to get the same answer all the time? What if we had 100 pictures
Message 9 of 11 , Jun 7, 2008
Hello, Baris. On Friday, June 6, 2008, at 1:50:37 PM, you wrote:

>>From the first time I saw XP I thought it is a very good methodology.
> But I have a questions about unit testing.
> As you may already know, in image processing most of the things work
> with a probability. Let's say you have a method that finds the
> location of faces in a given image, and this algorithm works well 98%
> of the time, and gives false trues (a location that doesn't contain
> any face). How do you test such a method?

What do we want to know about the method? If we have a given saved
pictures, do we want it to get the same answer all the time?

What if we had 100 pictures and it correctly classified 95 of them?
What if it did 99?

Ron Jeffries
www.XProgramming.com
Inigo Montoya: You are wonderful!
Man in Black: Thank you. I have worked hard to become so.
• Yes, good articles. Ron and Chet wrote those in response to this exact type of issue, if memory serves.... Specifically, there was a thread around things that
Message 10 of 11 , Jun 7, 2008
Yes, good articles. Ron and Chet wrote those in response to this exact type
of issue, if memory serves.... Specifically, there was a thread around
things that couldn't be TDD'd, and those guys do a decent job of answering
the question 'Can graphical stuff get developed using TDD?'

Chris.

On Fri, Jun 6, 2008 at 11:46 PM, Carfield Yim <carfield@...> wrote:

> be you can take a look?
>
> http://www.xprogramming.com/xpmag/dbcShotgunGray.htm
> http://www.xprogramming.com/xpmag/dbcShootingThePicture.htm
> http://www.xprogramming.com/xpmag/dbcShootingForEffect.htm
> http://www.xprogramming.com/xpmag/dbcImprovingDrawingCode.htm
> http://www.xprogramming.com/xpmag/dbcShootingInCircles.htm
>
> On Sat, Jun 7, 2008 at 1:50 AM, Baris Evrim Demiröz <demirozba@...>
> wrote:
> > Hello all!
> > From the first time I saw XP I thought it is a very good methodology.
> > But I have a questions about unit testing.
> > As you may already know, in image processing most of the things work
> > with a probability. Let's say you have a method that finds the
> > location of faces in a given image, and this algorithm works well 98%
> > of the time, and gives false trues (a location that doesn't contain
> > any face). How do you test such a method?
> >
> >
> > ------------------------------------
> >
> > To Post a message, send it to: extremeprogramming@...
> >
> > To Unsubscribe, send a blank message to:
> extremeprogramming-unsubscribe@...
> >
> >
> >
> >
> >
>
> ------------------------------------
>
> To Post a message, send it to: extremeprogramming@...
>
> To Unsubscribe, send a blank message to:
> extremeprogramming-unsubscribe@...
>
>
>
>
>

[Non-text portions of this message have been removed]
• ... 3. brute-force acceptance tests make a good-enough regression detector that you can now start refactoring the low-level code to add its unit tests.
Message 11 of 11 , Jun 7, 2008
> 1. You've got an automated regression test suite
> you can qualify algorithm improvements against.
>
> 2. You can add test files indefinitely, qualifying each