Tests vs. Examples OpenSpace Session
Imagine venn diagram with tests & examples overlapping.
What examples cannot be used as tests?
* All examples are potentially useful as tests.
What tests cannot be expressed as Examples?
* Tests which are a procedure to find unknown problems (e.g. soak test, random
* Tests that don’t have defined scripts or inputs (e.g. exploratory tests)
Imagine venn diagram with examples as subset of tests.
An example can become a test either by running it manually it by hooking up a
test runner to run it automatically.
An example illustrates (or elaborates on) a requirement.
A test can prove that a requirement hasn't been met. It can indicate that the
requirement has been met to some degree but it cannot prove
it because a single failing test is all that is required to disprove it.
An example is not necessarily a good test; we may choose from all examples.
Imagine venn diagram with "examples" as subset of "all possible tests" and
intersecting another "tests actually run" bubble which is also
a subset of "all possible tests".
Advantages/Disadvantages of Various Words which we could use as part of new name
for the workshop & mailing list.
- "baggage for testers", everyone really
- is it being used as a verb or noun?
+ implies execution
+ finds defects (regression = "bug repellant")
- Implies certain tool requirements that have nothing to do with communicating
- mindshift (challenge)
+ different word (clear that it is a mindshit! No "baggage")
- too general = confusion as to what it means
+ prevent defects (which would be waste if found later)
+ includes analysts
+ implies different tool requirements that focus on communication.
- baggage (formal/proof of correctness)
- too general
Note: Example + "runner" = Test
We tried out the name Example-Driven Development (Tool) but had some concerns:
* Not clear that it is full system level and not unit level.
1-403-827-2967 or me@...
Author of the Jolt Productivity Award winning book "xUnit Test Patterns -
Refactoring Test Code". Learn more at http://xunitpatterns.com/index.html