When
you are writing test cases , whether they are unit tests, tests for
test-driven development, regression tests, or any other kind, there are
some functions you will find yourself doing each time. These include the basics of an application, a mechanism for launching tests, and a way to report results. You
could write these functions each time you start testing a new feature
or you could write them once and leverage them every time. A test harness, at its most simple, is just that—a system to handle the elements you’ll repeat each time you write a test. Think of it as scaffolding upon which you will build your tests.
A basic test harness will contain at least the following elements: application basics, test launching, and result reporting. It may also include a graphical user interface, logging, and test case scripting. It should be noted that harnesses will generally be written for one language or runtime (C/C++, Java, .Net, etc.). It is hard to write a good harness which will work across runtimes.
To run a test, the first thing you need is an application. Each operating system has a different way to write an application. For example, on windows if you want any GUI, you need things like a message pump and a message loop. This handles the interaction with the operating system. The harness will include code to start the program, open any required files, select which cases are run, etc.
The next thing you need is a way to actually launch the tests. Most of the time a test case is just a function or method. This function does all of the actual work of the test case. It calls the API in question, verifies the results, and informs the framework whether the case passed or failed. The
test harness will provide a standardized way for test cases to
advertise themselves to the system and an interface by which they will
be called. In the most simple system, a C/C++ harness might advertise its cases by adding function pointers to a table. The
harness may provide some extra services to the test cases by allowing
them to be called in a specified order or to be called on independent
threads.
The third basic pillar of a test harness is a way to inform the user of which cases pass and which fail. They provide a way for test cases to output messages and to report pass/fail results. In the most basic harness, this could be just a console window in which test cases will print their own messages. Better than that is a system which automatically displays each test case name and its results. Usually there is a summary at the end of how many cases passed or failed. This could be textual or even a big red or green bar. More advanced systems have built-in logging systems. They
provide a standardized way for the test cases to output trace messages
informing the user of each important call as it is being made. The
harness may simply log to a text file but it may also provide a
parsable format like XML or even interact directly with a database for
result retention.
At Microsoft, many groups write their own test harnesses which are specialized to support the unique needs of the organization. For example, my team uses a harness called Shell98 which has, among other things, support for device enumeration. Good examples of a freely available test harnesses are the xUnit series of tests like cppUnit, nUnit, and jUnit. These are designed for unit testing and are not very feature-rich. I’ve used cppUnit which is very basic and nUnit which is pretty slick. The xUnit harnesses do not do any logging and you do not have control over the order in which tests are run. They are intended for a user to run and visually inspect the results. The harness I use allows for scripting and outputs its results to a database.
http://blogs.msdn.com/b/steverowe/archive/2006/04/27/585758.aspx
No hay comentarios.:
Publicar un comentario