NUnit and Microsoft Unit Tests use an approach of attributes on a method to identify a test which is expected to throw an exception, to differentiate tests which are expected to not throw an exception and just report success or failure based on Assertions (AreEqual, IsFalse, IsTrue, etc).  For individual tests, this approach is very effective.

Recently, I wrote two complex classes which required iterative testing on sets of parameters.  One project at work used some complex logic to evaluate arguments.  This required over 80 lines of parameter combinations to be validated.  This project used the Microsoft Unit Test architecture.

The other project was a personal project for a Url parser which was designed to validate patterns used by the HttpListener object and be less restrictive on content in the host name and other items.  This project uses NUnit 3.0.  Both of these projects needed iterative testing to cover the many variations in behavior from combinations of parameters.  And the results could be either assertions, or expected exceptions.

The example here is for NUnit.  Microsoft Unit Tests use attributes for similar iteration.  The CSV parsing in the NUnit tester is handled by a handy assembly named CsvReader available via NuGet.  And the CSV or TSV file defining the iteration tests is copied to the output folder, where the test assembly expects it.

This file contains the test data definition. There are some key columns in this file which makes it a good template:

  • The DataRow column: similar to the Microsoft Unit Test architecture, this column marks the row of the test for use in output to identify any row with problems.
  • Any column containing {empty} represents an empty string to reduce ambiguity.
  • The column ThrowsException is either {empty} to identify tests expected to be exception-free and verified with Assert.*(), or contains the class name of the Exception expected.
  • ExceptionContains is a further test of an an exception: it can be {empty}, or be a string expected to be contained in the exception message text to further qualify the exception expected.
  • Notes is a column not used in the test: it is to help the poor developer who can’t decrypt your specific intent from the test properties.

All other columns are properties for the test.  Columns containing expected results will sometimes appear as “n/a” because an exception is expected, so the “n/a” is a semaphore that an exception occurs in test.

This file contains the test class for the UrlTest, which uses the TSV file as its source.

Line 16 Contains the TestData class which provides the test properties to the iterative test method at line 59.  The properties are loaded into a StringDictionary object, which uses the column name (or property name) as the key for its value.

The test method at line 59 converts the content to local variables, then uses an if() statement to determine how to call the work method: expecting an error, or not.  In case there is a need to debug a single line of the TSV (i.e. a specific test case), the commented code in lines 86-89 can be used, by adjusting the targeted line’s DataRow value to the test parameters to debug.

That’s about it.  This method can be easily adjusted for other iterative tests.  The beauty of it is that is can used to test many arguments/one answer, or many arguments/many answers, and also handle exceptions thrown regardless of expectation.

No Comments
dotNET.ClassLibraries, dotNET.Techniques, Technology.Thoughts, Tips and Tricks, Visual Studio