Re: Writing New Constraints for NUnit
"Charlie Poole" <[email protected]> Sun, 7 Sep 2008 17:13:20 -0700
| Newsgroups | gmane.comp.windows.dotnet.nunit.user |
|---|---|
| Message-ID | <001801c91147$b3e836a0$6401a8c0@ferrari> |
This is a multi-part message in MIME format. --===============0704520312== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0019_01C9110D.07895EA0" This is a multi-part message in MIME format. ------=_NextPart_000_0019_01C9110D.07895EA0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Yes, we're eating our own dogfood! I added a number of tests that use 2.5 features. So using TD.Net is not possible till there's a release that supports it. I think you're running into the same problem reported by somebody else on one of our lists - the Nant build script for nunit.framework contains an entry for FactoriesAttribute.cs rather than TestCases attribute. I fixed this only recently. I suggest trying to get the very latest source. If you still have problems, let me know so I can update CVS correctly (One mistake I sometimes make is to update from within the src directory. Make sure you do it from the top level directory to get everything.) Charlie _____ From: Simone Busoli [mailto:[email protected]] Sent: Sunday, September 07, 2008 4:17 PM To: Charlie Poole Cc: [email protected] Subject: Re: [Nunit-users] Writing New Constraints for NUnit Thanks for the guide. I'm trying to write the BinarySerializableConstraint but I'm having some issues with building and testing NUnit. So far I had been able to build and run tests from within VS via TD.NET, but now I noticed that base test classes use NUnit 2.5 features so I had to rely on something else. I tried building NUnit with nant but no success, it complaints about missing files, specifically a Factory[something].cs. Then, since I can run the build with VS 2008, I downloaded NUnit 2.5 alpha 3 (won't trust the build on my machine) and tried with that, but I'm getting red all over the place in the tests for the constraints, while everything's green for the other stuff in Tests and Syntax namespaces. At a quick look it seems that what's failing are the tests defined in the ConstraintTestBase class (FailsWithBadValues, ProvidesProperFailureMessage), those using the TestCases attribute in other words. On Sun, Sep 7, 2008 at 7:35 PM, Charlie Poole <[email protected]> wrote: Hi All, I've been asked to provide more info on the increasingly complicated process of adding a new constraint to NUnit. Since I still expect it to change - and hopefully get simpler - I'm doing it on these mailing lists as "current but subject to change" information. This is basically a "how-to" but I've added a postscript about why the design is as it is at the end. Actually, adding a constraint is easy. It's documented here: http://nunit.org/?p=customConstraints <http://nunit.org/?p=customConstraints&r=2.5> &r=2.5 It gets more complicated when you also want to work with the syntax of NUnit's fluent interface for Asserts. I generally write constraints test first, jumping between the tests and the constraint and between the syntax tests and the syntactic elements. That's hard to describe, so I'll just tell you what's needed at the end here. 1. The Constraint I assume you're writing a simple constraint, one that does not operate on other constraints. Inherit your constraint class from Constraint and override the two abstract members Match() and WriteDescriptionTo(). Be sure to set the value of the protected field "actual" in your Match method, since that is used in any error message. Depending on the constraint you are writing, you might need to override other methods, but leave them alone unless you need to do it. If your constraint has a generic variation, be sure to define it within the scope of an #if NET_2_0 condition. If it operates on any collections, try to make it work with IEnumerable only and avoid copying the collection. An important rule for constraints is to check for invalid arguments and throw an exception. Do NOT just return false. While this may work for simple cases, it will mess up more complicated usage where constraints are combined with various operators. And, of course, it may hide an error. 2. Constraint Tests Derive one or more classes from ConstraintTestBase. Each of those will be used to test one instance of the constraint. How many you need depend on how many variations there are within your constraint. Be sure to create separate tests for generic variations -within an #if statement. Write any extra tests you need, adding them to the same classes or within a separate fixture. NOTE: If you stop here, your constraint is useable by creating it with new. You can write statements like: Assert.That( xxx, new MyConstraint(...) ); NOTE: See below if your constraint takes modifiers 3. Syntactic Elements You will need to pick one or more key words for use in the NUnit syntax. Pick something that will be meaningful while keeping it reasonably short. Currently, you must add your syntactic element in three places. I'm looking at using code generation to eliminate this duplication. The following classes need to be updated: * PartialConstraintExpression - look at how other simple constraints are implemented here. If your constraint takes modifiers, see below. Otherwise, you will just need a method similar to this one for TypeOf: public ConstraintExpression TypeOf(Type expectedType) { return this.Append(new ExactTypeConstraint(expectedType)); } * Is/Has - you decide which is better, or consult on the list if you're convinced that you need a new class. Add a method similar to this: public static ConstraintExpression TypeOf(Type expectedType) { return new PartialConstraintExpression().TypeOf(expectedType); } * ConstraintFactory: Add a method similar to this: public ConstraintExpression TypeOf(Type expectedType) { return Is.TypeOf(expectedType); } NOTE: It's important to keep the actual logic in the PartialConstraintBuilder class, with the others depending on it. NOTE: If your constraint takes no args, use properties rather than methods - it's easier to read. 4. Syntax Tests Derive one or more classes from SyntaxTest. You will need one for each different syntax you need to test. If you forget to add code to any of the three classes above, these tests will tell you. At this point you're done... UNLESS you want to add modifiers to your class. If your constraint will take any modifiers, I suggest you do all of the above first, then add the first one and make it work, then add the others. Come back to this note if/when you want to add a modifier... Welcome Back! In the NUnit syntax, a modifier is a property or method that returns the object itself, after making some sort of state change. Existing examples of modifiers are Within(), IgnoreCase and Descending. Here's a simple modifier: public EqualConstraint IgnoreCase { get { caseInsensitive = true; return this; } } With the above addition, it's possible to write new EqualConstraint("hello").IgnoreCase but NOT Is.EqualTo("hello").IgnoreCase In order to make the syntax work, it's currently necessary to create a ConstraintModifier class. These classes inherit from ConstraintModifier and are usually implemented as nested classes within the constraint itself, so as to allow access to private members of the constraint. Here's a simplified version of the one for EqualConstraint: public class Modifier : ConstraintModifier { private EqualConstraint constraint; public Modifier(EqualConstraint constraint, ConstraintExpression builder) : base(constraint, builder) { this.constraint = constraint; } public Modifier IgnoreCase { get { constraint.caseInsensitive = true; return this; } } } After implementing the above, the three syntax classes need to be changed so our methods return EqualConstraint.Modifier. ConstraintFactory and Is/Has require no further changes but PartialConstraintExpression will now look like this: public EqualConstraint.Modifier EqualTo(object expected) { EqualConstraint constraint = new EqualConstraint(expected); return new EqualConstraint.Modifier(constraint, this.Append(constraint)); } That's "all" there is to it! :-) Charlie Theoretical PostScript: In case you wonder Why we neeed all the classes: We want to provide a good object model, while *at*the*same*time* having only reasonable choices appear in the intellisense. And we want whatever you can compile to make sense as much as is possible, so that there aren't too many runtime checks. So we need separate PartialConstraintExpression and ConstraintExpression classes to make sure you can't enter ...And.And... or ...Null.Null... and similar meaningless stuff. We need Is/Has to provide static methods to get the expression started. We need ConstraintFactory, to supply instance methods for use by those who prefer to derive test classes from AssertionHelper. We could eliminnate that class by eliminating the feature. In particular, the Modifier classes are needed because we are dealing with a ConstraintExpression, but want the choices offered to the user to represent those provided for an Equal Constraint. The Modifier class ties together the whole expression and the "leading edge" constraint in a way that lets us have both. If the user ends the expression, the Modifier's implementation of IConstraint is used. If the user types "IgnoreCase" the EqualConstraint is used. If the user types "And" or "Or", that's handled by the ConstraintExpression. I'm working on a few approaches to this extra class. ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100 <http://moblin-contest.org/redirect.php?banner_id=100&url=/> &url=/ _______________________________________________ Nunit-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nunit-users ------=_NextPart_000_0019_01C9110D.07895EA0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META http-equiv=3DContent-Type content=3D"text/html; = charset=3Dus-ascii"> <META content=3D"MSHTML 6.00.6000.16705" name=3DGENERATOR></HEAD> <BODY> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>Yes,=20 we're eating our own dogfood! I added a number of tests that use 2.5=20 features.</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>So=20 using TD.Net is not possible till there's a release that supports=20 it.</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2></FONT></SPAN> </DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>I=20 think you're running into the same problem reported by somebody else on=20 one</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>of our=20 lists - the Nant build script for nunit.framework contains an entry=20 for</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>FactoriesAttribute.cs rather than TestCases attribute. I fixed = this only=20 recently.</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2></FONT></SPAN> </DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>I=20 suggest trying to get the very latest source. If you still have = problems,=20 let</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>me=20 know so I can update CVS correctly</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2></FONT></SPAN> </DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>(One=20 mistake I sometimes make is to update from within the src=20 directory.</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>Make=20 sure you do it from the top level directory to get=20 everything.)</FONT></SPAN></DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2></FONT></SPAN> </DIV> <DIV><SPAN class=3D937500600-08092008><FONT face=3DArial color=3D#0000ff = size=3D2>Charlie</FONT></SPAN></DIV><BR> <BLOCKQUOTE dir=3Dltr=20 style=3D"PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px = solid; MARGIN-RIGHT: 0px"> <DIV class=3DOutlookMessageHeader lang=3Den-us dir=3Dltr align=3Dleft> <HR tabIndex=3D-1> <FONT face=3DTahoma size=3D2><B>From:</B> Simone Busoli=20 [mailto:[email protected]] <BR><B>Sent:</B> Sunday, September = 07, 2008=20 4:17 PM<BR><B>To:</B> Charlie Poole<BR><B>Cc:</B>=20 [email protected]<BR><B>Subject:</B> Re: [Nunit-users] = Writing=20 New Constraints for NUnit<BR></FONT><BR></DIV> <DIV></DIV> <DIV dir=3Dltr>Thanks for the guide. I'm trying to write the=20 BinarySerializableConstraint but I'm having some issues with building = and=20 testing NUnit. So far I had been able to build and run tests from = within VS=20 via <A href=3D"http://TD.NET">TD.NET</A>, but now I noticed that base = test=20 classes use NUnit 2.5 features so I had to rely on something else. I = tried=20 building NUnit with nant but no success, it complaints about missing = files,=20 specifically a Factory[something].cs. Then, since I can run the build = with VS=20 2008, I downloaded NUnit 2.5 alpha 3 (won't trust the build on my = machine) and=20 tried with that, but I'm getting red all over the place in the tests = for the=20 constraints, while everything's green for the other stuff in Tests and = Syntax=20 namespaces. At a quick look it seems that what's failing are the tests = defined=20 in the ConstraintTestBase class (FailsWithBadValues,=20 ProvidesProperFailureMessage), those using the TestCases = attribute in=20 other words.<BR><BR> <DIV class=3Dgmail_quote>On Sun, Sep 7, 2008 at 7:35 PM, Charlie Poole = <SPAN=20 dir=3Dltr><<A=20 href=3D"mailto:[email protected]">[email protected]</A>></SPAN> = wrote:<BR> <BLOCKQUOTE class=3Dgmail_quote=20 style=3D"PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: = #ccc 1px solid">Hi=20 All,<BR><BR>I've been asked to provide more info on the increasingly = complicated<BR>process of adding a new constraint to NUnit. Since I = still=20 expect<BR>it to change - and hopefully get simpler - I'm doing it on = these<BR>mailing lists as "current but subject to change"=20 information.<BR><BR>This is basically a "how-to" but I've added a = postscript=20 about<BR>why the design is as it is at the end.<BR><BR>Actually, = adding a=20 constraint is easy. It's documented here:<BR><A=20 href=3D"http://nunit.org/?p=3DcustomConstraints&r=3D2.5"=20 = target=3D_blank>http://nunit.org/?p=3DcustomConstraints&r=3D2.5</A> = It gets=20 more<BR>complicated when you also want to work with the syntax<BR>of = NUnit's=20 fluent interface for Asserts.<BR><BR>I generally write constraints = test=20 first, jumping between the<BR>tests and the constraint and between = the=20 syntax tests and<BR>the syntactic elements. That's hard to describe, = so I'll=20 just<BR>tell you what's needed at the end here.<BR><BR>1. The=20 Constraint<BR>I assume you're writing a simple constraint, one that=20 does<BR>not operate on other constraints. Inherit your = constraint<BR>class=20 from Constraint and override the two abstract members<BR>Match() and = WriteDescriptionTo(). Be sure to set the value<BR>of the protected = field=20 "actual" in your Match method, since<BR>that is used in any error = message.=20 Depending on the constraint<BR>you are writing, you might need to = override=20 other methods, but<BR>leave them alone unless you need to do = it.<BR><BR>If=20 your constraint has a generic variation, be sure to = define<BR>it=20 within the scope of an #if NET_2_0 condition. If it operates<BR>on = any=20 collections, try to make it work with IEnumerable<BR>only and avoid = copying=20 the collection.<BR><BR>An important rule for constraints is to check = for=20 invalid<BR>arguments and throw an exception. Do NOT just return=20 false.<BR>While this may work for simple cases, it will mess up=20 more<BR>complicated usage where constraints are combined = with<BR>various=20 operators. And, of course, it may hide an error.<BR><BR>2. = Constraint=20 Tests<BR>Derive one or more classes from ConstraintTestBase. Each=20 of<BR>those will be used to test one instance of the = constraint.<BR>How many=20 you need depend on how many variations there are<BR>within your = constraint.=20 Be sure to create separate tests<BR>for generic variations -within = an #if=20 statement.<BR><BR>Write any extra tests you need, adding them to the = same<BR>classes or within a separate fixture.<BR><BR>NOTE: If you = stop here,=20 your constraint is useable by<BR>creating it with new. You can write = statements like:<BR> Assert.That( xxx, new MyConstraint(...)=20 );<BR><BR>NOTE: See below if your constraint takes = modifiers<BR><BR>3.=20 Syntactic Elements<BR>You will need to pick one or more key words = for=20 use<BR>in the NUnit syntax. Pick something that will = be<BR>meaningful while=20 keeping it reasonably short.<BR><BR>Currently, you must add your = syntactic=20 element in<BR>three places. I'm looking at using code = generation<BR>to=20 eliminate this duplication. The following<BR>classes need to be=20 updated:<BR><BR>* PartialConstraintExpression - look at how=20 other<BR> simple constraints are implemented here. = If<BR> your=20 constraint takes modifiers, see below.<BR> Otherwise, you will = just=20 need a method similar<BR> to this one for = TypeOf:<BR><BR> public=20 ConstraintExpression TypeOf(Type expectedType)<BR> {<BR> =20 return this.Append(new=20 ExactTypeConstraint(expectedType));<BR> }<BR><BR>* Is/Has - you = decide=20 which is better, or consult<BR> on the list if you're convinced = that=20 you need<BR> a new class. Add a method similar to=20 this:<BR><BR> public static ConstraintExpression TypeOf(Type=20 expectedType)<BR> {<BR> return new=20 = PartialConstraintExpression().TypeOf(expectedType);<BR> }<BR><BR>*=20 ConstraintFactory: Add a method similar to this:<BR><BR> public = ConstraintExpression TypeOf(Type expectedType)<BR> {<BR> =20 return Is.TypeOf(expectedType);<BR> }<BR><BR>NOTE: It's = important=20 to keep the actual logic in the<BR>PartialConstraintBuilder class, = with the=20 others depending<BR>on it.<BR><BR>NOTE: If your constraint takes no = args,=20 use properties<BR>rather than methods - it's easier to = read.<BR><BR>4.=20 Syntax Tests<BR>Derive one or more classes from SyntaxTest. You = will<BR>need=20 one for each different syntax you need to test.<BR>If you forget to = add code=20 to any of the three classes<BR>above, these tests will tell = you.<BR><BR>At=20 this point you're done... UNLESS you want to add<BR>modifiers to = your class.=20 If your constraint will take<BR>any modifiers, I suggest you do all = of the=20 above first,<BR>then add the first one and make it work, then = add<BR>the=20 others. Come back to this note if/when you want<BR>to add a=20 modifier...<BR><BR>Welcome Back!<BR><BR>In the NUnit syntax, a = modifier is a=20 property or method<BR>that returns the object itself, after making = some=20 sort<BR>of state change. Existing examples of modifiers = are<BR>Within(),=20 IgnoreCase and Descending. Here's a = simple<BR>modifier:<BR><BR> public=20 EqualConstraint IgnoreCase<BR> {<BR> get { = caseInsensitive=20 =3D true; return this; }<BR> }<BR><BR>With the above addition, = it's=20 possible to write<BR> new = EqualConstraint("hello").IgnoreCase<BR>but=20 NOT<BR> Is.EqualTo("hello").IgnoreCase<BR><BR>In order to make = the=20 syntax work, it's currently<BR>necessary to create a = ConstraintModifier=20 class.<BR>These classes inherit from ConstraintModifier<BR>and are = usually=20 implemented as nested classes<BR>within the constraint itself, so as = to=20 allow<BR>access to private members of the constraint.<BR><BR>Here's = a=20 simplified version of the one = for<BR>EqualConstraint:<BR><BR> public=20 class Modifier : ConstraintModifier<BR> {<BR> = private=20 EqualConstraint constraint;<BR><BR> public=20 Modifier(EqualConstraint constraint,<BR> = =20 ConstraintExpression = builder)<BR> =20 : base(constraint, builder)<BR> {<BR> = =20 this.constraint =3D constraint;<BR> = }<BR><BR> =20 public Modifier IgnoreCase<BR> {<BR> = =20 get { constraint.caseInsensitive =3D true; return this; = }<BR> =20 }<BR> }<BR><BR>After implementing the above, the three = syntax=20 classes<BR>need to be changed so our methods return=20 EqualConstraint.Modifier.<BR>ConstraintFactory and Is/Has require no = further=20 changes but<BR>PartialConstraintExpression will now look like=20 this:<BR><BR> public EqualConstraint.Modifier EqualTo(object=20 expected)<BR> {<BR> EqualConstraint constraint =3D = new=20 EqualConstraint(expected);<BR> return new=20 = EqualConstraint.Modifier(constraint,<BR>this.Append(constraint));<BR>&nbs= p;}<BR><BR>That's=20 "all" there is to it! :-)<BR><BR>Charlie<BR><BR>Theoretical=20 PostScript:<BR><BR>In case you wonder Why we neeed all the=20 classes:<BR><BR>We want to provide a good object model, while=20 *at*the*same*time*<BR>having only reasonable choices appear in the=20 intellisense. And<BR>we want whatever you can compile to make sense = as much=20 as<BR>is possible, so that there aren't too many runtime = checks.<BR><BR>So=20 we need separate PartialConstraintExpression = and<BR>ConstraintExpression=20 classes to make sure you can't<BR>enter ...And.And... or = ...Null.Null... and=20 similar<BR>meaningless stuff.<BR><BR>We need Is/Has to provide = static=20 methods to get the<BR>expression started.<BR><BR>We need = ConstraintFactory,=20 to supply instance methods<BR>for use by those who prefer to derive = test=20 classes<BR>from AssertionHelper. We could eliminnate that<BR>class = by=20 eliminating the feature.<BR><BR>In particular, the Modifier classes = are=20 needed because we<BR>are dealing with a ConstraintExpression, but = want=20 the<BR>choices offered to the user to represent those = provided<BR>for an=20 Equal Constraint. The Modifier class ties together<BR>the whole = expression=20 and the "leading edge" constraint<BR>in a way that lets us have = both. If the=20 user ends the<BR>expression, the Modifier's implementation of=20 IConstraint<BR>is used. If the user types "IgnoreCase" the=20 EqualConstraint<BR>is used. If the user types "And" or "Or", that's=20 handled<BR>by the ConstraintExpression.<BR><BR>I'm working on a few=20 approaches to this extra=20 = class.<BR><BR><BR><BR><BR><BR><BR>---------------------------------------= ----------------------------------<BR>This=20 SF.Net email is sponsored by the Moblin Your Move Developer's=20 challenge<BR>Build the coolest Linux based applications with Moblin = SDK=20 & win great prizes<BR>Grand prize is a trip for two to an Open = Source=20 event anywhere in the world<BR><A=20 = href=3D"http://moblin-contest.org/redirect.php?banner_id=3D100&url=3D= /"=20 = target=3D_blank>http://moblin-contest.org/redirect.php?banner_id=3D100&am= p;url=3D/</A><BR>_______________________________________________<BR>Nunit= -users=20 mailing list<BR><A=20 = href=3D"mailto:[email protected]">[email protected]= eforge.net</A><BR><A=20 href=3D"https://lists.sourceforge.net/lists/listinfo/nunit-users"=20 = target=3D_blank>https://lists.sourceforge.net/lists/listinfo/nunit-users<= /A><BR></BLOCKQUOTE></DIV><BR></DIV></BLOCKQUOTE></BODY></HTML> ------=_NextPart_000_0019_01C9110D.07895EA0-- --===============0704520312== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ --===============0704520312== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Nunit-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nunit-users --===============0704520312==--