Performance Tip (Unit tests and ANT)
You've probably heard about the need to write unit tests before, and some of you may be religious about writing them. I won't get into all the how and why about unit tests. But...
I'll admit I've never been that good about writing unit tests for my CFML code. However, I should know better. We have over 15000 unit tests for ColdFusion, it's these tests that I depend on every day, especially when making changes that risk breaking backward compatibility in the language.
So tonight I was playing with CFUNIT and ANT. With this combination I was able to discover a serious deadlock issue. Finding a serious problem so quickly and in my first few tests has changed my mind about cfml unit tests. I will write them as soon as I can in every project and you should too.
In general unit tests are good for making sure you don't break functionality. But if something works with 1 users and not with 5 it's still broken. To test this, Ant has a tag called
Note: You should still run a real-world load test against you application, before you put it into production, with a tool like OpenSta or Segue SilkPerformer. Using the unit tests will make sure that small chunks of code don't interfere with each other. Using a full load test tool makes sure the whole sections of the application don't interfere with each other and that your servers can handle the load you are expecting.
Here is the ANT build.xml I wrote to run my unit tests. To do a simple test of the functionality I run the "ant main" task and to test performance I run the "ant performance" task.
<project name="CFUnit" default="main" basedir=".">
<taskdef name="CFUnit" classname="net.sourceforge.cfunit.ant.CFUnit"/>
<property name="domain" value="http://localhost:8500/web/applications/MyProject/" />
<target name="main">
<CFUnit testcase="${domain}components/users/test/UserBean.cfc" verbose="true" />
<CFUnit testcase="${domain}components/users/test/UserDaoRead.cfc" verbose="true" />
<CFUnit testcase="${domain}components/users/test/UserDaoCreate.cfc" verbose="true" />
<CFUnit testcase="${domain}components/users/test/UserDaoCreate2.cfc" verbose="true" />
<CFUnit testcase="${domain}components/users/test/UserDaoUpdate.cfc" verbose="true" />
<CFUnit testcase="${domain}components/users/test/UserDaoDelete.cfc" verbose="true" />
<!--
-->
</target>
<!--
Run the tasks 10 times with 5 threads
-->
<target name="performance">
<parallel threadCount="5">
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
<antcall target="main"/>
</parallel>
</target>
</project>
This might help too. This is the Ant build.xml I have in my root folder. This ant file includes all of the build.xml files in sub-folders. From here I can run all of my unit tests at once.
<project name="CFUnit" default="main" basedir=".">
<taskdef name="CFUnit" classname="net.sourceforge.cfunit.ant.CFUnit"/>
<property name="domain" value="http://localhost:8500/web/applications/MyProject/" />
<target name="main">
<ant antfile="${basedir}/users/test/build.xml" dir="${basedir}/users/test" target="main">
<property name="domain" value="${domain}"/>
</ant>
<ant antfile="${basedir}/products/test/build.xml" dir="${basedir}/products/test" target="main">
<property name="domain" value="${domain}"/>
</ant>
</target>
<target name="performance">
<ant antfile="${basedir}/users/test/build.xml" dir="${basedir}/users/test" target="performance">
<property name="domain" value="${domain}"/>
</ant>
<ant antfile="${basedir}/products/test/build.xml" dir="${basedir}/products/test" target="performance">
<property name="domain" value="${domain}"/>
</ant>
</target>
</project>
The part I don't understand (not having researched enough) is how to test bits realistically. For example, I have a saveMember() routine. It takes 10 parameters. Ideally I would test out of bounds, invalid datatypes, data that is too large, etc. How would I write a unit test using cfunit and ant to do this? Does it require writing 50 tests to cover all of the combinations?