In the first part of this post series on the topic “Advanced Rule Testing”, I’ve explained some theory of advanced rule testing. At the end, I promised to present some more advantages. Here they come: Some testing techniques that are only possible, if you use advanced rule testing with Visual Rules.

1. Use functions to analyze your test results

With a conventional rule test, it is not possible to apply functions to output variables in order to check for an expected value. A good example is the count() function. In some cases, it is important, that, based on the given inputs, a list should be returned empty or with a specific number of elements. We had that in numerous projects using our Dynamic Application Framework where we dynamically build up and extend tables by modifying lists.

Just envoke the count() function in the test rule. Take the result and write it into an output variable of type Integer of the test rule and you get a value that you can compare your expected results to. Another function often used in test rules is the round() function. Based on the environment and the test data, slight differences may be accepted in the tests. Examples are shown in the pictures below.

2. Performance tests

Have you checked out the statistics feature of Visual Rules? It tells you exactly the duration for executing a rule element. Usually it is a fragment of a millisecond. But in a batch run with multiple input records, execution times of flow rule parts can vary between the records. With the times cumulated, low performance hotspots in your rule model can be identified. To tell the long story quickly: What you can do with a dedicated test rule are two things:

Set a timestamp before and after you call the rules to be tested. Use the diffTimestamp function to keep track of the time consumed by steadily growing rules during the development lifecycle.

Call the rules to be tested in a loop. Set a counter or load mock data prior to looping over your selected rules. Only that way, you can get representative performance statements.

Mind, though, that those tests will create performance statements that can only be compared with figures measured on the same system. In other environments, performance will be different, based on parallelization, system configuration, and server load!

Model using round(), count() and a performance measurement

Model using round(), count() and a performance measurement

 

Comparision that shows, how different examples show different aspects of the testing

Comparision that shows, how different examples show different aspects of the testing

3. Portfolio tests

That brings us right to the next use case that requires its own dedicated test rule. Sometimes, the business side of a Visual Rules project wants to run the whole portfolio of data records through the rules. This task is important before going live. Application owners and users need to know whether there are gaps or deviations from the new rules to the current system’s results. We call those tests Portfolio tests. They are very simple, really.

Create a test rule with three sub-rules. The first sub-rule is called ‘Get test data’. It contains a ‘database query action’ or a ‘CSV read action’ to retrieve the portfolio data.

The second sub-rule is called ‘Compute’. This rule is evoked in a loop, once for each of the results of ‘Get test data’. It mainly prepares the incoming data and loads it into the actual business rules (the ones, that are tested).

In the same loop, next comes the ‘Publish results’ sub-rule. It transforms the results into the target format you need. Usually a database or a CSV file. This sub-tree might also contain another rule that reads reference results and writes them next to the results generated by your test rules. Now the only thing left to do is kicking-off execution of portfolio test rules. You can do that e.g. by using the built-in test editor of Visual Rules or, if you use the Visual Rules Execution Platform, by calling the rules via web service.

The following picture shows a very simple portfolio test. Usually you would add validations, reference data enrichments from other sources, etc.

Portfolio Test Rule

Portfolio Test Rule

Those three Visual Rules testing best practices have been around for a while. We regularly use them in our projects. They are quickly implemented, and deliver huge value throughout the rule authoring and the development phase.

It would be interesting to see, how you are testing rules. Which modeling techniques do you use? Do you face the same challenges that are described here? Or maybe others that you can complement with additional testing tips?

About The Author

Christof Pitzer

My job at Bosch Software Innovations is to support clients in realizing Visual Rules projects. Specialized in risk rating projects, I helped Risk Analysts in Germany, Switzerland, the US and other countries with implementing business logic with Visual Rules. In former jobs I worked as QM consultant and Requirements Engineer.

2 Responses

  1. Robert

    Sounds to me as if the new Batch Platform would be an ideal candidate for Portfolio tests. I can see the three steps you described here as a core concept there.

    In addition to that, Batch Platform also has job scheduling so the Portfolio tests could be run automatically at night.

    @Christof
    What do you think?

    Reply
  2. Christof Pitzer

    Hi Robert,
    you are right. Both, the Batch Platform and the Portfolio Test as described above can be mutually exchanged.

    However, each concept shows characteristics that make it more or less suitable for certain use cases.

    The Portfolio Test uses the onboard features of the VR Rule Modeler. It can be set up locally and always runs on the currently available models in the VR workspace. Therefore the effects of model changes can quickly be analyzed and assest.

    The Batch Platform on the other hand requires deployed artifacts. Here rule versions are embedded into a rule lifecycle with automated test execution, build, etc. It needs to be set up and configured for the productive use (which might include clustering or scheduling for example).

    In summary I would say, even though both techniques for running a batch can be used for different use cases, the Portfolio Test should rather be used as a development tool, where as the Batch Platform is used for scheduled, productive batch executions.

    Reply

Leave a Reply

Your email address will not be published.