JMeter Performance Testing in Cognos Analytics
Date: 03/05/2017

By Chris Chapman

Determining the optimal tuning settings for a Cognos Analytics environment can be a challenging and confusing exercise. Even when there is an understanding of how a specific setting behaves, there is no guarantee until it is implemented, that it will have the desired effect within a specific environment. Additionally, Cognos environments evolve over time as user bases shift and reporting requirements change.   This leaves many Cognos administrators guessing and checking as they make settings changes to their Cognos environments. This methodology, unsurprisingly, is problematic.

A way to resolve this problematic practice is to run a performance test. Conducting a performance test is a great way to validate an environment after changes have been made.

And here lies the challenge.  Setting up a performance test can be intimidating. What tool should be used? What is the cost? Where to start with the script? Out-of-the-box solutions can be very costly and still require scripting and setup.

Fortunately, there is JMeter for Cognos Analytics.

What exactly is JMeter?

JMeter is a pure Java application that is part of the Apache project. It is designed to perform load tests and performance tests against Web applications and resources. The flexibility of JMeter provides users the ability to test against a growing list of different systems. More can be read on JMeter, including download information, from the link at the end of this article.

IBM released their own JMeter test script for Cognos 10.2.x. The link to the Proven Practices article is provided below. The script and article are informative for those working with JMeter or testing for the first time. However, for extended use, or use in a continually changing environment, I have found this script to be inflexible in its design. JMeter is full of features like the ability to read in variables from CSV files, or to properly handle SSL connections. These features greatly improve the usability of a JMeter script to provide a more robust and thorough test. It is highly recommended to become familiar with the JMeter documentation to learn about these additional features and incorporate them into a script.

Let’s take a look at a couple of practical examples that a JMeter test could be used for in Cognos.

Regression Test

Large scale changes to an environment, such as a model change, can be difficult and time consuming to test. It often can lead to the tedious process of launching individual reports to validate them one-by-one. With JMeter, you can execute a test that will run all reports in your environment to ensure they are properly working. Or, you can scale the test to only run reports that are impacted by a recent change (e.g., only selecting reports impacted by the model change). This test will cycle through the list of reports and run them one-by-one. You can then view the results in JMeter to see if any reports returned errors. An example of the results output is in the below screenshot.

JMeter_Regression Test.png

You can see JMeter highlighting the report that returned the error. This assertion (a test element that detects responses) records that an error occurred. There is no specific insight into what specific error was returned. However, using XPath, you can create an assertion that can grab the specific error Cognos returned. The outputs shown can also be saved to a CSV file and opened in Excel if additional reporting or sorting is needed.

This simple script can save many valuable man hours individually validating reports after major environment changes.

Load Test

The same script used for the regression test can also be used for a more intense load test against the system. Instead of running reports one-by-one, multiple concurrent users can be simulated running multiple reports. This load can then be used to test for bottlenecks or instabilities in the environment.

The Thread Group element in JMeter contains the information for defining the specific load profile you are seeking. The key components are:

  • Number of Threads (users): Each thread will execute the test plan in its entirety. Multiple threads are used to simulate multiple users.
  • Ramp-Up Period (in seconds): Tells JMeter the amount of time to ramp-up to the total number of threads/users defined.
  • Loop Count: Number of times a thread will loop through each element of a test.
  • Scheduler: This can be used to provide the test with a start and stop time.

A sample configuration could look similar to the screenshot below. This configuration will simulate 5 users, over a 20 minute test, and will ramp up to the max number of users over a 5 minute period.

JMeter_Thread Group.png

Most likely you will not want to run all reports in your environment for the load test. Instead, it may make more sense to choose a profile of reports that represent typical usage in your environment. Or, select high priority or high cost reports that are more susceptible to failure under heavier workloads.

Like the regression test, JMeter will provide test results to review. In this instance, we are looking at report performance over time. Is report performance or runtime being impacted as the test ramps up to full load? The Response Time Graph listener will provide a graphical view that shows report runtimes over the length of the test. Other listeners (a test element that records results) can provide more granular detail, such as the Aggregate Report.

JMeter_Response time graphc.png


Taking the time to learn JMeter and create a script for your environment can help you save time and identify problems before they hit the end users.  You can check out the aforementioned links here:


IBM Cognos BI 10.2.1.x JMeter Stability Package:

I hope you find this article useful.  If you would like to learn more please reach out to us at:

Australia Singapore, Philippines, Thailand     United States
Cornerstone PMsquare | A Cornerstone Group Company     PMsquare

Call +61 1300 840 048 or
email Piers Wilson
Call +65 6635 1700 or
email Carsten Brandt
    Call +1 (630) 607 0570 or
    email Chris Loechel

Blog post shared courtesy of PMsquare LLC