With the latest versions of TM1, several of the customers I work with have experienced first-hand how much both general performance and TM1 Contributor behavior have improved. In this article I review ways to run stress tests in order to find out whether your TM1 model is scalable and reasonably stable.
What to Test?
There are a number of different aspects of your model that you will most likely want to test including:
- What is the number of concurrent users the model can face?
- What is the response time per task would be under heavy load?
- How busy the CPUs and memory would be?
- What is the bottom line for the server to go down in various scenarios?
Through multiple TM1 engagements with our customers, we came up with the idea of creating a stress testing tool to simulate intense user activities on the TM1 server. Then we could detect how the new environment would handle such a load. Also, we wanted to test the TM1 model in terms of potential locking problems when a lot of users are working at the same time and how it would affect the average time required to perform a particular task within a Contributor.
Ultimately, Apache JMeter was selected as the proper tool. This was due to a number of reasons, primarily: it is open source software, a 100% pure Java application, and a supporting TM1 plug-in. Apache JMeter delivers several handy features including the ability to create complete test plans, containing diverse listeners and test script recorders. It also enables you to simulate different load types on a server and you can analyze application performance and TM1 model behaviour under heavy load with a number of concurrent users.
How Stress Testing Works
First we created a number of TM1 users for various user roles (for us it was READ/WRITE roles and applications by country Australia/New Zealand) and we filled a user list specifying the approval hierarchy elements by user. Then we created customized test plans doing the steps that different types of users would normally do. These steps included opening different tabs in TM1 Contributor, data input in the cells which could be random or defined by dimension (i.e. enter Sales Plan numbers), selecting drop-downs from picklists when entering the data and reading the cube view with summary calculations that having a number of business rules and consolidations.
We also decided on the scenarios to be run with a number of user sessions to be run and ramp-up speed for them. It could be easily be configured in the Apache tool. We started with the scenarios such as 100 concurrent users login in 2 minutes and ended up with more realistic numbers like 60 users in 3 minutes.
When running the tests using Apache tool you can monitor what is going on in realtime and check the results in the reports showing statistics on time per transaction, throughputs and graphs. We also found the TM1 plug-in which provides graphical representations of theTM1 server activity, including classic TM1 top thread view, to be quite useful. For more information on the IBM Cognos TM1 Server Monitor Plug-in for Apache JMeter, click here.
After you finish running the tests, or if you are lucky enough to finally crash the server (in our case we were), the statistics and logs give you very interesting feedback. Now you can work on the potential issues getting the reports like this:
Our stress test results were quite thorough as we could see how the CPUs and memory were consumed by TM1 and Tomcat services depending on the load and what the critical number of users for our infrastructure was. Having all this statistic analysis, we managed to determine capacity needed and adjust the capacity of the infrastructure by increasing the number of CPUs. Additionally, we were able to
recommend and optimize virtual machine settings (i.e. static memory allocation instead of dynamic) and adjust TM1 parameters responsible for memory utilization.
That was the outcome the customer was expecting and we worked hard together to get it. There was one other pleasant outcome we did not anticipate: the Stress testing results made a strong impression on the customers which helped them sustain the confidence in TM1 applications.
TM1 stress testing is a sound practice benefiting the project and providing the real outcome for both developers and customers. Apache JMeter can be very useful in creating customized test plans for TM1 models and simulating different load types on TM1 applications. TM1 performance can be significantly improved and stability of the solution secured.
If you'd like more information on how you can improve TM1 performance using stress testing, please contact us on the details below:
|Australia||Singapore, Philippines, Thailand||United States|
|Cornerstone||PMsquare | A Cornerstone Group Company||PMsquare|
|Call +61 1300 840 048 or
email Piers Wilson
|Call +65 6635 1700 or
email Carsten Brandt
| Call +1 (630) 607 0570 or
email Chris Loechel