Showing posts with label Performance Testing. Show all posts
Showing posts with label Performance Testing. Show all posts

Create a load test plan with JMeter

Related articles:

Become a technical tester while learning JMeter
Performance testing checklist

New posts: 

Selenium Basics



JMeter concepts used in this example:

- controllers: if, simple, foreach

- regular expression extractors

- user defined variables

- dynamic http requests

- response assertions

- duration assertions

- executing javascript functions

- bean shell assertions

- reading and writing user variables in a bean shell script

- writing to the jmeter log

- using listeners (view results tree)



Load Test Plan - requirements



Main URL: http://jmeter.apache.org/

step 1. Open the http://jmeter.apache.org/changes.html

step 2. Do a few validations

         a. check that version number = Version 2.10
        b. check that page load time < 1000ms
 
step 3. At the bottom of the changes.html page, there are many bug fixes links

step 4. Get the ids of all bug fixes

step 5. Go through all bug fix ids and repeat the following actions:

         a. Print the bug order number and bug id in the JMETER log
 
b. If the bug id < 58000
 
b1. Open the bug page

b2. Run a few assertions on the bug page

b2.1 Check that ASF Bugzilla exists in the page

b2.2 Check that the bug id exists in the page




Load Test Plan set up



1. Create a Thread Group with the following settings:


action to be taken after a sampler error = continue
number of threads (users) = 10
ramp-up period = 1
loop count = 1

2. Create a Simple Controller inside of the thread group; name it "Practice Tests"

3. Create a Simple Controller inside of Practice Tests; name it "Open Bug pages based on the results of a regular expression"

4. In the last Simple Controller, create the following:

- User Defined Variables config element

* it will include the variables used in the requests and assertions


- Only Once Controller

* it will be used for opening the changes.html page only once
* name it "Open The Changes page"



- ForEach controller

* it will be used for opening the bug id pages
* name it "Open Bug Id pages"

5. Add a View Results Tree listener at the same level with the "Practice Tests" Controller; this will be used for getting results from the load test



Click the image for the original photo




Create the Tess Plan details

1. in the User Defined Variables section, add the following variables:

BugIds

used for getting all bug fix id values from the changes.html page
the variable will store a list of bug id strings
default value = 0

ResultValue

used for getting the individual bug ids from the BugIds variable
default value = 0

NumberOfBugs

used for getting the bug count in the changes.html page
default value = 0



Click the image for the original photo



2. in the "Open The Changes page" controller, create a http request with the following details:


*** this matches requirement 1
Name = Changes Page
Server Name or IP = jmeter.apache.org
Path = /changes.html


3. Add 2 assertions to the "Changes Page" http request


*** this matches requirement 2


a. response assertion

Name = check version number
Response Field To Test = text response
Patterns to test = Version 2.10



Click the image for the original photo



b. duration assertion

Name = check page load time
Duration to assert = 1000 ms


Click the image for the original photo



4. Add a Regular Expression extractor post-processor to the "Changes Page" http request


*** this matches requirement 4
Name = GetBugsIds

Response Field To Check = body
Reference Name = BugIds (defined in User Defined Variables)
Regular Expression = (id=([0-9]{5}))+ 
Template = $1$
Match = -1 (important if using with foreach controller)
Default Value = nothing

This is how the regular expression will work:

- it will find all id = xxxxx values in the page and add them to the BugIds variable
- the BugIds variable will be a collection of bug ids



Click the image for the original photo




5. Add the following to the "Open Bug Id pages" ForEach controller:

*** this matches requirement 5

- Controller Settings:

input variable prefix = BugIds (defined in User Defined Variables and instanced by the GetBugsIds regular exp extractor)
output variable name = ResultValue (defined in User Defined Variables)

This is how the controller works: 
   for each iteration, it will assign one bugid value from BugIds to the ResultValue variable.


Click the image for the original photo




- Create a bean shell sampler for displaying the bug count and id in the jmeter log:

*** this matches requirement 5a

Name = Print Bug Number and ID (inline)
String myNumberOfBugs = vars.get("NumberOfBugs");
Double myBugNumber = 0; 
myBugNumber = Double.parseDouble(myNumberOfBugs); 
myBugNumber = myBugNumber + 1;
vars.put("NumberOfBugs", myBugNumber.toString()); 
String myResultValue = vars.get("ResultValue"); 
String myBugId = "";
myBugId = myResultValue.substring(3,8);
log.info("bug number: " + myBugNumber.toString() + " - bug id = " + myBugId);

Click the image for the original photo



- Create an If Controller so only specific bug id pages are opened:

*** this matches requirement 5b


Name = Open Bug Details page if bug id < 58000
Condition = ${__javaScript('${ResultValue}'.substring(3\,8))} < 58000
  This condition evaluates if the bug id (xxxxx) is smaller than 5800;
  If the condition is true, then the next http request is executed;
  If the condition is false, then the next http request is not executed;


Click the image for the original photo
    

- In the "Open Bug Details page if bug id < 58000" If Controller, create a http request:

                                                                                                       *** this matches requirement 5b1


Name = Bug Details Page
Server name or IP = issues.apache.org/bugzilla
Path = /show_bug.cgi?${ResultValue}


Click the image for the original photo

 

- In the "Bug Details Page" http request, add a bean shell assertion:

                                                                                                       *** this matches requirement 5b2


Name = assertions
String myResultValue = vars.get("ResultValue"); 
String myBugId = "";
myBugId = myResultValue.substring(3,8); 
String body= new String(ResponseData); 
if(!body.contains("ASF Bugzilla")) 
log.warn("- could not find ASF Bugzilla in the page");
else
log.info("- found ASF Bugzilla in the page");
if(!body.contains(myBugId) ) 
log.warn("- could not find the bug id in the page");
  else    
log.info("- found the bug id in the page");


Click the image for the original photo



Run the load test

Performance Testing checklist

I read a few days ago the Performance Testing Checklist from the ministryoftesting.com site and remembered about an old intention of creating a checklist with elements of a load test.

My last performance testing project was probably 2 years ago so the following info comes from memory.

I only worked on load and performance testing for web apps and web services so far.

The checklist applies to both with minimal changes.

If you prefer a mindmap to a long checklist, this is the link to it.

ANALYSIS


  • What will be measured?
    • application (web pages) response time
    • throughput
    • server resource utilization
  • Pass/Fail criteria
    • PASS: system performance during performance test <= baseline system performance
    • FAIL
      • system crashes during the load test
      • system becomes un-responsive
      • system performance during load test > baseline system performance
  • Benchmark the system's performance under highest production load
    • web pages' response times --> from Google Analytics
    • server resource utilization --> from the Server Monitoring tools
    • highest production application load
      • busiest times of the app --> from Google Analytics
      • highest number of concurrent users --> from the web server logs

  • Define user scenarios

    • get the most used pages from Google Analytics
    • create a few user scenarios using the most used pages
    • identify parameters of each page
    • create test data for the parameters of each page

  • Issues to be monitored
    • application becomes not responsive
    • application generates lots of errors under load
    • the servers resources are heavily used
    • application's response time is very high
  • Performance testing environment isolation from production
    • confirm that none of the components of the performance testing environment are used in production



PREPARATION


  • Prepare test data
    • Extract data from the production environment
    • Modify it for the performance test
    • Generate enough test data so it will not be re-used fast
  • Performance test environment
    • Application Environment
      • Application Servers
        • Web Servers
        • Database Servers  
        • Media Servers
        • Search Servers
      • Networking devices
        • Load Balancer
        • Firewall
    • Performance Testing Environment
      • Performance Testing Servers
        • Management Server
        • Agent Servers
  • Application Server Monitoring
    • Web Servers - Performance Counters
      • CPU
      • Memory
      • Disk
      • Network
      • Operating System
      • Web Server
      • Application (.NET)
    • Database Servers - Performance Counters
      • CPU
      • Memory
      • Disk
      • Network
      • Operating System
      • DB counters
  • Application stability before the performance testing
    • Run complete functional testing of the app
    • Confirm that no critical issues are present
  • Create Scripts for the performance testing user scenarios
    • Wait time between scripts steps
      • random times
      • script recording times
    • Create transactions for script steps
    • Create parameters in the scripts and connect them to the test data
    • Synchronize users in the script
  • Remove any 3rd party dependencies' impact on the application
    • 3rd party dependencies
    • banners
    • partner sites
    • google analytics
    • point all the 3rd party domains to 127.0.0.1
  • Identify browsers and bandwidth types to be used for scripts
  • Set up the model of loading users (user ramping)


EXECUTION


  • Start monitoring of the application environment
  • Start the performance test
    • Let the performance test go for a long time
    • Stop the performance test
  • Collect results
    • Performance testing tool reports
    • Server performance counters
    • Web logs
  • Analyze the results by comparing them with the baseline info
  • Report issues
  • Fix issues
    • Make changes in the application environment
    • Fix code bugs
    • Fine tune the performance testing scripts
  • Repeat the performance test






Become a technical tester while learning JMETER

I watched yesterday a few videos about JMETER from this site: http://blazemeter.com/blog/jmeter-tutorial-video-series.

I heard about JMETER before as the free web site load testing tool but never used it.

My experience includes working with Load Runner for a couple of years for performance and load testing so I thought that there is not much to learn from a free tool.

After watching the videos, I am still convinced that the difference between the paid tool (Load Runner) and the free one (JMETER) is big since the sophistication of Load Runner is high.

But, JMETER can work pretty well in many cases too.

I am digressing from what I had in mind .....

After watching the videos, I realized that learning to use JMETER can have a very nice side effect for a tester.

There are many things that have to be mastered before creating and running a successful load test:

- regular expressions: important for finding values in the web page content

- web proxy: a web proxy is being used by JMETER to record a web script

- creating scripts: after the web proxy generates the script, the script needs to be modified so that the script parameters can be "parameterized" (not great English, I know; what i mean is that the script should be able to use multiple values for the same parameter; imagine that a web page has a category parameter; the script should be able to use sports, business, home and decor as possible values for the category parameter); the "parameterization" is done usually through external data files

- find, save and re-use browser session variables: the session variable is generated usually as soon as the user opens the site and all other site pages need it; this uses regular expressions most of the time

- assertions: assertions will verify that the page returned for a web request is actually the correct page; this is done by checking for information in the web page content

- set up load test configuration info: to do this, a basic understanding of HTTP requests, HTTP headers, cookies, cache is important

- listeners: the listeners provide the results for each web script after the script is executed

- results reports: these reports include information like throughput, samples, min, max, average, etc

- server concepts: to run a load test, multiple computers will be needed: some of them will just send the requests to the target system (slaves), then you have the master server which controls the load test and the target system which hosts the application under test

So far, I just summarized some concepts that can be easily found in the JMETER user guide.

Learning JMETER will not be easy or quick especially for a tester that does not have a technical background and has no previous exposure to load tests and load testing tools.

What is the second benefit, however, of learning JMETER?

Learning about all key concepts for JMETER will improve the technical knowledge of the tester a great deal.

Technical knowledge is very handy when deciding to start on test automation or even on security testing.

It also allows the technical tester to do much more than just black-box functional testing.






Chat with a tester on load testing basics


Related articles

Become a technical tester while learning JMETER
Performance Testing Checklist



me

hi utester
how are you?
Are you online?

utester:

Hi Alex,
I'm well. And you?

me:
i am ok
do you have time for a chat?

utester:

Yes

me:

ok
so what exactly are you interested in?
i need to say from the beginning that i did not participate in load testing projects at uTest
but in my mind, they are not exactly load testing
load testing involves using a tool like Load Runner

unless i am mistaken, load testing for uTest means that lots of users use the app in the same time
outside of utest, load testing means something else

utester:

What expected in load testing?

me

in general, lets say that load testing is needed for a site

there are different types of tests that can be done
1. a test that shows you how many concurrent users that site can have before it goes down
2. a test that shows you when the site performance degrades, meaning what is the number of users when the performance of the site goes above 5 seconds, lets say

a load test has multiple components:
- the first one is to know what you want to do
- do you want to simulate users with multiple browsers or not?
- do you want to simulate different internet connection bandwidths?
- how should the users be loaded? should you start with 50 concurrent users from the beginning or start with 5 and add 5 more every few minutes?
- what are the most common scenarios that your users are using on the site?

are you following so far?

utester:

Yes. I see
We don't need report any bugs to customer in load testing, right?

me:

there are no bugs.
there are problems instead like for example:
- when you have 50 users, the time for opening a page is 20 seconds

or 

-when you have 75 concurrent users, the server reboots

load testing is a multi-discipline activity as you have to do multiple things as follows:

- record scripts that will simulate the most important user scenarios for the site; this involves some coding

- prepare data for the scripts parameters

- monitor servers performance counters

- finish your functional testing before the load test as the site must be stable

- run the load test and gather data from the server monitoring

- analyze the load test results and make improvements either on the server side, application side or scripts

- repeat everything from beginning

utester:

can we do all above tasks when using Load Runner tools?

me:

yes

load runner is the tool.

but the server monitoring is done on the server level.

unfortunately, load runner is extremely expensive so only big companies can afford it.

a free tool for load testing web site is http://jmeter.apache.org/.

but it is probably rather limited.

it could be a starting point though.

utester:

Thanks. Is this support web test only?

me

yes, i think so

utester

I checked the battery of some android devices in last month. I run more applications in background, and wait until battery is zero.

Is it also load test?

me

no

a load test assumes that you have multiple concurrent users on the app

that is a battery test :)

utester

Yes. I see. thanks for clarifying.

me

you are welcome

a load test for mobile apps is quite difficult

and it could only test the backend of the app

since everybody is using the mobile app on his own device

if the mobile app is connected to a backend system, having 100 users using the mobile app in the same time might put some load on the backend system

utester

Sure. I also tested using Android Simulators (from SDK). But have not tried connected with 100 users

me

because it is not possible, each user uses 1 device.

was this chat useful for you?

utester

Yes. This is very helpful.

I think I need to practise with JMeter first.

I don't know which site I should try to practice?

me

try http://demo.testfire.net/default.aspx or create a test blog site and practice your load tests on it with a low number of users.

use a low number of users, 2 or 3

2 or 3 users should not be a big deal for a sample load test

utester

ok. thank you so much

me

utester, are you ok with me sharing this chat with the uTest forum after removing your name from it?

there may be other testers with similar questions

utester

OK. No problem

me

great

utester

I'm very happy when sharing with other testers :)

me

if you want to discuss about other topics in the future, let me know :)

utester

Yes Many thank you

me

you are welcome