JMeter - Load Development LifeCycle

GUI Console

The GUI console is for:

• script development and debugging only.
• for small loads (<50 simultaneous users)

Avoid XML processing

XML processing is CPU intensive because it need to build a DOM and will rapidly consume all the CPU cycles. As a general rule, the performance of XML centric applications will perform 4-10 slower than applications using binary protocols.

Avoid:

• XPath Extractor
• XML Schema Assertion
• XML Assertion
• XPath Assertion

Prefer:

• Regular Expression extractor
• Response Assertion or Size assertion
• Response Assertion or Size assertion

(Non-Gui|Command Line|Batch) Mode

Start the load in Non-Gui Mode

Different data and log files for each test run, for example:

jmeter -n -t testplan.jmx -l testplan_01.jtl -j testplan_01.log
jmeter -n -t testplan.jmx -l testplan_02.jtl -j testplan_02.log

where:

• -n run JMeter in nongui mode
• t is the the test plan
• l is the the data. It create a top-level listener for the test run. This is in addition to any Listeners defined in the test plan.
• j define the log file. Versions of JMeter after 2.3.1 support variables in the log file name. If the filename contains paired single-quotes, then the name is processed as a SimpleDateFormat format applied to the current date, for example:

log_file='jmeter_'yyyyMMddHHmmss'.tmp' This can be used to generate a unique name for each test run.

JMeter Java options Configuration

Configure the JMeter Java options to meet your requirements.

• Set -Xmx option value (default 512Mo)
• Change MaxNewSize option in jmeter file to respect the original ratio between MaxNewSize and -Xmx.
• Tune GC options (if you master this domain).

Performance Metrics

• A single JMeter client running on a 2-3Ghz CPU (recent cpu) can handle 300-600 threads depending on the type of test.
• Maximum hits per second: Assuming you use strong resources for your test, keep the maximum hits per second lower than 1000 per second.
• Bandwidth: Keep the amount of bandwidth lower than 300MBs.

(Remove|disable) all listeners

Remove all listeners (such as “View Results Tree” or any of the graphical reporting listeners) Listeners receive Sample Results and do some processing with it, this takes resources. They can cause the console to freeze without providing any prior alert.

The following Listeners no longer need to keep copies of every single sample. Instead, samples with the same elapsed time are aggregated. Less memory is now needed:

• Aggregate Report
• Aggregate Graph
• Distribution Graph

The best listeners to use for a long-term, high-load test are:

• Aggregate Listener,
• Graph Listener,
• and Spline Listener.

Generate Reports after the run

Always generate Graphs and Reports AFTER the Load Test.

Set CSV as output

To minimise the amount of memory needed, use the Simple Data Writer, and use the CSV format.

Set CSV as output for SaveService because:

• XML is verbose, it takes resources to be written (CPU and memory)
• for analysis, CSV is great
• There are many unnecessary result data.

In the user.properties file, add:

jmeter.save.saveservice.output_format=csv
jmeter.save.saveservice.data_type=false
jmeter.save.saveservice.label=true
jmeter.save.saveservice.response_code=true
jmeter.save.saveservice.response_data.on_error=false
jmeter.save.saveservice.response_message=false
jmeter.save.saveservice.successful=true
jmeter.save.saveservice.thread_name=true
jmeter.save.saveservice.time=true
jmeter.save.saveservice.subresults=false
jmeter.save.saveservice.assertions=false
jmeter.save.saveservice.latency=true
jmeter.save.saveservice.bytes=true
jmeter.save.saveservice.hostname=true
jmeter.save.saveservice.thread_counts=true
jmeter.save.saveservice.sample_count=true
jmeter.save.saveservice.assertion_results_failure_message=false
jmeter.save.saveservice.timestamp_format=HH:mm:ss
jmeter.save.saveservice.default_delimiter=;
jmeter.save.saveservice.print_field_names=true

Distributed (Remote) Testing

If you reach the limits of one machine, you can switch to distributed or remote testing. JMeter defaults are not fine for efficient remote testing, so in user.properties, add:

mode=StrippedBatch

to:

• remove some data from the Sample Results as the response body
• send Sample Results as Batches and not for every sample reducing CPU, IO and network roundtrips