18.65M

Category:

programming

Performance testing

2.

2
Intro
I am Maksim
Petrov
QA automation engineer

3.

3
Agenda
⊗ Introduction to performance testing
⊗ Performance testing fallacies
⊗ Collected metrics
⊗ Performance test tools
⊗ Example of performance test cases
⊗ Best practices
⊗

6
What is performance testing?
⊗ Performance testing of defined as a type of non-functional
software testing to ensure software applications will
perform well under expected workload
⊗ The goal of performance testing isn’t to find bugs but to
eliminate performance bottlenecks
⊗ The focus of performance testing is checking a software
program’s
⊚ Speed
⊚ Scalability
⊚

7.

7
Why do performance testing?

8.

Benefits of performance
testing
⊗ Improve optimization and load capability
⊗ Identify discrepancies and resolve issues
⊗ Measure the accuracy, speed, and stability of the
software
⊗ Validate the fundamental features of the software
⊗ Performance Testing allows keeping your users
happy
⊗
8

9.

When should you consider
performance?
5
Forecasting
4
3
2
1
Deploymen
t phase
Planning
and design
phase
Production
phase
Developme
nt phase
9

10.

10
Types of performance testing

11.

11
Types of performance testing
⊗ Load testing
⊗ Stress testing
⊗ Stability testing
⊗ Spike testing
⊗ Volume testing
⊗ Scalability testing
⊗ Capacity testing
⊗ Recovery testing
⊗ Failover testing
⊗ Configuration
testing
⊗ Compare testing

12.

12
Types of performance testing
⊗ Load testing
⊗ Stress testing
⊗ Stability testing
⊗ Spike testing
⊗ Volume testing
⊗ Scalability testing
⊗ Capacity testing
⊗ Recovery testing
⊗ Failover testing
⊗ Configuration
testing
⊗ Compare testing

13.

Load
testing
(maximum
load)
load
over max
max
med
min
at rest
time
13

14.

14
Load
testing
load
over max
max
med
min
at rest
time

15.

15
Stability testing
load
over max
max
med
min
at rest
time

16.

Stress
testing
& recovery
testing
load
over max
max
med
min
at rest
time
16

17.

17
Configuration testing
load
over max
max
med
min
at rest
time

18.

Common performance
problems
⊗ Startup time
⊗ Poor response time
⊗ Poor scalability
⊗ Bottlenecking:
⊚
⊚
⊚
⊚
⊚
CPU utilization
Memory utilization
Network utilization
Operating system limitations
Disk usage
18

19.

19
Performance testing process
1
2
3
4
5
6
7

20.

20
Performance testing process
1
2
3
4
5
6
7

21.

21
Performance testing process
1
2
3
4
5
6
7

22.

22
Performance testing process
1
2
3
4
5
6
7

23.

23
Performance testing process
1
2
3
4
5
6
7

24.

24
Performance testing process
1
2
3
4
5
6
7

25.

25
Performance testing process
1
2
3
4
5
6
7

26.

26
Performance testing process
1
2
3
4
5
6
7

27.

2.
PERFORMANCE
TESTING
FALLACIES

28.

Performance testing fallacies

29.

29
Performance testing fallacies
⊗ Performance testing is the last step in
development
⊗ More hardware can fix performance issues
⊗ The testing environment is close enough
⊗ What works now, works across the board
⊗ One performance testing scenario is enough

30.

30
Performance testing fallacies
⊗ Testing each part equals testing the whole
system
⊗ What works for them, works for us
⊗ Software developers are too experienced to
need performance testing
⊗ A full load test tells everything
⊗ Test scripts are actual users

31.

3.
COLLECTED
METRICS

32.

Performance testing metrics:
parameters monitored
⊗
⊗
⊗
⊗
⊗
⊗
⊗
⊗
⊗
⊗
⊗
Processor Usage
Memory use
Disk time
Bandwidth
Private bytes
Committed memory
Memory pages/second
Page faults/second
CPU interrupts per second
Disk queue length
Network output queue
length
⊗ Network bytes total per
second
⊗ Response time
⊗ Throughput
⊗ Amount of connection
pooling
⊗ Maximum active sessions
⊗ Hit ratios
⊗ Hits per second
⊗ Rollback segment
⊗ Database locks
⊗ Top waits
⊗ Thread counts
⊗ Garbage collection
32

33.

4.
PEFRORMANCE
TEST TOOLS

34.

34
Performance test tools

35.

5.
EXAMPLE
PERFORMANCE
TEST CASES

36.

36
Example

37.

6.
BEST PRACTICES

38.

38
Best practices
⊗ Test as early as possible in development
⊗ Follow a devops approach
⊗ Consider users, not just servers
⊗ Understand performance test definitions
⊗ Build a complete performance model
⊗ Define baselines for important system
functions
⊗ Consistently report and analyze the results