Week 08 _09Requirements Validation

1. REQUIREMENTS VALIDATION

DR. CHANDRA REKA

2. OUTLINE

INTRODUCTION V&V
V&V REQUIREMENTS
REQUIREMENT VALIDATION TECHNIQUES
OTHERS

3. Learning Outcomes

At the end of this lecture, you
should be able to:
◦ Describe the purpose of
Requirements V&V
◦ Elaborate the process
involved in Requirements V&V
◦ Discuss the various techniques
for performing Requirements
V&V

4. Aim of Validating Requirements

The purpose of Requirements Validation is defined as follows:
◦ Ensure that all requirements support the delivery of value to
the business, fulfill its goals and objectives, and meet a
stakeholder needs

5. What is V&V ?

What is V&V ?
Verification:
◦ the process to discover ambiguous requirements
◦ Question to ask: Did we build the product right
(clearly?)
Validation:
◦ the process to discover unneeded requirements.
◦ Question to ask: Does it satisfies customer needs?)
“As a {user}, I want do {do
something} so that {perceived
benefit}.
https://www.bridging-the-gap.com/validate-requirements-babok-6-6/
Values!

6.

7.

• Specific objective:
identify and resolve software problems and high-risk
issues early in the software life cycle
the discovery of defects in a system
the assessment of whether or not the system is usable in
an operational situation
SWE307
7

8. Requirements Validation and Verification

At every stage during the (requirements)
process
Elicitation
Checking back with the elicitation sources
Analysis
Checking that the domain description and requirements are
correct
Specification
Checking that the defined system requirement will meet the
user requirements under the assumptions of the
domain/environment
Checking conformity to well-formed rules, standards…
SWE307
8

9. Requirements Validation and Verification

When do we perform V&V?
Michael et al (2011)
SWE307
9

10. Requirements V&V Process

REQUIREMENTS V&V
PROCESS

11. Requirements V&V Process

Requirements V&V Process
Specifications based on natural language can be ambiguous.
Example:
Requirements for a project management system – the system shall
generate a project status report once every month.
Is it ok to generate one report in the last week of May and then in the 1st week of
June?
What will happen to a project of 15 days? Does the system generate a report?
Analyst must validate understanding of requirements with customer.
Analyst must also verify which date is suitable to the business operation?
SWE307
11

12. Validation and Verification Criteria

i.
Completeness
Are there any requirements conflicts?
all parts are present(current), each part is fully developed
no TBDs (to be determined)
• "The system shall handle a peak load of (TBD) transactions per
second."
no nonexistent references
• "A record of all transactions is retained in the Transaction File,"
which is undefined.
no missing specification items
• No verification provisions
• No interface specifications
no missing functions
• No backup functions
• No recovery functions
no missing products
• Test tools
• Output postprocessors
SWE307
12

13. Validation and Verification Criteria

ii. Consistency
Are there any requirements conflicts?
provisions do no conflict with each other or with
governing specifications and objectives
internal consistency
• "Master real-time control interrupts shall have top priority at all times."
• "A critical-level interrupt from the security subsystem shall take precedence
over all other processes and interrupts."
external consistency
• "Spec: All safety parameters are provided by the pre-processor system, as
follows: . . . "
• "Preprocessor system spec: The preprocessor initializes all safety parameters
except for real-time control safety parameters, which are self-initialized. . . “
traceability
• misinterpretations, such as assuming that "on-line" storage implies a
requirement for random access storage
SWE307
13

14. Validation and Verification Criteria

iii. Feasibility
life-cycle benefits of the system exceed its life-cycle
costs
verifying system satisfies functional and
performance requirements
validating that systems will be maintainable,
reliable and human-engineered
identifying and resolving any high-risk issues
human engineering
• Will the specified system provide a satisfactory way for
users to perform their operational functions?
• Will the system satisfy human needs at various levels?
• Will the system help people fulfil their human potential?
SWE307
14

15. Validation and Verification Criteria

iii. Feasibility (further explained)
resource engineering
• Can a system be developed that satisfies the
specified requirements (at an acceptable cost in
resources)?
• Will the specified system cost effectively
accommodate expected growth in operational
requirements over its life-cycle?
program engineering
• Will it be cost-effective to maintain?
• Will it be cost-effective from a portability standpoint?
• Will it have sufficient accuracy, reliability, and
availability to cost-effectively satisfy operational
needs over its life cycle?
SWE307
15

16. Validation and Verification Criteria

iii. Feasibility (further explained)
risk
• technical risk – achievable levels of overhead in a
multiprocessor operating system; achievable levels of
computer security protection
• cost schedule risk - availability and reliability of the
underlying virtual machine (hardware, operating
system, database management system, compiler)
upon which the specified software will be built
• environmental risk - expected volume and quality of
input data; availability and performance of
interfacing systems
• interaction effects – second-order effects caused by
introduction of new system
SWE307
16

17. Validation and Verification Criteria

iv. Testability
specifications must be specific, unambiguous, and
quantitative
eg. of specifications that are not testable
Not testable
The software shall provide
accuracy
sufficient
to
support effective flight
control.
Testable
SWE307
The
software
shall
compute aircraft position
within
the
following
accuracies:
±50 feet in the horizontal;
±20 feet in the vertical.
The
software
shall
The
software
shall provide a 99.9999
provide
realtime percent assurance of
response
to
sales information privacy (or
activity queries.
"reliability,'
The system shall respond ''availability,''
or
to
"human safety," when
Type A queries in .2 these
terms
are
seconds;
undefined)
Type B queries in < 10
seconds;
Type C queries in 42
minutes;
Not testable
17

18. Requirements Validation and Verification

Goals,
• Requirements Validation
• Check right product is being built
• Ensure software being developed (or
modified)
will satisfy stakeholders
• Check the software requirements
specification
against stakeholders goals and
requirements
• Requirements error costs are high
which makes validation
very important
SWE307
• Fixing a requirements error after
delivery may cost up
to 100 times the cost of fixing an
implementation error
requirements
Validation
Requirements
Specification
Verification
Design
Software
system
18

19. Example: RIGHT application

20. Requirements Validation

“…confirmation,
through the provision
of objective evidence,
that the requirements
for a specific intended
use or application
have been fulfilled.”
ISO/IEC/IEEE 29148:2018(E)(2018)
SWE307
Questions to ask:
Does our
problem
statement
accurately
capture the real
problem?
Did we account
for the needs of
all the
stakeholders?
20

21. Requirements Validation

◦ Outcome:
Discovering the ambiguous
requirements
◦ unclear requirements
◦ conflicting requirements
◦ errors in requirements elicitation and analysis
◦ lack of conformance to quality standard
SWE307
21

22. Requirements Validation Checking

• Validity check
o functions proposed should align
with system needs
• Consistency check
o no conflicting requirements
• Completeness check
o include all requirements and
constraints
• Realism check
o requirements can actually be
implemented
• Verifiability check
o should be able to write test cases
for requirements
SWE307
22

23. Requirements Validation Techniques

REQUIREMENTS
VALIDATION
TECHNIQUES

24. Requirements Validation

◦ Requirements validation is the process of
checking that requirements define the
system that the customer really wants.
◦ Requirements validation is critically
important because errors in a requirements
document can lead to extensive rework
costs

25. Requirement Validation Process

26. Requirements Validation Technique

• Prototyping
• Requirement Testing
• Test Case Generation
• Automated consistency analysis
• Requirements review
• Inspection
SWE307
26

27. A number of requirements validation techniques

1. Requirements reviews
The requirements are analyzed systematically by a team of reviewers who check for
errors and inconsistencies.
2. Prototyping
This involves developing an executable model of a system and using this with end-users
and customers to see if it meets their needs and expectations. Stakeholders experiment
with the system and feed back requirements changes to the development team.
3. Test-case generation
Requirements should be testable.

28. A number of requirements validation techniques

4. Automated Consistency Analysis
called consistency checking, for automatic detection of errors, such as type errors,
nondeterminism, missing cases, and circular definitions, in requirements specifications.
The technique is designed to analyze requirements specifications expressed in the SCR
(Software Cost Reduction) tabular notation.
5. Inspection
Software inspection is a control technique for ensuring that the documentation
produced during a given phase remains consistent with the documentation of the
previous phases and respects preestablished rules and standards.

29. Requirements Validation Technique

• Prototyping
• useful in validating the SRS
• helps requirements engineer to understand
customer requirements
• serves as a media for communication
• prototype built and demonstrated to end users
for feedback
• 2 types:
• throwaway prototype – identify requirements that are
not properly understood and resolve inconsistencies
• evolutionary prototype – built from initial requirements
and gradually refined
SWE307
29

30. Throwaway Prototype

• Throwaway prototypes are
developed from the initial
requirements but they are not used
for the final product
• Throwaway prototype has a short
project timeline and easier and
faster to develop the interface.
• Throwaway prototypes actually do
nothing, it’s just presentation only
for a limited purpose.
https://medium.com/@pavithrajayasinghe9529/throwaway-prototyping-vsevolutionary-prototyping-8302be3baf33

31. Evolutionary Prototype

• Evolutionary Prototyping is considered
to be the most fundamental form of
prototyping and this prototyping type is
also known as breadboard
prototyping.
• The main concept of this prototyping
type is to build a robust prototype and
constantly improve it.
https://medium.com/@pavithrajayasinghe9529/throwaway-prototyping-vsevolutionary-prototyping-8302be3baf33

32. Requirements Validation Technique

• Requirements testing
• useful in validating the SRS
(Software Requirements
Specification)
• test cases defined for each
requirement
o difficulty in deriving test cases
may indicate incomplete or
ambiguous requirements
• may be used for final test of
system
• allows testing department to be
involved early in project
• testers should be experienced
• test-case driven (TCD)
inspections
SWE307
Raja (2009)
32

33. Requirements Validation Technique

• Automated consistency analysis
o CASE tool used to check for inconsistency
o automatic detection of error
o requires requirements to be expressed in a
formal or structured manner
SWE307
33

34. Requirements Validation Technique

• Inspection
• introduced by Fagan in 1976
• effective technique for detecting errors (50 –
90% errors may be detected)
• a process management tool (always formal)
used to improve quality of the development
process
• detects incomplete or ambiguous
requirements in SRS
• collect defect data to analyze the quality of
the process
SWE307
34

35. Requirements Validation Technique

• Requirements review
• regular reviews should be held while the
requirements definition is being formulated.
• both client and contractor staff should be
involved in reviews
o consists of software engineers, users, and other
stakeholders
• reviews may be formal (with completed
documents) or informal
• good communications between developers,
customers and users can resolve problems at an
early stage.
• review team performs systematic analysis of the
requirements
SWE307
35

36. Requirements Validation Technique

• Requirements review (contd.)
• specification is examined
• problems such as consistency, omissions,
and errors are detected and corrected
• checks requirements conforms to standards
• review meeting is held
SWE307
36

37. Requirements Validation Technique

• Requirements review (contd.)
• checklist produced (for eg.):
• Verifiability
• Is the requirement realistically testable?
• Comprehensibility
• Is the requirement properly understood?
• Traceability
• Is the origin of the requirement clearly
stated?
• Adaptability
• Can the requirement be changed
without a large impact on other
requirements?
SWE307
37

38. More Requirements Validation Technique

• Brainstorming
• validate requirements specification
document
• Storyboarding
• requirements are printed on paper and are
pinned in a logical sequence on a wall or a
board
SWE307
38

39. Brainstorming

◦ Brainstorming is used in requirement gathering to get as many ideas as possible from
group of people. Generally us
◦ It can be used to help elicit new ideas and features for the application
◦ define what project or problem to work on,
◦ to diagnose problems , Identify possible solutions and resistance to proposed solutions
◦ to identify possible solutions to problems, and clarify details of opportunities.
◦ Idea generation
◦ The primary goal during idea generation is to set down as many ideas as possible, focusing on
breadth of ideas, not necessarily depth.
◦ Idea reduction
◦ The primary goal during idea reduction is to analyze all the ideas generated. Idea reduction
includes pruning, organizing, ranking, grouping, refining, and so on.

40. Storyboarding

◦ Storyboarding was one of the main
techniques used during the process of
requirements elicitation. It allowed great
visual content.
◦ Recent studies show that the combination
of stories and sketches is helpful to reason
about a future application, errors and
contextual information.
◦ Widely adapted in user-centred software
engineering

41. More Techniques

SWE307
Maalem and Zarour (2016)
41

42. CONCLUSION

◦ V & V ARE VERY CRUCIAL
◦ Errors in requirements are pervasive, dangerous, and costly [Faulk 1995]
◦ Old story: https://www.tmctechnologies.com/do-you-vv-why-softwareverification-and-validation-is-so-important/
◦ A 1992 study found that the major source of safety-related software errors in
NASA’s Voyager and Galileo spacecraft were errors in functional and interface
requirements [Lutz 1993].

43. EXERCISE

Compare validation and verification. Tabulate the comparison

44. Reference Materials

◦ Boehm, B. (1984). Verifying and Validating Software Requirements and
Design Specifications. IEEE Software. 1(1).
◦ Michael, J. B., Drusinsky, D., Otani, T. W., Shing, M-T. (2011). Verification and
Validation for Trustworthy Software System. IEEE Software.
◦ Raja, U. A. (2009). Empirical Studies of Requirements Validation Techniques. Paper
presented at the 2009 2nd International Conference on Computer, Control and
Communication.
◦ Sommerville, I. (2015). Software Engineering (10th Edition). Pearson Publishing.
0133943038.
◦ ISO/IEC/IEEE. (2018). Systems and software engineering — Life cycle processes —
Requirements engineering. ISO/IEC/IEEE 29148:2018(E).
◦ Wallace, D. R., Fujii, R. U. (1989). Software Verification and Validation: An
Overview. IEEE Software.
◦ Heitmeyer et.al (1996). Automated Consistency Checking of Requirements
Specifications. ACM Transactions on Software Engineering and Methodology, Vol.
5, No. 3, July 1996, Pages 231–261.
◦ https://www.tmctechnologies.com/do-you-vv-why-software-verification-and-validation-is-soimportant/
SWE307
https://medium.com/@pavithrajayasinghe9529/throwaway-prototyping-vsevolutionary-prototyping-8302be3baf33
44