Software Testing and Maintenance

Some Concepts

•     Verification

•     Validation

•     Testing

•     Debugging

•     Errors

•     Faults

•     Failures

Verification and Validation

•     Verification

– Are we building the product right?

– Does the system conform to its specification?

•     Validation

– Are we building the right product?

– Does the system meet the customer’s expectation?

Objectives of Software Testing

•     Prevention of bugs

•     Discovery of bugs

– Defect Testing

•     Purpose of testing is to make quality visible

•     Overall process

– Software Quality Assurance

– I.e. the overall process of defining how software quality can be achieved and how the development organization knows that the software has the required level of quality

Defect Testing

•       The goal of defect testing is to discover defects in the program

•       A successful defect test is a test which causes a program to behave in an anomalous way

•       Testing shows the presence of defects

– Not the absence

Debugging Process

•      Locate Fault

•      Understand Error/Fault

•      Design Fault Repair

•      Understand Consequences of Fault Repair

•      Repair Fault

•      Retest Program

Testing vs. Debugging

•      Testing is the process of establishing the existence of defects.

•      Debugging is the process of locating and correcting the defects.

Faults, Failures, and Errors

•     Human error...

– E.g. Design mistake, programming mistake, misunderstood requirement

•     ... may lead to a fault in the program...

– E.g. a design that does not match the requirement, incorrect source code, incorrect documentation

•     ... that may cause a failure when executing.

– A deviation from the required behaviour

       Defined in: IEEE Standard 729










Automated Testing

•         Mostly on unit test level

•         Used for

–      test of parameter range

–      regression testing

•         Example of tools:

–      Test manager

–      Test data generator

–      File comparator

–      Dynamic analyser

–      Simulator


Test Specification

•         Collection of test cases

•         May contain the following information

–      Overall goal of tests

–      Overall test guidelines

–      List of requirements that are being tested

–      Description of test cases

Test Case

•         Purpose

–      Test Component or Functionality

•         May contain the following information

–      Description of the input data

–      Description of the expected behaviour

–      Description of the expected outcome

–      Test guidelines

Test Case – Example 1

Name: Test.Req.Func.Login

Requirement(s) Tested: Req.Func.Login

Precedent test cases: None

Input (Test Sequence): Fill in a valid username in the field username, and a valid password in the field password

Expected behaviour from the system: Logs in the user

Expected outcome: The user is logged in to the system and a welcome window is shown

Test Case – Example 2

Name: Test.Req.Func.Logout

Requirement(s) Tested: Req.Func.Logout

Precedent test cases: Test.req.func.login

Input (test sequence): Click on the button logout in the window

Expected behaviour from the system: logs out the user from the system

Expected outcome: A window showing a text with the message “you are now logged out” is shown, and the user can no longer use the functions that require the user to be logged in

Trouble report

•         May contain the following

–      Location: Where did the problem occur?

–      Timing: When did it occur?

–      Symptom: What was observed?

–      Cause: What caused the problem?


Test Report

•         Summary of your testing activities

•         What tests have been conducted?

•         How many tests have been conducted?

•         How many tests have failed?

•         etc.

Software Changes

•         80-90% of a product’s lifecycle

•         Once a system is put to use,

–      new requirements emerge

–      existing requirements change, as the business changes

–      part of the system may have to be modified to

»      correct faults that causes failures in operation

»      improve its performance or other non-functional characteristics

Strategies for Software Change

•         Software Maintenance

–      Changes to the software are made as requirements change, but the fundamental structure remains the same

•         Architecture Transformation

–      Significant changes are made to the architecture of the system

•         Software re-engineering

–      No new functionality is added to the system. Instead, it is made easier to understand and change. Usually, no major architectural changes are made

•         Combination of the above


Maintenance is more Expensive than Development

•         Team stability

–      Lack of common background with the system

•         Contractual responsibility

–      Maintenance contracts are written separately

–      Maintenance contracts may be given to other organisation

–       = No incentive to write maintainable code from the start

•         Staff skills

–      Maintenance often seen as boring

–       = Junior programmers are assigned to maintenance

•         Program age and structure

–      System degrades, becomes brittle.

–       = Harder to understand how to make a change


Change Implementation

•         Ideally, the change implementation stage of the maintenance process should modify

–      system specification

–      design

–      implementation

–      ... to reflect the changes to the system

•         New requirements that reflect the system change are proposed, analysed, and validated

•         System components are redesigned and implemented, and the system is re-tested

•         ... Then, there is the problem with urgent changes...