- Establishing Confidence that a program does what it is supposed to do.
- The process of executing a program or system with the intent of finding errors.
- Detecting specification errors and deviations from the specifications.
- Any activity aimed at evaluating an attribute or capability of a program or system.
- The measurement of software quality.
- Verifying that a system satisfies its specified requirements or identifying the difference between expected and actual results.
Testing
Testing is the process of executing a program (or Part of it) with the intention or goal of finding errors.
Testing objectives
Primary role of testing is not demonstration of correct performance but the exposure of hidden defects.
G J Myers
Why testing
Testing is primarily a validation task.
The longer the defect remains, the more expensive it is to remove it.
Areas impacted by modification should be identified and retested for regression.
Most usability aspects surface while testing.
Test phases
1. Establish test objectives
2. Design test case
3. Write test case
4. Review test cases
5. Execute the tests
6. Examine test results
Points to ponder
1. Testing is largely a problem of economics.
2. Exhaustive input testing is impossible
3. Each test case should provide maximum yield.
4. Yield is the probability that the test case will expose a previously undetected error.
5. Investment is measured by the time and the cost to produce, execute and verify test and communicate.
6. Investment is limited by schedule and budget
7. Art of test case design is really art of selecting those test cases with highest yield.
8. Second most important testing consideration is sequence of integration.
Test strategies
1. Black box: test to the specification (don't look inside code)
2. White box: test to the code (don’t look inside specification)
Testing strategies
1. Black box testing: testing of a system or component whose inputs, outputs and general functions are known, but whose contents or implementation are unknown or irrelevant.
2. Structural testing/white box/glass box testing: testing that takes into account internal mechanism of a system or component. Types include branch testing, path testing etc. synonymous with glass box testing and white box testing.
Black box Vs. White box
Black box (Specification based): use the specification document as your basis for testing coverage.
e.g. test every function described in the document.
White box (implementation based): use the code as your basis for testing coverage.
e.g. test every branch of every if-statement.
Black box / white box – a comparison
Black box
1. test are derived from functional design specification
2. will test to fail hidden function.
3. data driven.
4. require exhaustive input testing to detect all errors.
White box
1. requires knowledge of internal program structure and code
2. will fail to detect missing functions
3. logic driven testing
4. require exhaustive path testing to detect all errors.
Levels of testing
1. Unit testing- done by the developer at module level.
2. Integration testing: conducted by the project team in work parallelism.
3. System testing: conducted by project team or by separate testing team if any.
4. Acceptance testing: conducted by client either in developers site or at his site.
Black box testing
1. Conducted for integration testing, system testing, acceptance testing
2. Test case design methods
3. Equivalence partitioning method.
4. Boundary value analysis method.
5. Cause effect graph method
6. State transition testing.
7. Use case based testing.
Unit testing
1. Testing logical pieces of work functions, subroutines or logical units.
2. Generally conducted by the developer
3. Objectives for unit testing may be stated as
3.1. does the logic works properly.
3.2. is all the necessary logic present.
4. Unit test documentations to be maintained as a permanent record.
Integration testing
Both integration and testing are carried out in this phase
Units are combined one at a time and tested, till entire software integrated.
Integration as per policy
1.top down.
2. bottom up
3. modified methods.
In top down, functionality could be tested much early.
In bottom up, functionality checked at the end.
In either case interfaces are tested.
Bottom up testing
Terminal modules are tested in isolation
Next set of modules to be tested are modules that directly call these tested modules along with. Previously tested terminal modules
Repeat process until top is reached.
Bottom up testing requires module driver to feed test case input to interface of module being tested.
advantages
1. Advantageous if major flaws occur at bottom of program.
2. Test condition easier to create.
3. Test result easier to view.
4. Disadvantages.
5. Program as an entity does not exist until last module is added- design flaws will be detected only at the end.
Top down testing
1. Only module unit tested in isolation is top module in program structure
2. After this, modules directly called by this module are merged one by one.
3. Repeat process until all modules have been combined and tested
Stub module is rarely as simple as return statement because calling modules usually expects output parameters from called module
Usual approach is to wire in a fixed output, that is always return from stub
In some cases, stub module may return out to be bigger than simulated module
Advantages
It combines module testing, integration testing, and small amount of external function testing
Test cases can be written as external input once the I/O modules are inserted.
Feasibility or design flows can be detected early in the project
No need for module drivers
Disadvantages
Stubs may not be very simple
Modified top down
Each module is unit tested in isolation before it is integrated using top down
Requires stubs and drivers for each modules
Sandwich testing
Compromise candidate
Top down and bottom up testing are started simultaneously
Integration starts from both top and bottom and meet somewhere in the middle
Meeting point shall be determined by examining structure of program
Suitable for large programs such as OS.
Modified sandwich testing
Lower levels are tested in bottom up fashion
Modules in upper levels are unit tested before integration using top-down method
Regression testing
Testing a defect fix
Testing that a defect fix has not caused some other errors.
Both are done by regression testing by running a se of test cases run previously (capture and play back tools can be used)
System testing
All this stage, both the target hardware and software are available
Test cases derived using black box methodology from SRS
System testing has to be taken up only with the target hardware
In addition to functionality, other aspects such as usability, performance, recovery etc. tested.
System testing is not a process of testing the function of the complete system which has already been achieved by function testing
System testing is the process of attempting to demonstrate how the program does not meet its objective.
Facility testing
Test – Determine if every facility mentioned in objectives has been implemented
Procedure – Scan the objectives line by line and compare with user manual.
Examples – prompt the user to select one of the alternatives user should be able to specify range of values
Volume testing
Test - subject the program to heavy volumes of data
Procedure: Check fields, records and files to see if their sizes can accommodate all expected data (use an automated tool to create records)
Example: large volumes of data in client/ server applications
Load / performance testing
Test – Load the system with activity that simulates legitimate user activity. Statistics collected to predict what performance and response times user are likely to get.
Procedure: conduct load test by creating virtual users. Use a load test tool and create typical scenarios to simulate load. Use think time to simulate authentic user behavior.
Stress testing
Test: subject the application under test to peak volume of data in a short time. It is extremely important test for system that normally operates below capacity but may be severely stressed during peak demand. It is a type of performance testing.
Procedure: conduct stress test by creating virtual users. Use a test tool and create typical scenarios to simulate. Do not use think times as the idea is to exercise the system to fullest extent.
What must be in place before load test/ stress test?
Hardware in terms of DB server and application server must be functional
Network must be in place or statistics obtained will not be correct
Application must be ready in terms of core DB functionality like inserting records, updating records or preparing major reports.
Test scenarios must be ready
Realistic test DB must be created
Usability testing
Check for human factor problems
-Are output meaning full?
-Are error diagnostics straight forward?
-Does GUI have conformity of syntax conventions, format, style, abbreviations?
-Is it easy to use?
usability
1. It should not
-Annoy intended user in function or speed
-Take control from the user without indicating when it will be returned
2. It should
-Provide on-line help or user manual
-Consistent in its function and over all design
Security testing
Devise test cases that subvert the program’s security check:
1.Obtain passwords
2.Access idle terminals
3.Imitate valid users
4.Guess passwords
5.Check permissions of different user groups/ users
6.Check database security
7.Create more user than allowed in user group
8.Delete user groups like supervisor/ admin
9.Rename user groups like supervisor/ admin
Storage testing
Detect amount of main and secondary storage requirements of program
Determine capacity of system to store transaction data on a disk or in other files e.g. ten thousand records of 512 bytes in a single flexible disk?
Recovery testing
Determine ability of user to recover data or restart after a failure.
Verifies both recovery process and component of process of recovery
Examples of recovery testing:
-Loss of input capability
-Loss of communication lines
-Loss of database integrity
-Application system failure
-Operator mistake
Recovery ability check
Adequate back up data is preserved
-Back up is stored in a secure location
-Recovery procedures are documented
-Recovery personnel are assigned and trained
-Procedure for performing manual aspects of recovery are adequate
Install ability testing
This tests the installation procedure on:
-Clean system
-Over a previous version of itself
-Uninstall it completely
-Check for auto play functionality
-Check custom install, partial install, add install if any
-Check online registration if any
-Check installation of third party components
Acceptance testing
Final stage before handing over to the customer
Usually carried out by the customer
Test cases executed with actual data
Generally conducts subset of test conducted in system testing
Alpha testing
Tested at developer site by customer
Developer “looks over shoulder” and records errors and usage problems
Test conducted in a controlled environment
Beta testing
Formal acceptance test of product impossible
Beta testing conducted by one or more customer sites by end user of software
Live application environment cannot be controlled by developer
Customer records all problems encountered and reported to developer at regular intervals
Defect classification- Origin
Requirements defect
Design defect
Coding defect
Documentation defect
Bad fixes
What is a test tool?
Software that aids in planning, developing, executing, of the testing process for another software package with the intention of detecting errors, tracking and reporting them
They aid in setting up an unsupervised test facility for software testing.
Advantages of software testing tools
Testing is formalized
Testing process is automatically documented
Test plans can be reused
Defect tracking is systematic
Efficient because test scripts are developed that can be used in subsequent builds
Types of testing tools
Test planning, management and error tracking
Reviews and inspections
Test generation
Test execution
No comments:
Post a Comment