Software testing, cost estimation, quality management, and configuration management

Building software right needs more than design: it must be tested, its cost and schedule estimated, its quality assured and its versions controlled. This topic covers testing levels and test-case design, metrics, algorithmic cost estimation (COCOMO), software quality assurance (SQA), standards such as ISO 9001 and CMMI, and software configuration management (SCM).

Key concepts

Testing levels

• Unit testing: tests individual units (functions, classes) in isolation, usually by developers, using drivers (call the unit) and stubs (stand in for called modules).
• Component testing: tests composite components through their interfaces (interface misuse, misunderstanding, timing errors).
• Integration testing: tests combined modules. Top-down (needs stubs), bottom-up (needs drivers), sandwich, big-bang (all at once). Regression testing repeats tests after changes.
• System testing: whole system against requirements — functional plus performance, stress, load, security, recovery, usability testing.
• Acceptance testing: by the customer to decide whether to accept. Alpha testing at the developer's site with users; beta testing at the user's site in a real environment.

Test case design

• Black-box (functional): based on specification — equivalence partitioning, boundary value analysis, cause–effect graphs/decision tables, state-transition testing.
• White-box (structural): based on code — statement, branch, path coverage; basis path testing using cyclomatic complexity; loop testing; data-flow testing.

Test automation: tools run tests and compare results automatically (e.g. JUnit, Selenium); essential for regression testing.

Metrics for testing: defect density (defects per KLOC), test coverage, defect removal efficiency $DRE = E/(E + D)$ ($E$ = errors found before delivery, $D$ = defects found after), cyclomatic complexity.

Algorithmic cost modelling

• Effort $= A \times \text{Size}^B \times M$, where size is in KLOC or function points, $A$ a constant, $B$ an exponent (diseconomy of scale when $B > 1$), $M$ a product of cost drivers.
• Basic COCOMO (Boehm): three modes — organic (small, familiar), semi-detached (medium), embedded (tight constraints).
• COCOMO II has application composition, early design and post-architecture models.

Project duration and staffing: schedule depends on effort, not linearly — adding people to a late project makes it later (Brooks' law). Staffing follows a Rayleigh-type curve (Putnam model).

Software quality assurance (SQA)

• Planned, systematic activities to ensure product and process meet standards (reviews, testing, defect tracking, measurement).
• Formal technical reviews (FTR): meetings (typically 3–5 people, under ~2 hours, prepared in advance) to find errors in work products; walkthroughs and inspections (Fagan) are types. Review the product, not the producer.
• Formal approaches to SQA: proof of correctness using mathematical specification; cleanroom software engineering (formal specification, correctness verification, statistical usage testing).
• Statistical SQA: collect and categorise defects, trace each to its root cause, and apply the Pareto principle (80% of defects come from 20% of causes) to fix the vital few. Six Sigma (DMAIC) aims for 3.4 defects per million opportunities.

Framework for software metrics: measures → metrics → indicators; product, process and project metrics; direct (LOC, defects) vs indirect (quality).

Metrics for analysis and design models: function points (from inputs, outputs, inquiries, internal files, external interfaces), cohesion and coupling metrics, fan-in/fan-out, the CK metrics suite for OO design (WMC, DIT, NOC, CBO, RFC, LCOM).

ISO standards: ISO 9001 (quality management systems, applied to software through ISO 90003), ISO/IEC 12207 (software life-cycle processes), ISO/IEC 25010 (product quality model), ISO/IEC 15504 (SPICE, process assessment).

CMMI (Capability Maturity Model Integration, SEI) staged maturity levels:

1. Initial (ad hoc) 2. Managed (project management, requirements) 3. Defined (organisation-wide standard process) 4. Quantitatively managed (statistical control) 5. Optimising (continuous improvement).

SQA plan: purpose and scope, management, standards, reviews and audits, testing, problem reporting and corrective action, tools, records.

Configuration management (SCM)

• Planning: identify configuration items (code, documents, test data), naming conventions, responsibilities, baselines, tools.
• Baseline: a reviewed, approved version that changes only through formal change control.
• Change management: change request → impact and cost analysis → decision by a Change Control Board (CCB) → implement → verify → release.
• Version and release management: a version is an instance differing in some way; a release is a version delivered to customers. Version control systems (Git, SVN) support check-in/check-out, branching and merging.
• CASE tools for SCM: version control, build tools, issue trackers.

High-yield points

• Stubs for top-down integration; drivers for bottom-up.
• Alpha test at the developer's site; beta at the customer's site.
• Basic COCOMO organic: $E = 2.4\,\text{KLOC}^{1.05}$, $D = 2.5\,E^{0.38}$.
• $V(G) = E - N + 2P$ = decision points + 1.
• CMMI: 1 Initial, 2 Managed, 3 Defined, 4 Quantitatively managed, 5 Optimising.

Formulas

Basic COCOMO: $E = a\,(\text{KLOC})^b \ \text{person-months},\qquad D = c\,E^{d} \ \text{months},\qquad N = E/D$

Mode	$a$	$b$	$c$	$d$
Organic	2.4	1.05	2.5	0.38
Semi-detached	3.0	1.12	2.5	0.35
Embedded	3.6	1.20	2.5	0.32

$E$ = effort, $D$ = development time, $N$ = average staff.

Cyclomatic complexity: $V(G) = E - N + 2P$ (edges, nodes, connected components) $=$ number of decision points $+ 1$.

Worked example

An organic project has 10 KLOC. Find effort and duration (basic COCOMO).

1. $E = 2.4 \times 10^{1.05}$. $10^{1.05} = 10 \times 10^{0.05} \approx 10 \times 1.122 = 11.22$. $E \approx 26.9$ PM.
2. $D = 2.5 \times 26.9^{0.38}$. $\ln 26.9 \approx 3.292$; $\times 0.38 = 1.251$; $e^{1.251} \approx 3.49$. $D \approx 8.7$ months.
3. Staff $\approx 26.9 / 8.7 \approx 3.1$ people.

Exam tips

• Boundary value analysis and equivalence partitioning are black-box; basis path is white-box.
• CMMI level 3 = Defined, level 5 = Optimising.
• A baseline can change only through formal change control.

Exam-style questions

From Calmon's question bank, which has 26 questions on this topic.

1. Which test automation tool is mainly used for performance (load) testing?

• A. JUnit
• B. Selenium
• C. LoadRunner
• D. Cucumber

Answer: C. LoadRunner simulates many virtual users to measure system performance under load; JUnit is for unit tests, Selenium for browser UI tests and Cucumber for BDD tests.

2. Which model, developed by SEI, defines maturity levels used to assess and improve an organization's software development process?

• A. ISO 9126
• B. CMMI
• C. COCOMO
• D. Function Point Analysis

Answer: B. CMMI (Capability Maturity Model Integration) appraises process maturity on five levels and guides process improvement; COCOMO and FPA are estimation techniques and ISO 9126 is a product quality model.

Practise ACtE08 questions →