Testing Glossary.csv

Name,Subject(s),Definition,Parent(s),Synonym(s),Notes
Unit Testing,Code (modules/functions),"""Testing the individual modules [of a program]"" (van Vliet, 2000, p. 438). See also (Patton, 2006, p. 109).",N/A,"Module Testing (Patton, 2006, p. 109), Component Testing (Peters and Pedrycz, 2000, p. 444)","Because of the definition of ""component"" (""a part of a system that can be tested in isolation"" (International Software Testing Qualifications Board, 2022)), this seems to apply to the testing of both modules and specific functions."
Integration Testing,Code (module interfaces),"""Testing the composition of modules"" that is done incrementally (van Vliet, 2000, pp. 438-439). See also (Patton, 2006, p. 109).",N/A,N/A,
System Testing,Code (most or all),"""Test[ing] the whole system against the user documentation and requirements specification"" (van Vliet, 2000, p. 439). See also (Patton, 2006, p. 109).",N/A,N/A,
Acceptance Testing,"Code, documentation (requirements)","Testing similar to system testing that is ""often performed under supervision of the user organization"" that focuses on usability (van Vliet, 2000, p. 439) and the needs of the customer(s) (Peters and Pedrycz, 2000, p. 492).",System Testing?,N/A,
Installation Testing,"Code, documentation (installation)","Testing that focuses on the portability of the product, especially ""in an environment different from the one in which is has been developed"" (van Vliet, 2000, p. 439).",Acceptance Testing?,N/A,"Not one of the four levels of testing identified by the IEEE standard (Peters and Pedrycz, 2000, p. 445)."
Bottom-up Testing,Code (most or all),"Integration testing that uses test drivers: ""tool[s] that generate[] the test environment for a component to be tested"" (van Vliet, 2000, p. 410). See also (Patton, 2006, p. 109).",Integration Testing,N/A,
Top-down Testing,Code (most or all),"Integration testing that uses test stubs: tools that ""simulate[] the function of a component not yet available"" (van Vliet, 2000, p. 410) by providing ""fake"" values to a given module to be tested (Patton, 2006, p. 110).",Integration Testing,N/A,
Big Bang Testing,Code (most or all),"Integration testing of the system resulting from ""integrat[ing] all modules in a single step"" (Peters and Pedrycz, 2000, p. 489).",Integration Testing,N/A,
Sandwich Testing,Code (most or all),"Bottom-up and top-down testing performed together ""by defining a certain target layer in the hierarchy of the modules"" and working towards it from either end (Peters and Pedrycz, 2000, p. 491).","Integration Testing, Bottom-up Testing?, Top-down Testing?",N/A,
Black-Box Testing,"Code, documentation","""Testing based on an analysis of the specification of the [software]"" (International Software Testing Qualifications Board, 2022).",N/A,"Specification-Based Testing (International Software Testing Qualifications Board, 2022)",
Specification Testing,"Code, documentation (requirements)",,Black-Box Testing,Static Black-Box Testing,"(Patton, 2006, p. 56) calls this ""Static Black-Box Testing"", but mentions that it is the process of ""testing the specification""; I think ""specification testing"" is a better name."
Terminology Checklist,Documentation,"Identifying specifications that are potentially unrealistic, vague, or incomplete (Patton, 2006, p. 61).",Specification Testing,N/A,This seems like a black-box review (similar to the Code Reviews).
Coverage-Based Testing,Tests (coverage),"Testing derived from requirements ""depicted as a graph, where the nodes denote elementary requirements and the edges denote relations between [them]"" (van Vliet, 2000, p. 425).",Specification Testing,N/A,
Functional Testing,Code,"Testing performed by ""entering inputs, receiving outputs, and checking the results"" (Patton, 2006, p. 64).",Black-Box Testing,"Dynamic Black-Box Testing, Behavioural Testing",Is this the most common name?
Exploratory Testing,Code,"Testing features that are determined by exploring the software (Patton, 2006, p. 65).",Functional Testing,N/A,"An alternative to dynamic black-box testing when a specification is not available (Patton, 2006, p. 65)."
Equivalence Partitioning,Tests (effficiency),"The process of dividing the infinite set of test cases into a finite set that is just as effective (Patton, 2006, p. 67).",Functional Testing,Equivalence Classing,"The opposite of this, ""exhaustive testing"", is ""probably not feasible"" (Peters and Pedrycz, 2000, p. 461)."
Data Testing,Code,"Testing ""that information the user inputs [and] results"", both final and intermediate, ""are handled correctly"" (Patton, 2006, p. 70).","Functional Testing, Structural Testing",N/A,Could this be more specific?
Boundary Condition Testing,Code,"Testing that uses data ""at the edge[s] of the planned operational limits of the software"" (Patton, 2006, p. 72).",Data Testing,N/A,"Boundary conditions should be tested using ""the valid data just inside the boundary, …the last possible valid data, and …the invalid data just outside the boundary"" (Patton, 2006, p. 73) as well as the data on the boundary itself (Peters and Pedrycz, 2000, p. 460)."
Buffer Overrun Testing,Code,,Boundary Condition Testing,N/A,Is this a meaningful distinction?
Sub-Boundary Condition Testing,Code,"Testing that uses data at the edge(s) of boundary conditions ""that are internal to the software [but] aren't necessarily apparent to an end user"" (Patton, 2006, p. 75).","Data Testing, Boundary Condition Testing?",N/A,
"Default, Empty, Blank, Null, Zero, and None Testing",Code,,Data Testing,N/A,Does this (and other types of data testing) have a better name? Does it make sense for these to be separate from data testing?
"Invalid, Wrong, Incorrect, and Garbage Data Testing",Code,,Data Testing,N/A,
Syntax-Driven Testing,Code,"Testing based on ""the syntax or constraint of input domains defined in requirement specification"" (Intana et al., 2020, p. 260).",Data Testing,N/A,
Decision Table-Based Testing,Code,"Testing based on ""the combinations of conditions and the resulting actions"" (International Software Testing Qualifications Board, 2022) from requirements (Peters and Pedrycz, 2000, pp. 448, 450).",Data Testing,N/A,
State Testing,Code,"Testing ""the program's logic flow through its various states"" (Patton, 2006, p. 79) by checking that state variables are correct after different transitions (p. 83).",Functional Testing,"State Transition Testing, Finite State Testing (International Software Testing Qualifications Board, 2022)",Usually uses a state transition diagram (add to gen. term. page).
Performance Testing,Code,"Testing to determine how efficiently software uses resources (including time and capacity) ""when accomplishing its designated functions"" (International Software Testing Qualifications Board, 2022).",State Testing?,N/A,
Race Conditions and Bad Timing Testing,Code,,State Testing,N/A,Does this have a better name?
Repetition Testing,Code,"Testing ""the same operation over and over"", potentially up to ""thousands of attempts"" (Patton, 2006, p. 86).",State Testing,N/A,Find a source that highlights the rationale: making sure resources are used efficiently.
Stress Testing,Code,"Testing ""the software under less-than-ideal conditions"" to ""determine what external resources and dependencies it has"" (Patton, 2006, p. 86).",State Testing,N/A,
Load Testing,Code,"Testing the software with as large of a load as possible (e.g., large inputs, many peripherals) (Patton, 2006, p. 86).",State Testing,N/A,What is the purpose of this?
White-Box Testing,Code,"""Testing based on an analysis of the internal structure of [software]"" (International Software Testing Qualifications Board, 2022).",N/A,"Clear-box Testing (Patton, 2006, p. 55), Glass Box Testing (Peters and Pedrycz, 2000, p. 439)",
Structural Analysis,Code,"""The process of carefully and methodically reviewing the software design, architecture, or code for bugs without executing it"" (Patton, 2006, p. 92).",White-Box Testing,Static White-Box Testing,Is this the most common name?
Code Review,Code,"Structural analysis that focuses on ""find[ing] problems with the software"" in a group (Patton, 2006, pp. 92-93).",Structural Analysis,N/A,
Formal Review,Code,"Code review ""that follows a defined process with a formally documented output"" (International Software Testing Qualifications Board, 2022).",Code Review,N/A,"This process can include preparing in advance, setting expectations, and reporting on findings (Patton, 2006, p. 93), as well as checking for standards (Patton, 2006, pp. 96, 99-103)."
Informal Review,Code,"Code review ""that does not follow a defined process and has no formally documented output"" (International Software Testing Qualifications Board, 2022).",Code Review,N/A,
Peer Review,Code,"A less formal review where two or three developers review code that one of them has written as a group (Patton, 2006, p. 94).","Formal Review (Patton, 2006, p. 94), Informal Review (Patton, 2006, p. 94)","Buddy Review (Patton, 2006, p. 94)","Multiple variations; see (Patton, 2006, p. 94) and (van Vliet, 2000, p. 414)."
Walkthrough,Code,"A formal review where the author of some code presents it to a small group (Patton, 2006, p. 95) using test data to ""walk through"" its execution (van Vliet, 2000, p. 416).",Formal Review,N/A,"May have defined roles (Peters and Pedrycz, 2000, p. 484)."
Inspection,Code,"A formal review where code is presented by someone other than its author to a small group of people (Patton, 2006, p. 95), each with defined roles (Patton, 2006, p. 95); (Peters and Pedrycz, 2000, p. 439).",Formal Review,N/A,
Correctness Proofs,Code,"The use of ""highly formal methods of logic"" (Peters and Pedrycz, 2000, p. 438) to prove that a program and its specification are equivalent (Peters and Pedrycz, 2000, p. 485).",Formal Review,N/A,"Requires a formal specification (van Vliet, 2000, p. 418)."
Structural Testing,Code,"Testing that uses ""information …from seeing what the code does and how it works"" (Patton, 2006, p. 106).",White-Box Testing,Dynamic White-Box Testing,Is this the most common name?
Code Coverage,"Code, Tests (coverage)","""Test[ing] the program's states and the program's flow among them"" (Patton, 2006, p. 117).",Structural Testing,"Control-Flow Coverage (van Vliet, 2000, pp. 421)",What is the purpose of this?
Statement/Line Coverage,"Code, Tests (coverage)",,Code Coverage,N/A,Line vs. statement coverage?
Branch Coverage,"Code, Tests (coverage)","Testing that attempts to, ""at each branching node in the control graph, …[choose] all possible branches …at least once"" (van Vliet, 2000, p. 421).",Code Coverage,N/A,"(Patton, 2006, p. 119) says that this is ""the simplest form of path testing"", but I don't think this is correct."
Path Coverage,"Code, Tests (coverage)","Testing that ""attempt[s] to cover all the paths in the software"" (Patton, 2006, p. 119).",Code Coverage,N/A,
Condition Coverage,"Code, Tests (coverage)","Branch coverage that ""takes the extra conditions on the branch statements into account"" (e.g., all possible inputs to a Boolean expression) (Patton, 2006, p. 120).",Code Coverage,"Multiple Condition Coverage, Extended Branch Coverage",
Fault Seeding,"Code, Tests (coverage)","The introduction of faults to estimate the number of undiscovered faults in the system (van Vliet, 2000, p. 427).",Structural Testing,N/A,
Mutation Testing,"Code, Tests (coverage)","Running sets of tests on mutants of a system to maximize the number of mutants identified by a given test set (van Vliet, 2000, p. 429).",Structural Testing,N/A,Could this be more specific?
Strong Mutation Testing,"Code, Tests (coverage)","Mutation testing at the program level (van Vliet, 2000, p. 429).",Mutation Testing,N/A,
Weak Mutation Testing,"Code, Tests (coverage)","Mutation testing at the component level (van Vliet, 2000, p. 429).",Mutation Testing,N/A,
Gray-Box Testing,Code,"Testing ""the software as a black-box"" alongside limited information about ""what makes the software work"" (Patton, 2006, p. 218).",N/A,N/A,
Regression Testing,Code,"Retesting code using existing tests (Patton, 2006, p.232) ""to make sure there are no unwanted changes [to the software's behaviour]"" (Peters and Pedrycz, 2000, p. 481).",N/A,N/A,
Metamorphic Testing,Code,"Testing based on relations between changes to the input(s) and changes to the output(s) (Kanewala and Yueh Chen, 2019, pp. 67-68).",Black-Box Testing?,N/A,