Overview
Testing is a set of activities conducted to facilitate the discovery and/or evaluation of properties of one or more items under test. Each test, known as a test case, exercises a set of predefined test activities, developed to drive the execution of the test item to meet test objectives; including correct implementation, error identification, quality verification, and other valued details. The test environment is usually designed to be identical, or as close as possible, to the anticipated production environment. It includes all facilities, hardware, software, firmware, procedures, and/or documentation intended for or used to perform the testing of software. UAT and OAT test cases are ideally derived in collaboration with business customers, business analysts, testers, and developers. These tests must include both business logic tests as well as operational environment conditions. The business customers (product owners) are the primary stakeholders of these tests. As the test conditions successfully achieve their acceptance criteria, the stakeholders are reassured the development is progressing in the right direction. * User acceptance test (UAT) criteria (in agile software development) are usually created by business customers and expressed in a business domain language. These are high-level tests to verify the completeness of a user story or stories 'played' during any sprint/iteration. * Operational acceptance test (OAT) criteria (regardless of using agile, iterative, or sequential development) are defined in terms of functional and non-functional requirements; covering key quality attributes of functional stability, portability, and reliability.Process
The acceptance test suite may need to be performed multiple times, as all of the test cases may not be executed within a single test iteration. The acceptance test suite is run using predefined acceptance test procedures to direct the testers on which data to use, the step-by-step processes to follow, and the expected result following execution. The actual results are retained for comparison with the expected results. If the actual results match the expected results for each test case, the test case is said to pass. If the quantity of non-passing test cases does not breach the project's predetermined threshold, the test suite is said to pass. If it does, the system may either be rejected or accepted on conditions previously agreed between the sponsor and the manufacturer. The anticipated result of a successful test execution: * test cases are executed, using predetermined data * actual results are recorded * actual and expected results are compared, and * test results are determined. The objective is to provide confidence that the developed product meets both the functional and non-functional requirements. The purpose of conducting acceptance testing is that once completed, and provided the acceptance criteria are met, it is expected the sponsors will sign off on the product development/enhancement as satisfying the defined requirements (previously agreed between business and product provider/developer).User acceptance testing
User acceptance testing (UAT) consists of a process of verifying that a solution works for the user. It is not system testing (ensuring software does not crash and meets documented requirements) but rather ensures that the solution will work for the user (i.e. tests that the user accepts the solution); software vendors often refer to this as "Beta testing". This testing should be undertaken by the intended end user, or a subject-matter expert (SME), preferably the owner or client of the solution under test and provide a summary of the findings for confirmation to proceed after trial or review. InOperational acceptance testing
Operational acceptance testing (OAT) is used to conduct operational readiness (pre-release) of a product, service or system as part of a quality management system. OAT is a common type of non-functionalAcceptance testing in extreme programming
Acceptance testing is a term used in agile software development methodologies, particularly extreme programming, referring to the functional testing of a user story by the software development team during the implementation phase. The customer specifies scenarios to test when a user story has been correctly implemented. A story can have one or many acceptance tests, whatever it takes to ensure the functionality works. Acceptance tests are black-box system tests. Each acceptance test represents some expected result from the system. Customers are responsible for verifying the correctness of the acceptance tests and reviewing test scores to decide which failed tests are of highest priority. Acceptance tests are also used as regression tests prior to a production release. A user story is not considered complete until it has passed its acceptance tests. This means that new acceptance tests must be created for each iteration, or the development team will report zero progress.Types of acceptance testing
Typical types of acceptance testing include the following ; User acceptance testing:This may include factory acceptance testing (FAT), i.e. the testing done by a vendor before the product or system is moved to its destination site, after which site acceptance testing (SAT) may be performed by the users at the site. ; Operational acceptance testing:Also known as operational readiness testing, this refers to the checking done to a system to ensure that processes and procedures are in place to allow the system to be used and maintained. This may include checks done to back-up facilities, procedures for disaster recovery, training for end users, maintenance procedures, and security procedures. ; Contract and regulation acceptance testing : In contract acceptance testing, a system is tested against acceptance criteria as documented in a contract, before the system is accepted. In regulation acceptance testing, a system is tested to ensure it meets governmental, legal and safety standards. ; Factory acceptance testing : Acceptance testing conducted at the site at which the product is developed and performed by employees of the supplier organization, to determine whether a component or system satisfies the requirements, normally including hardware as well as software. ; Alpha and beta testing : Alpha testing takes place at developers' sites and involves testing of the operational system by internal staff, before it is released to external customers. Beta testing takes place at customers' sites and involves testing by a group of customers who use the system at their own locations and provide feedback, before the system is released to other customers. The latter is often called "field testing".Acceptance criteria
According to the Project Management Institute, acceptance criteria is a "set of conditions that is required to be met before deliverables are accepted." Requirements found in acceptance criteria for a given component of the system are usually very detailed.List of acceptance-testing frameworks
* Concordion, Specification by example (SbE) framework ** Concordion.NET, acceptance testing in .NET * Cucumber, a behavior-driven development (BDD) acceptance test framework ** Capybara, Acceptance test framework for Ruby web applications ** Behat, BDD acceptance framework for PHP ** Lettuce, BDD acceptance framework for Python * Cypress * Fabasoft app.test for automated acceptance tests * Framework for Integrated Test (Fit) ** FitNesse, a fork of Fit * Gauge (software), Test Automation Framework from Thoughtworks * iMacros * ItsNat Java Ajax web framework with built-in, server based, functional web testing capabilities. * Maveryx Test Automation Framework for functional testing, regression testing, GUI testing, data-driven and codeless testing of Desktop and Web applications. * Mocha, a popular web acceptance test framework based on Javascript and Node.js * Playwright (software) * Ranorex * Robot Framework * Selenium * Specification by example (Specs2) * WatirSee also
* Acceptance sampling * Conference room pilot * Development stage * Dynamic testing * Engineering validation test * Grey box testing * Test-driven development * White box testing * Functional testing (manufacturing)References
Sources
*Further reading
*External links
*