This chapter is from the book
This chapter is from the book
This chapter is from the book
Adapter QA
Quality assurance is a tedious task but extremely important. The best architecture and design is not very useful if the software does not comply with requirements, and if it has too many bugs. Fixing software bugs is a continuous process that involves constant checking and fixing. Adapter QA is a more complicated task that considers the involvement of not one but many applications in a scenario. Two levels of QA are basic to adapter development: adapter unit tests and integration scenario tests.
Adapter unit tests are focused on testing the adapter features and capabilities with reference to the adapter requirements. They involve testing the connectivity between the adapter and the business application, the fail over, and other recovery mechanisms programmed into the adapter. Adapter unit tests are important to ensure the basic level of integration between the business application and adapter on a one-on-one basis.
However, adapter unit tests are not enough to certify the adapter fit for participating in integration scenarios. That is the job of integration scenario tests, sometimes known as end-to-end testing. These tests are more complex, and involve a series of adapters collaborating in a simulated integrating scenario using test data. Sometimes, these tests are run for days to ensure the adapters do their job over a long period of time without crashing or corrupting any data.
Ironically, these tests expose bugs in operating systems—bugs in business applications, middleware, and so on. It is only in the integration scenario tests that the entire infrastructure associated with the scenario (including hardware, operating systems, databases, applications, adapters, and other middleware) are tested as a single unit of operation.
The usual system tests or integrated system tests conducted as part of software development are slightly different from integration scenario testing. The objective of the system test is to identify bugs in a particular business application. The objective of integration scenario testing is to ensure that all points of integration work as expected. The scope of testing is therefore much bigger and involves many potential points of failure.
Setup of the QA Environment
It is no surprise that setting up the QA environment for integration scenario testing, as well as a separate adapter unit test, is not a small task. Most QA teams prefer to maintain their test environments for a long time after they are set up. It takes a lot of effort to build a valid QA environment, complete with meaningful test data, decent hardware configurations, and defining appropriate scenarios. Automating the repetitive testing cycles is very important to maintain consistency between testing different versions of the adapters, and so on. Without a consistent QA environment, it is hard to isolate bugs, or even prove that past bugs don't exist in new versions of the software.
Selecting Valid Test Data
Access to good quality test data is always a challenge. Sometimes, QA teams consider copies of production data (data from applications in production environments) as good test data. This principle is problematic, however, because in many instances production data only represents the valid conditions. Test exceptions, business rules, security policies, and so on require an incredibly comprehensive set of test data. Most of the time, the quality of test data increases with the number of testing cycles.
QA teams need to expand their testing procedures and include test data for end-to-end integration scenarios. This requires test data from potentially multiple applications in different databases. This task of defining the appropriate set of test data should begin as soon as the integration requirements are known. Test plans and test databases should be defined and populated as soon as possible to avoid lengthy QA cycles in the later half of the development schedules.
Identifying Regression Test Cases
How many times should you test the adapter? If a minor change has been made to an adapter, or if an adapter has some bugs fixed, what kind of testing is required before it is deployed? These are some of the questions that QA engineers, project managers, and IT staff face regularly in any integration project. Regression tests define a set of test cases that represent critical aspects of the integration scenario. Not all adapter test cases are required for regression tests. Generally, each adapter has a specific set of features or functions relevant to a particular integration scenario. All features of all adapters probably are not deployed or even required to fulfill a specific integration scenario.
Hence, it is important to identify a series of test cases that represent the critical functionality in play for a particular integration scenario, and reference them as regression test cases. Regression test cases are useful for testing minor upgrades to adapters or bug fixes or maintenance releases of adapters. They provide the adapter QA team with enough testing procedures to test adapters quickly before deploying them in a production environment. However, if the adapters have undergone significant changes, it may be better to test using the full set of test cases.
Developing a Test Harness
A very useful part of adapter QA is the availability of a test harness. A test harness is a self-contained testing tool that is part of the adapter. It helps in conducting quick and easy adapter unit tests without major QA infrastructure. It also helps developers test their adapters because other adapters or even applications may not be ready for a comprehensive adapter QA. Developers and project managers can decide to include a test harness as part of the development schedule. However, the effectiveness of such a harness is likely to go down if it is not kept in sync with the adapter and the business application.