DataWarehouse - Testing

A test is performed by an independent group of testers after the functionality is developed, before it is shipped to the customer to verify that the system is conformed to the specification.

In a “test-driven software development” model, unit tests are written first.

Testing is questioning a product in order to evaluate it.

Two general type of test:

• unit (application)
• or system (hardware)

Testing can only reveal the presence of errors, not the absence of errors

Edsger Dijkstra

Generating test data that is sparse, incomplete, and in some cases fully complete and fully correct allows testing of:

• the domain alignment
• the normalization,
• missing keys,
• nulled attributes,
• relationships,
• zero-key generation,
• elimination of duplicates,
• etc…

• the start and end-dates of specific records (ie: the delta processing detection),
• Data Reconciliation (known also as balancing) between all area (between the source system and the staging area, the operational data store and the staging area, …)
• the transactional volume. If you receive 100 records by day and then suddenly 20, you may have an error in the change data capture process.

When you can't directly access the data source (e.g. web-service data source), the data can be stored in a staging area just for testing purposes later.

Volume tests is a functional test via a high volume of generated data to make sure:

• the partitioning is right
• the performance is acceptable
• the indexing works

and to test:

• the joins
• and the load cycle performance as well.

Tests are frequently grouped by where they are added in the software development process, or by the level of specificity of the test.

• Unit testing refers to tests that verify the functionality of a specific section of code, usually at the function level. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors
• Integration testing. Works to expose defects in the interfaces and interaction between integrated components (modules).
• System testing. Tests a completely integrated system to verify that it meets its requirements.
• System integration testing verifies that a system is integrated to any external or third party systems defined in the system requirements.
• Regression testing is any type of software testing that seeks to uncover software errors by partially retesting a modified program. The intent of regression testing is to provide a general assurance that no additional errors were introduced in the process of fixing other problems. Regression testing is commonly used to efficiently test the system by systematically selecting the appropriate minimum suite of tests needed to adequately cover the affected change. Common methods of regression testing include rerunning previously run tests and checking whether previously fixed faults have re-emerged. “One of the main reasons for regression testing is that it's often extremely difficult for a programmer to figure out how a change in one part of the software will echo in other parts of the software.
• Acceptance testing can mean one of two things:
  • A smoke test is used as an acceptance test prior to introducing a new build to the main testing process, i.e. before integration or regression.
  • Acceptance testing performed by the customer, often in their lab environment on their own hardware, is known as user acceptance testing (UAT). Acceptance testing may be performed as part of the hand-off process between any two phases of development.

Performance testing is executed to determine how fast a system or sub-system performs under a particular workload.

It can also serve to validate and verify other quality attributes of the system, such as:

• scalability,
• reliability
• and resource usage.

Load testing is primarily concerned with testing that can continue to operate under a specific load, whether that be:

• large quantities of data
• or a large number of users.

This is generally referred to as software scalability. The related load testing activity is often referred to as endurance testing.

• Volume testing is a way to test functionality.
• Stress testing is a way to test reliability.
• Load testing is a way to test performance.

There is little agreement on what the specific goals of load testing are. The terms load testing, performance testing, reliability testing, and volume testing, are often used interchangeably.

• Stability testing. checks to see if the software can continuously function well in or above an acceptable period. This testing is oftentimes referred to as load (or endurance) testing.
• Security testing. processes confidential data to prevent bad data security and system intrusion by hackers
• Internationalization and localization. It will verify that the application still works, even after it has been translated into a new language or adapted for a new culture (such as different currencies or time zones).
• Destructive testing attempts to cause the software or a sub-system to fail, in order to test its robustness.
• Usability testing. check if the user interface is easy to use and understand.

How do you determine “Concurrent” users?

• Oracle adopt the 10-20% rule:'
  • 10-20% of users will be logged in to Oracle BI at any one time
  • 10-20% of those users logged in will actually be running a query

So for 3000 “Total” users, you could expect concurrency of between 30 and 120 “Concurrent users”

Take time zones into account e.g. for a “Global” implementation divide the “Total” number of users by 3.

• SAT = site acceptance test
• FAT = Factory acceptance test
• UAT = User acceptance test
• USR = User system requirement

These must be the same thing. This should be a document explaining exactly what the user (customer) wants from the system. Not how it's going work, but what it's going to do. “I want a machine to make coffee, and it must have the option for white, black, espresso and chicken soup. Oh, and sugar. Oh and it must have two different sizes of cups. And a biscuit dispenser.” etc etc. This should be signed off between supplier and customer at the outset so there is an agreed set of deliverables that can be measured at the end. (Things like two different sizes of cups is not a good example. These should be specified in fluid ounces, millilitres whatever. This is a specification document, not just a description.)

Here the supplier is converting the needs (the 'what') in to the methods (the 'how'). This document defines for him how he intends to fulfil the URS. At the same time, and following on from the URS, some form of test specification should be beginning to emerge. How can we tell if it's working OK? What will be the agreed measures?

Once the machine/system/software has been assembled at the supplier's factory there should be a formal test with checkpoints and performance measures to prove that it is capable of meeting the URS. NB This does not mean it has to meet the URS there and then. That may be impossible until it's finally plumbed in on site. Very often customers will ask to be present at the FAT to witness this before giving permission for the system to be shipped to their site. Some projects will have stage payments on successful completion of a witnessed FAT.

Carried out at user's premises with equipment installed and ready to run. Must prove that it can do everything agreed in the URS. This is the point where the user satisfies himself that the system is substantially as requested and accepts delivery of the equipment. From that point he gets invoiced for the system. Further work may continue by the supplier, but this should be agreed and defined in change notices or deviations to original specifications, and I would expect most of this will represent an additional charge.

• Software_testing
• Regression testing
• Difference betweeen SAT, FAT, UAT, USR?