Testing Definitions and Scope

Recently I was asked about our testing strategy here in FS (Financial Services).
Before I could answer I drew a simple diagram on the white board to layout some simple definitions. I created a digital version of the diagram and thought I would share it with everyone.
Test Definitions and Scope

All tests take advantage of our FS Test utilities and frameworks. All tests are run multi-threaded and are thread safe. They take advantage of home state switching which facilitates testing against the target database.

Type Test Scope Tools Comments
Unit Tests Individual Java beans/services JUnit, JMock Tests mock out dependencies
DAO Tests Test DAO services with Hibernate and target databases JUnit, Hibernate, Spring Ensures that the DAO layer properly works with the real database. Allows for testing database features (Triggers, stored procs, constraints, views)
Integration Tests Test application without requiring Application Server to be started. Target database and transactional scope can be tested JUnit, Hibernate, Spring Ensures that the business logic works correctly with target database. Great for quickly testing scenarios and edge cases. Tests can use services to setup state in preparation of testing other services. Many use cases and edge cases can be tested easily and efficiently at this level.
Web Tests Application Server started. Tests interact through website UI JUnit, Hibernate, Spring, Selenium Test the complete application. Great for site regression and use case tests. Allows for testing across multiple WAR files.

Home State Switching

All of our tests are written with JUnit. Our custom FSJUnit test runner runs the tests concurrently taking advantage of our multi-core machines. Home State Switching allows our tests to redirect to separate database schemas ensuring the consistency of our initial state while avoiding state collisions. Essentially the test specifies what version of home state is required using our FSTestFramework. FSTestFramework uses an existing home state schema or prepares one (create, import, migrate) if a matching one does not exist. The test thread is assigned a special UUID which is then tied to the test schemas. This is implemented using our custom FSTestJDBCDriver which proxies it’s connections and knows how to redirect to alternate database schemas based on the thread’s UUID. The code under test remains 100% production as the driver takes care of the redirection. The test home state is rolled back upon completion of the test ready for the next test to take advantage of.
Below is the architecture used for Integration Testing:
Testing across multiple WARs

Using the new Home State Switching mechanism has allowed us to test across multiple WARs as long as they are deployed in the same application Server. In the diagram below we have a WAR that takes care of preparing the home states for use similar to the Integration Testing described above. The FSTestFilter is a servlet filter that is configured to intercept all http traffic checking for UUID’s coming in through the call (cookie, parameter, …). If a UUID is detected then the thread is stamped with the UUID by communicating directly with the FSTestJDBCDriver. This forces all subsequent JDBC calls to redirect to a different schema. One of our projects has 5 different deployed WAR files that can be tested together using the same home state. One WAR may change data during a test that the other WAR can detect.
Here at FS we have invested much time and effort into our frameworks to facilitate our testing. We feel that having good testing tools will help in the actual design of our applications. Another way to think about it is: Not having the tools actually influences design decisions.
cheers…
Kevin

It's only fair to share...
Share on FacebookGoogle+Tweet about this on TwitterShare on LinkedIn

Leave a Reply