Embellished Exception Chaining

In the final days of development for QLM Sourcing 2.0, we created a testing framework that wrapped around our existing integration test infrastructure. Here’s an example of what our newer web integration tests look like:

public void testCreateNewRfqWithDisabledSelectableTemplate() throws Exception {

        QLMSourcingWebTestActionSequence actionSequence = new QLMSourcingWebTestActionSequence(this);

        actionSequence.setTestDataValue("templateDescription", "testCreateNewRfqWithDisabled");
        actionSequence.setTestDataValue("templateEnabled", Boolean.FALSE);
        actionSequence.setTestDataValue("templateSelectable", Boolean.TRUE);
        actionSequence.setTestDataValue("templateReportDatabaseTableNamePrefix", "tableNamePrefix");
        actionSequence.setTestDataValue("templateAdobeFormName", "formName");

        File tempTemplateUploadFile = File.createTempFile("testCreateNewRfqWithDisabledSelectableTemplate", null);
        actionSequence.setTestDataValue("templateFormTemplateUploadFilename", tempTemplateUploadFile.getCanonicalPath());

        actionSequence.actions(new LoginPageWalk());
        actionSequence.actions(new SystemAdministrationWalk());
        actionSequence.actions(new TemplateListWalk());
        actionSequence.actions(new CreateTemplateWalk());
        actionSequence.actions(new BackToInboxWalk());
        actionSequence.actions(new RfqCreateWalk()).expects(new SelectionBoxDoesNotContainOptionExpectation("rfqTemplate", "testCreateNewRfqWithDisabled"));
        actionSequence.actions(new LogoutWalk());

        actionSequence.assertSequence();
    }

In this type of test, the central or main construct is that of an action. An action is the combination of a page walk and an expectation. A page walk is some unit of work that is executed during a test and an expectation is the expected outcome of executing that walk. For example, there is an action in the above snippet that consists of an RfqCreateWalk with an expectation SelectionBoxDoesNotContainOptionExpectation. When this action gets executed, the walk is performed which in this case navigates its way through a page and performs the necessary work to create an actual RFQ (filling the appropriate form, clicking the submit button, etc). Once this has happened, the expectation is checked which in this case is an assertion that a selection box does not contain a particular option.

A test can consist of many of these actions, all of which are tied together by a construct called QLMSourcingWebTestActionSequence. Once all the actions have been added to the sequence with their expectations (or none, it isn’t neccesary to expectations), the sequence is run by calling assertSequence(). The framework then iterates through all the actions executing their walks and expectations.

There are some benefits to this approach. One of the interesting things is that our web integration tests end up reading and looking much more like requirements (thanks to the JMock style approach syntax). We also end up reusing walks and expectations for different tests meaning we keep things centralized and easy to maintain/change when they need to be. Essentially, we’ve “object-oriented” our tests (or at least attempted to).

One of the rather annoying deficiencies that existed (up until a few days ago) was that of a misleading stack trace during a failure or error. The trace would look something like this (not related to the previous snippet of code):

com.qstrat.qlmutilities.exception.QLMNavigationRuntimeException: Expected QLM Page Class is :com.qstrat.qlmsourcing.pages.secure.admin.RolePermissionsList Found:com.qstrat.qlmsourcing.pages.AccessDenied
	at com.qstrat.webutilities.webtest.utils.BasePageHarness.assertCurrentQLMPageClass(BasePageHarness.java:213)
	at com.qstrat.webutilities.webtest.utils.BasePageHarness.clickOnLinkWithId(BasePageHarness.java:236)
	at com.qstrat.qlmsourcing.webtest.pageharness.secure.admin.SystemAdministrationHarness.roleACLAdministration(SystemAdministrationHarness.java:150)
	at com.qstrat.qlmsourcing.webtest.pagewalk.RolePermissionsWalk.executeWalk(RolePermissionsWalk.java:11)
	at com.qstrat.qlmsourcing.webtest.pagewalk.PageWalk.execute(PageWalk.java:35)
	at com.qstrat.qlmsourcing.webtest.pagewalk.QLMSourcingWebTestAction.executePageWalk(QLMSourcingWebTestAction.java:51)
	at com.qstrat.qlmsourcing.webtest.pagewalk.QLMSourcingWebTestActionSequence.assertSequence(QLMSourcingWebTestActionSequence.java:34)
	at com.qstrat.qlmsourcing.webtest.TestPermissionsEnforced.testAdminPermissionsEnforced(TestPermissionsEnforced.java:207)
	?

The key to this trace is in the last line referring to line 207 in TestPermissionsEnforced. The problem is we can’t pinpoint exactly where this error occurred. If we look at the first snippet of code, the last line of code is actionSequence.assertSequence(). If you notice, the trace above is referring to this method call. Since the framework simply iterates through the actions in the assertSequence method and executes them one by one (after they have all been registered via the actions() method call), we have no sense as to which action in the test code the error/failure corresponds to.

One way to remedy this problem is to use exception chaining in a slightly embellished way. Up until a few days ago the assertSequence method looked like this:

public void assertSequence() throws Exception {
		for (Iterator iter = qlmSourcingWebTestActions.iterator(); iter.hasNext();) {
			QLMSourcingWebTestAction qlmSourcingWebTestAction = (QLMSourcingWebTestAction) iter.next();
			qlmSourcingWebTestAction.executePageWalk(this);
			qlmSourcingWebTestAction.checkExpectations(this);
		}
	}

But was then changed to:

public void assertSequence() throws QLMSourcingWebTestActionException {
		for (Iterator iter = qlmSourcingWebTestActions.iterator(); iter.hasNext();) {
			QLMSourcingWebTestAction qlmSourcingWebTestAction = (QLMSourcingWebTestAction) iter.next();
            try {
                qlmSourcingWebTestAction.executePageWalk(this);
			    qlmSourcingWebTestAction.checkExpectations(this);
            } catch(Throwable t) {
                  QLMSourcingWebTestActionException traceException = qlmSourcingWebTestAction.getTraceException();
                  traceException.setStackTrace(getFilteredStackTraceElements(traceException));
                  traceException.initCause(t);
                  throw traceException;
            }
        }
	}

There is an added catch clause that catches all possible throwables. JUnit throws errors that are subclasses of Error and we want to handle those as well. In this catch, we grab a hold of the trace exception belonging to the action that was just run. This trace exception was originally created upon the registering of the action. Recall the actions() method used to register/add actions to the sequence:

actionSequence.actions(new RfqCreateWalk()).expects(new SelectionBoxDoesNotContainOptionExpectation("rfqTemplate", "testCreateNewRfqWithDisabled"));

The code for this method was changed to:

public QLMSourcingWebTestAction actions(PageWalk pageWalk) {
		QLMSourcingWebTestAction qlmSourcingWebTestAction = new QLMSourcingWebTestAction(pageWalk);
        qlmSourcingWebTestAction.setTraceException(new QLMSourcingWebTestActionException());
		qlmSourcingWebTestActions.add(qlmSourcingWebTestAction);
		return qlmSourcingWebTestAction;
	}

Essentially, by creating a new instance of our QLMSourcingWebTestActionException here we are obtaining a snapshot of the stack right after the action has been registered in the test method. This enables us to hold on to this trace for as long as we like; right up to the point where a throwable gets thrown during the execution of the action. Once this happens we pop off the first element in the stack trace (we don’t want to see the line in which the exception was actually created, but everything before it), then set the cause of our QLMSourcingWebTestActionException to the throwable we just caught during the execution of the action.

And we now get a stack trace that looks like:

com.qstrat.qlmsourcing.webtest.pagewalk.QLMSourcingWebTestActionException
	at com.qstrat.qlmsourcing.webtest.TestPermissionsEnforced.testAdminPermissionsEnforced(TestPermissionsEnforced.java:150)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at junit.framework.TestCase.runTest(TestCase.java:154)
	at com.qstrat.qlmsourcing.webtest.QLMSourcingWebTestCase.runTest(QLMSourcingWebTestCase.java:114)
	at junit.framework.TestCase.runBare(TestCase.java:127)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:118)
	at junit.framework.TestSuite.runTest(TestSuite.java:208)
	at junit.framework.TestSuite.run(TestSuite.java:203)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:478)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:344)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)
Caused by: com.qstrat.qlmutilities.exception.QLMNavigationRuntimeException: Expected QLM Page Class is :com.qstrat.qlmsourcing.pages.secure.admin.EditUserAclOverrides Found:com.qstrat.qlmsourcing.pages.AccessDenied
	at com.qstrat.webutilities.webtest.utils.BasePageHarness.assertCurrentQLMPageClass(BasePageHarness.java:213)
	at com.qstrat.webutilities.webtest.utils.BasePageHarness.clickOnLinkWithId(BasePageHarness.java:236)
	at com.qstrat.qlmsourcing.webtest.pageharness.secure.admin.UserListHarness.editUserAcl(UserListHarness.java:39)
	at com.qstrat.qlmsourcing.webtest.pagewalk.EditUserAclOverridesWalk.executeWalk(EditUserAclOverridesWalk.java:11)
	at com.qstrat.qlmsourcing.webtest.pagewalk.PageWalk.execute(PageWalk.java:35)
	at com.qstrat.qlmsourcing.webtest.pagewalk.QLMSourcingWebTestAction.executePageWalk(QLMSourcingWebTestAction.java:51)
	at com.qstrat.qlmsourcing.webtest.pagewalk.QLMSourcingWebTestActionSequence.assertSequence(QLMSourcingWebTestActionSequence.java:35)
	at com.qstrat.qlmsourcing.webtest.TestPermissionsEnforced.testAdminPermissionsEnforced(TestPermissionsEnforced.java:207)
	... 16 more

Pinpointing which registered action in the test method was the culprit.

It's only fair to share...
Share on FacebookGoogle+Tweet about this on TwitterShare on LinkedIn

Leave a Reply