TestPlan automation is organized into a series of scripts, call test units, and something known as a LogicalRoot→ which groups several scripts into a logical test.

If these scripts encounter a problem they set the status of the test unit to failed. Usually this failed status propagates and results in the entire LogicalRoot being marked as failed as well.

This document will show some other manners to explicitly alter the status.

Checkpoint

A checkpoint is a condition which must be met in order for the script to be considered passed. That is, should any checkpoint fail the unit will be marked as failed.

The simplest form of this is simply the Checkpoint→ function:

Checkpoint numComp 10 < (response //div/number)

Should this checkpoint fail the script will normally terminate and write an entry in the output logs. Exactly what Checkpoint→ does depends on when it is called, but you can certain failures are always recorded.

In normal test execution this behaves like a hard checkpoint, though while verifying a state it behaves like an "if" checkpoint.

If in doubt, the normal Checkpoint→ function usually behaves in a reasonable fashion.

Soft Checkpoints

Instead of having the test unit immediately stop it is possible to have it keep going until the end of the script. This is useful in cases where you wish to check many things and report errors on all of them.

This can be done with a SoftCheckpoint→.

SoftCheckpoint eqString Name (response //name)
SoftCheckpoint numComp 0 == (response count(//*[@class='error']))

In any of the soft checkpoints fail the script will continue, but at the end of the script a failure propagation begins just like any checkpoint.

Soft checkpoints are only written to the logs if they fail.

Hard Checkpoints

Unlike the generic Checkpoint→ function, the HardCheckpoint→ always fails immediately and always writes an entry in the log files.

This would typically be used within verify scripts, in a place where you wish to guarantee a log file entry is written and the script stops immediately.

If Checkpoints

Sometimes you wish to have a hard checkpoint but only record a log entry if the test fails.

IfCheckpoint numComp 4 == (binOp %Value% + 1)

XPath/Response Checkpoints

A vast majority of your checkpoints will deal with the current response, such as checking for the presence of a particular element. To make this easier the functions Check→ and CheckNot→ are available.

Check //div/p[contains(string(),'hello')]

This checks that the given element exists in the document. The opposite is also available.

CheckNot //div/address

Be careful that CheckNot has some limitations when used with rather dynamic pages -- as the element may simply not exist now.

These functions aren't exactly the same as a checkpoint, but achieve the same purpose of causing the script to fail if the item isn't found. If you wish to have something that behaves exactly like a checkpoint you may simply use the check→ condition.

Checkpoint check //div/address

Explicit Pass and Fail

The checkpoint functions all take a condition to determine whether they pass or fail. There are however many cases where you wish to explicitly mark a unit as failed, or explicitly record a pass indicator.

The function Fail→ may be used to record a failure in the logs and fail the current unit.

if check //item/failed
  set %Message% as response //item/failed/@message
  Fail The request failed with the message: %Message%
end

Similarly, Pass→ may be used to record a successful entry in the log files. This is similar to a hard checkpoint which has passed.

Pass We've reached here so everything is okay!

Both of these functions are useful in cases where the log files are further processed, or used, for monitoring. They are also just helpful to identify known situations and make debugging of failures a lot easier.

Exec and Logical Root

The failure status of a test unit usually propagates and results in the entire LogicalRoot→ failing. If you wish to isolate a piece of your test the best way to do this is by using the exec→ function.

# instead of:
call MyChecks
# use
exec MyChecks

exec→ sets up a new LogicalRoot→ and therefore failures will be isolated. The script which called exec→ keeps processing as though nothing has gone wrong.