If you successfully retrieve an XML document but are unable to extract any data from it, or every Check statement fails, then you are likely a victim of the XML Namespace malarkey.

Essentially you have a document that looks like this:

<tag>
  <record/>
</tag>

And //record is unable to find anything.

XML Namespace

First thing is to look at the retrieved XML document and see if it is using any namespaces.

Seeing the Response

There are a few ways to do this. You can dump the current response to the console:

RawGetURL %MyXMLDocumentURL%
SetRawResponse

DumpCurrentResponse STDOUT

Or on the command-line you can specifiy Debug.Web=/tmp/web.txt which provides a copy of all web traffic.

Also note in the first example that if you intend to work with XML documents directly you should use RawGetURL→, otherwise you'll be left to the whims of the backing agent (none of which work with namespaces at the moment).

Checking for namespace

Look through the entire XML for entries like the following:

<tag xmlns="http://some.domain.com/dump" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" >

This element has two namespaces defined, the default one being http://some.domain.com/dump and the xsi namespace being http://www.w3.org/2001/XMLSchema-instance.

What this means for you is that any element under this one actually exists in the default namespace (unless it has a specific prefix like xsi.

That means in this document:

<tag xmlns="http://some.domain.com/dump">
  <record/>
</tag>

There is no element //record, that record element is actually part of the default namespace, which in xpath is not specifiable. Indeed XPath 1 offers no easy solution whatsoever.

Getting your result

You first off have to tell our XML system what namespace you'd like to talk about. They all require prefixes -- while XML has this notion of a default, XPath does not.

To create a referencable namespace you need to add an element to the XMLNS→ map. In this case you'd like to do this:

set %XMLSNS:x% http://some.domain.com/dump
Check //x:record

You'll find that the result is now found and works as expected.

Criticism

There is no need to tell us how plainly ridiculous this system is. It was designed by a committee of people to solve a problem nobody was having.

We have tried very hard to find a way to make XPath resolve without namespaces, but the parsers we're using simply don't allow it. We're very sorry about that.

Should anybody ever build an appropriate XPath 2 parser there is another option. In XPath 2 you can specify //*:record which will find a record element in any namespace.