A recent article discussing the recent challenges to enterprise service bus (ESB) products by XML/SOA gateway products contained a sentence that I found extremely puzzling.

I know what the author was trying to say, but this sentence really is full of epic fail. "Packet" visibility is even more irrelevant to XML than it is for HTML or any other application layer protocol, for that matter. 

The problem with putting "XML" and "packet" together is that application layer data is almost never contained within any single packet, and if you're going to interpret, act on, or manipulate the actual application messages (i.e. the XML, the HTML, the application protocol) then you have to assemble the packets into a document or message first.

"Packet" level visibility is a term used to describe network devices like routers, switches, traditional firewalls and network drivers. These types of products work on a packet-processing level; they look at individual packets, at the IP and TCP characteristics contained within the headers, and little more. Packet-processing devices aren't designed to provide "deep" visibility into application layer protocols because they aren't designed to reassemble the documents and messages.

Packet processing is to delivering applications what fingers are to an individual. Having just one means you might be able to identify the person/application, but you don't *know* anything else about either. Basically, there may be some amount of application data that is valuable in any given packet that might be of limited use, say in identification of application type for rate shaping / classification purposes. This is often what's behind the use of the term "deep packet inspection" as it relates to applications. Identification.

In the context of any network-positioned device, like an XML or SOA gateway, an application delivery controller, or an XML firewall, visibility and processing must necessarily be at the application layer. The contents of any single given packet are irrelevant and, in the case of XML, practically useless.

XML must be parsed and put into a format which can be interpreted by a machine, and that means that it must be reassembled first. While "streaming" parsers appear to do this on a per-packet basis that is not completely accurate, for it is often the case that a specific element will be nested deep enough and be large enough to span two packets, which breaks the packet-processing model completely. Streaming simply means that the XML is being interpreted as the document is being reassembled; the document is still being viewed as application data, not necessarily individual packets.

What the author was trying to convey the sense that XML & SOA gateways are capable of reassembling XML documents and processing them, providing security and routing and message enrichment functionality just like an ESB, because though they are "network" devices, they are also full-proxies.

But that's not "deep XML packet inspection", or even just "deep packet inspection". That's flow or even message-based processing, not packet processing. The terms "packet inspection" and "[insert application layer protocol here]" should never be used concurrently in the same sentence.

Unless you're trying to explain why it is that packet processing is teh fail when it comes to true visibility into and manipulation of application messages.

Imbibing: Water