I have been doing a lot of work on metadata searching lately, including basic XMLSELECT filtering, as part of our Metadata Security Plug-ins, in our effective permissions viewers, as well as the free Metadata Explorer utility. XMLSELECT is a feature in the SAS Open Metadata Interface that allows you to apply filters to metadata queries; you could think of it as a “Where Clause” for metadata.
One of the known limitations about XMLSELECT, that I’d not really paid much attention to in the past, relates to querying in potentially lengthy attributes – those with “V” lengths. This is documented in the SAS 9.3 Metadata Model: Reference document in the SAS Metadata Model Conventions section. It’s also documented in the equivalent SAS 9.2 page. Here’s the relevant quote from the documentation:
“V” in Length Attribute
Attributes that have no practical length limitation are represented with a “V” in the Length attribute, for example, “V64”. The “V” indicates the property is variable length (arbitrarily large). The documented length (64) is the maximum length of the string that can be stored before an overflow algorithm is invoked. Storing a string that exceeds the documented length causes one or more TextPage objects and corresponding associations that connect them to the original object to be created to store the string. Each TextPage object holds an additional 1,000 characters of text.
Use of the overflow algorithm has performance overhead associated with it. XMLSELECT processing also will not search overflow text in attempts to qualify an object for selection.
To get an idea of which objects and attributes this applies to, I went through the documentation for all of the metadata model types in SAS 9.2 and SAS 9.3 and extracted this table of attributes that have “V” lengths:
| SAS Version | Model Type | Attribute | “V” Length | Has Subtypes? |
|---|---|---|---|---|
| 9.2 & 9.3 | AttributeProperty | DefaultValue | V1024 | No |
| 9.2 & 9.3 | CalculatedMeasure | Expression | V1000 | No |
| 9.2 & 9.3 | ConditionalPrecedence | Condition | V1000 | No |
| 9.2 & 9.3 | Cube | CubeName | V1024 | No |
| 9.2 & 9.3 | DeployedComponent | InstallationLocation | V256 | Yes |
| 9.2 & 9.3 | Document | URI | V256 | No |
| 9.2 & 9.3 | GlobalFormula | Expression | V1000 | Yes |
| 9.2 & 9.3 | PermissionCondition | Text | V1000 | No |
| 9.2 & 9.3 | Prompt | PromptInfo | V512 | No |
| 9.2 & 9.3 | PromptGroup | GroupInfo | V4096 | No |
| 9.2 & 9.3 | Property | DefaultValue | V1024 | No |
| 9.3 | Search | Details | V256 | No |
| 9.2 & 9.3 | SXLEMap | ValidationPath | V64 | No |
| 9.2 & 9.3 | TextStore | StoredText | V1000 | No |
| 9.2 & 9.3 | XPath | LocationPath | V64 | No |
These are just the super types that have documented “V” lengths. The attributes will, of course, also be found in any subtypes (which is why I’ve indicated them in the table above).
XMLSELECT processing with “V” lengths was also documented for SAS 9.1.3. I haven’t included any SAS 9.1.3 type/attribute info in the table above as yet, mainly because our new V3 plug-ins are supported for use with SAS 9.3 and 9.2 only (our older V1 plug-ins are used with SAS 9.1.3).
So if you’re planning on searching any attributes with “V” lengths you might want to consider that those attribute values could span multiple TextPage objects and XMLSELECT may not be appropriate in that instance. The word/phrase you are looking for might not be completely in the first TextPage object. It might be beyond the first TextPage, or even be split over the first and second TextPage objects. If this is likely then you will probably want to extract the metadata objects, filtering as much as you can on non-“V”-length attributes, and then post-process/filter the results in the client (SAS, Java, .NET etc).
We were also investigating these “V” length attributes as part of the development effort for a new product we are working on at Metacoda for fast metadata search and collaboration for both business & technical users alike. Watch this space for more info, or alternatively if you’re going to SAS Global Forum 2013 in San Francisco next year then visit us at the Metacoda stand to see it in action.