Tag: Metadata API

SAS Metadata “V” Lengths

"V" Lengths I have been doing a lot of work on metadata searching lately, including basic XMLSELECT filtering, as part of our Metadata Security Plug-ins, in our effective permissions viewers, as well as the free Metadata Explorer utility. XMLSELECT is a feature in the SAS Open Metadata Interface that allows you to apply filters to metadata queries; you could think of it as a “Where Clause” for metadata.

One of the known limitations about XMLSELECT, that I’d not really paid much attention to in the past, relates to querying in potentially lengthy attributes – those with “V” lengths. This is documented in the SAS 9.3 Metadata Model: Reference document in the SAS Metadata Model Conventions section. It’s also documented in the equivalent SAS 9.2 page. Here’s the relevant quote from the documentation: Continue reading “SAS Metadata “V” Lengths”

Metacoda’s Metadata Explorer Plug-in

Update 15Jun2015: The Metadata Explorer plug-in discussed in this blog post was first made publicly available with Metacoda Plug-ins V3.0. It has been enhanced in subsequent releases and now supports additional SAS platform versions beyond those originally mentioned in this post (such as SAS 9.4). For more information on supported SAS versions, please see the Metacoda Plug-ins System Requirements page.

My work at Metacoda involves lots of metadata, as does my work as in SAS® platform administration. I spend a lot of time looking inside SAS metadata repositories, searching for objects, looking at their attributes, following their associations to other objects, and generally getting to grips with the SAS Metadata Model. I write programs that use the SAS Open Metadata Interface, primarily using the SAS Java Metadata Interface, but sometimes using the SAS Language Interface to Metadata usually via PROC METADATA.

When I originally started to explore metadata, I used the metabrowse feature in Foundation SAS. I really liked it, but I’m not always working on, or have easy access to, a machine that has Base SAS installed (and I wanted to do some more extensive searching). I do spend a lot of time inside the SAS Management Console though. I tried using the XML Metadata Interface in SAS Management Console 9.1 for a short while but found it made me think too much about how to do the query, rather than what it was I was looking for. I also found working with the raw XML results returned by the XML Metadata Interface plug-in was a bit challenging too and it distracted me from my original purpose of finding metadata. As an aside, the SAS Management Console XML Metadata Interface plug-in is no longer available by default in SAS Management Console 9.2 and 9.3, but it is still available. If you need it, you can find instructions on how to enable it in a SAS Global Forum 2012 paper.

Now, as many programmers do when they can’t quite find what they are looking for, they write their own tools, and this is what I did. “scratching an itch” is a term I often hear programmers use to describe this. My goals for this ideal metadata exploration tool, which evolved over time, were along these lines: Continue reading “Metacoda’s Metadata Explorer Plug-in”

SAS Platform Object Framework MakeFolder Metadata Utility

Back in July last year I posted MKDIRMD Macro: Creating Tree Folders in Metadata about a SAS macro I wrote to create tree folders in a SAS metadata repository using the SAS Open Metadata API. Just recently I discovered an alternative method using a utility already present in a SAS 9.2 installation: MakeFolder.

I stumbled on MakeFolder in passing whilst looking in the SASPlatformObjectFramework directory (which is /usr/local/SAS/SASPlatformObjectFramework/9.2 on my Linux machine). If you haven’t seen it before, that directory contains a number of tools such as the Batch Export and Import tools for batch migration/promotion of metadata and the ValidateServer utility mentioned in SAS Usage Note 42523: ValidateServer utility to check availability of SAS® 9.2 Business Intelligence servers. I noticed the SASPlatformObjectFramework directory contained a few other utilities too, including MakeFolder which sounded very much like it might, well, make a folder! Since there are already operating system commands to make file system folders I guessed it would probably make metadata folders and so be worth a closer look.

I couldn’t find any documentation for MakeFolder. A search on support.sas.com yielded nothing. The other SASPlatformObjectFramework utilities documented –help parameters so I thought I would try that out first. It displayed the following nice usage message: 

userid@host:~$ /usr/local/SAS/SASPlatformObjectFramework/9.2/MakeFolder --help
usage: MakeFolder [options...] folderPath
options include:
 -profile <profile>     Metadata server connection profile.  Can be used in
                        place of the -host, -port, -user, and -password options.
 -user <userID>         User login identity. Required if -profile is not set or
                        if the profile does not contain connection credentials.
 -password <password>   User login password. Required if -profile is not set or
                        if the profile does not contain connection credentials.
 -host <hostname>       Metadata server host. Required if -profile is not set.
 -port <port>           Metadata server port. Required if -profile is not set.
 -?,--help              Print help information.
 -domain <domain>       User authentication domain
 -log <log-file>        Log file or directory.
 -makeFullPath          create intermediate folders if necessary

It all looked quite straightforward, so I tried it out in a development environment. Firstly to create a simple top level folder (/testsimple):

userid@host:~$ /usr/local/SAS/SASPlatformObjectFramework/9.2/MakeFolder -host localhost -port 8563 -user adminpaul@saspw -password thesecretpassword /testsimple
INFO  *** Make Folder started on 26/05/2011 ***
INFO  Server localhost:8563, user adminpaul@saspw.
INFO  Folder /testsimple created.
MakeFolder has completed.

That worked fine and looking in the SAS Management Console Folders tab I could see the folder had been created.

There was also a -makeFullPath to create intermediate parent folders too. I first tried it out without -makeFullPath:

userid@host:~$ /usr/local/SAS/SASPlatformObjectFramework/9.2/MakeFolder -host localhost -port 8563 -user adminpaul@saspw -password thesecretpassword /test/multiple/levels
INFO  *** Make Folder started on 26/05/2011 ***
INFO  Server localhost:8563, user adminpaul@saspw.
ERROR Parent folder /test/multiple/ does not exist.
MakeFolder has completed.

As expected, that produced an error message since I didn’t already have the /test/multiple parent folders. I ran it again, this time with the -makeFullPath parameter:

userid@host:~$ /usr/local/SAS/SASPlatformObjectFramework/9.2/MakeFolder -host localhost -port 8563 -user adminpaul@saspw -password thesecretpassword -makeFullPath /test/multiple/levels
INFO  *** Make Folder started on 26/05/2011 ***
INFO  Server localhost:8563, user adminpaul@saspw.
INFO  Folder /test/ created.
INFO  Folder /test/multiple/ created.
INFO  Folder /test/multiple/levels created.
MakeFolder has completed.

The folder and its parents were all created successfully.

MakeFolder indeed looks very useful. However, since I haven’t seen any official SAS documentation for it, I would err on the conservative side and, unless I hear otherwise, assume it’s not currently supported. Perhaps if someone from SAS Institute is reading they might be able to provide a bit more info about its status.

So now I know two ways to script the creation of metadata folders.

MKDIRMD Macro: Creating Tree Folders in Metadata

Update 10Jul2015: After 5 years, the macro discussed in this blog post is now very dated. If you still have SAS 9.1.3 then it should be ok, but for newer versions of SAS software (such as 9.2, 9.3 and 9.4) there are better options provided as standard by SAS Institute:

1) SAS Platform Object Framework MakeFolder Metadata Utility: see my later blog post

2) SAS 9.4 has the Make Folder (sas-make-folder) command line tool documented in the SAS 9.4 Intelligence Platform: System Administration Guide, Third Edition” under “Batch Tools for Metadata Management”.

The source code for the MKDIRMD macro mentioned below has a check for SAS 9.2 at the end, that should include later versions of SAS (if used with later versions of SAS). However, I would strongly recommend using one of the newer supported methods instead (as mentioned above).

The inspiration for this post came from a question at sasprofessionals.net about code to create tree folders in a SAS® metadata repository. It sounded like a good challenge considering that:

  • Creating a Tree object in metadata is relatively straightforward, but finding the correct parent Tree object to associate it with takes more work. You can’t search by the parent folder name in isolation because there might be multiple folders with the same name at different locations in the tree (e.g. there could be 2 folders both named Reports at /ACME/Sales/Data and /ACME/HR/data). The parent folder could be specified by its object id but that’s a bit tedious. A neater method would be to allow the parent path to be specified in absolute terms (e.g. /ACME/HR/Data/) and get the code to walk down the tree to find the correct unique parent Tree object (e.g. Data under HR under ACME).
  • It should be able to create top level folders in the root of the tree as well as folders at lower depths. Top level folders are represented slightly differently in metadata than lower level folders. They don’t have a ParentTree association but are instead associated with a SoftwareComponent.
  • It should fail if a folder with the same name already exists in the parent folder. The metadata API allows you from create duplicate named folders in the same parent folder.
  • It should check all return codes and fail fast – it is being used to update metadata after all.
  • It would be good if it worked in both SAS 9.1 and SAS 9.2

I had some metadata API fun writing a MKDIRMD macro that satisfied all of the above. Here is an example of it being used:
%mkdirmd(name=ACME, parent=/);
%mkdirmd(name=Sales, parent=/ACME/);
%mkdirmd(name=StoredProcesses, parent=/ACME/Sales/);
%mkdirmd(name=Data, parent=/ACME/Sales/);
%mkdirmd(name=HR, parent=/ACME/);
%mkdirmd(name=StoredProcesses, parent=/ACME/HR/);
%mkdirmd(name=Data, parent=/ACME/HR/);

I have uploaded the MKDIRMD.SAS code I wrote so it can be used by anyone that may have a need for it, or by those who might want to review it as an example program that uses the SAS Open Metadata Interface to query and update metadata. It includes examples of XMLSelect filters on both attribute criteria and association paths. I am publishing this code under the GNU LGPL license so it can be freely used. I have tested the code in both SAS 9.1.3 SP4 and SAS 9.2 M3 but it is provided “as-is” and should be used with care.

Be aware that with any code like this, that updates your metadata, it is strongly recommended that you have a recent backup (that is known to work) and test the code in a non-production environment. I prefer testing things like this in a private administrator environment (Lev9) since even development environments should be considered valuable.

Found the SAS® 9.2 Open Metadata Interface documentation

I find myself referring to the SAS® Open Metadata Interface documentation on a very regular basis and have been keenly awaiting the availability of said documentation for SAS 9.2. I have been scanning the SAS 9.2 Product Documentation A-Z Listing every now and then hoping to see it appear one day but so far it has been absent. I just assumed it hadn’t been released yet. Today, however it occurred to me that perhaps I should try using the support.sas.com search facility instead (how come I didn’t think of that before I wondered). Expecting only hits relating to the SAS 9.1.3 docs, I was very pleasantly surprised to find SAS 9.2 Open Metadata Interface links in the results too.

In case anyone else is looking for this documentation, and to help me find it again, here are the relevant documents I found:

SAS® 9.2 Language Interfaces to Metadata
This document does appear in the A-Z index under Base SAS (and has for a while I think). It can be found here:

SAS® 9.2 Open Metadata Interface: Reference and Usage
This document is not yet in the A-Z index but can be found here (as discovered by searching):

SAS® 9.2 Metadata Model: Reference
This document is not yet in the A-Z index but can be found here (as discovered by searching):

SAS® 9.2 Java Metadata Interface (JavaDoc)
This JavaDoc can be found (and has for a while) in the SAS AppDev Studio focus area at http://support.sas.com/rnd/gendoc/bi92/api/metadata/index.html

Now, of course, I am kicking myself wondering how long all of this documentation has actually been available while I have been waiting for it to appear in the A-Z list :) Can’t dwell on that now though – I have lots of reading ahead.

BTW – does anyone know if there is any published documentation for the SAS 9.2 Management Console Plug-in API available yet?