Month: July 2010

Identity Hierarchy

In my upcoming SAS Forum Australia & New Zealand 2010 presentation Best Practices with SAS® 9 Metadata Security, I mention that is important for platform administrators to have an understanding of the Identity Hierarchy concept. In this post I provide a bit more information about the identity hierarchy, and associated identity precedence, than time allows in the presentation.

Learning about the Identity Hierarchy

If you want to lean more about this topic, I would suggest any or all of the following:

  • Read the SAS® 9.2 Intelligence Platform Security Administration Guide, Chapter 3 Users, Groups, and Roles, specifically the section titled Identity Precedence
  • Read the SAS® 9.1.3 Intelligence Platform Security Administration Guide, Second Edition, Chapter 2 Understanding Authorization, specifically the section titled To Whom Can Permissions Be Assigned?
  • Attend the SAS training course SAS Platform Administration: Fast Track. I think this is one of the best courses available from SAS and is an essential foundation for any platform administrator. I should point out that I do teach this course from time to time for SAS Institute Australia so I might be a little biased ;)

Visualizing the Identity Hierarchy

The identity hierarchy consists of the user, all of the groups that they are a direct member of, all of the groups that they are an indirect member of via nested group memberships, as well as the SASUSERS and PUBLIC groups that they are implicit members of. All of these identities can be arranged in a hierarchy. The following screenshot shows the identity hierarchy for a fictitious user Kate Knowles:

This screenshot was generated and extracted from the identity hierarchy tab available in our Metacoda Security Plug-ins User Reviewer. This tab provides a visualization of the identity hierarchy for any selected user in metadata. If you are wondering what the various icons mean:

  • represents a user (in this particular case a user bulk-loaded into metadata from AD or LDAP);
  • represents a normal group;
  • represents a group that has been bulk loaded into metadata (from AD or LDAP); and
  • represents a group that ultimately contains itself (via a circular reference)

At the top of the identity hierarchy (level 0), with the highest identity precedence, is Kate Knowles herself. Underneath Kate (at level 1), and with the next highest identity precedence, are the groups that Kate is a direct member of ( Northern Region HR for example). These level 1 groups are those where you will find Kate listed on the group’s Members tab. Underneath those level 1 groups are the level 2 groups (e.g. Northern Region), that have the level 1 groups as direct members (i.e. you will find the level 1 group listed on the Members tab of the level 2 group). Kate is an indirect member of the level 2 groups. This process continues until we exhaust all of the nested groups, which is at level 3 for Kate. Finally, at the deepest levels are the implicit groups: SASUSERS for everyone with a metadata identity; and PUBLIC for everyone with valid credentials.

You can see the screenshot shows two different views of the identity hierarchy: 1) the tree view shows the member relationships between the groups as well as the shortest path by which the user is a member of any given group; and 2) a flattened table view which just shows the user and the direct, indirect and implicit groups in order of identity precedence.

The identity hierarchy shown above is a simplified smaller representation of a complete identity hierarchy. It has had redundant duplicate groups that have the same or lower identity precedence removed, so that groups are only shown once at their highest level of identity precedence. To illustrate this a simpler identity hierarchy is shown below which includes a duplicate group reference. In this example, Tara Thompson is a direct member of both the Australia and the Asia/Pacific groups – these are shown at level 1. The Australia group is itself a direct member of the Asia/Pacific group too, and so the Asia/Pacific group is also a level 2 group and shown greyed out below – ordinarily we hide this lower precedence group from the identity hierarchy. You might notice that there is no need for Tara to be a direct member of the Asia/Pacific group because she already has that membership indirectly through the Australia group membership. In fact this multi-level membership has the potential to cause unwanted conflicts. I plan to post about this type of identity hierarchy issue later on.

Importance of the Identity Hierarchy

The identity hierarchy is important because it is used to determine identity precedence which plays a key role in resolving conflicts. It can also be used to find any shared logins accessible to the user, via their group memberships, by walking the tree looking for logins on any of the groups.

To quote from the SAS® 9.2 Intelligence Platform Security Administration Guide:

Identity precedence affects authorization decisions when a user has more than one relevant permission setting because of the user’s group memberships. Identity precedence affects login priority when someone has more than one login in an authentication domain. Identity precedence is not relevant for roles

In SAS 9.1.3 the identity hierarchy is only used to resolve authorization decisions and find shared logins. Its use in choosing a higher priority login is new in SAS 9.2.

A good understanding of the identity hierarchy allows you to understand and troubleshoot conflicts, and hopefully help you plan to avoid them where possible.

Potential Identity Hierarchy Issues to Avoid

Most of the time the identity hierarchy is quite straightforward, but you can also get into some odd situations. Some of the potential issues to avoid include:

  1. As shown above, in the identity hierarchy for Tara, a group can potentially appear at multiple levels in an individual’s identity hierarchy. Whilst this can be normal, care may need to be taken with access controls to avoid potentially unwanted outcomes. I plan to post an example soon which provides an example of an unwanted outcome due to an identity hierarchy conflict and “non-implicit-group permission denials”.
  2. It is possible to end up with circular references in the identity hierarchy. This occurs when a group contains itself as an indirect member through a nested group membership. The Metacoda Security Plug-ins User Reviewer detects these and tags the group with a special icon . The identity hierarchy shown earlier for Kate provides an example of this: Kate is a direct member of the Human Resources Managers group which itself is a direct member of the Human Resources group and that group in turn is a direct member of the Human Resources Managers group (it’s own parent).
  3. Whilst it’s possible to add the implicit groups SASUSERS and PUBLIC as direct members of other groups, this is not a recommended practice. It is however quite normal to add the implicit groups to roles with SAS 9.2. In another quote from the SAS® 9.2 Intelligence Platform Security Administration Guide:

    To avoid introducing unnecessary complexity, don’t make PUBLIC or SASUSERS a member of another group. For example, if you make PUBLIC a member of GroupA, then a user who is an indirect member of GroupA (through his automatic membership in PUBLIC) has GroupA as his lowest precedence membership. This contradicts the usual expectation that every user’s lowest precedence membership is PUBLIC

To be continued …

I am planning on following up this post with some additional examples and practices that build upon this discussion of the identity hierarchy, but that’s it for now. I hope this post is useful to those who may want to find out more than I can discuss in my forum presentation. Please let me know if you have any comments or questions.

Promoting Job Flows

I am occasionally asked whether it is possible to promote job flows between metadata repositories (e.g. DEV to TEST to PROD) using the SAS package file (SPK) import/export promotion feature. In past consulting work I have needed to promote job flows containing hundreds of jobs all interlinked with dependencies, so this is a topic close to my heart :)

In the early days of SAS 9.1.3 import/export of job flows was not supported and so the only choices availably back then were to either manually reconstruct the jobs flows in the target environment or do a full replication/promotion (what used to be called full promotion in SAS 9.1.3 is now called replication in SAS 9.2). Manually reconstructing jobs flows was a tedious and error-prone process especially with large and complicated job flows. Full replication/promotion was often not appropriate because it would overwrite the target environment completely. But that’s all history (and has been for a while) …

With SAS 9.2 there is no question about it – import and export of job flows is standard with SAS 9.2 so I will ignore it for the rest of this post and concentrate on SAS 9.1.3 instead.

Since the import/export of job flows was not initially supported in SAS 9.1.3, I occasionally talk to people who assume it’s still the case now (which is understandable). It’s great to see the smile spread across their faces when they find out it is actually possible with SAS 9.1.3 today – I remember back to how I felt when I first found out. I was faced with the prospect of manually reconstructing some awe-inspiring job flows and contacted SAS on the off-chance that someone might know a better way – which is how I was lucky enough to get to try out an early version. You can imagine how happy I was.

The ability to perform partial promotion of job flows crept relatively quietly into SAS 9.1.3 with a couple of hotfixes some time back. There is a usage note about it now too: Usage Note 31008: Hot Fixes 913SMC04 and 14JPS02 provide the capability to add job flows using partial promotion

So if you need to promote job flows between SAS 9.1.3 environments here are some pointers that you might find useful:

  • Promote the job flows using the SAS Management Console 9.1 BI Manager plug-in. Last time I tried I couldn’t export job flows from SAS Data Integration Studio 3.4 (even with the latest hotfixes at the time). I last looked many months ago though, so happy to be corrected if that has changed since.
  • Make sure the SAS Management Console 9.1 client installation you will be using (most likely your workstation) has been updated to at least the level of hotfix 913SMC04. I would tend to install the most recent SAS Management Console 9.1 hotfix 913SMC08 instead.
  • This is already in the instructions for 913SMC08, but you also need to ensure the client installation you will be using (most likely your workstation) has been updated to the level of SAS Foundation Services 1.4 hotfix 14JPS02. You might recognize SAS Foundation Services 1.4 from mid-tier installations, but it also contributes the BI Manager plug-in to SAS Management Console 9.1 client installations.
  • Using BI Manager in the source repository track down the folder containing the job flows you want to export and export them into a SAS package (SPK) file. If you can’t find the job flows try looking in the /Shared Data folder. Here is a screenshot showing job flows being exported from SAS Management Console 9.1:
  • Using BI Manager in the target repository import the job flows from the previously exported SAS package (SPK) file. Once imported you will need to re-schedule the flows.

MKDIRMD Macro: Creating Tree Folders in Metadata

Update 10Jul2015: After 5 years, the macro discussed in this blog post is now very dated. If you still have SAS 9.1.3 then it should be ok, but for newer versions of SAS software (such as 9.2, 9.3 and 9.4) there are better options provided as standard by SAS Institute:

1) SAS Platform Object Framework MakeFolder Metadata Utility: see my later blog post

2) SAS 9.4 has the Make Folder (sas-make-folder) command line tool documented in the SAS 9.4 Intelligence Platform: System Administration Guide, Third Edition” under “Batch Tools for Metadata Management”.

The source code for the MKDIRMD macro mentioned below has a check for SAS 9.2 at the end, that should include later versions of SAS (if used with later versions of SAS). However, I would strongly recommend using one of the newer supported methods instead (as mentioned above).

The inspiration for this post came from a question at sasprofessionals.net about code to create tree folders in a SAS® metadata repository. It sounded like a good challenge considering that:

  • Creating a Tree object in metadata is relatively straightforward, but finding the correct parent Tree object to associate it with takes more work. You can’t search by the parent folder name in isolation because there might be multiple folders with the same name at different locations in the tree (e.g. there could be 2 folders both named Reports at /ACME/Sales/Data and /ACME/HR/data). The parent folder could be specified by its object id but that’s a bit tedious. A neater method would be to allow the parent path to be specified in absolute terms (e.g. /ACME/HR/Data/) and get the code to walk down the tree to find the correct unique parent Tree object (e.g. Data under HR under ACME).
  • It should be able to create top level folders in the root of the tree as well as folders at lower depths. Top level folders are represented slightly differently in metadata than lower level folders. They don’t have a ParentTree association but are instead associated with a SoftwareComponent.
  • It should fail if a folder with the same name already exists in the parent folder. The metadata API allows you from create duplicate named folders in the same parent folder.
  • It should check all return codes and fail fast – it is being used to update metadata after all.
  • It would be good if it worked in both SAS 9.1 and SAS 9.2

I had some metadata API fun writing a MKDIRMD macro that satisfied all of the above. Here is an example of it being used:
%mkdirmd(name=ACME, parent=/);
%mkdirmd(name=Sales, parent=/ACME/);
%mkdirmd(name=StoredProcesses, parent=/ACME/Sales/);
%mkdirmd(name=Data, parent=/ACME/Sales/);
%mkdirmd(name=HR, parent=/ACME/);
%mkdirmd(name=StoredProcesses, parent=/ACME/HR/);
%mkdirmd(name=Data, parent=/ACME/HR/);

I have uploaded the MKDIRMD.SAS code I wrote so it can be used by anyone that may have a need for it, or by those who might want to review it as an example program that uses the SAS Open Metadata Interface to query and update metadata. It includes examples of XMLSelect filters on both attribute criteria and association paths. I am publishing this code under the GNU LGPL license so it can be freely used. I have tested the code in both SAS 9.1.3 SP4 and SAS 9.2 M3 but it is provided “as-is” and should be used with care.

Be aware that with any code like this, that updates your metadata, it is strongly recommended that you have a recent backup (that is known to work) and test the code in a non-production environment. I prefer testing things like this in a private administrator environment (Lev9) since even development environments should be considered valuable.

Found the SAS® 9.2 Open Metadata Interface documentation

I find myself referring to the SAS® Open Metadata Interface documentation on a very regular basis and have been keenly awaiting the availability of said documentation for SAS 9.2. I have been scanning the SAS 9.2 Product Documentation A-Z Listing every now and then hoping to see it appear one day but so far it has been absent. I just assumed it hadn’t been released yet. Today, however it occurred to me that perhaps I should try using the support.sas.com search facility instead (how come I didn’t think of that before I wondered). Expecting only hits relating to the SAS 9.1.3 docs, I was very pleasantly surprised to find SAS 9.2 Open Metadata Interface links in the results too.

In case anyone else is looking for this documentation, and to help me find it again, here are the relevant documents I found:

SAS® 9.2 Language Interfaces to Metadata
This document does appear in the A-Z index under Base SAS (and has for a while I think). It can be found here:

SAS® 9.2 Open Metadata Interface: Reference and Usage
This document is not yet in the A-Z index but can be found here (as discovered by searching):

SAS® 9.2 Metadata Model: Reference
This document is not yet in the A-Z index but can be found here (as discovered by searching):

SAS® 9.2 Java Metadata Interface (JavaDoc)
This JavaDoc can be found (and has for a while) in the SAS AppDev Studio focus area at http://support.sas.com/rnd/gendoc/bi92/api/metadata/index.html

Now, of course, I am kicking myself wondering how long all of this documentation has actually been available while I have been waiting for it to appear in the A-Z list :) Can’t dwell on that now though – I have lots of reading ahead.

BTW – does anyone know if there is any published documentation for the SAS 9.2 Management Console Plug-in API available yet?

Metadata Reporting with SAS® Software

I was reminded today of the “Metadata reporting with SAS® software” presentation I did at a SNUG meeting way back in Sep 2007. It seems such a long time ago now, but when I read through it again it is still relevant today. The presentation is mainly about the use of PROC METADATA and the SAS XML Libname Engine (SXLE). It also lists a few metadata reports that I have found useful in the past and were done using this METADATA/SXLE technique:

Sample Report: list of objects currently checked-out (locked) by SAS DI Studio users

Sample Report: reverse chronological list of all check-ins performed by SAS DI Studio users, with links to review the jobs and tables associated with each check-in



Sample Report: list of SAS DI Studio jobs that need to be re-deployed for scheduling

Sample Report: comparison of SAS DI Studio jobs from 2 repositories to identify those that need to be promoted

The presentation is available in PDF format as slides-only as well as slides-with-notes that provide a little more information. The slides show code fragments and so here are a couple of complete, albeit basic, examples that can be used as starting points for further work:

  • ProcMetadataSample1.sas: this is a basic example to extract and print user information (id, name, title and email address) for any user whose name begins with ‘Paul’ (using XMLSELECT)
  • ProcMetadataSample2.sas: extract group information only for those groups that have a user member whose user name begins with ‘SAS’ – this is an example of XMLSELECT filtering based on associations

It was around this time that I had the idea to pursue metadata reporting by creating plug-ins for the SAS Management Console using Java. This ultimately resulted in the creation of our Metacoda Security Plug-ins product used to review and report on security metadata.