Config Notes: SAS Mid-Tier (Linux) IWA with Fallback

Update 26Sep2018: This post is now a few years old and naturally technology and security have progressed in that time. For more up to date information regarding delegation and, in particular, the requirement for constrained delegation when working with Windows Defender Credential Guard in Windows 10 and Windows Server 2016, please see Stuart Rogers’ very useful SAS Global Forum 2018 Paper: SAS 9.4 on Microsoft Windows: Unleashing Kerberos on Apache Hadoop.

Continuing on the theme of configuring a SAS 9.4 M2 platform on Linux to use Integrated Windows Authentication (IWA), in this post I’m going to jot down some notes on steps 12-15 – configuring the SAS mid-tier on a Linux server for IWA with fallback to form-based authentication (when IWA is not available). This includes delegation, so that IWA users of mid-tier apps like SAS Studio are able to get IWA access to a SAS Workspace Server (and avoid having to store their passwords in metadata or switch to using SAS Token Authentication).

If you’re wondering what happened to steps 1-11, I’ll try get to those earlier steps in future posts. I’m starting at step 12 because someone recently asked me a question about configuring an IWA mid-tier and so it seemed like a good idea to get this blog post done first. Of course, when actual implementing, it’s always good to start at the beginning, building up the foundations, and verifying those first steps are working well before moving on to the next steps. So these steps assume you already have a working implementation where SAS desktop applications (like SAS Management Console & SAS Enterprise Guide) are able to connect to the SAS metadata server using IWA, and also to get IWA access to an appropriately configured SAS Workspace Server.

I have found the best mid-tier related documentation resources for this type of configuration are these ones:

… and there are a few others listed in a previous blog post.

One of the reasons I’m writing this post is to get down some notes on a config that worked for me. The documents referenced above cover a variety of scenarios including plain basic web authentication with an XML file-based UserDatabaseRealm, an LDAP JNDIRealm, IWA (SPNEGO) without fallback, as well as fallback to form-based SAS authentication. Getting the right mix of settings, that didn’t conflict with each other, took me a long time to determine (my mid-tier takes about 20 minutes to restart whenever I want to test a modified config). Along the way I encountered pop-up basic web authentication dialogs when IWA should have worked, and infinite browser-refresh loops for the SAS Logon Manager when IWA was disabled in the browser and I was expecting fallback to SAS authentication. This post is about the final config that worked for me. I know I’ll be referring to this post again, and I hope it proves helpful to others too.

Here goes … Continue reading “Config Notes: SAS Mid-Tier (Linux) IWA with Fallback”

SAS Visual Analytics Guest Access with IWA Fallback

Yesterday I wrote a post about configuring a SAS® 9.4 M2 installation on Linux for Integrated Windows Authentication (IWA) with mid-tier fallback form-based authentication to handle situations where IWA was not available or was disabled. I also repeated this configuration with a SAS Visual Analytics 7.1 installation (based on SAS 9.4 M2). This means that domain users within an organisation, who can participate in IWA, can simply open a browser, navigate to SAS Visual Analytics, and be logged in automatically using their Windows login. Other users without a domain account, on a machine that is not in the domain, or who have deliberately disabled IWA in their browser, will see the familiar SAS Logon Manager login form where they can manually provide a user id and password.

One of the other reasons I built this configuration was to find out what happened with SAS Visual Analytics Guest Access in an IWA fallback configuration like this. Essentially, I wanted to find out if I could get maximum flexibility by supporting IWA users, form-based authentication users, and guest/anonymous access all at the same time.

One of the reasons I wanted to test this was a reference I remembered seeing in the SAS documentation. The Web Authentication section of the SAS 9.4 Intelligence Platform: Security Administration Guide, Second Edition, lists one of the limits of Web Authentication as “Not compatible with anonymous access”. This is also repeated in the PUBLIC Access and Anonymous Access section too.

It makes sense that anonymous access is not compatible with web authentication in a standard non-fallback configuration. If authentication is automatic and it fails then access is denied. An IWA fallback configuration is slightly different though – you have a choice whether to do web authentication or SAS authentication (e.g. IWA or non-IWA). If you choose SAS authentication then perhaps anonymous access might still be available as an option. I decided to test it out.

I ran 4 test scenarios to see how they were handled in an IWA with fallback configuration:
Continue reading “SAS Visual Analytics Guest Access with IWA Fallback”

IWA with SAS 9.4 M2 on Linux

I’ve just finished a challenging but very rewarding experience configuring a SAS 9.4 M2 platform on Linux to use Integrated Windows Authentication (IWA), for both server and mid-tiers ….. without using Quest Authentication Services.

The SAS platform has supported IWA on Linux since SAS 9.3 but until recently has only supported it when you “purchase, install, and configure an additional third-party product (Quest Authentication Services 4.0)”.

I’ve been wanted to do a SAS + Linux + IWA config for a while but had put it off because of the Quest requirement. What brought it back to the front of my mind was talking to someone recently about implementing IWA for a SAS Visual Analytics installation on Linux. They wanted to provide seamless login via IWA for most users, but also provide form-based logins for people who couldn’t use IWA.

I remembered seeing this section from the What’s New in SAS 9.4 (SAS 9.4 Intelligence Platform):

In the second maintenance release for SAS 9.4, Integrated Windows Authentication on Linux systems no longer requires the use of Quest Authentication Services. SAS can leverage the libraries that are shipped with the supported operating system or that are provided in most third-party authentication solutions.

It sounded like SAS 9.4 M2 would allow me to build such a config, without using Quest, and use the standard Linux libgssapi_krb5 package instead. At the same time I also remembered reading a great SAS Global Forum paper by Zhiyong Li on mid-tier fallback authentication: this is where you can configure the SAS mid-tier to fallback to form-based authentication in situations where IWA is not available or has been disabled (like you might do when you want to login using a different second identity). These both sounded like great challenges [ and fun :) ], so I set about confirming my understanding of these possibilities with SAS 9.4 M2 by doing both at the same time.

After a few days of research, implementation, testing and debugging, I finalized the config last night. I got quite a buzz out of some of the mind-bending troubleshooting sessions and it was a very rewarding outcome. Other than a few relatively minor issues to resolve, it is all working very well now.

If I get some time I’ll try to write up a few blog posts with more detail on the steps, issues, troubleshooting techniques and resolutions. In the meantime here’s an outline of the approach I took:
Continue reading “IWA with SAS 9.4 M2 on Linux”

SAS & IWA: Verifying Trusted for Delegation Status

Update 26Sep2018: This post is now several years old and naturally technology and security have progressed in that time. For more up to date information regarding delegation and, in particular, the requirement for constrained delegation when working with Windows Defender Credential Guard in Windows 10 and Windows Server 2016, please see Stuart Rogers’ very useful SAS Global Forum 2018 Paper: SAS 9.4 on Microsoft Windows: Unleashing Kerberos on Apache Hadoop.

I mentioned in a previous post that host machines need to be Trusted for Delegation when a SAS® software component, such as a SAS Workspace Server, needs to make outgoing connections to secondary servers when the initial incoming connection was made using Integrated Windows Authentication (IWA).

When a server needs to be Trusted for Delegation, it takes a domain administrator to change the machine account in Active Directory. I rarely have domain admin privileges when working at customer sites so I usually can’t do this for myself. :( This post describes the method I use, as a lowly domain user, to verify that a Windows server has been configured in Active Directory as Trusted for Delegation.

The screenshot below shows an example of what the domain admin might see in the Properties dialog Delegation tab for the machine account in Active Directory (via the Active Directory Users and Computers tool (under Start > All Programs > Administrative Tools).

This machine account (P1001) is not yet trusted for delegation. The domain admin would click the radio button for “Trust this computer for delegation to any service (Kerberos only)“.

Once the domain admin has advised that the change has been applied, we can test it out from the SAS platform. What happens if the test still fails? I like to double check that the server is definitely trusted for delegation before I move on to checking other things. Everyone makes mistakes from time to time, even domain admins; maybe the wrong machine account was modified (it does happen). So to avoid wasting time later on, I like to verify this pre-requisite before moving on. I could ask the domain admin to email me a screenshot of the dialog to confirm, but they’re likely very busy people, so why not do it myself? I often don’t have access to the Active Directory Users and Computers tool so I have to find another way to verify trusted for delegation. This is where the very useful AdExplorer utility helps. It’s one of the SysInternals tools available for download from Microsoft. As the name suggests it provides an explorer interface to Active Directory so you can browse the objects and attributes.

Here’s an AdExplorer screenshot showing the same machine account (P1001) from the dialog shown earlier.

I have selected the userAccessControl attribute and can see it has the value 4128. This is not good; I’ll explain why in a moment :) Essentially it means that the machine is not trusted for delegation. What I would rather see, is the next screenshot where it has the value 528416 meaning it is trusted for delegation.

So where do these magic numbers come from, and how do we know that 528416 is trusted and 4128 is not? The userAccessControl value is a bitmap value (or bit array). There are a number of possible flags that can be set in this value which are documented in the Microsoft resource How to use the UserAccountControl flags to manipulate user account properties. The main flag of interest here is TRUSTED_FOR_DELEGATION with value 0x80000 (hex) or 524288 (decimal) which is listed in the document as:

TRUSTED_FOR_DELEGATION – When this flag is set, the service account (the user or computer account) under which a service runs is trusted for Kerberos delegation. Any such service can impersonate a client requesting the service. To enable a service for Kerberos delegation, you must set this flag on the userAccountControl property of the service account.

The (trusted) decimal value 528416 (hex 0x81020) we saw above consists of TRUSTED_FOR_DELEGATION (decimal 524288 hex 0x80000) + WORKSTATION_TRUST_ACCOUNT (decimal 4096 hex 0x1000) + PASSWD_NOTREQD (decimal 32 hex 0x0020). The (untrusted) decimal value 4128 (hex 0x1020) we saw earlier only consists of WORKSTATION_TRUST_ACCOUNT (decimal hex 0x1000) + PASSWD_NOTREQD (decimal 32 hex 0x0020). It’s missing the TRUSTED_FOR_DELEGATION value. You might see other values in your environment, including other flag values, but the important thing to check is that it includes the TRUSTED_FOR_DELEGATION value.

If you know of any other ways to verify a server’s trusted for delegation status (as a normal domain user) please let me know by leaving a comment.

For more posts in this series have a look at the IWA tag.

Dynamic Logging Reconfiguration in SAS® 9.3

One of the exciting new features in SAS 9.3 (well for me anyway) is the ability to replace or modify the logging configuration on-the-fly for a running server without needing to restart it. Previously in SAS 9.2 you could dynamically change the levels for loggers on servers (via SAS Management Console Server Manager Loggers tabs) but not modify the appender configuration. Now in SAS 9.3 you can completely change the logging configuration file and apply it without restarting the server. This is exciting because it means you can do something like add a new appender for a new custom log file for the metadata server in the middle of the day with out having to restart the entire platform of dependant services. That’s really neat!

I tried it out. I made a small change to the metadata server logconfig.xml, saved it, and then waited … watching the log file. I’d thought that maybe the servers had a watchdog thread to monitor changes to the log config file (a bit like the log4j’s configureAndWatch) but after watching and waiting for a little while nothing happened. I then checked the documentation (always a good idea) and found that you actually need to prod the server to tell it to change its logging config via proc iomoperate (also new in SAS 9.3).

This is the code I used to tell the SAS Metadata Server to reload its logging configuration file (logconfig.xml) – the one I had changed.

proc iomoperate;
connect host='sas93lnx01' port=8561 user='sasadm@saspw' password='thesecretpassword';
set log cfg="/opt/ebiedieg/Lev1/SASMeta/MetadataServer/logconfig.xml";
quit;

It worked. The metadata server immediately closed it’s current log file (SASMeta_MetadataServer_2012-01-10_sas93lnx01_1266.log) and opened a new one with a _0 suffix (SASMeta_MetadataServer_2012-01-10_sas93lnx01_1266.log_0) using the modified config. I wondered why it had added the _0 suffix? Subsequent executions increase the log file suffix (_1, _2, etc.). Perhaps it was done to delineate the old and new logging config in case someone wanted to parse the logs? If anyone knows how to make it continue to log to the same log file I’d love to know.

Of course this dynamic logging reconfiguration also means you can readily switch between normal and trace level logging config files to troubleshoot a problem and then switch back again quickly – all without a restart. Something like this:

* switch to trace log configuration;
proc iomoperate;
connect host='sas93lnx01' port=8561 user='sasadm@saspw' password='thesecretpassword';
set log cfg="/opt/ebiedieg/Lev1/SASMeta/MetadataServer/logconfig.trace.xml";
quit;

* >>> REPLICATE THE PROBLEM;
* switch back to normal log configuration;
proc iomoperate;
connect host='sas93lnx01' port=8561 user='sasadm@saspw' password='thesecretpassword';
set log cfg="/opt/ebiedieg/Lev1/SASMeta/MetadataServer/logconfig.xml";
quit;

For more info check out the following SAS documentation references: