Config Notes: SAS Mid-Tier (Linux) IWA with Fallback

Update 26Sep2018: This post is now a few years old and naturally technology and security have progressed in that time. For more up to date information regarding delegation and, in particular, the requirement for constrained delegation when working with Windows Defender Credential Guard in Windows 10 and Windows Server 2016, please see Stuart Rogers’ very useful SAS Global Forum 2018 Paper: SAS 9.4 on Microsoft Windows: Unleashing Kerberos on Apache Hadoop.

Continuing on the theme of configuring a SAS 9.4 M2 platform on Linux to use Integrated Windows Authentication (IWA), in this post I’m going to jot down some notes on steps 12-15 – configuring the SAS mid-tier on a Linux server for IWA with fallback to form-based authentication (when IWA is not available). This includes delegation, so that IWA users of mid-tier apps like SAS Studio are able to get IWA access to a SAS Workspace Server (and avoid having to store their passwords in metadata or switch to using SAS Token Authentication).

If you’re wondering what happened to steps 1-11, I’ll try get to those earlier steps in future posts. I’m starting at step 12 because someone recently asked me a question about configuring an IWA mid-tier and so it seemed like a good idea to get this blog post done first. Of course, when actual implementing, it’s always good to start at the beginning, building up the foundations, and verifying those first steps are working well before moving on to the next steps. So these steps assume you already have a working implementation where SAS desktop applications (like SAS Management Console & SAS Enterprise Guide) are able to connect to the SAS metadata server using IWA, and also to get IWA access to an appropriately configured SAS Workspace Server.

I have found the best mid-tier related documentation resources for this type of configuration are these ones:

• SAS Global Forum Paper SAS102-2014: An Advanced Fallback Authentication Framework for SAS® 9.4 and SAS® Visual Analytics by Zhiyong Li & Mike Roda (SAS Institute)
• The Support for Integrated Windows Authentication section in the SAS 9.4 Intelligence Platform: Middle-Tier Administration Guide, Second Edition
• The Web Authentication section in the SAS 9.4 Intelligence Platform: Middle-Tier Administration Guide, Second Edition

… and there are a few others listed in a previous blog post.

One of the reasons I’m writing this post is to get down some notes on a config that worked for me. The documents referenced above cover a variety of scenarios including plain basic web authentication with an XML file-based UserDatabaseRealm, an LDAP JNDIRealm, IWA (SPNEGO) without fallback, as well as fallback to form-based SAS authentication. Getting the right mix of settings, that didn’t conflict with each other, took me a long time to determine (my mid-tier takes about 20 minutes to restart whenever I want to test a modified config). Along the way I encountered pop-up basic web authentication dialogs when IWA should have worked, and infinite browser-refresh loops for the SAS Logon Manager when IWA was disabled in the browser and I was expecting fallback to SAS authentication. This post is about the final config that worked for me. I know I’ll be referring to this post again, and I hope it proves helpful to others too.

Here goes …

The SAS mid-tier is running on a Linux server machine named sasva71 (with a fully qualified name of sasva71.corp.example.com) in an AD domain CORP.EXAMPLE.COM. This is a small development installation and the SAS metadata server and SAS compute tier are also on the same machine. The Linux server has had Kerberos packages installed, had Samba installed and has been joined to the AD domain. SSSD/PAM/GSSAPI are being used to authenticate Linux users against AD. The metadata server accepts IWA connections, as does the SAS Object Spawner for IWA access to SAS Workspace Servers. The sasva71 Computer account in AD has been marked as Trusted for Delegation (as has the sasva71-SAS User service account being used for SAS metadata/compute tier authentication).

Web Service Account

Using Active Directory Users and Computers, I created an AD User account sasva71-HTTP as a service account for the SAS mid-tier to use. It was in the Computers container alongside the sasva71 Computer account and sasva71-SAS User service account, and had the following settings:

• General tab
  • First name: sasva71-HTTP
  • Display name: sasva71-HTTP
• Account tab
  • User logon name: sasva71-HTTP
  • Password never expires: checked
  • User cannot change password: checked
  • Do not require Kerberos preauthentication: unchecked (the SAS documentation says to check it but it’s optional – I left it unchecked)
• Delegation tab
  • Trusted this user for delegation to any service (Kerberos only)

… and a long random password.

I wanted to be able to delegate so that IWA users, authenticated at the mid-tier, can get IWA access to a SAS Workspace Server (for things like SAS Studio).

Kerberos Config File

The Linux server already had a working Kerberos config in the /etc/krb5.conf file. This was also used for the mid-tier. For reference purposes these are the relevant bits:

[libdefaults]
   default_realm = CORP.EXAMPLE.COM
   forwardable = true
# NOTE: proxiable=true breaks delegation for SAS apps/services (e.g. SAS Studio accessing a SAS Workspace Server)
#   proxiable = true
 
[realms]
   CORP.EXAMPLE.COM = {
      kdc = ad.corp.example.com
      admin_server = ad.corp.example.com
   }
 
[domain_realm]
   .corp.example.com = CORP.EXAMPLE.COM
   corp.example.com = CORP.EXAMPLE.COM
   .example.com = CORP.EXAMPLE.COM
   example.com = CORP.EXAMPLE.COM

Kerberos Keytab

I used the following (single line) command on a Windows machine to generate a keytab file which was securely moved to the sasva71 Linux server as /home/sas/sasva71-sas.keytab

ktpass -princ HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM -ptype KRB5_NT_PRINCIPAL
   -pass longrandompassword -crypto all -mapuser sasva71-HTTP@CORP.EXAMPLE.COM -mapOp set
   -out C:\sasva71-http.keytab

Verify SPNs

As specified in the SAS documentation, I ran the following on a Windows machine to verify the SPNs for the sasva71 machine:

setspn -f -q HTTP/sasva71.corp.example.com
Checking forest DC=corp,DC=metacoda,DC=com
CN=sasva71-HTTP,CN=Computers,DC=corp,DC=metacoda,DC=com
    HTTP/sasva71.corp.example.com
Existing SPN found!

… and …

setspn -l sasva71-HTTP
Registered ServicePrincipalNames for CN=sasva71-HTTP,CN=Computers,DC=corp,DC=metacoda,DC=com:
   HTTP/sasva71.corp.example.com

Test Kerberos Keytab

On the Linux server (sasva71), I made sure the keytab file was only accessible to the SAS installation account …

chown sas:sassys /home/sas/sasva71-http.keytab
chmod 600 /home/sas/sasva71-http.keytab

… inspected it’s contents:

klist -t -e -k /home/sas/sasva71-http.keytab
    Keytab name: FILE:/home/sas/sasva71-http.keytab
    KVNO Timestamp         Principal
    ---- ----------------- --------------------------------------------------------
       4 01/01/70 10:00:00 HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM (des-cbc-crc)
       4 01/01/70 10:00:00 HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM (des-cbc-md5)
       4 01/01/70 10:00:00 HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM (arcfour-hmac)
       4 01/01/70 10:00:00 HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM (aes256-cts-hmac-sha1-96)
       4 01/01/70 10:00:00 HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM (aes128-cts-hmac-sha1-96)

… and then tested it out (as the sas user):

kdestroy -A
kinit -k -t /home/sas/sasva71-http.keytab HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM
klist -ef
    Ticket cache: FILE:/tmp/krb5cc_0
    Default principal: HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM

    Valid starting     Expires            Service principal
    29/01/15 18:12:03  30/01/15 04:12:02  krbtgt/CORP.EXAMPLE.COM@CORP.EXAMPLE.COM
        renew until 30/01/15 18:12:03, Flags: FRIA
        Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96

The SAS documentation says to test it with the kinit command that is provided in the SASHOME\SASPrivateJavaRuntimeEnvironment\9.4\jre\bin directory, however on this Linux install there is no kinit in the /opt/sasva71/sashome/SASPrivateJavaRuntimeEnvironment/9.4/jre/bin directory.

SAS Web Application Server Config

I also downloaded and installed the Java Unlimited Strength Cryptography Extension (JCE) into my SAS Private JRE in the /opt/sasva71/sashome/SASPrivateJavaRuntimeEnvironment/9.4/jre/lib/security dir to have support for AES256.

setenv.sh

I added the following lines (bolded) into the /opt/sasva71/config/Lev1/Web/WebAppServer/SASServer1_1/bin/setenv.sh file:

...
JVM_OPTS="-Xmx3072m ...
JVM_OPTS="$JVM_OPTS -Djava.security.krb5.conf=/etc/krb5.conf"
#JVM_OPTS="$JVM_OPTS -Dsun.security.krb5.debug=true"
#JVM_OPTS="$JVM_OPTS -Dsun.security.jgss.debug=true"
#JVM_OPTS="$JVM_OPTS -Dsun.security.spnego.debug=true"
JAVA_OPTS="$JVM_OPTS $AGENT_PATHS ...

The java.security.krb5.conf property specifies which kerberos config file to use. The other comment lines set properties to turn on debugging output if required.

jaas.config

I added the following lines (bolded) to the end of the /opt/sasva71/config/Lev1/Web/WebAppServer/SASServer1_1/conf/jaas.config file (and added another 2 lines (bolded) the existing PFS section:

...
PFS {
...
    "idpropagation"="sspi"
    "sspisecuritypackagelist"="KERBEROS"
;
...

com.sun.security.jgss.krb5.initiate {
   com.sun.security.auth.module.Krb5LoginModule required
   doNotPrompt=true
   principal="HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM"
   useKeyTab=true
   keyTab="/home/sas/sasva71-http.keytab"
   storeKey=true
   debug=false
   ;
};

com.sun.security.jgss.krb5.accept {
   com.sun.security.auth.module.Krb5LoginModule required
   doNotPrompt=true
   principal="HTTP/sasva71.corp.example.com@CORP.EXAMPLE.COM"
   useKeyTab=true
   keyTab="/home/sas/sasva71-http.keytab"
   storeKey=true
   debug=false
   ;
};

Everything above was quite straightforward. It was the changes to server.xml, context.xml and web.xml that took me a few iterations to get working.

server.xml

I added the following line (bolded) into the /opt/sasva71/config/Lev1/Web/WebAppServer/SASServer1_1/conf/server.xml file and commented out the existing LockOutRealm/UserDatabaseRealm:

...
<Service name="Catalina">
    ...
    <Engine ...
        <Realm className="com.sas.vfabrictcsvr.realm.GSSContextEstablishedRealm" allRolesMode="authOnly" />
<!--
        <Realm className="org.apache.catalina.realm.LockOutRealm">
            <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/>
        </Realm>
-->
        <Host ...

… from memory, if I left LockOutRealm/UserDatabaseRealm uncommented I was getting lock-out errors with IWA logins. I later thought about trying to nest GSSContextEstablishedRealm inside LockOutRealm, but I already had it working by then and I wasn’t bothered about locking out failed login attempts on this isolated development environment. If you try it and it works (for IWA with fallback) please leave a comment below and let me know.

context.xml (SASLogon.xml)

I edited the /opt/sasva71/sashome/SASWebInfrastructurePlatform/9.4/Static/wars/sas.svcs.logon/META-INF/context.xml file because I wanted to preserve my changes with any rebuild/redploy, but I could have also changed the /opt/sasva71/config/Lev1/Web/WebAppServer/SASServer1_1/conf/Catalina/localhost/SASLogon.xml file as long as I was aware my changes would get wiped out on the next rebuild/redeploy.

The SasFallbackAuthenticatorValve line was added as highlighted below:

...
<Context docBase="../sas_webapps/sas.svcs.logon.war" path="/SASLogon">
    ...
    <ResourceLink ...
    <Valve className="com.sas.vfabrictcsvr.authenticator.SasFallbackAuthenticatorValve" authMethod="SPNEGO" />
</Context>

web.xml.orig (web.xml)

As above, I edited the /opt/sasva71/sashome/SASWebInfrastructurePlatform/9.4/Configurable/wars/sas.svcs.logon/WEB-INF/web.xml.orig file because I wanted to preserve my changes with any rebuild/redploy, but I could have also changed the /opt/sasva71/config/Lev1/Web/WebAppServer/SASServer1_1/sas_webapps/sas.svcs.logon.war/WEB-INF/web.xml file for temporary use.

In this file I uncommented the existing 401 <error-page> section to support fallback to form authentication. However, I found I had to leave the commented <security-constraint> / <login-config> section as commented. If I uncommented it I found that non-IWA clients (such as IE with IWA disabled) would get into browser refresh loops and never get to the SAS Login Manager form-based login page (looking at the HTTP traffic I could see the redirect request with fallback=true was getting a 401).

...
  </error-page>

  <error-page>
    <error-code<401>/error-code>
    <location>/WEB-INF/view/jsp/default/ui/401Fallback.jsp</location>
  </error-page>

...

  <!-- Enable SPNEGO authentication -->
  <!--
  <security-constraint>
...
  </security-constraint>
  <login-config>
    <auth-method>SPNEGO</auth-method>
    <realm-name>Tomcat Host Manager Application</realm-name>
  </login-config>
  -->

Rebuild/Redeploy

After these edits, I then ran SAS Deployment Manager (/opt/sasva71/sashome/SASDeploymentManager/9.4/sasdm.sh) twice: once to Rebuild Web Applications > Web Infrastructure Platform 9.4 only; and second to Deploy Web Applications > Web Infrastructure Platform 9.4 only.

Once the SAS Web Application Server was restarted I was able to get automatic login to SAS web applications (via IWA) using Internet Explorer (IE) on a Windows 7 domain member. Using the SAS Studio web application (via IWA) I was also able to run a SAS program on a SAS Workspace Server which showed delegation was working (I did find I also had to turn off proxiable in krb5.conf to get this working).

If I then went into IE options, turned off IWA, restarted IE, and visited SAS web applications (without IWA) I was presented with the SAS Login Manager form, where I could login using normal credentials, showing fallback authentication was working too.

If you’ve found this useful, or have any additional comments or tips on getting a SAS mid-tier on Linux working with IWA and fallback, please let me know by leaving a comment below.