The latest release of Metacoda Plug-ins, version 6.2, is now available and includes a new Activity Reviewer plug-in which can be used to review SAS® 9 platform log records from within SAS Management Console for the purposes of audit.
This blog post provides an overview of how it came to be, how it works, and what you can do with it.
I think the most requested queries we get about Metacoda Plug-ins are can it answer the audit type questions of “What does this person have access to?” and “Who has access to this object?”. Our Permissions Explorers can be used to answer those questions because the answers can be found in SAS 9 metadata. I think the next most common questions are “What has this person done?” and “What has been done to this object?”. These questions cannot be answered from metadata, you have to delve into the SAS server logs instead.
Our plug-ins have always been focused on answering current state questions from SAS metadata and so we have not been able to answer those history questions so far. We would always recommended people look at the Audit Performance Measurement (APM) part of the SAS Environment Manager Service Architecture Framework. However, we have never felt that was the very best answer because, rather than sending people off to another interface, it made more sense to be able to show them the answer inside SAS Management Console when they were already looking at the the person or object in question. Who wants to go somewhere else and start the query process again from scratch? For that primary reason, and a few other secondary ones, we decided to revisit this and build the Activity Reviewer plug-in.
- We wanted to be able to tightly integrate log queries within other existing Metacoda Plug-ins including:
- Being able to see user activity for a selected user in the User Reviewer
- Being able to see activity for a selected object in all of the other reviewers
- We wanted to be able to show log queries for the user very quickly, within seconds
- We wanted to be able to show very recent log entries i.e. within the last few minutes or seconds and not have to wait for an overnight batch job
- We wanted to help those customers who have not enabled SAS Environment Manager APM or have chosen not to do so for various reasons
- We wanted the ability to augment the log data with additional information as it was collected
Rather than start from scratch, it made sense to implement these features by using existing open source software that excels at collecting and making log records available. Additionally, people may already be using this software in their organizations and have existing skills for installing and maintaining it. The open source software we chose to integrate with was OpenSearch, for the purposes of storing log data and making it available for flexible queries via a REST API, and Fluentd (or td-agent), for watching log files, parsing and filtering log data and sending it to OpenSearch for storage and query. We provide some plug-in software for Fluentd to make it easier to watch and process SAS log files. Initially, this is only for SAS Metadata Server and SAS Object Spawner log files. The Activity Reviewer plug-in then targets OpenSearch with REST API queries to show the log records of interest.
This diagram shows a high level overview of how it all fits together. The blue components show additional 3rd party software components that needs to be installed. The orange items represent software plug-ins provided by Metacoda. The red arrows show the flow of log records from SAS logs into OpenSearch. The blue arrows represent the query of OpenSearch log records by the Metacoda Activity Reviewer plug-in.
Once everything has been setup, SAS logs are being watched, and the log records are being sent to OpenSearch, the Activity Review plug-in can be used to query those logs.
From the main interface, you can search for a SAS object and see logs records that relate to activity on that object. Alternatively, you could search for a SAS user and see logs records that relate to activity by that user. You can combine them and view log activity for a specific user on a specific object. There is also a search field where you can refine the results using OpenSearch query syntax. If you prefer you can also skip the user and object selection and just use the search field on its own.
The results can be exported into CSV or HTML format.
We have also integrated the Activity Reviewer into our other plug-ins. If you right mouse click on an object in another reviewer, such as the ACT Reviewer, you can choose to view log activity related to that specific object.
If you right mouse click on a user in the User Reviewer, you can choose to view log records related to activity by that specific user.
The Activity Reviewer plug-in is available to those Metacoda customers who license the enterprise package, where support for the initial setup and ongoing usage is included.
The static screenshots in this blog post only go some way to show the versatility and integration of the Activity Reviewer plug-in, so if you would like to see a live demo and ask some questions then please sign up for the SAS Ask the Expert webinar, on Tuesday 24 Oct 2023 at 4pm ET, where I will be presenting “How Do You Review Activity In SAS® Management Console?” You can register at https://www.sas.com/en_us/webinars/review-activity-sas-management-console.html
To test this plug-in, and any of the others with your own SAS 9 platform installation, you can register for a free 30 day evaluation.
Hi Paul,
The Activity Reviewer looks terrific ! It really fills a gap, since real time observability is lacking with a SAS 9 platform, as well as in-depth Metadata log auditing. SAS EV APM reports are too static and based on nightly extractions, overdue for monitoring. The linking feature with other plugins is extremely useful, this way tracking a mere user or in some cases, tracking ACT/ACE modifications with potentially large albeit unnoticed security impact becomes possible. This significantly enhances SLAs levels for a SAS 9 platform.
Cheers
Ronan
Hi Ronan,
Thanks for that feedback, it’s really good to hear.
We still have some way to go with this plug-in. If it proves to be popular then we would like to add some log augmentation too. Many of the default log messages in the metadata server logs omit some useful audit information. As you know, the SAS APM setup changes the logging configuration to log additional messages that are useful for audit. We would like to add some other things too. For example when objects are added, renamed, and deleted only the name and object id are present. We would like to show the metadata folder path for the object too. We can’t change the log message content in the metadata server log itself, but we are looking at adding a watcher process that looks for these messages appearing in OpenSearch, does a lookup, and writes the additional info as a log record with the same timestamp into another log file. That log file is also tailed and sent to OpenSearch. That way a query from the Activity Reviewer plug-in will show those augmented log records mingled in with the original log records to provide additional information to the admin.
Regarding your well-said point about “unnoticed security impact”, for those that might want more immediate alerts when access controls are changed, we usually recommend setting up regularly scheduled automated metadata security tests using our Metadata Security Testing Framework. That way if someone makes an unexpected/unwanted change to access controls an email alert will go out to interested parties so they can follow up promptly.
I also expect that subsequent questions to the ones mentioned in the post above might include “What WAS the change this person made?” and “What WAS the change made to this object?” We don’t have concrete ways to answer those questions as it would required diff-ing before and after metadata (and possibly more). One thought I have for a partial solution also involves the Metadata Security Testing Framework. As well as automated testing, it supports automated export of tests, where those tests represent some of the before-state for security metadata. For example, if you automate the export of ACTs and Protected Object tests every hour, and at a later time detect a change was made (either through automated testing or inspecting the logs), you can run that previously exported test. Any test failures will represent changes of security metadata between now and when the test was exported. If you want to swap “now” for some other point in time you could use a diff tool to look at changes in the 2 closest Metadata Security Test XML export files.
Once again thanks for your comments, they are always thoughtful and very much appreciated.
Cheers
Paul
Hi Paul,
You are more than welcome ! The Testing Framework is indeed a great source of information for revealing differences, changes between periods of time. I used to enable a second log file issued from the Metadata server
to single out the transactions dealing with security objects – Persons, Logins, IdentityGroups, ACT, ACE : creations, deletions, modifications, timestamp, acting user. I have never automated the parsing/extractions of the corresponding log files, this was available as a kind as Metadata security audit trail easy to manipulate. Your approach generalizes this to any kind of object or changes, enabling a full and comprehensive MD audit trail, with the integrated search pane into the SMC as a bonus :-).
Cheers
Ronan
Hi Ronan,
In that case you may also be interested in some upcoming blog posts too. The Activity Reviewer can work with the default logging config of the SAS 9 Metadata Server, but you can also make some changes to log some additional things related to access control changes (as you would have done). I plan to write some blog posts about how to find additional loggers including some that provide more information on the application and removal of ACTs on objects.
Cheers
Paul