Announcing Scanner for Jupyter: Response-as-Code and Advanced Threat Hunting

We’re excited to announce the release of Scanner for Jupyter, allowing users to analyze and visualize years of logs using Jupyter notebooks via the Scanner Python SDK.

Scanner for Jupyter is particularly helpful for unlocking two use cases:

1. Response-as-Code
2. Advanced threat hunting on historical logs

Response-as-Code

For teams working on a Response-as-Code strategy to speed up investigations and response, Scanner for Jupyter can help.

Traditionally when a detection alert is triggered, DR engineers need to use internal documentation to figure out what steps to take to investigate the alert and decide how to respond. This can be a slow, manual process.

With Scanner for Jupyter, teams speed up their investigation and response with Jupyter notebooks, which serve as a kind of powerful, dynamic documentation that performs investigations and generates reports for you.

When a detection alert is triggered, users execute the alert’s corresponding Jupyter notebook, running Scanner queries to pull in context from historical logs and generate visualizations, like network graphs and charts. This automation can meaningfully speed up investigation and response from hours to minutes.

Network graph visualization of AssumeRole chains

Here are some example Response-as-Code use cases from teams using Scanner for Jupyter:

• When an IAM AssumeRole operation is executed by an unusual role, run a notebook to visualize the network graph of all IAM AssumeRole operations to look for lateral movement. This is easy to spot since there will often be an unexpected long chain.
• After a spike in download volume from an S3 bucket, run a notebook to render all download flow across S3 buckets and notice outliers by the size and color of their nodes in a network graph.
• When a particular user has a large number of failed login attempts, run a notebook to fetch all historical context about that user over the past year from multiple log sources, and render a table and chart of all of the sensitive API calls they have tried to execute, noting any successful calls.

Advanced threat hunting on historical logs

Since Scanner provides fast search on years of historical logs, our customers are using advanced analysis features in Jupyter notebooks to look for trickier kinds of threats, like APTs (advanced persistent threats).

One aspect that makes APT threat hunting difficult is that a lot of APT activity can appear legitimate:

• Credential Compromise and Privilege Escalation: In AWS, this activity looks like CreateUser or CreateAccessKey API calls, which aren’t always malicious.
• Persistence Mechanisms: This might look like IAM policy changes in AWS, which are perfectly legitimate in some cases but malicious in others.
• Lateral Movement: This might take the form of new EC2 instance creation or termination, or changes in IAM policies, which may all be benign in some cases but threatening in others.

With Scanner for Jupyter, teams can quickly import ML libraries like scikit-learn to look for anomalies in these operations.

Here is a simple example showing how to use the IsolationForest classifier from scikit-learn in a Jupyter notebook to find anomalies in log events using properties like IP address, API call, time of day, and user role identifier: AnomalyDetection.ipynb.

Scanner for Jupyter makes it easy for teams to use the ML tools from the Jupyter ecosystem to detect APTs and other threats that are hard to find.

Since Scanner queries over years of historical logs are fast, this kind of advanced persistent threat hunting is now doable.

Usage

To use Scanner for Jupyter:

• Sign up for a demo to create a Scanner account
• Configure Scanner to index your logs in S3
• Retrieve an API key to use in your Jupyter notebooks
• Set up detection rules in Scanner
• Follow our getting started guide for Jupyter notebooks

We’re excited by these use cases and others that are unlocked when you can finally retain years of historical logs and search them at high speed.

‍