Latest Blog Posts

The convergence of AI agents, data lakes, and fast query engines is transforming how teams approach detection engineering and incident response. Rather than manually writing and tuning detection rules, or spending hours investigating alerts, AI agents with access to your data lake can accelerate both the development and operational sides of security detection.

Scanner CEO and Co-founder Cliff Crosland joins Cloud Security Podcast host Ashish Rajan for an in-depth conversation about the Security Data Lake journey, overcoming typical challenges, and preparing for the future.

Today we're announcing Scanner's Model Context Protocol (MCP) server: a remote MCP server that connects AI agents directly to your security data lake. This unlocks a new generation of AI-driven security operations: interactive investigations, AI-assisted detection engineering, and autonomous response workflows.

When building security detection systems on top of data lakes, you have three fundamentally different approaches to choose from. Each offers different trade-offs between complexity, efficiency, and detection capability. Understanding these three methods—scheduled query detections, streaming event-based detections, and streaming query detections—is critical for building effective detection engineering programs at scale.

Dave Bittner (Host, CyberWire Daily) and Cliff Crosland (CEO, Scanner) discuss why Security Data Lakes are becoming the foundation for AI agents operating in the SOC. To be truly effective, AI models need as much context as possible.

In the first part of this blog series we discussed why security data lakes are on the rise to overcome SIEM log storage limits and sky rocketing costs, the various options and tools available, and overcoming the first challenge of ingesting all your logs. Once all of your log sources are connected, the next challenge is to make the data queryable in a highly performant manner and run detections on top of it. This is where things get really interesting (often painful) for most organizations.

Security operations are undergoing a transformation. As log volumes explode and attack surfaces multiply, organizations are discovering that legacy SIEMs - once the centerpiece of enterprise detection and response - can no longer keep up. The answer? The security data lake: a modern architecture that combines low-cost object storage, flexible compute, and open schema to deliver scale, speed, and control.

This is a interview with Steven Wu, CTO of Scanner on filtera.io It was awesome to hear all of the nitty gritty details about how they've architected and optimized Scanner for super fast log search! We also talked about Scanner as a company and the exciting inflection point where the company finds itself. Read the full interview here.

Recently we conducted a deep dive of experiments into integrating Artificial Intelligence and Large Language Models (LLMs) into the Security Operations Center (SOC). Our goal was to see if AI could genuinely streamline security operations without inadvertently sidelining the invaluable human element.

In this episode of the CISO Series, Cliff Crosland, co-founder and CEO of Scanner, explains how their data lake approach can reduce SIEM costs by 80-90% while giving organizations full custody of their data in their own cloud storage. Joining him are Nick Espinosa, host of the Deep Dive Radio Show, and Howard Holton, COO and industry analyst at GigaOm. The conversation explores critical questions around data retention policies, the fundamental challenge of managing growing log volumes over time, and how AI copilots are bridging the gap between security analysts and software engineers in detection workflows.

We’re heading to Blue Team Con for the first time, and we are looking forward to connecting with people who deal with search and detection daily - the CISOs and Security Leaders. There are plenty of ways to meet up with us: book a one-on-one, at networking events, or catch Cliff’s session on Saturday.

We're heading to Black Hat with a new feature – Scanner Collect. We've built a data lake search indexing engine designed for raw, messy logs – whether you're working with tens of terabytes or petabytes of data. Look up anything: IPs, file hashes, command-line flags, or the weird string that just feels suspicious. And do...

Legacy SIEM platforms were designed to bring order to chaos. But today, many teams are actively searching for SIEM alternatives as data volumes grow and pricing models break under pressure. Security teams are increasingly finding themselves forced to drop critical log sources and limit retention windows, not because it’s smart, but because the pricing model leaves them no choice.

Security teams at modern tech companies are expected to move fast and most importantly stay secure while doing it. As infrastructure scales and complexity increases, alert volume follows suit. It’s not just about alert volume, it’s about how efficiently teams can cut through the noise and act on what matters.

This knowledge base article provides a step-by-step guide to monitoring Okta System Logs for threat detection. Learn how to collect logs via API, enrich them using Substation, and analyze them with Scanner for real-time detections.

AI is no longer just a buzzword in cybersecurity, it’s becoming a practical, powerful tool for making detection faster, more scalable, and (dare we say) a little less painful. Security teams are using AI to accelerate log analysis, draft detection rules, and lighten the cognitive load that comes with sifting through mountains of alerts. While we’re not handing over the SOC keys to an algorithm anytime soon, AI is already proving itself as a valuable tool for security engineers.

The old-school SIEM model is bloated, pricey, and capped at low volumes like 50GB/day and it is falling behind. Tools like Scanner.dev are stepping in, managing volumes like 1TB/day with ease and speed. Rather than offering everything in one package, Scanner pairs with tools like Torq, which delivers powerful SOC automation and case management. Replacing bloated legacy SIEMs for a mix of elite tools will sharpen efficiency and cut costs. Another benefit is that it will provide teams fast detection, budget-friendly log search, powerful automation, and easy case management, all of which engineers and CISOs both value. The old-school SIEM model is bloated, pricey, and capped at low volumes like 50GB/day and it is falling behind. Tools like Scanner.dev are stepping in, managing volumes like 1TB/day with ease and speed. Rather than offering everything in one package, Scanner pairs with tools like Torq, which delivers powerful SOC automation and case management. Replacing bloated legacy SIEMs for a mix of elite tools will sharpen efficiency and cut costs. Another benefit is that it will provide teams fast detection, budget-friendly log search, powerful automation, and easy case management, all of which engineers and CISOs both value.

In this CyberWire Daily Podcast episode with host Dave Bittner, Cliff Crosland, CEO and co-founder of Scanner.dev , discusses the evolution and benefits of Security Data Lakes, explaining how they provide a more flexible and cost-effective way to store large amounts of diverse security data compared to traditional data warehouses and SIEM tools. He emphasizes the advantages of the “bring your own data model” approach, where organizations maintain their data in their own cloud storage while allowing various vendor tools to analyze it, reducing vendor lock-in and enabling multiple teams to access and analyze the same data without duplication.

We’ve released two features to improve log search functionality and detection rule organization.

We’re excited to announce a major expansion of Scanner’s detection rule capabilities with ready-to-use rules across 12 critical log sources. This release brings our total to 214 detection rules, covering 11 MITRE ATT&CK tactics and 45 techniques, providing extensive security monitoring across your technology stack.

This article explores the key benefits of security data lakes, including advanced use cases for threat hunting, streamlined detection and response workflows, and their role in GenAI-powered analysis. It also addresses the challenges of managing such large-scale data environments and offers solutions for optimizing performance

We’re excited to announce that the Scanner Playground environment is now available for users to explore. The Playground is a hands-on, interactive way to experience Scanner, walking users through a threat investigation scenario involving cloud audit logs—specifically AWS CloudTrail logs. You can sign up now and start exploring at scanner.dev/demo.

In Episode 14 of the Unapologetically Technical Podcast Scanner CEO, Cliff Crosland, sits down with host Jesse Anderson to discuss his early experiences with distributed systems, including his work on creating graphs and entity resolution.

In E47 of Not Another Demo, Scanner CEO, Cliff Crosland got to sit down with host Matt Nelson to discuss Scanner’s modern SIEM technology that can be 1/10th the cost of traditional SIEM technology all while being much faster and efficient. Matt and Cliff spoke through some of the technical details of Scanner, how they have set themselves apart and where their company goes from here with their technology.

Our CEO and Co-founder, Cliff Crosland joined Cyber Security Tribe’s Dorene Rettas for a discussion on the Future of SIEM and advocates for a new design suited for the cloud: keeping a search index in cloud storage alongside your data lake.

Scanner.dev CEO Cliff Crosland joins Smashing Security Podcast hosts, Graham Cluley and Carole Theriault, to discuss how Scanner transforms raw log data into searchable insights, helping organizations handle security events more effectively. Cliff explains the challenges of traditional logging tools, the high cost of log retention, and how Scanner leverages Data Lakes to make log analysis faster, cheaper, and more efficient. The conversation also touched on the future role of AI in organizing and interpreting log data, making security monitoring more accessible and actionable.

Cliff Crosland joins Danielle Heberling and the Believe in Serverless community for a discussion about innovative ways Scanner uses AWS Lambda functions to perform fast full-text searches over large volumes of log data stored in S3.

We’re excited to announce the release of Detection Rules as Code, allowing Scanner users to manage their detection rules directly in their own GitHub repositories. This enables stronger collaboration, change management, and continuous delivery of detections.

We’re excited to announce the release of Scanner for Jupyter, allowing users to analyze and visualize years of logs using Jupyter notebooks via the Scanner Python SDK.

We’re excited to announce the release of our custom Splunk app, Scanner for Splunk, which makes it easy for users to leverage logs in S3 for advanced threat hunting and detection – all while staying entirely within the Splunk UI.

The amount of cryptocurrency stolen or mined via server hijacking annually is pretty staggering. It’s estimated that almost 1/20th of North Korea’s economy each year is the illegal acquisition of cryptocurrency, which is accomplished either by extracting crypto wallet private keys or running crypto miner malware on vulnerable servers.

Scanner CEO and Co-founder, Cliff Crosland, joins the Data Engineering Podcast host, Tobias Macey, for a conversation about how Scanner’s fast log search and threat detections API for your S3 makes the discovery and exploration of security threats easier, faster, and much cheaper.

A month ago, we started playing with a fun data set to push the limits of our product some more. We decided to dial it up to 11 and indexed a data set of 100 billion synthetic AWS CloudTrail log events with a cumulative size of 250TB.

We’re excited to announce the release of new statistical aggregation functions in Scanner’s query language, which helps you explore your logs in powerful ways.

A few days ago, Okta announced a breach of their support systems, which may have caused cookies and session tokens to be compromised. To help their users detect activity from attackers, they published a list of threat indicators, specifically IP addresses and user agents, that may be connected with activity from these attackers.

Scanner CEO and Co-founder, Cliff Crosland, joins Ready, Set, Cloud! Podcast host, Allen Helton, for a conversation about how and why we built Scanner’s security data lake, Rust, serverless Lambda functions, and goats.

At our previous startup, our application and security logs experienced a rapid increase in volume – and so did our log management bill.

We’re excited to announce the release of a few new features our customers have been asking for.

As almost all security teams will tell you, managing logs can be quite expensive, with common tools like Splunk and Datadog frequently becoming a top five budget item for the team. To reduce costs, teams sometimes move their logs into a data lake built on top of cheap object storage, like S3, and they use tools like Amazon Athena to query them. However, the data lake user experience is usually fairly slow and cumbersome: queries over large time ranges can take a few dozen minutes. We believe that data lakes can be extremely fast if indexed effectively. This is why we built Scanner.

Scanner CEO and Co-Founder Cliff Crosland had the pleasure of sitting down with Rustacean Station Podcast host Allen Wyma to discuss Scanner’s Next-gen Security Data Lake tool powered by Rust.

While building Scanner’s security data lake product, we’ve had to think hard about how to push hardware to its fundamental limits to handle the scale of our users’ security log data. Cloud services like AWS, GCP, and Azure provide a remarkable abstraction layer that allows software engineers to avoid thinking about physical hardware. Unfortunately, this abstraction comes at a cost. If you don’t take the time to understand what your software is doing with the underlying hardware, it is easy to build applications that needlessly burn massive amounts of CPU cycles, SSD IOPs, and router operations by doing unnecessary extra work.

July 19, 2023 Advanced Persistent Threats in your Security Logs Security Logs Role in Preventing Advanced Persistent Threats As we’ve built Scanner, we’ve learned a tremendous amount from our users about the important role that security logs play in detecting threats. Among the most insidious adversaries they face is the Advanced Persistent Threat (APT). These are complex, multi-staged, and stealthy cyber-attacks, sometimes sponsored by nation-states or cybercrime syndicates. Their primary aim is to gain unauthorized access to a network and maintain it undetected for an extended period, often months or even years. As a result, they can cause significant financial and reputational damage to companies.

At Scanner, we use serverless Lambda functions to perform fast full-text search over large volumes of logs in data lakes, and our queries need to be lightning fast. We use Rust for this use case, but we wanted to know how Rust compared with Go, Java, and Python in terms of performance. We pitted the four languages against one another to see which was the fastest, and here is what we found.

At Scanner, we use Amazon Lambda functions and Rust in our log query engine. While Rust is technically supported in Lambda functions, it is not as easy to set up as the officially blessed languages: Node.js, Python, Ruby, Java, Go, C#, and PowerShell. In this post, we will share the information we wished someone had given us we got started using Rust in Lambda functions.

Given the extremely fast startup time and low memory usage of Rust binaries, using Rust in a serverless environment like AWS Lambdas is appealing. Rust launches extremely quickly from a cold start and uses little memory compared to other languages.