Cloudflare DNS and HTTP logs are rich with security-relevant data. These logs capture crucial events like DNS queries, HTTP requests, and responses, which can provide powerful insights for incident response and threat hunting.

Querying Cloudflare DNS and HTTP logs helps identify suspicious activity quickly.

Incident response examples:

• Trace malicious DNS requests to identify potential command-and-control domains.
• Investigate suspicious HTTP requests that may indicate an attempted attack.
• Detect unusual spikes in DNS activity or suspicious HTTP headers.
• Identify unauthorized access attempts or suspicious API activity.

Threat hunting examples:

• Query for DNS requests to known malicious domains.
• Search for HTTP requests with anomalous user agents or suspicious patterns.
• Identify spikes in DNS queries for domains associated with phishing campaigns.
• Look for unusual HTTP request patterns that may indicate data exfiltration attempts.

However, due to their scale, Cloudflare DNS and HTTP logs can present challenges to overcome.

These log sources can be voluminous, often reaching hundreds of gigabytes to single digit terabytes per day, which makes traditional SIEM solutions prohibitively expensive. Paying for these logs in a SIEM can become a significant budget item, with costs of six to seven figures per year.

When dealing with hundreds of terabytes or even petabytes of logs, teams frequently move logs into object storage, like S3, for scalability and cost reduction.

However, with a data set of that scale in S3, querying for an IP address or other IOCs can take 12 hours or more using Amazon Athena.

These obstacles greatly hinder incident response efforts and create inefficiencies in threat hunting workflows.

Scanner offers a better approach. By indexing these logs directly in S3, Scanner provides fast, cost-effective search capabilities.

For volumes of 1TB or more, it is common to see cost savings of $500k or more compared to traditional SIEMs by using Scanner to search these logs directly in place in S3.

Incident response and threat hunting teams benefit from Scanner's speed, particularly for high-selectivity queries like IP address and IOC searches.

For example, finding an IP address or other IOC in a petabyte of logs will take tens of seconds, while Athena may take tens of hours.

Imagine an analyst on your team receives an alert: a phishing link in an email has been clicked by one of your employees. This alert comes from your email security tool, let's say Sublime Security. Like many similar tools, Sublime allows you to export its logs to one of your S3 buckets.

You also have Cloudflare DNS logs exporting to an S3 bucket. Your team uses Scanner to organize these logs in S3, enabling fast and easy searches.

Now, back to the analyst. The alert notifies them that a potentially malicious link has been clicked.

The analyst opens Scanner and searches for the domain of the malicious link in the Cloudflare DNS logs. These logs contain records of all the domain lookups performed by company devices as employees browse the web. The analyst quickly finds the specific DNS lookup event related to the malicious link. The next question is: "What websites did the employee visit next?"

Using Scanner's "Go To Context" feature, the analyst zooms in on the DNS activity of that employee for the subsequent five minutes after the phishing link was clicked. They discover that the employee visited another domain—a Sharepoint URL that appears suspicious.

The analyst decides to act on this finding. First, they create a new detection rule in Scanner to monitor Cloudflare DNS logs and generate an alert whenever that suspicious Sharepoint domain is accessed by an employee. Next, they save a search in Scanner to identify all employees who have visited the same suspicious Sharepoint domain.

Since Scanner is designed for rapid, needle-in-haystack searches across historical data, the analyst searches the past six months of Cloudflare DNS logs. They identify three employees in total who have visited the suspicious domain.

With this information, the analyst notifies the security team. They contact the three employees and investigate further. Using Scanner, the team digs into other log sources, such as Windows Event logs, focusing on the activity related to those three employees, looking for any signs of malware installation or compromise.

Scanner's ability to organize raw log data from multiple sources in S3 and perform fast searches over months or even years of data makes investigations like these efficient and thorough. By leveraging high-volume data sources like Cloudflare DNS logs, Scanner helps keep investigation costs under control while ensuring your team can respond swiftly to potential threats. There is no need to rehydrate or reingest logs back into a SIEM, which can take several hours and is expensive in terms of both ingest costs and wasted time for the security team.

Whether you're responding to incidents or hunting for threats, Scanner can help you quickly get to the answers you need.