Ramp Increases Security Visibility and Accelerates Threat Hunting with Scanner’s Security Data Lake

Before adopting Scanner, Ramp's security engineering and operations teams struggled under the weight of their existing log management infrastructure, which relied on Datadog for active logs. They could only retain 15 days of searchable logs due to the high cost. This was a significant barrier when fraud and security investigations needed to look past more than two weeks. Additionally, some logs such as Cloudflare logs were never ingested into Datadog due to their high volume.  

To get more visibility they archived logs in Amazon S3. This allowed them to retain high-volume but critical log sources and for a much longer retention period. When they needed to search and investigate through these logs, they would try to use AWS Athena. 

However, the queries with Athena would take a painfully long time (30 minutes or more), often even timing out. This would then force them to break down queries into smaller segments, for example searching on daily instead of monthly timespans. 

These challenges directly impacted critical, time-sensitive functions across the organization, including complex fraud investigations, resolution of engineering incidents, and the continuous improvement of detection engineering efforts.

Ramp adopted Scanner after a proof-of-concept demonstrated its ability to handle high-volume data cost-effectively and with extremely fast search, solving the critical log retention problem. One example of this was their ability to now retain their Celery infrastructure logs for a much longer time period. 

Ramp began onboarding essential log sources, starting with CloudTrail logs due to their high volume and ability to natively write to S3. They subsequently onboarded high-volume logs that were previously cost-prohibitive, including Cloudflare logs, as well as Zendesk, Okta, Salesforce, and Clickhouse logs. They currently ingest approximately 4 TB of log data a day, with plans to expand that even further.  

The real magic of Scanner is that their query times have cut down from 30+ minutes with Athena to just a minute or two. This dramatic increase in speed has enabled them to conduct response investigations and threat hunting tasks much more effectively. 

In addition to security logs, Ramp also has their application logs flowing into Scanner at a very high scale. Now their app teams can debug issues that span over long periods of time, which was impossible to do before. 

Detection engineers at Ramp are very vigilant about keeping up with the latest threat vectors via articles, social media posts, and intelligence feeds. They leverage Scanner to rapidly search for IOCs in their own environment (going back months) and to also build detection rules to detect new attack vectors in the future. Since many campaigns are usually longer than two weeks, the ability to query several months of data has given them crucial new levels of visibility and risk management that they lacked before. 

One of the key requirements for Ramp was to write detections-as-code. Scanner enables their team to create detection rules in their own GitHub repository, and easily leverage CI/CD best practices such as unit testing, change management, and collaboration.   

Ramp’s innovative security engineering team integrated Scanner’s APIs with external tools like Cotool and Linear to automate many investigation steps. Tickets originating from Scanner go to Linear. This triggers Cotool’s AI agents which leverage Scanner (and other tools) APIs to search and pull in context from multiple sources. By the time a detection engineer looks at the ticket, it has already been enriched with intelligent information that is critical to resolving it. Today there is always a human-in-the-loop who reviews the information, possibly conducts further analysis, and determines whether to close the ticket. 

The shift to Scanner delivered substantial improvements in performance and operational scope:

• Increased Visibility from 15 Days to 1 Year: Ramp was able to search up to a year of logs instead of the previous 15 day retention limit. They also successfully onboarded crucial, high-volume data sources that were previously excluded due to cost, enabling much broader visibility for incident response, threat hunting, and detection engineering.
• Hyper Fast Queries on S3: Queries that previously caused Athena to time out after 30 minutes now complete in about a minute with Scanner, drastically reducing investigation time.
• Streamlined Security Operations: Ramp uses Scanner for Threat Hunting, often setting up rules to look for future incidents. The detection engineering team was delighted by features that save time, such as values auto-complete while querying, which they noted was surprisingly rare in other tools.

As a design partner, Ramp has been guiding and influencing Scanner’s product roadmap with great success. They plan to continue doing so by leveraging Scanner for beyond just security use cases - expanding to DevOps and infrastructure logs as well. 

The security team is looking forward to using Claude Code with Scanner’s upcoming MCP Server to support additional AI use cases. Other planned capabilities include log monitors, anomaly detection, and more.