The cost of false positives - how we became a target

TL;DR: the business cost of security false positives

• Detection rates mislead: Security vendors love to advertise higher detection rates, but a lazy rule that flags a legitimate domain based on URL structure alone can drop signups, spike support tickets, and burn partner trust before anyone reviews the alert.
• Zero false positives: Even a 1 in 100,000 false positive rate can block thousands of legitimate transactions during peak shopping; cside works to a zero false positive policy on client-side detections and uses its own products to keep signal above noise.
• Before you ship: Before your next detection rule ships, decide whether the review process includes context on domain reputation and payload behavior, or whether URL-string matches will keep flagging your own infrastructure.

Short on time? See cside's in-browser Magecart and skimmer blocking. It covers everything below in one deployment.

This week, we identified an intriguing use case involving the WP3[.]XYZ attack (link to our blog post). It sparked interest across the community and led to better detection rates on platforms like VirusTotal (VirusTotal link).

While most appreciated our efforts, others criticized us for not identifying the root cause or recommending services to clean up hacked websites. We still want to keep the community aware of attacks like this, and we're working to do it better each time.

When false positives hit home

After publishing the blog, something unexpected happened: our main website, cside.com, was flagged as suspicious. A single false positive like this can disrupt technical workflows, a business's reputation, and its day-to-day operations.

For technical readers, this shows why accurate detection rules matter, while for non-technical readers, it demonstrates how such issues can escalate and affect trust and business continuity.

Naturally, we were caught off guard and immediately began investigating.

Upon closer inspection, it seemed that some lazy detection rules might have caused the issue.

I've worked in endpoint security for over four years before transitioning to a SOC (Security Operations Center) analyst role. This experience gave me firsthand insight into the challenges of managing alerts and detections, a topic closely tied to the real cost of false positives.

As someone who has written detection rules, I understand the temptation to take shortcuts. Early in my career, I made similar mistakes. With experience, I learned the difference between good and bad rules and the importance of using the right technologies for rule writing. This is why most security vendors employ multiple engines to address different types of attacks effectively.

In this case, the issue seemed to arise from two factors:

1. The WP3[.]XYZ subdomain: We use a service to generate brief descriptions of domains using internal data and AI (Domain Insights). While the descriptions aim to provide useful context, they are not intended for detection purposes.
2. Misinterpretation by AV vendors: Some antivirus vendors flagged the domain name simply because it appeared in our URL. This kind of detection, based solely on the URL structure, lacks context and can lead to unnecessary disruptions.

To help the community, we shared the complete code in our blog for other companies to improve their detections. However, flagging a domain serving malicious payloads without proper context is a problematic practice. We have addressed this issue in detail in another blog post (Are Threat Feeds Still Good in 2024?).

The real cost of false positives

False positives may seem insignificant at first glance, but their impact can be profound:

1. Operational disruption: Even a 1:100,000 false positive rate can have severe consequences if it disrupts critical transactions. For example, imagine a payment gateway being flagged incorrectly during a peak shopping season. This could block thousands of legitimate transactions, resulting in frustrated customers, lost revenue, and potential damage to the company's reputation.
2. SOC analyst fatigue: FPs can waste thousands of hours as analysts attempt to triage and investigate non-issues. This fatigue can lead to true positives (TPs) slipping through unnoticed.
3. Business impact: This is the most concerning consequence, as it can jeopardize a company's entire operation.

Let's dive deeper into point three.

At cside, we're a small, young startup that cares about contributing to the cybersecurity community. However, if security products flag our website as malicious, it could jeopardize everything we've worked for.

I remember the initial panic of seeing the detection. Fixing the issue mattered, but protecting the trust we've built with our customers mattered more. The time and energy spent addressing such problems could disrupt operations significantly, especially for small companies like ours. While large corporations might weather such challenges, for smaller organizations, it could spell disaster.

Why it matters

The real cost of false positives goes beyond technical challenges. It impacts businesses, customers, and livelihoods. Those who have experienced the repercussions firsthand know how deeply it hurts.

Our commitment at cside

At cside, we firmly believe in a zero false positive policy.

We strive to:

• Share whatever we know with the community openly.
• Use our own products to ensure accuracy and reliability.
• Minimize the impact of false positives while improving our detection capabilities.

We understand the cost of both false positives and cyberattacks, which is why we're committed to continuous improvement and collaboration with the broader community.

If you have questions or suggestions, feel free to reach out. We're always open to feedback.