A vendor-neutral
evaluation framework.
Sixteen questions to ask any attack-surface platform. Use it against us, use it against anyone: the questions are the questions, and the answers are how you tell platforms apart.
Can it find what you don’t know you own?
| Q1. From a single seed domain, how many of our 60+ subdomains do you find? | Look for: passive DNS, certificate transparency, ASN enumeration, Wayback, code-search. |
| Q2. How do you attribute cloud assets that don’t carry our brand? | Look for: bucket-naming heuristics, IP reputation linkage, SSL SAN cross-reference, code-repo signals. |
| Q3. When we refute an attribution, how does the model learn? | Look for: confidence scoring, suppression workflow, audit trail. Avoid: opaque ML black box. |
| Q4. What’s your false positive rate on discovered assets? | Look for: a number, with measurement methodology. Avoid: “industry-leading accuracy.” |
Can you explain the number?
| Q5. Is the scoring methodology public? | Look for: published whitepaper, exact weights, formula. Avoid: “proprietary algorithm.” |
| Q6. Can a vendor dispute a finding? What’s the SLA? | Look for: documented workflow, named analyst, < 5 business days. Avoid: “email support.” |
| Q7. How often does the rating refresh? | Look for: continuous (within 24h of a finding change). Avoid: monthly recompute. |
| Q8. Show me a low-rated company you couldn’t explain. | Look for: a candid answer with examples. Avoid: deflection. |
What disciplines are covered?
| Q15. How many compliance frameworks do you map automatically? | Look for: 10+, with named frameworks. Avoid: “customisable templates.” |
| Q16. Brand takedowns. Are they included or extra? | Look for: included, with SLA. Avoid: “professional services engagement.” |
| Q17. Native integrations with Jira, ServiceNow, Slack, SIEM? | Look for: out-of-the-box, with field mapping. Avoid: “via Zapier.” |
Can it land inside your team?
| Q18. Onboarding time from contract signed to first useful rating? | Look for: hours to days. Avoid: 4-6 weeks of scoping. |
| Q19. API access. Rate limits. Sandbox. | Look for: REST & webhooks, documented limits, free sandbox. Avoid: API as paid add-on. |
| Q20. Is pricing on the website? | Look for: tiered pricing, transparent inclusions. Avoid: “contact us.” |
Is the pricing comprehensible?
| Q21. What scales the price? Assets? Users? Vendors? | Look for: one clear axis. Avoid: per-user fees on a platform you already pay for. |
| Q22. Free trial or paid pilot? | Look for: a free rating, no scoping call. Avoid: 90-day paid POC. |
Run the same questions across every vendor, including us
The value of an evaluation framework comes entirely from applying it uniformly. Asking three vendors different questions produces three sales narratives; asking them the same sixteen produces something comparable, which is the only thing an internal review can act on.
So the guide is written to be used against us. Several questions in it are ones where a competitor answers better (install base, track record, breadth of historical data) and pretending otherwise would make the whole document worthless to the person it is written for.
A framework written to flatter its publisher is transparent to any experienced buyer, and publishing one costs more credibility than it wins deals.
Four that separate vendors quickly
How is an asset attributed to a company, and can you see the evidence? Discovery is easy; attribution is where outside-in platforms genuinely differ, and raising a finding against a vendor for an asset that is not theirs damages every other finding you present.
Is the scoring model published and versioned? An unpublished model cannot be evaluated, and an unversioned one makes trending meaningless because you cannot separate a posture change from a methodology change.
What happens to a finding we can disprove? The answer should be that it goes, quickly, on evidence. Platforms without a disposal mechanism accumulate disputed findings until practitioners stop reading the output.
What does this explicitly not cover? A vendor who cannot name their own blind spots either has not thought about them or is not telling you. Outside-in assessment cannot see segmentation, backup testing, access governance or incident response, and any answer implying otherwise should end the evaluation.
Questions to ask before the pricing conversation
Does pricing scale with seats or with rated companies? Per-seat pricing on a risk tool has a predictable outcome: the people who should see findings do not get logins, and credentials get shared anyway.
Are modules sold separately? A partial view of an attack surface is not a cheaper product; it is a misleading one. It lets a customer hold a rating that looks complete while a category of exposure sits outside their subscription.
What happens at renewal if we have rated companies we no longer work with? Rosters change constantly, and a licence model that cannot accommodate that is one you will end up working around rather than using properly.
Common questions
Is this guide actually vendor-neutral?
It is written to be used against us, and it includes questions where competitors answer better than we do. That is self-interested rather than noble, a framework written to flatter its publisher is transparent to any experienced buyer.
What is the single most revealing question?
What does this explicitly not cover. A vendor who cannot name their own blind spots either has not thought about them or is not telling you.
Why does attribution matter so much?
Because discovery is easy and attribution is where platforms genuinely differ. Raising a finding against a vendor for an asset that is not theirs damages the credibility of every other finding you present.
How many vendors should we shortlist?
Three is usually enough to see the spread of answers. Beyond that the marginal question stops being informative and the evaluation itself becomes the cost.
Should we run a trial?
Rate your own organisation first. It is the only assessment where you can check every finding against what you already know, which tells you whether the vendor ratings are worth acting on.
Ask us all sixteen
The fastest way to test a platform against this list is to run it on a domain you already know well.
The scoring model is published in full, so question two answers itself.