At a glance
- Problem
- Acting on intelligence that is stale, single-sourced, or circularly reported
- Goal
- A graded, corroborated judgement you can defend, with its confidence stated
- Two axes
- Source reliability and information credibility, graded separately
- Biggest trap
- Circular reporting: many voices, one origin, mistaken for corroboration
- Frameworks
- NATO Admiralty grading, structured analytic techniques, analytic confidence
- Rule
- Evidence over assertion; separate what is shown from what is claimed
Intelligence is a claim
The word “intelligence” lends authority that the underlying statement may not deserve. Every feed entry, report, and attribution is a claim made by someone, with a method, a confidence, and a set of assumptions behind it, most of which do not travel with the claim when it is copied into your tooling. Validation is the habit of restoring that context before you act. It is the difference between “this is malicious” and “source X, method Y, observed Z, assessed with moderate confidence, last seen on date W.”
Unvalidated intelligence is not free to consume; it has a cost when it is wrong. A false “malicious” verdict blocks legitimate traffic and erodes trust in the whole programme; a false “clean” lets a real threat through. Both are decisions, and both deserve evidence.
Grade the source and the information separately
The most useful habit intelligence work borrows from older tradecraft is grading two things independently: how reliable the source is, and how credible this specific piece of information is. The NATO Admiralty (or source-information) system formalises it, a letter A to F for the source, a number 1 to 6 for the information, so a rating like “B2” carries both judgements at once.
| Source reliability | Information credibility | |
|---|---|---|
| A | Completely reliable | 1 · Confirmed by other sources |
| B | Usually reliable | 2 · Probably true |
| C | Fairly reliable | 3 · Possibly true |
| D | Not usually reliable | 4 · Doubtful |
| E | Unreliable | 5 · Improbable |
| F | Cannot be judged | 6 · Cannot be judged |
Keeping the axes separate stops two classic errors: trusting a shaky claim because it came from a good source, and dismissing a solid observation because the source is unfamiliar. A reliable source can still relay something it has not verified; an unknown source can still show you a screenshot that speaks for itself.
Corroboration, done properly
Corroboration raises confidence, but only when the corroborating sources are genuinely independent. Three feeds agreeing is strong if the three arrived at the verdict by different methods and observations. It is meaningless if all three are re-publishing the same upstream feed. Before you count agreement as corroboration, ask where each source got its information, and whether any two of them actually share an origin.
The circular reporting trap
The failure that catches even experienced teams is circular reporting: a single original claim gets repeated by many outlets until the sheer number of voices feels like consensus, when it is really one source wearing several hats. An indicator appears in one feed, is ingested by five aggregators, is cited in two vendor blogs, and now “everyone says” it is malicious, on the strength of exactly one observation nobody has re-checked.
Volume is not corroboration. Ten sources tracing back to one origin is one source. Always follow a claim upstream to its earliest appearance; the confidence you can hold is set by that origin, not by the size of the echo around it.
Timeliness and decay
Intelligence decays, and different types decay at different rates. A malicious IP may be reassigned to an innocent tenant within weeks; a domain may be seized or expire; an actor's tooling may be retired. A verdict that was correct six months ago can be actively harmful today. So validation always asks when: when was this observed, when was it last confirmed, and is the window it describes the same window your decision concerns? An indicator with no timestamp is an indicator you cannot fully trust.
Relevance to your environment
A claim can be perfectly true and still not matter to you. Intelligence about an actor that targets a sector you are not in, or a platform you do not run, is accurate and irrelevant, and acting on it spends attention you needed elsewhere. Validation therefore includes a relevance test: does this threat plausibly reach my environment, my users, my technology? True-but-irrelevant is a category, and treating it as actionable is its own kind of false positive, a theme we develop in How to reduce false positives.
A validation checklist
- Origin. Who first made this claim, and by what method? Trace it upstream.
- Independence. Do the corroborating sources share that origin, or did they observe it separately?
- Evidence vs assertion. Is there an observable behind the verdict, or only a label? Separate what is shown from what is stated.
- Timeliness. When was it seen and last confirmed? Does that window match your decision?
- Relevance. Does this plausibly reach your environment at all?
- Confidence. Write down the grade and the reasoning, so the next reader inherits your judgement, not just your conclusion.
The output of validation is never a bare “malicious” or “clean.” It is a verdict with a stated confidence and the evidence behind it, so that acting on it, and defending that action later, rests on something more than the fact that a tool said so.
Validation at speed
The pressure in real triage is that this discipline takes time you do not have on every alert. A correlation platform helps by doing the tedious part: showing which sources actually agree, keeping provenance attached so you can trace a claim to its origin, and marking explicitly what could not be verified rather than smoothing it over. That is a deliberate design choice in Forensia, the verdict leads, but the evidence, the source agreement, and the honest gaps come with it, which is exactly the material validation needs.
Key takeaway
Intelligence is a claim, not a fact. Validate it by grading the source and the information separately, corroborating only with genuinely independent observations, tracing every claim upstream to defeat circular reporting, and checking that it is both current and relevant to you.
Then record your confidence and its evidence. A verdict you can defend is worth more than a confident one you cannot, and the habit of separating what is shown from what is merely asserted is the whole of the craft.
References & further reading
- NATO, the Admiralty (source-information) grading system, for rating source reliability and information credibility separately.
- Richards J. Heuer Jr., Psychology of Intelligence Analysis, on cognitive bias and structured analytic technique.
- US Government, Intelligence Community Directive 203, on analytic standards and expressing confidence.
- MISP Project, threat intelligence sharing, including confidence and source tagging in practice.
- OverWatch Labs, How to reduce false positives, on the true-but-irrelevant category.