guide
15 min read
FreeHow to Read and Validate ClusterHawk Reports
This guide teaches analysts how to read platform-generated reports, what to focus on, and how to validate claims against the underlying artifacts.
By Chawkr Reports
15/08/2025
How to Read and Validate ClusterHawk Reports
Purpose
This guide teaches analysts how to read a single platform-generated report, what to focus on, and how to validate claims against the underlying artifacts, following the system methodology. It reflects the current report format. Section numbers below match the headings the report emits.
Two report forms. Every analysis job produces the standard analysis report described here. A pilot investigation deliverable is the same report plus three manually-appended annexes (per-IP disposition, hunting pack, method glossary).
Quick Start: What to Extract First
For threat intelligence and actor tracking, prioritize these sections:
- Section 2.B (Cluster Portraits): extract JA3/JA3S/JARM fingerprints and the compound validation query for persistent actor tracking.
- Section 6 (Anomaly & Outlier Intelligence): identify CRITICAL anomalies (≥95th percentile) and, crucially, the Noise/Outlier Re-Analysis (6.B); the noise re-analysis surfaces structurally unique assets the main clustering misses.
- Section 4 (Hypothesis Engine): read the competing hypotheses and whether a non-discrimination cap was applied before trusting any single attribution.
- Section 5 (Action Matrices): Shared Recommendations and role-specific actions.
- Section 1 (Executive Summary): extractable for non-technical stakeholders.
The Cluster Portraits (Section 2.B) are the most valuable for long-term tracking: fingerprints and compound queries persist across infrastructure changes and enable cross-campaign correlation. Most other sections reference these profiles by cluster number.
Core Principles
- Cluster Portraits (Section 2.B) are the crown jewels: persistent tracking fingerprints (JA3/JA3S/JARM, compound query patterns, infrastructure traits) for long-term actor tracking and attribution.
- Read the noise, not just the clusters: the Noise/Outlier Re-Analysis (6.B) routinely isolates the single most interesting asset in a dataset. A pivot never reaches it; the re-analysis does.
- "So What?" mindset: focus on operational impact, defensive implications, and intelligence value, not raw lists.
- Multi-perspective clarity: sections address SOC, TI, and Hunting teams with role-specific actions.
- Data completeness: exact values belong in Section 11 (Technical Appendix); narrative sections summarize patterns.
- Anomalies are dataset-relative, not a verdict of "bad": an anomaly is a host that does not lean the way the rest of this dataset leans, a structural outlier against the dataset's own center, not against an absolute model of malice. Severity scales with how far the host sits from that center: scores at or above the 95th percentile are emitted as CRITICAL. Read "flagged" as "doesn't fit this population," then decide whether not fitting is interesting, which depends entirely on what you seeded (see Section 6).
- A quiet anomaly result is a real result: a job with zero or only low-severity anomalies is not a failure or an empty report; it means the dataset is structurally uniform (one template, one operator profile, tight cohort), so nothing meaningfully deviates from the bulk. The detector reports what is there; when the data is homogeneous, "nothing stands out" is the honest and correct finding.
- The compound is the finding: a single fingerprint match is noise; the multi-parameter validation query is the signature.
Reading Order and Focus Areas
1. Executive Summary & Recommendations
What to focus on:
- 1.1 Executive Summary: BLUF, Infrastructure Overview Table, Key Findings, Risk Assessment.
- 1.2 Priority Actions Reference: top actions (references Section 5).
- 1.3 Attribution Summary: the primary hypothesis with its rubric percentage, and note explicitly whether a non-discrimination cap was applied (references Section 4).
Reading strategy for executives:
- Start with 1.1 BLUF and the Risk Assessment table.
- Review 1.2 Priority Actions.
- Check 1.3 Attribution Summary, and read whether the verdict is capped (if so, treat attribution as undetermined).
- Drop into full sections only when deeper detail is needed.
2.A Infrastructure-Cluster Matrix (Section 2.A)
What to focus on:
- Cluster groupings by core technology and possible operational role.
- Defensive Priority (CRITICAL / HIGH / MEDIUM / LOW) and the Defensive Maturity assessment.
- MITRE ATT&CK tactics per group.
- The Infrastructure Inventory Summary (asset counts + primary providers per cluster).
2.B Comprehensive Cluster Portraits (Section 2.B)
This is the most valuable section: it contains persistent fingerprints for long-term actor tracking and campaign correlation. Each cluster portrait is structured as four numbered subsections plus a Multi-Perspective Assessment:
1. Threat Intelligence & Technology Stack
- Detection Signatures (tooling labels and their confidence splits, e.g.
CobaltStrike:50%/GoPhish:50%). - Product & Technology stack with prevalence; version analysis.
- Vulnerability Intelligence: which CVEs, on how many hosts, KEV status, EPSS. Ask whether a CVE fingerprints the product (incidental) or the operator's configuration posture (a signature).
2. Infrastructure & Network Profile
- Network Services / port profile: attack surface and operational role (e.g.
:3333admin panel vs:443-only landing pages vs+:25/:587self-hosted mail). - Certificate & Cryptography: TLS versions, cipher suites, issuer patterns, self-signed vs CA-issued.
- Industrial protocols (ICS): BACnet / Modbus / Siemens S7 / etc. presence when the assets expose OT services, a high-signal finding that usually sits outside standard IT detection tooling.
- Exposed databases: MongoDB / Redis / Elasticsearch / MySQL exposure and whether authentication is enabled.
- Geographic distribution.
3. Cluster Validation & Uniqueness: PREDICTIVE DEFENSE
- The compound, multi-parameter query that synthesizes the cluster's pattern, with its real-world match count.
- Critical: a 0-match (or near-zero) compound query = a highly distinctive pattern; high confidence that future matches belong to this actor/campaign. Use these as watchers for infrastructure expansion.
- A commodity match count (millions) flags a generic/background cluster; baseline it, don't track it as an actor.
4. Statistical Analysis & Overlap
- Shared components/fingerprints with other clusters (operational relationships).
- The medoid (most representative member) and borderline IP: what defines the cluster's identity and where it blurs.
- Cloud/provider distribution: infrastructure procurement strategy.
Multi-Perspective Assessment (closes each portrait)
- Organizational & Hosting Intelligence: provider mix and what it implies (compartmentalized procurement, bulletproof hosting, trusted-cloud abuse).
- Network Exposure & Distinguishing Features: the highest-importance differentiating features (per the Metric Amber feature-prevalence ratio, "Nx vs dataset"), the medoid's defining traits, and how the borderline IP diverges.
- Cluster Quality Metrics: the per-cluster Quartz/Obsidian/Topaz roll-up.
Why this section enables actor tracking:
- JA3/JA3S/JARM fingerprints persist across IP rotation: track actors even when nodes churn.
- Compound queries become predictive alerts: get notified when an actor deploys new infrastructure matching the pattern. The compound, not any single parameter, is what makes the alert high-precision.
- High Metric Amber differentiators identify signature behaviors: the more a feature is over-represented vs the dataset, the more distinctive and trackable it is.
- Shared fingerprints across clusters reveal campaign relationships: same operator often reuses a deployment template across operations.
- Portraits accumulate into threat-actor dossiers over time.
Pro tip for TI teams: extract the JA3/JA3S/JARM fingerprints and the full compound query from each portrait; add the compound queries to continuous monitoring; track fingerprint/template reuse across time; pull exact values from Section 11 for TI-platform integration. Never deploy a single-parameter match as a detection: it will be noise.
3. MITRE Mapping (Section 3)
What to focus on:
- Techniques by cluster with ATT&CK IDs and the observable evidence cited for each.
- Infrastructure progression patterns (attack flow across clusters).
- The Technique Summary narrative tying tooling co-detections to capability stacking.
4. Hypothesis Engine (Section 4)
What to focus on:
- The competing hypotheses, typically a null (H0: unrelated/typological), and 2-3 alternatives (multi-operator collection, coordinated service/PhaaS, red-team vs malicious, IAB/APT staging, etc.).
- For each: supporting evidence with cluster references, disconfirming indicators, and the per-dimension
+tierscoring (Feature Distinctiveness, LIME Coherence, Cluster Quality, Labeling Corroboration, Query Validation, Cross-Cluster Consistency). - The score as a rubric percentage, not a calibrated real-world probability: the report says so explicitly. Read it as relative ranking, not odds.
The Non-Discrimination Cap is the most important thing not to misread. When two hypotheses score within ~5% of each other, the engine caps them and states that the data cannot discriminate between them. A 75% / 73% split does not mean "two close but distinct possibilities": it means do not pick a winner; the evidence to separate them (victimology, payload, temporal batching) is not in passive metadata. Honor the cap; keep both hypotheses active and drive targeted collection.
Confidence interpretation for the uncapped spread: 80-100% High, 60-79% Medium-High, 40-59% Medium, 20-39% Low-Medium, 0-19% Low.
5. Action Matrices by Role (Section 5)
What to focus on:
- Shared Recommendations (All Roles): CRITICAL / HIGH / MEDIUM priority actions across teams.
- Threat Hunter Specific: hunt procedures, fingerprint hunts, anomaly investigation targets.
- SOC Analyst Specific: safe-to-deploy detections (favor zero/near-zero-match compound signatures for low false positives).
- Intelligence Analyst Specific: attribution tracking and campaign correlation.
- The Immediate-Block IP list (assets with confirmed offensive-tooling detections).
6. Anomaly & Outlier Intelligence (Section 6)
Read all of it: the most interesting asset in a dataset usually lives in 6.B, not 6.A.
Read every anomaly against the dataset you submitted. The detector finds hosts that deviate from this dataset's structural center; it does not score against a fixed model of "malicious." That makes interpretation entirely dependent on what you seeded:
- On a confirmed-bad / C2 seed, the anomalies are the hosts that break the malicious bulk, which can mean benign contaminants, a misrouted host, or an operator who is structurally different from the rest. "Flagged" here does not mean "the worst one"; it means "the one least like the others."
- On a mixed or broad seed, the anomalies are the genuine structural oddities: the bespoke host in a sea of commodity tooling, the standalone node deliberately separated from the fleet.
Same host, different dataset, different verdict. That is correct behavior, not instability. The detector tells you what is structurally different; you decide whether different is interesting. This is also why the verify-ground-truth step matters: confirm a flagged host against its own banner/cert before deciding what its difference means.
A low-anomaly or zero-anomaly result is a valid finding. If a job returns few or no anomalies, the dataset is structurally uniform: a tight, templated population where nothing meaningfully stands out. That is a real, honest answer (often a strong signal of single-template/single-operator infrastructure), not an empty report. Do not read a quiet anomaly section as "the analysis found nothing."
6.A Main Clustering Anomaly Analysis
- IPs with unstable cluster membership across the clustering runs (assigned to many distinct clusters, low neighbor consistency, high neighbor turnover). All flagged at CRITICAL severity.
- The Top-N most anomalous IPs with their distinct-cluster counts and key instability pattern.
- Common anomaly patterns and which clusters the instability concentrates in.
6.B Noise/Outlier Re-Analysis: the high-value move
- The noise partition (Cluster -1) is re-analyzed on its own, producing noise sub-clusters plus residual outliers.
- Method Vela flags the structurally anomalous noise IPs (with a percentile and a nearest-main-cluster pointer).
- This is where deliberately-separated infrastructure surfaces, e.g. a pure single-tool detection sitting outside every main cluster (a probable standalone control node). A same-indicator pivot never reaches these; the re-analysis does.
- Cross-system signal: an IP flagged in both 6.A (main clustering) and 6.B (Method Vela) is the highest-priority target in the dataset.
6.C Anomaly Intelligence Categorization Matrix: anomaly type → detection source → indicators → threat potential → recommended action.
6.D Anomaly Cluster Profiles: per-IP narrative, what defines its assigned cluster and exactly how it diverges.
6.E Tool-Detected Outlier Profiles: assets carrying offensive-tooling labels that landed in noise, with their similarity-group and reverse-similarity-group memberships (low-importance features shared with other noise IPs, an infrastructure-reuse / attribution-tracking signal).
7. Attribution, Operational Intelligence & Victim Analysis (Section 7)
7.1 Attribution Indicators Analysis
- Infrastructure behavioral patterns and key attribution challenges.
- Technical Tradecraft Assessment (default vs custom configs, OPSEC sophistication, cloud-fronting, domain impersonation).
- Procurement and selection patterns (multi-cloud strategy, regional hosting alignment, certificate procurement).
7.2 Infrastructure Role and Vulnerability Analysis
- Infrastructure role classification (landing pages / admin-management / relay-staging / full-stack) with evidence.
- Critical exposure identification: which CVEs / services / configs are highest-priority, with KEV/EPSS prioritization. The corresponding remediation and hardening actions live in Section 5, not here.
8. Defensive Intelligence & Gap Analysis (Section 8)
What to focus on:
- Detection Blind Spots (non-standard ports, certificate-metadata inspection gaps, cloud-fronted traffic).
- Coverage Assessment by asset type and protocol.
- ICS/OT and database-exposure gaps.
- Defensive opportunities (e.g. KEV exposure on the infrastructure itself).
9. Operational Threat Hunting & Detection Engineering (Section 9)
What to focus on:
- Infrastructure-based detection rules: deployment-ready, compound, with per-rule global-match counts and confidence.
- Favor the zero/near-zero-match compound rules for SOC deployment (low false positive risk).
- Network detection signatures (fingerprint lists) referencing Section 11 for exact values.
10. Operational Security & Risk Assessment (Section 10)
10.1 Operational Security Assessment: OPSEC sophistication tiers per cluster, infrastructure persistence indicators, operational tempo.
10.2 Risk Assessment Framework: risk factors with levels and justification; intelligence collection value; priority targets.
11. Technical Appendix (Section 11)
- 11.1 Infrastructure Asset Lists: per-cluster summaries with asset counts and classifications (incl. the Noise and Honeypot partitions).
- 11.2 Cross-Cluster Correlation Matrix: the top shared-indicator overlaps with confidence.
- 11.3 Infrastructure Profiles: detailed profiles for the most significant and most anomalous clusters, including a Noise Cluster profile from the re-analysis. Exact fingerprint values and the per-cluster Detection Query live here; this is the source for TI-platform integration.
12. Clustering Quality and Methodology Assessment (Section 12)
What to focus on:
- 12.1 Dataset Composition: Total IPs, Clustered Assets, Noise/Outliers (Cluster -1), Honeypots (Cluster -2), NIL (Cluster -3). The noise and honeypot counts set up Sections 6 and 11; note them early.
- 12.2 Quality Metrics: per-cluster ratings: Metric Quartz (stability), Metric Obsidian (distinctness), Metric Topaz (distribution), rolled into an Overall Evaluation (Very Good / Good / OK / Bad). Treat "OK" clusters as hypotheses.
- Detection Signature Summary: which tooling labels dominate (e.g. GoPhish) and which co-occur (e.g. Cobalt Strike).
- Anomaly Rate: the headline count and percentage flagged, all at CRITICAL severity.
Read the quality rating honestly. A high "Very Good" share means clean typological separation, not proof of a single operator. Separation is about how distinct the groups are, not who runs them.
Investigative (Pilot) Report
The pilot investigation deliverable is the standard analysis report above, delivered findings-first, with three extra annexes appended by the analyst.
- Annex A: Per-IP Disposition. The dataset composition (total / clustered / outliers -1 / honeypots -2 / NIL -3 / usable) and a table of the notable IPs from the submitted list (any IP with a tooling detection, a CRITICAL/HIGH anomaly, or a honeypot/NIL disposition), each with its cluster, signature, anomaly severity, and disposition. A companion CSV carries every submitted IP so the client can audit any single address.
- Annex B: Hunting Pack. The per-cluster compound queries with their real-world match-count band from Shodan, plus the confirmed-signature blocklist. The curated detection rules stay in Section 9. Annex B is the raw, copy-paste hunting material and the match-count context.
- Annex C: Method Glossary. Plain-language meanings of the metric/method codenames (Metric Quartz and the other quality metrics, Method Vela, Metric Opal/Ruby, and the clustering-method names) plus the confidence-band caveat: qualitative bands are evidence-strength judgements, not calibrated probabilities. Codenames stay opaque by design; the annex never discloses the underlying algorithms.
Machine Learning & Ensemble Clustering Interpretation
- Cluster Stability: high stability across varied initial conditions and resampling ⇒ higher confidence; low stability ⇒ treat as a hypothesis pending corroboration.
- Separation & Overlap: strong separation suggests meaningful operational differences; overlap may reflect shared providers or multi-tenant hosting (avoid actor claims without supporting evidence).
- Ensemble Meaning: agreement across diverse model views strengthens trust; divergence indicates alternative plausible groupings and should drive hypothesis testing.
- Size & Distribution: large, diffuse clusters are often background archetypes; small but cohesive clusters are frequently specialized and operationally significant.
- Data Sparsity Signals: missing/sparse features can be deliberate hardening/evasion, not just absence of data.
Outlier Handling Playbook (high vs low confidence)
-
High-confidence outlier (small but cohesive; consistent over time; strong unique features)
- Treat as a priority deep dive (specialized capability, test bed, or control plane).
- Actions: extract exemplars, craft narrow compound detections, add 0-match watcher queries.
- In the current format these often appear in 6.B (Method Vela) or as a Noise Cluster profile in 11.3: a tight, structurally-unique group separated from the main population is exactly the high-confidence case.
-
Low-confidence outlier (unstable labeling; inconsistent features; low assignment confidence)
- Treat as suspected noise or decoy until proven otherwise.
- Actions: recheck provider/ASN stratification, assess collection gaps/time skew, and run feature-family sensitivity tests (does it disappear when a feature family is withheld?). These map to the 6.A instability patterns.
Analyst Tips
- Treat "OK"-rated and low-stability clusters as hypotheses until corroborated by multiple fingerprints.
- Prefer high-specificity fingerprints (cert subjects / JA3 / JA3S / JARM, tight product-stack combos) for initial detections, and always deploy them as a compound, never a lone parameter.
- Use compound queries for predictive monitoring: 0 matches = unique patterns to watch; commodity matches = archetypes to baseline and alert on.
- When a hypothesis is capped (non-discrimination), keep competing hypotheses active and drive targeted collection; do not report a winner the data does not support.
- Always read 6.B. The asset that matters most is frequently the one the main clustering threw into noise.
- An IP flagged in both 6.A and 6.B is your single highest-priority investigation target.
- Outliers are double-edged: high-confidence outliers may reveal specialized or hidden capabilities (investigate first); low-confidence outliers can be noise or decoy/control nodes (investigate before discarding or acting).
- Verify that outlier traits are genuine (rare cert subjects, distinctive provider use) vs. collection bias or boundary effects.
