Writing on Cyber Risk Assessment

Short observations from recent engagements. Not marketing — just things we have learned that might be useful.

Why we stopped delivering report-only pen tests

Five years ago we delivered pen test reports. Findings list, CVSS scores, executive summary. The report would land in inboxes, get filed in a SharePoint, and the same findings would appear on the next year's test.

We changed the engagement model in 2021. Every pentest now includes a 4-hour remediation working session with the client engineering team, where we walk through findings, demonstrate exploits in a lab environment, and pair on fixing the top critical findings.

Remediation rates went from roughly 38% in the first year (industry standard) to 79% in the first year (our 2023-2024 data across 47 engagements).

The FAIR question that breaks board cyber discussions

Ask the CISO: 'If we suffer a ransomware event in the next 12 months, what is the 90th-percentile loss in dollars, and what is the probability you're attaching to that 90th-percentile estimate?'

Most cyber risk frameworks cannot answer this. Heat maps cannot. NIST CSF tiers cannot. FAIR can, with Monte Carlo simulation on loss event frequency × loss magnitude. The answer is a distribution, not a single number.

Boards have started asking variations of this question. Insurers are asking it explicitly during renewals. CISOs who cannot answer in this form are losing budget arguments to peers who can.

SOC 2 evidence collection is a security control, not a compliance task

Most SOC 2 programs treat evidence collection as compliance overhead. This is backward.

The continuous evidence collection itself is the control. If logs that should be running monthly are only collected when the auditor asks, the control is not really operating.

We have seen programs where Vanta or Drata shows green dashboards while underlying evidence is broken. The platform marks a control as 'compliant' if a person clicks 'confirm' once a quarter, but the underlying log collector failed three months ago.

When CMMC Level 2 actually requires a 3PAO assessment

CMMC Level 2 has two assessment paths: self-assessment for non-critical contracts, and 3PAO assessment for contracts involving CUI at higher impact levels.

The DoD published clarifications in late 2024 narrowing what qualifies for self-assessment. Many primes that planned to self-assess found their contracts now require 3PAO.

If you are in the defense industrial base and planning CMMC compliance, verify your contract requirements before assuming self-assessment is sufficient. 3PAO engagement lead times are running 9-14 months as of Q1 2026.

The vCISO role is not "fractional security leadership"

The phrase 'fractional CISO' suggests scaled-down full-time work. That's not what an effective vCISO engagement is.

An effective vCISO is a senior security leader embedded in your governance: attending the audit committee, owning the board cyber report, present at incident response coordination, named on regulator-facing documentation.

The vCISO who shows up 4 hours a week for a stand-up call is not really doing the job. Effective engagements are 20-30 hours a week minimum for the first 6 months.

Older notes available on request.