Insurers · regulatory crosswalk
OID Bulletin 2024-11, provision by provision.
Where the LLM Validation Report provides responsive documentation, where it serves as one input among others, and where the bulletin's expectations stay squarely with your own AIS Program. Stated plainly, because a crosswalk that claims everything proves nothing.
Spectralgraph LLM Validation Report (v3) mapped to Oklahoma Insurance Department Bulletin No. 2024-11, Use of Artificial Intelligence Systems in Insurance (issued November 14, 2024)
Scope statement. The Spectralgraph LLM Validation Report is an independent screening measurement of fabrication risk in a designated, insurer-hosted generative AI (large language model) system. It is not an audit, attestation, examination, or opinion, and it does not certify compliance with Bulletin 2024-11 or any law or regulation. This crosswalk identifies, provision by provision, where the Report provides responsive documentation ("Addressed"), where it can serve as one input among others ("Supporting"), and where the provision is outside the Report's scope entirely ("Outside scope"). Provisions marked Outside scope remain the insurer's responsibility through its own AIS Program, and Spectralgraph makes no representation regarding them.
Section 3 — AIS Program guidelines
| Bulletin provision (paraphrase) | Status | Where this appears in the Report / basis |
|---|---|---|
| §3 (intro): "verification and testing methods to identify errors and bias in Predictive Models and AI Systems" | Addressed, in part | Error identification for the designated generative AI system: entire Report (per-domain results, findings register SG-NNN, Appendix A full response register). The bias / unfair discrimination portion of this provision is outside scope; see Section D below. |
| §3.4: validating, testing, and retesting to assess generalization of AI System outputs upon implementation, including performance on unseen data | Addressed | Core function of the engagement. Method section documents leave-one-model-out instrument provenance; measurement is performed on the deployed model using the insurer's own question types. Flagged responses are individually verifiable by the insurer's staff, supporting the bulletin's "performance against expert review" pathway. |
| §3.3(c): assessments such as repeatability, reproducibility, traceability, model drift, and auditability of measurements | Addressed, by analogy | Written for Predictive Models; the Report provides the analogous evidence for the designated generative AI system: document control with provenance hashes, machine-readable findings with stable IDs (findings.csv), Validity & Re-measurement section, and a quarterly re-measurement trend record where Continuous Assurance is engaged (drift evidence). |
| §3.7: narrative description of the model's intended goals and how the model is developed and validated | Supporting | Scope & Engagement Basis section plus the insurer's configuration statement (model, adapters, quantization, as attested by the insurer) document the validated system. Development narrative of the model itself remains with the insurer or its vendor. |
| §3.1: oversight and approval process for development, adoption, or acquisition of AI Systems | Supporting | A pre-deployment or post-deployment measurement can serve as an input to the insurer's approval process. The process itself is the insurer's. |
| §2.2: documentation requirements "developed with Section 4 in mind" | Supporting | The Report is constructed as producible documentation: signed Summary Letter, defined severity thresholds, findings register, limitations, independence statement, document control, and machine-readable appendix. |
| §1.1–1.9: written AIS Program, governance accountability, proportionality, consumer notice | Outside scope | The AIS Program is the insurer's own written program. The Report is one exhibit within it, not a substitute for it. |
| §3.2: data practices (lineage, quality, integrity, bias analysis and minimization, suitability, currency) | Outside scope | Insurer data governance is not measured. Note: the engagement itself is designed to require no consumer data (question types and a designated endpoint only). |
| §3.5, §3.6: protection of non-public information; data and record retention | Outside scope | Information security and retention programs are not assessed. |
Section 3, Part 4.0 — Third-party AI systems and data
| Bulletin provision (paraphrase) | Status | Where this appears in the Report / basis |
|---|---|---|
| §4.1: due diligence methods to assess third-party AI Systems | Supporting | Independent measurement of a vendor-supplied, insurer-hosted model provides diligence evidence the insurer did not generate itself and the vendor did not attest to itself. |
| §4.3: activities to confirm third-party compliance over time | Supporting | Periodic independent re-measurement (Continuous Assurance) provides a recurring, dated record for the third-party file. |
| §4.2: contract terms with third parties (audit rights, regulator cooperation) | Outside scope | Contracting is the insurer's function. |
Section 4 — Examination document requests
| Bulletin provision (paraphrase) | Status | Where this appears in the Report / basis |
|---|---|---|
| Item 1.3(d): "documentation related to validation, testing, and auditing, including evaluation of Model Drift," reflective of the system's components "whether based on Predictive Models or Generative AI" | Addressed | The Report is this document for the designated generative AI system: dated, signed, reproducible validation and testing documentation. Quarterly re-measurement produces the Model Drift evaluation record over time. This is the provision the engagement is built around. |
| Item 1.3(c)(ii)(3): techniques, measurements, thresholds, and similar controls used | Addressed | Severity taxonomy with thresholds printed in-document; Method section (published method citations, patent application 19/724,790, instrument provenance); findings register with stable IDs. |
| Item 1.3(c)(ii)(2): information about data used in development and oversight of the specific system | Supporting, in part | Question-set provenance is documented (count and domains confirmed with the insurer in writing). Training-data lineage of the underlying model is outside scope. |
| Item 1.2 / 2.1: pre-acquisition and third-party due diligence documentation | Supporting | Where the measurement was performed as part of diligence, the Report and its findings.csv are producible artifacts of that diligence. |
| Item 2.4: validation, testing, and drift documentation for third-party systems | Addressed | Same basis as Item 1.3(d), applied to a vendor-supplied model hosted by the insurer. |
| Items 1.1, 1.3(a), 1.3(b), 2.2, 2.3: AIS Program records, coordinating bodies, data practice records, third-party contracts | Outside scope | Program, organizational, and contractual records are the insurer's own. |
Section D — Explicit exclusions (stated so the covered items are credible)
| Area | Position |
|---|---|
| Unfair discrimination / bias testing | Not measured. The Report measures fabrication risk. Demographic-fairness testing of underwriting or pricing outcomes is a distinct discipline and should be sourced separately. |
| Traditional Predictive Models (rating, underwriting scores, actuarial models) | Not measured. The instrument applies to generative language models only. |
| Data governance, information security, record retention | Not assessed. |
| Governance program design, consumer notices, legal compliance determinations | Not provided. The Report supplies measurement evidence; conclusions of law are the insurer's and its counsel's. |
Instrument note. Instrument performance characterization: approximately 0.85 AUC, fabricated-entity discrimination, length-controlled, leave-one-model-out validation across an 18-model benchmark. This figure is provisional and is reported with its estimation conditions in the Report's Method section.
Crosswalk v1.0, published 2026-07-10. Source: OID Bulletin No. 2024-11 (Nov. 14, 2024), read in full from oid.ok.gov. This document describes the LLM Validation Report v3 structure and does not modify any engagement's scope; the engagement letter governs.
Want this mapping against your own program?
Reply with your question types and the model you host, and the sample report shows exactly what the evidence looks like before you spend a dime.
Email the labREPLIES SAME DAY · EVERYTHING ANSWERED IN WRITING, ON THE RECORD