What the NIST AI Framework Actually Requires of AI Used in Underwriting

What the NIST AI framework actually requires of AI used in underwriting is, on its own, nothing. NIST’s AI Risk Management Framework is explicitly voluntary. The requirement that actually binds an underwriting model comes from a different, much more specific source: a 2023 CFPB circular that already closed the exact loophole a voluntary framework cannot close on its own.

NIST’s framework names explainability and interpretability as characteristics of trustworthy AI, worth pursuing because they are good practice. The CFPB’s circular turns a specific version of that same idea into a legal floor for any creditor using AI in a credit decision, with no exception for models too complex to explain. Understanding both documents together, not just the NIST framework in isolation, is what actually tells an engineering team what an underwriting model has to be able to do.

What NIST’s framework actually means by explainable

NIST’s AI RMF 1.0 draws a careful distinction that most discussions of “AI transparency” collapse into one vague idea. Transparency, explainability, and interpretability are three separate characteristics, and the framework defines each one by the specific question it answers. Transparency answers what happened in the system. Explainability answers how a decision was made, through “a representation of the mechanisms underlying AI systems’ operation.” Interpretability answers why that decision means what it means to the user, in the context of the system’s actual purpose.

For an underwriting model, that distinction is not academic. A model can be transparent, in the sense that its logs show exactly what data it ingested and what score it output, while still failing to be explainable, because nobody can describe the mechanism that turned that data into that score. It can be explainable at a technical level, describing the mechanism precisely, while still failing to be interpretable to the actual person who needs to understand why a specific applicant was denied credit, not just how the model works in general.

Why NIST’s voluntary standard is not the binding one

NIST says this directly: the framework is voluntary, rights-preserving, and intended to apply across sectors rather than to any one regulated activity. That framing matters because it means NIST itself is not the source of law here. An underwriting team that builds an explainable model because NIST’s framework recommends it has made a good engineering decision. It has not, by that fact alone, met a legal requirement.

The legal requirement comes from the CFPB. Circular 2023-03 addresses a specific question: whether a creditor using artificial intelligence or a complex credit model may rely on the checklist of reasons in the CFPB’s own sample adverse action forms, currently codified in Regulation B, when those sample reasons do not actually and specifically identify why a given application was denied. The CFPB’s answer is direct. Creditors “may not rely on the checklist of reasons provided in the sample forms… if those reasons do not specifically and accurately indicate the principal reason(s) for the adverse action,” and this obligation holds “regardless of whether the technology used to make them involves complex or black-box algorithmic models.”

That second clause is the one that actually governs engineering decisions. A team cannot point to model complexity as a reason the adverse action notice is vague. The Equal Credit Opportunity Act’s notice requirement does not bend to how the model works internally. It requires the specific, accurate principal reason regardless.

Where NIST’s categories map onto the CFPB’s actual demand

Read together, these two documents describe the same underlying problem from two different directions. The CFPB names the legal consequence: a vague reason, or a checklist reason that does not actually match what drove the model’s decision, fails ECOA regardless of model complexity. NIST names the technical capability that consequence actually demands: the model has to be interpretable enough, in NIST’s specific sense of answering why a decision matters to the person affected by it, to produce a principal reason that is both specific and accurate for that individual applicant.

A model that is explainable in NIST’s sense, meaning someone can describe the general mechanism by which it scores applicants, is not automatically interpretable in the sense the CFPB’s circular actually requires. General mechanism descriptions produce generic explanations. The CFPB’s circular specifically rejects generic explanations, even accurate ones about how the model works in the abstract, if they do not resolve to the actual principal reason for a specific applicant’s specific denial. Building a model that only achieves NIST-style explainability without individual-level interpretability produces exactly the gap the CFPB has already said will not satisfy ECOA.

A concrete example: the denial that looks fine on a dashboard

Picture an underwriting model that scores merchant applicants on expected chargeback risk, built with a feature-importance dashboard showing the ten factors that matter most across the entire applicant population: processing history length, industry category, average ticket size, and so on. That dashboard is a real, useful artifact, and in NIST’s terms it is a genuine piece of explainability. It describes the mechanism by which the model generally operates.

Now a specific applicant gets denied. The adverse action notice the team generates pulls from that same population-level dashboard and states the denial reason as “industry category,” because that factor ranks highest in the general importance ranking. For this particular applicant, though, the model’s own decision may have turned almost entirely on a different factor. That factor could sit much further down the population-level ranking, yet still matter more for applicants with this specific combination of characteristics. The notice is explainable in NIST’s general sense. It is wrong in the CFPB’s specific sense, because it does not actually and accurately state the principal reason for this applicant’s denial. That gap between a population-level explanation and an individual-level one is exactly where a model that looks compliant on a dashboard fails the actual legal requirement.

Closing that gap requires the model, or a validated companion explanation system, to trace the specific path from this applicant’s own inputs to this applicant’s own score, not just to report where that applicant’s result sits relative to the general population. A system built only to produce the population-level dashboard was never designed to answer the question the CFPB’s circular actually asks.

The governance function that ties this together

NIST’s framework has a specific function built for exactly this kind of gap between a voluntary standard and a binding legal requirement. GOVERN 1.1 requires that “legal and regulatory requirements involving AI are understood, managed, and documented.” For a payments or lending platform building underwriting AI, that is not a generic instruction to stay broadly aware of AI regulation. It specifically means mapping the CFPB’s adverse action notice requirements, and ECOA and Regulation B more broadly, into the organization’s AI governance process as a documented, binding constraint, not an optional best practice sitting alongside NIST’s own voluntary characteristics.

A team that implements NIST’s explainability and interpretability characteristics as general engineering good practice, without separately documenting that the CFPB’s circular makes a specific form of interpretability a legal requirement for this particular use case, has only done half of what GOVERN 1.1 itself calls for. The framework’s own governance function requires connecting the voluntary technical practice to the actual binding requirement, not treating them as the same thing, and not assuming that satisfying NIST’s general characteristic automatically satisfies the narrower legal one sitting underneath it.

What this actually means for how an underwriting model gets built

The practical requirement is narrower and more specific than “make the model explainable.” An underwriting model has to be able to produce, for any individual denied applicant, a principal reason that is specific to that applicant’s actual data and accurate to what the model actually weighted most heavily in that specific decision. A model that can only produce a general feature-importance ranking across its training population, without the ability to trace that ranking back to an individual applicant’s specific inputs, has NIST-style explainability without CFPB-compliant interpretability.

This has direct architectural consequences for how the system actually gets built. A model built around a post-hoc explanation layer has to be validated for accuracy against the model’s actual decision process, not just plausibility. The CFPB’s circular permits post-hoc explanation methods only when their accuracy can be verified against the underlying model. A model that cannot support that verification at the individual-decision level needs a different approach. Either a different modeling choice entirely, or a parallel interpretable model built specifically to generate adverse action reasons, validated carefully enough to track the production model’s actual behavior rather than a convenient approximation of it.

This is exactly the kind of requirement that has to be designed into an underwriting model from the start, not retrofitted once a denied applicant disputes a vague reason. Building that capability correctly is core work for an AI software development practice focused on this industry’s actual constraints, not a generic machine learning build. The underlying data pipeline that has to support individual-level reason generation, tracing a specific decision back to the specific inputs that drove it, is exactly what a focused data and analytics engineering practice needs to expose by design. And because this requirement has to hold for every single adverse decision the model makes in production, not just the ones a team happens to review, the quality assurance and continuous delivery discipline around the model has to test individual-level explanation accuracy as a release gate, not a one-time validation exercise.

Frequently Asked Questions

Is the NIST AI Risk Management Framework legally required for underwriting models?

No. NIST’s AI RMF 1.0 is explicitly voluntary and sector-agnostic. It is not a legal requirement on its own. What makes explainability non-optional for underwriting specifically is a separate, binding source: the CFPB’s Circular 2023-03, which applies existing ECOA and Regulation B adverse action notice requirements to AI-driven credit decisions.

Can a creditor use a black-box AI model for underwriting and just give a generic adverse action reason?

No. CFPB Circular 2023-03 states that a creditor may not rely on the checklist of reasons in CFPB’s own sample adverse action forms if those reasons do not specifically and accurately identify the principal reason for the denial, and that this obligation applies regardless of whether the underlying technology is a complex or black-box algorithmic model.

What is the difference between explainability and interpretability under NIST’s framework?

NIST’s AI RMF defines explainability as a representation of how an AI system’s mechanisms produced a given output, answering the question of how a decision was made. Interpretability addresses the meaning of that output in context, answering the question of why the decision matters to the user. Both are distinct from transparency, which addresses what happened in the system.

Sources: NIST AI Risk Management Framework (AI RMF 1.0) and CFPB Consumer Financial Protection Circular 2023-03.