AI for CIOs
What Belongs in a CIO’s AI Risk Register Before Expanding a Knowledge Assistant?
SimplSolutions Editorial Team · 9 min read
Published

The expansion decision is a governance decision, not just a product decision
A knowledge assistant can perform well in a controlled pilot and still be unready for wider use.
The pilot may involve one department, a small set of documents, known users, and a support team that can investigate problems quickly. Expansion changes the conditions. More employees bring different permissions, more sensitive questions, more unfamiliar terminology, and more pressure to connect the assistant to business systems.
The CIO’s question should therefore be more specific than “Does it answer questions accurately?”
What evidence shows that this assistant uses approved business knowledge, stays within its authorized scope, refuses requests it should not answer, and leaves a reviewable record when something goes wrong?
That question belongs in the AI risk register before an expansion decision—not as an afterthought after the assistant reaches more users.
What NIST provides—and what it does not
The NIST AI Risk Management Framework is intended to help organizations manage risks to people, organizations, and society associated with artificial intelligence. NIST describes the framework as voluntary and designed to support the incorporation of trustworthiness considerations into the design, development, use, and evaluation of AI systems.
That makes it a useful organizing reference for a CIO review. It does not certify a particular assistant, prove that a deployment is secure, or replace the organization’s own approval process.
For an internal knowledge assistant, the framework is most useful when translated into operator-owned questions:
- Which business knowledge is approved for use?
- Who is allowed to retrieve it?
- What evidence supports an answer?
- Which systems can the assistant reach or affect?
- What does it do when a request is outside its scope?
- Who reviews exceptions and disputed answers?
- Can the organization reconstruct what happened?
The risk register should make those questions concrete enough to support a go, conditional-go, or no-go decision.
Establish approved sources and answer evidence
A useful register does not merely list risks. It connects each risk to an owner, a test or record, a decision threshold, and a current disposition.
Approved source permissions
The first risk is not whether the assistant can find information. It is whether it is allowed to use that information for this audience and purpose.
The register should identify:
- The source systems and document collections in scope.
- The business owner for each source.
- The intended audience and access conditions.
- How withdrawn, superseded, or restricted material is handled.
- What evidence shows that the assistant’s retrieval behavior respects those permissions.
A source inventory alone is not enough. A CIO should ask for evidence that permissions are enforced in the assistant’s actual experience, including searches that cross departmental boundaries or involve restricted terms.
A practical status field might distinguish between “source approved,” “access mapping reviewed,” and “runtime behavior tested.” Those are different claims and should not be collapsed into one green indicator.
Evidence quality and answer traceability
An answer can sound plausible without being sufficiently supported. Before expansion, reviewers should know how the assistant indicates the basis for an answer and what happens when the available material is incomplete or contradictory.
The register should capture:
- What counts as acceptable supporting evidence.
- Whether users can identify the relevant source or passage.
- How stale, conflicting, or missing information is surfaced.
- Which answer types require human confirmation.
- How sampled answers are reviewed and recorded.
The objective is not to promise perfect answers. The objective is to establish a reviewable standard for deciding when an answer is useful, when it is uncertain, and when it should not be relied upon without a person.
For a finance or HR assistant, for example, a response about a policy exception may require more than a general policy citation. The register should state whether the assistant may summarize, interpret, or merely direct the user to an authorized human process.
Scope integrations and test refusals
Integration scope and permitted actions
Expansion often introduces risk through integrations rather than through the conversational interface itself. A read-only assistant and an assistant that can initiate a workflow are not the same risk object.
The CIO should require an inventory of:
- Connected applications and data stores.
- Data the assistant can retrieve.
- Actions it can request, initiate, or recommend.
- Identity and authorization boundaries.
- Human approval points for consequential actions.
- Failure and rollback expectations for connected workflows.
The register should record the narrowest useful scope, not only the desired future state. If the pilot needs policy documents but not operational system access, that distinction should remain explicit in the expansion decision.
This is also where vague language such as “integrated with the business” becomes unhelpful. Each integration should have a named owner, a stated purpose, and evidence that the assistant cannot silently exceed that purpose.
Out-of-scope refusal behavior
A responsible assistant must do more than answer questions well. It must decline requests that fall outside its authorized knowledge, user permissions, or business purpose.
The risk register should include tested refusal categories such as:
- Requests for restricted information.
- Requests involving another employee’s private or sensitive data.
- Questions outside the assistant’s approved knowledge domain.
- Requests to make decisions reserved for a designated role.
- Attempts to bypass normal approval or access processes.
- Questions where the source material is absent, conflicting, or too old for the use case.
The evidence should show the observed behavior, not merely a design intention. Reviewers should record whether the assistant refused, redirected, asked for clarification, or produced an answer—and whether that response matched the policy for the scenario.
A refusal is not automatically good governance. An overly broad refusal can block legitimate work; an overly narrow refusal can create unacceptable exposure. The CIO needs a documented decision rule for both failure modes.
Define escalation and audit evidence
Human escalation and accountability
Some questions should not end with an AI-generated answer. They should end with a handoff to a responsible person or process.
The risk register should define:
- Which topics require human review.
- Who owns the escalation queue or process.
- What information accompanies the handoff.
- How the user is told that review is required.
- What service expectation applies to unresolved cases.
- How recurring escalations feed back into scope and source governance.
Human review is meaningful only when the reviewer has enough context to understand the question, the answer, the supporting material, and the reason for escalation. A generic “contact support” message may not create a useful control.
The register should also record what happens when no reviewer is available. A system that quietly answers because escalation is inconvenient has a different risk posture from one that pauses or redirects the request.
Auditability and access-denial testing
A CIO should be able to ask, after an incident or disputed answer: Which user asked what, what sources were available, what answer was returned, what action followed, and who reviewed the case?
The register should specify the records needed to investigate that question, subject to the organization’s privacy and retention requirements. It should also identify who can review those records and how exceptions are handled.
Access-denial testing deserves its own entry. Testing only successful use cases does not establish that boundaries hold. Reviewers should test representative denied requests, including requests from users who are valid employees but lack permission for the requested material.
The evidence should record:
- The user or role context used for the test.
- The requested information or action category.
- The expected denial or redirect behavior.
- The observed response.
- Any information disclosed despite the denial.
- The corrective owner and retest status.
The purpose is not to claim that a finite test set proves every boundary is perfect. It is to make denied access a deliberate control that can be examined, repeated, and improved.
A worked example: Sam’s expansion decision
Consider Sam, a fictional CIO deciding whether to expand an internal policy assistant from the People Operations department to the wider business.
The pilot is useful. Employees can locate approved policy documents faster, and the department reports fewer repetitive questions. The expansion proposal would add several thousand employees and introduce requests about travel, procurement, workplace policy, and benefits.
Sam’s risk register shows the following:
| Area | Evidence available | Open concern | Expansion decision |
|---|---|---|---|
| Source permissions | Owners have approved the initial policy collection | Several regional documents have different audiences | Conditional: exclude unresolved regional sources |
| Evidence quality | Sampled answers link to source material | Conflicting policy versions are not consistently surfaced | Conditional: require a conflict-handling rule and retest |
| Integration scope | The assistant is read-only in the pilot | The expansion proposal includes a case-management connection | No for that integration until separately reviewed |
| Refusal behavior | Basic out-of-domain tests are documented | Cross-department access-denial tests are incomplete | No wider launch until denial testing is complete |
| Human escalation | People Operations can review selected questions | No owner is assigned for procurement questions | Conditional: assign an owner or remove that topic |
| Auditability | Conversation records are retained for investigation | Review access and retention responsibilities are unclear | Conditional: document ownership and review procedure |
Sam does not need every row to be risk-free. Sam does need to know which uncertainties are accepted, which are blocking, and who is accountable for resolving them.
The resulting decision might be: expand the assistant to approved policy collections for a defined group, keep it read-only, exclude unresolved regional material, complete access-denial testing, and review the decision after a stated period. That is more precise than either “the pilot was successful” or “AI is too risky.”
What should block expansion?
A CIO should treat the following as potential expansion blockers unless a designated risk owner formally accepts them:
- No authoritative owner for a source collection.
- No reliable way to distinguish approved from restricted material.
- No test evidence for users with different access rights.
- No defined behavior for missing or conflicting information.
- No human owner for high-consequence or disputed questions.
- No usable record of the request, response, source context, and review path.
- A proposed integration whose permissions or actions are not separately understood.
These are not universal legal requirements. They are practical decision criteria for an organization that wants its assistant to remain governable as usage grows.
Keep the register alive after the launch decision
Expansion is not the end of the review. New sources, changed permissions, new integrations, policy revisions, and recurring refusals can all change the risk profile.
The register should therefore include a review trigger, not just a review date. Examples include adding a new business domain, changing the assistant’s action scope, introducing sensitive data, changing the identity model, or seeing a pattern of unsupported answers.
The strongest operating habit is to treat the register as a record of decisions and evidence—not a compliance-shaped checklist. Each material claim should have an owner, a supporting record, and a clear statement of what remains uncertain.
The takeaway
Before expanding a knowledge assistant, a CIO should require evidence across six areas: approved source permissions, answer evidence, integration scope, refusal behavior, human escalation, and auditability—including access-denial testing.
NIST’s voluntary AI Risk Management Framework can help structure that conversation, but the expansion decision remains operator-owned. The key question is not whether the assistant appears capable. It is whether the organization can explain what the assistant is allowed to know, do, refuse, and record—and who is accountable when the evidence is not enough.



