Test the access boundary before adding another source to your AI assistant

More connected information is not automatically more useful

An assistant needs information relevant to its intended audience and task. A source that improves one answer can create an access problem for another user. Before adding it, record the owner, approved audience, data purpose and permitted actions. A source connector appearing in a demo does not establish that its permissions match your company rules.

Start with a written outcome for each request type: answer with evidence, ask for clarification, deny, or escalate. Keep the expected result independent of the vendor demonstration. Otherwise an impressive response can gradually become the acceptance standard even when it fails your actual boundary.

Build a small test set with deliberate failures

Use synthetic records for the initial evaluation. Include a current permitted procedure, a restricted record, an expired source, conflicting guidance and a question with no approved evidence. Add an unavailable source and a request to modify a record through a read-only connection.

For each test, record the audience, applicable source, expected result and the authorized reviewer. A denied request should not reveal restricted information in its explanation. A source conflict should remain a conflict rather than becoming a blended answer. An unsupported write should not be quietly treated as an approved action.

Judge evidence separately from fluency

Review the factual answer, source applicability and access behavior as different checks. A well-written response can use an outdated document. A correct general answer can still expose a fact to the wrong audience. A citation can point to a real file that does not support the claim.

Ask the reviewer to open the cited source and locate the supporting passage. Record the source version used in the test. If the system cannot expose enough evidence to verify a material answer, treat that as a gap rather than filling it with a confidence percentage.

Illustrative editorial photograph, not a customer result.

Test changes, not just the first successful answer

Remove or supersede a controlled test source and repeat the question. Does the assistant still use the old information? Ask the system owner to describe refresh behavior and failure handling, then inspect the actual result. Do not assume an uncached web request means every internal source is current.

Change a synthetic user's permission and repeat the restricted question. Distinguish source access from generated-output retention. Your security reviewers should verify the actual implementation and applicable data terms; this worksheet is not a security certification.

Record a decision that an average score cannot hide

Count supported answers, correct denials, stale-source failures and reviewer effort separately. Decide which failures are hard gates before testing. A high average answer score must not override an unacceptable access result. Keep a named owner, remediation step and retest date for every unresolved gap.

The NIST AI Risk Management Framework provides broader risk-management context. This test sheet is a narrow operational aid, not a claim of NIST certification or complete framework implementation.

Use the worksheet, then scope the connection

Get the AI Knowledge Access Test Sheet and run the same cases against the proposed setup. Request a demo to explore a bounded SimplBrain workflow and separately scoped SimplDev connections. Bring your permission model and failure cases, not just the list of systems you want connected.

Your working check