Assessing developing possible version of "HHSGPT" for OIDP-specific internal use

Section 1: Use Case Identifiers

Use Case ID: HHS-OASH-00001
Agency: HHS
Op Div/Staff Div: OASH
Use Case Topic Area: Mission-Enabling (internal agency support)
Is the AI use case found in the below list of general commercial AI products and services?
None of the above.
What is the intended purpose and expected benefits of the AI?
OIDP's HIV.gov: an exploration of possibly using a custom knowledge-based walled/isolated version of "Chat GPT" for our internal OIDP users to access information to help with internal operational efficiencies related to our work on national strategies. We have been meeting for almost 1 year with senior HHS officials working on AI to keep them up to date on our approach and efforts in both RAG LLM and generative AI. We describe our work as an internal library. The tool is called Ask ABE! (v3.0.0-alpha.1) - AI-Driven. Beneficial. Equitable (ABE). It's like having a digital librarian who can help us at OIDP easily access and use all the valuable resources from HIV.gov to support our work in ending new HIV transmissions! Our work is critical to ensuring the following issues are addressed: (1) Security: FedRAMP certified Azure Open AI 3.5 Turbo / 4o Global. (2) Privacy: Not tracking conversations nor usage. PII stripped. (3) Equity: HIV Language and HIV Language Guideline through internal system prompt. (4) Hallucinations: Internal system prompt - ABE HIV.gov knowledge base and conversation history. There is a low risk of this occurring compared to open systems like ChatGPT. (5) Risk: NIST AI Risk Management Framework: Govern, Map, Measure, Manage for Responsible AI, Observable AI, and Explainable AI (6) Certifications / Training / Support: Azure AI Certifications / Data Science DoJo / Corporate Thought Leadership Support: Responsible AI Framework.
Describe the AI system's outputs.
OIDP internal federal staff would possibly be able to ask questions, develop content, and revise existing content using a unique dataset of OIDP information. Inclusion of NIH HIV guideline is embedded in the internal prompting.
Stage of Development: Initiated
Is the AI use case rights-impacting, safety-impacting, both, or neither?
Neither

Section 2: Use Case Summary

Date Initiated: 09/2023
Date when Acquisition and/or Development began: N/A
Date Implemented: N/A
Date Retired: N/A
Was the AI system involved in this use case developed (or is it to be developed) under contract(s) or in-house?
N/A
Provide the Procurement Instrument Identifier(s) (PIID) of the contract(s) used.
N/A
Is this AI use case supporting a High-Impact Service Provider (HISP) public-facing service?
N/A
Does this AI use case disseminate information to the public?
N/A
How is the agency ensuring compliance with Information Quality Act guidelines, if applicable?
N/A
Does this AI use case involve personally identifiable information (PII) that is maintained by the agency?
N/A
Has the Senior Agency Official for Privacy (SAOP) assessed the privacy risks associated with this AI use case?
ongoing

Section 3: Data and Code

Do you have access to an enterprise data catalog or agency-wide data repository that enables you to identify whether or not the necessary datasets exist and are ready to develop your use case?
N/A
Describe any agency-owned data used to train, fine-tune, and/or evaluate performance of the model(s) used in this use case.
N/A
Is there available documentation for the model training and evaluation data that demonstrates the degree to which it is appropriate to be used in analysis or for making predictions?
N/A
Which, if any, demographic variables does the AI use case explicitly use as model features?
N/A
Does this project include custom-developed code?
N/A
Does the agency have access to the code associated with the AI use case?
N/A
If the code is open-source, provide the link for the publicly available source code.
N/A

Section 4: AI Enablement and Infrastructure

Does this AI use case have an associated Authority to Operate (ATO) for an AI system?
N/A
System Name: N/A
How long have you waited for the necessary developer tools to implement the AI use case?
N/A
For this AI use case, is the required IT infrastructure provisioned via a centralized intake form or process inside the agency?
N/A
Do you have a process in place to request access to computing resources for model training and development of the AI involved in this use case?
N/A
Has communication regarding the provisioning of your requested resources been timely?
N/A
How are existing data science tools, libraries, data products, and internally-developed AI infrastructure being re-used for the current AI use case?
N/A
Has information regarding the AI use case, including performance metrics and intended use of the model, been made available for review and feedback within the agency?
N/A