General Requirements Runs in existing university managed Microsoft Azure tenant. Follows Azure AI Foundry best practices. Offeror shall provide Virginia Tech access to product source code with ability to develop and deploy extensions or modifications. Maintain version history of product with the ability to roll back to previous versions. Allow for system prompt optimization to improve user experience, especially for university-centric queries. Self-service retrieval augmented generation (RAG) support. For example, allowing faculty and students to provide their own documents and data to supplement the selected LLM. Multiple data source support for RAG allowing self-service direct connection to various platforms (e.g. Microsoft OneDrive, Google Drive, Amazon S3). Centrally managed access to selected university data sources (e.g. Snowflake, SQL databases). Allow users to search and pull data from sources on the web. Image generation (e.g. based on model selection), with capabilities for image editing and in-painting. Video generation (e.g. based on model selection). Monitoring and Management Requirements Allow for federating and managing LLM resources and related configuration on Azure. Ensure resiliency and high availability of the ecosystem for all users. Monitor model usage with near-real time reporting and dashboards. Apply quotas for model usage for rate limiting and/or cost control. Monitor/audit user queries and user-provided data sources. Archive and/or purge all history and other data associated with a particular user (e.g. for account deprovisioning). Reporting and Billing Support Requirements Detailed reports of usage across multiple dimensions (e.g. user, group, department, model, time, etc). Integration to Azure cost management.
