ENTERPRISE-LEVEL TESTING The Department is actively seeking an enterprise-level testing partner to play a pivotal role in optimizing the efficiency and reliability of its MES. The primary goal of this collaboration is to provide augmented staff support in overseeing all testing activities across designated modules and vendors within the MES. The selected testing vendor will assume ownership of the User Acceptance Testing (UAT) test plan and its execution, ensuring a comprehensive testing framework that aligns with the program's objectives. The testing contractor shall also be responsible for managing the defect tracking and resolution processes, ensuring a proactive approach to identifying, documenting, and resolving issues that may arise during testing phases. The partnership will extend to active oversight and participation in module vendors' master test plans, fostering collaboration and alignment with the broader testing strategy. The testing contractor shall provide assistance to the State in gaining insight into module vendors' release management and code versioning, contributing to a seamless transition between software versions and adherence to best practices. Collaborative efforts extend to working closely with the State to ensure system integration and readiness before release to production, emphasizing a coordinated approach for optimal functionality. Lastly, the testing contractor will assume responsibility for oversight and assistance in meeting system federal certification requirements, ensuring that testing processes align with federal regulations and standards. This comprehensive approach emphasizes the dedication to improving the MES infrastructure with the ultimate goal of enhancing healthcare services for the residents of Colorado. Data Conversion Contractor shall be responsible for data validation when data is migrated, converted, Extract, Transform, and Load process (ETL) or moved to the UAT region. Contractor shall provide all test results from data validation for Department review and approval, prior to initiating functional testing activities. Contractor shall provide test plans, test cases, and test scripts for data migration, data conversion, and Extract, Transform, and Load process (ETL) testing and coordinate with the System Integrator (SI) as directed by the Department. Contractor shall verify that data is formatted correctly as per the specifications provided by the module vendor. Contractor shall document all data migration, conversion, and ETL test results during UAT data validation, as well as any deviations that have been discovered. Contractor shall submit a separate data migration and conversion report for Department review and approval, which includes all test results along with the number of defects, defect status, and any new Change Requests necessary for data migration or conversion to the UAT region. Contractor shall describe how its proposed services perform testing that complies with security standards and incorporates industry best practices in order to provide effective testing, test success tracking, and defect reporting. Contractor shall describe how its proposed services incorporate ownership of UAT responsibilities for system verification and production readiness. Contractor shall provide a Department-approved deliverable tracking method to ensure all project-related deliverables have been accounted for, scheduled, and coordinated with the Department’s Project Management Office. All deliverables shall be approved by the Department. Contractor shall document the approach to testing for each MES module in accordance with the UAT and End-to-End (E2E) Test Plans. Contractor shall utilize a defect tracking tool for the management and reporting of all defects identified in the UAT region. Contractor shall specify the solution functionality that is in and out of scope for testing as part of the UAT and E2E Test Plans in collaboration with and upon approval from the Department. Contractor shall define the test strategy as part of the UAT and E2E Test Plans, including objectives and required types of testing for each of the testing activities. Contractor shall specify, as part of the UAT and E2E Test Plans, each of the facilities and tools to be used, contractor resources required, and Department resources required, for each testing cycle. Contractor shall receive written approval from the Department before making a change to the contractor's test plan or test cases. Contractor shall participate in the development and execution of defect resolution testing or Corrective Action Plans (CAP) within Department-defined timeframes. Contractor shall allow the Department staff to access the testing tool used for creating test cases and defects. Contractor shall be required to train and educate their staff in support of the Department's MES solutions. This includes web-based and in-person education in various settings including provider association meetings, on-site, and advanced (scheduled) training sessions as approved by the Department. Defect Management Contractor shall create a Defect Management Plan that documents how system defects are identified, tracked, reported on, resolved, and documented. Contractor shall implement and maintain a defect management process to monitor system issues, report statistics related to system quality, and track defect resolution. Contractor shall propose solutions for all issues, problems, and defects identified. Contractor shall document all issues, problems, and defects identified. Contractor shall track and report weekly on the defects identified and the progress made toward resolution of defects, including metrics on number of tests completed, number deferred or canceled, results of the tests executed, defects identified by severity, and corrections undertaken. This report should display a reconciliation of all open and closed defects. Contractor shall perform end-to-end testing for all defects identified and provide testing results. Contractor shall submit the number of defects identified and corrected along with their severity ranking after each test cycle and upon request by the Department. Contractor shall provide a monthly status report which summarizes system changes, including enhancements and defect resolutions. Contractor shall utilize a defect tracking tool for management and reporting of system defects. Contractor shall provide authorized users access to the defect tracking tool. Contractor shall explain the process used to establish the exit criteria (e.g., numbers and types of defects, defect severity/priority), as part of the UAT and E2E Test Plans. Contractor shall define the defect resolution process as part of the UAT and E2E Test Plans. Contractor shall describe the format and content of test progress and defect reports as part of the UAT and E2E Test Plans. Contractor shall describe as part of the UAT and E2E Test Plans, the approach to testing at all levels when defects are resolved. Contractor shall submit the number of defects identified and corrected along with their severity ranking after each test cycle and upon request by the Department. Contractor shall perform retesting for all defects identified and provide testing results. Contractor shall track all defects in the system UAT environment through tracking tool(s) and documentation until the defect is resolved. Contractor shall develop a Test Evaluation and Management Plan (TEMP) which defines the Contractor’s testing vision, including the approach to testing, testing objectives, and the tools/processes and workflows used. Additionally, the TEMP shall define the test monitoring and controlling activities, defect management, and overall deployment activities. Contractor shall establish the testing tool environments including the implementation of an automated testing and defect/issue tracking tool. Contractor shall warrant that the system is operating as designed and all defects as evidenced during implementation have been addressed and are fully remediated to Department's satisfaction. Contractor shall provide and maintain a Defect/Issue Log. Contractor shall deliver a tool that provides, at a minimum: Online accessibility to track identified defects and issues both local and remote. The status of each defect/issue regarding findings, percentage of completion, and resolution. Documented results of the correction efforts and the result. The prioritized status of defects/issues. Summary and detailed reporting on all defects/issues in Excel supported format. Supporting attachment capabilities (e.g., scanned documents, documents, and spreadsheets). The ability to cross-reference each defect/issue to the RFP requirements or the Requirements Traceability Matrix (RTM). Integration with the technical and non-technical artifact management repository. Integration with the workflow tool to route and alert the necessary staff members.
