The President’s Management Agenda calls for using automation software to improve efficiency of government services to focus on emerging technologies including machine learning and autonomous systems. OMB is pursuing an Acquisition Modernization Plan that leverages leading edge technology to enable the acquisition workforce to shift its attention to higher-value work and focus more on activities that prioritize mission. This includes the Past Performance Evaluation performed by the acquisition workforce: a tedious, labor intensive process using the Contractor Performance Assessment Reporting System (CPARS).
Problem Statement:
CPARS is the central system through which the federal government collects and makes available contractor/vendor past performance information. However, anecdotal evidence suggests that the volume of records and the inability to rapidly identify relevant reviews, make it difficult for contracting officers to easily utilize the data for past performance evaluations. An AI solution for CPARS may significantly assist contracting officers conducting past performance evaluations by identifying relevant past performance assessment records in a more accurate, consistent, faster and useful manner, reducing the administrative workload on contracting officers while increasing the quality of the outcome.
Meet CORMAC’s Envisioning and Prediction Enhancing System (CREPES)
Thanks to our customer who articulated their pain point in a meeting, an idea was sparked, and we began building a solution in our Innovation Lab (iLAB). CORMAC’s CREPES, is a Software-as-a-Service (SaaS) product built in the Cloud that uses Machine Learning, NLP and Data Visualization to automate and augment the past performance solution. This was a byproduct of our Data Science Practice built to save COs and Evaluation Panels across the Federal government time and energy.
When using CREPES for performance searches, COs receive an unbiased list of contractors ranked by relevance of their past contracts to the current requirement (scope, dollar value, and period of performance). In addition, CREPES provides visual performance ratings and lets COs conduct detailed reviews if needed. COs can drill down into the full text of the CPARS or view the highlights identified through sentiment analysis, a Natural Language Processing (NLP) technique.
The CREPES product automatically meta-analyzes past performance evaluations. It employs artificial intelligence (AI) to help COs find past performance data quickly and efficiently. We use both factorized data and text data to locate and evaluate vendor performance records. That means CREPES uses both ratings and open-ended narratives to uncover performance data. Beyond the geek stuff, it’s simple and enjoyable to use.
CREPES’ Summary Reports page displays a snapshot of relevant records and allows users to drill down into each vendor’s information to read the full evaluation.
Collaboration with Federal Government
The DHS Procurement Innovation Lab (PIL) partnered with us on CREPES development beginning in 2019 to determine how AI could improve access to insights from CPARS. DHS awarded a multiphase contract under Artificial Intelligence for Past Performance Prototypes & Pilots Acquisition Contract. The CREPES product was evaluated to see how this could augment the existing past performance evaluation. CREPES product uses various AI methods such as Machine Learning (ML), Unsupervised Latent Dirichlet Allocation (LDA), Manifold learning and Natural Language Processing. CORMAC used various NLP and ML libraries for text and sentiment analysis to automate the labor-intensive past performance evaluation. CREPES blends information engineering with textual data using neural networks to summarize and predict which vendors are likely to succeed when awarded a contract. It utilizes in-memory computing on Amazon Web Services (AWS) to scan thousands of performance records and return the relevant vendor(s) based on specified criteria.
Where We Are Today
CORMAC is currently supporting Phase II, the goal of which is to achieve a production-ready, federally-accredited SaaS solution that could be used across the federal government. We continue to iteratively develop CREPES, primarily focusing on improving user experience by implementing Human Centered Design (HCD) and Design Thinking. DHS has arranged user group sessions with users across various federal agencies. In these user group sessions, we give a walkthrough of the CREPES tool and solicit feedback from users in real-time. These sessions provide direct insight into users’ needs, goals, and pain points. Through feedback from user group sessions, we have prioritized feature enhancements including increasing customization of displays and reports, configuration of roles and permissions, and more. We also continually train our AI model and refine its predictive accuracy by feeding it more and more data, ever improving explainability and transparency for user confidence.
Where We Are Headed
Acquisition professionals have recognized the power of using AI for CPARS. The General Services Administration (GSA) has planned a multiple-award, government-wide acquisition vehicle focused on CPARS AI solutions—and CREPES is a perfect fit.
We continue to explore additional use cases to expand CREPES functionality, including a module for market research and potential integrations with publicly available data to give users access to more centralized insights on potential vendors. We are also building a module to automate administrative processes such as collecting emails, storing documents and extracting DUNS numbers from the submitted responses.
Ultimately, we’re thrilled for the opportunity to help contracting officials make more efficient and effective use of CPARS data and improve the evaluation process for everyone involved!