Introducing Measure AIde!

A recent report by the American Hospital Association details growing concerns over the financial stability of hospitals and healthcare systems across the United States. Facilities face constant cost growth and insufficient reimbursement. Despite raising costs in drugs and supplies, labor remains the largest expense, accounting for 56% of total expenses in 2024. This financial strain threatens hospitals’ ability to provide high-quality, timely care to the communities they serve.

Adding to these pressures is the requirement to comply with numerous federal and local quality measurement reporting programs. While these initiatives aim to improve patient care and accountability, they also impose significant administrative burden. Much of the work to extract, validate, and submit these metrics is done manually and falls to existing healthcare staff or external consultants. A 2023 study at Johns Hopkins Hospital in Baltimore, Maryland, found that reporting 162 required quality measures consumed an estimated 108,000 person-hours annually, the equivalent of 52 full-time employees, at a cost exceeding $5 million per year in personnel expenses.

Although various vendor tools exist to automate this work, and CMS has developed Electronic Clinical Quality Measures (eCQMs) to enable automated data extraction from EHR systems, many hospitals continue to rely heavily on manual reporting for one or more of the following reasons.

• Poor and incomplete results. Unstructured EHR data such as narrative notes, dictations, and scanned documents remains a major obstacle. Because EHRs were designed for clinical workflows rather than quality measurement, much of the data is inconsistently structured or difficult for automation tools to interpret.
• Lack of trust. Automation tools often produce unreliable results which has lead to widespread skepticism among clinicians and staff.
• Privacy, security, and compliance concerns. Many automation tools depend on external data transfers or cloud-based processing, creating concerns about how protected health information (PHI) is accessed, stored, and transmitted.
• Workflow disruption. Automation solutions often require clinicians to document data in specific fields or templates, adding time and disrupting established habits. When perceived as burdensome or unnecessary, staff resistance is inevitable.
• Cost of technology and implementation. Healthcare software is expensive to purchase, integrate, and maintain. Although manual abstraction is labor-intensive, it represents a known and distributed operational cost.
• Interoperability gaps and data integration. Many hospitals operate multiple EHR platforms or participate in networks (such as ACOs) with disparate systems, making it difficult to aggregate quality data. Compiling eCQM reports across different EHRs often involves manually merging and deduplicating patient records which is a technically complex and time-consuming process.
• EHR vendor limitations and slow feature rollout. Hospitals depend on their EHR vendors to deliver eCQM capabilities, but vendor updates and enhancements often lag behind regulatory and operational needs.

Measure AIde is a project focused on automating healthcare quality measure reporting. Its goal is to deliver accurate results while reducing the workload and cost associated with manual reporting. To accomplish this, the project is being built around six guiding principles designed to overcome the reasons that reporting automation has not seen widespread success to this point.

1. Openness

Measure AIde will be developed and released as open-source software, ensuring transparency, collaboration, and broad accessibility. Open source offers several key advantages. It allows healthcare organizations to freely adopt and evaluate the system without vendor licensing costs. It builds trust by enabling independent review and verification of the codebase. And it supports extensibility, empowering facilities with unique workflows, data structures, or integration needs to adapt and extend the platform to fit their environments while contributing improvements back to the community.

The Measure AIde source code and documentation will be available on GitHub.

2. Data Diversity

Ignoring unstructured data is an unrealistic limitation imposed by many automation solutions. Measure AIde will treat unstructured data as a first-class citizen along with structured data when calculating quality measures. This will be accomplished through the use of specialized Large Language Models (LLMs) and natural language processing techniques to extract and structure meaningful insights from free-text and image-based records, ensuring that no clinically relevant information is ignored in the measurement process.

3. Local First

Measure AIde will be designed to run entirely within a healthcare facility’s local environment to support privacy, security and compliance concerns. This approach will shape the system’s architecture and implementation in many important ways. The following are a few key examples of how this will be reflected in the design.

• Simplicity and ease of deployment are critical since hospital IT teams, not external vendors, will manage the system.
• Accounting for environments with limited GPU availability.
• Providing flexibility with integrated systems such as databases, authentication systems, and internal networks, accommodating diverse IT environments without requiring major reconfiguration.

4. Progressive Automation

Healthcare facilities vary significantly in data quality, workflows, and readiness for automation. Measure AIde will implement a progressive automation framework that offers a tiered path toward full automation. Facilities can start with semi-automated workflows that include human-in-the-loop validation and correction, ensuring that quality teams remain engaged in verifying outputs, refining mappings, and adjusting logic as needed. This approach will foster confidence and transparency while enabling gradual, data-driven improvements. It will also be effective for processing unstructured data through non-deterministic methods powered by large language models (LLMs), where automation can be automatically scaled according to the confidence level of the results.

View the mockups for additional details and visual examples of this framework in action.

5. Modularity

The measure processing pipeline will consist of three primary stages: data extraction from source systems, data normalization to reconcile local vocabularies and system-specific implementations, and measure calculation using a canonical chart model. MeasureAIde will adopt a modular architecture that enables hospital systems to integrate or substitute solutions at each stage based on their specific requirements. This approach promotes flexibility, adaptability, and interoperability across diverse technical environments, supporting hospitals with varying levels of system maturity and infrastructure. It also enables healthcare facilities using less common EHRs or backend systems to benefit from the platform by only implementing a piece of the overall process.

6. Community Participation

The goal of Measure AIde is to create a collaborative platform where healthcare facilities can define, validate, and share quality measure calculations with other organizations through a common ecosystem. Given the vast number of federal, state, and local measures across the United States, it is impractical for any single organization to manage or define them all. By decentralizing this responsibility through community participation, Measure AIde can enable a more scalable and responsive approach to quality measurement which would allow hospitals and health systems to collectively advance measure development and accelerate adoption of new standards.

We recognize that the goals of this project are highly ambitious, and achieving them will require addressing complex technical challenges. The following are a few key examples.

• Managing large medical charts with LLMs. Developing methods to process records that exceed an LLM’s context window, whether through Retrieval-Augmented Generation (RAG) or alternative chunking and summarization techniques, will be essential for accuracy and scalability.
• Developing a measure domain-specific language (DSL). The platform requires a formal DSL to define, version, and execute quality measures in a structured, machine-interpretable way. This DSL will allow community engagement and decentralization of quality measure implementation.
• Establishing evidence and confidence scoring. We must design a framework for ranking evidence strength and quantifying model confidence, including methods for LLMs to evaluate and communicate their certainty in specific outputs.

We are actively seeking healthcare partners to collaborate with us in bringing this solution to life. If your organization is interested in participating or learning more please contact us at info@measureaide.org.