Report Confidence Scoring

Imagine having intelligent and automated review of medical reports.

One crucial layer of the ARE4 system is our Confidence Scoring Engine. Here, the system analyzes the physician’s level of certainty or confidence of her report. This is done using machine learning, artificial intelligence and statistical algorithms.

Each concept is examined for the confidence level of the physician making the report. An overall confidence level is determined via our proprietary statistical analysis and artificial intelligence software components.

There are hundreds of such common phrases that can color the results and no true natural language understanding can exist without taking this critical concept of sentiment into account.

Many medical reports are filled with such language. Here are a few more examples: “most likely, could be, suggestive, resembles, indeterminate, appears, may represent, possible, and consistent with.”

The pie chart shows the proportion of medical reports that are either clear cut (presence/absence) or have some degree of hedging.

We break the language into “for sure” hedging (“yes”) and “maybe” hedging. It’s consistently about 20% of the medical report total, a signficant amount.

For example, here are phrases found in actual radiology reports we’ve processed. Note that confidence can go “both ways” in that the physician can be confident she doesn’t know and as well as being confident in what she does know.

Example Real-World Phrase Physician’s Confidence Score
1.5 cm nodule may represent a hemorrhagic cyst. 4
Stable 8 mm posterior right hepatic lobe lesion, likely a hemangioma. 8
A larger 1.9 cm echo resembles a small calcified intramural fibroid. 6