Samsung Medison’s NerveTrack1 is an innovative ultrasound feature used to identify nerve structures in real time during the application of anesthesia. NerveTrack owes its accuracy and speed to AI models developed through a collaborative process between doctors and AI engineers.
Training of NerveTrack’s deep learning inference models requires thousands of annotated ultrasound reference images. The image annotation process is best performed by doctors with years of medical training and experience in identifying the small, elusive nerve structures; however, it can be difficult and time-consuming for those doctors, partly because annotation tools and methods are typically designed for computer engineers and data scientists who ultimately use the images to train the models.
To improve the annotation and modeling workflow and collaboration between the teams, Samsung Medison worked with the Intel® Geti™ platform; the computer vision AI platform’s intuitive user interface enabled a small group of doctors to annotate tens of thousands of images in just a few weeks prior to handing them off to the AI engineer team for model training.
The Challenge: Including Medical Experts in the AI Model Training Process
Samsung Medison, a global manufacturer of medical equipment, developed NerveTrack as a real-time assistant for doctors who administer ultrasound-guided regional anesthesia (UGRA) and pain management using the company’s high-resolution ultrasound machines. Anesthesiologists and other medical specialists watch the ultrasound display as they place their syringes at the nerve location, and NerveTrack helps the doctors to identify the nerve structures more quickly and accurately.
The NerveTrack model is powered by deep learning inference models that enable the system to identify nerve structures in real time. Initially, NerveTrack’s custom AI inference model was trained to detect nerves only in the wrist. To broaden its application, Samsung Medison focused next on expanding its capabilities so an AI inference model could also identify peripheral nerve structures in the elbow, shoulder, and neck.
Peripheral nerves are especially difficult to detect because they are very small and thin. “Noise” or artifacts in an ultrasound image can mask the nerve itself. Nerves don’t all look alike, which makes it critically important during the image annotation process to rely on the expertise of anesthesiologists and neurologists to identify nerves in a wide variety of images. Without the annotations, a deep learning–based AI model might not identify the nerves correctly or differentiate the tiny, organic structures from noise in the image. Nerves can also move relative to other structures in the body, so the ultrasound images are typically captured as a series of video frames.
Due to the small size, complexity, and variability of the nerve structures, Samsung Medison needed hundreds of thousands of ultrasound images to be annotated as references to train each NerveTrack model. AI engineers are not able to annotate the images themselves, however, as these computer experts lack deep knowledge of human anatomy. Instead, the annotations must be made by medical experts, such as anesthesiologists, neurosurgeons, and other clinicians, whose years of study and experience enable them to recognize and identify the nerve structures accurately.
A key challenge for Samsung Medison’s expansion of the NerveTrack application was to find time with the busy doctors and specialists who possessed the needed expertise for image annotation.
The Solution: Intel® Geti™ Platform Puts Power in Doctors’ Hands
For the first version of NerveTrack, Samsung Medison had gathered hundreds of thousands of ultrasound images of nerves in the wrist. The annotation methodology relied on open source and proprietary tools that were designed by and for AI engineers, and the doctors doing the annotation needed significant technical support.
For the next version of NerveTrack—which would require the addition of new inference models for nerve structures in the elbow, shoulder, and neck—Samsung Medison’s AI engineers knew they needed to collect tens of thousands of additional annotated images to construct the most accurate models.
Because of the challenges faced in developing the initial version of NerveTrack, Samsung Medison started exploring other solutions that could streamline and simplify the annotation process. For the next round of annotations, the medical equipment manufacturer turned to Intel’s intuitive, easy-to-use Intel Geti platform.
Intel provided Samsung Medison with early access to the Intel Geti platform to help simplify the annotation process for the development of their second version of NerveTrack inference models. Samsung Medison began by testing the new Intel Geti platform–enabled annotation process on a limited data set of only 13 ultrasound images for a proof of concept. Several medical experts participated in the successful proof of concept and found the Intel Geti platform to be intuitive and easy to use with its point-and-click graphical user interface and AI-enabled annotation assistants.
The Intel Geti platform offers annotation assistants that allow users to draw shapes and label images easily. After annotating a subset of images, the platform provides annotation predictions. The user accepts or corrects the predictions as needed, streamlining the labeling process. Those intuitive capabilities enabled the doctors to annotate images of nerve structures in the elbow, shoulder, and neck quickly and accurately with little need for technical support.
The Intel Geti platform’s active learning helps automate data annotation and minimizes the number of annotations required to train models. The active learning feature provided AI annotation predictions for Samsung Medison in their NerveTrack project, which translated into a faster and less laborious annotation process.
Figure 1. The image on the left represents ground truth, the target of the NerveTrack machine learning model. The image at right is annotated with the Intel Geti platform for inference model training proof of concept, based on 13 annotated ultrasound images.
Scaling the Intel Geti Platform Across Hospitals and Providers
Satisfied with the test results, Samsung Medison moved forward with the Intel Geti platform to collect tens of thousands of new annotations in only two months.
Several hospitals in South Korea contributed their ultrasound video image data to the project. Doctors were able to work independently and contribute their individual areas of expertise and knowledge. When all the annotated images were submitted, the variety of approaches helped reduce bias in the resulting data set. The expectation was that this would help to make the new NerveTrack model more accurate and adaptable to a greater variety of end users.
Intel® Technologies Support AI Inference, Optimization, and Deployment
The final data set of annotated images of nerve structures in the elbow, shoulder, and neck were then sorted and processed by Samsung Medison’s AI engineers and programmers to create the reference image database for the deep learning framework. That framework was used to train the second phase of the NerveTrack model and develop its AI inference engine for those nerve sites. Using the Intel® Distribution of OpenVINO™ toolkit, the model was optimized for the Intel® Core™ i3 processor that runs NerveTrack on the targeted ultrasound machine.
With support from the Intel Distribution of OpenVINO toolkit, these new NerveTrack models will be deployed to Samsung Medison’s high-resolution ultrasound machines following approval by regulatory agencies.
To train and deploy the NerveTrack models for the elbow, shoulder, and neck, Samsung Medison leveraged several Intel® technologies in an eight-step process:
- Obtain ultrasound images and videos
- Create an individual Intel Geti platform project for each part of the body
- Recruit medical experts to annotate small sample data sets by hand with platform
- Train individual models with the Intel Geti platform
- Use the Intel Geti platform’s active-learning feature for AI annotation predictions to annotate remaining image data
- Consolidate all data sets from each project in all hospitals using a REST API and train models with the consolidated data set
- Optimize and deploy the models using the Intel Distribution of OpenVINO toolkit
- Incorporate the new models into NerveTrack
About Samsung Medison NerveTrack
Samsung Medison, an affiliate of Samsung Electronics, is a global medical equipment company manufacturing ultrasound systems. Samsung Medison is known worldwide for its R&D capabilities and innovative use of advanced technologies. The company’s NerveTrack model is an AI-enabled inference model of human nerve structures that can serve as a reference for doctors who use Samsung Medison ultrasound devices to guide needle placement for local anesthesia and other treatments.
About Intel Geti Platform
The Intel Geti platform offers a single seamless graphical interface to label, train, and optimize machine learning models. The user-friendly platform uses active learning to make data annotation and model training intuitive and fast.