Adaptive Ultrasound Imaging with Physics-Informed NV-Raw2Insights-US AI

Back to Articles Adaptive Ultrasound Imaging with Physics-Informed NV-Raw2Insights-US AI Enterprise + Article Published April 28, 2026 Upvote - Walter Simson wsimson Follow nvidia Jay Carlson jaydcarlson Follow nvidia Tom Lassiter tslassit Follow nvidia Kevin Woo kevwoo Follow nvidia Sean Huver shuver Follow nvidia Introduction Raw2Insights Deployment System Capabilities Closing Perspective Introduction Ultrasound is one of the most widely used medical imaging modalities due to its safety, real-time capability, portability, and low cost. For decades, ultrasound images have been formed using a hand-engineered reconstruction pipeline that compresses rich raw sensor measurements into a final image while also making simplifying assumptions about physics, including a constant speed of sound throughout the body. In the era of AI and foundation models, a natural question emerges: can we move beyond the traditional beamforming pipeline, learn directly from raw ultrasound sensor data, and make use of information that is normally discarded during reconstruction? And if so, what new capabilities does that unlock? NVIDIA and researchers from Siemens Healthineers teamed up to find answers to these questions. The result of this work is a reconstruction model we are releasing called NV-Raw2Insights-US. Raw2Insights At its core, ultrasound is not an image—it’s sound. What clinicians ultimately see on the screen is a reconstructed picture built from millions of tiny echoes returning from the body. But in that reconstruction process, much of the original signal—the richness of how sound actually moved through tissue—is simplified or lost. Our approach starts earlier. Instead of working from finished images, NV-Raw2Insights-US learns directly from the raw signals captured by the ultrasound probe—the closest representation of how sound truly interacts with the body. This allows the model to “listen” more carefully and understand how each patient uniquely shapes those sound waves. Our vision is to enable end-to-end AI for ultrasound imaging and this is the first step towards that vision. We call this class of models Raw2Insights. In this first Raw2Insights application we estimate speed of sound for adaptive image focusing. The result is a system that can generate a personalized map of sound speed for each patient—and use it to correct the image in real time. What once required complex, time-consuming computation is now performed in a single AI pass. This is the shift from raw ultrasound channel data to actionable insight: an AI system that doesn’t just process ultrasound images, but actively understands and adapts to the physics of each individual patient. Deployment Typically raw ultrasound channel data is not easily accessible on clinical grade ultrasound scanners due to its high-bandwidth. Holoscan Sensor Bridge (HSB), is an open source FPGA IP developed by NVIDIA that allows high-bandwidth low latency data transfer to the GPU via (RDMA over Converged Ethernet). An Altera Agilex-7 FPGA development kit paired with NVIDIA Holoscan Sensor Bridge enables raw ultrasound channel data streaming from an ACUSON Sequoia ultrasound scanner’s DisplayPort outputs. We call this technology Data over DisplayPort. The NVIDIA HSB then packetizes the data and transmits it over Ethernet to NVIDIA IGX for data collection and AI inference. This demonstrates how modern high-performance computational capacity can be integrated with existing scanner architectures using…

This excerpt is published under fair use for community discussion. Read the full article at Hugging Face - Blog.