NV-Raw2Insights-US: AI that adapts ultrasound to the patient | Keryc
Ultrasound imaging isn't a photo: it's reconstructed sound. What if, instead of working on the final image, we taught AI to listen to the transducer's raw signals and understand how your body alters those waves? That's exactly what NV-Raw2Insights-US proposes: a model-based reconstruction pipeline that learns from the source, not from the already processed version.
What is NV-Raw2Insights-US
NV-Raw2Insights-US is a reconstruction model for ultrasound that operates directly on the transducer's raw channel data. Rather than accepting the simplifications of the classic beamforming pipeline, the model estimates key physical parameters—starting with sound speed in tissue—to focus and correct the image in real time.
Why does sound speed matter so much? Because traditional systems usually assume a constant speed everywhere in the body. That assumption introduces focusing errors and reduces clarity. By estimating a patient-specific sound speed map, NV-Raw2Insights-US adapts the image to the real physics of each case.
Raw data and physics of sound
In ultrasound, what reaches the system are millions of echoes: voltage signals on every channel of the transducer array. The classic pipeline compresses and transforms those echoes into an image, losing fine information about phase, relative amplitude and dispersion.
NV-Raw2Insights-US works with that raw channel data. By learning from the original signal it can exploit information that was previously discarded: tiny timing variations, phase differences between channels and patterns that reflect tissue heterogeneities.
Technically, the system incorporates ideas from differentiable beamforming and supervised learning to estimate sound speed and apply focusing corrections in a single inference pass. That reduces the need for costly iterative optimizations and opens the door to latencies compatible with clinical real-time use.
Architecture and deployment: from FPGA to GPU at the edge
Transferring and processing raw data demands bandwidth and low latency. This is where Holoscan Sensor Bridge (HSB) comes in: an open-source NVIDIA FPGA IP that enables high-rate transfers via RDMA over Converged Ethernet.
In the technical demo, an Altera Agilex-7 kit captures the DisplayPort output of an ACUSON Sequoia scanner and uses a technique called Data over DisplayPort to extract the raw channel data. HSB packets that information and sends it over Ethernet to an NVIDIA IGX system for collection and AI execution.
Inference runs on Blackwell-class GPUs using the Holoscan platform, designed for real-time sensor workloads on devices like NVIDIA IGX Thor and NVIDIA DGX Spark. The flow is: data capture -> streaming to GPU -> accelerated inference -> sound speed map -> feedback to the scanner to improve live image focusing.
Results and technical capabilities
Patient-specific sound speed estimation in a single AI pass, replacing iterative procedures.
Real-time focus correction, which sharpens images and reduces artifacts introduced by incorrect physical assumptions.
Modular architecture that separates capture (FPGA/HSB), transport (RDMA/Ethernet) and compute (GPU/Holoscan). This makes experimentation and deployment easier in different clinical environments.
The publication cites relevant work, including differentiable beamforming methods and studies on sound speed estimation with deep learning, which provide scientific backing for the approach.
Clinical and development implications
What does this mean for clinicians, engineers and startups? First, a practical route toward AI-native imaging: instead of applying networks to already reconstructed images, you learn from the primary signal, reducing systematic errors.
For developers, NV-Raw2Insights-US offers a platform to experiment with models that combine physics and deep learning. NVIDIA publishes resources so you can get started, including links to repositories, model weights and research datasets.
For clinical practice, improved focus and contrast can help tasks like identifying small lesions, guiding interventional procedures and more accurate assessment of anatomical structures. Remember this is investigational technology and not approved for sale or general clinical use.
Technical resources and references
Article on differentiable beamforming: "Ultrasound Autofocusing: Common Midpoint Phase Error Optimization via Differentiable Beamforming", IEEE Transactions on Medical Imaging, 2026. https://ieeexplore.ieee.org/document/11154013
The project was developed in collaboration with Siemens Healthineers, with contributions from Ismayil Guracar and Rickard Loftman. The technical demo shows how to integrate existing infrastructure (DisplayPort outputs from clinical scanners) with edge compute to enable Raw2Insights pipelines.
Thinking of ultrasound as sound, not as an image, changes how we design models. NV-Raw2Insights-US is a first technical step toward systems that understand the physics behind the signal and use it in real time to improve clinical decisions.
