3D Internal Anatomy Localization from External Body Geometry
Predicting 3D internal organ centroids and spatial uncertainty directly from optical surface scans and depth cameras using deep multi-scale cross-attention decoders. Developed at CAIR, IIT Mandi by Khushi Mhamane (Project Lead) & Sharon Melhi under the supervision of Dr. Deepak Raina.
Multi-Scale PointNet++ Encoder
Extracts hierarchical geometric surface features across SA1 (1,024 points), SA2 (256 points), and SA3 (64 points) scales, preserving local anatomical contours.
Target-Query Cross-Attention
104 learned organ queries attend to multi-scale body surface tokens, predicting spatial coordinate residuals relative to a canonical anatomical atlas.
3-Seed Ensemble Uncertainty
Predicts consensus centroids and computes root-mean-square seed disagreement across independent initializations, providing localized spatial confidence bounds.
Comparative Paradigm Analysis
How external geometric localization compares to standard diagnostic and procedural modalities.
| Modality | Ionizing Dose | Latency | Hardware Cost | Portability |
|---|---|---|---|---|
| Surface2Anatomy (Ours) | 0.0 mSv | < 100 ms | Optical / RGB-D Scanner | Bedside / Field |
| Diagnostic Whole-Body CT | 10 - 20 mSv | Minutes to Hours | \$500,000 - \$2,000,000 | Fixed Gantry Suite |
| Diagnostic MRI | 0.0 mSv | 30 - 60 Minutes | \$1,000,000+ | Shielded Magnet Room |
| Handheld Ultrasound | 0.0 mSv | Real-Time | \$5,000 - \$20,000 | Operator Acoustic Window |
Open Source Package & Checkpoints
Surface2Anatomy is packaged for production and research environments with zero placeholder weights.
Installs frozen PointNet++ multi-scale encoder, cross-attention decoders, and Ridge alignment models.
from surface2anatomy import SurfaceAnatomyModel
model = SurfaceAnatomyModel.from_pretrained()
result = model.predict("patient_surface.ply", target="spleen")
print(result.centroid_mm) # (-61.2, 34.8, 105.4)