Feature Matching Improvements Based on Hyperspectral Imaging

Stereo instrumentisan often used image based procedure in medical applicationsthatcurrentlydoes not provide any assistingfeature. Targeted use of stereo technology could makeassistance systems available to medical users. An essential componentof this is the correspondence analysis, which forms the basis for a 3D reconstruction. In order to create a dense depth map, featuresmust be found in both imagesof the stereo pairand assigned to one another with a matching score. Limitations in medicalin vivo image data are often reflections of the tissue surface due to the fluid filmand fast movements of the medical imaging tool. Theselimitations lead to gaps in the depth map and misinformation in the 3D reconstruction. In this work, we focus onremovingthese constraints. Therefore, it is necessary to develop anapplicable descriptor with the aid of ahyperspectral camera. Feature matching is often only donein grayscale converted RGB images. Hyperspectral images are a composite of several grayscaleimages that correspond to different wavelengths, in this studybetween 470 nm and620 nm. Under laboratory conditions, a HSI descriptor will be developed based on non-medical data. The higher information content of these HSI cubes (256x512x16 pixels) let expecttoreach robust features and better matching results compared to grayscaleimage matchingwhich will beevaluated in this study. By selective thresholding multiple bands are used to match features. Thus, bands that do not provide informationcan be discarded and matchingwill be done with the validbandsonly. This is not possible in the case ofconventional RGB camera sensors, as the interference affects all three bands. After final investigations under laboratory conditions, the HSI descriptor hasto be tested and evaluated with medical ex-vivo