Image Segmentation

Segmentation is the task of extracting regions of interest from images. Given a medical context like in our group, this usually means that in images from medical modalities (e.g. magnetic resonance imaging, computed tomography, or ultrasound), organs have to be delineated for quantitative measurements or shape analysis. Likewise, manifestations of disease, such as tumors, need to be traced for tracking disease status and assessing treatment effects.

Image segmentation has always played a major role in research at CIAN. In earlier projects, motion segmentation was used as part of tracking the movement of organs induced by breathing, and facial muscles were segmented for maxillofacial surgery planning. Current projects include gray matter–white matter segmentation in the spinal cord for multiple sclerosis research, detecting and delineating multiple sclerosis lesions in the brain, developing an augmented reality device for lymph node identification in cancer patients, and reconstructing the cornea surface for eye surgery planning.

The approaches to solve our tasks are as various as the tasks themselves. Methods employed in our group reach from classical ones like variational segmentation to the more recent deep learning techniques.


Dr Simon Pezold
Head of Image Segmentation
Gewerbestrasse 14
4123 Allschwil

Neurological diseases like Multiple Sclerosis have impact on the spinal cord. In this project we want to study the effect of the disease on the different compartments of the spinal cord: gray matter and its surrounding white matter (GM/WM).

To isolate these compartments we segment magnetic resonance images of the spinal cord.
Because manual segmentations are tedious and have high intra- and interrater variability, we aim at implementing a deterministic segmentation process on the computer that is capable of robustly and accurately mimicing the manual process.

We collaborate with the Radiological Physics of the University Hospital Basel for developing image acquisition techniques that produce unblurred images with high contrast between GM/WM. The developed segmentation algorithm is then used in collaboration with the Department of Neurology to calculate longitudinal atrophy rates of Multiple Sclerosis patients.

Persons: Antal Horvath, Philippe Cattin

Optical Coherence Tomography (OCT) is an emerging modality in ophthalmology. However, the three-dimensional measurement of the eye by OCT is still limited, e.g. by eye motion. We develop new methods to enable robust three-dimensional ophthalmic measurements – required for reliable diagnostic and surgical planning.

This requires sophisticating scanning, segmentation and reconstruction methods. Sophisticated scanning reduces the effect of eye motion and ensures the coverage needed for reliable reconstruction of the eye structures. Appropriate segmentation handles the low signal-to-noise ratio of OCT and the dynamic appearance of the structures depending on the scanning and eye orientation.

Persons: Jörg Wagner, Philippe Cattin

The aim of the research project MOONSTAR (Mobile Optical Navigated SPECT Camera with Augmented Reality) is to develop a novel approach to improve lymph node identification for biopsy by means of augmented reality (AR).

Squamous cell carcinoma in the head and neck region is one of the most prevalent cancers worldwide. Classic treatment of this malignancy involves the complete removal of the tumor together with the lymphatic basin (neck dissection). However, such a radical and complicated treatment is only required in one third of the patients. To prevent this overtreatment, sentinel lymph node biopsy is an important minimally invasive technique to improve the staging of cancer.

In order to find the relevant lymph nodes a radioactive tracer is injected into the patient's lymphatic tissue near the primary tumor. Simple one-dimensional gamma detectors are often unsuited to help the surgeon differentiating between lymph nodes, the tumor and surrounding tissue, all potentially activated by the tracer.

Our project supports the specialist to find and excise sentinel lymph nodes for further histologic analysis in order to assess the cancer. Optical images of the small biopsy incision are combined, augmented with gamma detector data from the tracer and displayed. This combination helps the surgeon to be more precise in finding sentinel lymph nodes and could stop overtreatment.

A recently conducted feasibility study shows the augmentation of an optical image of a vial, containing a radioactive liquid tracer, with the gamma activity data from that tracer. Three different distances of the source are shown in Figures 1–3. The good matching of the image overlays is explained by the underlying opto-geometric model necessary for the augmentation.

Persons: Peter von Niederhäusern, Philippe Cattin

Multiple sclerosis is the most common neurological disorder in young adults, with more than 2 million affected people worldwide. The disease causes formation of lesions in the central nervous system where the myelin sheaths of neurons are damaged. These lesions can be visualized with different MR sequences, depending on the individual development of each lesion. The detection and segmentation of those lesions offers an important diagnostic value in the medical practice and makes up a time consuming step in the development of new treatments, as large drug trials usually require the manual delineation of lesions in thousands of brain scans.

The so called lesion load is a measure often taken into account when conducting MS studies, where individual lesions are counted and the total amount is noted per patient. Another popular measure is the lesion volume, but due to the cumbersome process of acquiring it manually by segmenting it voxel-wise, it is not always gathered. Both of these measures are highly rater dependent, since two raters would either completely ignore or accept a lesion, as well as most likely disagree about the lesion boundaries of individual lesions if no clear guideline has been defined beforehand, resulting in a significant inter-rater variability.

Given an automatic measure, studies could be compared on the same grounds. We hence investigate different automatic methods to accomplish this task. Our most successful developments for brain tissue segmentation as well as lesion segmentation are both located in the area of deep learning, achieving state of the art segmentations in both domains.

Persons: Simon Andermatt, Philippe Cattin