PhD Candidate Prompt-based AI for Radiotherapy Segmentation

Accurate delineation of tumor target volumes is a critical step in the radiotherapy workflow. While deep learning models have successfully enabled automated segmentation of organs-at-risk, the delineation of tumor targets remains a major challenge due to large interpatient variability. Moreover, in contrast to healthy organs, delineation of tumors requires integration of clinical context such as tumor stage, surgery reports, and patient-specific risk profiles. Recent work has shown that large language model (LLM)-driven prompt-based segmentation can outperform vision-only approaches.

In this PhD project, you will develop and clinically translate next-generation multimodal AI models for prompt-driven segmentation, using clinical text and 3D imaging data in tandem. The goal is to move beyond static image-to-image auto-contouring and instead create dynamic, context-aware systems that adapt to each patient’s scenario.

During your PhD, you will work on the following work packages:

WP1: Development of prompt-based segmentation models

Inspired by LLMSeg (Oh et al., Nat. Commun. 2024) and nnInteractive (Isensee et al. 2025), you will build multimodal segmentation networks that integrate clinical prompts with image features. You will investigate fine-tuning strategies (e.g., prompt tuning, LoRA) and evaluate model performance in terms of accuracy, data efficiency, model uncertainty, and robustness across institutions.

WP2: Clinical scenario modeling and explainability

You will design methods for generating structured clinical prompts from medical records and evaluate how prompt variations affect model behavior. You will also develop explainability tools to support clinical evaluation, such as attention heatmaps and prompt-response diagnostics.

WP3: Clinical deployment in image-guided radiotherapy

You will translate the developed models to clinical settings, working closely with radiation oncologists and RTTs to evaluate feasibility in daily practice. Integration into adaptive radiotherapy workflows (e.g., MR-guided online re-planning) will be explored in our institutional roadmap for AI-enhanced radiotherapy.

WP4: Interactive AI for clinician-in-the-loop segmentation

To support clinical usability, you will develop interactive correction tools that allow radiation oncologists to quickly adjust AI-generated contours using minimal input (e.g., positive/negative scribbles or brush strokes). These user inputs will be incorporated into the segmentation network through guided refinement, allowing the AI to recalculate the target and OAR volumes in real-time based on clinician feedback. This approach supports a human-in-the-loop workflow

The Department of Radiotherapy at the University Medical Center Utrecht is the birthplace of the 1.5 T MR-Linac and a world leader in MR-guided radiotherapy. The department hosts over 40 PhD candidates working on topics ranging from advanced imaging physics to clinical implementation. This project will be part of the IMAGINE program and contribute to shaping the future of personalized, automated cancer care.

• You are an ambitious and creative candidate with an MSc in Artificial Intelligence, Biomedical Engineering, Technical Medicine, or a related field.
• You have strong programming skills (Python/Matlab), experience with deep learning and medical imaging, and a clear interest in AI applications in oncology.
• Familiarity with radiotherapy workflows or clinical data is a plus.
• You are proactive, collaborative, and capable of translating novel ideas into practical solutions.
• Strong communication and writing skills and the ability to publish in scientific journals are essential.

• A salary between € 3108 and € 3939 gross per month (salary scale OIO), based on full-time employment (36 hours).
• Year-end bonus of 8.3% and holiday allowance of 8%.
• Pension insurance with ABP. We take care of approximately 70% of the monthly contribution.
• 100% public transport reimbursement. Are you coming on foot, by bike or by car? You will then receive a reimbursement of € 0.18 per km (by car up to a max. of 40 km one way).
• Possibilities to develop yourself personally and professionally.
• The option to select additional employment benefits in exchange for gross salary, such as purchasing a bicycle and memberships.