Our team and collaborators have developed PanDerm, a large-scale multimodal foundation model for clinical dermatology. PanDerm is trained with self-supervised learning on more than 2 million real-world skin disease images collected from 11 clinical institutions worldwide, spanning four imaging modalities: close-up clinical photographs, dermoscopic images, histopathology slides, and total-body photographs. Unlike previous single-task or single-modality systems, PanDerm is a unified model that supports a broad set of real-world clinical workflows.
Key Results
In extensive evaluations across 28 clinical tasks, PanDerm demonstrated state-of-the-art performance across multiple clinical scenarios:
- Skin cancer screening and early melanoma detection
- Risk stratification and prognosis (e.g., risk of recurrence or metastasis)
- Differential diagnosis of common and rare skin conditions (128+ entities in some reader studies)
- Lesion segmentation, mole counting, and longitudinal monitoring of lesion change
- Skin type assessment and other auxiliary imaging tasks
Critically, PanDerm often achieves best-in-class results using only 5-10% of the labeled data typically required, highlighting its suitability for scenarios where expert annotations are scarce.
Multimodal, Workflow-Centric Design
PanDerm's key innovation is its ability to jointly process multiple imaging modalities, mirroring how dermatologists synthesize information from clinical close-ups, dermoscopic views, histopathology slides, and whole-body imaging. This multimodal design allows holistic analysis of skin disease, instead of focusing on a single image type or narrow task.
In practice, PanDerm serves as a clinical decision-support tool that provides diagnostic probability estimates and visual assessments, assisting clinicians in triage and screening, diagnostic refinement, monitoring changes over time, and assessing risk of progression or metastasis. PanDerm is consistently designed to augment rather than replace clinician judgment.
Real-World Relevance and Equity
- Resource-limited and primary care settings: PanDerm can significantly aid non-specialists, helping close dermatology expertise gaps where access to dermatologists is limited.
- Robustness with limited labels: Strong performance with very limited labeled data is a key practical advantage when curated dermatology datasets are small or imbalanced.
- Global collaboration and generalizability: The model is trained on data from multiple countries and institutions, supporting broader generalization across populations and care environments.
- Future work: Ongoing efforts focus on standardized evaluation protocols, cross-demographic performance assessment, and equitable behavior across age, sex, skin type, and geography before large-scale deployment.
Broader Ecosystem and Next Steps
PanDerm has been described as a blueprint for medical foundation models. Its success is driving development of a Unified Phenotype Foundation Model (UPFM) at Monash, aiming to generalize the multimodal approach from dermatology to whole-patient, multi-organ phenotyping across cardiovascular, neurological, and skin diseases. PanDerm is currently in an evaluation and translation phase, with further clinical validation and workflow integration underway before broad clinical rollout.
In the News
- Giving doctors an AI-powered head start on skin cancer — Monash University
- Giving doctors an AI-powered head start on skin cancer — University of Queensland
- New AI system for analysing skin diseases tested — MedUni Vienna
- The Latest AI Developments in Dermatology — Mayo Clinic Platform
- Monash project to build Australia's first AI foundation model for healthcare — Healthcare IT News
- Giving doctors an AI-powered head start on skin cancer — Medical Xpress
- AI boosts skin cancer diagnoses, study says — US News & World Report
- Multimodal AI technology boosts early detection of skin conditions — News-Medical.net
- New AI can improve non-derma's ability to diagnose skin conditions — MobiHealthNews