About this role
Scarlet’s Devices function is a team of clinicians, AI experts, and software engineers, working together to assess and certify the most innovative and impactful medical device software. We pride ourselves on delivering fast and efficient assessments to enable market access, new device updates and ongoing surveillance of a growing portfolio of medical devices. As we continue to scale our activities and certify more and more medical devices, we need additional medical AI expertise to provide support across all stages of the customer journey. You will play a key role in evaluating whether AI model performance testing, datasets, and clinical evidence are appropriate to support clinical benefits and performance claims under medical device regulations.

YOUR RESPONSIBILITIES

• Critically evaluate AI/ML performance testing methodologies, outcome measures, datasets, and statistical analyses to determine whether they adequately support clinical and performance claims
• Communicate complex statistical and AI/ML concepts clearly to internal teams, manufacturers, and customers
• Deploy your expertise externally by representing Scarlet at events and conferences
• Screen and action insights from the latest research on AI in healthcare
• Work with our Product, Engineering, Design and Applied ML functions to build and improve our systems

THE KEY SKILLS

• Education - A PhD in the area of AI in healthcare or Master’s (or higher) in Health Data Science or AI-focused medical statistics
• Work experience - Experience evaluating AI/ML model performance metrics, validation methodologies, and training/testing datasets in healthcare or regulated environments
• Work experience - Experience critically assessing whether statistical analyses and performance testing appropriately support clinical benefit and performance claims
• Work experience - Experience communicating complex AI in healthcare topics to non-technical audiences

DESIRABLE SKILLS

• Research experience - You have experience of designing or contributing to the development of interventional and non-interventional clinical studies
• Real world data - You have experience appraising or analysing real-world data and observational studies, including familiarity with bias frameworks (e.g. ROBINS-I, GRADE for observational evidence)
• LLM-native - You have experience with the deployment and evaluation of LLMs in healthcare
• Ferociously curious - You like going down rabbit holes, understanding deeply how things work, and challenging the status quo
• Problem-solver - You can identify and define scientific problems, defend a scientific position and ideate pragmatic solutions
• Highly adaptable - You have worked in different environments and like operating with autonomy on sometimes ambiguous tasks

THE INTERVIEW PROCESS

• Intro call with Sandy - 30 mins
• Interview with Yun or Ed - 30 mins
• Technical interview with Mihir - 1 hour
• Culture and values interview with James and Jamie - 2x30 mins
• Referencing & offer