Carrum Health: Staff Applied AI Product Engineer

As a Staff Applied AI Product Engineer, you will bridge the gap between cutting-edge AI capabilities and real-world patient needs. You will take a hands-on leadership role in architecting and integrating LLM-driven features into our core platform. While our foundation is built on Ruby on Rails (SOA) and React, you will lead the evolution of our stack to support AI-native workflows and build agents to support key business initiatives such as improving care navigation quality and efficiency.

You will work closely with Data and Product teams (including a dedicated Applied AI PM) to determine use cases that require LLMs rather than ML, build robust RAG (Retrieval-Augmented Generation) pipelines, select appropriate foundation models, iterate on prompts, and design the "glue" that turns raw model intelligence into reliable product features. You will partner with our DevOps team to leverage AWS/Azure for simple and cost-effective cloud infrastructure deployment, ensuring that systems are both performant and resilient. A key responsibility will be implementing comprehensive monitoring and observability practices to guarantee the high availability, security, and scalability of our new AI services. Additionally, you will collaborate directly with our internal clinical experts to validate model outputs, ensuring your AI agents are safe, accurate, and truly helpful in a healthcare setting.

As a senior technical leader, you will own the AI engineering strategy, moving us beyond "demos" into scalable, production-ready systems. You will establish patterns for building “smart products,” model evaluation, latency optimization, and cost management, while ensuring we maintain strict data privacy and HIPAA compliance. You will champion Responsible AI by operationalizing the internal guidelines set by our AI Council, ensuring that fairness checks, bias detection, and safety guardrails are strictly implemented to serve all patient demographics equitably. You will mentor the wider engineering team on AI-native development practices (such as prompt engineering and context management) and collaborate with product managers to identify high-leverage opportunities where AI can radically simplify the healthcare experience. You will partner with the VP of Engineering and Chief Product Officer to staff and align delivery teams on your initiatives, while working with product teams to define project timelines and milestones from inception to go-live.

This is a full time position, the salary range for this role is $190,000 - $260,000 depending on level of experience and geographic location.

You’re excited about this opportunity because you will...

• Define the Gen AI technical roadmap: You will build the foundation of Applied AI function at Carrum and have a direct impact on how Carrum leverages Generative AI to automate complex healthcare coordination and improve patient outcomes.


• Build "magic" features: Move fast to prototype, iterate and ship AI-powered experiences that feel magical to users, such as instant answer bots or automated appointment logistics.


• Architect for the future: Be the sole  expert on integrating vector databases, orchestration frameworks (like LangChain), and LLM APIs into a mature Service-Oriented Architecture.


• Bridge the gap: Act as the translator between the "stochastic" world of AI models and the "deterministic" world of software engineering, ensuring reliability and trust.


• Lead without ego: Mentor talented full-stack engineers on how to incorporate AI tools into their workflows, lifting the technical ceiling of the entire team.


• Own the outcome: Take ownership of t