ProjectVue by Philips
AR Guidance for High-Stakes Clinical Decision
Executive Summary
The Mission: Find a meaningful clinical application for AR within Philips Healthcare's ecosystem with no predefined problem, user or scope.
"Turning a blank innovation brief into a patented AR guidance system for one of obstetrics' most cognitively demanding procedures."
The Challenge: Identify a genuine market gap within Philips' healthcare portfolio, validate it through rigorous clinical research, and build a working prototype that could earn the trust of PhD researchers, clinical experts, and international leadership — as a 3-month intern.
The Role: XR Designer and Research Lead. Owned the complete pipeline from strategic market research through clinical user studies through HoloLens prototype development. Collaborated with PhD researchers, Mixed Reality developers, and Philips international leadership.
Business Impact: Patent filed for EM tracking and alignment visualization mapping. Recognized as Philips Design India's Most Innovative Project of the Year. Secured funding for MVP development targeting emerging healthcare markets. Microsoft Mixed Reality partnership explored for expanded clinical use cases.
The Brief Was a Blank Page
Philips' Innovation Program gave me an open brief — explore how Augmented Reality could create value somewhere within Philips Healthcare's ecosystem. No defined problem. No target user. No scope. Just a technology and a healthcare giant with decades of products, markets, and clinical relationships to understand.
For most interns that would be overwhelming. For me it was the most interesting design challenge I'd ever been handed — because it meant the first deliverable wasn't a wireframe or a prototype. It was a question worth asking.
So before I touched any design tool, I did what most designers skip when they're eager to build — I researched. Deeply. Systematically. Like someone who understood that the wrong problem, beautifully solved, is still the wrong problem.
I spent weeks mapping Philips' entire healthcare portfolio — Radiology, Ultrasound, Critical Care, Patient Monitoring, Surgery, Clinical Informatics. I reviewed ten years of AR applications across seven industries — construction, finance, education, aerospace, automotive, warehouse operations, and healthcare specifically. I mapped where competitors like Siemens, GE, and Lumina were playing and where nobody was.
And then I found it.
A clinical procedure called External Cephalic Version — ECV — where a breech baby is manually repositioned in the womb before delivery. High stakes. Time constrained. Cognitively demanding. And completely untouched by any AR solution anywhere in the market.
The most valuable design skill isn't knowing how to solve a problem. It's knowing which problem is worth solving.
ECV is one of obstetrics' most technically demanding procedures. When a baby is positioned feet or bottom first — breech — in the final weeks of pregnancy, doctors attempt to manually rotate the baby from outside the mother's abdomen. It requires precise spatial judgment, careful force application, and real-time guidance from ultrasound imaging.
The existing Philips ultrasound workflow showed a 2D image on a screen. Everything else — the spatial reasoning, the mental mapping, the procedural confidence — lived entirely inside the doctor's head, built from years of experience.
For an experienced OB-GYN that's manageable. For a midwife in training, or a doctor in a resource-limited setting, it's an enormous cognitive burden under genuine time pressure.
That was the gap. Not a missing feature. Not a UI problem. A fundamental mismatch between what the technology showed and what the human brain needed to act confidently and safely.
I brought this finding to Philips' Head of Design. The conversation that followed changed the entire scope of my internship.
Within weeks the project had grown beyond a single intern's brief. PhD researchers joined to tackle the 2D to 3D mapping mathematics. The Head of Design engaged on strategy. The Netherlands team was notified.
What started as an open innovation exploration had become a focused, funded research initiative with patent implications.
I didn't pitch an idea. I brought evidence of a gap that mattered and let the clinical reality make the case.
Before I could design anything meaningful I had to understand something most designers never get close to — what actually happens inside a clinical procedure at the moment of highest cognitive demand.
I ran a research program that went significantly deeper than standard UX interviews.
Literature Review Ten years of academic and clinical research on ECV outcomes, failure rates, training methodologies, and ultrasound guidance protocols. This gave me the clinical vocabulary and the evidence base to have credible conversations with medical professionals.
Expert Interviews — OB-GYNs, midwives, and sonologists. Not usability interviews. Clinical workflow interviews. I needed to understand not just what they did but how they thought while doing it — the mental models, the decision points, the moments of uncertainty.
Lab Visits — I went into the ultrasound lab to observe procedures firsthand. Watching a doctor work in real time revealed things no interview could surface — the body language of concentration, the way eyes moved between the screen and the patient, the verbal cues between team members during a procedure.
Diary Studies — I asked doctors to document their workflow in real time. This was the methodology that surfaced the most critical insight of the entire project.
What the diary studies revealed was deceptively simple and profoundly important:
The current Philips ultrasound workflow only showed 2D mapping of the fetal position. There was no tool to translate that 2D image into 3D spatial understanding.
Every ECV procedure, the doctor was doing complex 3D spatial reasoning entirely in their head — converting a flat screen image into a mental model of where the baby actually was in three-dimensional space, under time pressure, with the mother's comfort and the baby's safety depending on that mental translation being correct.
The PhD researchers were working on the mathematical problem — how do you accurately map 2D ultrasound data into 3D space? My job was the human problem — how do you present that 3D mapping in a way a doctor can trust and act on in the middle of a procedure?
Bodystorming and Roleplaying — I physically acted out ECV scenarios to understand the spatial and ergonomic constraints of the procedure. Where are a doctor's hands? Where are their eyes? What information do they need and when do they need it?
DIY Ultrasound Phantom — Working with the research team I built a physical phantom — a simulated body model — to test prototype interactions in a realistic procedural context. Not on a screen. In a space that approximated the clinical environment.
This research phase didn't just inform the design. It earned me credibility with the PhD researchers, the clinical advisors, and the Philips leadership who would eventually evaluate the work. You can't design for a clinical environment by staying at your desk.
The insight wasn't in the interviews. It was in watching a doctor's eyes move between a 2D screen and a patient and understanding exactly what cognitive work they were doing in that gap.
With the research grounded and the problem precisely defined, I owned the entire design and build pipeline — from concept through prototype. This wasn't a project where I handed off to developers. I was the designer, the prototyper, the build lead, and the research director simultaneously.
Storyboarding — I started with 2D storyboards to map the AR experience conceptually. How does the 3D fetal model appear in the HoloLens field of view? How does it update in real time as the ultrasound probe moves? How does the doctor interact with it without breaking procedural focus?
3D Model Development — Translating 2D ultrasound scans into accurate 3D fetal models required collaboration with PhD researchers in MRI imaging. Multiple co-creation sessions to refine medical model conversion — ensuring clinical accuracy without sacrificing real-time performance.
New Desktop Workflow Design — The AR guidance system needed a companion workflow on the existing Philips desktop application. I redesigned the navigation and data flow to integrate AR output seamlessly into the existing clinical infrastructure — because a new tool that disrupts an established workflow doesn't get adopted regardless of how good the technology is.
HoloLens Prototype — Iteration 1 — The first prototype focused on visual representation. Real ultrasound data, rendered in three dimensions, visible through the HoloLens headset in the clinical space. No interaction design yet — just proving the core concept worked and that the 3D spatial visualization was clinically useful.
HoloLens Prototype — Iteration 2 — The second iteration added interaction and guidance. Audio cues — designed using Philips' own speech and audio design guidelines — to direct the doctor's attention during the procedure without requiring them to look away from the patient. Interchangeable dummy data for training scenarios. A fully interactive AR experience grounded in real clinical workflow.
Tablet AR Application — A parallel prototype using Vuforia image recognition for settings where HoloLens wasn't available. This surfaced important technical constraints — marker size limitations restricted the 3D model scale — informing future development priorities.
I built all of this. Not alone — I collaborated with Unity developers to bring the HoloLens integration to full fidelity, and with the Mixed Reality team to explore expanded use cases including virtual clinical training rooms. But I owned the vision, the design decisions, and the research foundation that made every build decision defensible.
I didn't just design what it should look like. I built enough of it to prove it could work and then collaborated with the people who could make it work at full scale.
When the Prototype Became a Patent
There's a moment in every project where you realize the work has outgrown its original container. For Confident Care that moment came when the Netherlands team flew in.
The presentation I gave wasn't a portfolio review. It was a full MVP demonstration — research foundation, clinical rationale, prototype iterations, technical architecture, market opportunity, and future roadmap. I stood in front of Philips' international leadership and walked them through a system that hadn't existed six months earlier.
The response wasn't polite applause. It was strategic action.
Philips filed a patent for the EM tracking and alignment visualization mapping system — the core technical innovation that made real-time 3D fetal guidance possible. The patent wasn't filed under my name — it was filed under Philips, as is standard for innovation program work. But the concept, the clinical application, and the design framework that made it patentable came directly from the research and prototyping work I had led.
The project was recognized as Philips Design India's Most Innovative Project of the Year — not most innovative intern project, most innovative project across the entire design organization. Funding was secured to advance the prototype toward an MVP for emerging healthcare markets where ECV complications are most prevalent and clinical training resources are most scarce.
A Microsoft Mixed Reality partnership was explored for expanded use cases — virtual clinical training rooms, remote procedure guidance, multi-user collaborative scenarios.
My 3-month internship became a 6-month extended engagement because the work demanded it.
And then budget constraints ended the program. Philips couldn't sustain the Mixed Reality investment at the scale the project required. I left having built something that the organization couldn't immediately continue — but also couldn't ignore.
The patent exists. The proof of concept exists. The clinical research exists. The market gap I identified in week one still exists.
The project didn't end because it failed. It ended because it succeeded faster than the organization could follow.
What It Proved. What It Started.
ProjectVue taught me things about design that no classroom and no standard product brief could have.
Research is a strategic tool, not just a user tool. The diary studies and lab visits didn't just inform the interface — they gave me the clinical credibility to work alongside PhD researchers, present to international leadership, and have my design decisions taken seriously in a room full of domain experts. Research isn't preparation for design. It is design.
Ownership changes everything. Because I owned the full pipeline — research, concept, storyboard, prototype, development collaboration, stakeholder presentation — every decision connected to every other decision. There were no handoff gaps. No lost context. No "that's not my part." That end-to-end ownership is what allowed a 3-month project to produce patent-worthy output.
Complexity deserves craft. ECV is one of the most cognitively demanding procedures in obstetrics. The design response had to match that complexity — not simplify it away, but make it navigable. That philosophy has shaped every complex system I've designed since.
What I'd do differently:
Earlier usability testing with practicing midwives in low-resource settings — the primary target market for the MVP. The prototype was validated with clinical experts but the real-world adoption questions for emerging markets were left for a future phase that never came.
A more formal technology transfer plan. The patent was filed but the pathway from prototype to clinical product wasn't fully mapped. Next time I'd design that transition as deliberately as the product itself.
What this project started:
A career built on the belief that the most important design problems aren't the ones someone hands you — they're the ones you find by looking where nobody else is looking.
ECV was invisible to the AR market in 2018. Financial health was invisible to most AI products in 2024. Teacher efficiency was invisible to most edtech platforms in 2023.
That's the thread. Finding the gap. Building the conviction. Making it real.
Most Innovative Project of the Year wasn't the achievement. Finding the problem worth innovating on — that was the achievement.