Project Duration 1 Week
Team Members Orlagh O’Brien, Lochlann O’Regan, Edward Conmy & Tessy Thomas
Team Members Orlagh O’Brien, Lochlann O’Regan, Edward Conmy & Tessy Thomas
Key roles Conducting interviews / Design research / Design development / Visual + user interface design
Skills involved Interaction design / user research / user journey mapping / prototyping / usability testing / service design / conducting interviews & workshops / visual design / UI/UX Design
Tools used Figma / FigJam / Paper prototyping / Adobe Illustrator
⚠️ Content Warning: This case study includes medical imagery (bowel and colorectal surgery) that some viewers may find distressing. Viewer discretion is advised.
Visualisation of the redesigned gesture-based UI for colorectal surgeons
Brief
When removing a segment of bowel, colorectal surgeons need to make accurate and fast decisions. Prof. Ronan Cahill and his team at the Mater Hospital have a working prototype of an interface which has been trained to identify the ideal incision points using previous imaging data (i.e. images of the bowel with ICG introduced shows surgeons the ideal points for making those incisions) that then aid surgeons with making these decisions based on the same.
Our brief was to research and redesign this interface to make it more usable, and position it seamlessly within the workflow in a way that gives confidence to the operating surgeon.
Research
To start, we conducted interviews with surgeons and attended surgeries to understand the exact flow and usability of the current software state. We got many insights from the surgeons themselves, including discovering what their main challenges were and what they hoped for from the tool.
“Ideally, it’s automatic, a totally seamless confirmation of the decision.”
– Colorectal Surgeon 1, Mater Hospital
“I’d prefer a range of safety, not just a single pixel line.”
– Colorectal Surgeon 2, Mater Hospital
Ideation
We started brainstorming how design could intervene to help make the process simpler as well as fit its usage within the workflow seamlessly. We did this by rapid brainstorming, testing and prototyping different hand gestures as well as other sensor/gesture-based interactions which could help solve this.
We recognised feedback as critical. Surgeons needed immediate, clear responses from the system. A co-design workshop with 10 participants enabled us to test and iterate concepts. We mapped the surgical workflow, refined gesture use, and explored camera input to support faster incision planning. Our process was guided by tight feedback loops to ensure usability in a sterile, high-pressure environment
In addition to redesigning the interface to be more minimal and clear, we identified two key factors for integration into the software to significantly improve the level of assistance that the software could provide for surgeons.
01 We recognised the need to incorporate a feedback mechanism so surgeons would know the software is registering their actions
02 We introduced a guide to the different gestures to maintain recall and minimise any confusion
Motion gesture guide created in an icon form for easy identification and recall while using the software
The main changes we made to the interface were to introduce a clear, feedback mechanism which allowed surgeons to see their gestures being recognised and to add a gesture guide. We also cleaned up the interface of the software to make it more minimal and straightforward. Our revisualisation of the UI brought down the number of actions that surgeons had to carry out to mark the incision sections on the bowel significantly.
Improved interfaces: The redesigned interface of the loading screen along with the new guide for motion gestures. Clear instructions telling surgeons what actions needs to be taken next results in improved efficiency and seamless workflows
Motion detection feedback:The new interface has a guide that remains on the screen to improve surgeons' usability; clear indicators of colour changes when certain motions are detected will inform them with clarity and boost confidence while using the software
Improved section visualisation: Once the two points are selected, the software draws out a clear line and by removing the previous version's coloured sections, provides one optimum section for the surgeon to make precise incisions.
Impact
By introducing a guided gesture system and feedback mechanism, we reduced the steps to seven, significantly improving efficiency. This not only enhanced the surgeons' workflow but also reduced potential errors, directly impacting patient safety.
✏️ A huge thanks to my team: Orlagh O’Brien, Lochlann O’Regan, and Edward Conmy, who I worked with to conduct research, interviews, workshops, and ultimately redesign and reimagine the software in new, improved ways. Also many thanks to the team at the Mater Hospital for giving us the opportunity to embark on this project and help expand the potential of the great work they are doing.
