Interactive Workflow Management for Surgical Operations
Surgical operations are often the most risky and complex procedures in the medical field. With human lives at stake, every instrument needs to be prepared and positioned correctly, and, needless to say, surgeons find themselves needing to act with what anyone else would call “surgical” precision. Our client`s mission is to build the best digital technology and put it in the hands of surgical teams. The company focuses on real-time surgical procedure data acquisition and management, providing step-by-step interactive playbook for optimal teamwork in the operating room.
To help the Client enable their vision, our team delivered a proof of concept for an AI-based software solution that can recognize surgical instruments from a video stream. Working together with our Client`s specialists, our team has prepared a dataset and trained a model for surgical instrument detection, and extended this model to develop and build an application capable of:
- Detecting all surgical objects on a video footage
- Classifying all of the detected objects
- Tracking objects on the screen and producing events based on the changing of their respective positions
This application is to become an essential component in the interactive workflow management software for use during surgical operations.
The most innovative products on the market are results of validated learning, so as ExpORer Surgical. The first prototype contained full guidelines for checking surgical instruments, including AI-based abilities that allowed it to ensure all procedures were followed in real-time, and all instruments properly prepared for safe use. Then, with AI capabilities already developed, we were able to identify new opportunities and automate the work of the conveyor line that prepares surgical instruments for operations.
Our team prepared datasets, developed data transformation pipelines, trained models, developed and tested applications using the following technology stack:
- Pytorch, Scikit-learn, Resnet, OpenCV, MLflow, InceptionNet - For preparing and training image recognition models
- Numpy, Pandas, Matplotlib - For preparing and transforming the data
- Pillow, LabelMe - For image organization and processing