They don’t get fruitcakes or Christmas cards from grateful patients, but for decades robots have been helping doctors perform gallbladder removals, hysterectomies, hernia repairs, prostate surgeries and more. While patients lie unconscious on the operating table, robotic arms and grippers work on their bodies at certain stages in these procedures ― all guided by doctors using joystick-like controllers, a process that minimizes human hand tremor.

Now, a team of Johns Hopkins University and Stanford University researchers has reported a significant advance, training robots with videos to perform surgical tasks with the skill of human doctors.

The robots learned to manipulate needles, tie knots and suture wounds on their own. Moreover, the trained robots went beyond mere imitation, correcting their own slip-ups without being told ― for example, picking up a dropped needle. Scientists have already begun the next stage of work: combining all of the different skills in full surgeries performed on animal cadavers.

A new generation of more autonomous robots holds the potential to help address a serious shortage of surgeons in the United States, the researchers said.

Presented at the recent Conference on Robot Learning in Munich, the research comes almost four decades after the PUMA 560 became the first robot to assist in the operating room, helping with a brain biopsy in 1985.

The new work is currently undergoing review for publication in a journal. And the next-generation surgical robots will need to demonstrate safety and effectiveness in clinical trials, and receive approval from the Food and Drug Administration before they can become a fixture in hospitals.

While some studies have shown that robotic surgery can be more expensive to the overall health care system without performing significantly better than traditional surgery, a 2023 paper in AMA Journal of Ethics concluded that surgeons are becoming more experienced using the robots resulting in improvements.

Nonetheless, scientists and doctors are already touting the reliability, skill and increasing autonomy of surgical robots as an important step toward addressing a potential crisis. The combination of an aging population that will require more surgery, and a stagnant level of student doctors that has the United States on pace to experience a shortage of 10,000 to 20,000 surgeons by 2036, according to a report this year by the American Association of Medical Colleges.

“In our work, we’re not trying to replace the surgeon. We just want to make things easier for the surgeon,” said Axel Krieger, an associate professor at Johns Hopkins Whiting School of Engineering who supervised the research. “Imagine, do you want a tired surgeon, where you’re the last patient of the day, and the surgeon is super-exhausted? Or do you want a robot that is doing a part of that surgery and really helping out the surgeon?”

In 2020, the U.S. reported about 876,000 robot-assisted surgeries.

The robots used by Krieger and his colleagues were made from research kits supplied by the medical technology firm, Intuitive.

Ji Woong “Brian” Kim, a postdoctoral researcher working with Krieger, said the team has already developed a system “where you can talk to the robot like you would to a surgical resident. You can say things like, ‘Do this task.’ You can also say things like, ‘Move left’ and ‘Move right.’”

“In my mind, I thought they were still a couple of years behind what they have demonstrated here,” said Dipen J. Parekh, director of robotic surgery at the University of Miami Miller School of Medicine, who was not involved in the research.

But he stressed that many steps remain before robots are able to perform surgical procedures on their own.

“The stakes are so high,” he said, “because this is a life and death issue.” The anatomy of every patient differs, as does the way a disease behaves in patients.

“I look at [the images from] CT scans and MRIs and then do surgery,” by controlling robotic arms, Parekh said. “If you want the robot to do the surgery itself, it will have to understand all of the imaging, how to read the CT scans and MRIs.” In addition, robots will need to learn how to perform keyhole, or laparoscopic, surgery that uses very small incisions.

Teaching robots to learn by imitating actions on a video should reduce the need to program them to perform each individual movement required for a medical procedure, according to the researchers.

The team’s training method resembled the approach used in ChatGPT, except that instead of working with words, it employs a language that describes the position of the robot gripper and the direction it is pointing.

Researchers built their training model using videotapes of robots performing surgical tasks on practice suture pads. Each image in the video sequence is an arrangement of pixels that can be expressed in numbers. In simple terms, the model takes numbers that represent images and converts them into another set of numbers that represent different robot actions.

After training the robots, researchers produced a separate set of videos demonstrating that the robots could perform the surgical tasks in a different environment ― in pork and chicken.

“I thought it was very exciting. It’s the dawn of a new era,” said Amer Zureikat, who was not involved in the study but serves as director of robotic surgery at University of Pittsburgh Medical Center.

Zureikat too cautioned that the work, though “a significant first step,” must still overcome numerous hurdles. “The majority are logistical issues that should be rectified over time as artificial intelligence improves.”

Scientists and doctors will have to figure out how to handle common challenges of surgery, such as bleeding and improperly placed sutures.

“If a blunder occurs, who holds responsibility?” Zureikat asked. “Is it the doctor? Is it the AI developer? Is it the hospital facility? Is it the robot manufacturer?”

Privacy is also likely to emerge as a major issue. The robots discussed at the Munich conference were not trained using videotape of actual surgeries. However, they will need to train on videotapes of real surgeries if robots are to advance to the point where they can operate safely on their own. That will mean gaining permission from patients to have their surgical videos used to develop robot systems.

Zureikat said advances in the use of robot surgical equipment are likely to raise additional questions: “Are patients going to get equal access to the technology?’ and ‘Will surgeons rely so much on robots that they become less adept at performing surgery without them?”