In rural Tennessee, these cancer doctors see patients via hologram

On a recent weekday, I stood in front of a phone-booth-sized box—seven feet tall, four feet wide, and two feet deep—talking to a life-sized, three-dimensional image of a doctor who was 2,000 miles away in Tennessee. This wasn’t a Zoom call: It felt like he was in the room.

West Cancer Center, a Memphis-based healthcare and research group, is the first in the U.S. to begin using “hologram” technology, from a startup called Proto, for some of its doctor visits.

“We wanted to address discrepancies in cancer care in rural communities,” says Julie Flanery, the director of marketing and patient communication for the cancer center. “We have 11 clinics, and we wanted to be able to offer our specialty oncology care to patients without making them have to drive to urban markets. And this has been able to do that.”

In a clinic in Paris, Tennessee, a small town of 10,000, some cancer patients now use the tech to meet with doctors in Memphis. (When I tested it out, Proto had temporarily installed the tech as part of a conference at U.C. Berkeley’s Haas School of Business.)

Doctors say it’s possible to build a connection with patients that wouldn’t be doable in a typical telehealth meeting on someone’s phone. “With what I do— palliative care—communication is our procedure,” says Dr. W. Clay Jackson, an oncologist at West Cancer Center. “So, my body language is very important. How I sit, how I have my hands, if I’m engaged or not, all of those things are critical.”

In the past, Jackson says he never did new consultations using telehealth tech. “I felt like our care discussions were too complex to be done over the phone or by cellphone video for the initial consultation,” he says. “But now, I actually do remote consults in a rural area.”

Proto, the company that developed the technology, initially focused on entertainment. It spun out of Musion, a company known for using special effects to create posthumous shows with hologram-like versions of Frank Sinatra or Tupac. Founder David Nussbaum wanted to go farther, “beyond the novelty factor,” he says. He started working on the current tech in his living room, with help from engineer friends. The final large version now sells for $65,000.

The design uses a transparent window with a layer of clear 4K LCD resolution, and an empty box behind it to create depth. In the case of the clinic in Tennessee, a patient sits in an exam room with the box and a local staff member from the clinic. On the other end of the connection, the doctor sits in front of a camera—any 4K camera works, like an iPhone—with a plain-colored wall behind them. The tech’s software places the doctor on a white background inside the box, and adds shadows that move with them.

“The effect of shadow and reflection and the volumetric nature of it is part of the production process, so that shadow appears on the back wall,” says Nussbaum. “Those reflections appear on the floor. So it’s like my feet are touching the ground. It’s like I’m presenting inside of a space, because this is exactly what you would expect to see if a person were actually physically there.”

The comedian Howie Mandel, a self-described germaphobe, invested in Proto and has used the devices to do remote shows so he can avoid traveling. Some retail stores have used a smaller version of the tech to let customers quickly browse through 3D images of products that aren’t in stock. (With a tap on the screen and a swipe, it’s possible to turn the object and look at it from every angle.) NASA uses it when 3D printing devices, looking at a model before starting the printing process. At medical schools, the devices are used to let students look at detailed 3D models of body parts, like the heart. It can be used in a variety of different ways, though it could be particularly helpful for medical appointments.

“A doctor can be anywhere,” Nussbaum says. “The patient could be in the most rural area in the world and we can create the effect of physical interaction, physical and emotional connection where there may be thousands of miles of distance in between them.”

In India, a clinic is interested in using the tech to beam detailed versions of patients in remote areas to specialists elsewhere. That’s the opposite of the clinic in Tennessee, which beams the doctor to the patient. With a clear view of the patient, remote diagnosis is easier; a doctor could more easily give instructions to a nurse or assistant who is onsite with the patient in a remote area. Proto is also working on AI tech that can replicate a doctor’s appearance and then answer questions, in the doctor’s voice, in any language.

Of course, it’s not the same as actually having a doctor standing next to you. In the U.S., the Association of American Medical Colleges projects that there could be a shortage of 86,000 doctors by 2036. More people need to go to medical school, though the healthcare system—including the bureaucracy of health insurance—likely needs to change to convince enough potential doctors to actually join the profession. The shortage is especially acute in rural areas. But technology might help improve care while the gap exists.