Keywords:Geographical environment; Zhoushan Islands; Emergency -Received: 28 Oct 2022 Accepted: 05 Nov 2022 Published: 11 Nov 2022 J Short Name: ACMCR
⦁ Abstract⦁ Background: The geographical environment of Zhoushan Is- lands makes it difficult for people to get standardized emergency treatment. Based on the current two hot technologies, we devel- oped “5G+AR” remote consultation equipment to improve peo- ple’s medical treatment. ⦁ Methods: We selected 11 major islands with primary health centers, equipped these primary doctors with AR glasses. We recorded relevant data of the consultation from June 1, 2022 to October 28, 2022, such as basic patient information (gender and age), consultation information (diagnosis, consultation duration, Consultation distance), data transmission information (network la- tency). We performed a descriptive analysis of these data. ⦁ Result: 3 emergency cases using AR glasses were observed. During the consultation of 3 patients, the consultant felt no ob- vious video lag, and the latency was in the range of 42 to 53. All patients were well managed and had a good prognosis. ⦁ Conclusion: From a technical point of view, 5G+AR remote consultation technology is feasible and safe.
⦁ IntroductionZhoushan islands is located in the southeast coast of China. Zhoushan is made up of 2085 islands, with a total area of 22,200 square kilometers, including 20,800 square kilometers of sea area and 1,459 square kilometers of land area. In 2021, the permanent population will be 1.165 million. Like many developing regions, -Zhoushan faces skewed distribution of health professionals toward urban areas and health worker shortages that do not aptly address the population’s real health needs [1]. In addition, the special ge- ographical environment limits the flow of population, thus exac- erbating the contradiction between medical supply and demand. This contradiction is particularly prominent in emergency patients. Emergency patients on some islands need to take more than 3 hours to reach the nearest medical center. This situation delays the treatment time of patients and increase medical costs. Traditional remote consultation is not real-time and requires a lot of preparation before consultation. It makes remote consultation of emergency patients impossible. Therefore, we need to find a technology to meet the needs of remote consultation for emergen- cy patients. Augmented reality (AR) is a rising technology with great potential for medical field. Current AR technology is mainly used in medical education and surgical navigation [2,3,4]. There is a study that used lumbar computer tomography scans (CT) of two cadaver specimens to reconstruct 3D model. Then planned trajectories and 3D models were subsequently uploaded to an AR head-mounted device. Randomly, k-wires were placed either into the left or the right pedicle of a vertebra (L1-5) with or without AR-navigation (by holographic projection of the planned trajecto- ry). They found improved angular precision in the AR-navigation group [3]. We also found many studies on the application of AR technology in surgery [3,5,6]. It benefits from the real-time guid- ance ability of AR technology, which can meet the needs of sur- gery. But there are no studies on the application of AR technology in remote consultation, which also need real-time. China is one of the first countries to complete 5G infrastructure. 5G technology has two obvious advantages: high speed and low la- tency. 5G networks provide a high data transfer rate at 10 GB/s [7]. 5G networks have developed a “network splicing/slicing” scheme that divides the network architecture into multiple networks spe- cialized in one specific function. Splicing allows the 5G network to optimize resources toward the relevant functions being used. This scheme allows 5G networks to achieve higher data transfer speed, communication, reliability, and ultra-low-latency than 4G networks [7,8,9]. The advantages of 5G network satisfy the huge demand of bandwidth and delay for AR telemedicine. In order to solve the medical problems in Zhoushan Islands and similar areas, our team designed and produced a “5G+AR” head-mounted portable device, and established a consultation net- work, hoping to improve the medical environment in remote areas through this new technology. To evaluate the feasibility and safety -of the technique, recent consultation data were collected, and de- scriptive analysis was performed. ⦁ Methods ⦁ Introduction of the AR Glasses Figure 1 is the appearance of our product. The hardware tech- nology and part of the software technology of the AR glasses are provided by the Rokid company. It is equipped with monocular AR optical waveguide technology with high resolution display, which provided the user with see through daylight readable dis- plays, offering the combination of very large eye box and excellent real-world transmission in a compact format [10]. At the center of the glasses, there is an 8-megapixel auto-zoom lens that supports 1080P HD video. There are headphones and microphones on ei- ther side of the glasses, which support voice control and have high resolution even in noisy environments. The host includes battery, GPS module and 5G network module. It built to last up to 8 hours. Our 5G network service is provided by China Unicom.
⦁ Introduction of Remote Consultation System The remote consultation system is developed by our team. It con- sists of three parts: glasses client, consultant client and consultant allocation system. The glasses client is used to initiate consulta- tions. Each pair of glasses can be voice controlled to call a tele- medicine at any time. The consultant client can be mounted on the mobile phone or computer for the consultant to join the virtual conference room. The consultant allocation system pushes corre- sponding consultants according to the needs of each pair of glasses users. At present， the team of specialist doctors participating in the consultation was provided by Sir Run Run Shaw Hospital. During the consultation, the consultant can view the patient and the examination report from the first-person perspective and guide the treatment by marking the concerns on the screen and voice communication. Figure 2 is the initial interface of glasses client for remote consultation. Figure 3 is the consultant client. In Figure - 4，doctor in primary hospital is using AR glasses to communicate with consultants ⦁ Introduction of Auxiliary Diagnosis and Treatment System In each pair of AR glasses, we have auxiliary diagnosis and treat- ment system. We loaded the diagnosis and treatment procedures recommended by various disease guidelines into AR glasses and made corresponding decision trees. Based on the principle of de- cision tree, the system helps users to complete diagnosis and treat- ment according to the disease process specification. At present, the auxiliary diagnosis and treatment system can only guarantee the early treatment of the disease, and is mainly used in emergency scenarios. We developed this project mainly to standardize the ear- ly treatment of first aid, as well as emergency treatment in extreme network conditions. In Figure 5, we captured part of the content of the trauma decision tree and the corresponding images displayed in the glasses.
⦁ Data Collection and Analysis We officially started the application of “5G+AR” glasses in Zhoushan Islands on June 1, 2022. We selected 11 major islands with primary health centers (Figure 6), equipped these primary doctors with AR glasses, and trained them to use them. We record- ed relevant data of the consultation from June 1, 2022 to October 28, 2022, such as basic patient information (gender and age), con- sultation information (diagnosis, consultation duration, Consulta- tion distance), data transmission information (network latency).
⦁ ResultIn this study, 3 emergency cases using AR glasses were observed. One was a hypertensive emergency patient. Under the guidance of the consultant, the primary physician gave the patient timely hy- potension and successfully lowered the patient to the target range (Table 1). The other one was a patient with a ruptured liver with a huge 10*7*7cm intrahepatic hematoma，intact liver capsule，un- stable vital signs. Doctors at local hospitals have a difficult choice between observational, surgical, or interventional treatment. So, they contacted chief physician of Sir Run Run Shaw Hospital for remote consultation. The chief physician read the film in real -time through the camera of AR glasses. The consultation assessed grade IV liver injury, and suggested interventional treatment. After treatment, the patient’s blood pressure was stable, liver function was preserved completely, and the prognosis was good. The last patient was a patient with a sudden lisp. The primary physician considered the patient to have a stroke without performing a CT scan and wanted to treat the patient with aspirin. But he wasn’t sure, so he called for a consultation. The consultant stopped him and suggested that CT should be perfected. After the completion of craniocerebral CT, the possibility of stroke was considered, but the time window for thrombolysis and interventional surgery was ex- ceeded. Standardized antiplatelet and lipid-lowering therapy was given. After standardized treatment, the prognosis of the patient is good. The consultants are located in two regions, one is General Hospital of Run Run Shaw Hospital in Hangzhou, Zhejiang province, and the other is Putuo Branch of Run Run Shaw Hospital in Putuo District, Zhoushan City, Zhejiang Province. During the consulta- tion of 3 patients, the consultant felt no obvious video lag, and the latency was in the range of 42 to 53 (Table 1). All patients were well managed and had a good prognosis.
⦁ Discussion In China’s primary health centers, most doctors are trained in gen- eral practice. They are not trained in emergency medicine. Even in some places, doctors have received professional emergency train- ing, but due to too little exposure, lack of skilled application of knowledge. First aid has always been a weakness of rural doctors. Since the COVID-19 outbreak, the need for remote consultations has increased due to reduced population movements [12]. Com- pared with the traditional remote consultation equipment, our equipment is more portable, and the simple process is easier to use. Our AR glasses are inexpensive and can be popularized even in re- source-poor places. The consultation doctors conduct the consulta- tion from the first perspective, with stronger immersion and more efficient. Moreover, when facing extremely poor network condi- tions, the auxiliary diagnosis and treatment system built into AR glasses can help users to carry out standardized early treatment. We collected the transmission data during the consultation, and one of the most important parameters was the latency. Latency is the response delay between a device and the hosting server or target device, which is affected by the data transfer rate of a network and the amount of data needed to be transferred [7]. In telemedicine, these characteristics are essential for the consultant to respond adequately to changing circumstances and the quality of the vid- eo stream to provide appropriate information. Inadequate time or poor video quality could lead to damage to patients. Since we are the first to conduct remote AR consultation, we do not have a refer- ence value. Therefore, we chose the latency of 5G remote surgery as a reference. In December 2018, Chinese researchers conducted -a 5G remote surgery experiment. The surgeon, who was about 48 kilometers away from the operating room, remotely controlled two robotic arms to remove a 2cm×3cm×3cm portion of the pig’s liv- er. The operation lasted about 60 minutes and the total blood loss was about 5 ml. In addition, the average latency is less than 150 ms [11]. Current successful 5G robotic surgeries have an average latency of 76-150ms. Some studies have shown that the response time of laparoscopic surgeons to complex situations is 397 ± 19 ms [13]. As a result, surgeons may face a potential response time of 475-660 ms when performing 5G robotic surgery [14-16]. Some researchers believe that 150 ms is the maximum safety latency up- per limit [7]. The maximum latency in our consultation process was 53 ms, which seems safe at present. In the recent trial, we have not observed the use of auxiliary diag- nosis and treatment system, which may be related to the short ap- plication time of our AR glasses. During the operation of the auxil- iary diagnosis and treatment system, we will record the completion time of each process node to compare it with the completion time of each key node in the traditional treatment process. We hope to use this study to evaluate the effectiveness of AR glasses assisted diagnosis and treatment system, which is one of the main contents of our later research.
⦁ Conclusion From the perspective of communication technology and hardware technology, “5G+AR” remote consultation is feasible and safe. It is relatively smooth in the use of ordinary patients at present, but its application in emergency patients needs further evaluation.
References 1. Scaling up, saving lives task force for scaling up education and train-ing for health workers. World Health Organization. 2008. 2. Barteit S, Lanfermann L, Bärnighausen T, Neuhann F, Beiersmann C. Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games. 2021; 9(3): e29080. 3. Spirig JM, Roner S, Liebmann F, Fürnstahl P, Farshad M. Aug-mented reality-navigated pedicle screw placement: a cadaveric pilot study. Eur Spine J. 2021; 30(12): 3731-3737. 4. Zhang W, Liu X, Zheng B. Virtual reality simulation in training endoscopic skills: A systematic review[J]. Laparosc Endosc Robot Surg. 2021; 4(4): 8. 5. Li T, Li C, Zhang X, Liang W, Chen Y, Ye Y, Lin H. Augmented Reality in Ophthalmology: Applications and Challenges. Front Med (Lausanne). 2021; 8: 733241. 6. Goh GS, Lohre R, Parvizi J, Goel DP. Virtual and augmented reality for surgical training and simulation in knee arthroplasty. Arch Orthop Trauma Surg. 2021; 141(12): 2303-2312. 7. Pandav K, Te AG, Tomer N, Nair SS, Tewari AK. Leveraging 5G technology for robotic surgery and cancer care. Cancer Rep (Hobo-ken). 2022; e1595. 8. What is 5G | everything you need to know about 5G | 5G FAQ | Qual-comm [internet]. Qualcomm. 2020. 9. Ahmadi S. 5G network architecture. In: 5G NR [Internet]. Elsevier. 2019; 1-194. 10. Alexander A, Cameron. Optical waveguide technology and its appli-cation in head-mounted displays, Proc. SPIE 8383, Head- and Hel-met-Mounted Displays XVII; and Display Technologies and Appli-cations for Defense, Security, and Avionics VI. 2012; 83830E. 11. Liu R, Zhao G, Sun Y, et al. Animal experiment for 5G remote robot-ic surgery[J]. Chinese Journal of Laparoscopic Surgery (Electronic Edition), 2019. 12. Wang Y, Yang J, Ma H, Dong X, Xie G, Ye S, Du J. Application of telemedicine in the COVID-19 epidemic: An analysis of Gansu Province in China. PLoS One. 2021; 16(8): e0249872. 13. Zheng B, Janmohamed Z, Mackenzie C L. Reaction times and the decision-making process in endoscopic surgery - An experimental study[J]. Surgical Endoscopy. 2003; 17(9): 1475-1480. 14. 5G Tele-assisted Surgery in Shanghai - AIS TeleSurgeon [Internet]. 2019. 15. Tian W, Fan M, Zeng C, Liu Y, He D, Zhang Q. Telerobotic Spinal Surgery Based on 5G Network: The First 12 Cases[J]. Neurospine. 2020; 17(1): 114-120. 16. Acemoglu A, Peretti G, Trimarchi M, et al. Operating From a Dis-tance: Robotic Vocal Cord 5G Telesurgery on a Cadaver[J]. Annals of Internal Medicine. 2020; 173(11)
He Y, Lin HL, Hu L and Xia B. Preliminary Study on the Application of “5G+AR” Portable Equipment for Emergency in Zhoushan Islands. Annals of Clinical and Medical Case Reports 2022