Gamification and virtual reality for digital twin learning and training: Architecture and challenges
IMAGE: THE FIGURE ILLUSTRATES THE SET OF COMPONENTS OF THE ENVISIONED ARCHITECTURE THAT COLLABORATE TO REALIZE THE LEARNING/TRAINING WE ENVISION. WE EXPLAIN EACH COMPONENT, THEIR RELATIONS, AND HOW DIFFERENT STAKEHOLDERS ARE INVOLVED IN PERFORMING SPECIFIC ACTIVITIES. view more
CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.
This study proposes the integration of VR and gamification mechanisms to enable DT students and employees to experience quality learning and training experiences. This technology will be multi-purpose and suitable for different uses, including lesson planning and instruction. The gamification element will enhance learner motivation and make the entire experience more enjoyable. This benefits the learning process by enabling learners to acquire knowledge and create a favorable learning environment. We discussed the required components and necessary interconnections to adequately assemble and obtain the resulting gamified modeling tool. We highlighted some relevant issues that should be addressed in future research: the definition of a repository for learning experiences, the realization of techniques required to adapt and personalize user feedback, and the implementation of approaches for personalized and cooperative learning in the DT domain.
JOURNAL
Virtual Reality & Intelligent Hardware
ARTICLE TITLE
Gamification and virtual reality for digital twins learning and training: Architecture and challenges
Building the metaverse using digital twins at all scales, states, and relations
Peer-Reviewed PublicationIMAGE: THE 3D MOLECULAR MODELS IN THE FIGURE WERE GENERATED IN UNITYMOL, AND THE OTHER 3D MODELS WERE DOWNLOADED FROM THE INTERNET.THIS CONCERNS STRUCTURING A COMPLETE EVOLUTIONARY HIERARCHY FROM THE SUBATOMIC SCALE WITH NEUTRINOS AND QUANTUM DOTS TO THE SOLAR SYSTEM AND THE ENTIRE UNIVERSE. THE FLEXIBILITY OF DIGITAL TWINS IS FURTHER IMPROVED BY MAKING INVESTIGATIONS FROM THE MESOSCALE TO TWO RELATIVE SCALES. view more
CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.
Today’s technological development is equivalent to the iterative evolution of applications and ideas: from early graphics to visualization, to HCI, to virtual reality, to Digital Twins, and finally to the Metaverse. Since the birth of the first image display in 1950, computer graphics have undergone continuous development
in recent decades and continuously evolved. Computer graphics aim to create efficient visual communication, and this goal is achieved via visualization techniques. Visualization is the theory, method, and technology of using computer graphics and image processing technology to convert data into graphics or images and display them on a screen, and then perform interactive processing. Subsequently, with the development of computer hardware and realization of more interactive technologies, virtual reality was conceived.
Virtual reality is a further extension of visualization technology, focusing on bringing immersive experiences to users in a virtual world such that users become and feel part of a virtual world. Simultaneously, virtual reality technology emphasizes visual simulation and attempts to simulate various senses such as touch, taste, smell, and hearing. Digital Twins are a virtual entity that digitally recreate a physical entity. With the help of historical data, real-time data, and algorithm models, it is a technical means of simulating, verifying, predicting, and controlling the entire life cycle process of a physical entity. Compared with virtual reality, Digital Twins are more concerned with the synchronization of data and interaction between the virtual and real worlds. The Metaverse is a larger and more complex system than Digital Twins. Digital Twins originated from the industrialization of complex product development and is moving toward urbanization and globalization, whereas the Metaverse originated from the gaming and entertainment industry that builds human–human interaction and is moving from globalization to urbanization and industrialization. Although the Metaverse and Digital Twins focus on the connection and interaction between the real and virtual worlds, the essential difference between the two is that the Metaverse is directly oriented towards people, whereas Digital Twins are oriented towards things. The concept of the Metaverse represents the development direction of the next generation of the Internet, and via Digital Twins, the two major technology systems complement each other and will lead the fourth industrial revolution.
JOURNAL
Virtual Reality & Intelligent Hardware
ARTICLE TITLE
Building the Metaverse by Digital Twins at All Scales, State, Relation
Resource requirements of an edge-based digital twin service: An experimental study
IMAGE: THE TESTBED IS DEPICTED IN FIGURE 3 AND CONSISTS OF TWO MAIN BLOCKS: THE EDGE HOST, WHICH WE WILL REFER TO AS EDGE MACHINE, IN CHARGE OF OFFERING COMPUTATIONAL RESOURCES TO THE DT SERVICE, AS WELL AS PROVIDING LTE CONNECTIVITY TO THE SECOND BLOCK OF THE ARCHITECTURE TERMED ROBOTS MACHINE, I.E., THE PHYSICAL ROBOTS THAT CONVERSELY REQUEST AND CONSUME THE SERVICE. THE EDGE MACHINE IS EQUIPPED WITH 16GB RAM AND AN INTEL(R) CORE(TM) I7-7700HQ 4CPU@2.80 GHZ PROCESSOR. CONVERSELY, THE ROBOTS MACHINE WAS SUPPLIED WITH 16GB RAM AND AN INTEL(R) CORE(TM) I7-8550U 4CPU@1.80 GHZ PROCESSOR. view more
CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.
We presented the experimental evaluation and profile of an Edge-based Digital Twin solution designed for the remote control of robotic arms. The service was split into virtual network functions, deployed on a laboratory setup and offered to 6-axis Niryo One simulated robotic arms. In an initial measurement campaign, the robotic arms leveraged Gigabit Ethernet cables to join the service, to investigate the interconnection between the resources occupied only by the service and the number of remotely controlled robots. Our results show that the most critical function of the Digital Twin as a Service is the inverse kinematics computation, followed by the movement trajectory plan. Both these functions are taken over by the Motion Planning VNF. We then analyzed the impact of the commands imposed on the robots on the service profile. Particularly, measurements proved that the exploitation of low abstraction level commands can lead to relevant computational resources savings, thus great performance benefits; however, additional safety mechanisms are required if the trajectories are not predictable. Finally, in a second measurement campaign, LTE shortage in accommodating real-time DT applications has been empirically proved. For future research, the testbed is going to integrate enhanced radio communication technologies, e.g., 5G and Wi-Fi 6E, in order to meet the stringent real-time latency requirements involved by the operational DT. Moreover, in accordance with the results derived in this work, the need for an agile and automated network orchestration framework emerges to enhance resource usage efficiency and provide performance guarantees. We then intend to design and deploy an automated smart network service orchestrator able to ensure resource utilization and energy consumption optimization while avoiding running into service disruptions caused by shortage of allocated resources. In this regard, in compliance with the underlying and driving idea of 5GB and 6G, Artificial intelligence (AI) and Machine Learning (ML) may provide the key tools to achieve the aforementioned goals. ML, in particular, can support and leverage network slicing capabilities at best, in order to logically isolate resource pools dedicated to different industrial vertical applications.
JOURNAL
Virtual Reality & Intelligent Hardware
ARTICLE TITLE
Resource requirements of an edge-based digital twin service: An experimental study
Empowering digital twins with eXtended reality collaborations
Peer-Reviewed PublicationIMAGE: FIRSTLY, THE AR INTERFACE WOULD RUN ON AN AR DEVICE LIKE THE HOLOLENS, SO THE ASSEMBLERS COULD KEEP GOING WITH THEIR WORK WITHOUT MOVING FROM THE ELECTRICAL BOARD TO THE PERSONAL COMPUTER. SECONDLY, OBJECT RECOGNITION WOULD PREVENT WASTING TIME SEARCHING FOR THE INSTRUCTION/ANNOTATIONS REGARDING A PARTICULAR COMPONENT BY IMMEDIATELY VISUALIZING ALL THE INFO IN THE SHARED DOCUMENT. THE ADAPTATION OF THE HCLINT-DT FRAMEWORK FOR THIS USE CASE IS VISUALLY REPORTED IN THE FIGURE. IN ADDITION, THANKS TO THE ADOPTION OF THE HCLINT-DT FRAMEWORK, THE WORKERS’ EMPOWERMENT IS PROVIDED WITH THE USE OF STATE OF THE ART TOOLS AND XR DEVICES (E.G., HOLOLENS 2 AND HTC-VIVE), AS CONFIRMED BY THE ASSEMBLERS, ENGINEERS, AND TESTERS WHICH WERE INTERVIEWED DURING OUR ANALYSIS. THE COMMUNICATION IMPROVEMENT IS GIVEN BY THE NATURAL WORKING MECHANISM OF HCLINT-DT SHARED ANNOTATIONS. FINALLY, TRAINING IS A NATURAL CONSEQUENCE OF THE SAME HCLINT-DT SHARED ANNOTATION MECHANISM SINCE WORKERS COULD ANALYZE PREVIOUSLY ANNOTATED DOCUMENTS TO LEARN FROM OTHERS’ EXPERIENCES. view more
CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.
We introduced the HCLINT-DT framework to support the spread of human collaborative intelligence by leveraging DT and XR paradigms. To validate such an approach, we assessed a use case involving family photo albums through an online survey. The results show a general agreement on the ease of use of the AR interface and the overall experience, even if there was only a partial agreement in preferring AR. We also explored the adaptability of the proposed approach considering a use case drawn from a local industrial electrical engineering context. Here, the HCLINT-DT showed a good adaptability level. Further investigations could involve: (a) improving annotations retrieval, (b) analyzing how this annotation system could support machines (learning), and, (c) understanding more deeply the impact of such systems on human activities. Finally, instances of this framework could be created for additional areas and domains, other than the considered ones (e.g., education, marketing).
JOURNAL
Virtual Reality & Intelligent Hardware
ARTICLE TITLE
Empowering digital twins with eXtended reality collaborations
Digital twins and multi-access edge computing for IIoT
Peer-Reviewed PublicationIMAGE: THE PROPOSED NETWORK CAN BE CHARACTERIZED AS AN AGGREGATING DIGITAL TWIN NETWORK BECAUSE THE NODES ARE OF A LARGE VOLUME INSIDE THE INDUSTRY. EACH OF THE NODES IS NEAR THE OTHER NODES, WHICH CAN CAUSE DATA REDUNDANCY. WITH THE CORRECT COLLECTION AND TRANSMISSION METHODS IN THE DIGITAL TWIN, THIS CAN BE AVOIDED. THIS RESULTS IN LESS NETWORK CONGESTION AND LESS ENERGY BUT INCREASED COMPUTING PERFORMANCE AND MEMORY. FIGURE 1 SHOWS THE PROPOSED DIGITAL TWIN COMMUNICATION MODEL WITH A MULTI-ACCESS EDGE-CLOUD FRAMEWORK. THE PROPOSED IIOT COMMUNICATION MODEL, CONSISTS OF THREE LAYERS. THE FIRST LAYER CONSTITUTES THE PHYSICAL, CONNECTIVITY, OR NETWORK AND ABSTRACTION SUBLAYERS. SPECIFICALLY, IN THE PHYSICAL SUBLAYER, VARIOUS DEVICES ARE INSTALLED. IN THE CONNECTIVITY SUBLAYER, THE DEVICES ARE CONNECTED IN A WAY AND TOPOLOGY SO THAT THEY CAN PRODUCE AND TRANSFER DATA OVER TIME. IN THE ABSTRACTION SUBLAYER, THE DIFFERENCES BETWEEN THE VALUABLE INFORMATION AND THE TOTAL DATA GENERATED, HAVE BEEN REDUCED. THE SECOND LAYER COMPRISES ESS, A DATABASE, AND A BROKER. SPECIFICALLY, THE VALUABLE DATA PRODUCED ARE FORWARDED THROUGH THE BROKER DEVICE TO THE ESS FOR A GOOD REAL-TIME MANIPULATION AND MANAGEMENT AND FINALLY STORED FOR A SPECIFIC PERIOD OF TIME INSIDE THE LOCAL DATABASE. THE THIRD LAYER CONSISTS OF AN APPLICATION, A CS, AND A CLOUD DATABASE. SPECIFICALLY, THE APPLICATION UPDATES DATA STORED IN THE LOCAL DATABASE IN REAL TIME EFFICIENTLY, AND CLOUD SERVICES ARE PROVIDED FOR COMPLEX ANALYTICS AND LONGTERM STORAGE. view more
CREDIT: BEIJING ZHONGKE JOURNAL PUBLISING CO. LTD.
All recent technological findings can be collectively used to strengthen the industrial Internet of things (IIoT) sector. The novel technology of multi-access edge computing or mobile edge computing (MEC) and digital twins have advanced rapidly in the industry. MEC is the middle layer between mobile devices and the cloud, and it provides scalability, reliability, security, efficient control, and storage of resources. Digital twins form a communication model that enhances the entire system by improving latency, overhead, and energy consumption. Methods The main focus in this study is the biggest challenges that researchers in the field of IIoT have to overcome to obtain a more efficient communication environment in terms of technology integration, efficient energy and data delivery, storage spaces, security, and real-time control and analysis. Thus, a distributed system is established in a local network, in which several functions operate. In addition, an MEC-based framework is proposed to reduce traffic and latency by merging the processing of data generated by IIoT devices at the edge of the network. The critical parts of the proposed IIoT system are evaluated by using emulation software. The results show that data delivery and offloading are performed more efficiently, energy consumption and processing are improved, and security, complexity, control, and reliability are enhanced. The proposed framework and application provide authentication and integrity to end users and IoT devices.
JOURNAL
Virtual Reality & Intelligent Hardware
ARTICLE TITLE
Digital twins and multi-access edge computing for IIoT
