Xi'an Conservatory of Music
1. Project Overview The project is a cultural and technological integration project in response to the “Belt and Road Initiative” and the cooperation framework of BRICS countries. By integrating key technologies such as music big data analysis, 3D instrument modeling, virtual reality (VR), and immersive human-computer interaction, the platform aims to create a new environment for learning and experiencing traditional music cultures along the Belt and Road. It not only provides users with an immersive space to deeply understand the traditional music cultures of Belt and Road countries but also serves as an essential tool for promoting cultural exchange and understanding among BRICS countries, holding profound significance for the protection, inheritance, and innovative development of diverse cultures. 2. Technical Challenges and Innovations The project addresses core technical challenges such as intelligent calibration of music style data, precise transformation from 2D images to 3D models, construction of a comprehensive virtual music environment, cross-language interaction experiences, and real-time pitch recognition and feedback. Innovatively combining high-precision 3D modeling with VR technology, the platform offers users an unprecedented immersive experience in music culture. Additionally, the platform's real-time pitch recognition and feedback system, as well as the application of gesture recognition and haptic feedback technologies, bring innovative attempts to the field of music education, greatly enhancing users' interest and experience. There have already been several successful cases in recent years, including the 'Belt and Road' music culture virtual simulation teaching course developed by our team at the Xi'an Conservatory of Music.
Artificial Intelligence And Digital Services
Artificial intelligence
2024-2026
Xi'an, Shaanxi
The innovation and characteristics of this project are mainly reflected in the following aspects:Cross-cultural integration of music culture experience and education model: The platform integrates music cultural resources from countries along the "Belt and Road" and customizes platform content based on the unique accumulation and dissemination patterns of music culture in BRICS countries. It breaks through the geographical limitations of traditional music education, providing a diversified and cross-cultural music learning environment. Users can learn and experience different music styles from various countries and regions on the same platform. This integrated education model helps cultivate musical literacy and a broad cultural perspective in a globalized context. Combination of high-precision 3D modeling and virtual reality technology: Using high-precision 3D scanning and modeling technology, the platform can accurately restore musical instruments and performance scenes. Combined with VR and virtual simulation technology, it provides users with an immersive music culture experience. This combination not only enhances the sensory experience of users but also allows them to break through the boundaries of time and space, expand the horizons of thought, and understand and appreciate traditional music in new ways. Real-time pitch recognition and feedback system: The platform's proprietary pitch recognition technology can analyze users' performance in real time and provide feedback, which is an innovative attempt in the field of music education. This system not only helps users improve their playing skills but also greatly stimulates interest in learning, promotes communication among users, and enhances learning effectiveness. Innovative application of human-computer interaction technology: The platform simulates the playing experience of real musical instruments through gesture recognition and haptic feedback technology, allowing users to perform highly natural simulations and interactions in a virtual environment. This innovative human-computer interaction method increases user engagement and immersion, bringing new possibilities for the future development of music education. Cloud platform support and content update mechanism: Based on mature cloud computing technology and backend services, the platform supports a large number of concurrent visits and data transmission, ensuring the stability and scalability of the service. At the same time, regular content updates and third-party developer support ensure the continuous iteration and updating of platform content, continuously enhancing the platform's educational, teaching, and communication value in the field of music culture.
In 2019, the "Chime Bell Virtual Simulation Interaction System" won the first prize in the university group of the China Virtual Reality Competition. In 2020, the "Xi'an Drum Music Virtual Simulation Interaction System" won the Special Award for Visual Interaction Art at the China Virtual Reality Competition. In 2019, the "Belt and Road" virtual simulation teaching course was approved as a provincial-level virtual simulation experimental teaching course.
In 2022, the team developed a "Pitch Assessment Method Based on Deep Convolutional Neural Network (DCNN) and CTC Algorithm," which was granted a national invention patent.
