The emergence of a *new artistic approach* is disrupting the contemporary musical landscape. *Virtuosity no longer resides solely in performance*, but also emerges from the interaction between humans and technology. *The alliance between creativity and innovation* transcends traditional boundaries, inviting a redefinition of the contours of musical composition. The challenges that arise are both exciting and bewildering, imbued with an unexpected charm. In this context, the exploration of artificial intelligence systems proves essential. *The musical future is sketched at the crossroads of these new vibrations* and ancient traditions.
A considerable gathering assembled at the MIT Media Lab in September for an exceptional concert given by pianist Jordan Rudess, accompanied by violinist and vocalist Camilla Bäckman, along with an artificial intelligence model informally referred to as jam_bot. This latter, developed in collaboration with a team from MIT, made its debut as a work in progress.
Throughout the performance, Rudess and Bäckman exchanged signals and smiles characteristic of seasoned musicians, navigating harmoniously in a shared improvisation. The interactions between Rudess and the jam_bot introduced a form of exchange that was unfamiliar. During a captivating duo inspired by Bach, Rudess alternated between playing a few measures and letting the AI extend the music in a baroque style. With each turn of the model, a palette of expressions appeared on Rudess’s face: astonishment, concentration, curiosity. At the end of the piece, he confessed to the audience: “It’s a combination of immense pleasure and a true challenge.”
Profile of Jordan Rudess
Jordan Rudess stands out as one of the greatest keyboardists in history, according to a poll by Music Radar magazine. His fame primarily stems from his work with the award-winning progressive metal band Dream Theater, which begins a tour this autumn to celebrate its 40th anniversary. In parallel, he pursues a solo career, exemplified by the recent release of his album “Permission to Fly.” In addition to his artistic trajectory, Rudess serves as an educator, sharing his skills through online tutorials and running his own software company, Wizdom Music. His work merges rigorous classical foundations, acquired from the age of nine at the Juilliard School, with remarkable improvisational ability and a strong desire for experimentation.
Collaboration with MIT
Last spring, Rudess was named artist-in-residence at the MIT Center for Arts, Science, and Technology (CAST). There, he collaborated with the Media Lab’s reactive environments research group to design new music technologies powered by artificial intelligence. His main partners in this project are Lancelot Blanchard and Perry Naseck, respectively students at the Media Lab, specializing in generative AI and interactive media. This project is supervised by Professor Joseph Paradiso, a prominent figure in the lab, who combines physics and engineering with his musical passions.
The researchers set out to develop a machine learning model that channels Rudess’s distinctive musical style. In an article published by MIT Press in September, they articulated their vision of a “symbiotic virtuosity.” This approach aims to enable real-time duets between humans and machines, learning from each performance and thus creating music worthy of a concert in front of an audience.
Interactivity and visualization
Rudess provided the necessary data for training the AI while ensuring continuous testing and feedback. Naseck, on his part, studied the best methods to visualize this technology for the public. Indeed, the challenge was to create a platform that would allow the AI to weave its own relationship with the audience, similar to that observed in jazz concerts where musicians exchange signs of anticipation. This led to the creation of a sculptural installation, lit according to the chords played by the AI, which came to life behind Rudess during the concert on September 21.
This type of interaction is crucial for giving the audience insight into what will happen next. Naseck designed an original structure, programmed from scratch at the Media Lab, with the help of experts in mechanical design and fabrication. This kinetic sculpture, illustrating the contributions of the AI, successfully conveyed the emotion and energy generated by the performance: it swayed gently during Rudess’s control and opened and closed like a flower when the AI played majestic chords.
Future evolutions of musical virtuosity
The AI model, based on a neural network architecture called musical transformer, was developed by Blanchard from a model created by an assistant professor at MIT. This model works by predicting the most probable notes following a musical pattern, akin to those of large language models. Blanchard refined the model with recordings of Rudess, ensuring the AI could respond in real-time to his improvisations.
The relevance of the project lies in Rudess’s ability to anticipate the AI’s decisions and maintain control over his performance. Blanchard incorporated options allowing Rudess to activate different modes while playing, thus fostering a rewarding and fluid interaction between the human and the machine.
The research perspectives remain diverse, despite the end of Rudess’s residency. Naseck plans to refine the interactions, optimizing reactive sensors to capture more subtle movements from the musician. The desire to explore musical education through this technology is also present, as Rudess envisions that the model could potentially become a learning tool for students.
The exploration of artificial intelligence represents a natural advancement in Rudess’s journey, stemming from his growing interest in musical technology. His commitment is tinged with the desire to steer this evolution toward positive achievements, benefiting all musicians.
At MIT, Paradiso emphasizes the importance of integrating AI within a framework that benefits humanity. The synergy between these two entities could open up unprecedented perspectives, enabling progress toward new musical horizons.
Frequently asked questions about a model of virtuosity
What is a model of virtuosity in music?
A model of virtuosity in music refers to an approach to musical interpretation that emphasizes high technical skills and great creative agitation, allowing musicians to interact with artificial intelligence systems to produce new compositions in real time.
How can a model of virtuosity enhance musical performances?
A model of virtuosity can enrich musical performances by enabling dynamic improvisation between the musician and artificial intelligence, creating innovative musical dialogues that transcend traditional boundaries of music.
What skills are required to use a model of virtuosity?
To use a model of virtuosity, musicians need to have a solid technical mastery of their instrument, a well-developed improvisational ability, and an open mindset to collaborate with artificial intelligence systems.
What type of artificial intelligence is used in a model of virtuosity?
Models of virtuosity can use various types of artificial intelligence, including generative neural networks and musical transformers, that analyze and predict musical elements based on the musician’s performances.
What is the importance of interactivity in a model of virtuosity?
Interactivity is crucial as it allows the musician to control the AI’s musical decisions in real time, thus creating a unique and rewarding collaborative experience for the audience.
What results can be expected from using a model of virtuosity in concert?
The results of a concert using a model of virtuosity include innovative and dynamic performances, where the AI can respond to the musician’s actions, thereby creating unprecedented musical works that captivate and engage the audience.
How can a musician prepare to integrate a model of virtuosity into their practice?
To prepare, a musician should first familiarize themselves with AI technologies, understand the principles of collaborative improvisation, and participate in workshops or training sessions on using these systems.
Is a model of virtuosity only for professional musicians?
No, a model of virtuosity can also be used by amateur musicians eager to explore new forms of musical expression, as long as they have some mastery of their instrument and music.
What impact can a model of virtuosity have on the future of music?
A model of virtuosity could transform the future of music by redefining musicians’ roles, fostering innovation, and expanding creative possibilities through enriched collaboration between humans and machines.
