PDF (Español)
FLIP
How to Cite
Gómez Marín, D. . (2022). NEA (New Electronic Assistant), a system for musical computational co-creativity. Kepes, 19(26), 473–506. https://doi.org/10.17151/kepes.2022.19.26.15
More Citation Formats
Authors
Daniel Gómez Marín
Universidad Icesi
Abstract
This article presents a system for artistic collaborative computational creativity applied to music creation called the New Electronic Assistant (NEA). NEA is a system that can learn musical styles in a symbolic format, generate pieces following the learned styles and transform the results through real time
interaction. Throughout the text, NEA is introduced and framed within the concepts of computational collaborative creativity. To analyze the performance of NEA in real co-creation scenarios, a process of systematic generation of musical pieces is carried out from which fragments are extracted that are used in a validation experiment with human subjects. The results of the experiment suggest high levels of efficiency in the co-creation processes, related to the production time of new pieces, the surprise expressed by the participants of the experiment and the perceived value of the pieces generated by NEA. The experiment concludes with a section where the details of the pieces perceived as having the highest value by the participants in the experiment are discussed. The text closes with a reflection on the role of generative systems that require human interaction in real processes of collaborative creation.
Keywords:
References
Ames, C. (1989). The Markov process as a compositional model: A survey and tutorial. Leonardo, 22(2), 175-187.
Boden, M. A. (1994). What is creativity? En Dimensions of Creativity (pp. 75-117). Cambridge, Mass.: Bradford/The MIT press.
Boden, M. A. (2009). Conceptual spaces. En Milieus of creativity (pp. 235-243). Springer.
Briot, J. P., Hadjeres, G. & Pachet, F. D. (2020). Deep learning techniques for music generation (Vol. 1). Springer.
Candy, L. & Edmonds, E. (2002, October). Modeling co-creativity in art and technology. En Proceedings of the 4th conference on Creativity & cognition (pp. 134-141).
Catak, M., AlRasheedi, S., AlAli, N., AlQallaf, G., AlMeri, M. y Ali, B. (2021, September). Artificial Intelligence Composer. En 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT) (pp. 608-613). IEEE.
Chaillou, S. (2020). ArchiGAN: Artificial intelligence x architecture. En Architectural intelligence (pp. 117-127). Springer.
Collins, N. (2018). ‘…there is no reason why it should ever stop’: large-scale algorithmic composition. Journal of creative music systems, 3(1).
Csikszentmihalyi, M. (2013). Creativity: Flow and the psychology of discovery and invention (1st Ed.). Harper Perennial.
Csikszentmihalyi, M., Abuhamdeh, S. y Nakamura, J. (2014). Flow. En Flow and the foundations of positive psychology (pp. 227-238). Springer.
Davis, N. (2013). Human computer co-creativity: Blending human and computational creativity. En G. Smith y A. Smith (Eds.), Proceedings of the Doctoral Consortium of Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE) 2013, number WS-13-23 in AAAI Technical Report, 9-12. AAAI.
Eck, D. & Schmidhuber, J. (2002). A first look at music composition using lstm recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale, 103, 48.
Eigenfeldt, A., Burnett, A. & Pasquier, P. (2012, May). Evaluating musical metacreation in a live performance context. En Proceedings of the Third International Conference on Computational Creativity (pp. 140-144).
Gärdenfors, P. (2004). Conceptual spaces: The geometry of thought. MIT press.
Glines, P. W. (2022). Imposing Structure on Generated Sequences: Constrained Hidden Markov Processes (Doctoral Dissertation). Idaho State University.
Hämäläinen, M. (2018). Poem machine-a co-creative nlg web application for poem writing. En The 11th International Conference on Natural Language Generation Proceedings of the Conference. The Association for Computational Linguistics.
Herndon, H. & Dryhurst, M. (Hosts). (2021, August 17). Latent Visions, Promptism and the future of AI art with Adverb [Audio podcast episode]. In Intrdependence. https://bit.ly/3AgERtU
Hiller, L. A. & Baker, R. A. (1964). Computer Cantata: A study in compositional method. Perspectives of New Music, 3(1), 62-90.
Itti, L. & Baldi, P. (2009). Bayesian surprise attracts human attention. Vision research, 49(10), 1295-1306.
Jacob, M. (2019). Improvisational artificial intelligence for embodied co-creativity (Doctoral Dissertation). Georgia Institute of Technology.
Jordanous, A. (2017). Co-creativity and perceptions of computational agents in co-creativity. School of Computing, University of Kent.
Kahneman, D. & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives. Psychological review, 93(2), 136.
Kantosalo, A., Ravikumar, P. T., Grace, K. & Takala, T. (2020, September). Modalities, Styles and Strategies: An Interaction Framework for Human-Computer Co-Creativity. En ICCC (pp. 57-64).
Lubart, T. (2005). How can computers be partners in the creative process: classification and commentary on the special issue. International Journal of Human-Computer Studies, 63(4-5), 365-369.
Macedo, L., Cardoso, A., Reisenzein, R. & Lorini, E. (2009). Artificial surprise. Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence, 267-291.
Maguire, R., Maguire, P. & Keane, M. T. (2011). Making sense of surprise: an investigation of the factors influencing surprise judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(1), 176.
Mathews, M. V. (1991). The radio baton and conductor program, or: Pitch, the most important and least expressive part of music. Computer Music Journal, 15(4), 37-46.
Meyer, L. B. (1957). Meaning in music and information theory. The Journal of Aesthetics and Art Criticism, 15(4), 412-424.
MIDI Manufacturers Association. (1996). The complete MIDI 1.0 detailed specification. Los Angeles, CA, The MIDI Manufacturers Association.
Ó Nuanáin, C., Herrera, P. & Jordá, S. (2017). Rhythmic concatenative synthesis for electronic music: techniques, implementation, and evaluation. Computer Music Journal, 41(2), 21-37.
Ortony, A. & Partridge, D. (1987, August). Surprisingness and expectation failure: what's the difference? En IJCAI (pp. 106-108).
Pachet, F., Roy, P. & Barbieri, G. (2011). Finite-length Markov processes with constraints. In Twenty-Second International Joint Conference on Artificial Intelligence.
Pinch, T. J. & Trocco, F. (2004). Analog days. Harvard University Press.
Privato, N., Rampado, O. & Novello, A. (2022). Scramble Live: Combining LSTM and Markov Chains for Real-time Musical Interaction. En Proceedings of the 19th Sound Music Computing Conference (SMC'22).
Quilici, M. E. (2005). Creativity: Surprise and abductive reasoning. Semiotica, 2005(153), 325-342.
Reisenzein, R. (2000). Exploring the strength of association between components of emotion syndromes: The case of surprise. Cognition & Emotion, 10, 241-277.
Roberts, A., Engel, J., Mann, Y., Gillick, J., Kayacik, C., Nørly, S., ... & Eck, D. (2019). Magenta Studio: Augmenting Creativity with Deep Learning in Ableton Live. Proceedings of the International Workshop on Musical Metacreation (MUME).
Shih, Y. J., Wu, S. L., Zalkow, F., Muller, M. & Yang, Y. H. (2022). Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer. IEEE Transactions on Multimedia.
Teigen, K. H. & Keren, G. (2003). Surprises: Low probabilities or high contrasts? Cognition, 87, 55-71. DOI: 10.1016/s0010-0277(02)00201-9.
Wang, H., Ohsawa, Y., Hu, X. & Xu, F. (2015). Idea discovery: a context-awareness dynamic system approach for computational creativity. En Smart Modeling and Simulation for Complex Systems (pp. 99-111). Springer.
Wiggins, G. A. (2006). A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19(7), 449-458.
Wright, M. & Freed, A. (1997, September). Open SoundControl: A new protocol for communicating with sound synthesizers. En ICMC.
Boden, M. A. (1994). What is creativity? En Dimensions of Creativity (pp. 75-117). Cambridge, Mass.: Bradford/The MIT press.
Boden, M. A. (2009). Conceptual spaces. En Milieus of creativity (pp. 235-243). Springer.
Briot, J. P., Hadjeres, G. & Pachet, F. D. (2020). Deep learning techniques for music generation (Vol. 1). Springer.
Candy, L. & Edmonds, E. (2002, October). Modeling co-creativity in art and technology. En Proceedings of the 4th conference on Creativity & cognition (pp. 134-141).
Catak, M., AlRasheedi, S., AlAli, N., AlQallaf, G., AlMeri, M. y Ali, B. (2021, September). Artificial Intelligence Composer. En 2021 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT) (pp. 608-613). IEEE.
Chaillou, S. (2020). ArchiGAN: Artificial intelligence x architecture. En Architectural intelligence (pp. 117-127). Springer.
Collins, N. (2018). ‘…there is no reason why it should ever stop’: large-scale algorithmic composition. Journal of creative music systems, 3(1).
Csikszentmihalyi, M. (2013). Creativity: Flow and the psychology of discovery and invention (1st Ed.). Harper Perennial.
Csikszentmihalyi, M., Abuhamdeh, S. y Nakamura, J. (2014). Flow. En Flow and the foundations of positive psychology (pp. 227-238). Springer.
Davis, N. (2013). Human computer co-creativity: Blending human and computational creativity. En G. Smith y A. Smith (Eds.), Proceedings of the Doctoral Consortium of Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE) 2013, number WS-13-23 in AAAI Technical Report, 9-12. AAAI.
Eck, D. & Schmidhuber, J. (2002). A first look at music composition using lstm recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale, 103, 48.
Eigenfeldt, A., Burnett, A. & Pasquier, P. (2012, May). Evaluating musical metacreation in a live performance context. En Proceedings of the Third International Conference on Computational Creativity (pp. 140-144).
Gärdenfors, P. (2004). Conceptual spaces: The geometry of thought. MIT press.
Glines, P. W. (2022). Imposing Structure on Generated Sequences: Constrained Hidden Markov Processes (Doctoral Dissertation). Idaho State University.
Hämäläinen, M. (2018). Poem machine-a co-creative nlg web application for poem writing. En The 11th International Conference on Natural Language Generation Proceedings of the Conference. The Association for Computational Linguistics.
Herndon, H. & Dryhurst, M. (Hosts). (2021, August 17). Latent Visions, Promptism and the future of AI art with Adverb [Audio podcast episode]. In Intrdependence. https://bit.ly/3AgERtU
Hiller, L. A. & Baker, R. A. (1964). Computer Cantata: A study in compositional method. Perspectives of New Music, 3(1), 62-90.
Itti, L. & Baldi, P. (2009). Bayesian surprise attracts human attention. Vision research, 49(10), 1295-1306.
Jacob, M. (2019). Improvisational artificial intelligence for embodied co-creativity (Doctoral Dissertation). Georgia Institute of Technology.
Jordanous, A. (2017). Co-creativity and perceptions of computational agents in co-creativity. School of Computing, University of Kent.
Kahneman, D. & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives. Psychological review, 93(2), 136.
Kantosalo, A., Ravikumar, P. T., Grace, K. & Takala, T. (2020, September). Modalities, Styles and Strategies: An Interaction Framework for Human-Computer Co-Creativity. En ICCC (pp. 57-64).
Lubart, T. (2005). How can computers be partners in the creative process: classification and commentary on the special issue. International Journal of Human-Computer Studies, 63(4-5), 365-369.
Macedo, L., Cardoso, A., Reisenzein, R. & Lorini, E. (2009). Artificial surprise. Handbook of research on synthetic emotions and sociable robotics: New applications in affective computing and artificial intelligence, 267-291.
Maguire, R., Maguire, P. & Keane, M. T. (2011). Making sense of surprise: an investigation of the factors influencing surprise judgments. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(1), 176.
Mathews, M. V. (1991). The radio baton and conductor program, or: Pitch, the most important and least expressive part of music. Computer Music Journal, 15(4), 37-46.
Meyer, L. B. (1957). Meaning in music and information theory. The Journal of Aesthetics and Art Criticism, 15(4), 412-424.
MIDI Manufacturers Association. (1996). The complete MIDI 1.0 detailed specification. Los Angeles, CA, The MIDI Manufacturers Association.
Ó Nuanáin, C., Herrera, P. & Jordá, S. (2017). Rhythmic concatenative synthesis for electronic music: techniques, implementation, and evaluation. Computer Music Journal, 41(2), 21-37.
Ortony, A. & Partridge, D. (1987, August). Surprisingness and expectation failure: what's the difference? En IJCAI (pp. 106-108).
Pachet, F., Roy, P. & Barbieri, G. (2011). Finite-length Markov processes with constraints. In Twenty-Second International Joint Conference on Artificial Intelligence.
Pinch, T. J. & Trocco, F. (2004). Analog days. Harvard University Press.
Privato, N., Rampado, O. & Novello, A. (2022). Scramble Live: Combining LSTM and Markov Chains for Real-time Musical Interaction. En Proceedings of the 19th Sound Music Computing Conference (SMC'22).
Quilici, M. E. (2005). Creativity: Surprise and abductive reasoning. Semiotica, 2005(153), 325-342.
Reisenzein, R. (2000). Exploring the strength of association between components of emotion syndromes: The case of surprise. Cognition & Emotion, 10, 241-277.
Roberts, A., Engel, J., Mann, Y., Gillick, J., Kayacik, C., Nørly, S., ... & Eck, D. (2019). Magenta Studio: Augmenting Creativity with Deep Learning in Ableton Live. Proceedings of the International Workshop on Musical Metacreation (MUME).
Shih, Y. J., Wu, S. L., Zalkow, F., Muller, M. & Yang, Y. H. (2022). Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer. IEEE Transactions on Multimedia.
Teigen, K. H. & Keren, G. (2003). Surprises: Low probabilities or high contrasts? Cognition, 87, 55-71. DOI: 10.1016/s0010-0277(02)00201-9.
Wang, H., Ohsawa, Y., Hu, X. & Xu, F. (2015). Idea discovery: a context-awareness dynamic system approach for computational creativity. En Smart Modeling and Simulation for Complex Systems (pp. 99-111). Springer.
Wiggins, G. A. (2006). A preliminary framework for description, analysis and comparison of creative systems. Knowledge-Based Systems, 19(7), 449-458.
Wright, M. & Freed, A. (1997, September). Open SoundControl: A new protocol for communicating with sound synthesizers. En ICMC.
Downloads
Download data is not yet available.
