Providing an essential and unique bridge between the theories of signal processing, machine learning, and artificial intelligence (AI) in music, this book provides a holistic overview of foundational ideas in music, from the physical and mathematical properties of sound to symbolic representations. Combining signals and language models in one place, this book explores how sound may be represented and manipulated by computer systems, and how our devices may come to recognize particular sonic patterns as musically meaningful or creative through the lens of information theory.
Introducing popular fundamental ideas in AI at a comfortable pace, more complex discussions around implementations and implications in musical creativity are gradually incorporated as the book progresses. Each chapter is accompanied by guided programming activities designed to familiarize readers with practical implications of discussed theory, without the frustrations of free-form coding.
Surveying state-of-the art methods in applications of deep neural networks to audio and sound computing, as well as offering a research perspective that suggests future challenges in music and AI research, this book appeals to both students of AI and music, as well as industry professionals in the fields of machine learning, music, and AI.
Author(s): Shlomo Dubnov, Ross Greer
Publisher: CRC Press
Year: 2023
Language: English
Pages: 345
Cover
Endorsements
Half Title
Series Page
Title Page
Copyright Page
Dedication
Contents
Preface
Chapter 1: Introduction to Sounds of Music
1.1. From Sound to Features
1.2. Representation of Sound and Music Data
1.3. Acoustics and Basics of Notes
1.4. Perception and Cognition: Anticipation and Principles of Music Information Dynamics
1.5. Exercises
Chapter 2: Noise: The Hidden Dynamics of Music
2.1. Noise, Aleatoric Music, and Generative Processes in Music
2.2. History of Mathematical Theory of Music and Compositional Algorithms
2.3. Computer Modeling of Musical Style and Machine Improvisation
2.4. Markov Models and Language Models for Music
2.5. Music Information Dynamics
2.6. Exercises
Chapter 3: Communicating Musical Information
3.1. Music as Information Source
3.2. Lempel-Ziv Algorithm and Musical Style
3.3. Lossy Prediction Using Probabilistic Suffix Tree
3.4. Improved Suffix Search Using Factor Oracle Algorithm
3.5. Modeling of Musical Style
3.6. Estimating Music Information Dynamics Using Lempel-Ziv Complexity
3.7. Exercises
Chapter 4: Understanding and (Re)Creating Sound
4.1. Introduction to Digital Signal Processing
4.2. Short-Time Fourier Analysis and Perfect Reconstruction (COLA)
4.3. Griffin-Lim Phase Reconstruction
4.4. Filters and Transforms
4.5. Voice as a Source-Filter Model
4.6. Information Rate in Spectral Analysis
4.7. Exercises
Chapter 5: Generating and Listening to Audio Information
5.1. Concatenative and Recombinant Audio Synthesis
5.2. Audio Oracle
5.3. Audio Symbolization Using Music Information Dynamics
5.4. Accompaniment Using Query-Based Improvisation
5.5. Computational Aesthetics
5.6. Exercises
Chapter 6: Artificial Musical Brains
6.1. Neural Network Models of Music
6.2. Viewing Neural Networks in a Musical Frame
6.3. Learning Audio Representations
6.4. Audio-Basis Using PCA and Matrix Factorization
6.5. Representation Learning with Auto-Encoder
6.6. Exercises
Chapter 7: Representing Voices in Pitch and Time
7.1. Tokenization
7.2. Recurrent Neural Network for Music
7.3. Convolutional Neural Networks for Music and Audio
7.4. Pretrained Audio Neural Networks
7.5. Exercises
Chapter 8: Noise Revisited: Brains that Imagine
8.1. Why study generative modeling?
8.2. Mathematical Definitions
8.3. Variational Methods: Autoencoder, Evidence Lower Bound
8.4. Generating Music and Sound with Variational Autoencoder
8.5. Discrete Neural Representation with Vector-Quantized VAE
8.6. Generative Adversarial Networks
8.7. Exercises
Chapter 9: Paying (Musical) Attention
9.1. Introduction
9.2. Transformers and Memory Models in RNN
9.3. MIDI Transformers
9.4. Spectral Transformers
9.5. Attention, Memory, and Information Dynamics
9.6. Exercises
Chapter 10: Last Noisy Thoughts, Summary, and Conclusion
10.1. Music Communication Revisited: Information Theory of VAE
10.2. The Big Picture: Deep Music Information Dynamics
10.3. The Future of AI in Music and Man-Machine Creative Interaction
Appendix A: Introduction to Neural Network Frameworks: Keras, Tensorflow, Pytorch
Appendix B: Summary of Programming Examples and Exercises
Appendix C: Software Packages for Music and Audio Representation and Analysis
Appendix D: Free Music and Audio-Editing Software
Appendix E: Datasets
Appendix F: Figure Attributions
References
Index
