Human-instrument control surfaces

I want to share a list/note I've been compiling for the past few months, while thinking about different ways a live performer can control their instrument, and how I can apply some of these methods in my own live performances. But first, I think a bit more context about what I mean by "performer" and "instrument" might be helpful, since I'm approaching this topic from a very specific angle.

Anatomy of a Live Performance

Live performance is an artistic medium I've been working in for a while now — specifically, interacting with real-time systems to generate live sound and video. I use electronic instruments — both physical hardware devices, and software running on computers. Drawing from more traditional musical practices, I call the real-time system I interact with "the instrument," though it's usually made of multiple smaller devices, which each perform different functions. Sometimes the instrument produces sound, or visuals (live video), or both.

While elements of the instrument might cycle or fluctuate on their own, the presence of a human, who I call "the performer," is a vital part of the system. (In my own practice, "the performer" is generally me, and sometimes a collaborator.) The performer makes decisions and controls the instrument by physically interacting with it in some way, which changes its internal parameters, and results in a change in the output. Sometimes, lights or a screen can display information about the internal state of the system, but the main way that the performer observes this is actually the output itself. For example, playing a D note on a digital piano might cause the letter "D" to show up on the screen, but mainly, the performer is listening to the sound of the D note to understand what's happening in the instrument.

Let's take a simple example of a live performance to start with, and apply some of the terminology to a diagram about it.