Protocol is an interactive installation through which I attempt to realize a new form of human-machine symbiosis. It features a multi-modal interface and non-verbal communication system that networks and integrates a human participant with a group of intelligent digital agents that can “sense” and “communicate” with humans via sound, rhythmic patterns and electrical stimulation of the human participant’s skin. Through their interactions, the agents and the human attempt to develop a human-machinic “understanding” or “equilibrium.” The system is inspired and utilizes some of the tactile communication and sensory substitution techniques developed by Paul Bach-Y-Rita and others as well as “bottom-up” approaches to artificial intelligence such as reinforcement learning and subsumption architecture.

The installation consists of a group of drums (usually 2-4). Each drum behaves as an “intelligent agent”. To interact with them, a participant must put on a set of wearable electronic components around his or her trunk area. In addition to spatialized sound, these elements serve as the two-way communication interface between the human participant and the agents. The drums respond to agitation and concussive striking, and serve as the primary method by which the human communicates with the agent. The agents respond not only by processing the live, acoustic signals of the drums and generating their own sounds but also by electrically triggering muscle stimulation patterns in the participant. These patterns serve as a means for the entities to “touch” and manipulate the participant’s body. In addition, the electrical stimulation patterns, along with the rhythmic patterns generated by the participant, constitute a sort of informal protocol that both human and machine co-develop and that each must learn and adapt to. Through visceral and corporeal interactions, Protocol seeks to examine how humans and intelligent technological systems can intertwine, interrelate and give rise to posthuman, cybernetic experiences.

GitHub