A bio-responsive network data collection and visualization system
Exhibitions: 2007 – San Jose Institute of Contemporary Art (San Jose, CA), San Jose State University
Biomorphic Aggregator is a bio-responsive network data collection and visualization system where participants can influence the media they see by utilizing their body’s affective responses. It features biofeedback devices and custom data acquisition, analysis and visualization software to collect text and images from the web. The system measures the heart rate, respiration and galvanic skin response of the participant to influence what data is collected, how the data is collected and the characterstics of the visualization itself. A particpant may, for instance, influence how chaotic the movement of the media elements is by her heart rate and breathing pattern. She may be able to speed up, slow dow or fade the images in or out. If the system detects that a certain media element is particularly arousing to the participant, the system will attempt to collect more of that type of media from the web and add it to the visualization. Thus, in a circular relationship, media influences the body and the body influences media. The system is loosely based on news aggreagators or "feed reader" software. This is software that uses a web feed to retrieve syndicated, up-to-date web content such as weblogs, podcasts, and mainstream mass media websites. And in fact the system uses the same protocols (RSS and Atom) to gather much of the data. Biomorphic Aggregator presents a novel approach to browsing the Internet. It makes one conscious of their body and could perhaps begin to reveal it’s relationship to information and digital content.
Biomorphic Aggregator is part of larger investigations focusing on how our bodies experience and adapt to technological and digital media environments, the social contexts of these environments and how they may be reconfiguring bodily experience.
The system consists of biofeedback measurement equipment which the participant must wear in order to interact with the system. The participant’s physiological states are read via biofeedback sensors and signal encoding hardware that measures various physiological signals of the body – in this case heart rate, respiration rate and skin conductance or galvanic skin repsonse. This data is then fed into the data collection, analysis and visualization system.
Data Collection, Analysis and Generation
The system collects data from two primary sources: (1) the participant’s physical state, via a biofeedback device and (2) the web, primarily via rss feeds from various syndicated sites such as Yahoo! Flickr and Google News. The biofeedback data is read from the signal encoding hardware and sent continuously to a computer running the analysis and visualization engines.
The web searching is accomplished by randomly selecting from a list of pre-chosen keywords to perform a web search of the aforementioned sites. Common yet culturally significant terms are used. For example one search may include terms such as "beauty", "leadership", and "free speech". The images and titles or headlines from these searches are parsed out and sent to the analysis and visulization engines.
As the participant’s physical states are being monitored, the system analyzes her responses to media and alters it’s search method accordingly. For instance if a participant becomes particluarly aroused by certian image, the system will search for images and news headines that are similar to that image (Galvanic Skin Response is reliable marker of excitement or arousal. As one becomes excited or nervous for example, the skin becomes more conductive, which increases the amount of voltage in the circuit. Thus if there is a sudden spike in the GSR, it will trigger the system to perform this new search). The new set of text and images will then be added to the visualization.
The visual display of Biomorphic Aggregator is simply a stream of text and images containing all of the recently collected data from the web. Initially, the text and images will appear on the screen as a chaotic and pixelated jumble. However, once the participant has connected herself to the biofeedback device, the data from that device is used to affect the charactersistics of the visualization. In essence, the participant can make the visualization less chaotic via her breathing, thus allowing particular images to come to the fore one at a time (so the affective respose can be measured via GSR).
The interaction is not direct or one-to-one. It does not repeat or react directly, but rather slowly over time as the information (from the participant’s body) is gathered and analyzed. In a sense, it is the energy of the participant’s breathing, heart rate and skin conductance that influence the visualization.
The body’s relationship to digital images and networks
The seeds of this project stem from a desire to investigte the relationships between the world of digital images, computer networks and the material body. Digital images are no longer truly images, but computational processes. Their location and dissemination is made possible, more and more through the organizing logic of computer networks and protocols. Contrary to notions of disembodiment in the world of the information, Biomorphic Aggregator sees the body as an active agent in the world of digital media networks. It seeks to investigate the notion of the body as a perceptual filter or "enframing mechanism". Furthermore, if we look at both digital media and the body within the context of networks and protocols, the body could be seen as a protocological agent, a player in this emerging mode of social-technical organization.
By harnessing the body’s affective resposes, Biomorphic Aggregator seeks to catalyze our bodily senses so that we may better orient ourselves to the perceptions brought about by the proliferaton of media images through the organizing logic of networks and protocols.
The cultural context of media images
Media images exist within, and in relation to, larger cultural, economic and political environments. One simply cannot ignore the cultural context within which technologies arise, especially when dealing with images. Much as the body is used as a perceptual filter, Biomorphic Aggregator posits the notion of the body as a "cultural filter". By becoming more conscious of ones bodily proceesses one can perhaps better perceive the unstable nature of media images. To counteract their fixed cultural meanings and to better question their truth claims and reveal their contingencies.
The research being conducted can be expanded into many areas. Currently, we are looking at the development of both a “smart chair” and “smart wearables” as two possible areas of integration with the Biomorphic Aggregator system. This would relieve the participant of having to strap on any sensor harnesses and the like. Interaction with the system would then happen via the simple act of sitting down are puttin on a shirt.
Statistical models such Hidden Markov Models (HMM) and Regression Analysis can be used to understand the relationships between data and make predictions based on those relationships. These techniques are commonly used in affective computing and context-aware computing research.
While we are not specifically interested in making an intelligent computer that can “sense your mood”, the aforementioned techniques may still prove useful in better establishing relationships and states of mutual influence between human and machine.
MIT Affective Computing Group
Stern, R. M., Ray, W. J., and Quigley, K. S. Psychophysiological Recording Second Edition. Oxford: Oxford University Press, 2001.
Hidden Markov Models
Media & Cultural Theory
Bowen, Dore. “Imagine There’s No Image (It’s Easy If You Try): Appropriation in the Age of Digital Reproduction”, in Jones, Amelia (Ed.), A Companion to Contemporary Art Since 1945. Oxford: Blackwell Publishing, 2006, pp. 534-556.
Debord, Guy. Society of the Spectacle. New York: Zone Books, 1995.
Galloway, Alexander. Protocol: How Control Exists After Decentalization. Cambridge, MA: MIT Press, 2004.
Hansen, Mark B. N. New Philosophy for New Media. Cambridge, MA: MIT Press, 2004.
Massumi, Brian. Parables for the Virtual: Movement, Affect, Sensation. Durham and London, 2002.
Embodied Cognition, Perception, etc
Bergson, Henri. Matter and Memory. London: G. Allen & Unwin, 1962.
Varela, Thompson and Rosch. The Embodied Mind: Cognitive Science and Human Experience. Cambridge,MA: MIT Press, 1991.