HC(F)I (Human-Computer-Fish Interaction) (working title) is a collaborative art-science research initiative exploring human-animal interaction, communication and collaboration, as well the use of lay interpretations as a form of knowledge construction within a scientific research context. The focus of the research is the behavior and neurobiology of weakly electric fish. These animals generate small electrical pulses or waves (called electric organ discharges or EODs) in order to navigate their environment and communicate with other fish. Done in collaboration with Professor Gary Marsat of West Virginia University, this project will use machine learning algorithms to recognize human gestures and convert them to electromagnetic oscillations that the fish can understand. Participants will thus be able to interact and communicate with the fish, in essence acting as “virtual fish”, with the gallery acting as extension of the fish tank. The bioelectrical fluctuations of the EODs will also be used as modulation and trigger sources for modular electronic and software-based sound synthesizers; in essence sonifying the fish’s behavior and giving the animal a “voice”.

In addition, data collected from the fishes during these interactions, along with the interpretations of the human participants, will be used to construct a model that relates the fish’s sensory input to its electrogenic and behavioral responses. We are also investigating the possibility of developing a fish “chirp database” that gallery participants can contribute to (chirps being transient increases in EOD frequency). This will consist of monitoring and recording of the fish’s EODs and identifying these chirps. We would then correlate their occurrence with the behavior observed and interpreted by participants in the gallery at the time the chirp(s) occurred (e.g. “he looked frightened”, “she was being cautious”, etc).

Currently, we are working with the species Apteronotus leptorhynchus (brown ghost knifefish) and Apteronotus albifrons (black ghost knifefish).

GitHub