Search

C World

Conor's Blog/Portfolio

Category

Fall 2012

ROB3115 – A Neuro-Immersive Narrative

In-experience screenshot

ROB3115 is an interactive graphic novel that is influenced by the reader’s brainwaves. The experience is driven by the reader’s ability to cognitively engage with the story. ROB3115′s narrative and its fundamental interactive mechanic – the reader’s ability to focus – are tightly intertwined by virtue of a philosophical supposition linking consciousness with attention.

ROB3115 explores the intersection of interactive narrative, visual storytelling, and brain-computer interfacing. The experience, designed for an individual, puts the reader in the shoes of a highly intelligent artificial being that begins to perceive a sense of consciousness. By using a NeuroSky brainwave sensor, the reader’s brain activity directly affects the internal dialogue of the main character, in turn, dictating the outcome of his series of psychosomatic realizations. The system is an adaptation of the traditional choose-your-own-adventure. However, instead of actively making decisions at critical points in the narrative, the reader subconsciously affects the story via their level of cognitive engagement. This piece makes use of new media devices while, at the same time, commenting on the seemingly inevitable implications of their introduction into society.

This project was my thesis in graduating from Parsons with an M.F.A. in Design & Technology.

Brain Interface Lab

I recently founded the Brain Interface Lab with some colleagues from Parsons MFA Design & Technology and Columbia University. The lab is dedicated to supporting the open-source software and hardware development of brain-computer interfaces. Check out our website and all of the awesome stuff that was created during our first big event titled Hack-A-Brain:

Plasma Ball Concentration Game (openFrameworks + Neurosky’s EEG Mindset)

Project Summary

This project relates to the brain-computer interface work I’ve been doing for my thesis. As I will soon be creating generative animations that responds to brain activity, which are part of a digital graphic novel, I wanted to do a prototype of a visually complex animation that was dependent on a person’s brain activity. This project was written in openFrameworks and uses a Neurosky Mindset to link a player’s attention level to the intensity of electricity being generated from a sphere in the middle of the screen. The meat of the code is a recursive function that creates individual lightning strikes at a frequency inversely proportional to the attention parameter calculated by the Neurosky EEG headset. The project was visually inspired by the tesla coil and those cool electricity lamps that were really popular in the 90s (see below).

Once the connection between the Neurosky headset and the user’s computer has strong connectivity, the user can press the ‘b’ key (for brain) to link their EEG with the plasma ball. At any point the user can press the ‘g’ key (for graph) to see a HUD that displays a bar graph of their attention value on a scale from 0-100. The graph also shows the connectivity value of the device and the average attention value, calculated over the previous 5 seconds, being used to dictate the frequency of the electricity.

In order to get this application working on your computer, you must first download and install the Neurosky Thinkgear connector. You should be able to get it working with any bluetooth enabled Neurosky device; I’ve documented how to do so in the readme file on my github. You can get my code for the project on my Github page here: https://github.com/crussoma/conorRussomanno_algo2012/tree/master/Conors_Final

Also, if you just want to see the recursive electricity code working independent of a person’s EEG, download and install the app lightningBall (not lightnightBall_brain) from my github.

Project Video

To see this project in action check out my demo reel and jump to 35s.

Visual Inspiration

lightnightBall

lightnightLamp

Screenshots

Screen Shot 2013-03-06 at 5.29.23 PM

Screen Shot 2013-03-06 at 5.29.00 PM

References

My code uses some of the logic and algorithms Esteban Hufstedler’s processing sketch:http://www.openprocessing.org/sketch/2924

Additionally, a big shout out to Akira Hayasaka for writing the Neurosky openFrameworks addon that I used to pull this off:  https://github.com/Akira-Hayasaka/ofxThinkGear

‘Wetlands’ Architectural Renders

Project Summary

I spent the past 6 weeks working with the amazing and progressive artist Mary Mattingly on her project titled Wetlands. Most of her work explores the complex relationship between people and the Earth. Wetlands, currently in the design phase, is a self-sustained living environment that floats in the rivers outside of Philadelphia. The structure will be a low-cost floating barge with various components that explore DIY techniques of sustainability.

My Role

I worked with 2 other artists to create an architectural design for the structure that optimized the functional and design constraints. I helped with the concept drawings and took the lead on creating 3D renders of the design.

Renders

Project Presenation PDF: wetlands

Please Vote For An Awesome EEG Project!

Please take 10 seconds to vote for my New Challenge application:

Despite being rather silent on this blog recently, I’ve actually been quite busy. My ongoing thesis at Parsons MFA Design & Technology is an exploration of practical applications of wearable brain-computer interfaces. More on that to come.

Recently, some fellow designers, engineers, researchers, and myself applied for an award of up to 10K to explore if wearable BCIs could be used to find complimentary or alternative solutions for people suffering from attention disorders such as ADHD. If you support this cause, please click on the image above or the following link and click the “vote” button. You could comment here, but it would be better for you to comment on the application page itself in order to prove to the judges that people truly do care about this cause.

The application is as follows:

Project Title: Brain Design Lab – Finding Alternative Approaches to Addressing ADHD

People Involved:

  • Conor Russomanno (Director) – Conor is currently a 2nd Year in Parsons School of Design MFA Design & Technology Program. Conor did his undergraduate degree in engineering at Columbia University, and has been working with brain-computer interfaces for the past year. Check out his website at conorrussomanno.me.
  • Kristen Kersh – Candidate for MFA in Design & Technology at Parsons School of Design, Masters in Neuroscience and Education from Harvard University
  • James Ramadan – Received dual majors in biology and statistics from University of Virginia, currently does research in statistical analysis of quantitative EEG.
  • Amy Burns – Award winning reporter, spent more than 17 years in the multi-media industry, covering a diverse range of topics through the written word, social media, and the power of video.
  • Other members of the Brain Design Lab (our website is currently being built, braindesignlab.com)

The Problem

Our brains are dependent on the stimuli provided by our environment. Neuroplasticity is the notion that our neurons can be molded and re-purposed based on our experiences, even after critical stages of development. Currently, elite academic institutions such as Harvard, Columbia, and MIT are using functional magnetic resonance imaging (FMRI), magnetic resonance imaging (MRI), and electroencephalography (EEG) to research the brain’s ability to develop and change in response to stimuli. These studies have produced important findings with regards to a wide range of neurological diseases, traumatic brain injuries, and learning. In turn, these findings are being translated and applied to improved techniques in medicine, therapy, and education.

One of the main shortcomings of interfacing the brain is the ability to attain data outside of the confinement of a laboratory setting. There are very few studies done with a patient within the context of their normal environment, looking at how their home, what they eat, smell, see, hear, and touch affects the activity within their brain. Understandably, this is a very large challenge to address. If we are honored with receiving funds from the New Challenge competition, we intend to contribute to this pervasive challenge by addressing the issues of one of its sub communities, people suffering from attention disorders that affect their ability to focus and learn.

In 2007, the Center for Disease Control reported that 8.4% of American children aged 3-17 were at one point diagnosed with ADHD. Roughly 50% of children with attention disorders continue to experience issues as they progress into adulthood, and almost 60% of people diagnosed with these disorders are prescribed medication in an attempt to address the symptoms. It is vital that researchers continue to explore alternative and complimentary methods for solving attention-related disorders, and do not rely entirely on prescription medication to resolve the issue. Additionally, we believe that solutions to these problems have the potential to extend beyond the scope of individuals diagnosed with ADHD, and could be implemented by undiagnosed individuals trying to enhance their level of focus, learning ability, and productivity. It is this ubiquitous issue that we intend to examine.

Our Solution

To address this problem, my team of designers, engineers, and researchers has come together to found the Brain Design Laboratory (BDL). The goal of this community is to design, build, test, and rebuild non-invasive neurofeedback platforms that allow users to record environmental conditions over prolonged periods of time, while simultaneously tracking brain activity. In order to explore alternative techniques to addressing ADHD, we want to analyze the data that is recorded by these systems.

The systems will be comprised of a non-invasive headset that wirelessly sends brainwave data to a mobile phone and a central server, as well as a mobile application that tracks environmental stimuli both actively and passively. Passive stimuli will include variables such as location, noise, and movement, using GPS, audio inputs, and accelerometers. Actively recorded stimuli will include variables such as diet, activities, and moods, and will be input manually by the user. We believe this system will provide invaluable insight into how environmental stimuli correlate to variations in levels of attention. We will reach out to find user groups willing to test the platform. Eventually we hope to be able to provide real-time feedback to the user about how their environment is affecting their level of attention.

Currently, the potentials of commercial EEG have been used primarily for stationary recording and interaction, and do not serve as a good system for prolonged recording of brain activity. Some of the major shortcomings include comfort and attention to aesthetics. We believe that our diverse team of designers and engineers with experience in neuroscience, electrical engineering, as well as fashionable technology, can provide a new outlook on these problems, creating a system that is both wearable and functional. Lastly, we don’t want to just build technology; we strive to turn BDL into an open community of designers, researchers, patients, parents, and other organizations who are dealing with this problem.

Rough Budget

Item Cost Rationale
20xNeuroskyThinkgearChip $35 each The Neurosky Thinkgear chips (http://neurosky.com/Business/ThinkGearChipsets.aspx) are commercial
Electronics $2000 Bluetooth modules, Android testing platforms, electrodes, wires,
Materials $500 Garments, materials, and accessories for designing and building wearable devices. Including fabric, sewing equipment, hats, etc.
Website $500 We will use this money to establish our validity as an organization so that we can reach out to potential user groups for testing
Contingency Cost $1000 Miscellaneous expenditures

Our Qualifications

I first began trying to address this issue last spring when I designed and built a baseball cap with a sensor for recording brainwaves. To accompany the hat I developed a mobile application for Android that received and recorded the user’s EEG allowing for retroactive analysis of the data. The application also allowed the user to record a variety of moods and daily activities, the intention being to see how quantitative brain activity could be used to find new comparisons between the two. For more information about the project refer to: http://conorrussomanno.me/2012/06/19/interactive-android-application-for-eeg-biofeedback/

This Fall, with the support of former dean of Parsons The New School of Design’s Art Media and Technology department, Sven Travis, I founded the Brain Design Lab (BDL), a community focused on finding practical applications for brain-computer interfaces. Since it’s inception the community has grown and now has members both inside and outside of the New School community. Some of BDLs most prominent members include a recent graduate of Harvard’s Neuroscience and Education M.S. program, a University of Virginia graduate with a double major in biology and statistics, an award-winning journalist whose son suffers from an incredibly rare undiagnosed neurological disorder, and a graduate of Columbia University’s engineering program.

Recently we received $1,500 from the New School Student Activities Finance Committee to host a development jam titled Hack-A-Brain. The goal of the event is to explore the potentials of various front-line commercial EEG devices, while introducing New School students to the emerging industry of brain-computer interfacing (BCI). The Brain Design Lab has already connected with a number of individuals and organizations involved in the industry. Now we are looking to find additional support, make new connections, and apply novel design techniques to address problems related to the brain. We want to start by attempting to build user feedback applications for addressing attention disorders such as ADHD.

 

ABC No Rio – An Illustrated Short Story Prototype

I collaborated with two other artists, Tharit Firm Tothong and Giselle Wynn, on the creation of this illustrated short story for a class project. The piece pays tribute to ABC No Rio, an art gallery and concert space in the Lower East Side that has been in operation since the early 80s and was very active politically active during the late 80s and early 90s, acting as a sanctuary for society’s misfit demographics, as well as taking a strong stance of opposition to NYC’s heavy gentrification at the time. The piece speculates a fictional narrative from the point of view of a poor musician living in the slums of an overpopulated and depressed urban setting.

The piece is comprised of unpolished illustrations, done by myself and Giselle, as well as a collection of photographs of authentic artwork from within the walls ABC No Rio itself, taken by Firm. Firm also oversaw the design and layout of the composition.

Bull’s Eye – Hand-drawn Animation

This hand-drawn animation is of an archer readying and firing his bow:

Orbitorbs v2.1 – Solar System Simulator

Project Summary

This project is an extension of Orbitorbs v1.0.  I translated the code that I wrote in processing into Openframeworks, a C++ based programming language.  I added additional features that enabled more user control over the planetary system including:

  • The ability to pause the solar system simulation and edit planet parameters
  • A more intuitive interaction for editing planet parameters
  • The ability to turn on and off a function that links the computer microphone volume input to the strength of the gravitational constant dictating the force between the planets (activate by pressing the ‘e’ key and deactivate by pressing the ‘s’ key). The higher the volume, the higher the g-constant (directly proportional).

The algorithm uses 2-dimensional matrices to store the x and y parameters of the various planets and it implements Newton’s Law of Universal Gravitation:

newton

This project has the potential to be adapted into a new type of learning tool, allowing for a more fun and interactive method for teaching basic principles of physics including angular acceleration, gravitation, ideas of mass and density, and more.

Orbitorbs v2.1 (openframeworks) from Conor Russomanno on Vimeo.

The Code

If you want to play with this application or examine the code, please feel free to grab it from my github.

Blog at WordPress.com.

Up ↑