George Khut in collaboration with Dr Angie Morrow, Dr Melissa Yogui Watanabe, Caroline Dale, Dr Angelo Fraietta, Jason McDermott (ARUP), Frank Maguire, and students from UTS Faculty of Engineering & IT ‘Extreme Programming’ workshop

BrightHearts is an interdisciplinary research collaboration between electronic media artist Dr George Khut, and Dr Angie Morrow, a staff specialist at the Kids Rehab Department, of The Children’s Hospital at Westmead, in New South Wales, Australia.

In this exhibition, you are invited to interact with a collection of prototype iPhone and iPad apps that translate changes in your heart rate into colourful geometric artworks, that you can influence with your breath and mental/emotional focus.
The sensors used here are hospital-grade pulse oximeters (PureSAT OEMIII series) produced by Nonin – that we have adapted to transmit wirelessly via Arduino microcontrolers.

The BrightHearts research project is researching the potential of these playful, meditative interactions for the management and reduction of pain and anxiety experienced by children undergoing painful, recurrent clinical procedures as part of their treatment at hospital.

The recurrent nature of these procedures can result in a build-up of anticipatory anxiety, causing significant distress to the children, exacerbating the perceived intensity of the painful stimulus during treatment, and further complicating veinipuncture procedures in cases of extreme vasoconstriction. If left un-addressed, the intense distress and anxiety experienced during these procedures can lead to avoidance behaviors that may stay with an individual into adulthood.

Currently in its preliminary design-research phase, the next stage of the project will involve a pilot study followed by a clinical trial that will assess the effectiveness and impact of these interactions compared to standard iPad-based pain and anxiety management distraction methods.

The BrightHearts project was initiated by Angie after a chance meeting with George at an “Art-Science Speed Dating” event, hosted by the Powerhouse Museum in 2010. Research team members include Dr Melissa Yogui-Watanabe (research assistant), Caroline Dale (qualitative researcher), Dr Angelo Fraietta (electronics and software), Jason McDermott (visualisation software) and Frank Maguire (electronics and industrial design), and fourth-year students from the UTS Faculty of Engineering and IT, “Extreme Programming” workshop (heart rate analysis software).

The project has received funding from James Kirby Foundation (medical grant for design and clinical trials) and the Australian Network for Art and Technology and the Australia Council for the Arts (2011 Synapse artists residency grant). This project has been supported by the UTS Faculty of Engineering & IT: Interaction Design & Human Practices (IDHuP) Lab, and Creativity & Cognition Studio (CCS), and UTS School of Design Interactivation Studio.

As a continuation of my research into facilitating and working with audience-generated interpretive materials, I have been formulating a new body of work based around questions of historic and geographic interpretation, expanding from the body-focussed interactions I’ve been working over the past six years, to situation-focused interactions and reflections.

At this stage I am working on two separate but related interaction concepts: ‘Interactive Story Dice’ and the ‘World Telescope’. Both interaction concepts are intentionally open in scope – but have been developed with a view to deployment in community cultural development projects, arts in health and historic interpretation work.

The aim is to draw on local expertise (i.e. participants as having some special knowledge), and simultaneously engage participants in a process of listening to and reflecting on the experiences of other participants.

World Telescope…

The ‘World Telescope’ proposal (really just a working title!) is proposal for a mobile outdoor art project that combines pre-existing databases of world place names and coordinates with a hand-operated telescope-like interface, computer graphics and a spoken word story-sharing system. Audiences locate and describe places of personal significance (ancestral homelands, birthplaces, locations of loved-ones etc.) using a built in computerised voice recording system, and then LISTEN to other visitors stories (edited and moderated each night, by the artist) as they move around the Earth using the tilt and rotate interface. The work could take the form of a portable 1.5 meter-long device, mounted on tripod and powered by a solar-powered battery system. Audiences would rotate the device to point in the ACTUAL direction of a place on Earth they want to talk about or listen to (relative to the location of the telescope) – and then tilt it up or down to move closer or further away from their present location.

One idea I am considering presently – is to visualise place, country and region names on earth as vast collection of bilingually-labelled glowing points of light – distributed around an invisible sphere – not a representation of the earth per say – but a visualisation of human presence and relations across the earth.This idea is very inspired by the numerous start chart applications now available for the iPhone and Google Android.

Sensors inside the sculpture translate tilt and rotation data into global coordinates, from which bilingual place names, and participant voice recordings are mapped to. A Large LED text displays on the exterior of the device could reveal locations being viewed by the operator – transforming the device into DYNAMIC MILESTONE, providing casual onlookers with some clues as to the devices functionality i.e. “WIRADJURI COUNTRY/FORBES, NSW 386 km W”, or “SEOUL 8324 km NW” etc. Another option is to visualise the earth beneath the interaction as hollow – so that viewers point the telescope downwards, to look through to locations literally on the other side of the world.

Along the way I’ve been researching telescopes as augmented reality displays. Here are a few links that I’ve found particularly inspiring:

Jeffrey Shaw Melbourne Museum Animal Viewer
Lawrie & Friis
Unsworn IndustriesParascope
YDreams
Star Map for iPhone

How we visualise the Earth is a very interesting problem, especially how we represent this information in a 2 dimensional image. Buckminster Fuller’s Dymaxion Map is a very interesting approach to this problem – one that approaches the process of flattening out a sphere in a very dynamic way – making it very clear to the user the multiple aproaches one can take to this task. These elegant, folding polyhedral forms, suggest interesting possibilities for sculptures, along the lines of Brazillian artist Lygia Clarke’s folding ‘Bichos’ sculptures.

Interactive Story Dice… play and replay

The ‘Interactive Story Dice’ concept takes its inspiration from Rory O’Connor and Anita Murphy’s prize-winning Story Cubes improvisation game, (now also available as great looking iPhone application!) but focuses on the use of tilt sensors to record and then playback participant responses. Each face of the three die would be used to display a range of images designed to embody key moments in history or issues of significance for a given exhibition or community engagement process. Participants would then be invited to assemble the dice onto a sequence of images that has particular meaning for them. When they are ready, and in the presence of an exhibition facilitator, they can then contribute a spoken interpretation of this arrangement of images, relating it to their own experiences, beliefs and interests, and their understanding of other perspectives connected to the issues they have raised. Tilt sensors (and compass perhaps) readings are used to tag the orientation of each die. These stories are recorded along with dice tilt data, and played back on a separate but visually identical set of cubes, when that specific configuration of images (dice tilt data) is re-assembled. The experience of exploring seemingly random image combinations is transformed into a story-sharing experience.

In all these examples (including the Heart Library Project) I’m working with interaction as a framework for generating and reflecting on participant experiences–in ways that the participants themselves can access and reflect on.

I’ll be working on addressing hardware and software requirements for both of these interfaces during my residency at Fraser Studios over the coming three months, with assistance from Jos Mulder and Frank Maguire. Most of this will focus on assembling and testing tilt and rotation sensing functions with Arduino and Zigbee wireless communication.

Google Earth phone application…

A couple of weeks ago I took part in a three-day Feldenkrais Practitioner’s Advance Training event, led by my colleague Maggie Slattery, and produced by fellow practitioner Fran Chambers, in Howden, Southern Tasmania. Maggie had invited me to record some video-cued recall (VCR) interviews with practitioners at this training. Catherine Truman, another member of the Thinking Through The Body research group, also attended, as a practitioner, and also as a creative arts researcher working with neuro-anatomist and teacher Ian Gibbins at Flinders University, in South Australia.

The focus of this training event was on Feldenkrais Functional Integration work (F.I. for short) – the hands-on work that Feldenkrais practitioners do with clients on the table/bench in clinic – gentle manipulation, sensations that awaken patterns of moving and responding that can provide alternatives to painful and self-limiting movement habits we can unconsciously acquire as a result of injury or environmental factors.

Maggie and I wanted to continue our exploration of video-cue recall as a tool for studying what happens in a Feldenkrais Functional Integration lesson. Here’s a video we made last year as part of research at Thinking Through The Body: Catherine Truman operated the video camera, I was the client/subject and Maggie was the practitioner.

Note: this video is best listened to with a pair of over-ear headphones or through an amplifier with Left-Right Balance controls. You hear Maggie and I speaking simultaneously – Maggie in the left, and me in the right – which can be confusing – but works well with headphones – because they allow you to move your focus between our contrasting recollections.

You can hear me struggling for words to describe my own experience in the video… this was surprising and disappointing for me initially, but ultimately quite productive, in that its forced me to look deeper into what we call ‘experience’ – especially in relation to body-focused interactions.

At this Practitioner Training event in Tasmania, my aim was to learn more about the experience of touch and movement in this type of therapuetic/educational work – and to consider how the sensibilities at work in Feldenkrais (and somatic-bodywork more generally) might be incorporated into my work with body-focussed interactive art practice, and also as a student of the Feldenkrais Method – to explore the group – some details ofp it was we are working with when we do this work.

Maggie’s aim was to see how VCR might work as a training tool – to allow practitioners to reflect on the two very different perspectives operating within this interaction.

We agreed that the recordings we made where for private research purposes only – so I won’t be showing them to anyone else – but the whole event certainly raised some interesting questions and possible directions for further research.

We recorded a total of three separate Functional Integration sessions – and from these sessions, recorded four individual interviews and one dialogue – where both the practitioner and client watched and discussed the video of their Functional Integration (F.I.) session together.

The most striking thing about the interviews we recorded – was the seeming disparity between the recollections of the practitioner and client. Where the practitioner is negotiating through layers of observation, listening and experimentation, the client seems immersed in an often undifferentiated flow of gentle and sometimes powerful sensation, punctuated by moments of observation, insight and/or speculation as to what the practitioner was doing with them. This was Maggie’ primary interest in the VCR method for this workshop – to encourage the participants to consider the radically different perspective of the client, and what this might mean for the practitioner in relation to how they structure the session and communicate with the client. Underpinning this work with VCR was a set of much bigger questions that Maggie was asking the practitioners to explore: what is intention in this work with your clients?; how does intention evolve over the course of an FI session, and how do we meet physical resistance and stuckness in FI work? (Apologies to Maggie here for my gross simplifications of her workshop!).

Throughout each day Maggie led a series of Feldenkrais Awareness Thuough Movement (ATM) lessons – with a strong emphasis on how we negotiate resistance within our own bodies, and how effort plays out in this negotiation. Not being a practitioner myself – I found these lessons extremely challenging: lying on your back – raise your head and left foot off the ground but without effort or exertion – when you notice yourself efforting, back off from this, and see if you can find a way to do it without effort – can you do this in a way that is not about having to be good at it – but as a genuine question and exploration? These lessons certainly made me stop and consider some habitual approaches I bring to many tasks in life – the bull changing at the gate vs. the ants slowly nibbling away – the difference between trying to force some poorly conceived static outcome and coordinating a series of movement processes that have as their conclusion a certain bodily configuration. Big stuff – that reverberates at lots of different and very personal levels.

Each day we scheduled some time for the group to watch some extracts from the VCR interviews we had recorded – to observe these recollections together, and discuss various issues and aspects of the FI session that was recorded. Not surprisingly when it came to the VCR work, participants felt slightly uneasy with the idea of being video recorded. Invariably the presence of a camera produces a certain self-consciousness, and in the context of this training we had to emphasis that our intention was not to be judgmental – but simply to discover what we each could learn from the experience of observing each other and ourselves.

One issue we encountered in relation to the individual VCR interviews was the tendency to for people to feel like they where being assessed – assessed as practitioners – and assessed for the ability to recall as much as possible. All participants made a point of apologizing to me (the test operator/research) for not being able to recall much – whereas I thought they had done well to remember as much as they did. On the other hand – the conversational recollections – where both client and practitioner watched the video of their interaction together – where much more relaxed (alas – we didn’t record this conversation – but maybe that had something to do with the outcome!) – and this looks like a useful variation to pursue in future trainings.

Personally, the combination of all of this (awareness through movement lessons, coordinating and recording the interviews, editing the sound and video) left me feeling slightly exhausted and confused – what is experience? was I getting any closer to an understanding of what happens in this work with the body? What is the role of intention, language, etc. At one point I was reminded of the story of Plato’s Cave:

Plato imagines a group of people who have lived chained in a cave all of their lives, facing a blank wall. The people watch shadows projected on the wall by things passing in front of a fire behind them, and begin to ascribe forms to these shadows. According to Plato, the shadows are as close as the prisoners get to seeing reality. He then explains how the philosopher is like a prisoner who is freed from the cave and comes to understand that the shadows on the wall are not constitutive of reality at all, as he can perceive the true form of reality rather than the mere shadows seen by the prisoners.

…Our speculations on experience and learning in FI being like the speculations of the cave dwellers on origins of the strange shadows on their cave wall – unaware of the origins of these shadows in the actions taking place beyond their view in the world outside the cave.

The role of language and conversation also came under examination during this training event – and in this I was reminded of the work of hypnotherapist Milton Erickson <…very neat introduction via Wikipedia> – that a conversation between a client and practitioner may not always be what it seems, as exemplified by his Covert or Conversational Hypnosis methods.

All this also got me me thinking about the work of Shaun Gallagher and others on Body Image and Body Schema that I had come across some years ago while reading for my doctoral research – that there are levels of body experience and cognition we can not consciously access. To be honest I find this stuff very difficult reading – and its not a subject I understand well at all – but its obviously something I’m going to need to understand better if I’m to progress in this line of inquiry. One of the problems when working accros disciplines of creative arts practice, somatic bodywork, interaction-design theory and philosophy is the lack of a shared language for describing all of this. A quick Google for “Body Image Body Schema” led to a very interesting collection of writings on this very subject:

“Body image and body schema: interdisciplinary perspectives on the body”
edited by Helena de Preester, Veroniek Knockaert, John Benjamins Publishing, Amsterdam, NL, 2005.

A quick look through this collection on Google Books – reveals there’s already a lot of great research being done around this question of body experience – but its going to take me a while to work my way into it! Note: by interdisciplinary they mean between the disciplines philosophy, psychology, neuroanatomy and cognitive sciences – NOT somatic bodywork practice and interactive art! A book club anyone?

One of the papers in this book “Vectorial versus configural encoding of Body Space: A neural basis for a distinction between Body schema and Body image” by Jacques Paillard can be accessed online as a PDF. In this paper he includes an simplified representation of this distinction that I find very interesting:

Two levels of information processing in the nervous system, (modified from Paillard 1980) in

This is clearly a very busy research field for psychologists, philosophers, neuro-anatomists and the like – I’d be very interested to know of any bodywork practitioners, (and Feldenkrais especially) who are following this – if you are please get in touch!

Included in the exhibition
Super Human: Revolution of the Species, RMIT Gallery, Melbourne, Australia, 2009

Distillery (Alembic & Retort), continues my exploration of heart rate biofeedback – but this time concentrating entirely on sound as a way of tracking changes in heart rate patterning – with no interactive visuals.

Sensors embed in the surface of a simple table enable participants to slowly navigate through layers of electronic sound using emotionally-mediated changes in their heart rate patterning. Alluding to alchemical notions of slow-burning psychological ferment and distillation, the work re-frames our psycho-physiology as space of poetic, elemental transformations using the sounds of heat, condensation, air and stone. Interactions require a minimum commitment of ten minutes – as the work measures extremely low frequency oscillations in heart rate patterning that can take up to five minutes to become fully formed – hence the alchemical metaphor of distillation.

Audio documentation coming soon!

Exhibition presented by ANAT in association with the RMIT Gallery.
Artists: Justine Cooper, Jonathan Duckworth, Donna Franklin, Tina Gonsalves, Leah Heiss, George Poonkhin Khut, Angela Main, Brad Nunn, Jill Scott, Paul Thomas, Tissue Culture & Art Project, Mari Velonaki

Credits:

George Poonkhin Khut: interaction, sound and interface design
Sean O’Connell: sensor fabrication (stainless steel)
Vicky Sowry and Ruth Cross: ANAT curatorial team

Special thanks to Melinda Rackham, Dan Armstrong & Melinda, and the staff at RMIT Gallery

Super Human: Revolution of the Species
RMIT Gallery, Melbourne, Australia, 2009

Still images from the Heart Library Project, interactive video projection, as exhibited at Saint Vincent’s Public Hospital (Darlinghurst, NSW), 2009, demonstrating various patterns created by variations in the participant’s heart rate patterning.

Live video footage of the participant (lying on the backs, under the ceiling mounted video projection) is overlaid with colours and textures controlled by real time changes in the participant’s heart rate.

The photos with the holes in the centre, where taken of the actual ceiling project – the camera recording the participant’s image is located behind this hole.

Photographic documentation by Julia Charles.

Special thanks to Steve Evans, Rhys Cumpstone, Kate De Maere, Khahien Hyuhn, Margaret Kay, Palani Narayanan, and Yalla Foods.

Credits

George Khut: concept, interactive visuals and sound design
David Morris-Oliveros: visualisation software, photography, documentation
Naomi Derrick & Julia Landrey: interviews and experience mapping facilitation
Greg Turner: data analysis and mapping (Max-MSP)
Katarina Cvitkovic and Kim Vaughan: St Vincent’s Campus Arts Committee
Andræ Castaldi, Converse Media: video documentation

Level 4, Xavier Building, St. Vincent’s Public Hospital,
Darlinghurst, New South Wales, Australia
July 7th – 19th, 2009

Installed on Level 4 of the Xavier building, in a large waiting area just down the corridor from the Heart and Lung Centre, this was the first presentation of The Heart Library Project in a hospital setting.

Visitors to the Heart Library Project came from across the hospital community, from patients and their families, to nurses, pastoral care workers and cleaning staff, and we where able to record around fifty experience-maps over the twelve days of the exhibition.

This exhibition at St. Vincent’s demonstrated the impact and value of this type of art in a hospital setting – providing patients, visitors and staff alike with a space for quiet reflection and curious engagements. The video documentation (below) includes interview materials and drawing produced by participants at St. Vincents.

The exhibition was a produced in partnership with the St. Vincent’s Campus Arts Committee and dLuxMediaArts, with funding from an Australia Council Inter-Arts Office Research & Development Project grant.

Download Exhibition Programme Notes (PDF)

Credits

George Poonkhin Khut: Concept, art direction and sound
David Morris-Oliveros: Computer programming – visuals
Jason McSweeney: Computer programming – analysis
Caitlin Newton-Broad: Concept development and prototyping
Annie Laerkesen: Exhibition technical production
Andre Castaldi, Converse Media: Video documentation
Julia Charles: Photo documentation
Frank Maguire: Heart-sensor fabrication
Kaeko Tanigawa: Design consultant and fabrication assistant
Naomi Derrick & Julia Landrey: Body-mapping interviews

Special thanks to David Morris-Oliveros and Jason McSweeney for their patience and generosity; Katarina Cvitcovic, Kim Vaughan and Garry Nicholls for making this exhibition possible in the first place; Yalla Foods for the amazing opening event food; and all the volunteers who have helped to realise this project: Rhys Cumpstone, Naomi Derrick, Kaeko Tanigawa, Steve Evans, Palani Narayanan, Somaya Langley, Bec Dean, Kate de Mare, Julianne Campbell, Khahien Hyunh and Sam James.

Acknowledgements

The Heart Library Project: St. Vincent’s has been generously supported by the following organisations and individuals: the Australian Government through the Australia Council its arts funding and advisory body; dLux Media Arts; The St. Vincent’s Campus Art Committee; Yulla Foods; Fraser Studios and Queen Street Studios; UTS Gallery; UTS Creativity & Cognition Studios; Lizzie Muller & Kathy Cleland; Campbelltown Arts Centre; Lisa Colley and The Arts & Health Foundation; Bill & George Studio, Redfern; and Totugua Studios, St. Peters.

Backround – St. Vincent’s Campus Art Committee

The St Vincent’s Campus Art Committee’s was established in 2003 with an aim to consider, develop and promote the application of the humanities with a strong emphasis on fine arts within St Vincent’s Hospitals. St Vincent’s Campus Art Program manages a permanent collection of artworks exhibited throughout our buildings, together with two changing exhibition spaces on Xavier levels 3 and 4. At St Vincent’s, paintings, drawings, prints, photographs, ceramics and sculpture enliven the walls and the spaces of public areas and wards – embodying the hospitals commitment to duty of care, to the whole patient. All funding for the art program has been raised privately from individuals and organisations. The Campus Art Committee is comprised of volunteer representatives from across the hospital, business and arts community. The Heart Library Project builds on this work – extending the scope of their exhibition portfolio into the areas of live art and interactive media.

Press Coverage…

Press Coverage: Sydney Morning Herald, July 2009

Press Coverage: Sydney Daily Telegraph, July 2009

Drawings created by exhibition participants after their interaction with the Heart Library Project’s heart-rate controlled interactive video projection. Created and exhibited as part of “Enfoldings & Disclosures”, exhibition, UTS Gallery, Sydney, Australia, 2008.

This exhibition included all the key elements of The Heart Library Project: the biofeedback interaction, the facilitated experience-mapping process and the recording and exhibition of video interviews, made with participants who had interacted with the biofeedback part of the work.

For this show we used large format body-map templates – almost life-size, which invited a very expansive drawing style from many of the participants.

The video below compiles some of these interviews, along with edited footage of the drawings being made – filmed from above with ceiling mounted video cameras.

Credits

George Khut: project concept & production, interactive visuals and sound design
Caitlin Newton0Broad: concept development & production, interviews
Greg Turner: Max-MSP interaction design
David Morris-Oliveros: visualisation & lighting software, photography
Naomi Derrick: interviews
Sam James: video documentation
Tanya Creighton & Holly Williams/Anneke Jaspers: UTS Gallery curatorial team

Still images from the Heart Library Project, interactive video projection, as exhibited in “Enfoldings & Disclosures” exhibition at UTS Gallery, 2008, demonstrating various patterns created by variations in the participant’s heart rate patterning.

Live video footage of the participant (lying on the backs, under the ceiling mounted video projection) is overlaid with colours and textures controlled by realtime changes in the participant’s heart rate.

Special thanks to Julieanne Campbell and Bec Dean.

Credits

George Khut: concept, interactive visuals and sound design
Caitlin Newton-Broad: concept development, production, interviews
Greg Turner: data analysis and mapping (Max-MSP)
David Morris-Oliveros: visualisation software, photography, documentation
Naomi Derrick: interviews
Sam James: video documentation
Tanya Creighton, Holly Williams, Anneke Jaspers: Curatorial team

Research and Development collaboration with Caitlin Newton-Broad, as part of “The Living Room Project” artist residency, Performance Space, Redfern, Australia, 2007.

Established in 2004 by Sheila Ghelani, Caitlin Newton-Broad and Emma Wolukau-Wanambwa the Living Room Project was an artist led critical and creative group set up to support the development of their respective solo practices, all of which, at that time, were going through a period of transition.

George Khut worked with Caitlin Newton-Broad to research and develop approaches and networks for a hospital-based interactive art project, that would combine George’s work with biofeedback interactions, and Caitlin’s work with community story-sharing and live-art events.

During their residency they interviewed numerous arts-health practitioners, producers and therapists, to asses the viability of this type of work in a hospital setting, and to explore ways of facilitating the sharing of uncany, body-focussed experiences.

The body-map has been used in medical practice as a way for patients to localise sensation within and across their body. Caitlin and George used this format as an alternative to conventional interviewing methods. The process of silently gathering ones impressions in the form of an illustrated map, allows for the prioritisation of otherwise hard to describe sensations, and provides an evocative framework for the recollection of experiences.

Credits:
Caitlin Newton-Broad and George Khut: Interaction concept development
George Khut: Interaction design and audio-visual documentation systems
Neil Simpson: Production manager
Heidrun Lohr and Julia Charles: Photo documentation

‘This Secret Location’ (exhibition), as part of ‘Inbetween Time: Festival of Live Art & Intrigue’, Arnolfini, Bristol, Great Britain, February 2006.

‘I Took a Deep Breath…’ (exhibition), Perth Institute of Contemporary Art, as part of Biennial of Electronic Arts Perth (BEAP), Australia, 2007.

The third iteration of the Cardiomorphologies project, based on the same basic heart and breath visualisation scheme, but exploring permutations between hard and soft edged visuals from earlier versions of the work.

Credits

George Poonkhin Khut: Original concept, interaction design and art direction
Greg Turner: Data analysis, mapping systems, and video software (Max-MSP)
Lizzie Muller: Interaction design, curatorial research, BEAP 07 catalogue essay
Helen Cole: Curator (Live Arts), Inbetween Time, Arnolfini
Hannah Mathews: Curator, Perth Institute of Contemporary Art
Eva Fernandez: Photo documentation

Special thanks to Antoanetta Ivanova (BEAP 07), Fiona Winning, and Freemantle Arts Centre.