What music can learn from the movement of birds and fish

Poetry in motion … starlings flock above Brighton's derelict west pier. Photograph: James Warwick/Getty Images

A strange thing about music is the chemistry between the composer and the musician who plays the music. Music written hundreds of years ago comes alive in front of our eyes in the hands of today's musicians. As a composer, as much as I like to be in control, I love the moments and spaces where musicians put themselves in the mix. It creates an incredible chemical reaction – one which I can only imagine as I sit writing the music at my desk.
When I wrote my work for solo viola, Flux, I was particularly interested in the bowing arm – how fast the bow will travel against the strings to make an actual sound, the acceleration of it and the slowing down. Cassette tapes were still around when I was growing up in mid 80s in Japan, and I vividly remember the sensation of the tape winding down when the tape recorder's battery was running down, and how it felt both emotional and romantic listening to the stretched phrases and pitches going down.

This memory inspired me when working on Flux. I imagined the work starting energetically and feisty, before the melody gets longer and stretched out like on an old tape machine. As the bowing slows down, the music gets more and more emotional. I imagined it like looking like melting chocolate, or ice-cream, slowly moving into shape, elegantly. Every time it is performed, the viola player has to bring his or her emotion into it and decide for themselves how, precisely, to "wind down". As a composer, I have to hand over control.
I have spent many nights studying and admiring the beauty of swarming fish and birds on YouTube. I have always wanted to be part of a shoal or flock. Swarming birds and fish have a direction, but they are free, flying wildly but with an elasticity of overall shape, not competing to lead but flying in formation.
I wondered how I could express in music what I saw. How I could control the overall sound, while each individual phrase contributes to the overall shape. The shape of the flock or shoal is constantly moving; sometimes it becomes linear, at other times it spreads out to cover the whole sky. The movement and shapes formed by swarmings are utterly beautiful.
I imagined every line of my music to be like a single fish or bird in a swarm. How they get together, musically how they play in unison, and sometimes how they melt into a big, wide shape is something I can carefully orchestrate.
Frozen Heat is a piano work I composed when I was still at music college. A friend of mine asked me to write some music for her final exam at college, so I wrote this piece. I've disregarded a lot of works from my student days, but this is one of the few pieces people continue to play. I wanted to write something with a machine-like element, though not completely repetitive. I imagined that there is energy and bursting heat inside a frozen box like an ice cage, which are fighting to break through the box. I was attracted by the idea of being emotional inside, but cool on the outside. Wild but controlled.
When I hear a sound, I don't just hear the sound or see the colour (as some people say they do), but I can also see the movement of an object. I explore what would it be if I could put this musical material (it could be a phrase, or a chord played) in my mouth; what sort of taste and feeling would it have? What if I put this chord against my cheek, what sort of temperature does it have, or how would I feel it if I were to rub it against my cheek? Hearing music is a sensual experience of swarming.
Swarming needs constant movement and variation and I seek out collaborations with other artists wherever I can. I have recently worked with the experimental rock musician, David Sylvian, whose music I listened to in my bedroom in Japan when I was in my early teens - had I only known he would become one of my closest friends and that we would get the chance to work together 20 years later!
One of my most recent compositions, Fluid Calligraphy is a piece for violin and video that I composed in collaboration with the video artist Tomoya Yamaguchi. When this work is performed live, the speeds of the video are triggered by the dynamics of the violin, and in other parts the violinist loosely plays along with the video. I met Tomoya after seeing his beautiful exhibition (he is a painter, and had never made video works before I came along), and we became very good friends. I love how in his paintings he uses curved lines, but paints the background of the painting, and not the actual lines, so the lines are the colour of canvas, and the rest of the area is painted. I thought using the violin, which is a linear instrument, would work perfectly with his obsession with elegant curved lines in his paintings. Often in sound and video collaborations the sound is the one controlled live and the video is just secondary. I wanted the video to be as important a live element as the violin part. Each performance of Fluid Calligraphy can be a different length: the violin player has to make his interpretation and interact with the video, rather than worrying about catching up with the timing.
Collaborating – or working in swarms – taught me a new way of listening and a new approach to music. I learned not to try to push the sounds, but just let them "be" and let them sound. The same applies to the structures in the concert hall. What happens if we don't seat people and tell them what to do and what not to? I can't wait to watch when people will be swarming in the Old Vic Tunnels for Yellow Lounge on Friday.
• Dai Fujikura will be at Yellow Lounge at the Old Vic Tunnels, London SE1 to present his video installation Fluid Calligraphy, Flux and Frozen Heart will also be performed.