In other news, Rhodri Davies, avant-harpist and musician for the project, has arrived, and plugged all kinds of spindly electronics and interesting devices into our PA system.

Since the move back to Director, things have been very fraught and extremely busy. I haven’t used it for a while, and I am annoyed by how buggy and temperamental it seems to be. Notwithstanding this, the work is being done and there is a visual piece of increasing complexity in development. I’m using Myron/WebCamXtra to get the live video in, and the XMidi Xtra to receive MIDI data from Max/MSP, where all the sensing is being done. So, we have a Processing sketch running Carnivore, which sends data to Max/MSP via MaxLink, which gets converted to MIDI and sent to Director using XMidi. In addition, we have Max/MSP reading variable amplitude from some contact microphones, converting to MIDI, and sending on to Director, and finally, we have input from the iCube, which is MIDI being sent via Max/MSP to Director. This was an interesting problem – it seems that you can only open one MIDI channel using the XMidi Xtra, so the iCube input had to be filtered through Max/MSP to be sent on the same channel as the rest of the MIDI data.

And for my next trick, I will juggle all the components…

The Max Patch

After much lengthy wrangling with Max/MSP/Jitter, it looks like a change of strategy needs to be put in place. The most useful aspect of Jitter is that its powerful video processing objects can be controlled live, and externally, using a variety of controllers and inputs. The particular aesthetic of the piece we are trying to make uses lots of composite images layered together on a single frame of moving image. While Jitter is excellent at producing video effects and transitions, it seems that compositing more than two sources of video is an extremely difficult task. The first problem is managing the program flow, something that is straightforward in most ‘code’ based packages. In Max, the flow of events is very difficult to manage, with various matrix objects requiring bangs to output, and various other input objects also requiring bangs at different times. With just two inputs, flickering images have been the result (and I don’t understand how the jit.qball object works).

The result of this is that we are going to resort to using Director to composite the assets and live video, but I am hopeful that we can still take external input from the existing sources (Processing running Carnivore; amplitude input from Max/MSP; and rotary switch input from the iCube) by using OSC to get everything to communicate. Not having used OSC before, I am expecting a few bumps, but the priority is that there is something to see next Wednesday or before, and as such I will be prioritising the construction of the screen based piece before getting it to communicate with external inputs.

After a number of conversations, the plan of action has now been decided upon and a schematic drafted. The only issues left to resolve (apart from implementation) are what the ‘performance computer’ which is on view to the audience is actually going to do with all this input.

The ‘home brew’ video streaming technology we have been testing has not been a huge success. We started trialling the broadcast capabilities of Jitter, which of course, didn’t work. The next set of experiments was with Quicktime Broadcaster and Darwin Streaming Server. Neither of these worked either. The reason is the complexity of the network we are behind. We are streaming from behind a wireless router, which it seems impossible to punch a hole in the firewall of. The academic network this router is attached to is specifically designed to stop people streaming stuff out, for reasons of copyright and so on.

After much wrangling and a huge amount of staring and worry, we discovered last night that even if streaming *was* possible, there is no guarantee that the stream will be successfully opened in Quicktime Player, as our first off-campus tests opened up in RealPlayer without us asking. Because time is a factor, we made the decision not to throw any more days at the problem, and to concentrate on the rest of the piece.

After a number of tests with sheets, drapes, various projectors and rigging, we have decided not to use projections in the piece. For all sorts of reasons, it really wasn’t working. See this terrible image and you’ll see what I mean:

The result of this is a lot more complexity on the computer side of things. More on this in a moment.

Yesterday evening, we did a quick trial to test the possibility of using projections in the performance. After getting a reasonable projector from Information Services, we all got very excited at the possibilities for bringing the network activity into the space in a visual way. The downside is the familiarity of the projected image in performance work such as this. Since it is such a common trope, we will have to do some clever thinking to make it both relevant and fresh.

This morning has been spent rigging the projector in a semi-permanent way and rendering images to test with. We will reserve judgement on the projections until the tests have been completed, not least because using projections means we will probably have to paint the set.

So, today I took the iCube out of the box and tried to get it working. I have previously had it working very effectively using the Editor software, and this worked well today also, with all four sensors that are supplied as part of the starter kit working perfectly in that environment.

The next step was to try and get input into Max/MSP. This proved to be much less straighforward. There is an iCube object supplied with the hardware, but after following the installation instructions, Max was reporting an error loading and reporting a ‘bogus’ object. This means that the object didn’t load during Max’s startup. The object itself is supplied without a file extension, and as such, OSX was reporting that it was a Unix Executable rather than a Max Object. After some protracted googling, I figured out that the extension for Max objects used to be .mxd, and after adding this, the icon changed and all seemed to be well.

However, on booting Max the next time, it reported “error: iCube.mxd not available for PowerPC”. I have yet to find a solution to this, as removing the extension just stops it loading again, as before. I sent an email to the iCube tech support too.

The good news is that receiving input from the iCube into Max is actually a lot more straightforward than the manual makes you believe. You don’t actually need the iCube object at all to take input from it: you just need to configure it in the Editor and then use the ctlin or notein objects in Max to receive the input just like any other MIDI data. I’m struggling a little with my MIDI setup, but it seems to work providing the notein is listening to the right device (in my case a Quattro USB Audio interface). You do need to configure the sensor in Editor to send the right sort of MIDI data for the receiving object (ie, there’s no point sending velocity to a notein etc).

After thrashing about with Quicktime Broadcaster, Darwin streaming server, and eventually Jitter, I am of the opinion that the academic network I am behind is not going to let an rtsp stream out to the wider internet. I can’t actually make it go beyond my own computer at the moment.

I am waiting to get in touch with the technical people here at the University of Aberystwyth to see if streaming is actually possible.

I’ve been spending the day trying to see what Jitter can do. At the moment, I am still very limited in my skills base, but I am starting to get some interesting (though not groundbreaking) results with live video. This extension of the previously posted patch has an increment in place of the bline object to twiddle with the hue, which was a major breakthrough in terms of my understanding. I have also used the pack and unpack jitter objects to split up the signal, and I figured out how to make it operate from a bang (and therefore from the incoming Processing/MaxLink bridge).