Grigor Todorov

The Blue Violet River

The Blue Violet River was an interactive AR installation created for Abandon Normal Devices Festival 2021.

The installation took place on the Mersey Ferry and transformed the riverbank and landmarks of Liverpool into a fantasy world of creatures and sculptures. Eight custom-built tablet viewfinders were installed around the ferry, allowing visitors to look out across the River Mersey and see digital content accurately placed in relation to the real landscape.


Trailer

Walkthrough

The installation ran from 24 to 27 June 2021.

It was a technically complex project, combining real-time ship tracking, custom hardware, networked tablets, GNSS antennas, Arduino-based rotation tracking, a control system, live performance, and accessibility features.

The experience was made up of several connected systems:
  • Ship tracking: To track the ferry accurately, I used two Trimble Catalyst GNSS antennas: one positioned at the front of the ship, the bow, and one at the back, the stern. Each antenna was connected to a phone and powered by a power bank. The phone at the front received data from the bow antenna and transmitted it over UDP to the local network. The phone at the back received data from the stern antenna and also listened for the bow antenna data. By combining these two positions, I could calculate both the ferry's location and heading. The ferry location was calculated as the midpoint between the two antenna positions, while the heading was calculated from the vector between them. This gave us the position and rotation needed to drive the Unity scene displayed across all eight tablets.
  • Tablet viewfinders: Eight tablets were mounted to the ferry using custom plinths: four on the starboard side and four on the port side. Each plinth contained a wireless Arduino connected to the local network. When a visitor rotated a tablet, a metal rod connected to the tablet stand turned with it. This movement was read by a rotary encoder connected to the Arduino. The Arduino transmitted the rotation data over the network. Each tablet combined this rotation data with the ferry's GNSS position and heading to render the virtual world from the correct camera position and angle. This meant that each tablet acted like a fixed viewfinder into the same shared AR world, accurately aligned to its physical position on the ferry.
  • 3D printed tablet housing: Each tablet was placed inside a custom 3D printed housing. These housings made the tablets feel more like physical viewfinders than standard mobile devices, while also helping protect and secure them during the installation.
  • Networking: A 4G router was installed near the centre of the ship. This allowed all devices to communicate with each other over a local network and also connect to the internet. The internet connection was important because the GNSS antennas required access to Trimble RTX correction data. By combining satellite GNSS data with ground correction signals, the system could achieve very high location accuracy, in ideal conditions up to around 1 cm.
  • Control system: Because the ferry route varied between each run, I could not rely on a fixed timer or simple GNSS triggers to move through the different sections of the experience. Instead, I used a manual control system. A laptop sent UDP messages over the local network, and the tablets listened for those messages to change the AR content at the right moments. This gave the team more flexibility during each ferry journey and made it possible to adapt to real-world timing, route changes, and performance cues.
  • Audio: There was already an audio experience running onboard the ferry, so I could not use the ship's existing speaker system for our own sound. Instead, visitors were given headphones with built-in receivers. A radio transmitter onboard the ferry was used to broadcast the audio for the installation.
  • Performance: Each ferry trip also included a live dance performance. The AR experience, audio, physical movement of the ferry, and live performance all worked together as part of the same installation.
  • Accessibility: All eight tablet viewfinders were installed on the upper deck of the ferry. To make the experience more accessible, we also installed a large TV screen on the main deck, which played a recording of the AR experience for visitors who could not access the upper deck.
  • AR placement: A highly detailed photogrammetry map of Liverpool was used during development for AR content placement and occlusion. This allowed us to position digital creatures and sculptures in relation to real landmarks along the riverbank. Perfect alignment was difficult to guarantee in the live environment, so the occlusion feature was disabled for the final installation. However, the photogrammetry map remained extremely useful during development, placement, and testing. Because public testing was restricted at the time, I also built a simulation system. This used a simulated ferry path and the photogrammetry data to emulate the full journey, allowing us to test the experience without needing to be physically on the ferry.
Configuration systems:
  • River Mersey has the second-highest tidal range in the UK, with tidal movement of up to around 10 metres. To account for this, I built a tidal height offset into the configuration system. This could be adjusted during the installation to keep the virtual content aligned with the changing height of the river.
  • Each tablet received the ferry's position and rotation. An offset system was then used to place each virtual camera correctly relative to the centre point of the ship. Because the tablets were fixed in place on the ferry, no local positional tracking was needed.
  • All Arduinos transmitted UDP messages containing rotary encoder data. Each tablet could listen to any of the rotary encoder streams, and UI buttons in the configuration panel allowed us to switch between data sources when needed.
  • The tablets also used their internal gyroscopes for faster rotational response. The Arduino rotation stream was used for gradual correction relative to the ship, while the gyro helped the viewfinder feel more responsive in the moment.
Overseer phone app:
  • I also developed an overseer phone app that listened to all UDP signals across the system and provided warnings if a device stopped operating properly. Thankfully, the system performed very reliably throughout the festival. The antennas had 100% online time, and we never had a moment with fewer than seven of the eight tablets operational. Usually, the only downtime was when one tablet was being recalibrated. The main adjustment required during the festival was recalibrating for tide changes every few hours.
Network optimisation:
  • The 4G router used as the main network did not create a WiFi signal strong enough to cover the full 46-metre length of the MV Snowdrop. To solve this, I used a WiFi repeater to extend the network across the ferry. This allowed both GNSS phones to access the local network and the internet, which was essential for receiving the RTX correction signal used for higher accuracy.
  • As more devices were added to the system, including eight tablets, eight Arduinos, three phones, and the controller laptop, network latency became a problem. To fix this, I reduced the frequency of UDP messages and only transmitted data when it had changed since the previous message. An "alive" ping was still sent every few seconds to keep connections open and confirm that devices were still online.

Other notes:
  • Why use two GNSS antennas instead of one antenna and a compass? I initially tested a compass-based approach, but magnetic compasses were not reliable enough for this installation. They can be inaccurate near large metal structures or iron deposits, and the ferry itself is a large metal object. For that reason, ships usually use gyrocompasses, which are fascinating but also expensive pieces of equipment. Instead, I used two GNSS antennas positioned at the front and back of the ferry. By calculating the vector between those two positions, I could determine the ferry's heading relative to true north. This approach works well on a large vessel because there is enough distance between the antennas to calculate a stable forward direction. The Mersey Ferry was 46 metres long, which made this possible. Smaller ships or vehicles would require a more accurate or more specialised GNSS setup.
  • Why not calculate heading from the movement delta of one GNSS antenna? The ferry could rotate quickly before mooring. Depending on the tides, it could rotate 180 degrees in a matter of minutes to dock on the more favourable side. Because of this, calculating heading from movement direction alone would not have been reliable. The ferry's direction of travel and the direction it was facing were not always the same.
  • Why not use mobile phone GPS instead of dedicated GNSS antennas? Mobile phone GPS/GNSS is not accurate enough for this kind of installation. Phones cannot use the same ground correction signal, and atmospheric interference and other factors can reduce precision significantly. Even if two phones were positioned 40 metres apart, a location error of 4 to 5 metres could produce a very inaccurate forward vector. Since the entire AR world depended on accurate ferry position and heading, dedicated GNSS antennas were necessary.