> **Tech Lead** · 2017
> Source: https://www.simonbetton.com/case-studies/fight-for-territory.md (markdown only; there is no HTML page for this document).
> Index of case studies: https://www.simonbetton.com/case-studies/index.md

# Case study: Fight for Territory

## Quick facts

|  |  |
| --- | --- |
| **Role** | Tech Lead and lead developer, DDB New Zealand |
| **Company** | DDB New Zealand (DDB Group). Media placement by APN Outdoor |
| **Client** | Lion, for the Steinlager brand |
| **Product** | The camera, screen, API and operations software behind the Fight for Territory installation at Auckland Airport during the 2017 British and Irish Lions tour |
| **Timeline** | 2017. A first stable release, then a follow-up that added automatic playback when a camera or the network dropped out |
| **Scale** | 65 interactive digital screens across the domestic and international terminals, as reported by APN Outdoor and the agency |
| **Team** | I built the software that connected cameras, screens and operations, and chose the hardware. Two other front-end developers are credited on the campaign. Brand films, stills and the campaign idea were the creative team's |
| **Stack** | PHP (Slim 3, Twig, Propel 2), MySQL, Redis, Raspberry Pi 3, Intel NUC, tracking.js, GSAP TweenLite, Zepto, Webpack 2 |

## What the campaign needed from the software

Steinlager sponsored the All Blacks. Guinness sponsored the Lions. The campaign idea was that Steinlager bought every advertising screen in Auckland Airport, where touring teams and fans spend more time than in stadiums, and offered the screens to Guinness for free, provided Lions fans fought for them: stand in front of a screen and it turns your colour.

That idea only works if a row of unattended kiosks in an airport behaves like a single game for the length of a tour. A person stands in front of a portrait or 4K screen. A camera watches them. If a face stays in frame long enough the screen runs a detect sequence, then a branded takeover: black for the All Blacks, red for the Lions. The next face can take it back. The screens have to keep doing this while airport Wi-Fi drops, a USB cable gets knocked, and units get plugged in by installers who will not edit a config file.

The failure modes are boring and public. A frozen detect graphic in an international terminal is worse than a quiet brand film, and there is no deploy window in the middle of a tour.

## What I built, and what I did not

I was the lead developer and built the installation software: the Raspberry Pi camera page, the Intel NUC screen package, the JSON API that held territory state, and the operations dashboard that registered, paired and watched the units. Two other front-end developers are credited on the campaign; the Pi page, the screen package, the API and the dashboard are my work.

The campaign concept, the six takeover films and the instruction stills were the creative team's. Media placement was APN Outdoor's. I chose the Raspberry Pi 3 and Intel NUC pairing because the units had to fit inside the display housings. I did not procure the hardware, but I supported the installation on site, which in a few cases meant getting inside a baggage carousel to fit a unit.

## Constraints

The screen package had to be static HTML ingested by a signage CMS on each NUC. That CMS would only reach the outside world through its own proxy endpoint, with the target URL encoded as a query parameter. Every live request from a screen was therefore an encoded URL to a proxy I did not control, over plain HTTP.

The camera was a Raspberry Pi 3, powered over USB from the NUC driving the screen, running Chromium. Face detection had to happen in that browser on a 280×210 capture, because a native binary would have been a second deploy, a second language and a second thing to debug in a terminal.

Devices would be plugged in by people who were not going to configure them. A unit's identity had to survive a refresh and still be useless until someone in the office registered it and linked the camera to a screen.

Operations needed answers in seconds, from a phone, standing in a terminal: is this NUC online, is its Pi online, which side holds it, and can I force a flip from here.

## Architecture

Two kiosk frontends and one operations frontend share a Slim 3 JSON API and a MySQL schema. Pis post presence. Screens poll presence. Operations registers, links, locks and watches heartbeats. The CMS hosts the screen HTML and proxies its API calls. If a camera or the network fails, the screen plays takeovers on a timer.

```
person
  -> Raspberry Pi 3 webcam (Chromium, Media Capture API, tracking.js)
  -> POST presence percentage + LAN IP
  -> API + MySQL (display value, heartbeats, pairing)
  -> GET display JSON (through the signage CMS proxy)
  -> NUC screen (GSAP detect scene + HTML5 video takeover)
```

The Pi page and the screen package shared no code. They shared a JSON contract and a pair of identifiers in browser local storage.

## Decisions and trade-offs

### Presence is a percentage, not a boolean

A face is a noisy signal. tracking.js runs a Viola-Jones classifier and will fire on a poster, a reflection or someone walking past. One detection must not flip a 4K screen. So the Pi does not decide anything. It throttles the tracker's events to one every 250 ms, counts consecutive hits, and posts the ratio of hits to a "wait for trigger" window as a value from 0 to 100. If the face leaves for longer than a last-seen buffer the counter resets and the value falls. A 15 second watchdog posts a zero if nothing has moved, and duplicate zeros are not sent.

The screen turns that number into a state machine. Below a low threshold it shows the idle graphic. Crossing it starts the scan animation. Crossing a higher threshold (around 10) plays the takeover, locks animation, waits for the video to end, holds a no-trigger period so a lingering face does not immediately bounce the screen back, then unlocks.

The percentage bought hysteresis against walk-past false positives, let the screen start scanning before committing to a full-resolution video decode, and let operations see a unit warming up rather than a flag flicking on and off. The cost was tuning: threshold, throttle, accumulate window and cooldown all interact, so I made them remote settings. Each Pi reads a global tracker configuration, can override it per unit, and refreshes it about once a minute. Changing detection behaviour did not mean touching an SD card.

The percentage is also the only vision output the rest of the system ever sees. The screen never receives bounding boxes. Operations never sees a picture. What the platform stored about a person in front of a terminal webcam was a number and a LAN IP.

Campaign coverage described the screens as recognising which team a fan supported. The software I built detected a face holding position; the direction of the flip came from which side currently held the screen, with two preloaded films, black-to-red and red-to-black. Anyone could flip a screen. The game was that a Lions fan would walk up to a black screen and an All Blacks fan to a red one, and the software did not know or care who was standing there.

### Pairing is a human gate

On first load a Pi asks the API to mint an identifier, stores it in local storage and starts tracking. A NUC does the same for a display. Neither affects the live show until someone registers it in the dashboard and points the display at a Pi and the Pi at that display. The API answers with an application code beside the HTTP status: one code for "known but not registered", another for "not found". Screens treat both as "do not animate".

That is how you install a row of kiosks without a shared configuration file. Installers plug things in; units appear in the dashboard the moment they boot; a person links them. Identity is a minted string, not a MAC address, so a unit can be wiped and re-enrolled from the player, and retirement is a soft delete (status off, registration off) so a decommissioned NUC does not keep flipping the leaderboard. Pairing is a two-sided foreign key: moving a display's link from the dashboard clears the old Pi's pointer.

### Poll through the proxy, and make caching agree with the poll

The CMS proxy ruled out WebSockets. A screen instead issues a GET for a URL ending in .json four times a second. That survives a proxy, a cache header and a firewall that only likes HTTP, which is why it worked in an airport.

The cost is that caching becomes a correctness problem. If any layer caches the display document for longer than the poll interval, the screen lies. Hot identifier lookups sat behind a half-second application cache (Redis in production, APC locally), HTTP expiry and cache-control headers on the JSON were kept to one to five seconds, and writes from the Pi had to win that race. A stale zero looks like a dead room.

### Vision in the browser

Native OpenCV on the Pi would have been faster. It would also have been a second runtime to build, flash and debug at every site. tracking.js on a small muted video element was slow enough to need the throttle and a two-second accumulate window, and small enough to deploy by opening a URL. The classifier settings (initial scale 1, step size 3, edge density 0.1) are the usual trade: finer steps catch smaller faces and cost CPU the Pi does not have.

Operations also needed each Pi's local IP without anyone SSHing into it. The browser can learn that address from WebRTC: open a dummy peer connection against a STUN server, read the ICE candidates and SDP, parse the address. The player refreshed it every 30 seconds and attached it to each presence post so the dashboard could show where on the local network a Pi sat. That is the whole WebRTC story; nothing was streamed or controlled over it.

### Automatic mode, added in the second release

The first release assumed the camera and network would be there. Automatic mode was built and then improved remotely while the tour was running, once it was clear that in an airport a camera or a network link will go away, and it made failover a first-class runtime rather than a "please reboot" instruction. The screen samples the browser's online flag once a second and raises online, offline and change events on a small event bus. Offline, or with the Pi reporting unhealthy, the screen enters automatic mode and flips between the two films on a 60 second timer with the no-trigger delay zeroed so the loop can run unattended. When the Pi reports healthy again, automatic mode turns off and polling resumes.

### An operations dashboard that tells the truth in five seconds

Slim 3 and Twig render the leaderboard, displays and Pis behind session authentication. Lists show a five-second online window (a ping in the last five seconds is "online"; the display payload uses a wider five-hour window before it marks a camera offline), relative timestamps, registration toggles and the link between each NUC and Pi. The leaderboard can force a display to 100 and hold it there, or reset it to live zero. Editing a display loads its current API state so a locked value is visible before you save. If operations tick "fix value" the screen keeps that number even when the camera is quiet, which is how demos and rehearsals ran. Tracker settings are editable per Pi.

Realtime for operations is a two-second auto-reload with a hash flag so the checkbox survives the refresh. It is Bootstrap 3 and it is not fashionable. It answered the question a person on a phone in a terminal was asking.

## The JSON contract

The device API is JSON over HTTP with an open CORS policy and URLs ending in .json so that a CMS and a cache treat them as documents.

| Action | Shape |
| --- | --- |
| Mint a display or a Pi | GET, returns an identifier |
| Read a display | GET: value, linked Pi, fixed flag, camera-online flag |
| Read a Pi | GET: linked display and its value |
| Post presence | POST: presence value and LAN IP |
| Read tracker settings | GET: per-unit JSON, or the global fallback |
| Leaderboard | GET: grouped counts of display values |

The body carried its own status beside the HTTP status: 340 meant the identifier existed but was not registered; the not-found case reused the number 500, which collides with HTTP's server-error code and is not a choice I would repeat. If a Pi has been quiet for ten seconds the API reports zero for its display unless the value is fixed.

## Security: what I locked down and what I accepted

Locked down: dashboard sessions, hashed passwords using PHP's password hashing API, CSRF tokens on login (32 random bytes, session id rotated on check), Google reCAPTCHA on login, only registered units affecting the live show, and no image ever leaving the camera. The dashboard's home route is a deliberate 404; the product was not a public website.

Accepted: an open CORS policy on the device API, unauthenticated mint and presence posts for anyone who knew or could brute-force an identifier, and plain HTTP on the signage side because the CMS proxy gave me no choice, while the Pi talked HTTPS to the same host. The threat model was physical units on a venue LAN, not a public web app. I would still put a shared device secret on those posts today, and I would not repeat the mixed HTTP/HTTPS split or the wildcard origin.

## Outcome

The installation ran across Auckland Airport for the 2017 tour. APN Outdoor and the agency reported 65 interactive digital screens across the domestic and international terminals plus eight static billboards, and around 90,000 fans approaching a screen to force a turnover. The agency also reported sales uplifts over the campaign of 18% for Guinness and 5% for Steinlager.

The campaign won three D&AD Wood Pencils in 2018 across the Media and Experiential categories, a Gold Spike at Spikes Asia 2018 and a One Show Merit.

The engineering outcomes I would defend:

| Outcome | Detail |
| --- | --- |
| Presence as a percentage | The Pi posted 0 to 100, not a boolean, giving the screen hysteresis against walk-past false positives and a scan state before a full-resolution decode |
| Human registration gate | Units appeared in the dashboard on plug-in and stayed inert until a person linked camera to screen; no shared config file on site |
| Vision as a web page | tracking.js in Pi Chromium, deployed by opening a URL, with detection behaviour tunable remotely per unit |
| Automatic failover | Built and tuned remotely during the tour: with the camera or network down, the screen cycled takeovers on a 60 second timer and returned to live polling by itself |
| Operations in five seconds | Online windows, pairing, locked values and a forced flip from a phone in a terminal |

I did not keep uptime or false-positive figures from the installation.

## Technologies and standards

| Layer | Choices |
| --- | --- |
| Pi player | jQuery 3, Media Capture API, canvas, tracking.js, lodash.throttle, WebRTC ICE/STUN for LAN IP discovery |
| NUC screen | Zepto, GSAP TweenLite, HTML5 video, SVG chrome; portrait 1080×1920 and landscape 3840×2160 packages |
| Operations | Slim 3, Twig, Bootstrap 3, timeago.js, session auth, CSRF, reCAPTCHA |
| API and data | JSON over HTTP, Propel 2, MySQL/InnoDB with soft deletes, phpFastCache (Redis in production, APC locally) |
| Build | Webpack 2 and Sass for the dashboard and Pi player; a separate Sass and Uglify pipeline for the screen package, which had to stay a folder of HTML, CSS, JS, SVG, PNG and MP4 for the CMS |
| Hardware | Intel NUC per screen, Raspberry Pi 3 per camera powered over USB from the NUC |

Standards and platform APIs in play: HTML5 video and canvas, the Media Capture API, WebRTC ICE/STUN/SDP, local storage, W3C custom events, JSON, CORS, HTTP caching, PSR-7 via Slim, PHP password hashing.
