SoundOfTwitter Part 1: Local Application

Update 2023-03-25: I’ve had to take this down thanks to some fuckboy’s API changes.

One of my favourite sites, for years, has been Listen To Wikipedia, a soothing project in the idea of beauty and art coming from systems. Years back I set up the [Holiday by Moore’s Cloud] lights with a script that would automatically hook into the streaming of Wikipedia edit notifications and change in time to them, and the music if the web app was open concurrently.

I decided to use this platform as inspiration for a creative coding exercise, receiving input notifications from some popular streaming API and turning them into a soundscape that you could tune into.

While GitHub does offer a “live” view of the public activity on the website, there’s no streaming functionality and it would require constant polling. My second selection, Twitter, offered just what I wanted with their new V2 API offerings. I considered the sampled stream but I’m not sure how easy I could handle that on the small connection I was using, plus it might be too cacophonous with even 1% of the traffic being blasted through it.

I settled for the filtered stream, logged into my Twitter development, and got access to the developer API to create a new application.

I used Postman to set up some rules and cURLed against the endpoint to see what gave a good tweet:silence ratio. Fittingly enough, it turns out the stream of people tweeting the hashtag “#NowPlaying” is perfect. To enable that as a rule, I POSTed the following to the https://api.twitter.com/2/tweets/search/stream/rules endpoint.

{
    "add": [
        {"value": "#nowplaying", "tag": "now playing"}
    ]
}

Locally I made a blank index page and included howler.js to handle audio. For music I used a pack of public domain harp samples by Samulis on Freesound and converted them to MP3 and OGG using SoX. I named them sequential numbers (grouped by sound) and imported them like so:

let sounds = [];
for (let i = 100; i < 123; i++) {
    sounds.push(new Howl({
        src: ['sounds/' + i.toString() + '.ogg', 'sounds/' + i.toString() + '.mp3'],
        html5: true
    }));
}

This let me play them via a websocket quite easily:

const socket = new WebSocket('wss://localhost:5000');
socket.addEventListener('message', function (event) {
    sounds[Math.floor(Math.random() * (123 - 100))].play();
});

Reading the filtered Tweet stream was done with Node.js, I simply set up a WebSocket connection and a streaming HTTP connection. When the former received connections, each was added to a list. When the HTTP connection to Twitter received data, it send a ping message to each WebSocket listed as connected. Roughly it could be modelled like so:

const WebSocket = require('ws');
const wss = new WebSocket.Server({ port: 5000 });
let wsConnectionList = []
wss.on('connection', function connection(ws) {
    wsConnectionList = wsConnectionList.concat(ws)
});

const stream = needle.get(streamURL, {
    headers: {
        "User-Agent": "v2SampleStreamJS",
        "Authorization": `Bearer ${token}`
    },
timeout: 20000
});

stream.on('data', data => {
    wsConnectionList.forEach((ws) => {
        ws.send(JSON.stringify(json));
    });
});

Note the example above doesn’t handle disconnects from the WebSocket server, nor errors in the HTTP connection, that’s an exercise for the reader.

With this set up, you should be able to listen to the streaming sounds of Twitter.

Az

SoundOfTwitter Part 1: Local Application