SoundOfTwitter Part 3: VPS Time

TL;DR: Aural - Sound Of Twitter changed hosting and stream!

Well! A bit changed since last post. Most prominently: yesterday I received a missive from AWS, a preset alert that I was going over my established high water mark for expected bill this month (not even halfway through!). It looks like running API Gateway for websockets is spenno. Not hugely expensive, but in less than a week it was already $5, over half my AWS spend and looking to be way more expensive than hosting on an existing VPS.

To get it running on a VPS I went back to the work from my first post. While trying to test it locally again, I noticed the other problem: my Twitter API requests to the filtered stream endpoint had started terminating immediately upon connection. Logging into the Twitter Developer centre I discovered I’d gone over my limits for tweets delivered through the filtered/search stream. So I swapped back to the sampled stream and discovered it does come through but when attached to the music component it feels like a full-on blast. So as a naive way of paring down the stream I decided to only fire a websocket message on every 45th tweet. There was no real scientific reason, I just kept connecting at different levels (50, 25, 30, etc) until it sounded relatively mellifluous.

Once that was all set up and deployed I ran into my final problem: the website is delivered over HTTPS (still with AWS Amplify) and I didn’t want to remove TLS security but with it you can’t connect to an unsecured WebSocket connection. To get around this, I pointed a domain at the VPS that was running the WebSocket server, installed Certbot and set up a HTTPS server in my Node script. This can be easily done like below and integrated with the WebSocket server to upgrade the security on the connections:

const fs = require('fs');
var https = require('https');
const WebSocket = require('ws');

var privateKey = fs.readFileSync(process.env.SSL_KEY, 'utf8');
var certificate = fs.readFileSync(process.env.SSL_CHAIN, 'utf8');
var credentials = { key: privateKey, cert: certificate };
var httpsServer = https.createServer(credentials);

const wss = new WebSocket.Server({
  server: httpsServer

You’ll want to make sure the certificates are readable by whatever user is executing your Node script. It might make sense to copy them elsewhere but remember they need to be updated before they expire.

By now it should work! I ran into a slight problem with the sampled stream that after about 30 minutes it refuses to keep playing, so I set up an automatic restart on the script. I know that’s not “proper” and professionally I’d hunt down the root cause and a true solution but it was late and this is just a fun/cute project so I’m cutting that corner.