Az

SoundOfTwitter Part 2: Public Ready

Update 2023-03-25: I’ve had to take this down thanks to some fuckboy’s API changes.

TL;DR: Aural - Sound Of Twitter

After my last post discussing building a local site that lets you “listen” to Twitter, I decided to put it on the public-facing internet. But rather than just whack it on a VPS I decided to expand my horizons by putting a bunch of components through AWS.

My desired setup was:

  • Static website with AWS Amplify that could talk to;
  • An API Gateway configured for websocket that would register connections and disconections via;
  • Lambda serverless functions that would update;
  • DynamoDB which would receive connection info and new tweet notifications and stream the latter to;
  • Another Lambda which would read the DynamoDB for live connection IDs in API Gateway and let them know a tweet had arrived.

And it all worked (after much finagling)!

Of course, this doesn’t count for how the Tweets get into DynamoDB, and that is where my hubris overtook me. To prevent me from having to spin up a new server just to run this script continuously, I tried setting up a scheduled CI/CD job in GitLab that would execute the job for 20 minutes, and then finish the job and the next scheduled job would start. The main problem here is the disconnections from the Twitter API weren’t always counted and no matter how I wrangled it some jobs would fail and we’d be left with 20 minutes of silence. Shorter jobs didn’t work either, sometimes we’d lose sound for even longer as the scripts attempted to reconnect and were rebuffed by Twitter’s API limiting.

However I realised I had a server running some Node.js-based Twitter bots so I just added this as an extra process to be managed by PM2.

If you want to check it out, it’s called Aural and it’s available now! I haven’t tested compatibility beyond my PC and iPhone, if you run into problems let me know via Twitter or Facebook and I can take a look. I’m still working on it, so it may go through periods of unavailability while everything is smoothed out.

If you want to try this yourself, don’t forget your IAM policies for access to all the resources you’ll need!