Introducing Rockpool

I've spent roughly a year completely rewriting the code running Aus GLAM Blogs, and it's finally ready for use. The old codebase was built with MeteorJS. This was a really useful framework when I was first learning JavaScript, because it allowed me to build a usable webapp with logins and routing, without really being that good at coding. As my JavaScript and general coding knowledge improved, Meteor became increasingly frustrating. I was spending more time fiddling with versions and dependencies than I was writing code. Integration with the npm module world was imperfect. I also realised there were some larger structural problems with the app. So I foolishly decided to build a new version from scratch.

After fiddling with a few options, including writing it in Python instead of JavaScript, I ended up using express, with the existing MongoDB backend and - a key component - passwordless for user logins. I'm calling the new software Rockpool - because it allows people to form a small conversational community without having to constantly swim in the bigger ocean of information. It's a crappy metaphor but I had to call it something.

What I learned

You might be wondering how on earth it took me (more than) a whole year to rebuild something that is really not that complicated. The answer is partially that I was doing this in my spare time, partially that it got more complex as I completed each part and became more ambitious, and partially that I used this as a learning opportunity for a number of new things I hadn't done before. I spent quite a lot of time thinking about how to implement new features I really wanted: several would require blog owners to sign in to the app, but I didn't want to be responsible for keeping passwords secure. I initially considered passport, but then I found Florian Heinemann's passwordless which sends a one-use login link every time a user wants to log in. Given users may only ever log in one or two times, this was a perfect solution. Unfortunately, Florian hasn't updated the code for a while, and there was a problem with one of the dependencies. So I had to fork the code and make a minor adjustment, and then publish that fork as a new npm module.

I got some experience publishing my own scoped npm module from scratch when I realised that obtaining RSS feed information from a website URL and vice-versa was something I might want to do in other coding projects. That module - feedfinder - also gave me a small project to use for finally learning unit testing with mocha, which I went on to use in a much bigger way in Rockpool. I've written before about using mocha, and how whilst it sometimes felt like it was slowing me down, it almost certainly made the project go faster with better code in the long run.

I played around with some roll-my-own AJAX, learned the basics of React, and then rejected it as far too fiddly and decided to learn and use VueJS instead. It turned out that using mocha to do unit testing really helped me when structuring Vue components, (even though I didn't actually test them with mocha), because it directed me to a kind of API-driven structure for the app's routes and function triggers.

I learned a bit about MongoDB indexes (though it's quite likely I haven't got this right), and finally got my head around Docker Compose. I've been using Docker Compose to run my Mastodon server for a while, but putting Rockpool into Docker containers helped me to understand what's actually going on. In some ways, Rockpool was just a vehicle for learning a bunch of web technologies and frameworks.

What it means for Aus GLAM Blogs

That's a lot of tech jargon for people uninterested in software development! Here's what it means if you follow or have registered your blog with Aus GLAM Blogs:

Improved performance

The old app connected to a remotely hosted database, and I hadn't set up any indexes. There were also problems with the way my database queries were set up. As a result, it was really slow - especially on first loading a page. The new app is much faster.

Update your own blog details

The new software allows blog owners to log in and update the Twitter and Mastodon accounts associated with their blog(s), change the category, or delete the blog from the app entirely (though not old posts). If you already have a blog on Aus GLAM Blogs you'll need to 'claim' your blog. If you ever change the title of you blog, you can use the update feature without actually changing your blog's category, and it will grab the new title.

Integrates with Mastodon

Aus GLAM Blogs will now post to Mastodon as well as to Twitter. Mastodon integration was actually the main thing I wanted to add, originally, but I got a bit carried away. If you want to add your Mastodon account to your blog listing (so you are mentioned on Mastodon when you publish a new blog post) you need to log in to glamblogs.newcardigan.org, add you mastodon details, and claim your blog.

Hashtags and content warnings

If your blog is tagged glam blog club then when the bot tweets or toots it will now use the #glamblogclub hashtag. The Mastodon bot will also use content warnings for key words I think might benefit from it.

Improved Search

You can now search by keyword or phrase across tags, titles and authors, or browse by tag. I've also added some filters for 'this month' and 'last month'¹.

Tags are also lightly normalised, as I described last year.

Better options for Pocket

For some time readers have been able to subscribe to the main feed to have posts added directly to their Pocket list. Unfortunately I neglected to create a way to unsubscribe. I've now added that and also an option to exclude particular blogs from the feed if there are some that you find irrelevant or uninteresting. I also removed the function that adds all the tags in the original blog post as tags in Pocket, because it turned out to be really annoying. You can always add Pocket tags manually if you want.

If you have a Pocket account registered with Aus GLAM Blogs, you will already have an account to log in the new app, using the same email address you use for Pocket. If your Pocket username is not an email address, there was no way to migrate your account over to the new app, but I think that only affects me.

Improved admin features

You won't notice this directly (probably), but the new software gives administrators more control over what's going on. I can now suspend a blog if, say, someone gets temporarily hacked, or begins posting things not in line with 'community standards'. This is a gentler option that simply removing the blog completely. It's also now much easier to delete a blog if it goes offline or the feed starts failing for some other reason.

Running your own Rockpool instance

The last reason why this took a bit longer than I anticipated is that I wanted other people to be able to use the code in a meaningful way. It's one thing to put your code up on GitHub. It's quite another to build it in a way that other people can actually understand how to install and use it. I had to think about documentation, and what needed to be adjustable settings rather than hard-coded values. I've probably got some things wrong, but I hope that it's reasonably straightforward to set up your own instance of Rockpool for your own community, should you wish to do so. I'd be really interested to talk to anyone who wants to give it a go.

The whole thing runs in two Docker containers using Docker Compose, so the good news is that you shouldn't have to worry much about getting you environment set up properly or maintaining dependencies. I anticipate that the most difficult thing will be getting past Twitter's Bot Police to get your Twitter API keys.

If you want to try it for your own community, check out the installation instructions on GitHub with the code. I'm more than happy to provide any help or advice.

What now?

Aus GLAM Blogs is now running on Rockpool. You can find out more from the help page, or the Rockpool docs. If you have a blog on Aus GLAM Blogs it would be great if you could log in and claim it, and check out the app.

If you have any feedback I'm keen to hear it. If you spot a bug, please report it. A good thing about having rewritten everything is that it is now much easier for me to make updates and run tests, so if they're needed, updates and bug fixes can be done relatively quickly. Special thanks to Alissa who helped me with some great testing, and even submitted a pull request to fix some bugs.

Having said that, I'm hoping to take a bit of a rest from Rockpool for a while. I've got a shelf full of books to read, and a couple of new features to add to ephemetoot.