Introducing SiteImp
It has been quite the ride, but I pushed DNS on siteimp.com last night. Siteimp is designed for non-technical users, entrepreneurs and digital marketers who are concerned with their site’s performance. We schedule a full site audit, run it and generate a report that’s full of recommendations on how to improve their site’s performance.
Genesis story
Siteimp started life as part of my automated testing rig. The first component was a crawler named Trixie. Trixie is a very fast nodejs based crawler that goes through sites, gobbles up links, tests them and builds lists of pages along with http status codes. Whenever I deployed an application, I would run Trixie to make sure that all my links worked and all my pages returned the response code I expected.
From there, I started writing some really simple performance tests. Trixie would map out applications and send the map over to another set of performance testing scripts. They would download each page, perform a variety of tests and generate metrics.
Within a couple of weeks, my simple performance tests had made me a much better software developer. Getting a consistently generated set of performance metrics every deploy helped me zero in on performance, deliver more stable applications and provide much better user experiences. So I kept adding to my performance testing rig and Speed was born.
(Note - Trixie and Speed are named after Speed Racer characters.)
Then Google started talking about site performance and everything changed. Less technical people started talking about website performance. A lot of their businesses started to depend on crafting fast web experiences but they had one hell of a lot to learn.
That’s when I started to feel really sorry for small business owners and online entrepreneurs. Google told them they needed to learn dev ops but that’s a really intimidating learning path for less technical people. Ops is hard and the internet is complicated. My side of the industry has some amazing tools but they’re complicated. I felt like we were forcing these people down a learning path without starting them off with simple tools that they could digest.
Then I got an idea. What if I launched Trixie at their websites, ran a rig of performance tests and fed them metrics? They wouldn’t have to learn testing and could just focus on learning how to optimize stuff. I built the first version over a weekend and starting running reports for some brave friends.
I knew I had something right away. Running performance tests across an entire site from the same setup gave much better insight than running Lighthouse reports manually. It pointed a finger at the really big, system wide problems and really helped people prioritize what changes to make. I could add some of my knowledge of ops, browsers, content management systems and code, write some recommendations and we could seriously speed up people’s websites.
Siteimp was born.
The name
It didn’t take many reports for me to realize that some Wordpress plugins were a bigger problem than others. Without mentioning any names or even sectors, it became pretty clear that Wordpress users were fucked if they didn’t get a faster way to deal with contact forms. So, I knew I had to build a static form generator that wouldn’t do stupid things, like installing bootstrap (for a second time) across an entire fucking website just so they could have one contact form on one page.
That actually happens.
So, I knew I would work with forms and sites so I started looking for domains. It’s hard to find short .com domains so it came down to formdsd/sitedsd or formimp/siteimp. Siteimp was born.
The tech
Under the hood, Siteimp has two components and a database.
Trixie, Speed and friends are all nodejs and a lot of puppeteer. There’s an Express server in there, but that won’t make sense until the next iteration. (Hint - developers/agencies that employ developers, you’ll want it)
They feed all the data into postgres.
Then through the magic of python and django, all that data turns into stuff. The first iteration can output data in all sorts of ways. I’m using the second iteration for my own stuff, so I have a dashboard. Everything starts in json. It’s currently built to export reports into pdf or Excel formats, but I can output it in a lot of different ways. The sky is the limit and I’m really excited to help other developers work with the API.
Dev wise, I built it solo so push everything to main. I use release tagging to make regression testing easier. It uses my automated deployment rig which is a little absurd because it means I hit Siteimp with Siteimp when I deploy Siteimp.
The marketing site is Hugo, hosted on DigitalOcean’s app platform.
The future
First, I’m going to start writing a lot more about web performance. I have a series of articles ready to go about optimizing this site’s performance. That way, you’ll get a sense of how you could use Siteimp to speed up your own sites.
From there, I need to learn more. I need to talk to as many paying users as possible and collect as much feedback as I can. At some point, I have to accept that I’m really bad at design, give up and get help but I’m as stubborn as I am optimistic.
Final words
Building Siteimp was a ridiculous amount of fun and using Siteimp has made me a much better software developer. Please let me know if I can help.