Over-engineered Side Projects have Utility

I remember my first corporate job. The recruiter, hired just two months prior, hadn’t gotten a single candidate into the pipeline. In my interview, he kept asking: “Why didn’t you write about the things you clearly know?” I had trimmed my resume (advice from some blog) and lacked experience with big companies. He helped me rewrite it.

For that job, I’d studied everything: inverted trees, Big O, dynamic programming, sorting algorithms. My interviewers couldn’t stop me from whiteboarding. I was hired!

Six months later, I was debugging form validations… again!

We’re sold a dream. The hacker mindset, the thrill of technology, the magic of code coming to life. Many of us fell for programming through video games: Imagine making a character do something cool! Reality? You’ll debate the max length of an email field.

Most software jobs orbit around CRUD: Input a field into the database, then output it on the screen. Working on scalability means tweaking the database indexes. Innovation means choosing between SaaS tools to glue into the stack.

The skills that you practiced for, the ones that you spent countless hours reading and applying, they’ll gather dust. But do not despair.

If you want to build something complex, to learn something new, to challenge yourself, then you are gonna need a Side Project. Not just any side project. An over-engineered side project. This will be your playground for learning what work won't teach you.

This isn't about productivity, it's about intentional, glorious overkill.

• A weather app with a homemade ML model that photographs the sky to predict the weather.
• A to-do list with Web+Assembly + Blockchain. (Why? Why Not?)
• A blog on a custom PHP framework with adaptive caching, auto asset inlining, and a fancy CLI interface... wait, that's this blog.

Most developers go with something simple for blogging. Wordpress, Ghost, or Hugo. Mine started as hand typed static HTML files. But before the week was over, it transformed into a distributed system in disguise.

The framework nobody asked for

Routing? A solved problem. But did I know how? I built it from scratch: hundreds of lines of regex, deprecated PHP functions, and inefficiency. It worked.

Dynamic meta tags? I built it WordPress-style. As the blog was evolving, I found myself wanting to add additional meta tags on each post, but not the same list everywhere. It did cause my system to crash at some point, but how else was I supposed to learn?

Caching? When you are serving static files, performance is not really an issue. But when you add all these database calls, dynamic fields and what not, you need a caching strategy. I learned this the hard way when one of my articles reached the #1 spot on Hackernews. My server melted while I was at work. I downloaded a terminal on my android phone and connected via SSH (when it let me), just so I could reboot the server. I'd get a few more requests in before the server would crash again.

My solution? Adaptive caching. When a page gets 10 requests per seconds, the database call to fetch articles is cached for a minute. When it reaches 50 requests per second, the comments module is cached as well. When requests reach 100 per seconds, the resulting HTML page is cached. But this conflicted with my view and reading counter. So I added async calls to continue inserting to the db even after the page has been rendered.

$post = $db->get("BlogPosts", cache: true, threshold: 50)->getActiveArticleBySlug($slug);
...
$router->get("/blog/:slug", controller: "BlogPost", cache: true, threshold: 100);

I also created a CLI tool to manage migration and code generation. Whenever I updated a field in the database, I ran a command to generate the properties in all the necessary classes, and flagged the places where required fields were missing.

In the same vein, the CLI tool also handled the deployment pipeline. No more scp deployment. Now my framework could handle deploying itself and provisioning the server.

Oh, and I've reimplemented some of my favorite jQuery functions right into my own js library. Along the way, I roll up my own CDN.

Was any of this necessary?

No. But did I learn more than any job could teach me? Absolutely. A lot of the random things I implemented in my own blog became the basis to solving problems for my employer. For example, badly implementing a router, brought up some issues that I later found at my employer as well. That async logging system was a godsend for my employer. My company's AB testing tool was something that was originally intended to run on my blog. If it wasn't for the folks at VWO, I would be a billionaire by now...

By building overly complex stuff, I learned useful skills that I wouldn't have learned any other way. The corporate world rewards efficiency. Side projects reward curiosity. You don't have to apologize for over-engineering your own tools. You are not wasting time, you are compounding knowledge. And someday, that "useless" project will be the reason you actually understand the problem at work.

---