Reflections on greenfielding MilkRun

When I joined MilkRun in June 2018, the technical situation was… not good. Seeing as I couldn’t even sign up due to errors, I had expected issues. But when I saw a timestamp primary key field which was also used as a foreign key, and other WTFs, I knew I was going to rewrite this thing from the ground up.

This was the first project I had truly greenfielded. I have built a lot of new services and repos but there was always other stuff it needed to hook into, and some set of expectation that new code should be similar to the old. This was all fresh, though! So I figured out a set of ideas/principles/strategies I’d develop with, based on what I think worked and didn’t work at previous jobs (and what I was technically competent at), and stuck to it for the next year or so.

I enjoy exercises like this because it’s also how I developed most of my management beliefs: researching, experimenting, iterating, then trying it out on a larger scale. I went into Cozy knowing exactly what I was going to try from a management/leadership/cultural perspective. And I went into MilkRun’s rewrite knowing just what I’d do from a technical level.

Since retrospective posts are fun and easy to write, I’m going to break down most of these topics into future posts. There are a handful of bigger “strategies” that I had:

• React Single Page App frontend. I really only knew KnockoutJS from Cozy, which I was not going to use again. Honestly I don’t much like SPAs in theory but I’d rather program them than dynamically rendered apps, so I went with it.
• Ruby API using Grape, Sequel, and Postgres. I was very happy with this at Cozy and saw no reason to change it.
• All complex state on the backend; move everything possible out of the frontend. Avoid complex patterns like Redux, and avoid the need for most frontend unit tests. Keep the backend extensively unit tested. We had thousands of frontend unit tests at Cozy because we had way, way too much state there.
• Frontend tested mostly through integration tests. Webdriver-based integration tests worked really well at Cozy and I wanted to continue it.

And finally, there were a set of pet peeves I had from various jobs that I was going to try to do differently and see what happened.

• Do not host anything myself (use AWS only for S3). Most everything was in Heroku, except frontends in Netlify.
• Use more off-the-shelf tooling. We wrote too many of our own things at Cozy, such as our “inline schema migrator,” and manually doing prefetch/eager loading. I used Sequel’s standard migrator, sequel-annotate, and Sequel’s tactical eager loading gem.
• Do not use datasets in models, to avoid multiple database round trips for the same data.
• No polymorphic associations.
• Rigorous database constraints instead of model validations.
• Sidekiq for async jobs. I really like RabbitMQ but it’s hard to find cheap hosting for it and the libraries and tooling aren’t so great. So I used Sidekiq and built some additional stuff to get some of the things I like from AMQP.

The order I write about them won’t have any rhyme or reason, it’ll just be when I want to write.