We found a few things, like e.g. isolated tests, isolated boot, etc which can take you in a stepwise way toward an isolation goal, provided you know what that is.
I think I have a good idea what you mean by “isolated boot” and “isolated tests”, but do you have any links to more discussion/references/examples of this? Specifically in the context of Rails apps?
We work with engines that are in-repo gems. So isolated boot can literally mean just comment out everything in your Gemfile that your chosen subset of engines should not depend on and boot. It’ll probably crash hard and crash fast and there’s your work for you. Mostly moving stuff in places like application.rb into engine initializers etc.
Isolated tests are slightly trickier but it’s the same idea: with your subset loaded, run tests. Probably nothing will work at first and you’ll spend a lot of time just to get a single test to run let alone even pass. But it gets easier.
We basically carved out a framework. It’s an interesting learning experience.
The idea of using tests (and selective loading) for isolation in Rails apps is not new.
Yes absolutely agree, the idea is not new. And I hope I haven’t given the impression that it was our idea (although I personally hadn’t read anything about it before we started).
But as they say, 1% inspiration, 99% perspiration. People may have thought of it before at Shopify but nobody had actually tried it. We now have it as a part of the CI pipelines that are run on every PR.
For folks interested in digging in to prior approaches, there is a bunch of older and newer material listed on https://cbra.info/resources, which I still try to keep up to date
People may have thought of it before at Shopify but nobody had actually tried it.
That’s a matter of definition IMO. In-repo gems still exist, don’t they? They were part of an effort to investigate isolating code completely, and I always saw them as an alternative for code that was quite isolated already in terms of implementation and / or semantics.
Yes that’s fair, and extracting a gem is the analogy I like best for what we did. However extracting something at the base of a giant legacy system is quite different from extracting an API wrapper, for example. I’ve done both, the former was a quite different experience.
The only other thing I’d say in response to the original question asked is don’t focus too much on tooling (including Packwerk). Just do the work, and you’ll learn as you go. Yes, for isolated testing you need a pipeline that runs tests in isolation. We also have a pipeline to boot in production to weed out other issues. These are important to avoid regressions.
But other than stuff like that, the focus should really be on the code and what it’s telling you. You learn as you work. I learned way more in a few months heads down doing the extraction than I had in years reading, listening to discussions, etc.
If you want a metric, measure how fast your tests run. They should be way faster in isolation. That’s a genuine indication that you’re doing something right.
That makes a lot of sense, however I don’t think that’s the only useful metric. It depends on your goals. So I would express this differently :
Don’t focus too much on tooling (including Packwerk). Determine your goals first, derive a way to get there, and then use tooling to implement it.
There’s tons of value to be unlocked by turning accidental into intentional architecture that are not related to running tests quickly.
(Additionally I think it can be dangerous to “listen to the code” if that’s your only or main source of information. The code may be really bad, and what it’s telling you may not be helpful to achieve your goals)
