Using Packwerk to Modularize a Large Monolith - Seeking Feedback and Suggestions

Message originally sent by slack user U7832IJ3CGA

But if we didn’t have to do this…it’d be a much easier sell. :wink:

Message originally sent by slack user U7832IJ3CGA

Imagine adopting this in Shopify in 2016 before the componentization work happened. Might have raised some concerns.

Message originally sent by slack user U783MOJYF8Z

I am trying to imagine it right now :smile:

I mean, we can also see how it is adopted at shopify in applications that don’t have a component folder structure yet.

Usually these applications don’t have your “team annotations” in files, and our core monolith didn’t either.

So the first step is to decide which package a file should be in. And then we want to reflect that in the code somehow. People have already been using folders in some form for that (sometimes with, sometimes without a separate namespace) before we had packwerk.

So I guess the applications that we’ve looked at are mostly different from yours in that file level annotations never existed.

Message originally sent by slack user U783MOJYF8Z

Do namespaces and team annotations not align at all in your code?

Message originally sent by slack user U7832IJ3CGA

So I guess the applications that we’ve looked at are mostly different from yours in that file level annotations never existed.
Yeah, we went that route instead of moving files around.

Do namespaces and team annotations not align at all in your code?
No not really. We’ve really kept a traditional Rails folder structure, probably far far too long.

We have recently started a components folder and made some small extractions thanks to the leadership from @stephan. However, 98% of code is in app.

Message originally sent by slack user U783MOJYF8Z

For some added context - not all applications at shopify have taken the approach to put everything into components that are determined upfront. Some have started with just a few empty components that responsibilities were transferred into step by step.

But in general, our components model business domains, and their relationship to teams is secondary.

Message originally sent by slack user U783MOJYF8Z

I don’t want to sound ignorant, but I really don’t understand the friction you’d encounter with moving files. If it’s just an experiment, maybe you don’t have to merge to master?

If it’s more than an experiment, wouldn’t it be super confusing to think about how parts of the application relate to each other if those parts are invisible?

Message originally sent by slack user U7832IJ3CGA

But in general, our components model business domains, and their relationship to teams is secondary.

I really believe this is the way to go. The hardest part is making the transition from The Rails Way to packages and components. I’m looking for any way to help the Rails community find a repeatable, iterative way through the transition.

Message originally sent by slack user U783MOJYF8Z

yeah me too :slightly_smiling_face: I’m also not saying packwerk is perfect. It’s one thing that has worked for us on multiple large apps

Message originally sent by slack user U7832IJ3CGA

If it’s just an experiment, maybe you don’t have to merge to master?
When I say experiment, what I mean to say is, for monoliths who are making their very first efforts to draw boundaries, they are likely to get it wrong. We won’t know if we drew the boundaries correctly at commit time. We just need to try something (the experiment) and over time, reflect and review.

wouldn’t it be super confusing to think about how parts of the application relate to each other if those parts are invisible?
Of everything we’ve said so far here, this has me reconsidering the value of “virtual packages”. I really do want a forcing function to make these conversations happen and moving all the files would certainly put this paradigm shift front-and-center.

The “virtual packages” idea seems like a baby-step to make it easier to make this change feel smaller and therefore make it easier to get buy-in from the hundreds of developers who need to get work done every day.

Message originally sent by slack user U78EKGG059M

Over at Root Insurance we did move all the files into what we proposed as their future home. We also made it clear this was a rough first pass and as we make the boundary hard we should consider if we got the home right in the first place and not be afraid to move a file - or even split it if it does too much and should belong in multiple homes

Message originally sent by slack user U78EKGG059M

Over time so far we have mostly been right - but this journey is far from complete - we have also been wrong a few times and moved files - its pretty easy to do in Ruby/Rails

Message originally sent by slack user U783MOJYF8Z

Yep - from my experience it is best to make it really obvious which package a file belongs to, because that highlights the cases where it’s wrong. You put it way more succinctly than I could have Tim.

Folders make it obvious when browsing the repository, which is mostly what people do when they are implementing new code, or looking up the implementation of a method they’re calling.

Packwerk can make it even more obvious and also help trigger these conversations for legacy code.

Message originally sent by slack user U783MOJYF8Z

The most painful part is where we discovered that the boundary has to go right through one file, and if that’s and active record, maybe even through the table

Message originally sent by slack user U783MOJYF8Z

Most of those cases took months or years to discover.

I can only imagine!

Message originally sent by slack user U783MOJYF8Z

Meaning we’re still discovering some :smile:

Message originally sent by slack user U78EKGG059M

because that highlights the cases where it's wrong - so much this Philip