Module Design Conventions? or Maybe Just a Conversation :)

System Design
1

@pinzonjulian ‘s post, [Practical advice after Shopify’s Packerk Retrospective]( Practical advice after Shopify’s Packerk Retrospective ) got me thinking–especially after I saw Eileen Uchitelle’s RailsWorld 2024 talk, The Myth of the Modular Monolith, and this post from Brandon Weaver.

I couldn’t get away from the thought that we seem to lack a basic consensus on what modularity is, what problem modularity is trying to solve, and what are sound practices for implementing modules.

I ended up writing a series of articles on this (this one is probably the most relevant). Not because I’m an expert or authority (I’m not), but because I wanted to sort out my thoughts–and because any conversation needs a starting point. :face_with_monocle:

In the article, I quoted this passage from 99 Bottles of OOP on object-oriented design because I think it is equally applicable to modularity:

Where you once optimized code for understandability, you now focus on its changeability. Your code is less concrete but more abstract—you’ve made it initially harder to understand in hopes that it will ultimately be easier to maintain. This is the basic promise of Object-Oriented Design (OOD): that if you’re willing to accept increases in the complexity of your code along some dimensions, you’ll be rewarded with decreases in complexity along others. OOD doesn’t claim to be free; it merely asserts that its benefits outweigh its costs.

(emphasis added)

If modularity efforts require all stakeholders to buy in to the validity of the exercise, then there needs to be clarity around the theory, the practice, and the goal of modularity. How else can they measure the benefits against cost?

I also strongly feel that the creation of conventions to guide implementation would be hugely helpful. I propose four to start (at least one of which is sure to be controversial). Again, I don’t claim to be “right,” I just want to start a conversation: if not these conventions, then what?

  1. Packs/modules are only for domain logic. No controllers, views, or other UI objects should be allowed (just use namespaces to organize UI objects, if needed).
  2. Only coupling between packs/modules should be scrutinized and regulated for purposes of dependency graphs. Coupling with the UI layer is expected behavior (although this doesn’t mean abandoning interfaces).
  3. Only data objects should be passed to and from packs/modules.
  4. Packs/modules must own their own data (persistence).

I argue that autonomy should be the overarching goal of modularity, and that these conventions promote autonomy. I would love to hear your thoughts. :nerd_face:

2

Thanks for your articles, Damian! I am especially grateful for the diligent use of references… I feel too often folks, while standing on the shoulders of giants, obscure their thinking by not doing this one simple trick :wink: Very much appreciated!

There is so much in those articles, that I will instead just ask one question about what you posted here. Regarding your first convention: Based on your discussion, I would understand if you said “don’t commingle controllers+vies and domain logic.“ but you are in fact saying “Packs/modules are only for domain logic“ (my emphasis). I am not sure where you make the argument that would allow me to arrive at that proposed restriction in usage. Did I miss it?

3

Hi, @stephan! Thanks for engaging! I hoped that posting here would result in more conversation. I am most interested in exploring these ideas and finding good usage patterns.

I know all three articles are a lot, but the third one (the one I linked to above) is really the most important one. I mainly wrote the other two just to get my thoughts in order–a surprisingly difficult task sometimes. :face_with_monocle:

This section of that article, What’s In, What’s Out, addresses this topic directly. As I understand things, the main goal of Packwerk (and modularity in general) is to reduce interconnections between different areas of responsibility in an app (to use general language). Another goal, a corollary of sorts, is to avoid circular dependencies between these different components of the system.

My argument is that the controllers and views are the UI of the app. They represent accidental complexity–accidental to the fact that the app is a web app. As such, they should be separated from the domain logic, the essential complexity of our application. I think this view fits with the currently accepted convention of “fat models, skinny controllers,” but I recognize that the separation of UI and domain logic is not usually this clean in most Rails apps: domain logic often bleeds into controllers and even views.

But, if we are able to maintain this separation, then the need to modularize controllers and views evaporates in my view, since

  • There is no need to restrict or manage the “connections” between the UI and the domain logic: this is the intended functioning of the system, and
  • There is no danger of circular dependencies if the domain logic never calls controllers or views. Why would it?

To draw an analogy, in the few Hanami apps I’ve seen that utilize Slices (analogous to Packwerk Packs), they usually contain a full array of actions, views, and domain logic. If the goal is to limit interconnections between slices (or packs), this strikes me as counterproductive: actions and views are magnets for domain logic interaction. In this implementation, they are drivers of connections between slices.

As an aside, it also strikes me that the scheme used to define these slices, an “admin” slice for example, do not align well with the natural seams in the domain logic, leading to more interconnections between slices. It feels to me like the slice composition scheme is being driven more by the logical categorization of controllers and views than by the natural seams in the domain logic. I think this practice loses sight of the original goal of creating slice/packs.

This begs an important question: what is the value of including controllers and views in packs? What benefit is derived? I recognize that there can be a need to manage and categorize controllers and views (like “admin”), but I argue that namespaces are entirely sufficient for this.

Keeping the UI out of the domain logic prevents UI organizational needs from contaminating our analysis of the domain and, most importantly, lets us focus more of our attention on the domain–where it belongs.

4

I understand and agree with the idea of separating UI and domain! (Although: I also have examples where I want “full vertical stacks”…) What I am questioning is whether the idea should lead us to not put the UI into packages at all. You could put all of the UI into one package and you would get some protection as to whether you indeed don’t have circular dependencies. If you don’t to be overwhelmed by having to state all those dependencies on the domain packs, don’t enforce dependencies (but do enforce privacy!).

As soon as you do this, you likely realize you want not one UI package but multiple because they are actually doing different things. And then you’ll find there is some structure to the UI stuff as well: e.g., some shared parts that multiple UI packages depend on.

5

I don’t think (hope?) that is the currently accepted convention :smiley:

6

Oh, also - I did end up creating my own packwerk retrospective… I gave a talk about it at SF Ruby last week. I will post the video once its out!

2 Likes
7

I keep checking youtube and the SF Ruby conf website for this but still nothing yet :sob: I really want to hear the talk as well as the CISCO one on their decoupling effort.

8

Found it: https://www.youtube.com/watch?v=py_vjWTmAwg

9

@stephan Great talk! :star_struck: Thank you!

10

@stephan Thank you for responding! I can’t believe it’s taken me so long to circle back! :scream:

Can you give some examples of this? I mean, I get that there will be partials and all. Is that what you mean? Have you found value in enforcing rules around this with Packwerk?

I guess this is where I’m missing something. Privacy to what end? As you said in your talk, everything has a cost. Is placing rails controllers and views in packs worth the cost? Whatever the benefits are of doing so, could the same be achieved without packs and the cost of packs?

To be clear, these are all genuine questions. I’m here to learn. I have been experimenting with Hanami and I’ve had a similar conversation to this one with the core members there (I am a Hanami maintainer as well, as a result of work I did on the hanami-router gem).

With Hanami slices as with packs, I don’t see the value of including UI as being worth the cost. It seems to me that the cost of placing UI components inside packs or slices is greater than that for other components (whatever you call them, models, service objects, domain logic, etc.), with less benefit.