This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.
Message originally sent by slack user U70K2MT6OHQ
Hey all! Wanted to say this community has been such a great source of knowledge for me in my modularization journey and truly appreciate your drive and expertise!
I’m a previous Rails developer and advocate of packwerk now in Laravel/PHP world trying to lead modularization concepts in these tools. Reading through the packwerk retrospective I was interested in learning more about this quote →
… some components are domains, while others are designed around the functional role they play in the application. A checkout flow is a function defined as the code required for a customer to initiate a checkout and pay for their order.
One of the things I enjoy about Laravel is the event bus is a first class feature. Do you think some of these function components could be broken down more accurately across domains if an event bus was available by default in Rails?
I disagree with the gist of that quote from the blog post - components/checkouts was explicitly designed to encompass the domain of checkout, not just the process of a buyer checking out. That’s why checkout configuration is in there too, which the blog post authors bring up as a drawback. I believe that it depends on your perspective and the goals you’re trying to achieve whether “by domain” or “by function” is the better split for you.
yes, eventing would have been helpful in many cases in Shopify’s journey, and I believe it’d be useful in most similar efforts. However, we couldn’t get enough buy-in for any specific eventing solution for use within the monolith during my time there. I ended up pushing for and working on a project for a domain-level event bus that could be used between and within services, with (some) transactional guarantees based on the outbox pattern. I left before that was finished, but I believe that it’s still being worked on.
On another note though - eventing is very loose coupling and definitely can be overused.
Plain method calls are easier to understand, debug and predict. It‘s a tool that is good to have in the toolbox but it doesn’t solve all problems well.
Yep I totally agree, and I understand this is a nuanced topic . Curious about what push back you got in terms of buy-in Philip. Was it difficult to introduce novel concepts because of the code/team size?
Some aspect was just size - hard to change the trajectory of a large container ship, and many people with opinions.
The main disagreement was between on the one hand people who wanted to keep the traceability and optional transactionality of method calls, which would have been at most amenable to in-process synchronous eventing, and on the other side people saying that synchronous eventing doesn’t decouple runtime aspects like performance, with subscribers able to slow publishers down or interrupt them with exceptions.
We had a pretty solid proposal for a synchronous event bus that would catch exceptions and would allow publishers to define and enforce subscriber execution time budgets, but in the end it didn’t fly.
To clarify - there were various mechanisms, mostly ad-hoc but some more elaborate, already in place for eventing between services (Shopify had hundreds of services in addition to the giant monolith during my time there).
The disagreements were about how to do eventing within the monolith.
What are your thoughts on Rails Event Store, if you have any <@U70TIGAX94P>? I believe it has the transactional outbox pattern and can support in-process and out of process events, sync and async. Although the async option seems to just be jobs?
TBH I haven’t looked too closely into it. IIRC, when we were looking into eventing, RES was relatively new and marketed as a solution for event sourcing which we were convinced we didn’t want to do (too much of an architectural departure from vanilla Rails; which means too many potential incompatibilities and too much work for the benefits we would have gained). I think event sourcing would have blown up the size of our architectural transition by an order of magnitude at least.
There is also the fact that at Shopify scale, we were unlikely to be able to adopt any relatively-new library that hadn’t been proven at similar scale. When we looked into the outbox pattern, there was a lot of pushback from our inhouse database team, and for good reason. Events with outbox pattern would have increased database load by something like 30-50% according to our projections, and we discussed lots of possible solutions some of which would have required major contributions to the MySQL implementation. We knew we wouldn’t just be able to drop in a library like RES.
(One of the solutions we discussed was to use a table backed by MySQL’s BLACKHOLE engine, meaning a table that isn’t persisted. We’d stream the binlog into Kafka via debezium.)
Looking at RES now, I’m also thinking that it does a lot of things. Since the introduction of domain-level, and monolith-internal, events was a big change, and we’d put the new pattern / tooling into many hundreds of developers hands at the same time, I would have preferred to have a more restricted, simplified approach. Otherwise had we found out that the pattern for some reason doesn’t work out for us it’d be even more difficult to revert. We also had a lot of problems with libraries making Rails upgrades difficult etc so generally were quite conservative regarding adoption of new third party dependencies.
If I were to do this again I would definitely take a closer look at RES but I believe some of the reasons we didn’t adopt it are still applicable.
Synchronous eventing can be replaced by callbacks in many cases.
Now that’s a can of worms… Should they, though? One of the point of events is decoupling, which callbacks don’t help with. (I am assuming you are referring to ActiveRecord callbacks here.)
. Curious about what push back you got in terms of buy-in Philip. Was it difficult to introduce novel concepts because of the code/team size?
really appreciate your thoughts and insight.
Should they, though? One of the point of events is decoupling, which callbacks don’t help with. (I am assuming you are referring to ActiveRecord callbacks here.)