How to scale and segment different parts of an app using engines?

This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.

Message originally sent by slack user U7835KE8YO1

for the folks that have separated more into engines, are you able to spin up parts of your app as a scaling strategy (api engine needs more resources) or to segment parts in environments (admin/back office tool in a VPN)? i read a couple hints that this is technically possible (boot inquirer here) but was curious how feasible it ends up being?

Message originally sent by slack user U78E2Q4FDIQ

We don’t do this at Gusto. It’s been kicked around but there’s lots of things to figure out that are kind of a headache.

When you talk about this style of scaling strategy, I can only see one thing that this would help with: memory consumption. The CPU/IO-bound performance should be about the same. The memory consumption of just the engine needs to be less than the whole app for this to make sense.

If there’s a strong desire to explore this, I recommend seeing it as a stepping stone toward a separately running application. Here’s how this evolves from my perspective:

• Carve out a piece of the monolith into an engine.
• Solidify the boundaries between the engine<->monolith to only speak values.
• Separate the data store of the monolith/engine to be separate.
• Turn the communication at the boundary into a local network request, e.g. HTTP, grpc, etc. Pick your poison.
• Turn the local network request into a non-local network request by moving the application to a different set of machines.

Message originally sent by slack user U78E2Q4FDIQ

If any of the above steps are difficult, that might tell you to reconsider the extraction. Example: If it’s hard to have the interfaces at the boundaries only speak in values (i.e. not ActiveRecord objects), that might tell you that your code is more coupled than you thought.