This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.
Message originally sent by slack user U70TIGAX94P
I want to comment on the performance concerns brought up on the thread above.
Packwerk started out optimized for a super fast feedback loop. We focused on keeping the constant overhead low so that it would start instantly, scan a single file quickly from cold boot, specifically for editor integration.
The thing is that packwerk needs to know a few pretty detailed things about your application’s auto loader configuration and it was a little cumbersome to make that available.
Plus, at that point, almost nobody was using editor integration with packwerk. So we made packwerk boot the application. Pro: a lot less configuration, as packwerk can just get what it needs from the application. Con: startup got a lot slower. Spring alleviates that a bit. But we definitely deprioritized editor integration at that point.
I think it should be possible to reduce in-editor latency further by running a persistent process in the background, cutting startup time to zero. The analysis of a single file itself should still be super fast, as packwerk is not doing a lot of work on that.
For batch analysis, e.g. whole code base, it’s a different picture. Last time I checked most of the time was spent parsing Ruby code. I’d hope that with the work that Kevin Newton is doing on https://github.com/ruby/yarp, that problem will be solved soon.
One thing I was wondering was whether application boot is really necessary. At least for Gusto, we don’t use custom inflections or custom boot paths, so the default inflections and boot paths work for us. I took advantage of this in packs, since I used a vanilla inflector library and assume each package has its own rails-like boot paths.
Speeding up Ruby parsing will help a lot! I’m working on integrating a Rust extension to an experimental fork of packwerk to play around with a different, faster parser. We should be able to swap in yarp or any other parser easily if it’s faster and/or better supported
IIRC the application is booted to extract inflections and autoload paths.
Before we started booting the application the autoload paths were dumped into a file and checked by packwerk validate, which IMO worked reasonably well. Inflections were more complex since we couldn’t figure out how to reliably dump them, so we created a new source of truth, which never felt like a great solution.
BTW - are you sure you are not using custom inflections? Gems can introduce them. IIRC the graphql gem does.
Regarding custom inflections and such, I tried using a fork of packwerk that doesn’t boot the application and uses default inflections and when I pointed Gusto at it, deleted the cache, and ran bin/packwerk update, nothing changed.
If we switch to “dump what you need,” that would unblock the rust port by allowing it to consume those inflections more reliably and more exactly replicate the behavior.
That makes total sense. The rust port actually no longer needs to be unblocked! In Gusto’s monolith it’s now a drop-in replacement and we could always have a helper ruby gem which dumps whatever it needs if we wanted to enable more.
