This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.
Message originally sent by slack user U71TN2WF04X
Hi, are uses of factory_bot in tests dependency violations? I know packwerk does not take this as a violation, but it does seem like one. Has anyone digged into this? Should I just ignore it since packwerk doesn’t take it as a violation?
We ignore the factorybot violations.
I agree it has significant drawbacks, but factorybot is so effective at setup for tests that we didn’t explore very long the idea of trying to do without it.
Message originally sent by slack user U71V6G2PDJS
Controversial opinion (maybe), this is my belief:
It should be an ambition to rid yourself of Factorybot. Even outside of a packs context it is detrimental to the health of your application. Factorybot is a bandaid on top of suboptimal design. And it leads you to a place where your database is one large connected graph of which it becomes harder and harder to work with just a part of that graph.
Factorybot enables the “big setup” antipattern for testing. It hides it under the covers, but it’s still there. So not only do you have to deal with a large amount of setup and slow tests as a consequence but Factorybot is also removing the feedback that your tests should be giving you.
The basic idea in packwerk is that each pack should stand alone. So not having ActiveRecord associations between packs is a good idea. And so we should be able to construct any object within a pack without needing to construct any object outside of the pack. This means that we don’t need Factorybot to setup data outside our pack.
Personally I prefer handrolling my data setup. And I prefer to do it using the applications API over the ActiveRecord API whenever possible. I think we as a community are overusing the ActiveRecord API in a lot of places where it doesn’t belong. I would much rather have User.signup in my tests than User.create.
I hope this is helpful - if nothing else than brining another perspective 
Our packs have dependencies… managed, acyclic dependencies, but dependencies none-the-less. Even though the production code within each pack is restricted to use the public API of another pack, that other pack has to be in a suitable state to enable the test flow for the target pack.
The only other way to do that setup is to essentially reproduce factorybot capabilities in API-targeted features, ie reimplement factorybot.
I’d be interested to see real-world examples of other systems that address this tension without factorybot.
#showme
Message originally sent by slack user U70VMMV37TJ
We’re currently focused on fixing our violations in our production code over testing code.
I agree with Jacob’s assessment of the issues with FactoryBot. And it can cause feature delay as some of the big test setup is so complex that it’s difficult to reason about.
Message originally sent by slack user U71V6G2PDJS
I don’t claim to have all (or any) of the answers, I’ve merely run into a lot of difficult to test code bases and from observation I believe Factorybot to have played a part in leading developers down the path that lead to this difficulty. But this is just beliefs and I don’t claim to have hard evidence on any of this. My best piece of “evidence” is that I am currently working with a codebase that doesn’t employ factorybot and I don’t miss it. We have a few hand rolled “factorybot implemetations” if you will.
The point I was trying to make and maybe not making very clearly is that I favour using the real production code ways of creating objects over the fake “let’s just create some data using ActiveRecord” ways of doing it. It’s not a problem that your code has dependencies. It is a problem if you have to create data in tables that belong to something outside of the pack you’re working with. In other words: If you need a customer to create a user and these are living in different packs it’s not a problem to instantiate a customer object through the customer packs exposed API to create a user object. It is a problem if you have to use factorybot to create the customer in the database to create a user object or test the logic in your user pack.
Perhaps we’re talking about testing at different levels. I’m coming at this from the unit testing angle. If talking about integration tests or higher level tests the picture changes a bit in terms of what data we should expect to be present. I would still argue that I would prefer having actual domain level code being able to produce the relevant state over low level data manipulation techniques though.
@gpassero Thank you for prompting me to clarify my thinking on this.
<@U71V6G2PDJS> I appreciate the sentiment with which you bring your thoughts. Yes, I’m speaking about integration tests (in this case, system tests).
My answer to <@U71TN2WF04X> original question is: yes, they are. And as discussed in this thread, they don’t tend to rise to the top of concerns to where they should be addressed first.
We have a couple of (large) packs where we’re working to separate the database connections on a per pack basis. In those cases, the question how to create test data becomes more pressing.
Note that if we’re talking about specific kinds of tests where this pattern is more useful than in others, then we can make gradual progress here:
I see many packs that could very well be gems. Many well-designed gems bring their own testing tools and expose them as part of their API (take typhoeus’ direct stubbing for example).
What if we were to challenge ourselves to only use the exposed test API for unit tests (cos is it really a unit test if we need to invoke other packs?). That should help limit what factories (or their alternatives) need to do. And just maybe those APIs are powerful enough to back the needed integration tests too?
tangent: Interestingly, without splitting app and test code into two packs, we currently don’t have tooling to say “test code can depend on a test API of a pack”.
Message excluded from import.