This message was imported from the Ruby/Rails Modularity Slack server. Find more info in the import thread.
Message originally sent by slack user U712P4S1P5U
Dear All, I have a few questions about integration tests. If we have Rails API based application that is modularised with packwerk and a FE app that uses a framework like React and together they make up the web app, In your experience and understanding whats the best way to write integration tests for them? Any specific tools you recommend for this ? Can things like capybara still be a solution here somehow ?
It appears contract testing with the FE assumption that all contracts on the BE are always up to date is resulting in brittle specs overall and this is making it even more challenging looks like for us at work especially as the domain gets more complex and the need for integration tests is more important for at least critical flows. Also whats the best way to integrate this into the CI build for both BE & FE so that at both ends we could keep our systems in check and make them catch new changes such that we build a more robust app.
P.S: If I need to ask this question in <#C02TLU33RNW|packwerk> or in another channel, my apologies in advance, please let me know and I’ll be happy to post there.
@gpassero My apologies for not being able to respond earlier, by contract testing I was primarily referring to using something like Pactflow(one can see how it works at a high level from here)
P.S: I haven’t used it personally however I’m aware that our team tried doing this and I currently do not have enough information on how successfully it was used in our current workplace, however it appears that we don’t seem to be actively using it currently.
The thing that we don’t have a robust solution for integration tests currently makes me ask the original question in the post above on what are some best practices and recommended tools out there to test API backends in Rails against the FE apps that are mainly powered through JS frameworks like React for instance.
Doesn’t have much to do with packwerk, but it can run in CI and will give you the ability to test your entire application from the perspective of a user interacting with your software
Cypress is how we do E2E testing. Our E2E tests live in the frontend/React repo and we have the Rails app(s) running in Docker. Every time there’s a merge in a Rails repo, a new Docker image gets pushed to AWS ECR, and the E2E tests subsequently run against that via Cypress.
That said, I wish we had more/better contract tests. The E2E tests are very expensive and my unscientific sense is that they throw multiple false positives for every real failure that they catch.
Thank you very much Taylor and Mark for your suggestions and experiences shared with regard to Cypress. My sincere apologies for not being able to respond sooner
We currently also use Cypress and Mark, just like you mentioned:
Our E2E tests live in the frontend
We also experience a lot of flaky specs
These specs don’t seem to be that reliable as a result and this becomes all the more crucial a thing to address as our product grows and the related domain knowledge that grows as as a part of it.
We are also currently experimenting with Cypress and related tool called Cypress Play(I’m not sure about the specifics yet - I’m not currently working on it directly)
<@U71N3KA8TMX> From a process standpoint since you mentioned you do a lot of your testing in the FE repo, how exactly is the BE team members involved in writing or reviewing or overseeing the Cypress tests to have covered or having sufficient coverage from a logic stand point at least for all the new flows that get added from time to time?
Is it that these Cypress tests in your team are currently mainly owned by the FE team alone ?
That said, I wish we had more/better contract tests. The E2E tests are very expensive and my unscientific sense is that they throw multiple false positives for every real failure that they catch
I’m also curious, does this slow down the push to production or go live for every new bug fix or enhancement that goes live on the BE side with regard to your company?
At our company, we actually don’t run the Cypress tests for new BE related pushes to production. It doesn’t feel like an ideal approach, however it goes back to the same thing, we also find it expensive and we experience false positives(these are the reasons I’m aware of, there probably could be something else that I’m missing, I’m not closely working on it - just sharing my current understanding) so don’t see much value in it currently to run it as part of the CI build on the BE
We have a mechanism to mark flaky tests as such, track them, and get tickets created to fix them, so in general I don’t believe they end up blocking production deploys (ie if a known-flaky test fails, we don’t stop the deploy over it).
The E2E suite runs on every merge to development and in most cases we are deploying production daily or less, so any flaky tests are usually already known and accounted for before we get to production deploys :shrug: I have mixed to negative feelings about this as an overall process, because I think reduces some of the urgency to make flaky tests less flaky or question whether we really need them, but pragmatically it seems like it works OK.
if a known-flaky test fails, we don’t stop the deploy over it → Thank you Mark We have something similar at our end too to label them as such and have them as non blockers.
About this, thank you for letting me know your thoughts about this.
What are your thoughts about the below from here ?
From a process standpoint since you mentioned you do a lot of your testing in the FE repo, how exactly is the BE team members involved in writing or reviewing or overseeing the Cypress tests to have covered or having sufficient coverage from a logic stand point at least for all the new flows that get added from time to time?
Is it that these Cypress tests in your team are currently mainly owned by the FE team alone ?
Our engineering department is organized into “pods”, each of which has a mix of FE/BE/fullstack engineers, and pods are responsible for their own E2E tests. So theoretically, E2E test ownership is shared between FE and BE engineers. Anecdotally, I do think the FE and fullstack engineers end up spending more time writing and maintaining the E2E stuff compared with the more backend focused engineers, just because the toolchain involved is more familiar to them and the tests are defined in the FE codebase.
Thank you very much Mark for letting me know how this is done in your organisation I sincerely appreciate your taking out your valuable time to answer these questions for me, thank you
We have something similar at our end too to label them as such and have them as non blockers.