Testing stubs

[MarkMurphy]

Would love to see this library include support for stubbed testing. The aws gem does this and it's pretty rad. The existing third party gems like rspec-stripe and stripe-ruby-mock are ok but I'm not overly fond of their approach to it. Doesn't seem as elegant as it could be and they also tend to fall behind in terms of keeping up with the updates made to this library which makes it hard to update either library.

[moneytree-doug]

👍

[bparanj]

I would love to see an in-memory implementation of the stripe-ruby gem that is fast to use in test environment.

[j15e]

FYI we are working on an update of stripe-ruby-mock that will work with the latest API. But indeed a solution provided by Stripe itself would be great, it would be much easier for them to be in sync with their own API.

[bparanj]

I totally agree with you. Ideally the vendor should provide the implementation that can be used in test environments.

[brandur]

Hi everyone, thanks a lot for logging this and the follow-up discussion. Language bindings are certainly a feature we want to provide, but we have a few other dependencies that we need to get out before a sustainable version of them will be available. The biggest issue is that the bindings can't currently create an accurate representation of the responses return by our various API endpoints (this information currently lives in another disconnected project).

I'm going to close this issue and move its tracking to one of our internal boards, but we have our eye on it.

@j15e Awesome work on stripe-ruby-mock! We'll make sure to reference this if/when we attempt an official implementation. Hopefully we can produce a mostly compatible API or even share/re-use some of the implementation.

[brandur]

Actually, we're going to re-open this issue to ease tracking. Tagging as "speculative" "future" for now. Apologies for the churn!

[MarkMurphy]

Hey guys, it's been just over a year since I brought this up. Just wondering if there's any updates or movement on this ie. has there been further discussions, planning or code started?

[brandur]

Hi @MarkMurphy, thanks for keeping us accountable here!

So the answer is not completely satisfying: yes, there's been movement on the front, but we're still not particularly close to an implementation of enough quality that it's ready for public consumption. You can see in #347 that I added a basic machine readable spec and started moving some tests over to it, but ran into some trouble internal to Stripe in that it was difficult to create a spec that was complete (specifically our "polymorphic" endpoints like "sources" were difficult to spec out).

That problem has been solved, but we still aren't generating a spec that's fully accurate. About a month back I did some work to generate an OpenAPI spec (a.k.a. Swagger). I got about 60% of the way there, but didn't follow through on it yet.

The reason that I'm talking about specs a lot is that I think that this sort of machine readable description will be a dependency to having up-to-date and accurate testing stubs that we can ship out with the gem. I expect the order of operations to look a little like:

1. Get generated spec into the gem and start using it internally (a little like in Use generated API responses in stripe-ruby tests #347).
2. Expose a public interface for its use by all users.

Apologies for the slow progress here. I'm still hoping that we can get this in, but it's still largely being done in peoples' spare time right now, and so a completion date is difficult to estimate.

[MarkMurphy]

@brandur appreciate the update! It's satisfying enough (for me anyway) just to know that it's somewhat of a work in progress.

[brandur]

@MarkMurphy Okay, thanks! Will definitely keep you appraised is anything more substantial comes about.

[Systho]

Could you maybe add a section in the readme about the third-party gems (rspec-stripe and stripe-ruby-mock) until you provide an official stub ?

[brandur]

I'd consider a PR on that one, but I don't personally have enough direct experience with 3rd party stubs to be able to recommend any in particular.

[exocode]

Has someone a solution for this?
Stripe could get quite complex. And without writing tests I feel unsafe. It would be a neck-breaker to charge customers with unexpected code behaviour, because it's not tested.

• How do you deal with that?
• How do you test?
• Are you running everything with stubs?
• Are you using the outdated / poorly maintained gems?
  https://github.com/stripe-ruby-mock/ has potential, but also a ton of PR requests which are not implemented, since years.

I feel totally uncomfortable at the moment.

Please give me some suggestions, thank you

[jacobjlevine]

Can stripe-mock be used for our test suites? Or is that just intended for this gem's test suite? If it is possible to use it for our own purposes, is there good guidance anywhere for how to go about doing that?