Kyle/fsspec integration #60
Conversation
* Streaming list results * Collect remaining object metas * Fused list stream * Also fuse BytesStream * fix tests * Improve list docs
* Return list result as arrow * Check if arro3-core is installed * Add tests * fix ci? * update set up python * downgrade requires python in test env * Update list docstring examples
…lopmentseed#39) * Return buffer protocol object from get_range and get_ranges * Fix docs
* Support range in GetOptions * Add test for get with options
* Rename package to object-store-py * fix ci * more renames
* Fix typing and docs website * Update versions
* Rename to obstore * more renames
* Add custom exceptions * Add exceptions to docs
* Add put options * Add put if not exists test
* remove `range` from `GetOptions` for now * skip test
…entseed/object-store-rs into kyle/fsspec-integration
I think that usually means that it's falling back to instance authentication? Maybe we should default that to off for simpler errors |
How to do that? |
|
I think there'll be a bit of a learning curve for us to figure out the right set of auth configuration and ensure it's documented correctly. So far, whenever I've hit auth errors, I've been able to re-check my credentials and get it working |
* Add __repr__ to store classes * Add __repr__ to pyi files * containment
|
#62 has an example of working tests against |
|
|
||
| def __init__( | ||
| self, | ||
| store: ObjectStore, | ||
| store, |
There was a problem hiding this comment.
Why remove the ObjectStore typing here?
There was a problem hiding this comment.
It wasn't importing, and this is only typing. Perhaps something with my environment.
There was a problem hiding this comment.
Yeah it's a type-only object. Perhaps it should be changed in the future to be an actual Python object that can be imported at runtime. The from __future__ import annotations should allow this to not be evaluated.
* wip use moto for tests * edit * Add basic tests with moto * remove print * anonymous s3 connection * lint
|
Perfect, I'll build on that. "Unsigned" seems wrong, but oh well. |
|
Er, something has gone very wrong with my git here. I'll get back to it tomorrow with any luck. |
|
The branch you're trying to merge into is also behind. It's fine if you switch the target branch to |
|
I'll close this as I believe the only PR you intend to work on is #63, but reopen if I'm wrong. |
Running this test fails:
even though that URI is gettable with requests, returning valid XML. Thoughts?