Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for satstac.ItemCollection #29

Merged
merged 7 commits into from
Dec 6, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Top-level functions

open_stac_catalog
open_stac_collection
open_stac_item_collection
open_stac_item

.. currentmodule:: intake_stac
Expand All @@ -26,5 +27,6 @@ Catalog Objects

StacCatalog
StacCollection
StacItemCollection
StacItem
catalog.StacEntry
6 changes: 3 additions & 3 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ Intake-stac is an open source project and Python package for discovering,
exploring, and loading spatio-temporal datasets.

Intake-stac provides Intake Drivers for SpatioTemporal Asset Catalogs (STAC).
It provides tools for opening STAC ``Catalogs``, ``Collections``, and ``Items``
as Intake catalogs. Intake and Intake-xarray provide the tooling for loading
assets described in STAC into Xarray objects.
It provides tools for opening STAC ``Catalogs``, ``Collections``,
``ItemCollections``, and ``Items`` as Intake catalogs. Intake and Intake-xarray
provide the tooling for loading assets described in STAC into Xarray objects.

.. toctree::
:maxdepth: 2
Expand Down
33 changes: 30 additions & 3 deletions docs/source/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
Tutorial
========

Intake-stac provides is pretty simple. It provides a thin interface that
combines `sat-stac` and Intake. It's basic usage is shown below:
Intake-stac simply provides a thin interface that combines `sat-stac` and
Intake. It's basic usage is shown below:

To begin, import intake:

Expand Down Expand Up @@ -85,6 +85,33 @@ Once you have identified a dataset, you can load it into a ``xarray.DataArray``
using Intake's `to_dask()` method:

.. ipython:: python

:okwarning:
da = entry.to_dask()
display(da)

Combining with `sat-search`
---------------------------

Intake-stac integrates with `sat-search` to faciliate dynamic search and
discovery of assets through a STAC-API. To begin, construct a search query
using `sat-search`:

.. ipython:: python

import satsearch

results = satsearch.Search.search(
collection='landsat-8-l1',
bbox=[43.16, -11.32, 43.54, -11.96],
sort=['<datetime'], #earliest scene first
property=["landsat:tier=T1"])
items = results.items()
display(items)

In the code section above, `items` is a `satstac.ItemsCollection` object.
Intake-stac can turn this object into an Intake catalog:

.. ipython:: python

catalog = intake.open_stac_item_collection(items)
list(catalog)
4 changes: 2 additions & 2 deletions intake_stac/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
del get_versions

import intake # noqa: F401
from .catalog import StacCatalog, StacCollection, StacItem
from .catalog import StacCatalog, StacCollection, StacItem, StacItemCollection

__all__ = ["StacCatalog", "StacCollection", "StacItem"]
__all__ = ["StacCatalog", "StacCollection", "StacItem", "StacItemCollection"]
32 changes: 30 additions & 2 deletions intake_stac/catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,15 @@ def __init__(self, stac_obj, **kwargs):
else:
raise ValueError(
"Expected %s instance, got: %s"
% (type(self._stac_cls), type(stac_obj))
% (self._stac_cls, type(stac_obj))
)

metadata = self._get_metadata(**kwargs.pop("metadata", {}))

name = kwargs.pop("name", self._stac_obj.id)
try:
name = kwargs.pop("name", self._stac_obj.id)
except AttributeError:
name = str(type(self._stac_obj))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewhanson - here is another place where we were tripped up while using a ItemCollection. There is not an id attribute on the class object. Not sure if it is possible, but it would be nice if ItemCollection inherited from satstac.Thing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jhamman In sat-stac an ItemCollection isn't really a STAC ItemCollection, it's more of a Single File STAC as per the extension:
https://github.com/radiantearth/stac-spec/tree/dev/extensions/single-file-stac

The idea behind this was that a self-contained STAC catalog would be like a regular STAC catalog, except as you pointed out there's no id, nor are there any other catalog fields:
https://github.com/radiantearth/stac-spec/blob/dev/catalog-spec/catalog-spec.md#catalog-fields

I've posted an issue for STAC recommending that all the Catalog fields get added to the Single File STAC extension:
radiantearth/stac-spec#691


super().__init__(name=name, metadata=metadata, **kwargs)

Expand Down Expand Up @@ -125,6 +128,31 @@ def _get_metadata(self, **kwargs):
)


class StacItemCollection(AbstractStacCatalog):
"""
Intake Catalog represeting a STAC ItemCollection
"""

name = "stac_item_collection"
_stac_cls = satstac.ItemCollection

def _load(self):
"""
Load the STAC Item Collection.
"""
for item in self._stac_obj:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewhanson - I could use a sanity check on this section.

self._stac_obj is a satstac.ItemsCollection. Is this the best way to unpack this into individual items?

Alternatively, we can/should we be grouping these items by collection?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure grouping is necessary. The most common use case I think is a catalog with a single collection of data. Additionally, a user might want to group them according to any of the other metadata fields, so I think we can leave it to the user to decide how to sort/use them.

self._entries[item.id] = LocalCatalogEntry(
name=item.id,
description="",
driver=StacItem,
catalog=self,
args={"stac_obj": item},
)

def _get_metadata(self, **kwargs):
return kwargs


class StacCollection(AbstractStacCatalog):
"""
Intake Catalog represeting a STAC Collection
Expand Down
Loading