Skip to content

Commit

Permalink
docs: update tools summary
Browse files Browse the repository at this point in the history
  • Loading branch information
spwoodcock committed Jul 12, 2024
1 parent e60dd80 commit af0cc95
Show file tree
Hide file tree
Showing 5 changed files with 59 additions and 52 deletions.
6 changes: 3 additions & 3 deletions docs/modules/backend.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ website](https://hotosm.github.io/osm-rawdata)).

### [PostgresClient()](https://hotosm.github.io/osm-rawdata/api/#postgrespy)

This program extracts data from a local postgres data, or the remote
Underpass one. A boundary polygon is used to define the area to be
This program extracts data from a local postgres data, or via a
remote database. A boundary polygon is used to define the area to be
covered in the extract.

### [MapImporter()](https://hotosm.github.io/osm-rawdata/api/#importerpy)

Imports data into a postgres database that is using the Underpass
Imports data into a postgres database that is using the raw-data-api
schema. This currently supports three input formats, Geojson, all OSM
formats, and Parquet files from Overture.

Expand Down
22 changes: 10 additions & 12 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,18 @@

The HOT Tech Team is working to improve the user experience across
multiple tools, and working towards an end to end data flow between
projects. Since often a disaster response, or humanitarian mapping
campaign, a project manager or mapper may use the [Tasking
Manager](https://hotosm.github.io/tasking-manager/), [Export Tool](https://hotosm.github.io/osm-export-tool/),
[fAIr](https://fair-dev.hotosm.org/),
[FMTM](https://hotosm.github.io/fmtm/), etc... For more information,
here's a short [E2E Presentation](techdoc/e2e.pdf).
projects.

During disaster response, or a humanitarian mapping campaign, a project
manager or mapper may require:

To improve maintainability of the code base, the Text Team has been
working on modularizing all of our projects. Rather than duplicating
functionality used by multiple projects, that functionality has been
migrated to standalone python modules so they can be shared. There is
more detailed information on modularization [here](modules.md).
- High quality base imagery.
- Digitized map features, derived from this imagery.
- Field-verified information about these features.
- Data exported in a useful format for further analysis.

The goal of our proposed [E2E Flow](./e2e-overview.md) is to fill in these
gaps to aid an effective response.

## Block Model Diagram

Expand All @@ -23,4 +22,3 @@ more detailed information on modularization [here](modules.md).
## Overall Architecture

![detailed-e2e](./images/hot-components-model.png)

Binary file removed docs/techdoc/e2e.pdf
Binary file not shown.
75 changes: 43 additions & 32 deletions docs/tools-summary.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ raster or vector-based basemaps) for usage.**
### Input Data

- Raster Basemaps:
- Google, Bing, ESRI, Mapbox et al all provide web basemaps we can use.
- Google, Bing, ESRI, Mapbox all provide web basemaps we can use.
- Open Aerial Map (OAM) provides more bespoke basemaps of particular areas
of interest (AOIs), over a certain time period. Often higher resolution.
- Vector Data:
Expand Down Expand Up @@ -73,15 +73,13 @@ With the above as context, HOT's tools roughly can be categorised as such:

### Input

| Tool | Description |
| --------------- | -------------------------------------------- |
| OAM | Get base imagery. |
| Raw Data API | Extract data from OSM easily _for software_. |
| Underpass (OSM) | Assure data quality. |
| Tool | Description |
| ------------ | -------------------------------------------- |
| Drone TM | Get base imagery. |
| OAM | Stored and accessible base imagery. |
| Raw Data API | Extract data from OSM easily _for software_. |

> Note: input may be into our own tools, or workflows of others.
>
> Each tool is fully open to use by the public.
### Output

Expand All @@ -92,40 +90,51 @@ With the above as context, HOT's tools roughly can be categorised as such:
| FMTM | Add extra information to digitised features in the field. |
| Export Tool | Extract data from OSM easily _for humans_. |

### Drone TM

Drone TM is the newest tool being developed by HOT.

It's purpose is to fill the gap of reliable, high-quality, base imagery
that can be used during the digitisation process.

Given an AOI, an area can be subdivided into task areas and flight
plans generated for each users specific drone.

In a **collaborative** manner a large amount of high quality base
imagery can be generated very quickly and with minimal cost.

### OpenAerialMap (OAM)

OAM should underpin all of our tools.

Integrations into other tools allow for basemaps to be loaded to better
inform mapping.
It is essentially a repository of base imagery, which is ideally
high-resolution for a specifically targetted mapping area.

The thing with satellite imagery is that you have lots of different
resolutions available, through different providers.
Imagery can be uploaded from Drone TM, or any other openly available
data source, by the community.

You need to use a mix of providers and decide on the best quality
usable for mapping.
The imagery is made available to other tools via the Tile Map Service
protocol (TMS).

Many providers (ESRI, Bing, etc) provide free base maps we can use,
but OAM should be more targeted - the highest resolution imagery we can
get over a time period of interest, for a specific AOI that may be used
in TM or FMTM.
### Raw Data API

### Raw Data API & Underpass
Uses an innovative database structure to make OSM data much more easily
searchable with excellent performance.

They use an innovative database structure to make OSM data much more usable.
The idea to make OSM data more accessible, searchable / filterable, and
create extracts in formats that can be consumed either by software or users.
This database is made accessible behind an API, where users (or software)
can request OSM data within specific filter criteria.

Underpass is a quality control tool that will sit on top of Raw Data API.
Data can be exported in various formats, from GeoJSON to GeoPackage and Flatgeobuf.

### Export Tool

The purpose of Export Tool is to:

- Take the users input for what data they need, over what area.
- Calls Raw Data API to extract and filter the data.
- Receives back the data in the user requested format, e.g. a JSON
for use in another tool, or a geopackage for use in GIS software by a user.
- Receives back the data in the user requested format.

> This is essentially the more user-centric frontend for raw-data-api.
### Tasking Manager

Expand All @@ -139,8 +148,11 @@ The mapping is crowdsourced, or organised by NGOs.

Resulting data needs to be validated.

Another tool, MapSwipe, can help with this & may be part of the solution.
However, TM allows mapping to be done **collaboratively**.
> Another tool, MapSwipe, also fills this niche but takes a much more minimalistic
> approach.
>
> It allows users without any prior OSM or mapping experience to contribute more
> effectively, from a mobile device or web browser.
### fAIr

Expand All @@ -160,8 +172,7 @@ Again, the mapping can be crowdsourced, but more likely organised by NGOs / govs
The FMTM mapping could also provide a feedback loop to TM, helping to
validate that features were mapped correctly.

As with TM, another tool, StreeComplete, can help with this & may be
part of the solution.

However, FMTM allows mapping to be done **collaboratively**.

> As with TM, another tool, StreeComplete, can help with this & may be
> part of the solution.
>
> However, FMTM allows mapping to be done **collaboratively**.
8 changes: 3 additions & 5 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ nav:
- User Roadmap: https://docs.fmtm.dev/user-roadmap/
- Field Pilots:
- Monrovia: projects/FMTM/Monrovia E2E Request.pdf
- Drone TM:
- Repo: https://github.com/hotosm/drone-tm
- fAIr:
- Overview: techdoc/overview/fAIrProductPitch.pdf
- Docs: https://fair-dev.hotosm.org/
Expand All @@ -74,16 +76,12 @@ nav:
- Docs: https://hotosm.github.io/raw-data-api/
- Roadmap: https://github.com/orgs/hotosm/projects/29
- Task Tracker: https://github.com/orgs/hotosm/projects/29/views/2
- Underpass:
- Docs: https://hotosm.github.io/underpass/
- Roadmap: https://github.com/orgs/hotosm/projects/23/views/3
- Task Tracker: https://github.com/orgs/hotosm/projects/23
- Open Aerial Map:
- Overview: techdoc/overview/OAMProductPitch.pdf
- Browser: https://hotosm.github.io/oam-browser/
- API: https://hotosm.github.io/oam-api/
- Export Tool:
- Docs: https://hotosm.github.io/osm-export-tool/
- Repo: https://github.com/hotosm/osm-export-tool
- Roadmap: https://github.com/orgs/hotosm/projects/29
- Task Tracker: https://github.com/orgs/hotosm/projects/29/views/2
- Roadmap: roadmap.md
Expand Down

0 comments on commit af0cc95

Please sign in to comment.