|
1 | 1 | { |
2 | 2 | "cells": [ |
3 | 3 | { |
| 4 | + "attachments": {}, |
4 | 5 | "cell_type": "markdown", |
5 | 6 | "id": "a0eb29a0", |
6 | 7 | "metadata": {}, |
7 | 8 | "source": [ |
8 | 9 | "## eoAPI Demo\n", |
9 | 10 | "\n", |
10 | 11 | "\n", |
11 | | - "This notebook will try to go over the different [eoAPI](https://github.com/developmentseed/eoAPI) services using the latest Open data from Maxar acquired during the Hawaii fired of August 2023\n", |
| 12 | + "This notebook will review the different [eoAPI](https://github.com/developmentseed/eoAPI) services using the latest Open data from Maxar acquired during the Hawaii fires of August 2023\n", |
12 | 13 | "\n", |
13 | 14 | "\n", |
14 | 15 | "### Maxar Open Data\n", |
15 | 16 | "\n", |
16 | | - "> Pre and post event high-resolution satellite imagery in support of emergency planning, risk assessment, monitoring of staging areas and emergency response, damage assessment, and recovery. These images are generated using the Maxar ARD pipeline, tiled on an organized grid in analysis-ready cloud-optimized formats.\n", |
| 17 | + "> Pre and post-event high-resolution satellite imagery in support of emergency planning, risk assessment, monitoring of staging areas and emergency response, damage assessment, and recovery. These images are generated using the Maxar ARD pipeline, tiled on an organized grid in analysis-ready cloud-optimized formats.\n", |
17 | 18 | "\n", |
18 | 19 | "\n", |
19 | 20 | "\n", |
20 | 21 | "### STAC and COGs\n", |
21 | 22 | "\n", |
22 | | - "Maxar releases open data for select sudden onset major crisis events. In addition of putting the data (as nice COGs) freely available on AWS they are also adding STAC (static) metadata alongside the images. Having the STAC items already created makes ingestion into PgSTAC database really easy (we don't have to create the items ourself and thus having to read the images).\n", |
| 23 | + "Maxar releases open data for select sudden-onset major crisis events. In addition to putting the data (as nice COGs) freely available on AWS, they are also adding STAC (static) metadata alongside the images. Having the STAC items already created makes ingestion into the PgSTAC database easy (we don't have to produce the items ourselves and thus have to read the images).\n", |
23 | 24 | "\n", |
24 | | - "\n", |
25 | | - "To learn more about how we ingest MAXAR OpenData STAC catalog into pgstac see https://github.com/vincentsarago/MAXAR_opendata_to_pgstac\n" |
| 25 | + "To learn more about how we ingest Maxar OpenData STAC catalog into PgSTAC see https://github.com/vincentsarago/MAXAR_opendata_to_pgstac.\n" |
26 | 26 | ] |
27 | 27 | }, |
28 | 28 | { |
|
61 | 61 | ] |
62 | 62 | }, |
63 | 63 | { |
| 64 | + "attachments": {}, |
64 | 65 | "cell_type": "markdown", |
65 | 66 | "id": "5237534c", |
66 | 67 | "metadata": {}, |
67 | 68 | "source": [ |
68 | 69 | "## Collections\n", |
69 | 70 | "\n", |
70 | 71 | "\n", |
71 | | - "In this section we will:\n", |
72 | | - "- list all the collection stored in the pgSTAC database (backend of the eoapi STAC service\n", |
73 | | - "- talk about the collection metadata (and extensions)\n", |
74 | | - "- see how to visualized the collections on a Map using collection's Spatial Extent " |
| 72 | + "In this section, we will:\n", |
| 73 | + "\n", |
| 74 | + "- List all the collections stored in the pgSTAC database (backend of the eoapi STAC service\n", |
| 75 | + "- Talk about the collection metadata (and extensions)\n", |
| 76 | + "- See how to visualize the collections on a Map using the collection's Spatial Extent " |
75 | 77 | ] |
76 | 78 | }, |
77 | 79 | { |
|
119 | 121 | ] |
120 | 122 | }, |
121 | 123 | { |
| 124 | + "attachments": {}, |
122 | 125 | "cell_type": "markdown", |
123 | 126 | "id": "51e448c1", |
124 | 127 | "metadata": {}, |
125 | 128 | "source": [ |
126 | | - "Each collection should be in form of \n", |
| 129 | + "Each collection should be in the form:\n", |
127 | 130 | "```json\n", |
128 | 131 | "{\n", |
129 | 132 | " \"id\": ...,\n", |
|
157 | 160 | "}\n", |
158 | 161 | "```\n", |
159 | 162 | "\n", |
160 | | - "**Important**: in eoapi, we are trying to set an example of good practice which is why we choose to add `optional` extension in our collections. The `item_assets` extension is quite useful because it quickly tell what assets are found within all the collection's items. This can help creating visualization for the whole collections without knowing much about its items." |
| 163 | + "**Important**: in eoapi, we are trying to set an example of good practice, so we choose to add `optional` extension in our collections. The `item_assets` extension is helpful because it quickly tells what assets are found within all the collection's items. This can help create visualizations for the whole collection without knowing much about its items." |
161 | 164 | ] |
162 | 165 | }, |
163 | 166 | { |
|
414 | 417 | ] |
415 | 418 | }, |
416 | 419 | { |
| 420 | + "attachments": {}, |
417 | 421 | "cell_type": "markdown", |
418 | 422 | "id": "b2a46c84", |
419 | 423 | "metadata": {}, |
420 | 424 | "source": [ |
421 | 425 | "### Vizualizing the collections \n", |
422 | 426 | "\n", |
423 | | - "Each collections should have at least one spatial extent (combined) so let's use those to create a GeoJSON and add it to the map" |
| 427 | + "Each collection should have at least one spatial extent (combined), so let's use those to create a GeoJSON and add it to the map:" |
424 | 428 | ] |
425 | 429 | }, |
426 | 430 | { |
|
661 | 665 | "source": [ |
662 | 666 | "# As mentioned, the collection's spatial extent can be a combination of multiple bbox\n", |
663 | 667 | "\n", |
664 | | - "# Here we use the `/collections/{collectionId}` endpoing\n", |
| 668 | + "# Here we use the `/collections/{collectionId}` endpoint\n", |
665 | 669 | "\n", |
666 | 670 | "collection_info = httpx.get(\"https://stac.eoapi.dev/collections/MAXAR_Maui_Hawaii_fires_Aug_23\").json()\n", |
667 | 671 | "\n", |
|
699 | 703 | ] |
700 | 704 | }, |
701 | 705 | { |
| 706 | + "attachments": {}, |
702 | 707 | "cell_type": "markdown", |
703 | 708 | "id": "5fd7a387", |
704 | 709 | "metadata": {}, |
705 | 710 | "source": [ |
706 | 711 | "#### Temporal Extent\n", |
707 | 712 | "\n", |
708 | | - "Each collection can have spatial and temporal extent. Same as for the spatial extent, a collection can have multiple temporal extent but with it's first one representing the combined `min/max` of all the intervals." |
| 713 | + "Each collection can have spatial and temporal extents. As for the spatial extent, a collection can have multiple temporal extents but with its first one representing the combined min/max of all the intervals." |
709 | 714 | ] |
710 | 715 | }, |
711 | 716 | { |
|
732 | 737 | "metadata": {}, |
733 | 738 | "source": [ |
734 | 739 | "## Items\n", |
735 | | - "Let's checks the items for the Hawaii Fire collections: `MAXAR_Maui_Hawaii_fires_Aug_23`\n", |
| 740 | + "Let's check the items for the Hawaii Fire collections: `MAXAR_Maui_Hawaii_fires_Aug_23`\n", |
736 | 741 | "\n", |
| 742 | + "In this section, we will:\n", |
737 | 743 | "\n", |
738 | | - "In this section we will:\n", |
739 | | - "- list all items for a specific collection\n", |
740 | | - "- talk about the `limit` parameter \n", |
741 | | - "- visualize all items on a map\n", |
742 | | - "- talk about the item metadata" |
| 744 | + "- List all items for a specific collection\n", |
| 745 | + "- Talk about the `limit` parameter\n", |
| 746 | + "- Visualize all items on a map\n", |
| 747 | + "- Talk about the item metadata" |
743 | 748 | ] |
744 | 749 | }, |
745 | 750 | { |
|
1061 | 1066 | "id": "ad0efb74", |
1062 | 1067 | "metadata": {}, |
1063 | 1068 | "source": [ |
1064 | | - "As you can see bellow, the `/items` endpoints returned only 10 items. To return more data we need to either use the `pagging` mechanism or use the `limit` query parameter to get more items per requests " |
| 1069 | + "As you can see below, the `/items` endpoints returned only 10 items. To return more data we need to either use the `paging` mechanism or use the `limit` query parameter to get more items per request." |
1065 | 1070 | ] |
1066 | 1071 | }, |
1067 | 1072 | { |
|
1294 | 1299 | ] |
1295 | 1300 | }, |
1296 | 1301 | { |
| 1302 | + "attachments": {}, |
1297 | 1303 | "cell_type": "markdown", |
1298 | 1304 | "id": "844f4575", |
1299 | 1305 | "metadata": {}, |
1300 | 1306 | "source": [ |
1301 | 1307 | "#### Find acquisition times\n", |
1302 | 1308 | "\n", |
1303 | | - "By definition, every item should have either a `datetime` or a `start/end_datetime` property. For the Maxar dataset, we are assuming that `datetime` are acquisition times." |
| 1309 | + "By definition, every item should have either a `datetime` or a `start/end_datetime` property. For the Maxar dataset, we are assuming that `datetime` is acquisition times." |
1304 | 1310 | ] |
1305 | 1311 | }, |
1306 | 1312 | { |
|
1356 | 1362 | ] |
1357 | 1363 | }, |
1358 | 1364 | { |
| 1365 | + "attachments": {}, |
1359 | 1366 | "cell_type": "markdown", |
1360 | 1367 | "id": "2693e87a", |
1361 | 1368 | "metadata": {}, |
1362 | 1369 | "source": [ |
1363 | | - "Note: The `catalog_id` represent a unique acquisition Id. In our case, this mean that all items are derived from 5 different acquisitions. \n", |
| 1370 | + "Note: The `catalog_id` represents a unique acquisition ID. In our case, this means that all items are derived from 5 different acquisitions. \n", |
1364 | 1371 | "\n", |
1365 | 1372 | "> The unique identifier for the item, as assigned by the provider, Maxar. This is also referred to as the \"acquisition ID\".\n", |
1366 | 1373 | "\n", |
1367 | 1374 | "ref: https://ard.maxar.com/docs/select-and-order/select-stac-collection-file/#properties-object\n", |
1368 | 1375 | "\n", |
1369 | 1376 | "\n", |
1370 | | - "Let's look back at the item's datetime. If we remove the `seconds`" |
| 1377 | + "Let's look back at the item's datetime. If we remove the `seconds`:" |
1371 | 1378 | ] |
1372 | 1379 | }, |
1373 | 1380 | { |
|
1390 | 1397 | ] |
1391 | 1398 | }, |
1392 | 1399 | { |
| 1400 | + "attachments": {}, |
1393 | 1401 | "cell_type": "markdown", |
1394 | 1402 | "id": "7de68313", |
1395 | 1403 | "metadata": {}, |
1396 | 1404 | "source": [ |
1397 | | - "We get 5 different dates, which might correspond to 5 different aquisition (Keep this in mind for later)" |
| 1405 | + "We get 5 different dates, which might correspond to 5 different acquisitions (keep this in mind for later)." |
1398 | 1406 | ] |
1399 | 1407 | }, |
1400 | 1408 | { |
| 1409 | + "attachments": {}, |
1401 | 1410 | "cell_type": "markdown", |
1402 | 1411 | "id": "4ca2e374", |
1403 | 1412 | "metadata": {}, |
1404 | 1413 | "source": [ |
1405 | 1414 | "## Assets \n", |
1406 | 1415 | "\n", |
1407 | | - "So we have **123** items for the `MAXAR_Maui_Hawaii_fires_Aug_23` collection, and each item have **4** assets (this is also found at collection level in the `item_assets` extension." |
| 1416 | + "So we have **123** items for the `MAXAR_Maui_Hawaii_fires_Aug_23` collection, and each item has **4** assets (this is also found at the collection level in the `item_assets extension`)." |
1408 | 1417 | ] |
1409 | 1418 | }, |
1410 | 1419 | { |
|
1484 | 1493 | ] |
1485 | 1494 | }, |
1486 | 1495 | { |
| 1496 | + "attachments": {}, |
1487 | 1497 | "cell_type": "markdown", |
1488 | 1498 | "id": "7bc1a304", |
1489 | 1499 | "metadata": {}, |
1490 | 1500 | "source": [ |
1491 | 1501 | "# Raster\n", |
1492 | 1502 | "\n", |
1493 | | - "In eoAPI we have a raster API which is connected to the PgSTAC database. The service is built using [titiler-pgstac](http://github.com/stac-utils/titiler-pgstac) and can be used to visualize `Item` or `Mosaics` (multiple items).\n", |
| 1503 | + "In eoAPI, we have a raster API connected to the PgSTAC database. The service is built using [titiler-pgstac](http://github.com/stac-utils/titiler-pgstac) and can be used to visualize `Item` or `Mosaics` (multiple items).\n", |
1494 | 1504 | "\n", |
1495 | 1505 | "\n", |
1496 | 1506 | "Endpoint: https://raster.eoapi.dev" |
|
1625 | 1635 | ] |
1626 | 1636 | }, |
1627 | 1637 | { |
| 1638 | + "attachments": {}, |
1628 | 1639 | "cell_type": "markdown", |
1629 | 1640 | "id": "3a87b045", |
1630 | 1641 | "metadata": {}, |
1631 | 1642 | "source": [ |
1632 | | - "The `/collections/{collectionId}/items/{itemId}/info` endpoint returned metadata for 3 assets (the raster ones). We now know a bit more about each assets (datatype, zoom levels, number of bands) which can help us for creating tiles urls.\n", |
| 1643 | + "The `/collections/{collectionId}/items/{itemId}/info` endpoint returned metadata for 3 assets (the raster ones). We now know a bit more about each asset (datatype, zoom levels, number of bands), which can help us create tiles urls.\n", |
1633 | 1644 | "\n", |
1634 | 1645 | "\n", |
1635 | 1646 | "##### Asset on Map\n", |
1636 | 1647 | "\n", |
1637 | | - "To visualize an asset on a Map we need to construct a `Tile URL`. To ease the task we can use the raster's service `/tilejson.json` endpoint, but here are the requirement:\n", |
| 1648 | + "To visualize an asset on a Map, we need to construct a `Tile URL`. To ease the task we can use the raster's service `/tilejson.json` endpoint, but here are the requirements:\n", |
1638 | 1649 | "\n", |
1639 | 1650 | "- HAVE TO pass `assets` or `expression` parameter\n", |
1640 | | - "- CAN pass `min/max zooms` (which will avoid under/overzooming)\n", |
| 1651 | + "- CAN pass `min/max zooms` (which will avoid under/over-zooming)\n", |
1641 | 1652 | "- CAN pass `rescale` parameter if datatype is not compatible with PNG/JPEG output format\n", |
1642 | | - "- CAN pass `asset_bidx` parameter to select band combination\n" |
| 1653 | + "- CAN pass `asset_bidx` parameter to select band combination" |
1643 | 1654 | ] |
1644 | 1655 | }, |
1645 | 1656 | { |
|
1934 | 1945 | ] |
1935 | 1946 | }, |
1936 | 1947 | { |
| 1948 | + "attachments": {}, |
1937 | 1949 | "cell_type": "markdown", |
1938 | 1950 | "id": "b6ad993a", |
1939 | 1951 | "metadata": {}, |
1940 | 1952 | "source": [ |
1941 | | - "The `ms_analytic` asset has more than 3 bands and is also of `Uint16` data type, so to be able to visualize it we will need to:\n", |
| 1953 | + "The `ms_analytic` asset has more than 3 bands and is also of `Uint16` data type, so to be able to visualize it, we will need to:\n", |
1942 | 1954 | "- select 1 or 3 bands\n", |
1943 | | - "- apply some linear rescaling between min/max value (we can an approximation using the `/statistics` endpoint)" |
| 1955 | + "- apply some linear rescaling between min/max values (we can make an approximation using the `/statistics` endpoint)" |
1944 | 1956 | ] |
1945 | 1957 | }, |
1946 | 1958 | { |
|
2366 | 2378 | ] |
2367 | 2379 | }, |
2368 | 2380 | { |
| 2381 | + "attachments": {}, |
2369 | 2382 | "cell_type": "markdown", |
2370 | 2383 | "id": "3a02b6ae", |
2371 | 2384 | "metadata": {}, |
2372 | 2385 | "source": [ |
2373 | 2386 | "## Mosaics\n", |
2374 | 2387 | "\n", |
2375 | | - "As mentioned and shown, with `titiler-pgstac` we can visualize item's asset but the real power of the raster api is to be able to create virtual mosaic dynamically and merging multiple items together on the fly\n", |
| 2388 | + "As mentioned and shown with `titiler-pgstac`, we can visualize an item's asset, but the real power of the raster API is to be able to create a virtual mosaic dynamically and merge multiple items on the fly.\n", |
2376 | 2389 | "\n", |
2377 | | - "learn more: http://github.com/stac-utils/titiler-pgstac\n", |
| 2390 | + "Learn more: http://github.com/stac-utils/titiler-pgstac.\n", |
2378 | 2391 | "\n", |
2379 | | - "Let's create `mosaics` for a specific datetime we found in previous step.\n", |
| 2392 | + "Let's create `mosaics` for a specific DateTime we found in the previous step.\n", |
2380 | 2393 | "\n", |
2381 | 2394 | "> ['2023-08-09T21:10', '2023-08-09T21:20', '2023-08-12T21:11', '2023-08-12T21:12', '2023-08-12T21:18']" |
2382 | 2395 | ] |
|
2431 | 2444 | ] |
2432 | 2445 | }, |
2433 | 2446 | { |
| 2447 | + "attachments": {}, |
2434 | 2448 | "cell_type": "markdown", |
2435 | 2449 | "id": "6dd5f539", |
2436 | 2450 | "metadata": {}, |
|
2439 | 2453 | "\n", |
2440 | 2454 | "```json\n", |
2441 | 2455 | "{\n", |
2442 | | - " // PgSTAC accept multiple language for filter, here we will just cql2-json\n", |
| 2456 | + " // PgSTAC accepts multiple languages for filtering; here we will use cql2-json\n", |
2443 | 2457 | " \"filter-lang\": 'cql2-json',\n", |
2444 | | - " // We tell PgSTAC to `register` a `search` request with the following filter\n", |
| 2458 | + " // We tell PgSTAC to `register` a `search` request with the following filter:\n", |
2445 | 2459 | " \"filter\": {\n", |
2446 | 2460 | " \"op\": 'and', \n", |
2447 | 2461 | " \"args\": [\n", |
|
2451 | 2465 | " \"args\": [{\"property\": \"collection\"}, [collection_id]]\n", |
2452 | 2466 | " },\n", |
2453 | 2467 | " {\n", |
2454 | | - " // Item's datetime property HAS to intersect with the start/enddate\n", |
| 2468 | + " // Item's datetime property HAS to intersect with the start/end date\n", |
2455 | 2469 | " \"op\": \"t_intersects\", \n", |
2456 | 2470 | " \"args\": [\n", |
2457 | 2471 | " {\"property\": \"datetime\"}, [startdate, enddate]\n", |
2458 | 2472 | " ]\n", |
2459 | 2473 | " },\n", |
2460 | 2474 | " // titiler-pgstac accept some additional metadata\n", |
2461 | 2475 | " // https://stac-utils.github.io/titiler-pgstac/advanced/metadata/\n", |
2462 | | - " // One is really useful: `bounds`. When creating a mosaic, the tiler will have no idea \n", |
2463 | | - " // where the items will be prior trying to create each tile. To avoid trying to request \n", |
| 2476 | + " // One is useful: `bounds`. When creating a mosaic, the tiler will have no idea \n", |
| 2477 | + " // where the items will be before trying to create each tile. To avoid trying to request \n", |
2464 | 2478 | " // tiles where we know we don't have any items, we can add the collection's extent to the \n", |
2465 | 2479 | " // mosaic metadata. The tiler service will then return the bounds in the tilejson document for the \n", |
2466 | 2480 | " // client application.\n", |
|
2475 | 2489 | "\n", |
2476 | 2490 | "API Response:\n", |
2477 | 2491 | "\n", |
2478 | | - "The raster service will return a `searchid` hash (also called `mosaic id`), wich we can use to construct a tile URL." |
| 2492 | + "The raster service will return a `searchid` hash (also called `mosaic id`), which we can use to construct a tile URL." |
2479 | 2493 | ] |
2480 | 2494 | }, |
2481 | 2495 | { |
|
2601 | 2615 | "tiles.add_to(m)\n", |
2602 | 2616 | "m" |
2603 | 2617 | ] |
2604 | | - }, |
2605 | | - { |
2606 | | - "cell_type": "code", |
2607 | | - "execution_count": null, |
2608 | | - "id": "31d3649e", |
2609 | | - "metadata": {}, |
2610 | | - "outputs": [], |
2611 | | - "source": [] |
2612 | 2618 | } |
2613 | 2619 | ], |
2614 | 2620 | "metadata": { |
|
0 commit comments