Skip to content

Fix STAC API URL handling and enhance notebook execution tracking #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
emmanuelmathot merged 2 commits into from
Dec 10, 2025
Merged

Fix STAC API URL handling and enhance notebook execution tracking #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4ba2e68
fix: update STAC API URL handling and improve notebook execution trac…
emmanuelmathot Dec 10, 2025
8614dde
fix: improve URL formatting for collection updates in STACCollectionM…
emmanuelmathot Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion operator-tools/manage_collections.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,11 @@ def create_or_update_collection(
click.echo("❌ Collection data missing 'id' field", err=True)
return False

url = f"{self.api_url}/collections"
url = (
f"{self.api_url}/collections/{collection_id}"
if update
else f"{self.api_url}/collections"
)
method = "PUT" if update else "POST"

try:
Expand Down
149 changes: 22 additions & 127 deletions operator-tools/submit_stac_items_notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -56,23 +56,15 @@
},
{
"cell_type": "code",
"execution_count": 32,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"✅ Configuration loaded\n"
]
}
],
"outputs": [],
"source": [
"# STAC API Configuration\n",
"STAC_API_URL = \"https://stac.core.eopf.eodc.eu/\"\n",
"\n",
"# Webhook Configuration\n",
"WEBHOOK_URL = \"http://localhost:12001/samples\"\n",
"WEBHOOK_URL = \"http://localhost:12000/samples\"\n",
"\n",
"print(\"✅ Configuration loaded\")"
]
Expand All @@ -86,30 +78,23 @@
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Area of Interest: [12.4, 41.8, 12.6, 42.0]\n",
"Time Range: 2025-07-01T00:00:00Z to 2025-07-31T23:59:59Z\n"
]
}
],
"outputs": [],
"source": [
"# Area of Interest (AOI) - Bounding box: [min_lon, min_lat, max_lon, max_lat]\n",
"# Example: Rome area\n",
"aoi_bbox = [12.4, 41.8, 12.6, 42.0]\n",
"# aoi_bbox = [12.4, 41.8, 12.6, 42.0]\n",
"# Example 2: Majorca area (2.1697998046875004%2C39.21097520599528%2C3.8177490234375004)\n",
"# aoi_bbox = [2.16, 39.21, 3.82, 39.78]\n",
"# Example 3: France Full\n",
"# aoi_bbox = [-5.14, 41.33, 9.56, 51.09]\n",
"# Example 4: Lagoon From Venice to Trieste\n",
"aoi_bbox = [12.0, 44.4, 14.0, 45.0]\n",
"\n",
"# Time range\n",
"start_date = \"2025-07-01T00:00:00Z\"\n",
"end_date = \"2025-07-31T23:59:59Z\"\n",
"start_date = \"2025-01-01T00:00:00Z\"\n",
"end_date = \"2025-12-31T23:59:59Z\"\n",
"\n",
"print(f\"Area of Interest: {aoi_bbox}\")\n",
"print(f\"Time Range: {start_date} to {end_date}\")"
Expand All @@ -124,55 +109,9 @@
},
{
"cell_type": "code",
"execution_count": 27,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"📚 Available Collections (12 total):\n",
"\n",
" - sentinel-2-l2a\n",
" The Sentinel-2 Level-2A Collection 1 product provides orthorectified Surface Reflectance (Bottom-Of-...\n",
"\n",
" - sentinel-3-slstr-l1-rbt\n",
" The Sentinel-3 SLSTR Level-1B RBT product provides radiances and brightness temperatures for each pi...\n",
"\n",
" - sentinel-3-olci-l2-lrr\n",
" The Sentinel-3 OLCI L2 LRR product provides land and atmospheric geophysical parameters computed for...\n",
"\n",
" - sentinel-2-l1c\n",
" The Sentinel-2 Level-1C product is composed of 110x110 km2 tiles (ortho-images in UTM/WGS84 projecti...\n",
"\n",
" - sentinel-3-olci-l2-lfr\n",
" The Sentinel-3 OLCI L2 LFR product provides land and atmospheric geophysical parameters computed for...\n",
"\n",
" - sentinel-3-olci-l1-efr\n",
" The Sentinel-3 OLCI L1 EFR product provides TOA radiances at full resolution for each pixel in the i...\n",
"\n",
" - sentinel-1-l1-grd\n",
" The Sentinel-1 Level-1 Ground Range Detected (GRD) products consist of focused SAR data that has bee...\n",
"\n",
" - sentinel-3-slstr-l2-frp\n",
" The Sentinel-3 SLSTR Level-2 FRP product provides global (over land and water) fire radiative power.\n",
"\n",
" - sentinel-3-olci-l1-err\n",
" The Sentinel-3 OLCI L1 ERR product provides TOA radiances at reduced resolution for each pixel in th...\n",
"\n",
" - sentinel-1-l1-slc\n",
" The Sentinel-1 Level-1 Single Look Complex (SLC) products consist of focused SAR data, geo-reference...\n",
"\n",
" - sentinel-3-slstr-l2-lst\n",
" The Sentinel-3 SLSTR Level-2 LST product provides land surface temperature.\n",
"\n",
" - sentinel-1-l2-ocn\n",
" The Sentinel-1 Level-2 Ocean (OCN) products for wind, wave and currents applications may contain the...\n",
"\n"
]
}
],
"outputs": [],
"source": [
"# Connect to STAC API\n",
"catalog = Client.open(STAC_API_URL)\n",
Expand Down Expand Up @@ -203,49 +142,30 @@
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"🔍 Searching collection: sentinel-2-l2a\n",
"🎯 Target collection for processing: sentinel-2-l2a\n"
]
}
],
"outputs": [],
"source": [
"# Choose the source collection to search\n",
"source_collection = \"sentinel-2-l2a\" # Change this to your desired collection\n",
"\n",
"# Choose the target collection for processing\n",
"target_collection = \"sentinel-2-l2a-staging\" # Change this to your target collection\n",
"target_collection = \"sentinel-2-l2a\" # Change this to your target collection\n",
"\n",
"print(f\"🔍 Searching collection: {source_collection}\")\n",
"print(f\"🎯 Target collection for processing: {target_collection}\")"
]
},
{
"cell_type": "code",
"execution_count": 29,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"✅ Found 10 items (after filtering).\n",
"\n"
]
}
],
"outputs": [],
"source": [
"# Search for items\n",
"search = catalog.search(\n",
" collections=[source_collection],\n",
" bbox=aoi_bbox,\n",
" datetime=f\"{start_date}/{end_date}\", # Adjust as needed\n",
" limit=100, # Adjust limit as needed\n",
" limit=200, # Adjust limit as needed\n",
")\n",
"\n",
"# Collect items paginated results and clean them (workaround for issue #26)\n",
Expand Down Expand Up @@ -286,7 +206,7 @@
},
{
"cell_type": "code",
"execution_count": 33,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -327,34 +247,9 @@
},
{
"cell_type": "code",
"execution_count": 35,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"📤 Submitting 10 items to pipeline...\n",
"\n",
"✅ Submitted: S2C_MSIL2A_20250728T100051_N0511_R122_T33TTG_20250728T153115\n",
"✅ Submitted: S2C_MSIL2A_20250728T100051_N0511_R122_T32TQM_20250728T153115\n",
"✅ Submitted: S2B_MSIL2A_20250713T100029_N0511_R122_T33TUG_20250713T123724\n",
"✅ Submitted: S2B_MSIL2A_20250713T100029_N0511_R122_T33TTG_20250713T123724\n",
"✅ Submitted: S2B_MSIL2A_20250713T100029_N0511_R122_T32TQM_20250713T123724\n",
"✅ Submitted: S2C_MSIL2A_20250708T100051_N0511_R122_T33TTG_20250708T155705\n",
"✅ Submitted: S2C_MSIL2A_20250708T100051_N0511_R122_T32TQM_20250708T155705\n",
"✅ Submitted: S2B_MSIL2A_20250703T100029_N0511_R122_T33TUG_20250703T122001\n",
"✅ Submitted: S2B_MSIL2A_20250703T100029_N0511_R122_T33TTG_20250703T122001\n",
"✅ Submitted: S2B_MSIL2A_20250703T100029_N0511_R122_T32TQM_20250703T122001\n",
"\n",
"📊 Summary:\n",
" - Successfully submitted: 10\n",
" - Failed: 0\n",
" - Total: 10\n"
]
}
],
"outputs": [],
"source": [
"# Submit all found items to the pipeline\n",
"if items:\n",
Expand Down Expand Up @@ -439,7 +334,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.1"
"version": "3.11.0"
}
},
"nbformat": 4,
Expand Down
3 changes: 2 additions & 1 deletion stac/sentinel-2-l2a.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,5 +306,6 @@
"https://stac-extensions.github.io/product/v0.1.0/schema.json",
"https://stac-extensions.github.io/sat/v1.0.0/schema.json",
"https://stac-extensions.github.io/scientific/v1.0.0/schema.json"
]
],
"eodash:rasterform": "https://raw.githubusercontent.com/EOPF-Explorer/eodash-assets/refs/heads/main/forms/bandsform.json"
}