docs: update README.md with code

Author: wietzesuijker

README.md

@@ -1,123 +1,122 @@
 # EOPF GeoZarr Data Pipeline
 
-Automated pipeline for converting Sentinel Zarr datasets to cloud-optimized GeoZarr format with STAC catalog integration and interactive visualization.
+Automated Kubernetes pipeline for converting Sentinel Zarr datasets to cloud-optimized GeoZarr format with STAC catalog integration.
 
-## Quick Start (30 seconds)
+## Quick Start
 
 ```bash
-# 1. Submit workflow
 export KUBECONFIG=.work/kubeconfig
 kubectl create -f workflows/run-s1-test.yaml -n devseed-staging
-
-# 2. Monitor
-kubectl logs -n devseed-staging -l workflows.argoproj.io/workflow=<name> -c main -f
+kubectl get wf -n devseed-staging -w
 ```
 
-📖 **New here?** [GETTING_STARTED.md](GETTING_STARTED.md) • **Details:** [Full docs below](#submitting-workflows)
+📖 **First time?** See [GETTING_STARTED.md](GETTING_STARTED.md) for full setup
+🎯 **Monitor:** [Argo UI](https://argo-workflows.hub-eopf-explorer.eox.at)
 
 ## What It Does
 
-**Input:** STAC item URL → **Output:** Interactive web map in ~15-20 minutes
-
-```
-Convert (15 min) → Register (30 sec) → Augment (10 sec)
-```
-
-**Supports:** Sentinel-1 GRD (SAR) • Sentinel-2 L2A (optical)
-
-**Prerequisites:** Kubernetes with [platform-deploy](https://github.com/EOPF-Explorer/platform-deploy) • Python 3.11+ • [GETTING_STARTED.md](GETTING_STARTED.md) for full setup
-
-## Submitting Workflows
+**Input:** STAC item URL → **Output:** Cloud-optimized GeoZarr + Interactive map (~15-20 min)
 
-| Method | Best For | Setup | Status |
-|--------|----------|-------|--------|
-| 🎯 **kubectl** | Testing, CI/CD | None | ✅ Recommended |
-| 📓 **Jupyter** | Learning, exploration | 2 min | ✅ Working |
-| ⚡ **Event-driven** | Production (auto) | In-cluster | ✅ Running |
-| 🐍 **Python CLI** | Scripting | Port-forward | ⚠️ Advanced |
+**Supports:** Sentinel-1 GRD, Sentinel-2 L2A
+**Stack:** Argo Workflows • [eopf-geozarr](https://github.com/EOPF-Explorer/data-model) • Dask • RabbitMQ • Prometheus
+**Resources:** 6Gi memory, burstable CPU per workflow
 
-<details>
-<summary><b>kubectl</b> (recommended)</summary>
+## Monitoring
 
 ```bash
-export KUBECONFIG=.work/kubeconfig
-kubectl create -f workflows/run-s1-test.yaml -n devseed-staging -o name
-kubectl logs -n devseed-staging -l workflows.argoproj.io/workflow=<wf-name> -c main -f
+# Health check
+kubectl get wf -n devseed-staging --field-selector status.phase=Running
+
+# Recent workflows (last hour)
+kubectl get wf -n devseed-staging --sort-by=.metadata.creationTimestamp | tail -10
 ```
-Edit `workflows/run-s1-test.yaml` with your STAC URL and collection.
-</details>
 
-<details>
-<summary><b>Jupyter</b></summary>
+**Web UI:** [Argo Workflows](https://argo-workflows.hub-eopf-explorer.eox.at)
 
+## Usage
+
+### kubectl (Testing)
 ```bash
-uv sync --extra notebooks
-cp notebooks/.env.example notebooks/.env
-uv run jupyter lab notebooks/operator.ipynb
+kubectl create -f workflows/run-s1-test.yaml -n devseed-staging
 ```
-</details>
 
-<details>
-<summary><b>Event-driven</b> (production)</summary>
+**Namespaces:** `devseed-staging` (testing) • `devseed` (production)
 
+### Event-driven (Production)
 Publish to RabbitMQ `geozarr` exchange:
 ```json
-{"source_url": "https://stac.../items/S1A_...", "item_id": "S1A_IW_GRDH_...", "collection": "sentinel-1-l1-grd-dp-test"}
+{"source_url": "https://stac.../items/...", "item_id": "...", "collection": "..."}
 ```
-</details>
-
-<details>
-<summary><b>Python CLI</b></summary>
 
+### Jupyter Notebooks
 ```bash
-kubectl port-forward -n core svc/rabbitmq 5672:5672
-export AMQP_PASSWORD=$(kubectl get secret rabbitmq-password -n core -o jsonpath='{.data.rabbitmq-password}' | base64 -d)
-uv run python examples/submit.py --stac-url "..." --collection sentinel-2-l2a
+uv sync --extra notebooks
+cp notebooks/.env.example notebooks/.env
+uv run jupyter lab notebooks/
 ```
-</details>
 
-**Related:** [data-model](https://github.com/EOPF-Explorer/data-model) • [platform-deploy](https://github.com/EOPF-Explorer/platform-deploy) • [Testing report](docs/WORKFLOW_SUBMISSION_TESTING.md)
+See [examples/](examples/) for more patterns.
 
 ## Configuration
 
-<details>
-<summary><b>S3 & RabbitMQ</b></summary>
-
 ```bash
-# S3 credentials
+# S3 credentials (OVH S3)
 kubectl create secret generic geozarr-s3-credentials -n devseed \
-  --from-literal=AWS_ACCESS_KEY_ID="<key>" \
-  --from-literal=AWS_SECRET_ACCESS_KEY="<secret>"
+  --from-literal=AWS_ACCESS_KEY_ID="..." \
+  --from-literal=AWS_SECRET_ACCESS_KEY="..." \
+  --from-literal=AWS_ENDPOINT_URL="https://s3.de.io.cloud.ovh.net"
+
+# S3 output location
+# Bucket: esa-zarr-sentinel-explorer-fra
+# Prefix: tests-output (staging) or geozarr (production)
 
-# RabbitMQ password
+# Get RabbitMQ password
 kubectl get secret rabbitmq-password -n core -o jsonpath='{.data.rabbitmq-password}' | base64 -d
-```
 
-**Endpoints:** S3: `s3.de.io.cloud.ovh.net/esa-zarr-sentinel-explorer-fra` • RabbitMQ: `geozarr` exchange • [UIs](https://workspace.devseed.hub-eopf-explorer.eox.at/): [Argo](https://argo-workflows.hub-eopf-explorer.eox.at) • [STAC](https://api.explorer.eopf.copernicus.eu/stac) • [Viewer](https://api.explorer.eopf.copernicus.eu/raster)
-</details>
+# STAC API endpoints
+# STAC API: https://api.explorer.eopf.copernicus.eu/stac
+# Raster API: https://api.explorer.eopf.copernicus.eu/raster
+```
 
 ## Troubleshooting
 
-<details>
-<summary><b>Logs & Issues</b></summary>
-
 ```bash
-kubectl get wf -n devseed-staging -w
+# Check workflow status
+kubectl get wf -n devseed-staging --sort-by=.metadata.creationTimestamp | tail -5
+
+# View logs
 kubectl logs -n devseed-staging <pod-name> -c main -f
-kubectl logs -n devseed -l sensor-name=geozarr-sensor --tail=50
+
+# Check resources
+kubectl top nodes
 ```
 
-**Common fixes:** Workflow not starting → check sensor logs • S3 denied → verify `geozarr-s3-credentials` secret • RabbitMQ refused → `kubectl port-forward -n core svc/rabbitmq 5672:5672` • Pod pending → check resources
-</details>
+**Common issues:**
+- **Workflow not starting:** Check sensor logs: `kubectl logs -n devseed -l sensor-name=geozarr-sensor`
+- **S3 errors:** Verify credentials secret exists
+- **Pod pending:** Check node capacity with `kubectl top nodes`
+
+**Performance:** S1 GRD (10GB): 15-20 min • S2 L2A (5GB): 8-12 min • Increase if >20GB dataset
+
+See [GETTING_STARTED.md](GETTING_STARTED.md#troubleshooting) for more.
 
 ## Development
 
 ```bash
-uv sync --all-extras && pre-commit install
-make test  # or: pytest tests/ -v -k e2e
+# Setup
+uv sync --all-extras
+pre-commit install
+
+# Test
+pytest tests/ -v  # 100/100 passing
+
+# Deploy
+kubectl apply -f workflows/template.yaml -n devseed
 ```
 
-**Deploy:** Edit `workflows/template.yaml` or `scripts/*.py` → `pytest tests/ -v` → `docker buildx build --platform linux/amd64 -t ghcr.io/eopf-explorer/data-pipeline:dev .` → `kubectl apply -f workflows/template.yaml -n devseed` • [CONTRIBUTING.md](CONTRIBUTING.md)
+**Project structure:** `workflows/` (manifests) • `scripts/` (Python utils) • `tests/` (pytest) • `notebooks/` (tutorials)
+
+**Documentation:** [CONTRIBUTING.md](CONTRIBUTING.md) • [GETTING_STARTED.md](GETTING_STARTED.md)
 
 ## License