Skip to content

Commit 2467dcb

Browse files
agniveshadhikarilmazuel
agniveshadhikari
and
lmazuel
authored
[FarmBeats, samples] Initial samples (Azure#19398)
* Initial samples * minor corrections * Sync and async each sample * Updates to samples * minor doc up * Adding cascade delete and satellite flow samples * Addiing farm hierarchy samples * Updates to readme for new samples * minor update to doc * master -> main * Restructuring the samples. Working around windows proactor policy. Updating readme. Introducing .env file for credentials. * Fixing typo * Another typo * More typos * Added more description to env vars everywhere * Blurb about async http clients. * Auth snippet change * List -> Table for description of samples. * satellie to satellite Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
1 parent eb5e43b commit 2467dcb

14 files changed

+2085
-0
lines changed

‎sdk/agrifood/azure-agrifood-farming/samples/.env‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
FARMBEATS_ENDPOINT = "https://<your-farmbeats-resource-name>.farmbeats.azure.net"
2+
AZURE_TENANT_ID = "<your-tenant-id>"
3+
AZURE_CLIENT_ID = "<your-client-id>"
4+
AZURE_CLIENT_SECRET = "<your-client-secret>"

‎sdk/agrifood/azure-agrifood-farming/samples/README.md‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
---
2+
page_type: sample
3+
languages:
4+
- python
5+
products:
6+
- azure
7+
- azure-farmbeats
8+
urlFragment: farmbeats-samples
9+
---
10+
11+
# Azure FarmBeats samples for Python client
12+
13+
The following sample programs demonstrate some common use case scenairos for the [FarmBeats Python client library][python_sdk].
14+
15+
| Files | Description |
16+
| ------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
17+
| [sample_hello_world.py][hello_world_sample] | This sample demonstrates the most basic operation that can be performed - creation of a Farmer. Use this to understand how to create the client object, how to authenticate it, and make sure your client is set up correctly to call into your FarmBeats endpoint. |
18+
| [sample_farm_hierarchy.py][farm_hierarchy_sample] | This sample demonstrates how to create a simple [farm hierarchy][farm_hierarchy_docs], comprising of Farmer, Farm, Field and Boundary. |
19+
| [sample_farm_hierarchy_complete.py][farm_hierarchy_complete_sample] | This builds on the previous sample to add to the previous sample, and demonstrate the relationships between Crops, Crop Varities, Seasons and Seasonal Fields and the other farm hierarchy objects. |
20+
| [sample_satellite_download.py][satellite_download_sample] | This sample demonstrates FarmBeats' satellite integrations and how to ingest satellite imagery into the platform, and then download them onto a local directory. This demonstrates key concepts of how to create a job in FarmBeats, and how to poll on the completion of the job using the SDK. In the [corresponding async sample][satellite_download_async_sample] we demonstrate how to handle concurrent download of potentially hundreds of files by limiting the max concurrency using semaphores. |
21+
| [sample_attachments.py][attachments_sample] | This sample demonstrates FarmBeats' capabaility of storing arbitrary files in context to the various farm hierarchy objects. We first attach some files onto a farmer and a farm, and then download all existing attachments for the farmer onto a local directory. |
22+
| [sample_cascade_delete.py][cascade_delete_sample] | This sample demonstrates the usage of cascade delete jobs to perform cleanup of farm hierarchy objects. A cascade delete job handles the recursive deletion of all dependent data for the given parent resource. Use this to clean up the sample resources that you create in the other samples. |
23+
24+
Additionally, for each sample, there are corresponding files in the [`samples/async`][async_samples] directory that demonstrate the same scenarios using the async FarmBeats client. Using the async approach will offer much better performance when parts of the code can be executed concurrentlly. This is especially important when downloading files, such as in the satellite data download sample, and in the attachments sample.
25+
26+
27+
## Prerequisites
28+
29+
To run the samples, you need:
30+
31+
- A [python][get_python] environment. Supported versions are 2.7 and 3.6+.
32+
- An Azure subscription. Create a free subscription [here][azure_free_sub].
33+
- A FarmBeats resource. See [installation docs][install_farmbeats] to create a new FarmBeats resource.
34+
35+
Additionally, there are some specific prerequisites if you want to leverage third party integrations into FarmBeats:
36+
37+
- A subscription to a [supported weather provider][weather_docs], to run the weather sample.
38+
39+
## How to run the samples
40+
41+
### Install the dependencies
42+
43+
To run the samples, you need to install the following dependencies:
44+
```bash
45+
pip install azure-agrifood-farming azure-identity aiohttp python-dotenv
46+
```
47+
_Note: You can use your preferred async http client to use the async FarmBeats client, instead of using aiohttp._
48+
49+
### Set up the credentials for authentication
50+
51+
We use [azure-identity][azure_identity]'s [DefaultAzureCredential][azure_identity_default_azure_credential] to authenticate to your FarmBeats instance. If you have followed the [installation docs][install_farmbeats], you should already have an application created, and the appropriate RBAC roles assigned.
52+
53+
Set the following variables to the appropriate values either in the [`.env`][dot_env_file] file, or alternatively in your environment variables.
54+
55+
- `AZURE_TENANT_ID`: The tenant ID of your active directory application.
56+
- `AZURE_CLIENT_ID`: The client ID of your active directory application.
57+
- `AZURE_CLIENT_SECRET`: The client secret of your active directory application.
58+
- `FARMBEATS_ENDPOINT`: The FarmBeats endpoint that you want to run these samples on.
59+
60+
_Note: There are alternate mechanisms of authentication supported by `azure-identity`. Check out the docs [here][azure_identity]_.
61+
62+
Once these are set correctly, you can go ahead and run the sample_hello_world.py sample to make sure everything works correctly.
63+
64+
```bash
65+
python sample_hello_world.py
66+
```
67+
68+
If everything worked fine, you should see an output like this:
69+
```
70+
Creating farmer, or updating if farmer already exists... Done
71+
Here are the details of the farmer:
72+
ID: contoso-farmer
73+
Name: Contoso
74+
Description: Contoso is hard working.
75+
Created timestamp: 2021-06-21 10:21:11+00:00
76+
Last modified timestamp: 2021-06-22 21:01:35+00:00
77+
```
78+
79+
80+
<!-- Product docs aka.ms links-->
81+
[farm_hierarchy_docs]: https://aka.ms/FarmBeatsFarmHierarchyDocs
82+
[weather_docs]: https://aka.ms/FarmBeatsWeatherDocs/
83+
[install_farmbeats]: https://aka.ms/FarmBeatsInstallDocumentationPaaS/
84+
85+
<!-- Links to samples files -->
86+
[async_samples]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agrifood/azure-agrifood-farming/samples/async
87+
[hello_world_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_hello_world.py
88+
[attachments_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_attachments.py
89+
[cascade_delete_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_cascade_delete.py
90+
[satellite_download_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_satellite_download.py
91+
[farm_hierarchy_complete_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_farm_hierarchy_complete.py
92+
[farm_hierarchy_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/sample_farm_hierarchy.py
93+
[satellite_download_async_sample]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/async/sample_satellite_download_async.py
94+
[dot_env_file]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agrifood/azure-agrifood-farming/samples/.env
95+
96+
<!-- Microsoft/Azure related links -->
97+
[azure_free_sub]: https://azure.microsoft.com/free/
98+
[azure_identity]: https://pypi.org/project/azure-identity/
99+
[azure_identity_default_azure_credential]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity#defaultazurecredential
100+
[python_sdk]: https://pypi.org/project/azure-agrifood-farming/
101+
102+
<!-- Links to external sites -->
103+
[get_python]: https://www.python.org/

‎sdk/agrifood/azure-agrifood-farming/samples/async/sample_attachments_async.py‎

Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT license.
3+
4+
"""
5+
FILE: sample_attachments_async.py
6+
7+
DESCRIPTION:
8+
This sample demonstrates FarmBeats' capabaility of storing arbitrary files
9+
in context to the various farm hierarchy objects.
10+
We first attach some files onto a farmer and a farm, and then download all
11+
existing attachments for the farmer onto a local directory.
12+
13+
USAGE:
14+
```python sample_attachments_async.py```
15+
16+
Set the environment variables with your own values before running the sample:
17+
- `AZURE_TENANT_ID`: The tenant ID of your active directory application.
18+
- `AZURE_CLIENT_ID`: The client ID of your active directory application.
19+
- `AZURE_CLIENT_SECRET`: The client secret of your active directory application.
20+
- `FARMBEATS_ENDPOINT`: The FarmBeats endpoint that you want to run these samples on.
21+
"""
22+
23+
from typing import Dict
24+
from azure.core.exceptions import ResourceNotFoundError
25+
from azure.identity.aio import DefaultAzureCredential
26+
from azure.agrifood.farming.aio import FarmBeatsClient
27+
from azure.agrifood.farming.models import Farmer
28+
from pathlib import Path
29+
import asyncio
30+
import os
31+
from dotenv import load_dotenv
32+
33+
34+
async def sample_attachments_async():
35+
36+
farmbeats_endpoint = os.environ['FARMBEATS_ENDPOINT']
37+
38+
credential = DefaultAzureCredential()
39+
40+
client = FarmBeatsClient(
41+
endpoint=farmbeats_endpoint,
42+
credential=credential
43+
)
44+
45+
farmer_id = "contoso-farmer"
46+
farm_id = "contoso-farm"
47+
attachment_on_farmer_id = "contoso-farmer-attachment-1"
48+
attachment_on_farm_id = "contoso-farm-attachment-1"
49+
attachment_on_farmer_file_path = "../test.txt"
50+
attachment_on_farm_file_path = "../test.txt"
51+
52+
53+
if not (os.path.isfile(attachment_on_farmer_file_path) and
54+
os.path.isfile(attachment_on_farm_file_path)):
55+
raise SystemExit(
56+
"Please provide the paths to the files you want to upload."
57+
)
58+
59+
# Ensure farmer exists, create if necessary.
60+
print(f"Create/updating farmer with id {farmer_id}...", end=" ", flush=True)
61+
await client.farmers.create_or_update(
62+
farmer_id=farmer_id,
63+
farmer=Farmer()
64+
)
65+
print("Done!")
66+
67+
# Ensure farm exists, create if necessary.
68+
print(f"Create/updating farm with id {farm_id}...", end=" ", flush=True)
69+
await client.farms.create_or_update(
70+
farmer_id=farmer_id,
71+
farm_id=farm_id,
72+
farm=Farmer()
73+
)
74+
print("Done!")
75+
76+
# Create attachment on farmer
77+
try:
78+
print(f"Checking if attachment with id {attachment_on_farmer_id} already exists "
79+
f"on farmer with id {farmer_id}...", end=" ", flush=True)
80+
await client.attachments.get(
81+
farmer_id=farmer_id,
82+
attachment_id=attachment_on_farmer_id
83+
)
84+
print("Attachment already exists. Not updating file.")
85+
86+
except ResourceNotFoundError:
87+
print("Attachment doesn't exist")
88+
print("Creating attachment...", end=" ", flush=True)
89+
90+
# Open file with buffering set to 0, to get a IO object.
91+
file_to_attach_on_farmer = open(
92+
attachment_on_farmer_file_path,
93+
"rb",
94+
buffering=0)
95+
96+
await client.attachments.create_or_update(
97+
farmer_id=farmer_id,
98+
attachment_id=attachment_on_farmer_id,
99+
resource_id= farmer_id,
100+
resource_type="Farmer",
101+
file=file_to_attach_on_farmer)
102+
103+
print("Done!")
104+
105+
# Create attachment with farm
106+
try:
107+
print(f"Checking if attachment with id {attachment_on_farm_id} already exists " +
108+
f"on farm with id {farm_id}...", end=" ", flush=True)
109+
await client.attachments.get(
110+
farmer_id=farmer_id,
111+
attachment_id=attachment_on_farm_id
112+
)
113+
print("Attachment already exists. Not updating file.")
114+
115+
except ResourceNotFoundError:
116+
print("Attachment doesn't exist")
117+
print("Creating attachment...", end=" ", flush=True)
118+
119+
# Open file with buffering set to 0, to get a IO object.
120+
file_to_attach_on_farm = open(
121+
attachment_on_farm_file_path,
122+
"rb",
123+
buffering=0)
124+
125+
await client.attachments.create_or_update(
126+
farmer_id=farmer_id,
127+
attachment_id=attachment_on_farm_id,
128+
resource_id= farm_id,
129+
resource_type="Farm",
130+
file=file_to_attach_on_farm)
131+
132+
print("Done!")
133+
134+
print("Proceeding to download all attachments on the farmer. " +
135+
"Press enter to continue...")
136+
input()
137+
138+
139+
print("Getting a list of all attachments " +
140+
f"on the farmer with id {farmer_id}...", end=" ", flush=True)
141+
farmer_attachments = client.attachments.list_by_farmer_id(
142+
farmer_id=farmer_id,
143+
)
144+
print("Done!")
145+
146+
# Using a semaphore to limit the number of concurrent downloads.
147+
semaphore = asyncio.Semaphore(2)
148+
149+
150+
print("Downloading attachments with a maximum concurrency "+
151+
"of two downloads at a time...")
152+
153+
# Setting up a async function (a coroutine) to download each attachment
154+
async def download(attachment, semaphore):
155+
async with semaphore:
156+
downloaded_attachment = await client.attachments.download(
157+
farmer_id=farmer_id,
158+
attachment_id=attachment_on_farmer_id
159+
)
160+
out_path = Path(
161+
"../data/attachments/" +
162+
f"{attachment.resource_type}/{attachment.resource_id}" +
163+
f"/{attachment.id}/{attachment.original_file_name}"
164+
)
165+
166+
# Make sure the dirs to the output path exists
167+
out_path.parent.mkdir(parents=True, exist_ok=True)
168+
169+
print(f"Saving attachment id {attachment.id} to {out_path.resolve()}")
170+
with open(
171+
out_path,
172+
'wb'
173+
) as out_file:
174+
async for bits in downloaded_attachment:
175+
out_file.write(bits)
176+
177+
await asyncio.gather(
178+
*[download(attachment, semaphore) async for attachment in farmer_attachments]
179+
)
180+
181+
print("Done!")
182+
183+
await client.close()
184+
await credential.close()
185+
186+
187+
if __name__ == "__main__":
188+
189+
load_dotenv()
190+
191+
asyncio.get_event_loop().run_until_complete(sample_attachments_async())

‎sdk/agrifood/azure-agrifood-farming/samples/async/sample_cascade_delete_async.py‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT license.
3+
4+
"""
5+
FILE: sample_cascade_delete_async.py
6+
7+
DESCRIPTION:
8+
This sample demonstrates
9+
- Getting a filterd list of farmers based on last modified timestamp
10+
- Queuing a cascade delete job on a farmer, and polling for it to complete
11+
12+
USAGE:
13+
```python sample_cascade_delete_async.py```
14+
15+
Set the environment variables with your own values before running the sample:
16+
- `AZURE_TENANT_ID`: The tenant ID of your active directory application.
17+
- `AZURE_CLIENT_ID`: The client ID of your active directory application.
18+
- `AZURE_CLIENT_SECRET`: The client secret of your active directory application.
19+
- `FARMBEATS_ENDPOINT`: The FarmBeats endpoint that you want to run these samples on.
20+
"""
21+
22+
from azure.identity.aio import DefaultAzureCredential
23+
from azure.agrifood.farming.aio import FarmBeatsClient
24+
import os
25+
from datetime import datetime, timedelta
26+
from random import randint
27+
from isodate import UTC
28+
import asyncio
29+
from dotenv import load_dotenv
30+
31+
32+
async def sample_cascade_delete_async():
33+
34+
farmbeats_endpoint = os.environ['FARMBEATS_ENDPOINT']
35+
36+
credential = DefaultAzureCredential()
37+
38+
client = FarmBeatsClient(
39+
endpoint=farmbeats_endpoint,
40+
credential=credential
41+
)
42+
43+
job_id_prefix = "cascade-delete-job"
44+
45+
# Getting list of farmers modified in the last 7 days
46+
print("Getting list of recently modified farmer id's... ", end="", flush=True)
47+
farmers = client.farmers.list(
48+
min_last_modified_date_time=datetime.now(tz=UTC) - timedelta(days=7)
49+
)
50+
farmer_ids = [farmer.id async for farmer in farmers]
51+
print("Done")
52+
53+
# Ask for the id of the farmer which is to be deleted.
54+
print(f"Recentely modified farmer id's:")
55+
print(*farmer_ids, sep="\n")
56+
farmer_id_to_delete = input("Please enter the id of the farmer you wish to delete resources for: ").strip()
57+
if farmer_id_to_delete not in farmer_ids:
58+
raise SystemExit("Entered id for farmer does not exist.")
59+
60+
# Deleting the farmer and it's associated resources. Queuing the cascade delete job.
61+
62+
job_id = f"{job_id_prefix}-{randint(0, 1000)}"
63+
print(f"Queuing cascade delete job {job_id}... ", end="", flush=True)
64+
cascade_delete_job_poller = await client.farmers.begin_create_cascade_delete_job(
65+
job_id=job_id,
66+
farmer_id = farmer_id_to_delete
67+
)
68+
print("Queued. Waiting for completion... ", end="", flush=True)
69+
await cascade_delete_job_poller.result()
70+
71+
print("The job completed with status", cascade_delete_job_poller.status())
72+
73+
await client.close()
74+
await credential.close()
75+
76+
77+
if __name__ == "__main__":
78+
79+
load_dotenv()
80+
81+
asyncio.get_event_loop().run_until_complete(sample_cascade_delete_async())

0 commit comments

Comments
 (0)