Skip to content

Commit 03f1449

Browse files
Jg1255Jg1255
authored
[Eventhub] TableCheckpointstore base branch (Azure#19726)
adding checkingpointstore for table. This is my initial pull request.
1 parent 64128ba commit 03f1449

File tree

15 files changed

+213
-0
lines changed

15 files changed

+213
-0
lines changed

‎sdk/eventhub/azure-eventhub-checkpointstoretable/CHANGELOG.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Release History
2+
3+
## 1.0.0b1 (Unreleased)
4+
- Template package (can update this after implementation is done)

‎sdk/eventhub/azure-eventhub-checkpointstoretable/MANIFEST.in‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
include *.md
2+
include azure/__init__.py
3+
include azure/eventhub/__init__.py
4+
include azure/eventhub/extensions/__init__.py
5+
recursive-include tests *.py
6+
recursive-include samples *.py

‎sdk/eventhub/azure-eventhub-checkpointstoretable/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
[![Build Status](https://dev.azure.com/azure-sdk/public/_apis/build/status/azure-sdk-for-python.client?branchName=main)](https://dev.azure.com/azure-sdk/public/_build/latest?definitionId=46?branchName=main)
2+
3+
# Azure EventHubs Checkpoint Store client library for Python using Tables
4+
5+
Azure EventHubs Checkpoint Store is used for storing checkpoints while processing events from Azure Event Hubs.
6+
This Checkpoint Store package works as a plug-in package to `EventHubConsumerClient`. It uses Azure Tables as the persistent store for maintaining checkpoints and partition ownership information.
7+
8+
9+
# Getting started
10+
11+
### Prerequisites
12+
13+
- Python2.7, Python 3.6 or later.
14+
- **Microsoft Azure Subscription:** To use Azure services, including Azure Event Hubs, you'll need a subscription. If you do not have an existing Azure account, you may sign up for a free trial or use your MSDN subscriber benefits when you [create an account](https://azure.microsoft.com/).
15+
16+
- **Event Hubs namespace with an Event Hub:** To interact with Azure Event Hubs, you'll also need to have a namespace and Event Hub available. If you are not familiar with creating Azure resources, you may wish to follow the step-by-step guide for [creating an Event Hub using the Azure portal](https://docs.microsoft.com/azure/event-hubs/event-hubs-create). There, you can also find detailed instructions for using the Azure CLI, Azure PowerShell, or Azure Resource Manager (ARM) templates to create an Event Hub.
17+
18+
- **Azure Storage Account:** You'll need to have an Azure Storage Account and create a Azure Table Storage to store the checkpoint data with entities. You may follow the guide [creating an Azure Table Storage Account]
19+
(https://docs.microsoft.com/azure/storage/tables/table-storage-overview).
20+
21+
# Key concepts
22+
23+
Bullet point list of your library's main concepts.
24+
25+
# Examples
26+
27+
Examples of some of the key concepts for your library.
28+
29+
# Troubleshooting
30+
31+
Running into issues? This section should contain details as to what to do there.
32+
33+
# Next steps
34+
35+
More sample code should go here, along with links out to the appropriate example tests.
36+
37+
# Contributing
38+
39+
If you encounter any bugs or have suggestions, please file an issue in the [Issues](<https://github.com/Azure/azure-sdk-for-python/issues>) section of the project.
40+
41+
![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Feventhub%2Fazure-eventhub-checkpointstoretable%2FREADME.png)

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/__init__.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__path__ = __import__("pkgutil").extend_path(__path__, __name__)

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/checkpointstoretable/__init__.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
6+
from ._version import VERSION
7+
8+
__version__ = VERSION
9+
10+
from ._tablestoragecs import TableCheckpointStore
11+
12+
__all__ = [
13+
"TableCheckpointStore",
14+
]

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/checkpointstoretable/_tablestoragecs.py‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# --------------------------------------------------------------------------------------------
2+
# Copyright (c) Microsoft Corporation. All rights reserved.
3+
# Licensed under the MIT License. See License.txt in the project root for license information.
4+
# --------------------------------------------------------------------------------------------
5+
6+
class TableCheckpointStore:
7+
"""A CheckpointStore that uses Azure Table Storage to store the partition ownership and checkpoint data.
8+
This class implements methods list_ownership, claim_ownership, update_checkpoint and list_checkpoints.
9+
:param str table_account_url:
10+
The URI to the storage account.
11+
:param table_name:
12+
The name of the table for the tables.
13+
:type table_name: str
14+
:param credential:
15+
The credentials with which to authenticate. This is optional if the
16+
account URL already has a SAS token. The value can be a SAS token string, an account
17+
shared access key, or an instance of a TokenCredentials class from azure.identity.
18+
If the URL already has a SAS token, specifying an explicit credential will take priority.
19+
:keyword str api_version:
20+
The Storage API version to use for requests. Default value is '2019-07-07'.
21+
:keyword str secondary_hostname:
22+
The hostname of the secondary endpoint.
23+
"""
24+
25+
def __init__(self, **kwargs):
26+
pass
27+
28+
def list_ownership(self, namespace, eventhub, consumergroup, **kwargs):
29+
pass
30+
31+
def list_checkpoints(self, namespace, eventhub, consumergroup, **kwargs):
32+
pass
33+
34+
def update_checkpoint(self, checkpoint, **kwargs):
35+
pass
36+
37+
def claim_ownership(self, ownershiplist, **kwargs):
38+
pass

‎sdk/eventhub/azure-eventhub-checkpointstoretable/azure/eventhub/extensions/checkpointstoretable/_version.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# ------------------------------------
2+
# Copyright (c) Microsoft Corporation.
3+
# Licensed under the MIT License.
4+
# ------------------------------------
5+
6+
VERSION = "1.0.0b1"

‎sdk/eventhub/azure-eventhub-checkpointstoretable/dev_requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
-e ../../../tools/azure-sdk-tools
2+
../../core/azure-core
3+
-e ../../../tools/azure-devtools

0 commit comments

Comments
 (0)