Merge pull request #42 from bitol-io/dev

v2.2.2 release

Author: pflooky

.github/workflows/docs-site-deploy.yaml

@@ -0,0 +1,32 @@
+name: docs-site-deploy
+on:
+  push:
+    tags:
+      - "v*"
+permissions:
+  contents: write
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Configure Git Credentials
+        run: |
+          git config user.name github-actions[bot]
+          git config user.email 41898282+github-actions[bot]@users.noreply.github.com
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+      - uses: actions/cache@v4
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+      - run: pip install mkdocs-open-in-new-tab mike
+      - run: pip install mkdocs-material
+      - run: mkdocs build
+      - run: mike deploy --push --update-aliases ${{ github.ref_name }} latest

.gitignore

@@ -1,2 +1,3 @@
 
 .DS_Store
+.idea

CHANGELOG.md

@@ -1,5 +1,16 @@
 This document tracks the history and evolution of the **Open Data Contract Standard**.
 
+# v2.2.2 - 2024-01-05 - OPEN
+
+* Change `dataset.description` data type from `array` to `string`
+* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`
+* Change `price.priceAmount` data type from `string` to `number`
+* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`
+* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`
+* Update [examples](docs/examples) to adhere to JSON schema
+* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml)
+* Add in mkdocs for creating documentation website
+
 # v2.2.1 - 2023-12-18 - APPROVED
 
 * Reformat quality examples to be valid YAML.

README.md

@@ -7,10 +7,10 @@ Welcome!
 Thanks for your interest and for taking the time to come here! ❤️
 
 ## Executive summary
-This standard describes a structure for a **data contract**. Its current version is 2.2.1. It is available for you as an Apache 2.0 license. Contributions are welcome!
+This standard describes a structure for a **data contract**. It's current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
 
 ## Discover the open standard
-Discover the [Open Data Contract Standard](./docs/README.md). This file contains some explanations and several examples. More [examples](./examples/README.md) have been added to v2.2.1.
+Discover the [Open Data Contract Standard](docs/index.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
 
 ## What is a Data Contract?
 
@@ -29,7 +29,8 @@ A data contract defines the agreement between a data producer and consumers. A d
 
 ### JSON Schema
 
-JSON Schema for ODCS can be found [here](schema/odcs-json-schema.json). You can import this schema into your IDE for validation of your YAML files. The links below show how you can import the schema:
+JSON Schema for ODCS can be found [here](schema/odcs-json-schema.json). You can import this schema into your IDE for 
+validation of your YAML files. Links below show how you can import the schema:
 
 - [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom)
 - [VS Code](https://code.visualstudio.com/docs/languages/json#_json-schemas-and-settings)
@@ -60,3 +61,28 @@ Formerly known as the data contract template, this standard is used to implement
 
 ### How does PayPal use Data Contracts?
 PayPal uses data contracts in many ways, but this [article](https://medium.com/paypal-tech/the-next-generation-of-data-platforms-is-the-data-mesh-b7df4b825522) from the [PayPal Technology blog](https://medium.com/paypal-tech) gives a good introduction.
+
+### Mkdocs mike versioning
+To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
+
+```bash
+pip install mike
+cd open-data-contract-standard                    #ensure you are inside the repo
+mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
+mike set-default --push latest                    #by default, users will go to latest
+```
+
+#### Deploying a new version
+Given that the Github action [here](.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
+created, all that is required is to:
+1. [Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
+2. Put in new tag version for release (follows pattern v*)
+3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
+
+#### Delete version
+If a version tag was pushed that was incorrect, it can be deleted via:
+
+```bash
+mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
+mike delete <incorrect tag> --push
+```

docs/contributing.md

@@ -0,0 +1,85 @@
+# Open Data Contract Standard
+
+## Executive summary
+First off, thanks for taking the time to contribute! ❤️
+
+All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉
+
+You do not have to be a member of AIDA User Group to contribute, although becoming a member is free. Strength is always in the number. Check [it out](https://aidausergroup.org/join/).
+
+> And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:
+> - Star the project.
+> - Tweet about it.
+> - Refer this project in your project's readme.
+> - Mention the project at local meetups and tell your friends/colleagues.
+
+<!-- omit in toc -->
+## Table of Contents
+
+- [Code of Conduct](#code-of-conduct)
+- [I Have a Question](#i-have-a-question)
+- [I Want To Contribute](#i-want-to-contribute)
+- [Suggesting Enhancements](#suggesting-enhancements)
+- [Improving The Documentation](#improving-the-documentation)
+- [Join The Project Team](#join-the-project-team)
+
+
+## Code of Conduct
+
+This project and everyone participating in it is governed by the
+[Open Data Contract Standard Code of Conduct](blob/master/CODE_OF_CONDUCT.md).
+By participating, you are expected to uphold this code. Please report unacceptable behavior
+to [@jgperrin](https://github.com/jgperrin).
+
+
+## I Have a Question
+
+** New **
+
+AIDA User Group also opened its Slack for Data Contract discussion. It is an alternate way of contributing to this project. The Slack channel is now [available](https://aidaug.slack.com/archives/C05UZRSBKLY).
+
+You have to be a member of AIDA User Group (it's free) to have access to our Slack channel. All the details are [here](https://aidausergroup.org/welcome/).
+
+> If you want to ask a question, we assume that you have read the available [Documentation](https://github.com/AIDAUserGroup/open-data-contract-standard).
+
+Before you ask a question, it is best to search for existing [Issues](https://github.com/AIDAUserGroup/open-data-contract-standard/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first.
+
+If you then still feel the need to ask a question and need clarification, we recommend the following:
+
+- Open an [Issue](https://github.com/AIDAUserGroup/open-data-contract-standard/issues).
+- Provide as much context as you can about what you're running into.
+
+We will then take care of the issue as soon as possible.
+
+## I Want To Contribute
+
+> ### Legal Notice <!-- omit in toc -->
+> When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license.
+
+### Suggesting Enhancements
+
+This section guides you through submitting an enhancement suggestion for Data Contract Template, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions.
+
+<!-- omit in toc -->
+#### Before Submitting an Enhancement
+
+- Make sure that you are using the latest version.
+- Read the [documentation](https://github.com/AIDAUserGroup/open-data-contract-standard) carefully and find out if the functionality is already covered.
+- Perform a [search](https://github.com/AIDAUserGroup/open-data-contract-standard/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one.
+- Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. 
+
+<!-- omit in toc -->
+#### How Do I Submit a Good Enhancement Suggestion?
+
+Enhancement suggestions are tracked as [GitHub issues](https://github.com/AIDAUserGroup/open-data-contract-standard/issues).
+
+- Use a **clear and descriptive title** for the issue to identify the suggestion.
+- Provide a **step-by-step description of the suggested enhancement** in as many details as possible.
+- **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you.
+- **Explain why this enhancement would be useful** to most Open Data Contract Standard users. You may also want to point out the other projects that solved it better and which could serve as inspiration.
+
+### Improving The Documentation
+Please contact [@jgperrin](https://github.com/jgperrin). Examples are always welcome.
+
+## Join The Project Team
+Please contact [@jgperrin](https://github.com/jgperrin).

examples/all/full-example.yaml docs/examples/all/full-example.yamlexamples/all/full-example.yaml renamed to docs/examples/all/full-example.yaml

@@ -9,14 +9,14 @@ uuid: 53581432-6c55-4ba2-a65f-72344a91553a
 # Lots of information
 description:
   purpose: Views built on top of the seller tables.
-  limitations: null
-  usage: null
+  limitations: Data based on seller perspective, no buyer information
+  usage: Predict sales over time
 tenant: ClimateQuantumInc
 
 # Getting support
 productDl: product-dl@ClimateQuantum.org
 productSlackChannel: '#product-help'
-productFeedbackUrl: null
+productFeedbackUrl: https://product-feedback.com/sellers
 
 # Physical parts / GCP / BigQuery specific
 sourcePlatform: googleCloudPlatform
@@ -25,14 +25,14 @@ datasetProject: edw # BQ dataset
 datasetName: access_views # BQ dataset
 
 kind: DataContract
-apiVersion: 2.3.0 # Standard version (follows semantic versioning, previously known as templateVersion)
+apiVersion: v2.2.2 # Standard version (follows semantic versioning, previously known as templateVersion)
 
 type: tables
 
 # Physical access
-driver: null
-driverVersion: null
-server: null
+driver: jdbc:postgresql
+driverVersion: 1.0.0
+server: localhost:5432
 database: pypl-edw.pp_access_views
 username: '${env.username}'
 password: '${env.password}'
@@ -42,7 +42,7 @@ schedulerAppName: name_coming_from_scheduler # NEW 2.1.0 Required if you want to
 dataset:
   - table: tbl
     physicalName: tbl_1 # NEW in v2.1.0, Optional, default value is table name + version separated by underscores, as table_1_2_0
-    priorTableName: null # if needed
+    priorTableName: seller_table # if needed
     description: Provides core payment metrics
     authoritativeDefinitions: # NEW in v2.2.0, inspired by the column-level authoritative links
       - url: https://catalog.data.gov/dataset/air-quality
@@ -59,15 +59,14 @@ dataset:
         logicalType: date
         physicalType: date
         isNullable: false
-        description: null
+        description: Reference date for transaction
         partitionStatus: true
         partitionKeyPosition: 1
         clusterStatus: false
         clusterKeyPosition: -1
         criticalDataElementStatus: false
-        tags: null
-        classification: null
-        encryptedColumnName: null
+        tags: []
+        classification: public
         transformSourceTables:
           - table_name_1
           - table_name_2
@@ -90,24 +89,23 @@ dataset:
         clusterStatus: true
         clusterKeyPosition: 1
         criticalDataElementStatus: false
-        tags: null
-        classification: null
-        encryptedColumnName: null
+        tags: []
+        classification: restricted
       - column: rcvr_cntry_code
         isPrimary: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table.
         primaryKeyPosition: -1
         businessName: receiver country code
         logicalType: string
         physicalType: varchar(2)
         isNullable: false
-        description: null
+        description: Country code
         partitionStatus: false
         partitionKeyPosition: -1
         clusterStatus: false
         clusterKeyPosition: -1
         criticalDataElementStatus: false
-        tags: null
-        classification: null
+        tags: []
+        classification: public
         authoritativeDefinitions:
           - url: https://collibra.com/asset/742b358f-71a5-4ab1-bda4-dcdba9418c25
             type: businessDefinition
@@ -162,14 +160,10 @@ stakeholders:
   - username: mhopper
     role: Data Scientist
     dateIn: 2022-10-01
-    dateOut: null
-    replacedByUsername: null
   - username: daustin
     role: Owner
     comment: Keeper of the grail
     dateIn: 2022-10-01
-    dateOut: null
-    replacedByUsername: null
 
 # Roles
 roles:

…/postgresql-adventureworks-contract.yaml …/postgresql-adventureworks-contract.yamlexamples/all/postgresql-adventureworks-contract.yaml renamed to docs/examples/all/postgresql-adventureworks-contract.yaml examples/all/postgresql-adventureworks-contract.yaml renamed to docs/examples/all/postgresql-adventureworks-contract.yaml

@@ -11,6 +11,7 @@ driver: "org.postgresql.Driver"
 description: {}
 database: "adventureworks"
 dataset:
+  tables:
   - table: "department"
     physicalName: "department"
     description: "Lookup table containing the departments within the Adventure Works\

docs/examples/fundamentals/table-column-description.yaml

@@ -0,0 +1,20 @@
+version: 1.0.0
+kind: DataContract
+uuid: 53581432-6c55-4ba2-a65f-72344a91553a
+type: tables
+status: current
+datasetName: my_table
+quantumName: my_quantum
+dataset:
+  - table: tbl
+    description: Provides core payment metrics
+    dataGranularity: Aggregation on columns txn_ref_dt, pmt_txn_id
+    columns:
+      - column: txn_ref_dt
+        businessName: Transaction reference date
+        logicalType: date
+        physicalType: date
+        description: Reference date for the transaction. Use this date in reports and aggregation rather than txn_mystical_dt, as it is slightly too mystical.
+        sampleValues:
+          - 2022-10-03
+          - 2025-01-28

docs/examples/index.md

@@ -0,0 +1,53 @@
+# Examples of Data Contracts
+
+## Executive summary
+This folder contains mainly excerpt of data contracts to illustrate specific sections & behaviors.
+
+## Table of content
+* [Full example](#full-example)
+* [Fundamentals & demographics](#fundamentals)
+* [Datasets & schema](#dataset-and-schema)
+* [Data quality](#data-quality)
+* [Pricing](#pricing)
+* [Stakeholders](#stakeholders)
+* [Roles](#roles)
+* [Service-level agreement](#service-level-agreement)
+* [Other properties](#other-properties)
+
+## Full example
+
+- [Full example](all/full-example.yaml)
+- [Postgres AdventureWorks](all/postgresql-adventureworks-contract.yaml)
+
+## Fundamentals
+
+- [Table and column](fundamentals/table-column-description.yaml)
+
+## Dataset and schema
+
+- [Table with single column](schema/table-column.yaml)
+- [Table with columns and partitioning](schema/table-columns-with-partition.yaml)
+
+## Data quality 
+
+- [Column accuracy](quality/column-accuracy.yaml)
+- [Column completeness](quality/column-completeness.yaml)
+- [Column validity](quality/column-validity.yaml)
+
+## Pricing
+This section covers pricing when you bill your customer for using this data product. Pricing is experimental in v2.2.0 of the data contract.
+
+## Stakeholders
+
+- [Stakeholders example](stakeholders/basic-four-dpo.yaml)
+
+## Roles
+
+- [Service and operational roles](roles/service-and-operational-roles.yaml)
+
+## Service-level agreement
+
+- [Database SLA](sla/database-table-sla.yaml)
+
+## Other properties
+Coming soon.