Skip to content

Commit c0c90d6

Browse files
pflookypflooky
committed
Fix links when copying markdown files into docs folder
1 parent ef07ba9 commit c0c90d6

File tree

6 files changed

+306
-7
lines changed

6 files changed

+306
-7
lines changed

‎.github/workflows/docs-site-deploy.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
mkdocs-material-
3030
- run: pip install mkdocs-open-in-new-tab mike
3131
- run: pip install mkdocs-material
32-
- run: cp README.md docs/index.md
32+
- run: cat README.md | sed 's/docs\///g' | sed 's/CONTRIBUTING.md/contributing.md/g' > docs/index.md
33+
- run: cat CHANGELOG.md | sed 's/docs\///g' > docs/changelog.md
3334
- run: cp CONTRIBUTING.md docs/contributing.md
34-
- run: cp CHANGELOG.md docs/changelog.md
3535
- run: mike deploy --push --update-aliases ${{ github.ref_name }} latest

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ This document tracks the history and evolution of the **Open Data Contract Stand
1616
* Reformat quality examples to be valid YAML.
1717
* Type of definition for authority have standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.
1818
* Add in `isUnique`, `primaryKeyPosition`, `partitionKeyPosition`, and `clusterKeyPosition` to `column` definition.
19-
* Add [JSON schema](schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
19+
* Add [JSON schema](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
2020
* Integrated as part of [Bitol](https://lfaidata.foundation/projects/bitol/).
2121
* Reformat Markdown tables.
2222

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Thanks for your interest and for taking the time to come here! ❤️
1010
This standard describes a structure for a **data contract**. It's current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
1111

1212
## Discover the open standard
13-
Discover the [Open Data Contract Standard](docs/index.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
13+
Discover the [Open Data Contract Standard](docs/standard.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
1414

1515
## What is a Data Contract?
1616

@@ -23,13 +23,13 @@ A data contract defines the agreement between a data producer and consumers. A d
2323
* Security & stakeholders.
2424
* Custom properties.
2525

26-
![Data contract schema](./docs/img/data-contract-v2.2.1-schema.svg "Data contract schema")
26+
![Data contract schema](docs/img/data-contract-v2.2.1-schema.svg "Data contract schema")
2727

2828
*Figure 1: illustration of a data contract, its principal contributors, sections, and usage.*
2929

3030
### JSON Schema
3131

32-
JSON Schema for ODCS can be found [here](schema/odcs-json-schema.json). You can import this schema into your IDE for
32+
JSON Schema for ODCS can be found [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json). You can import this schema into your IDE for
3333
validation of your YAML files. Links below show how you can import the schema:
3434

3535
- [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom)
@@ -82,7 +82,7 @@ mike set-default --push latest #by default, users will go to
8282
```
8383

8484
#### Deploying a new version
85-
Given that the Github action [here](.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
85+
Given that the Github action [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
8686
created, all that is required is to:
8787
1. [Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
8888
2. Put in new tag version for release (follows pattern v*)

‎docs/changelog.md‎

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,117 @@
1+
This document tracks the history and evolution of the **Open Data Contract Standard**.
2+
3+
# v2.2.2 - 2024-01-05 - OPEN
4+
5+
* Change `dataset.description` data type from `array` to `string`
6+
* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`
7+
* Change `price.priceAmount` data type from `string` to `number`
8+
* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`
9+
* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`
10+
* Update [examples](examples) to adhere to JSON schema
11+
* Full example from README directs to [full-example.yaml](examples/all/full-example.yaml)
12+
* Add in mkdocs for creating documentation website
13+
14+
# v2.2.1 - 2023-12-18 - APPROVED
15+
16+
* Reformat quality examples to be valid YAML.
17+
* Type of definition for authority have standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.
18+
* Add in `isUnique`, `primaryKeyPosition`, `partitionKeyPosition`, and `clusterKeyPosition` to `column` definition.
19+
* Add [JSON schema](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
20+
* Integrated as part of [Bitol](https://lfaidata.foundation/projects/bitol/).
21+
* Reformat Markdown tables.
22+
23+
# v2.2.0 - 2023-07-27 - APPROVED
24+
25+
* New name to Open Data Contract Standard.
26+
* `templateName` is now called `standardVersion`, v2.2.0 parsers should account for this change and support both to avoid a breaking change.
27+
* Added support for `authoritativeDefinitions` at the table level.
28+
* Added many examples.
29+
* Various improvements and typo corrections.
30+
* Finalization of fork under AIDA User Group.
31+
32+
# v2.1.1 - 2023-04-26 - APPROVED
33+
34+
* Open source version.
35+
* Additional value field `valueExt` in SLA.
36+
37+
# v2.1.0 - 2023-03-23 - APPROVED
38+
39+
## Data Quality
40+
The data contract adds elements specifically for interfacing with the Data Quality tooling.
41+
42+
Additions:
43+
* quality (table level & column level check):
44+
* templateName (called standardVersion since v2.2.0)
45+
* dimension
46+
* type
47+
* severity
48+
* businessImpact
49+
* scheduleCronExpression
50+
* customProperties
51+
* columns
52+
* isPrimaryKey
53+
54+
## Physical names
55+
The data contract is a logical construct; we add more specific links to the physical world.
56+
57+
## Service-level agreement
58+
The service-level agreements not previously used are more detailed to follow the DP QoS pattern. See SLA.
59+
60+
## Other
61+
Removed the weight for system ratings from the data contract. Their default values remain.
62+
63+
# v2.0.0 - SUPERSEED BY V2.1.0
64+
65+
## Guidelines & Evolution
66+
* [Type case](https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format)
67+
* Support for SemVer versioning.
68+
* Tags can have values.
69+
70+
## Additions
71+
* Version of contract definition: v2.0.0. A breaking change with v1.
72+
* Description:
73+
* Purpose (text field).
74+
* Limitations (text field).
75+
* Usage (text field).
76+
* Domain.
77+
* Dictionary section:
78+
* Identification of masked column (encryptedColumnName property), example: the email_decrypted column would be masked by email_encrypted.
79+
* Flag for critical data element.
80+
* Added keys for transformation data (sources, logic, description).
81+
* Sample values.
82+
* Ability to specify links to authoritative sources at the column level (authoritativeDefinitions).
83+
* Business name.
84+
* List of stakeholders:
85+
* Username (user account).
86+
* Role.
87+
* Date in.
88+
* Date out.
89+
* Replaced by.
90+
* Service levels: agreements & objective [orginal inspiration](https://medium.com/@jgperrin/meet-cactar-the-mongolian-warlord-of-data-quality-d7bdbd6a5398).
91+
* Price / cost.
92+
* Name changes to match PPaaS type case.
93+
* Product data:
94+
* productDl.
95+
* productSlackChannel.
96+
* productFeedbackUrl.
97+
* Renamed `tables` key to `dataset`.
98+
* Removed `owner` key. Owner is now a stakeholder role.
99+
* Additional quality keys:
100+
* description.
101+
* toolName.
102+
* toolRuleName.
103+
* Custom Properties.
104+
* Product dates:
105+
* generalAvailabilityDate.
106+
* endOfSupportDate.
107+
* endOfLifeDate.
108+
109+
# v1 - DEPRECATED
110+
* Description of the data quantum/data artifact.
111+
* Roles.
112+
* Schema:
113+
* Tables, columns.
114+
* Data quality.
115+
* System rating weightage.
116+
* Ratings:
117+
* System, user, etc.

‎docs/contributing.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# Open Data Contract Standard
2+
3+
## Executive summary
4+
First off, thanks for taking the time to contribute! ❤️
5+
6+
All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉
7+
8+
You do not have to be a member of AIDA User Group to contribute, although becoming a member is free. Strength is always in the number. Check [it out](https://aidausergroup.org/join/).
9+
10+
> And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:
11+
> - Star the project.
12+
> - Tweet about it.
13+
> - Refer this project in your project's readme.
14+
> - Mention the project at local meetups and tell your friends/colleagues.
15+
16+
<!-- omit in toc -->
17+
## Table of Contents
18+
19+
- [Code of Conduct](#code-of-conduct)
20+
- [I Have a Question](#i-have-a-question)
21+
- [I Want To Contribute](#i-want-to-contribute)
22+
- [Suggesting Enhancements](#suggesting-enhancements)
23+
- [Improving The Documentation](#improving-the-documentation)
24+
- [Join The Project Team](#join-the-project-team)
25+
26+
27+
## Code of Conduct
28+
29+
This project and everyone participating in it is governed by the
30+
[Open Data Contract Standard Code of Conduct](blob/master/CODE_OF_CONDUCT.md).
31+
By participating, you are expected to uphold this code. Please report unacceptable behavior
32+
to [@jgperrin](https://github.com/jgperrin).
33+
34+
35+
## I Have a Question
36+
37+
** New **
38+
39+
AIDA User Group also opened its Slack for Data Contract discussion. It is an alternate way of contributing to this project. The Slack channel is now [available](https://aidaug.slack.com/archives/C05UZRSBKLY).
40+
41+
You have to be a member of AIDA User Group (it's free) to have access to our Slack channel. All the details are [here](https://aidausergroup.org/welcome/).
42+
43+
> If you want to ask a question, we assume that you have read the available [Documentation](https://github.com/AIDAUserGroup/open-data-contract-standard).
44+
45+
Before you ask a question, it is best to search for existing [Issues](https://github.com/bitol-io/open-data-contract-standard/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first.
46+
47+
If you then still feel the need to ask a question and need clarification, we recommend the following:
48+
49+
- Open a [New Issue](https://github.com/bitol-io/open-data-contract-standard/issues/new).
50+
- Provide as much context as you can about what you're running into.
51+
52+
We will then take care of the issue as soon as possible.
53+
54+
## I Want To Contribute
55+
56+
> ### Legal Notice <!-- omit in toc -->
57+
> When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license.
58+
59+
### Suggesting Enhancements
60+
61+
This section guides you through submitting an enhancement suggestion for Data Contract Template, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions.
62+
63+
<!-- omit in toc -->
64+
#### Before Submitting an Enhancement
65+
66+
- Make sure that you are using the latest version.
67+
- Read the [documentation](https://github.com/AIDAUserGroup/open-data-contract-standard) carefully and find out if the functionality is already covered.
68+
- Perform a [search](https://github.com/bitol-io/open-data-contract-standard/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one.
69+
- Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset.
70+
71+
<!-- omit in toc -->
72+
#### How Do I Submit a Good Enhancement Suggestion?
73+
74+
Enhancement suggestions are tracked as [GitHub issues](https://github.com/bitol-io/open-data-contract-standard/issues).
75+
76+
- Use a **clear and descriptive title** for the issue to identify the suggestion.
77+
- Provide a **step-by-step description of the suggested enhancement** in as many details as possible.
78+
- **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you.
79+
- **Explain why this enhancement would be useful** to most Open Data Contract Standard users. You may also want to point out the other projects that solved it better and which could serve as inspiration.
80+
81+
### Improving The Documentation
82+
Please contact [@jgperrin](https://github.com/jgperrin). Examples are always welcome.
83+
84+
## Join The Project Team
85+
Please contact [@jgperrin](https://github.com/jgperrin).

‎docs/index.md‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8149/badge)](https://www.bestpractices.dev/projects/8149)
2+
3+
# Open Data Contract Standard (ODCS)
4+
5+
Welcome!
6+
7+
Thanks for your interest and for taking the time to come here! ❤️
8+
9+
## Executive summary
10+
This standard describes a structure for a **data contract**. It's current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
11+
12+
## Discover the open standard
13+
Discover the [Open Data Contract Standard](standard.md). This file contains some explanations and several examples. More [examples](examples/index.md) can be found here.
14+
15+
## What is a Data Contract?
16+
17+
### The basics of a data contract
18+
A data contract defines the agreement between a data producer and consumers. A data contract contains several sections:
19+
* Fundamentals.
20+
* Schema.
21+
* Data quality.
22+
* Service-level agreement (SLA).
23+
* Security & stakeholders.
24+
* Custom properties.
25+
26+
![Data contract schema](img/data-contract-v2.2.1-schema.svg "Data contract schema")
27+
28+
*Figure 1: illustration of a data contract, its principal contributors, sections, and usage.*
29+
30+
### JSON Schema
31+
32+
JSON Schema for ODCS can be found [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json). You can import this schema into your IDE for
33+
validation of your YAML files. Links below show how you can import the schema:
34+
35+
- [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom)
36+
- [VS Code](https://code.visualstudio.com/languages/json#_json-schemas-and-settings)
37+
38+
## Contributing to the project
39+
Check out the [CONTRIBUTING](./contributing.md) file.
40+
41+
## Articles
42+
* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879)
43+
* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148)
44+
* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
45+
* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
46+
* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d)
47+
* 2023-09-10 - [Data Contracts 101](https://medium.com/p/568a9adbf9a9)
48+
* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/)
49+
* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/)
50+
* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130)
51+
* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1)
52+
* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3)
53+
* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/)
54+
55+
If you spot an article about the Open Data Contract Standard, make a pull request!
56+
57+
## Vendors
58+
59+
Vendors who natively support the Open Data Contract Standard.
60+
61+
A non-exhaustive list of organizations offering solutions natively compatible with ODCS, such as data catalogs, and data quality platforms.
62+
63+
* [DQC.ai | DQ PLATFORM](https://www.dqc.ai/dqc-platform) - [Enhancing Data Quality with ODCS: A Standard Ensured by the DQ Platform
64+
](https://www.dqc.ai/post/enhancing-data-quality-with-odcs-a-standard-ensured-by-the-dq-platform)
65+
66+
## More
67+
68+
### History
69+
Formerly known as the data contract template, this standard is used to implement Data Mesh at [PayPal](https://about.pypl.com/). Starting with v2.2.0, it is maintained by a 501c6 non-profit organization called [AIDA User Group (Artificial Intelligence, Data, and Analytics User Group)](https://aidaug.org). On November 30th, 2023, [AIDA User Group](https://aidaug.org) and the [Linux Foundation AI & Data](https://lfaidata.foundation/) joined forces to create [Bitol](https://bitol.io). Bitol englobes ODCS and future standards & tools.
70+
71+
### How does PayPal use Data Contracts?
72+
PayPal uses data contracts in many ways, but this [article](https://medium.com/paypal-tech/the-next-generation-of-data-platforms-is-the-data-mesh-b7df4b825522) from the [PayPal Technology blog](https://medium.com/paypal-tech) gives a good introduction.
73+
74+
### Mkdocs mike versioning
75+
To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
76+
77+
```bash
78+
pip install mike
79+
cd open-data-contract-standard #ensure you are inside the repo
80+
mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
81+
mike set-default --push latest #by default, users will go to latest
82+
```
83+
84+
#### Deploying a new version
85+
Given that the Github action [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
86+
created, all that is required is to:
87+
1. [Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
88+
2. Put in new tag version for release (follows pattern v*)
89+
3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
90+
91+
#### Delete version
92+
If a version tag was pushed that was incorrect, it can be deleted via:
93+
94+
```bash
95+
mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
96+
mike delete <incorrect tag> --push
97+
```

0 commit comments

Comments
 (0)