Skip to content

Commit 0c2a09c

Browse files
jgperrinjgperrin
authored
Merge pull request #17 from pflooky/dev-position
Dev position
2 parents b97c0c5 + d79105d commit 0c2a09c

File tree

2 files changed

+55
-7
lines changed

2 files changed

+55
-7
lines changed

‎docs/README.md‎

Lines changed: 20 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -124,13 +124,16 @@ dataset:
124124
columns:
125125
- column: txn_ref_dt
126126
isPrimary: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table.
127+
primaryKeyPosition: -1
127128
businessName: transaction reference date
128129
logicalType: date
129130
physicalType: date
130131
isNullable: false
131132
description: null
132133
partitionStatus: true
134+
partitionKeyPosition: 1
133135
clusterStatus: false
136+
clusterKeyPosition: -1
134137
criticalDataElementStatus: false
135138
tags: null
136139
classification: null
@@ -146,26 +149,32 @@ dataset:
146149
- 2020-01-28
147150
- column: rcvr_id
148151
isPrimary: true # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table.
152+
primaryKeyPosition: 1
149153
businessName: receiver id
150154
logicalType: string
151155
physicalType: varchar(18)
152156
isNullable: false
153157
description: A description for column rcvr_id.
154158
partitionStatus: false
159+
partitionKeyPosition: -1
155160
clusterStatus: true
161+
clusterKeyPosition: 1
156162
criticalDataElementStatus: false
157163
tags: null
158164
classification: null
159165
encryptedColumnName: null
160166
- column: rcvr_cntry_code
161167
isPrimary: false # NEW in v2.1.0, Optional, default value is false, indicates whether the column is primary key in the table.
168+
primaryKeyPosition: -1
162169
businessName: receiver country code
163170
logicalType: string
164171
physicalType: varchar(2)
165172
isNullable: false
166173
description: null
167174
partitionStatus: false
175+
partitionKeyPosition: -1
168176
clusterStatus: false
177+
clusterKeyPosition: -1
169178
criticalDataElementStatus: false
170179
tags: null
171180
classification: null
@@ -193,15 +202,19 @@ dataset:
193202
| dataset.table.columns | | Yes | Array. A list of columns in the table. |
194203
| dataset.table.columns.column | | Yes | The name of the column. |
195204
| dataset.table.columns.column.isPrimaryKey | | No | Boolean value specifying whether the column is primary or not. Default is false. |
196-
| dataset.table.columns.column.businessName | | No | the business name of the column. |
197-
| dataset.table.columns.column.logicalType | | Yes | the logical column datatype. |
198-
| dataset.table.columns.column.physicalType | | Yes | the physical column datatype. |
205+
| dataset.table.columns.column.primaryKeyPosition | | No | If column is a primary key, the position of the primary key column. Starts from 1. Example of `account_id, name` being primary key columns, `account_id` has primaryKeyPosition 1 and `name` primaryKeyPosition 2. Default to -1. |
206+
| dataset.table.columns.column.businessName | | No | The business name of the column. |
207+
| dataset.table.columns.column.logicalType | | Yes | The logical column datatype. |
208+
| dataset.table.columns.column.physicalType | | Yes | The physical column datatype. |
199209
| dataset.table.columns.column.description | Description | No | Description of the column. |
200-
| dataset.table.columns.column.isNullable | | No | indicates if the column may contain Null values; possible values are true and false. Default is false. |
201-
| dataset.table.columns.column.partitionStatus | | No | indicates if the column is partitioned; possible values are true and false. |
202-
| dataset.table.columns.column.clusterStatus | | No | indicates of the column is clustered; possible values are true and false. |
210+
| dataset.table.columns.column.isNullable | | No | Indicates if the column may contain Null values; possible values are true and false. Default is false. |
211+
| dataset.table.columns.column.isUnique | | No | Indicates if the column contains unique values; possible values are true and false. Default is false. |
212+
| dataset.table.columns.column.partitionStatus | | No | Indicates if the column is partitioned; possible values are true and false. |
213+
| dataset.table.columns.column.partitionKeyPosition | | No | If column is used for partitioning, the position of the partition column. Starts from 1. Example of `country, year` being partition columns, `country` has partitionKeyPosition 1 and `year` partitionKeyPosition 2. Default to -1. |
214+
| dataset.table.columns.column.clusterStatus | | No | Indicates of the column is clustered; possible values are true and false. |
215+
| dataset.table.columns.column.clusterKeyPosition | | No | If column is used for clustering, the position of the cluster column. Starts from 1. Example of `year, date` being cluster columns, `year` has clusterKeyPosition 1 and `date` clusterKeyPosition 2. Default to -1. |
203216
| dataset.table.columns.column.classification | | No | Can be anything, like confidential, restricted, and public to more advanced categorization. Some companies like PayPal, use data classification indicating the class of data in the column; expected values are 1, 2, 3, 4, or 5. |
204-
| dataset.table.columns.column.authoritativeDefinitions | | No | list of links to sources that provide more detail on column logic or values; examples would be URL to a GitHub repo, Collibra, on another tool. |
217+
| dataset.table.columns.column.authoritativeDefinitions | | No | List of links to sources that provide more detail on column logic or values; examples would be URL to a GitHub repo, Collibra, on another tool. |
205218
| dataset.table.columns.column.encryptedColumnName | | No | The column name within the table that contains the encrypted column value. For example, unencrypted column `email_address` might have an encryptedColumnName of `email_address_encrypt`. |
206219
| dataset.table.columns.column.transformSourceTables | | No | List of sources used in column transformation. |
207220
| dataset.table.columns.column.transformLogic | | No | Logic used in the column transformation. |

‎examples/schema/table-columns-with-partition.yaml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
- table: tbl
2+
columns:
3+
- column: rcvr_cntry_code
4+
businessName: Receiver country code
5+
logicalType: string
6+
physicalType: varchar(2)
7+
isPrimaryKey: true
8+
primaryKeyPosition: 1
9+
partitionStatus: true
10+
partitionKeyPosition: 1
11+
clusterStatus: false
12+
- column: rcvr_id
13+
businessName: Receiver identification number
14+
logicalType: string
15+
physicalType: varchar(20)
16+
isPrimaryKey: true
17+
primaryKeyPosition: 2
18+
partitionStatus: false
19+
clusterStatus: false
20+
- column: year
21+
businessName: Year of transaction
22+
logicalType: integer
23+
physicalType: int
24+
isPrimaryKey: false
25+
partitionStatus: true
26+
partitionKeyPosition: 2
27+
clusterStatus: true
28+
clusterKeyPosition: 1
29+
- column: amount
30+
businessName: Transaction amount
31+
logicalType: double
32+
physicalType: double
33+
isPrimaryKey: false
34+
partitionStatus: false
35+
clusterStatus: false

0 commit comments

Comments
 (0)