Skip to content

Commit 58030c2

Browse files
asuncepedaasuncepeda
committed
readme and ci/cd files
1 parent 11c36bd commit 58030c2

File tree

7 files changed

+111
-2
lines changed

7 files changed

+111
-2
lines changed

‎.github/workflows/iterate_snowflake_cd.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
##################################################
2+
### Visit https://github.com/tinybirdco/ci ###
3+
### for more details or custom CI/CD ###
4+
##################################################
5+
6+
name: Tinybird - CD Workflow
7+
8+
on:
9+
workflow_dispatch:
10+
push:
11+
paths:
12+
- 'iterate_snowflake/**'
13+
branches:
14+
- main
15+
jobs:
16+
cd:
17+
uses: tinybirdco/ci/.github/workflows/cd.yml@v4.1.0
18+
with:
19+
data_project_dir: ./add_column_snowflake_ds
20+
secrets:
21+
tb_admin_token: ${{ secrets.TB_ADMIN_TOKEN_ADD_COLUMN_SNOWFLAKE_DS }}
22+
tb_host: https://api.tinybird.co

‎.github/workflows/iterate_snowflake_ci.yml‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
##################################################
2+
### Visit https://github.com/tinybirdco/ci ###
3+
### for more details or custom CI/CD ###
4+
##################################################
5+
6+
name: Tinybird - CI Workflow
7+
8+
on:
9+
workflow_dispatch:
10+
pull_request:
11+
paths:
12+
- 'iterate_snowflake/**'
13+
branches:
14+
- main
15+
types: [opened, reopened, labeled, unlabeled, synchronize, closed]
16+
17+
concurrency: ${{ github.workflow }}-${{ github.event.pull_request.number }}
18+
19+
jobs:
20+
ci:
21+
uses: tinybirdco/ci/.github/workflows/ci.yml@v4.1.0
22+
with:
23+
data_project_dir: ./add_column_snowflake_ds
24+
secrets:
25+
tb_admin_token: ${{ secrets.TB_ADMIN_TOKEN_ADD_COLUMN_SNOWFLAKE_DS }}
26+
tb_host: https://api.tinybird.co

‎DEVELOPMENT/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
1. Create a new workspace in Tinybird using the UI and choose to use the `Web Analytics` Template. This will generate a new Workspace and push all the resources from the Web Analytics template
44
2. Create a repository in Github (private or public)
5-
3. Connect the workspace with the repository using the UI. Follow the steps and push all the changes to the repo
5+
3. Connect the workspace with the repository using the [documentation about versions](https://www.tinybird.co/docs/production/working-with-version-control#connect-your-workspace-to-git-from-the-cli)
66
4. At this point, you should have a workspace connected to Git and using the release 0.0.0
77
5. (Optional) You can use `mockingbird` to start fake data to our workspace. Just go to https://mockingbird.tinybird.co/ and follow the steps to start sending data to your workspace
8-
6. (Optional) You can run the script `utils/query_api.sh` to generate fake requests to the endpoints of the workspace
8+
6. (Optional) You can run the script `utils/query_api.sh` to generate fake requests to the endpoints of the workspace

‎iterate_snowflake/.tinyenv‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
VERSION=0.0.2

‎iterate_snowflake/README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Use cases for iteration of Snowflake Data Sources
2+
3+
First of all, you must connect your Tinybird workspace with your Git repository using the [documentation about versions](https://www.tinybird.co/docs/production/working-with-version-control#connect-your-workspace-to-git-from-the-cli)
4+
5+
### Create a Snowflake Data Source
6+
7+
- Create the Snowflake Connection in your main tinybird workspace using Tinybird CLI, as explain in the [iteration documentation](https://www.tinybird.co/docs/ingest/snowflake). Snowflake connections reside always in the main branch.
8+
- Create a new git branch
9+
- Create a new Snowflake .datasource data file in the branch, as in the example PR. You shouldn't need fixtures as the Data Source will be populated from the connection, but you can add fixtures and/or tests if needed.
10+
- For information abount Snowflake options read the [connector documentation](https://www.tinybird.co/docs/ingest/snowflake)
11+
- Commit an create a new PR/MR from the branch to `main`
12+
- Wait for CI. You can check in the temporary workspace branch (automatically created and named like `tmp_ci_*`) that everything works as expected.
13+
- Merge and wait for CD. Now you should have your Data Source in the main workspace, using the connection, and the temporary CI branch will be deleted.
14+
15+
[Example Pull Request](https://github.com/tinybirdco/use-case-examples/pull/tbd)
16+
17+
18+
### Update Snowflake Data Source with a new column, with same connection
19+
20+
- Create another git branch
21+
- Add the new column to the schema and query
22+
- Commit and create an ew PR from the branch to `main`
23+
- Wait for CI to finish. You can check in the temporary workspace branch (automatically created and named like `tmp_ci_*`) that everything works as expected.
24+
- Merge and wait for CD. Now you should have your Data Source updated in the main workspace, using the connection, and the temporary CI branch will be deleted.
25+
26+
[Example Pull Request](https://github.com/tinybirdco/use-case-examples/pull/tbd)
27+
28+
### 4th Delete a Snowflake Data Source
29+
30+
- Create another git branch
31+
- Remove the .datasource data file
32+
- Commit and create an ew PR from the branch to `main`
33+
- Wait for CI to finish. You can check in the temporary workspace branch (automatically created and named like `tmp_ci_*`) that everything works as expected.
34+
- Merge and wait for CD. Now you should have your Data Source updated in the main workspace, using the connection, and the temporary CI branch will be deleted.
35+

‎iterate_snowflake/requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
tinybird-cli==5.7.0

‎iterate_snowflake/scripts/exec_test.sh‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
2+
#!/usr/bin/env bash
3+
4+
fail=0;
5+
6+
# We set this environment variable to avoid the CLI to show the warning about a new version
7+
export TB_VERSION_WARNING=0
8+
9+
for t in `find ./tests -name "*.test"`; do
10+
echo "** Running $t **"
11+
echo "** $(cat $t)"
12+
if res=$(bash $t $1 | diff -B ${t}.result -); then
13+
echo 'OK';
14+
else
15+
echo "failed, diff:";
16+
echo "$res";
17+
fail=1
18+
fi
19+
echo ""
20+
done;
21+
22+
if [ $fail == 1 ]; then
23+
exit -1;
24+
fi

0 commit comments

Comments
 (0)