Skip to content

Commit 01fb13f

Browse files
committed
add new labs
1 parent 5f67e2f commit 01fb13f

File tree

1 file changed

+50
-6
lines changed

1 file changed

+50
-6
lines changed

README.md

Lines changed: 50 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -365,7 +365,7 @@ This is a widely-used, powerful open-source stack for monitoring and alerting.
365365

366366
<details>
367367

368-
<summary><b style="font-size: 1.2em;">🧪 Factor House Local Labs 🧪</b></summary>
368+
<summary><b style="font-size: 1.2em;">Factor House Local Labs</b></summary>
369369

370370
<br>
371371

@@ -375,7 +375,7 @@ Each lab is designed to be modular, hands-on, and production-inspired, helping y
375375

376376
<div align="center">
377377
<a href="https://github.com/factorhouse/examples/tree/main/fh-local-labs" target="_blank" rel="noopener noreferrer">
378-
<img src="./images/fh-local-labs.png" alt="Factor House Local Labs" width="600"/>
378+
<img src="https://raw.githubusercontent.com/factorhouse/examples/refs/heads/main/images/fh-local-labs.png" alt="Factor House Local Labs" width="600"/>
379379
</a>
380380

381381
<br>
@@ -388,15 +388,15 @@ Each lab is designed to be modular, hands-on, and production-inspired, helping y
388388

389389
<details>
390390

391-
<summary><b style="font-size: 1.2em;">🎮 Mobile Game Top K Analytics 🎮</b></summary>
391+
<summary><b style="font-size: 1.2em;">Mobile Game Top K Analytics</b></summary>
392392

393393
<br>
394394

395395
This project walks through how to build a complete real-time analytics pipeline for a mobile game using a modern data stack. It simulates live gameplay data, processes it in real time to calculate performance metrics, and displays the results on an interactive dashboard.
396396

397397
<div align="center">
398398
<a href="https://github.com/factorhouse/examples/tree/main/projects/mobile-game-top-k-analytics" target="_blank" rel="noopener noreferrer">
399-
<img src="./images/mobile-game-top-k-analytics.gif" alt="Mobile Game Top K Analytics" width="600"/>
399+
<img src="https://raw.githubusercontent.com/factorhouse/examples/refs/heads/main/projects/mobile-game-top-k-analytics/images/mobile-game-top-k-analytics.gif" alt="Mobile Game Top K Analytics" width="600"/>
400400
</a>
401401

402402
<br>
@@ -409,15 +409,15 @@ This project walks through how to build a complete real-time analytics pipeline
409409

410410
<details>
411411

412-
<summary><b style="font-size: 1.2em;">🔄 CDC with Debezium on Real-Time theLook eCommerce Data 🗄️</b></summary>
412+
<summary><b style="font-size: 1.2em;">CDC with Debezium on Real-Time theLook eCommerce Data</b></summary>
413413

414414
<br>
415415

416416
This project unlocks the power of the popular [theLook eCommerce dataset](https://console.cloud.google.com/marketplace/product/bigquery-public-data/thelook-ecommerce) for modern event-driven applications. It uses a re-engineered [real-time data generator](https://github.com/factorhouse/examples/tree/main/datagen/thelook-ecomm) that transforms the original static dataset into a continuous stream of simulated user activity, writing directly to a PostgreSQL database.
417417

418418
<div align="center">
419419
<a href="https://github.com/factorhouse/examples/tree/main/projects/thelook-ecomm-cdc" target="_blank" rel="noopener noreferrer">
420-
<img src="./images/thelook-datagen.gif" alt="CDC with Debezium on Real-Time theLook eCommerce Data" width="600"/>
420+
<img src="https://raw.githubusercontent.com/factorhouse/examples/refs/heads/main/datagen/thelook-ecomm/images/thelook-datagen.gif" alt="CDC with Debezium on Real-Time theLook eCommerce Data" width="600"/>
421421
</a>
422422

423423
<br>
@@ -428,6 +428,50 @@ This project unlocks the power of the popular [theLook eCommerce dataset](https:
428428

429429
</details>
430430

431+
<details>
432+
433+
<summary><b style="font-size: 1.2em;">A Practical Guide to Data Lineage on Kafka Connect with OpenLineage</b></summary>
434+
435+
<br>
436+
437+
The lab demonstrates how to capture real-time data lineage from Kafka Connect using a custom Single Message Transform (SMT) - `OpenLineageLifecycleSmt`. It builds a complete pipeline that tracks data from a source connector to S3 and Iceberg sinks, with the full lineage graph visualized in Marquez.
438+
439+
<div align="center">
440+
<a href="https://github.com/factorhouse/examples/blob/main/projects/data-lineage-labs/lab1_kafka-connect.md" target="_blank" rel="noopener noreferrer">
441+
<img src="https://raw.githubusercontent.com/factorhouse/examples/refs/heads/main/projects/data-lineage-labs/images/connector-lineage.gif" alt="A Practical Guide to Data Lineage on Kafka Connect with OpenLineage" width="600"/>
442+
</a>
443+
444+
<br/>
445+
446+
[**➡️ Click Here to Explore the Lab**](https://github.com/factorhouse/examples/blob/main/projects/data-lineage-labs/lab1_kafka-connect.md)
447+
448+
</div>
449+
450+
</details>
451+
452+
<details>
453+
454+
<summary><b style="font-size: 1.2em;">End-to-End Data Lineage from Kafka to Flink and Spark</b></summary>
455+
456+
<br>
457+
458+
An end-to-end tutorial for establishing data lineage across Kafka, Flink, Spark, and Iceberg. This lab begins by tracking data from a single Kafka topic through parallel pipelines: a Kafka S3 sink connector for raw data archival, a Flink job for real-time analytics, another Flink job for Iceberg ingestion, and a downstream Spark batch job that reads from the Iceberg table.
459+
460+
<div align="center">
461+
<a href="https://github.com/factorhouse/examples/blob/main/projects/data-lineage-labs/lab2_end-to-end.md" target="_blank" rel="noopener noreferrer">
462+
<img src="https://raw.githubusercontent.com/factorhouse/examples/refs/heads/main/projects/data-lineage-labs/images/end-to-end-lineage.gif" alt="End-to-End Data Lineage from Kafka to Flink and Spark" width="600"/>
463+
</a>
464+
465+
<br/>
466+
467+
[**➡️ Click Here to Explore the Lab**](https://github.com/factorhouse/examples/blob/main/projects/data-lineage-labs/lab2_end-to-end.md)
468+
469+
</div>
470+
471+
</details>
472+
473+
<br/>
474+
431475
_Stay tuned—more labs and projects are on the way!_
432476

433477
## Prerequisites

0 commit comments

Comments
 (0)