Updated dashboards deployment to use standard lakeview dashboard definitions

[AdityaMandiwal]

Changes

• Updated dashboards deployment to use standard lakeview dashboard definitions. Removed lsql dependency.
• Improved telemetry by logging unique hashed inputs

Linked issues

Resolves #132

Tests

• manually tested
• added unit tests
• added integration tests
• added end-to-end tests
• added performance tests

[github-actions]

✅ 445/445 passed, 3 flaky, 40 skipped, 5h25m32s total

Flaky tests:

• 🤪 test_e2e_workflow_for_patterns_exclude_patterns (7m9.542s)
• 🤪 test_e2e_workflow_for_patterns_exclude_output (4m30.647s)
• 🤪 test_e2e_workflow_for_patterns_table_checks_storage (4m27.178s)

_{Running from acceptance #3315}

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.20%. Comparing base (d200468) to head (08ebb6e).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...tabricks/labs/dqx/installer/dashboard_installer.py	90.14%	7 Missing ⚠️
src/databricks/labs/dqx/telemetry.py	97.87%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #950      +/-   ##
==========================================
+ Coverage   90.09%   90.20%   +0.11%     
==========================================
  Files          64       64              
  Lines        6138     6188      +50     
==========================================
+ Hits         5530     5582      +52     
+ Misses        608      606       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

This PR migrates dashboard configuration from a YAML-based format using the lsql library to a native JSON-based Lakeview dashboard definition format. The change standardizes dashboard creation to use Databricks SDK's native Lakeview API directly.

Key Changes:

• Replaced YAML dashboard configuration with JSON format
• Implemented custom DashboardMetadata class to replace the lsql library dependency
• Refactored DashboardInstaller to use native Lakeview SDK methods for dashboard creation and publishing

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
src/databricks/labs/dqx/queries/quality/dashboard/dashboard.yml	Removed legacy YAML dashboard configuration
src/databricks/labs/dqx/queries/quality/dashboard/dashboard.json	Added new JSON-based dashboard configuration with native Lakeview format
src/databricks/labs/dqx/installer/dashboard_installer.py	Replaced lsql library with custom implementation using native Databricks SDK Lakeview API

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.

[mwojtyczka]

Pls also remove databricks-labs-lsql from the dependencies in pyproject.toml

[AdityaMandiwal]

Pls also remove databricks-labs-lsql from the dependencies in pyproject.toml

Done

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.