Skip to content

Conversation

@m-abulazm
Copy link
Contributor

@m-abulazm m-abulazm commented Nov 13, 2025

Changes

What does this PR do?

Relevant implementation details

  • Support username and password in spark options instead of jdbc url
  • This makes sure non-url compliant chars in the password are supported

@m-abulazm m-abulazm self-assigned this Nov 13, 2025
@m-abulazm m-abulazm requested a review from a team as a code owner November 13, 2025 15:23
@codecov
Copy link

codecov bot commented Nov 13, 2025

Codecov Report

❌ Patch coverage is 95.45455% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 65.14%. Comparing base (660a23d) to head (53df4bf).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...ricks/labs/lakebridge/reconcile/connectors/tsql.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2146      +/-   ##
==========================================
+ Coverage   65.10%   65.14%   +0.04%     
==========================================
  Files         100      100              
  Lines        8493     8503      +10     
  Branches      884      885       +1     
==========================================
+ Hits         5529     5539      +10     
  Misses       2774     2774              
  Partials      190      190              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

✅ 51/51 passed, 11 flaky, 3m50s total

Flaky tests:

  • 🤪 test_validate_mixed_checks (172ms)
  • 🤪 test_validate_non_empty_tables (8ms)
  • 🤪 test_validate_invalid_source_tech (172ms)
  • 🤪 test_validate_invalid_schema_path (1ms)
  • 🤪 test_validate_table_not_found (1ms)
  • 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (20.391s)
  • 🤪 test_transpiles_informatica_to_sparksql (20.749s)
  • 🤪 test_transpile_teradata_sql (21.951s)
  • 🤪 test_transpile_teradata_sql_non_interactive[True] (24.449s)
  • 🤪 test_transpile_teradata_sql_non_interactive[False] (5.845s)
  • 🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (11.503s)

Running from acceptance #3122

@m-abulazm m-abulazm enabled auto-merge November 19, 2025 10:04
@asnare asnare changed the title Handle jdbc passwords containing special chars Handle JDBC passwords containing special chars Nov 20, 2025
Copy link
Collaborator

@gueniai gueniai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

connector = TSQLServerDataSourceUnderTest(mock_spark, mock_ws)

columns = connector.get_schema("labs_azure_sandbox_remorph", "dbo", "Employees")
columns = connector.get_schema("labs_azure_sandbox_remorph", "dbo", "reconcile_in")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this come from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manually created by me.

}

def reader(self, query: str) -> DataFrameReader:
def reader(self, query: str, options: dict | None = None) -> DataFrameReader:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def reader(self, query: str, options: dict | None = None) -> DataFrameReader:
def reader(self, query: str, options: Mapping[str, OptionalPrimitiveType] | None = None) -> DataFrameReader:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cant import OptionalPrimitiveType so using object

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try:

from pyspark.sql.readwriter import OptionalPrimitiveType

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this cannot be imported as well. I copied it so we can use it.

Copy link
Collaborator

@sundarshankar89 sundarshankar89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@m-abulazm m-abulazm added this pull request to the merge queue Dec 8, 2025
Merged via the queue into main with commit 0349ea8 Dec 8, 2025
10 checks passed
@m-abulazm m-abulazm deleted the fix/recon/jdbc_password branch December 8, 2025 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants