Skip to content

Conversation

@sandyfog
Copy link

@sandyfog sandyfog commented Dec 19, 2025

Purpose

In streaming pipelines, a common Flink SQL pattern is to filter rows before they reach the Paimon sink:

INSERT INTO paimon_table
SELECT * FROM upstream WHERE delete = 0;

When a row flips from delete = 0 to delete = 1, Flink first emits an UPDATE_BEFORE (-U) for the previously qualified row.
The succeeding UPDATE_AFTER (+U) is dropped by the filter, so only the -U record arrives at Paimon.
The current PartialUpdateMergeFunction does not treat this lone -U as a deletion.
Consequently, the stale row remains in the table, breaking correctness.

Linked issue: close #6862

Tests

API and Format

Documentation

@sandyfog
Copy link
Author

@JingsongLi PTAL, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Partial-update merge-function keeps stale row when the last message is a lone UPDATE_BEFORE (-U)

1 participant