Override Fysnc Calls for HDFS #27468
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…x parallelism in dynamic inference mode This closes apache#24962.
…ile introduced by wrong operation (cherry picked from commit a6b9881)
(cherry picked from commit d0ba000)
…reaming and batch read (cherry picked from commit fc6f288)
…ify logic (cherry picked from commit 5617d62)
…resume materialized table (cherry picked from commit 24040de)
…able (cherry picked from commit 132d2fe)
…ent when RowFormat is PLAIN_TEXT (cherry picked from commit 27287a1)
This closes apache#24984 (cherry picked from commit 7fc3aac)
…rval in continuous mode
…acting method parameter names from UDF This closes apache#25019
…esumeOperation (cherry picked from commit 2efeb20)
This closes apache#24975 (cherry picked from commit 87be68a)
This closes apache#24975 (cherry picked from commit 90fc679)
…per Timer Cleanup
…en't updated after failover. (cherry picked from commit 615d197)
…g file-merging directories (apache#25031)
…f file-merging directories (apache#25052)
…le deletion before quit (apache#25067)
…XPLAIN (cherry picked from commit 5e48853)
…ith the case where the user value of the map state is null # Conflicts: # flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/sync/AbstractForStSyncState.java
… state backend temporarily
* chore: bump angular to v19 * chore: bump angular to v20
…thonEnvUtils logging (apache#26841) Co-authored-by: Rahul Sud <rahul.sud@gmail.com>
(cherry picked from commit d95f056) Co-authored-by: Gabor Somogyi <gabor_somogyi2@apple.com>
…elated logic in TaskStateAssignment
… exchange with channel state to avoid UnsupportedOperationException
…tRescaleWithMixedExchangesITCase.testRescaleFromUnalignedCheckpoint When one task has multiple inputs, and the unaligned checkpoint will be disabled for all inputs once one input exchange does not support unaligned checkpoint. It caused no inflight buffers, but UnalignedCheckpointRescaleWithMixedExchangesITCase.testRescaleFromUnalignedCheckpoint always wait for checkpoint with inflight buffers. Explicitly specifying rebalance can avoid the forward exchange.
…-merging manager (apache#26971) (cherry picked from commit ee73d0f)
…ng manager (apache#26978) (cherry picked from commit f3bd4a4)
…ing committables in batch mode (apache#27013) In apache#26433, we removed the EOI marker in the form of Long.MAX_VALUE as the checkpoint id. Since streaming pipelines can continue to checkpoint even after their respective operators have been shut down, it is not safe to use a constant as this can lead to duplicate commits. However, in batch pipelines we only have one commit on job shutdown. Using any checkpoint id should suffice in this scenario. Any pending committables should be processed by the ComitterOperator when the operator shuts down. No further checkpoints will take place. There are various connectors which rely on this behavior. I don't see any drawbacks from keeping this behavior for batch pipelines.
… objects. * [FLINK-37747][runtime] Use old subtask count for restored committable objects * trying to adapt test * Attempt to backport SinkV2ITCase from FLINK-37747 * Add more test use cases for SinkV2ITCase * Fix SinkV2ITCase test * Setup TestSinkV2WithPostCommitTopology global committer * Rename test SinkV2ITCase streaming source to be more specific --------- Co-authored-by: Eric Nascimento <ericsn2021@gmail.com> Co-authored-by: David Wang <dwang4@atlassian.com>
Collaborator
Contributor
|
This looks like you may need to pull in the latest |
davidradl
suggested changes
Jan 24, 2026
Contributor
davidradl
left a comment
davidradl
left a comment
There was a problem hiding this comment.
There is no description for this change or Jira (the title should have the Jira number) . Due to the size of this change, the motivation and proposed solution should be discussed on the dev list first, then get agreement whether a Flip is required.
github-actions
bot
added
the
community-reviewed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Override Fysnc Calls for HDFS