Skip to content

Core: HadoopFileIO to take list of filesystem schemas to enable trash for #15111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
steveloughran wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Core: HadoopFileIO to take list of filesystem schemas to enable trash for #15111

steveloughran wants to merge 7 commits into from
+260 −22

Conversation

@steveloughran
Copy link
Contributor

@steveloughran steveloughran commented Jan 22, 2026

Addresses #15093

  • New (documented) option iceberg.hadoop.delete-trash-schemas
  • Default value is "hdfs, viewfs"
  • Uses Trash.moveToAppropriateTrash() to resolve the target FS, and will query hdfs for the
    active trash policy.
  • Restores behavior that delete(missing-path) is a no-op; trash operations raise FNFE here
  • Test changes as appropriate

@steveloughran
New configuration option iceberg.hadoop.delete-trash-schemas
5456b3b 
* declares whether to look @ trash policy before delete
* defaults are currently "hdfs" and "viewfs", though I'm tempted to make the
  default "".
@steveloughran
Core: HadoopFileIO to take list of filesystem schemas to enable trash…
fe62659 
… for

Addresses apache#15093
@steveloughran
Core: HadoopFileIO to take list of filesystem schemas to enable trash…
efec561 
… for

Addresses apache#15093
* Default Filesystems are hdfs: and viewfs: schemas
* Tests conditionally enable file: schema as trash
* Tests set up FS with/without trash enablement.
* Also address issue that delete(missing-path) must always succeed; trash policy fails here.
@steveloughran
use Trash.moveToAppropriateTrash() to resolve
b018f7d 
+ docs.
+ test sets in a list of schemas when enabling local trash so verifying that the option splitting works.
@steveloughran steveloughran marked this pull request as draft January 23, 2026 11:09
@steveloughran
TestHadoopFileIO tests
d39925a 
1. verify setting schemas to "" means default hdfs value goes
2. verify that a failure in moveToTrash() is caught and downgraded.

Test case apache#2 is the key one, as it shows that delete works even if trash move somehow failed.

+ fix doc trailing space failure.
@steveloughran
checkstyle
ae098fc
@steveloughran
spotlessness of javadocs. Need to understand the layout rules there.
9340a74
@steveloughran steveloughran marked this pull request as ready for review January 25, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

core docs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@steveloughran