Skip to content

Conversation

@dlmarion
Copy link
Contributor

No description provided.

@dlmarion dlmarion added this to the 4.0.0 milestone Sep 26, 2025
@dlmarion dlmarion self-assigned this Sep 26, 2025
*
* @since 4.0.0
*/
static CompactableFile create(URI uri, Range range, long estimatedSize, long estimatedEntries) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 4.0, but not referenced. Is this a bug?

*
* @since 4.0.0
*/
static CompactableFile create(URI uri, Range range, long estimatedSize, long estimatedEntries) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is public API, should probably keep and maybe add test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remaining CompactableFile.create method is only used in tests and in RatioBasedCompactionPlanner.FakeFileGenerator. Are users expected to create CompactableFile implementations? If not, I'm thinking that these create methods on the interface should be removed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are users expected to create CompactableFile implementations?

I have found those to be useful when writing unit test for different plugins. Your plugin will be passed those and will not need to create them when running in accumulo.

Copy link
Member

@ctubbsii ctubbsii Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "useful" (especially for unit testing) imply "should be new public API", though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the CompactableFile.create methods and just replaced their references in 3 or 4 places with new CompactableFileImpl.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the methods is existing public API that was added in 2.1.0 and should be deprecated. The other was added when added ranges to metadata file entries.

Does "useful" (especially for unit testing) imply "should be new public API", though?

Yes, these are useful in the public API for two reasons. They can be used by anyone who writes one of the compaction plugins and wishes to unit test their plugin. Mocking these objects is non trivial because they have non trivial hashCode and equals functions, so attempting to mock them in a user unit test would likely be wrong and if correct would involve a lot of duplicative effort. Plugins will likely need to do set operations on these that require hashCode and equals. The easiest option for the user would be to use CompactableFileImpl in their unit test if these do not exist.

Copy link
Member

@ctubbsii ctubbsii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks good. I would double check that some things aren't public API, though.

Comment on lines 99 to 105
public TermSource(TermSource other) {
this.iter = other.iter;
this.term = other.term;
this.notFlag = other.notFlag;
this.seekColfams = other.seekColfams;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is public API. Maybe users have a utility for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted changes to this class in 700ec8c

Comment on lines 401 to 407
public static String stringTopKey(SortedKeyValueIterator<Key,Value> iter) {
if (iter.hasTop()) {
return iter.getTopKey().toString();
}
return "";
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably also public API

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted changes to this class in 700ec8c

Comment on lines -138 to -141
/**
* Candidates that are malformed.
*/
INVALID
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised we don't need this.

Comment on lines 485 to 500
/**
* Contains additional metadata in a reserved area not for tablets
*/
public static class ReservedSection {
private static final Section section = new Section(RESERVED_PREFIX, true, null, false);

public static Range getRange() {
return section.getRange();
}

public static String getRowPrefix() {
return section.getRowPrefix();
}

}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class serves as schema documentation. We don't have a need to scan the whole reserved range, so we don't use it directly, but it should still exist to provide a mechanism in code to evaluate the schema of the metadata table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted changes to this class in 700ec8c

Comment on lines -100 to -108
public int getVersion() {
return version;
}

public static boolean needsUpgrade(final String json) {
var rootData = GSON.get().fromJson(json, Data.class);
int currVersion = rootData.getVersion();
return currVersion < VERSION;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we read the version from the serialized metadata elsewhere now? It seems important.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like these methods were added in #3876 , and then the Upgrader11to12 class was modified in #3988 and removed the code that was calling these methods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants