Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added enhancement in the API to support for Google BigTable #337

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Added enhancement in the API to support for Google BigTable #337

wants to merge 1 commit into from

Conversation

vim89
Copy link

@vim89 vim89 commented Jun 19, 2020 •
edited
Loading

Background -

Basically Google BigTable doesn't have namespaces & name descriptors Check this
Hence, during createRelation task we have to skip calling getter/setter methods of namespaces & name descriptors viz. getNamespaceDescriptor() and createNamespace()
There were 2 issues -

  1. I had created an issue
  2. Another similar one was created back in 2017.

What changes were proposed in this pull request?

  1. Create new class variable tableType in HBaseTableCatalog
    tableType variable by default is initialized to value "hbase"
  2. Add getter and setter methods to overwrite tableType variable
  3. Create if else branch in createTableIfNotExist() in HBaseRelation class based on tableType variable set in catalog, so as to skip calling namespace getter methods if API is used to perform write into Google BigTable
  4. Illustrate the usage for writing into Google BigTable iin README.md

How was this patch tested?

  1. Unit test is added, HBaseTableCatalogSuite.scala
  2. Manual testing is performed thoroughly and I'm using this in one of my project & running since 7 months in production, therefore I think now this is stable & right time to create pull request to merge into master branch

Regards,
Vitthal

@vim89
Added enhancement in the API to support for Google BigTable.
57379fd 
Basically Google BigTable doesn't have namespaces & name descriptors Check this - https://cloud.google.com/bigtable/docs/hbase-differences#namespaces.
Hence, during createRelation task we have to skip getter/setter methods of namespaces & name descriptors.

I had to create an if else branch for that based on "tableType" property in HBaseTableCatalog class.
"tableType" variable takes value "bigtable" when using with Google BigTable and by default value will be "hbase" if not explicitly specified.
vim89
vim89 commented Jun 19, 2020
View reviewed changes
Copy link
Author

@vim89 vim89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed, Tested and good for merging into master

@vim89 vim89 mentioned this pull request Jun 19, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@vim89