-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Properties transformer #50
Conversation
@Cerfoglg map the related issues, please. We are gonna merge this, when it is integrated with tests and in the final state of refactoring. |
@VincenzoFerme Ah, the issue regarding properties was only opened for the collector of the properties. See the pull I made for the collectors regarding it |
Moreover make sure that you save only once the same |
|
compressed = io.BytesIO(res.read()) | ||
decompressed = gzip.GzipFile(fileobj=compressed) | ||
inspectData = decompressed.readline() | ||
# Retrieve the configuration file that was sent to spark |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this is a common function to multiple transformers. Are you going to refactor it in this pull request or in a new one? If it's a new one, open an issue about this and point to this comment
@Cerfoglg ping me when it's ready to the final review |
@VincenzoFerme Should be ready now |
👍 |
|
Related to benchflow/collectors#31

host_properties
. There should be a single value that let you identify if the host is the same, e.g.,ID
commons
library ofdata-transformers
. I marked it in Develop a library that abstracts common functionalities of data-transformers #5