Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properties transformer #50

Merged
merged 5 commits into from
May 19, 2016
Merged

Properties transformer #50

merged 5 commits into from
May 19, 2016

Conversation

Cerfoglg
Copy link
Contributor

@Cerfoglg Cerfoglg commented Apr 17, 2016

Related to benchflow/collectors#31

   

@VincenzoFerme
Copy link
Member

@Cerfoglg map the related issues, please. We are gonna merge this, when it is integrated with tests and in the final state of refactoring.

@Cerfoglg
Copy link
Contributor Author

@VincenzoFerme Ah, the issue regarding properties was only opened for the collector of the properties. See the pull I made for the collectors regarding it

@VincenzoFerme
Copy link
Member

VincenzoFerme commented Apr 20, 2016

Moreover make sure that you save only once the same host_properties. There should be a single value that let you identify if the host is the same, e.g., ID

@Cerfoglg
Copy link
Contributor Author

  • Move code away from the main

compressed = io.BytesIO(res.read())
decompressed = gzip.GzipFile(fileobj=compressed)
inspectData = decompressed.readline()
# Retrieve the configuration file that was sent to spark
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is a common function to multiple transformers. Are you going to refactor it in this pull request or in a new one? If it's a new one, open an issue about this and point to this comment

@VincenzoFerme
Copy link
Member

@Cerfoglg ping me when it's ready to the final review

@VincenzoFerme VincenzoFerme mentioned this pull request May 19, 2016
23 tasks
@Cerfoglg
Copy link
Contributor Author

@VincenzoFerme Should be ready now

@VincenzoFerme
Copy link
Member

👍

@VincenzoFerme
Copy link
Member

:shipit:

@VincenzoFerme VincenzoFerme merged commit b8e04f0 into benchflow:dev May 19, 2016
@Cerfoglg Cerfoglg deleted the properties-transformer branch May 31, 2016 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

2 participants