Skip to content
This repository has been archived by the owner on Mar 11, 2020. It is now read-only.

Do performance testing of Pylama and Coala #59

Open
alexclear opened this issue Nov 8, 2016 · 12 comments
Open

Do performance testing of Pylama and Coala #59

alexclear opened this issue Nov 8, 2016 · 12 comments
Assignees
@alexclear @artemsnezhny
Labels
type/comparing type/test
Milestone
The very end

Comments

@alexclear
Copy link
Member

No description provided.

@alexclear
Copy link
Member Author

  1. Set Netdata to an Ubuntu VM using Ansible

@alexclear
Copy link
Member Author

  1. Get top 50 most-starred repos

@alexclear
Copy link
Member Author

  1. run sloccount on top-50 most starred repos

@alexclear
Copy link
Member Author

alexclear commented Nov 8, 2016 •
edited
Loading

  1. Run pylama on all of them

@alexclear
Copy link
Member Author

alexclear commented Nov 8, 2016 •
edited
Loading

  1. Run Coala on all of them

@alexclear
Copy link
Member Author

  1. Observe metrics in Netdata

@alexclear
Copy link
Member Author

https://github.com/gitinsky/ansible-role-netdata

@alexclear alexclear added this to the Week #47 milestone Nov 8, 2016
@alexclear alexclear changed the title Create a plan for performance testing Do performance testing of Pylama and Coala Nov 8, 2016
@alexclear
Copy link
Member Author

  1. Answer the following questions:
  • What tool is faster?
  • Are we IO-bound or CPU-bound?
  • What functions are most time consuming?

@alexclear
Copy link
Member Author

Take tests three times because we respect physics

@artemsnezhny
Copy link
Member

Simple test of execution time for 11 repositories (ansible-2.2.1.0-0.1.rc1,beets-1.4.1,boto-2.43.0,django-1.8.16,flask-0.11.1,ipython-5.1.0,pandas-0.19.1,requests-2.12.1,scrapy-1.2.1,sentry-8.10.0,tornado-4.4.2):

Coala: ~30 minutes
Pylama: memory exhaustion

@artemsnezhny
Copy link
Member

Last Coala result is invalid. In end of messages.txt:

[ERROR][20:19:58] An unknown error occurred. This is a bug. We are sorry for the inconvenience. Please contact the developers for assistance. During execution of coala an exception was raised. This should never happen. When asked for, the following information may help investigating:

artemsnezhny added a commit that referenced this issue Nov 29, 2016
@artemsnezhny
Up RAM to 3Gb. Start Coala in non-interactive mode. Valid execution t…
4369b79 
…ime result for Coala #59
@artemsnezhny
Copy link
Member

Result of 'time' utility for Coala

real	279m28.419s
user	388m30.549s
sys	122m16.393s

@alexclear alexclear modified the milestones: Week #51, Week #47 Dec 6, 2016
@artemsnezhny artemsnezhny modified the milestones: The very end, Week #51 Jan 26, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@alexclear alexclear

@artemsnezhny artemsnezhny

Labels
type/comparing type/test
Projects
None yet
Milestone
The very end
Development

No branches or pull requests
2 participants
@alexclear @artemsnezhny