Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the FileNotFoundError of get_model_features() #2052

Merged
merged 1 commit into from
Jun 27, 2019
Merged

Fix the FileNotFoundError of get_model_features() #2052

merged 1 commit into from
Jun 27, 2019

Conversation

yanghangliu
Copy link
Contributor

There is no /usr/share/libvirt/cpu_map.xml file existing on RHEL8,
but instead of this file with /usr/share/libvirt/cpu_map directory.
If a test which calls the get_model_features() is runned on RHEL8,
an FileNotFoundError will be thrown resulting in making this test
failed.

Signed-off-by: Yanghang Liu [email protected]

@yanghangliu yanghangliu reopened this Apr 23, 2019
@yanghangliu
Copy link
Contributor Author

yanghangliu commented Apr 23, 2019

The test about virsh.dumpxml is failed because of a calling to get_model_features()

I can get the test about virsh.dumpxml passed by rewriting get_model_features()
The test result is as followed:

[root@RHEL8 tp-libvirt]# avocado run --vt-type libvirt virsh.dumpxml
JOB ID : 09e5870078b9f98aee9c3846a305c075c3e93f16
JOB LOG : /root/avocado/job-results/job-2019-04-23T10.44-09e5870/job.log
(01/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_default.domname: PASS (60.94 s)
(02/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_default.domuuid: PASS (50.69 s)
(03/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_inactive.domname: PASS (64.86 s)
(04/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_inactive.domuuid: PASS (65.54 s)
(05/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.minimum_match.domname: PASS (53.46 s)
(06/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.minimum_match.domuuid: PASS (51.42 s)
(07/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.exact_match.domname: PASS (67.70 s)
(08/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.exact_match.domuuid: PASS (84.53 s)
(09/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.strict_match.domname: PASS (19.07 s)
(10/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_cpu.strict_match.domuuid: PASS (22.95 s)
(11/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_security.domname: PASS (22.98 s)
(12/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_shutoff.with_security.domuuid: PASS (22.48 s)
(13/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_default.domid: PASS (29.03 s)
(14/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_default.domname: PASS (27.94 s)
(15/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_default.domuuid: PASS (23.00 s)
(16/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_inactive.domid: PASS (22.08 s)
(17/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_inactive.domname: PASS (24.18 s)
(18/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_running.with_inactive.domuuid: PASS (22.65 s)
(19/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_default.domid: PASS (28.98 s)
(20/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_default.domname: PASS (20.85 s)
(21/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_default.domuuid: PASS (21.49 s)
(22/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_inactive.domid: PASS (23.22 s)
(23/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_inactive.domname: PASS (22.65 s)
(24/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.non_acl.vm_paused.with_inactive.domuuid: PASS (23.12 s)
(25/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_default.domname: PASS (61.99 s)
(26/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_default.domuuid: PASS (51.39 s)
(27/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_inactive.domname: PASS (26.51 s)
(28/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_inactive.domuuid: PASS (31.13 s)
(29/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.minimum_match.domname: PASS (43.65 s)
(30/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.minimum_match.domuuid: PASS (45.04 s)
(31/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.exact_match.domname: PASS (39.13 s)
(32/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.exact_match.domuuid: PASS (46.52 s)
(33/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.strict_match.domname: PASS (47.60 s)
(34/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_shutoff.with_cpu.strict_match.domuuid: PASS (45.62 s)
(35/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_default.domid: PASS (62.56 s)
(36/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_default.domname: PASS (51.48 s)
(37/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_default.domuuid: PASS (40.69 s)
(38/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_inactive.domid: PASS (39.44 s)
(39/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_inactive.domname: PASS (35.60 s)
(40/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_running.with_inactive.domuuid: PASS (38.47 s)
(41/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_default.domid: PASS (41.50 s)
(42/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_default.domname: PASS (40.48 s)
(43/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_default.domuuid: PASS (46.10 s)
(44/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_inactive.domid: PASS (39.63 s)
(45/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_inactive.domname: PASS (37.14 s)
(46/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_none.vm_paused.with_inactive.domuuid: PASS (33.98 s)
(47/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_read_secure.vm_shutoff.with_security.domname: PASS (34.83 s)
(48/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.normal_test.acl_test.grant_read_secure.vm_shutoff.with_security.domuuid: PASS (37.69 s)
(49/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.none_domain: PASS (28.58 s)
(50/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.invalid_domid: PASS (27.26 s)
(51/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.hex_domid: PASS (24.46 s)
(52/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.invalid_domuuid: PASS (27.04 s)
(53/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.invalid_domname: PASS (27.13 s)
(54/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.additional_args: PASS (27.78 s)
(55/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.not_exist_options: PASS (25.84 s)
(56/56) type_specific.io-github-autotest-libvirt.virsh.dumpxml.error_test.acl_test.security_info: PASS (48.41 s)
RESULTS : PASS 56 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 2203.85 s

output = open(conf_dir + file, "r").read()
model = ET.fromstring(output)
for feature in model.findall("model/feature"):
features.append(feature.get('name'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code dealing with xml for both scenarios is the same logic, please simplify the code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @yafu-1, please re-factor to avoid redundancy.

#Find model in /usr/share/libvirt/cpu_map
if os.path.isdir(conf_dir) and os.path.exists(conf_dir):
filelist = os.listdir(conf_dir)
for file in filelist:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

file is a keyword in python.

output = open(conf_dir + file, "r").read()
model = ET.fromstring(output)
for feature in model.findall("model/feature"):
features.append(feature.get('name'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @yafu-1, please re-factor to avoid redundancy.

if nested_model is not None:
nested_model_name = nested_model.get('name')
if os.path.isfile(conf) and os.path.exists(conf):
output = open(conf, 'r').read()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to use with statements.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for helping me review the code.
I will fix the code soon.

for feature in model_n.findall('feature'):
features.append(feature.get('name'))
break
if os.path.isfile(conf) and os.path.exists(conf):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for the "and" part for judgement

features.append(feature.get('name'))
break
# Find model in dir /usr/share/libvirt/cpu_map
if os.path.isdir(conf_dir) and os.path.exists(conf_dir):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above

@@ -1,5 +1,5 @@
[provider]
uri: https://github.com/autotest/tp-libvirt.git
uri: file:///root/code/tp-libvirt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove this change.

filelist = os.listdir(conf_dir)
for file_name in filelist:
if model_name in file_name:
with open(conf_dir + file, "r").read() as output:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. It would break here as file_name to be used ?
  2. please use os.path.join()

continue
else:
break
# Find model in dir /usr/share/libvirt/cpu_map
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix indentation

Copy link
Member

@balamuruhans balamuruhans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes required, otherwise code looks good to me. Thanks!

@@ -4024,26 +4025,36 @@ def get_model_features(model_name):
"""
features = []
conf = "/usr/share/libvirt/cpu_map.xml"
conf_dir = "/usr/share/libvirt/cpu_map/"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conf = "/usr/share/libvirt/cpu_map.xml"
# check the libvirt version to branch out instead of path
if libvirt_version.version_compare(5, 0, 0):
    conf = "/usr/share/libvirt/cpu_map/"
    # we should raise the error if the path doesn't available
    if not os.path.exists(conf):
        test.error()
    with open(conf, 'r') as output:
        :::
else:
    :::

Copy link
Contributor Author

@yanghangliu yanghangliu May 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@balamuruhans
Thank you very much for helping review the code and pointing out my code errors !

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The utils_misc.py file path is "~/avocado-vt/virttest/utils_misc.py"
The libvirt_version.py file path is "tp-libvirt/provider/ libvirt_version.py "
I can not import libvirt_version.version_compare() directly in this utils_misc.py.
Should I write a function to get libvirt version, or what can I do so that I can import the libvirt_version.version_compare() ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggested to move the code from tp-libvirt/provider/libvirt_version.py to avocado-vt/virttest/ and use it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dzhengfy @sathnaga request for your acknowlegement.

Copy link
Member

@sathnaga sathnaga May 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yanghangliu @dzhengfy @balamuruhans looks like moving libvirt_version.py needs a change in many tests, for now we can copy libvirt_version.py to virttest and use in this testcase and slowly change import in other testcases through different PR and once all movement happened, we can safely remove tp-libvirt/provider/libvirt_version.py

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, once moving to virttest, we will have to send separate PR to tp-libirt to adopt it by changing all the testcases using it from virttest and remove the file in provider

features.append(feature.get('name'))
break
if not libvirt_version.version_compare(5, 0, 0):
if os.path.exists(conf):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For compare version, no need to judge the file exist or not. If not exist, it should be error. Also the following part.

@lgtm-com
Copy link

lgtm-com bot commented Jun 26, 2019

This pull request introduces 163 alerts when merging 132ce0f into 40c351f - view on LGTM.com

new alerts:

  • 163 for Module-level cyclic import

@balamuruhans
Copy link
Member

@yanghangliu, we have to fix the cyclic import issue, I have tried to clean utils_misc.py in #2103 to break the cyclic imports. Feel free to review it. Thanks!

  There is no /usr/share/libvirt/cpu_map.xml file existing on RHEL8,
  but instead of this file with /usr/share/libvirt/cpu_map directory.
  If a test which calls the get_model_features() is runned on RHEL8,
  an FileNotFoundError will be thrown resulting in making this test
  failed.

Signed-off-by: Yanghang Liu <[email protected]>
@kylazhang
Copy link
Contributor

kylazhang commented Jun 27, 2019

The import cycle resolved by remove the virsh import. Thanks

@kylazhang kylazhang merged commit 71c1605 into avocado-framework:master Jun 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants