Test discovery errors crash nose

[jpellerin]

I thought I'd submitted this on Friday, sorry if it shows up twice...

What steps will reproduce the problem?

1. nosetests doctest.py

What is the expected output? What do you see instead?

Expected: nose-reported errors, logged by .addError.
Got: traceback (other failures are possible with different plugins
active):

/home/john/src/nose/nose/case.py:321: DeprecationWarning: class Tester is
deprecated; use class doctest.DocTestRunner instead
self.inst = self.cls()
Traceback (most recent call last):
File "/usr/bin/nosetests", line 8, in
load_entry_point('nose==0.10.0b1', 'console_scripts', 'nosetests')()
File "/home/john/src/nose/nose/core.py", line 188, in init
argv=argv, testRunner=testRunner, testLoader=testLoader)
File "/usr/lib/python2.5/unittest.py", line 767, in init
self.parseArgs(argv)
File "/home/john/src/nose/nose/core.py", line 236, in parseArgs
self.createTests()
File "/home/john/src/nose/nose/core.py", line 249, in createTests
self.test = self.testLoader.loadTestsFromNames(self.testNames)
File "/home/john/src/nose/nose/loader.py", line 408, in
loadTestsFromNames
return unittest.TestLoader.loadTestsFromNames(self, names, module)
File "/usr/lib/python2.5/unittest.py", line 565, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/home/john/src/nose/nose/loader.py", line 363, in
loadTestsFromName
module, discovered=discovered)
File "/home/john/src/nose/nose/loader.py", line 280, in
loadTestsFromModule
test_classes + test_funcs)
File "/home/john/src/nose/nose/loader.py", line 279, in
tests = map(lambda t: self.makeTest(t, parent=module),
File "/home/john/src/nose/nose/loader.py", line 486, in makeTest
return self.loadTestsFromTestClass(obj)
File "/home/john/src/nose/nose/loader.py", line 449, in
loadTestsFromTestClass
for case in filter(wanted, dir(cls))]
File "/home/john/src/nose/nose/loader.py", line 496, in makeTest
return MethodTestCase(obj)
File "/home/john/src/nose/nose/case.py", line 321, in init
self.inst = self.cls()
File "doctest.py", line 1970, in init

TypeError: Tester.init: must specify mod or globs

What version of the product are you using? On what operating system?

http://python-nose.googlecode.com/svn/branches/0.10.0-stable revision 309.
Ubuntu Linux.

Please provide any additional information below.

Obviously these particular Python files didn't expect to have their tests
discovered by nose. I expected errors during test discovery to be
reported the same way as errors in running discovered tests. It seems
that doesn't (always?) happen. Would a patch be accepted, or is this
regarded as preferable behaviour?

Google Code Info:
Issue #: 86
Author: [email protected]
Created On: 2007-09-02T19:36:33.000Z
Closed On:

[jpellerin]

A patch would be great. Errors during discovery should always (for reasonable cases)
produce nose.case.Failure instances, without killing the whole test run.

Google Code Info:
Author: [email protected]
Created On: 2007-09-02T23:48:19.000Z

[jpellerin]

I have added a patch which fixes this error (with test case!)

Open questions about the patch:

1. Is it wise to catch all Exceptions in loader:makeTest (given what can happen I
   don't see another way).
2. Is the try except block in loader.makeTest broad enough? i.e. should it be in elif
   isclass(obj) or around all the attempts to make tests

Unfortunately, nosetests doctest.py still fails, but gets further:

y/home/localjames/svn/nose/nose/case.py:320: DeprecationWarning: class Tester is
deprecated; use class doctest.DocTestRunner instead
self.inst = self.cls()
Traceback (most recent call last):
File "/usr/bin/nosetests", line 8, in
load_entry_point('nose==0.10.0b1.dev-r309', 'console_scripts', 'nosetests')()
File "/home/localjames/svn/nose/nose/core.py", line 188, in init
argv=argv, testRunner=testRunner, testLoader=testLoader)
File "/usr/lib/python2.5/unittest.py", line 768, in init
self.runTests()
File "/home/localjames/svn/nose/nose/core.py", line 263, in runTests
result = self.testRunner.run(self.test)
File "/home/localjames/svn/nose/nose/core.py", line 62, in run
test(result)
File "/home/localjames/svn/nose/nose/suite.py", line 133, in call
return self.run(_arg, *_kw)
File "/home/localjames/svn/nose/nose/suite.py", line 164, in run
test(orig)
File "/home/localjames/svn/nose/nose/suite.py", line 133, in call
return self.run(_arg, *_kw)
File "/home/localjames/svn/nose/nose/suite.py", line 164, in run
test(orig)
File "/home/localjames/svn/nose/nose/suite.py", line 133, in call
return self.run(_arg, *_kw)
File "/home/localjames/svn/nose/nose/suite.py", line 164, in run
test(orig)
File "/home/localjames/svn/nose/nose/case.py", line 63, in call
return self.run(_arg, *_kwarg)
File "/home/localjames/svn/nose/nose/case.py", line 147, in run
result.addError(self, err)
File "/home/localjames/svn/nose/nose/proxy.py", line 100, in addError
self.assertMyTest(test)
File "/home/localjames/svn/nose/nose/proxy.py", line 81, in assertMyTest
% (self.test, id(self.test), test, id(test)))
File "/home/localjames/svn/nose/nose/case.py", line 72, in repr
return "Test(%r)" % self.test
File "/usr/lib/python2.5/doctest.py", line 2279, in repr
return self._dt_test.filename
AttributeError: 'str' object has no attribute 'filename'

Google Code Info:
Author: [email protected]
Created On: 2007-09-08T17:29:38.000Z

[jpellerin]

The latest error seems to be due to the fact that some instances of DocTestCase are
being created with strings instead of tests. Here is the init

class DocTestCase(unittest.TestCase):

def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
             checker=None):

    unittest.TestCase.__init__(self)
    self._dt_optionflags = optionflags
    self._dt_checker = checker
    self._dt_test = test
    self._dt_setUp = setUp
    self._dt_tearDown = tearDown

so the "test" argument is a string. I will carry on looking for the point where
this is created by nose.

Google Code Info:
Author: [email protected]
Created On: 2007-09-09T12:36:30.000Z

[jpellerin]

This affects unittest as well. Not sure this is an excuse, since unittest won't get
itself into that situation (no discovery)

james@palermo:/home/localjames/svn/nose/sandbox$ cat test_doctest.py
import doctest
import unittest

TestDoc = doctest.DocTestCase

if name == "main":
unittest.main()

james@palermo:/home/localjames/svn/nose/sandbox$ python test_doctest.py

EE

Traceback (most recent call last):
File "test_doctest.py", line 9, in
unittest.main()
File "/usr/lib/python2.5/unittest.py", line 768, in init
self.runTests()
File "/usr/lib/python2.5/unittest.py", line 805, in runTests
result = self.testRunner.run(self.test)
File "/usr/lib/python2.5/unittest.py", line 708, in run
result.printErrors()
File "/usr/lib/python2.5/unittest.py", line 676, in printErrors
self.printErrorList('ERROR', self.errors)
File "/usr/lib/python2.5/unittest.py", line 682, in printErrorList
self.stream.writeln("%s: %s" % (flavour,self.getDescription(test)))
File "/usr/lib/python2.5/unittest.py", line 642, in getDescription
return test.shortDescription() or str(test)
File "/usr/lib/python2.5/doctest.py", line 2208, in shortDescription
return "Doctest: " + self._dt_test.name
AttributeError: 'str' object has no attribute 'name'

Google Code Info:
Author: [email protected]
Created On: 2007-09-09T14:03:54.000Z

[jpellerin]

Well this patch fixes it.

I'm not sure this second patch should go in as it addresses the very marginal case
that seems to occur when unittest tries to load the tests from DocTestCase.

Google Code Info:
Author: [email protected]
Created On: 2007-09-09T14:29:32.000Z

[jpellerin]

I'm going to reopen but postpone for post 0.10.0.

What I'm thinking right now is that loadTestsFromNames() is going to have to wrap
each call to loadTestsFromName() in a try/except block, but I don't want to make a
major change like that so close to a release, and I want to think about it some more.

Google Code Info:
Author: [email protected]
Created On: 2007-09-10T20:00:30.000Z

[jpellerin]

Has this got forgotten?

Google Code Info:
Author: [email protected]
Created On: 2008-01-27T00:59:56.000Z

[jpellerin]

Google Code Info:
Author: [email protected]
Created On: 2008-03-23T02:15:36.000Z

[jpellerin]

Wrapping the .loadTestsFromName() call in .loadTestsFromNames() fails to catch this,
I guess because the real loading happens lazily later on. The attached patch renames
the "real" .loadTestsFromName() with an initial underscore, and calls that from a
little wrapper function that catches "everything" and returns a Failure. Seems to do
the trick for "nosetests doctest.py", but I haven't tried to analyze all the possible
lazy cases! Makes my head hurt...

Google Code Info:
Author: [email protected]
Created On: 2008-03-23T21:19:08.000Z

[jpellerin]

I need to think about this one some more. If laziness goes away, as we've discussed I
think for 0.11 or later, does this get simpler -- or at least simpler to reason about?

I'm going to put this under 0.11 for now to hold it open for more discussion.

Google Code Info:
Author: [email protected]
Created On: 2008-03-25T02:25:52.000Z

[jpellerin]

I'm sure it would make it easier to think about.

As I say, I didn't really try to figure it out, and I'm sure you know better than I
do how the lazy loading works.

Google Code Info:
Author: [email protected]
Created On: 2008-03-25T18:47:19.000Z

[jpellerin]

Pushing this to post-1.0, when test loading stops being lazy.

Google Code Info:
Author: [email protected]
Created On: 2009-04-19T15:59:13.000Z