Could not open the database: 'module' object has no attribute 'Connection'

[Azef1]

Hi,
I try to install this module to use with livestatus but in my brokerd.log i have this error:
Could not open the database: 'module' object has no attribute 'Connection'
I installed releases 1.4.2 because i have pymongo 3.

I started with default params in logstore_mongodb.cfg:

define module {
module_name logstore-mongodb
module_type logstore_mongodb
mongodb_uri mongodb://localhost/?safe=false ; Set to your value
# If you are running a MongoDB cluster (called a “replica set” in MongoDB),
# you need to specify it's name here.
# With this option set, you can also write the mongodb_uri as a comma-separated
# list of host:port items. (But one is enough, it will be used as a “seed”)
#replica_set
#database
#collection
#max_logs_age
}

The logs results is:

INFO: [broker] Correctly loaded livestatus as a very-new-style shinken module :)
INFO: [broker] Correctly loaded logstore-mongodb as an old-new-style shinken module :|
INFO: [broker] [LogstoreMongoDB] Get an LogStore MongoDB module for plugin logstore-mongodb
INFO: [broker] Trying to init module: logstore-mongodb
INFO: [broker] I correctly loaded the modules: [logstore-mongodb]
INFO: [broker] [Livestatus Broker] Go run
ERROR: [broker] [LogStoreMongoDB] Could not open the database: 'module' object has no attribute 'Connection'
ERROR: [broker] [livestatus] Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/shinken/basemodule.py", line 164, in start_module
self._main()
File "/usr/lib/python2.7/site-packages/shinken/basemodule.py", line 296, in _main
self.main()
File "/var/lib/shinken/modules/livestatus/module.py", line 250, in main
self.from_q.put(msg)
File "", line 2, in put
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
raise convert_to_error(kind, result)
RemoteError:
Traceback (most recent call last):
File "/usr/lib64/python2.7/multiprocessing/managers.py", line 240, in serve_client
request = recv()
ImportError: No module named logstore-mongodb.module
ERROR: [broker] The external module livestatus goes down unexpectedly!
INFO: [broker] Setting the module livestatus to restart

I checked my database in mongo :
I have a "shinken" database and "logs" collection

So i edited logstore_mongodb.cfg to set database and collection:

define module {
module_name logstore-mongodb
module_type logstore_mongodb
mongodb_uri mongodb://localhost/?safe=false ; Set to your value
# If you are running a MongoDB cluster (called a “replica set” in MongoDB),
# you need to specify it's name here.
# With this option set, you can also write the mongodb_uri as a comma-separated
# list of host:port items. (But one is enough, it will be used as a “seed”)
#replica_set
database shinken
collection logs
#max_logs_age
}

But i have same error : Could not open the database ....

My mongo works well with webui2. Have an idea to solve the error ?

[mohierf]

If you need to store logs in a MongoDB database without LiveStatus, you should use this module: https://github.com/shinken-monitoring/mod-mongo-logs. If you need LS, I cannot help ... sorry

[Azef1]

I have an idea:
I went to source code: https://github.com/shinken-monitoring/mod-logstore-mongodb/blob/master/module/module.py#L174 on line 174. I have determined that the real error message is 'module' object has no attribute 'Connection'
And after several research in google (see hyperopt/hyperopt#241), this error is due by a deprecated command pymongo.Connection was depricated in pymongo version 2.4.
I use pymongo 3 because it is required in webui2.
If you go to manual of pymongo: http://api.mongodb.org/python/current/tutorial.html you can see to make a connection they use MongoClient command instead of old pymongo.Connection.

So i think but not sure:
1- Add in the source code client = MongoClient() to create a MongoClient
2- Replace in line 153 and 155 pymongo.Connection(.....) by MongoClient('.....') command.

What do you think ?
I will try this fix tomorrow.

[mohierf]

You can try this fix ... or you can have a look in the source code of the mongo-logs module how it is managed.

[Azef1]

i applied the modifications (mongoClient) now the connection to mongo v3 is works !!

[1452074100] INFO: [broker] Trying to init module: logstore-mongodb
[1452074100] INFO: [broker] I correctly loaded the modules: [logstore-mongodb]
[1452074100] INFO: [broker] [Livestatus Broker] Go run
[1452074100] INFO: [broker] [Livestatus Broker] Livestatus query thread started
[1452074100] INFO: [broker] [Livestatus Broker] listening on tcp port: 50000

i created a fork here: https://github.com/Azef1/mod-logstore-mongodb

But a new error in brokerd.log, i have several lines:

[mongo-logs] This line is invalid: [1452074035] INFO: [Shinken] Checking brokers...
[mongo-logs] This line is invalid: [1452074097] INFO: [broker_noeyy6z1] [Graphite] Configuration - host/port:
[mongo-logs] This line is invalid: [1452074035] INFO: [Shinken] Checked 1 brokers

I think it's a problem about the function manage_log_brok (https://github.com/Azef1/mod-logstore-mongodb/blob/master/module/module.py#L205) but i don't understand this function.
What is a normal functioning to see lot of [mongo-logs] This line is invalid: in brokerd.log ?

[mohierf]

Once again, have a look at my mongo-logs module, because I copied and modified the log line analyzer of LiveStatus because I found some missing information ...
The file is: https://github.com/shinken-monitoring/mod-mongo-logs/blob/master/module/log_line.py

[Azef1]

Ok now i haven't error this line is invalid... i followed your advice.
But now a new error :)

[livestatus] Warning: The mod logstore-mongodb raise an exception: 'LiveStatusLogStoreMongoDB' object has no attribute 'logs_cache', I'm tagging it to restart later

[mohierf]

I suppose you copied too much code ... you must debug your python code.

[Azef1]

Hi,
All works now, you were right, i copied too much code :)
In my brokerd.log i have not error and the last lines are:

[1452156607] INFO: [broker] Trying to init module: logstore-mongodb
[1452156607] INFO: [broker] I correctly loaded the modules: [logstore-mongodb]
[1452156607] INFO: [broker] [Livestatus Broker] Go run
[1452156608] INFO: [broker] [Livestatus Broker] Livestatus query thread started
[1452156608] INFO: [broker] [Livestatus Broker] listening on tcp port: 50000
[1452156609] INFO: [broker] [LogStoreMongoDB] Next log rotation at Fri Jan 8 00:05:00 2016

So i think all is ok!!!!!
Now i will install Nagvis.
My fork is here: https://github.com/Azef1/mod-logstore-mongodb and you can test it.
I will be very happy if it's possible to update the original module (https://github.com/shinken-monitoring/mod-logstore-mongodb) from my frok.

[mohierf]

You can submit a Pull Request from your fork and the module maintainer will be able to merge it ...

[kirillfish]

Got into the same trouble while trying to parallelize hyperopt. @Azef1, please, explain in more detail how precisely can I use your code to make MongoTrials work after all.

[mohierf]

@Azef1 : can you submit a PR from your forked module ?

[jbnitro]

Hello,
I have a same problem

Error on close database: 'Database' object is not callable. If you meant to call the 'disconnect' method on a 'MongoClient' object it is failing because no such method exists.

but I installed the fork of Azef1 and I have mongo-logs and logstore-mongodb on my shinken

I need to remove one module or not ?

I need to install livestatus module for nagvis and for livestatus it needs logstore-mongodb or mongo-logs

With module of Azef1 it can replace mongo-logs ?

J'ai installé le fork de Azef1 en plus du module mongo-logs donc j'ai les deux modules logstore-mongodb et mongo-logs
Son module peut remplacer mongo-logs ?
Pour avoir le webui2 il faut obligatoirement mongo-logs ?

[mohierf]

For WebUI2 you do not need this module nor the mongo-logs one ... the mongo-logs module is made to store the Shinken logs in a mongo DB and the WebUI2 uses this module to display an history and availability from the logs, but it is not mandatory.

The database error that occured is mainly because of the difference betweeen Mongo2 and Mongo3 versions ... for the fork of @Azef1 I cannot help 😉

[jbnitro]

Oui voila donc pour avoir un webui2 fonctionnel il faut garder mongo-logs, mais pour nagvis il lui faut livestatus qui a besoin de logstore-mongodb, c'est le problème.

Enfin la j'ai testé la version de GuusH avec le fix de lermit et j'ai plus d'erreur dans le broker enfin rien qui bloque je crois

[mohierf]

C'et cela, la WebUI est fonctionnelle même si le module mongo-logs n'est pas installé. Il n'y aura simplement pas d'historique ni de disponibilité ... faute de source pour les récupérer.

[Azef1]

Ok for NagVis, i installed Livestatus module + mod-logstore-null module here
I abandoned the mod-logstore-mongodb because it's not stable. But if you want i could explain what i did in the code.
In Nagvis if you have an error "Problem Backend: live_1. The object doesn't exist" using Servicegroup object, you need to edit this file var/lib/shinken/modules/livestatus/mapping.py and add

    'staleness': {
        'description': 'FIXME',
        'function': lambda item, req: 0,
    },

See here

Quickly to explain my code:
WebUI using mongo3 and mod-logstore-mongodb using a function for mongo2
So i replaced pymongo.Connection(...) by the function for mongo3: MongoClient(...)