Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing of code with asyncio version of redis-om can't be done #527

Open
belyak opened this issue Jun 19, 2023 · 3 comments
Open

Testing of code with asyncio version of redis-om can't be done #527

belyak opened this issue Jun 19, 2023 · 3 comments

Comments

@belyak
Copy link

belyak commented Jun 19, 2023

requirements.txt:

redis-om==0.1.2

models.py:

import aredis_om


class SampleModel(aredis_om.JsonModel):
    field: str = aredis_om.Field(index=True)

func.py:

from aredis_om import NotFoundError

from models import SampleModel


async def find_user(name):
    try:
        return await SampleModel.find(SampleModel.field == name).first()
    except NotFoundError:
        return None

test_func.py:

from unittest import IsolatedAsyncioTestCase

from aredis_om import Migrator

from func import find_user


class MyTest(IsolatedAsyncioTestCase):
    async def asyncSetUp(self) -> None:
        await super().asyncSetUp()
        await Migrator().run()

    async def test_1(self):
        result = await find_user('test')
        self.assertIsNone(result)

    async def test_2(self):
        result = await find_user('test')
        self.assertIsNone(result)

outputs an error:

Ran 2 tests in 0.014s

FAILED (errors=1)

Error
Traceback (most recent call last):
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/connection.py", line 847, in read_response
    response = await self._parser.read_response(
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/connection.py", line 414, in read_response
    await self.read_from_socket()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/connection.py", line 395, in read_from_socket
    buffer = await self._stream.read(self._read_size)
  File "/usr/lib/python3.10/asyncio/streams.py", line 669, in read
    await self._wait_for_data('read')
  File "/usr/lib/python3.10/asyncio/streams.py", line 502, in _wait_for_data
    await self._waiter

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/andy/works/redis-om-connection-test-bug/test_func.py", line 11, in asyncSetUp
    await Migrator().run()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/aredis_om/model/migrations/migrator.py", line 163, in run
    await self.detect_migrations()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/aredis_om/model/migrations/migrator.py", line 118, in detect_migrations
    await conn.ft(cls.Meta.index_name).info()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/commands/search/commands.py", line 867, in info
    res = await self.execute_command(INFO_CMD, self.index_name)
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/client.py", line 518, in execute_command
    return await conn.retry.call_with_retry(
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
    return await do()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/client.py", line 492, in _send_command_parse_response
    return await self.parse_response(conn, command_name, **options)
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/client.py", line 539, in parse_response
    response = await connection.read_response()
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/connection.py", line 869, in read_response
    await self.disconnect(nowait=True)
  File "/home/andy/.venvs/redis-om-connection-test-bug/lib/python3.10/site-packages/redis/asyncio/connection.py", line 737, in disconnect
    self._writer.close()  # type: ignore[union-attr]
  File "/usr/lib/python3.10/asyncio/streams.py", line 338, in close
    return self._transport.close()
  File "/usr/lib/python3.10/asyncio/selector_events.py", line 698, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 750, in call_soon
    self._check_closed()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

As you can see, the tests are the same, but the first is OK and the second fails.
Besides of that, if to debug, there will be an exception "got Future attached to a different loop".
But if I change the test it works ok, however, in a real project it could be problematic to do like this:

models_hacked.py:

from unittest import IsolatedAsyncioTestCase

from aredis_om import Migrator, get_redis_connection

from models import SampleModel
from func import find_user


class MyTest(IsolatedAsyncioTestCase):
    async def asyncSetUp(self) -> None:
        SampleModel._meta.database = get_redis_connection()

        await super().asyncSetUp()
        await Migrator().run()

    async def test_1(self):
        result = await find_user('test')
        self.assertIsNone(result)

    async def test_2(self):
        result = await find_user('test')
        self.assertIsNone(result)
@belyak
Copy link
Author

belyak commented Jun 19, 2023

Finally, in my project, I've used an overloaded db method, like this:

models.py:

import aredis_om


class MyBaseModel(aredis_om.JsonModel):
    @classmethod
    def db(cls):
        return aredis_om.get_redis_connection()


class SampleModel(MyBaseModel):
    field: str = aredis_om.Field(index=True)


class AnotherSampleModel(MyBaseModel):
    field: int

@XChikuX
Copy link

XChikuX commented Jun 20, 2023

You can add the Meta class to change the connection parameters without needed a BaseClass

from aredis_om import (
    Field,
    HashModel,
    JsonModel,
    EmbeddedJsonModel,
    Migrator,
    get_redis_connection,
)
redis_conn = get_redis_connection(
    url=f"redis://10.9.9.4:6379", decode_responses=False, password="random"
)
class Matched(HashModel):
    user_id1: str
    user_id2: str

    @root_validator()
    def assign_pk(cls, values):
        values["pk"] = f"{values['user_id1']}:{values['user_id2']}"
        return values

    class Meta:
        database = redis_conn

Remember to call await Migrator().run() It can be tricky to know when to call this to create indexes.

@belyak
Copy link
Author

belyak commented Jun 21, 2023

You can add the Meta class to change the connection parameters without needed a BaseClass

from aredis_om import (
    Field,
    HashModel,
    JsonModel,
    EmbeddedJsonModel,
    Migrator,
    get_redis_connection,
)
redis_conn = get_redis_connection(
    url=f"redis://10.9.9.4:6379", decode_responses=False, password="random"
)
class Matched(HashModel):
    user_id1: str
    user_id2: str

    @root_validator()
    def assign_pk(cls, values):
        values["pk"] = f"{values['user_id1']}:{values['user_id2']}"
        return values

    class Meta:
        database = redis_conn

Remember to call await Migrator().run() It can be tricky to know when to call this to create indexes.

It will be validated only once then, the unit-testing assumes that all the resources are reinitialized for each case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants