Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Method to detect required redis modules is flawed #220

Open
JordanTreDaniel opened this issue May 1, 2023 · 0 comments
Open

Method to detect required redis modules is flawed #220

JordanTreDaniel opened this issue May 1, 2023 · 0 comments

Comments

@JordanTreDaniel
Copy link

JordanTreDaniel commented May 1, 2023

Desc

  • The app cannot properly detect the fact that my redis db does indeed have RedisSearch (or any other module).
  • I have a redis labs db set up, (not on my local. on redis labs) and it can be connected to (both by the project, and from the CLI).

Proof of Redis DB:

  • Here is an image of the dashboard, showing the db instance, along with included modules
    • image
  • Here is an active connection from the cli, also showing the modules (as the program would see them)
redis-13165.c9.us-east-1-4.ec2.cloud.redislabs.com:13165> MODULE LIST
1) 1) "name"
   2) "searchlight"
   3) "ver"
   4) (integer) 20606
2) 1) "name"
   2) "timeseries"
   3) "ver"
   4) (integer) 10809
3) 1) "name"
   2) "bf"
   3) "ver"
   4) (integer) 20405
4) 1) "name"
   2) "ReJSON"
   3) "ver"
   4) (integer) 20406
5) 1) "name"
   2) "graph"
   3) "ver"
   4) (integer) 21009

Problem:

  • The code expects client.modules to be declared, while it is not.
  • Here is the (json-encoded) "client" that the program does detect, printed from within the _check_redis_module_exist method:
{
    "redis_version": "6.2.7",
    "redis_git_sha1": 0,
    "redis_git_dirty": 0,
    "redis_build_id": 0,
    "redis_mode": "standalone",
    "os": "Linux 5.4.0-1093-aws x86_64",
    "arch_bits": 64,
    "multiplexing_api": "epoll",
    "gcc_version": "7.5.0",
    "process_id": 11628161,
    "run_id": "3648691c19356deaab411ed73164b7beef648819",
    "tcp_port": 13165,
    "server_time_usec": 1682964253000000,
    "uptime_in_seconds": 125400,
    "uptime_in_days": 1,
    "hz": 10,
    "lru_clock": 0,
    "config_file": "",
    "connected_clients": 1,
    "client_longest_output_list": 0,
    "client_biggest_input_buf": 0,
    "blocked_clients": 0,
    "maxclients": 30,
    "cluster_connections": 0,
    "used_memory": 4565096,
    "used_memory_human": "4.35M",
    "used_memory_rss": 4565096,
    "used_memory_peak": 6677576,
    "used_memory_peak_human": "6.36M",
    "used_memory_lua": 32768,
    "mem_fragmentation_ratio": 1,
    "mem_allocator": "jemalloc-5.2.1",
    "loading": 0,
    "rdb_changes_since_last_save": 14,
    "rdb_bgsave_in_progress": 0,
    "rdb_last_save_time": 1682838854,
    "rdb_last_bgsave_status": "ok",
    "rdb_last_bgsave_time_sec": 0,
    "rdb_current_bgsave_time_sec": -1,
    "aof_enabled": 0,
    "aof_rewrite_in_progress": 0,
    "aof_rewrite_scheduled": 0,
    "aof_last_rewrite_time_sec": -1,
    "aof_current_rewrite_time_sec": -1,
    "aof_last_bgrewrite_status": "ok",
    "aof_last_write_status": "ok",
    "total_connections_received": 24,
    "total_commands_processed": 66,
    "instantaneous_ops_per_sec": 0,
    "total_net_input_bytes": 10055791,
    "total_net_output_bytes": 174809710,
    "instantaneous_input_kbps": 0.07,
    "instantaneous_output_kbps": 1.31,
    "rejected_connections": 0,
    "sync_full": 0,
    "sync_partial_ok": 0,
    "sync_partial_err": 0,
    "expired_keys": 0,
    "evicted_keys": 0,
    "keyspace_hits": 0,
    "keyspace_misses": 13,
    "pubsub_channels": 0,
    "pubsub_patterns": 0,
    "latest_fork_usec": 0,
    "migrate_cached_sockets": 0,
    "total_forks": 0,
    "total_error_replies": 0,
    "role": "master",
    "connected_slaves": 0,
    "master_repl_offset": 0,
    "repl_backlog_active": 0,
    "repl_backlog_size": 1048576,
    "repl_backlog_first_byte_offset": 0,
    "repl_backlog_histlen": 0,
    "used_cpu_sys": 0.0,
    "used_cpu_user": 0.0,
    "used_cpu_sys_children": 0.0,
    "used_cpu_user_children": 0.0,
    "used_cpu_sys_main_thread": 0.0,
    "used_cpu_user_main_thread": 0.0,
    "cluster_enabled": 0,
    "db0": {
        "keys": 13,
        "expires": 0,
        "avg_ttl": 0
    }
}

Solutions:

  • My apologies for not offering a more complete solution, but I tried to find another way to query the connection itself from python, but the documentation for the code used is a bit sparse.
  • Comment out datastore/providers/redis_datastore.py line 98, which calls _check_redis_module_exist. This will allow the connection to happen.
  • Find another way to check for the modules. In this case, I am still not sure if it will work since not everything in REDIS_REQUIRED_MODULES = [ {"name": "search", "ver": 20600}, {"name": "ReJSON", "ver": 20404} ] is inside of the output from the MODULE LIST output above. search may be replaced/interchangeable with searchlight
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant