V3.0.0 release (#67)

* v2.6.1 (#62) * bump version 2.6.1 * Added tests and example for custom typing for compression (#59) * Added pipelining to methods pop,setdefault (#61) * Added pipelining to methods pop,setdefault * Changed use SET redis command to set and get within one command * Changed housecleaning * Added mypy strict to ci * Changed mypy and python3.12 don't seem to mix well * Added python versions 13 and 14 * Changed seems python 3.14 is not yet available through ubuntu os * Changed seems python3.14 alpha is available * Changed seems python3.14 alpha is available but it's not useable * Changed mypy and python3.13 also don't mix well * Deleted python3.13 from CI, cryptography package fails on ubuntu base * Changed removed and updated comments * Changed changed name of variable * Changed pop method from two operations to single * Added tests, and handling of edge cases * Removed redundant test file * Changed moved import to top of the file * add possibility to pass connection (#63) * added possibility to pass existing redis instance * fix redis is not a generic class * extends test of testing connection * move test to correct place and make test code more verbose --------- Co-authored-by: Jan Vollmer <[email protected]> * Added RedisDict encoding/decoding to JSON * Changed to convention dir layout * Changed moved test to dev-requirements * Changed updated packages for dev/test * Changed path for pylama and mypy * Added PYTHONPATH as env for the unittest * Changed, setup the project for versioning * Changed, handling of version missing pragmatic * Added failed test for json encoding/decoding * Removed EnumType, not available for python3.9 * Added the _default_decoder to the tests * Changed readme, updated links and small changes * Changed removed python2 methods, changed the methods keys,values,items to Iterators * Changed enabled all types are tested test again * Changed mypy import issue * Changed mypy importing issue * Changed mypy imports fun * Changed mypy, imports * Changed add verbose to mypy for debuggin * Changed missed adding the new file to git * Changed revert github action changes * Changed small housekeeping * Changed updated docstrings * Changed initial start for Sphinx docs * Add doctype test to ci * Changed working version of build docs, but still needs work * Changed passes different linters * Added missing methods from dict to RedisDict * Changed skip new method tests for older than python3.9 * Changed handling for new method case other does not adhere to mapping * Added example of nested types * Changed updated path for readme link --------- Co-authored-by: Jan Vollmer <[email protected]>
Attumm · Nov 1, 2024 · 9863e59 · 9863e59
1 parent ca07bc8
commit 9863e59
Show file tree

Hide file tree

Showing 30 changed files with 2,203 additions and 470 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -40,13 +40,23 @@ jobs:
 
       - name: Install test dependencies
         run: |
-          pip install -r test-requirements.txt
+          pip install -r dev-requirements.txt
 
       - name: Run Pylama
         run: |
-          pylama redis_dict.py -i E501,E231
+          python -m pylama -i E501,E231 src
+
+      - name: Run mypy strict
+        run: |
+          mypy
+
+      - name: Doctype Check
+        run: |
+          darglint src/redis_dict/
 
       - name: Run Unit Tests
+        env:
+          PYTHONPATH: src
         run: |
           coverage run -m unittest discover -p "*tests.py"
 

diff --git a/.gitignore b/.gitignore
@@ -7,9 +7,13 @@ build
 dist
 venv
 .venv
+.venv_*
+dev_venv
 
 .hypothesis/
 
-
-.coverage
+.coverage*
 htmlcov
+
+
+.idea/
diff --git a/README.md b/README.md
@@ -1,4 +1,5 @@
 # Redis-dict
+[![PyPI](https://img.shields.io/pypi/v/redis-dict.svg)](https://pypi.org/project/redis-dict/)
 [![CI](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml/badge.svg)](https://github.com/Attumm/redis-dict/actions/workflows/ci.yml)
 [![codecov](https://codecov.io/gh/Attumm/redis-dict/graph/badge.svg?token=Lqs7McQGEs)](https://codecov.io/gh/Attumm/redis-dict)
 [![Downloads](https://static.pepy.tech/badge/redis-dict/month)](https://pepy.tech/project/redis-dict)
@@ -52,7 +53,6 @@ In Redis our example looks like this.
 
 ### Namespaces
 Acting as an identifier for your dictionary across different systems, RedisDict employs namespaces for organized data management. When a namespace isn't specified, "main" becomes the default. Thus allowing for data organization across systems and projects with the same redis instance.
-
 This approach also minimizes the risk of key collisions between different applications, preventing hard-to-debug issues. By leveraging namespaces, RedisDict ensures a cleaner and more maintainable data management experience for developers working on multiple projects.
 
 ## Advanced Features
@@ -101,7 +101,6 @@ dic['gone'] = 'gone in 5 seconds'
 Efficiently batch your requests using the Pipeline feature, which can be easily utilized with a context manager.
 
 ```python
-from redis_dict import RedisDict
 dic = RedisDict(namespace="example")
 
 # one round trip to redis
@@ -229,14 +228,11 @@ This approach optimizes Redis database performance and efficiency by ensuring th
 Following types are supported: 
 `str, int, float, bool, NoneType, list, dict, tuple, set, datetime, date, time, timedelta, Decimal, complex, bytes, UUID, OrderedDict, defaultdict, frozenset`
 ```python
-from redis_dict import RedisDict
-
 from uuid import UUID
 from decimal import Decimal
 from collections import OrderedDict, defaultdict
 from datetime import datetime, date, time, timedelta
 
-
 dic = RedisDict()
 
 dic["string"] = "Hello World"
@@ -265,14 +261,39 @@ dic["default"] = defaultdict(int, {'a': 1, 'b': 2})
 dic["frozen"] = frozenset([1, 2, 3])
 ```
 
+
+
+### Nested types
+Nested Types
+RedisDict supports nested structures with mixed types through JSON serialization. The feature works by utilizing JSON encoding and decoding under the hood. While this represents an upgrade in functionality, the feature is not fully implemented and should be used with caution. For optimal performance, using shallow dictionaries is recommended.
+```python
+from datetime import datetime, timedelta
+
+dic["mixed"] = [1, "foobar", 3.14, [1, 2, 3], datetime.now()]
+
+dic['dic'] = {"elapsed_time": timedelta(hours=60)}
+```
+
+### JSON Encoding - Decoding
+The nested type support in RedisDict is implemented using custom JSON encoders and decoders. These JSON encoders and decoders are built on top of RedisDict's own encoding and decoding functionality, extending it for JSON compatibility. Since JSON serialization was a frequently requested feature, these enhanced encoders and decoders are available for use in other projects:
+```python
+import json
+from datetime import datetime
+from redis_dict import RedisDictJSONDecoder, RedisDictJSONEncoder
+
+data = [1, "foobar", 3.14, [1, 2, 3], datetime.now()]
+encoded = json.dumps(data, cls=RedisDictJSONEncoder)
+result = json.loads(encoded, cls=RedisDictJSONDecoder)
+```
+
+
 ### Extending RedisDict with Custom Types
 
 RedisDict supports custom type serialization. Here's how to add a new type:
 
 
 ```python
 import json
-from redis_dict import RedisDict
 
 class Person:
     def __init__(self, name, age):
@@ -301,43 +322,30 @@ assert result.name == person.name
 assert result.age == person.age
 ```
 
-```python
->>> from datetime import datetime
->>> redis_dict.extends_type(datetime, datetime.isoformat, datetime.fromisoformat)
->>> redis_dict["now"] = datetime.now()
->>> redis_dict
-{'now': datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)}
->>> redis_dict["now"]
-datetime.datetime(2024, 10, 14, 18, 41, 53, 493775)
-```
-
-For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/extend_types_tests.py).
+For more information on [extending types](https://github.com/Attumm/redis-dict/blob/main/tests/unit/extend_types_tests.py).
 ### Redis Encryption
 Setup guide for configuring and utilizing encrypted Redis TLS for redis-dict.
-[Setup guide](https://github.com/Attumm/redis-dict/blob/main/encrypted_redis.MD)
+[Setup guide](https://github.com/Attumm/redis-dict/blob/main/docs/tutorials/encrypted_redis.MD)
 
 ### Redis Storage Encryption
-For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/encrypt_tests.py).
+For storing encrypted data values, it's possible to use extended types. Take a look at this [encrypted test](https://github.com/Attumm/redis-dict/blob/main/tests/unit/encrypt_tests.py).
 
 ### Tests
 The RedisDict library includes a comprehensive suite of tests that ensure its correctness and resilience. The test suite covers various data types, edge cases, and error handling scenarios. It also employs the Hypothesis library for property-based testing, which provides fuzz testing to evaluate the implementation
 
 ### Redis config
 To configure RedisDict using your Redis config.
 
-Configure both the host and port.
+Configure both the host and port. Or configuration with a setting dictionary.
 ```python
 dic = RedisDict(host='127.0.0.1', port=6380)
-```
 
-Configuration with a dictionary.
-```python
 redis_config = {
     'host': '127.0.0.1',
     'port': 6380,
 }
 
-dic = RedisDict(**redis_config)
+confid_dic = RedisDict(**redis_config)
 ```
 
 ## Installation
@@ -348,4 +356,3 @@ pip install redis-dict
 ### Note
 * Please be aware that this project is currently being utilized by various organizations in their production environments. If you have any questions or concerns, feel free to raise issues
 * This project only uses redis as dependency
-
diff --git a/test-requirements.txt → dev-requirements.txt b/test-requirements.txt → dev-requirements.txt
@@ -1,23 +1,32 @@
+astroid==3.2.4
 attrs==22.2.0
 cffi==1.15.1
 coverage==5.5
 cryptography==43.0.1
+darglint==1.8.1
+dill==0.3.9
 exceptiongroup==1.1.1
 future==0.18.3
 hypothesis==6.70.1
+isort==5.13.2
 mccabe==0.7.0
-mypy==1.1.1
+mypy==1.13.0
 mypy-extensions==1.0.0
+platformdirs==4.3.6
 pycodestyle==2.10.0
 pycparser==2.21
 pydocstyle==6.3.0
 pyflakes==3.0.1
 pylama==8.4.1
 pylint==3.2.7
-redis==4.5.4
+redis==5.2.0
+setuptools==75.3.0
 snowballstemmer==2.2.0
 sortedcontainers==2.4.0
 tomli==2.0.1
-types-pyOpenSSL==23.1.0.0
-types-redis==4.5.3.0
-typing_extensions==4.5.0
+tomlkit==0.13.2
+types-cffi==1.16.0.20240331
+types-pyOpenSSL==24.1.0.20240722
+types-redis==4.6.0.20241004
+types-setuptools==75.2.0.20241025
+typing_extensions==4.12.2
diff --git a/encrypted_redis.MD → docs/tutorials/encrypted_redis.MD b/encrypted_redis.MD → docs/tutorials/encrypted_redis.MD
diff --git a/pyproject.toml b/pyproject.toml
@@ -0,0 +1,133 @@
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "redis-dict"
+version = "3.0.0"
+description = "Dictionary with Redis as storage backend"
+authors = [
+    {name = "Melvin Bijman", email = "[email protected]"},
+]
+readme = "README.md"
+
+requires-python = ">=3.8"
+license = {text = "MIT"}
+dependencies = [
+    "redis>=4.0.0",
+]
+classifiers = [
+    "Development Status :: 5 - Production/Stable",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Information Technology",
+    "Intended Audience :: Science/Research",
+    "Topic :: Internet",
+    "Topic :: Scientific/Engineering",
+    "Topic :: Database",
+    "Topic :: System :: Distributed Computing",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+    "Topic :: Software Development :: Object Brokering",
+    "Topic :: Database :: Database Engines/Servers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Typing :: Typed",
+]
+
+keywords = [
+    "redis", "python", "dictionary", "dict", "key-value",
+    "database", "caching", "distributed-computing",
+    "dictionary-interface", "large-datasets",
+    "scientific-computing", "data-persistence",
+    "high-performance", "scalable", "pipelining",
+    "batching", "big-data", "data-types",
+    "distributed-algorithms", "encryption",
+    "data-management",
+]
+
+[project.optional-dependencies]
+dev = [
+    "coverage==5.5",
+    "hypothesis==6.70.1",
+
+    "mypy>=1.8.0",
+    "mypy-extensions>=1.0.0",
+    "types-pyOpenSSL>=24.0.0.0",
+    "types-redis>=4.6.0",
+    "typing_extensions>=4.5.0",
+
+    "pylama>=8.4.1",
+    "pycodestyle==2.10.0",
+    "pydocstyle==6.3.0",
+    "pyflakes==3.0.1",
+    "pylint==3.2.7",
+    "mccabe==0.7.0",
+
+    "attrs==22.2.0",
+    "cffi==1.15.1",
+    "cryptography==43.0.1",
+    "exceptiongroup==1.1.1",
+    "future==0.18.3",
+    "pycparser==2.21",
+    "snowballstemmer==2.2.0",
+    "sortedcontainers==2.4.0",
+    "tomli==2.0.1",
+    "setuptools>=68.0.0",
+    "darglint",
+    "pydocstyle",
+]
+
+docs = [
+    "sphinx",
+    "sphinx-rtd-theme",
+    "sphinx-autodoc-typehints",
+    "tomli",
+    "myst-parser",
+]
+
+
+[tool.setuptools]
+package-dir = {"" = "src"}
+packages = ["redis_dict"]
+
+[tool.setuptools.package-data]
+redis_dict = ["py.typed"]
+
+[tool.coverage.run]
+source = ["redis_dict"]
+branch = true
+
+[tool.coverage.report]
+exclude_lines = [
+    "pragma: no cover",
+    "def __repr__",
+    "if __name__ == .__main__.:",
+    "raise NotImplementedError",
+    "if TYPE_CHECKING:",
+]
+show_missing = true
+
+[tool.mypy]
+python_version = "3.8"
+strict = true
+mypy_path = "src"
+files = ["src"]
+namespace_packages = true
+explicit_package_bases = true
+
+[tool.pylama]
+ignore = "E501,E231"
+skip = "*/.tox/*,*/.env/*,build/*"
+linters = "pycodestyle,pyflakes,mccabe"
+max_line_length = 120
+paths = ["src/redis_dict"]
+
+[project.urls]
+Homepage = "https://github.com/Attumm/redisdict"
+Documentation = "https://github.com/Attumm/redisdict#readme"
+Repository = "https://github.com/Attumm/redisdict.git"
+Changelog = "https://github.com/Attumm/redisdict/releases"
diff --git a/scripts/build_dev_checks.sh b/scripts/build_dev_checks.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+set -e
+
+rm -rf dev_venv
+python3 -m venv .venv_dev
+source .venv_dev/bin/activate
+
+pip install --upgrade pip
+pip install -e ".[dev]"
+
+pip freeze > dev-requirements.txt
+
+# Type Check
+python -m mypy
+
+# Doctype Check
+darglint src/redis_dict/
+
+# Multiple linters
+python -m pylama -i E501,E231 src
+
+# Unit tests
+python -m unittest discover -s tests
+
+# Docstring Check
+# pydocstyle src/redis_dict/
+
+deactivate
diff --git a/scripts/build_docs.sh b/scripts/build_docs.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+set -e
+
+rm -rf docs/Makefile docs/build/* docs/source/*
+
+#python3 -m venv .venv_docs
+
+source .venv_docs/bin/activate
+pip install --upgrade pip
+pip install -e ".[docs]"
+
+pip freeze
+
+python3 scripts/generate_sphinx_config.py
+
+sphinx-apidoc -o docs/source src/redis_dict
+
+cd docs
+make html
+
+echo "Documentation built successfully in docs/build/html/"