Config v6 (#50)

* Expose Python 3.11 support * add a tox.ini config for easier local testing * Drop unsupported Python versions and update syntax accordingly * Add Flake8 linting to tox * Fix semver deprecations warnings * Export typing using PEP-561 py.typed * Add typing check and expose linting settings * Simplify github actions workflow - run test and coverage in a single pass - use official codecov action to upload coverage - add typing analysis - run flake8 once and rely on settings * Fix all lints * Update contributing guide and document `tox` usage * Ignore python:S4790 intentional Sonar errors * reverse commit: Drop unsupported Python versions and update syntax accordingly * update CONTRIBUTING.md * remove MyPy check * universal bdist_wheel * intro config v6 json format * add comparators * conditions check * dependent flag logging into the same log_entries * configclient get_config fixes * dependency loop check * testmatrix comparators_v6 * testmatrix segments * testmatrix dependent flag * testmatrix: AndOr * dependent flag logging * comments * TODO: percentage_rule_attribute * sdk key validation check * percentage_rule_attribute log * move sha256 calculation into a function * finalize no percentage_rule_attribute error handling * introduce typed value in override + test fixes * linter fixes * cleanup * github test fix * custom percentage attribute * IS NOT IN SEGMENT fix * operator updates * update tests * lint fixes * circular dependency test * new evaluation logging (WIP) * github action: python 2.7 support * fix user json key order on python 2.7 * Remove the u prefix from unicode strings on python 2.7 in the eval log tests * evaluation logging * lint fixes * fix tests * test_options_within_targeting_rule * lint fix * typo fix * handling the modified config json format * evaluation log test + generator + data * lint fix * Adjust evaluation and update evaluation tests * In case of local only flag overrides mode, we accept any SDK Key format * rename comparators * NOT STARTS WITH ANY OF (hashed), NOT ENDS WITH ANY OF (hashed) comparators * eval log tests: validation error handling * lint fixes * consistent trim logic during evaluation * log fix * test incorrect json * evaluation log update: hashed value + max 10 length lists * indentation fix * matched_evaluation_rule -> matched_targeting_rule, matched_evaluation_percentage_rule -> matched_percentage_rule * evallogging: list logging fix * update matrix tests + eval log tests * Fix list_truncation.txt * Turn off python 2.7 build * attr_value_from_datetime + attr_value_from_list * inline salt, segment for handling flag override * fix python 3.5 test * check if the prerequisite key exists * remove unnecessary served_value get * add `any of` tests to testmatrix_comparators_v6 * rename config members (comparision_rule -> user_condition, segment_rule -> segment_condition) * config descriptor * github-ci fix * fix python 2.7 tests * github ci: python 3.5 * Adjust config model and tests to config v6 schema changes * matrix test update + new cleartext comparators * unicode tests * segments_old matrix test + segment eval log fix * python 2.7 unicode support * fix evaluation log test on python <= 3.5 * type mismatched user attribute warning * python 3.12 support * lint fix + review fixes * python 2.7 fix * github ci: win test fix * review fixes + exceptions in prerequisite flag evaluation * lint fix * don't force users to pass user object attributes as strings * test: evaluation_details_matched_evaluation_rule_and_percentage_option * DefaultValue and SettingType mismatch warning * forced setting_type check * python 2.7 support (timezone, unicode/str handling) * Read cache upon each setting read (lazy cache fix) * fix coding convention * bump version 9.0.0 * ConfigService's refresh offline warning * comment updates * remove unsupported value error log --------- Co-authored-by: Axel H <[email protected]>
configcat · Nov 30, 2023 · 1aed3cb · 1aed3cb
1 parent 241805f
commit 1aed3cb
Show file tree

Hide file tree

Showing 114 changed files with 3,715 additions and 721 deletions.
diff --git a/.github/workflows/python-ci.yml b/.github/workflows/python-ci.yml
@@ -14,10 +14,11 @@ on:
 jobs:
   test:
 
-    runs-on: ubuntu-20.04
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        python-version: ["2.7", "3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["2.7", "3.5", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
+        os: [ windows-latest, macos-latest, ubuntu-20.04 ]
 
     steps:
     - uses: actions/checkout@v3
@@ -36,7 +37,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip 
-        pip install pytest pytest-cov mock flake8
+        pip install pytest pytest-cov parameterized mock flake8
         pip install -r requirements.txt
 
     - name: Lint with flake8

diff --git a/configcatclient/config.py b/configcatclient/config.py
@@ -0,0 +1,293 @@
+from enum import IntEnum
+
+CONFIG_FILE_NAME = 'config_v6'
+SERIALIZATION_FORMAT_VERSION = 'v2'
+
+# Config
+PREFERENCES = 'p'
+SEGMENTS = 's'
+FEATURE_FLAGS = 'f'
+
+# Preferences
+BASE_URL = 'u'
+REDIRECT = 'r'
+SALT = 's'
+
+# Segment
+SEGMENT_NAME = 'n'  # The first 4 characters of the Segment's name
+SEGMENT_CONDITIONS = 'r'  # The list of segment rule conditions (has a logical AND relation between the items).
+
+# Segment Condition (User Condition)
+COMPARISON_ATTRIBUTE = 'a'  # The attribute of the user object that should be used to evaluate this rule
+COMPARATOR = 'c'
+
+# Feature flag (Evaluation Formula)
+SETTING_TYPE = 't'  # 0 = bool, 1 = string, 2 = int, 3 = double
+PERCENTAGE_RULE_ATTRIBUTE = 'a'  # Percentage rule evaluation hashes this attribute of the User object to calculate the buckets
+TARGETING_RULES = 'r'  # Targeting Rules (Logically connected by OR)
+PERCENTAGE_OPTIONS = 'p'  # Percentage Options without conditions
+VALUE = 'v'
+VARIATION_ID = 'i'
+INLINE_SALT = 'inline_salt'
+
+# Targeting Rule (Evaluation Rule)
+CONDITIONS = 'c'
+SERVED_VALUE = 's'  # Value and Variation ID
+TARGETING_RULE_PERCENTAGE_OPTIONS = 'p'
+
+# Condition
+USER_CONDITION = 'u'
+SEGMENT_CONDITION = 's'  # Segment targeting rule
+PREREQUISITE_FLAG_CONDITION = 'p'  # Prerequisite flag targeting rule
+
+# Segment Condition
+SEGMENT_INDEX = 's'
+SEGMENT_COMPARATOR = 'c'
+INLINE_SEGMENT = 'inline_segment'
+
+# Prerequisite Flag Condition
+PREREQUISITE_FLAG_KEY = 'f'
+PREREQUISITE_COMPARATOR = 'c'
+
+# Percentage Option
+PERCENTAGE = 'p'
+
+# Value
+BOOL_VALUE = 'b'
+STRING_VALUE = 's'
+INT_VALUE = 'i'
+DOUBLE_VALUE = 'd'
+STRING_LIST_VALUE = 'l'
+UNSUPPORTED_VALUE = 'unsupported_value'
+
+
+def get_value(dictionary, setting_type):
+    value_descriptor = dictionary.get(VALUE)
+    if value_descriptor is None:
+        raise ValueError('Value is missing')
+
+    if setting_type not in list(map(int, SettingType)):
+        raise ValueError('Unsupported setting type')
+
+    expected_value_type, expected_py_type = SettingType.get_type_info(setting_type)
+    if expected_value_type is None:
+        raise ValueError('Unsupported setting type')
+
+    value = value_descriptor.get(expected_value_type)
+    if value is None:
+        raise ValueError('Setting value is not of the expected type %s' % expected_py_type)
+
+    return value
+
+
+def get_value_type(dictionary):
+    value = dictionary.get(VALUE)
+    if value is not None:
+        if value.get(BOOL_VALUE) is not None:
+            return bool
+        if value.get(STRING_VALUE) is not None:
+            return str
+        if value.get(INT_VALUE) is not None:
+            return int
+        if value.get(DOUBLE_VALUE) is not None:
+            return float
+
+    return None
+
+
+class SettingType(IntEnum):
+    BOOL = 0
+    STRING = 1
+    INT = 2
+    DOUBLE = 3
+
+    @staticmethod
+    def get_type_info(setting_type):
+        return setting_type_mapping.get(setting_type, (None, None))
+
+    @staticmethod
+    def from_type(object_type):
+        if object_type is bool:
+            return SettingType.BOOL
+        if object_type is str:
+            return SettingType.STRING
+        if object_type is int:
+            return SettingType.INT
+        if object_type is float:
+            return SettingType.DOUBLE
+
+        return None
+
+    @staticmethod
+    def to_type(setting_type):
+        return SettingType.get_type_info(setting_type)[1]
+
+    @staticmethod
+    def to_value_type(setting_type):
+        return SettingType.get_type_info(setting_type)[0]
+
+
+setting_type_mapping = {
+    SettingType.BOOL: (BOOL_VALUE, bool),
+    SettingType.STRING: (STRING_VALUE, str),
+    SettingType.INT: (INT_VALUE, int),
+    SettingType.DOUBLE: (DOUBLE_VALUE, float)
+}
+
+
+class PrerequisiteComparator(IntEnum):
+    EQUALS = 0
+    NOT_EQUALS = 1
+
+
+class SegmentComparator(IntEnum):
+    IS_IN = 0
+    IS_NOT_IN = 1
+
+
+class Comparator(IntEnum):
+    IS_ONE_OF = 0
+    IS_NOT_ONE_OF = 1
+    CONTAINS_ANY_OF = 2
+    NOT_CONTAINS_ANY_OF = 3
+    IS_ONE_OF_SEMVER = 4
+    IS_NOT_ONE_OF_SEMVER = 5
+    LESS_THAN_SEMVER = 6
+    LESS_THAN_OR_EQUAL_SEMVER = 7
+    GREATER_THAN_SEMVER = 8
+    GREATER_THAN_OR_EQUAL_SEMVER = 9
+    EQUALS_NUMBER = 10
+    NOT_EQUALS_NUMBER = 11
+    LESS_THAN_NUMBER = 12
+    LESS_THAN_OR_EQUAL_NUMBER = 13
+    GREATER_THAN_NUMBER = 14
+    GREATER_THAN_OR_EQUAL_NUMBER = 15
+    IS_ONE_OF_HASHED = 16
+    IS_NOT_ONE_OF_HASHED = 17
+    BEFORE_DATETIME = 18
+    AFTER_DATETIME = 19
+    EQUALS_HASHED = 20
+    NOT_EQUALS_HASHED = 21
+    STARTS_WITH_ANY_OF_HASHED = 22
+    NOT_STARTS_WITH_ANY_OF_HASHED = 23
+    ENDS_WITH_ANY_OF_HASHED = 24
+    NOT_ENDS_WITH_ANY_OF_HASHED = 25
+    ARRAY_CONTAINS_ANY_OF_HASHED = 26
+    ARRAY_NOT_CONTAINS_ANY_OF_HASHED = 27
+    EQUALS = 28
+    NOT_EQUALS = 29
+    STARTS_WITH_ANY_OF = 30
+    NOT_STARTS_WITH_ANY_OF = 31
+    ENDS_WITH_ANY_OF = 32
+    NOT_ENDS_WITH_ANY_OF = 33
+    ARRAY_CONTAINS_ANY_OF = 34
+    ARRAY_NOT_CONTAINS_ANY_OF = 35
+
+
+COMPARATOR_TEXTS = [
+    'IS ONE OF',                  # IS_ONE_OF
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF
+    'CONTAINS ANY OF',            # CONTAINS_ANY_OF
+    'NOT CONTAINS ANY OF',        # NOT_CONTAINS_ANY_OF
+    'IS ONE OF',                  # IS_ONE_OF_SEMVER
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF_SEMVER
+    '<',                          # LESS_THAN_SEMVER
+    '<=',                         # LESS_THAN_OR_EQUAL_SEMVER
+    '>',                          # GREATER_THAN_SEMVER
+    '>=',                         # GREATER_THAN_OR_EQUAL_SEMVER
+    '=',                          # EQUALS_NUMBER
+    '!=',                         # NOT_EQUALS_NUMBER
+    '<',                          # LESS_THAN_NUMBER
+    '<=',                         # LESS_THAN_OR_EQUAL_NUMBER
+    '>',                          # GREATER_THAN_NUMBER
+    '>=',                         # GREATER_THAN_OR_EQUAL_NUMBER
+    'IS ONE OF',                  # IS_ONE_OF_HASHED
+    'IS NOT ONE OF',              # IS_NOT_ONE_OF_HASHED
+    'BEFORE',                     # BEFORE_DATETIME
+    'AFTER',                      # AFTER_DATETIME
+    'EQUALS',                     # EQUALS_HASHED
+    'NOT EQUALS',                 # NOT_EQUALS_HASHED
+    'STARTS WITH ANY OF',         # STARTS_WITH_ANY_OF_HASHED
+    'NOT STARTS WITH ANY OF',     # NOT_STARTS_WITH_ANY_OF_HASHED
+    'ENDS WITH ANY OF',           # ENDS_WITH_ANY_OF_HASHED
+    'NOT ENDS WITH ANY OF',       # NOT_ENDS_WITH_ANY_OF_HASHED
+    'ARRAY CONTAINS ANY OF',      # ARRAY_CONTAINS_ANY_OF_HASHED
+    'ARRAY NOT CONTAINS ANY OF',  # ARRAY_NOT_CONTAINS_ANY_OF_HASHED
+    'EQUALS',                     # EQUALS
+    'NOT EQUALS',                 # NOT_EQUALS
+    'STARTS WITH ANY OF',         # STARTS_WITH_ANY_OF
+    'NOT STARTS WITH ANY OF',     # NOT_STARTS_WITH_ANY_OF
+    'ENDS WITH ANY OF',           # ENDS_WITH_ANY_OF
+    'NOT ENDS WITH ANY OF',       # NOT_ENDS_WITH_ANY_OF
+    'ARRAY CONTAINS ANY OF',      # ARRAY_CONTAINS_ANY_OF
+    'ARRAY NOT CONTAINS ANY OF'   # ARRAY_NOT_CONTAINS_ANY_OF
+]
+COMPARISON_VALUES = [
+    STRING_LIST_VALUE,  # IS_ONE_OF
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF
+    STRING_LIST_VALUE,  # CONTAINS_ANY_OF
+    STRING_LIST_VALUE,  # NOT_CONTAINS_ANY_OF
+    STRING_LIST_VALUE,  # IS_ONE_OF_SEMVER
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF_SEMVER
+    STRING_VALUE,       # LESS_THAN_SEMVER
+    STRING_VALUE,       # LESS_THAN_OR_EQUAL_SEMVER
+    STRING_VALUE,       # GREATER_THAN_SEMVER
+    STRING_VALUE,       # GREATER_THAN_OR_EQUAL_SEMVER
+    DOUBLE_VALUE,       # EQUALS_NUMBER
+    DOUBLE_VALUE,       # NOT_EQUALS_NUMBER
+    DOUBLE_VALUE,       # LESS_THAN_NUMBER
+    DOUBLE_VALUE,       # LESS_THAN_OR_EQUAL_NUMBER
+    DOUBLE_VALUE,       # GREATER_THAN_NUMBER
+    DOUBLE_VALUE,       # GREATER_THAN_OR_EQUAL_NUMBER
+    STRING_LIST_VALUE,  # IS_ONE_OF_HASHED
+    STRING_LIST_VALUE,  # IS_NOT_ONE_OF_HASHED
+    DOUBLE_VALUE,       # BEFORE_DATETIME
+    DOUBLE_VALUE,       # AFTER_DATETIME
+    STRING_VALUE,       # EQUALS_HASHED
+    STRING_VALUE,       # NOT_EQUALS_HASHED
+    STRING_LIST_VALUE,  # STARTS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # NOT_STARTS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ENDS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # NOT_ENDS_WITH_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ARRAY_CONTAINS_ANY_OF_HASHED
+    STRING_LIST_VALUE,  # ARRAY_NOT_CONTAINS_ANY_OF_HASHED
+    STRING_VALUE,       # EQUALS
+    STRING_VALUE,       # NOT_EQUALS
+    STRING_LIST_VALUE,  # STARTS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # NOT_STARTS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # ENDS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # NOT_ENDS_WITH_ANY_OF
+    STRING_LIST_VALUE,  # ARRAY_CONTAINS_ANY_OF
+    STRING_LIST_VALUE   # ARRAY_NOT_CONTAINS_ANY_OF
+]
+SEGMENT_COMPARATOR_TEXTS = ['IS IN SEGMENT', 'IS NOT IN SEGMENT']
+PREREQUISITE_COMPARATOR_TEXTS = ['EQUALS', 'DOES NOT EQUAL']
+
+
+def extend_config_with_inline_salt_and_segment(config):
+    """
+    Adds the inline salt and segment to the config.
+    When using flag overrides, the original salt and segment indexes may become invalid. Therefore, we copy the
+    object references to the locations where they are referenced and use these references instead of the indexes.
+    """
+    salt = config.get(PREFERENCES, {}).get(SALT, '')
+    segments = config.get(SEGMENTS, [])
+    settings = config.get(FEATURE_FLAGS, {})
+    for setting in settings.values():
+        if not isinstance(setting, dict):
+            continue
+
+        # add salt
+        setting[INLINE_SALT] = salt
+
+        # add segment to the segment conditions
+        targeting_rules = setting.get(TARGETING_RULES, [])
+        for targeting_rule in targeting_rules:
+            conditions = targeting_rule.get(CONDITIONS, [])
+            for condition in conditions:
+
+                segment_condition = condition.get(SEGMENT_CONDITION)
+                if segment_condition:
+                    segment_index = segment_condition.get(SEGMENT_INDEX)
+                    segment = segments[segment_index]
+                    segment_condition[INLINE_SEGMENT] = segment