Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dependencies, update localization #545

Merged
merged 7 commits into from
Sep 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"VARIANT": "3.11",
// Options
// "NODE_VERSION": "lts/*",
"POETRY_VERSION": "1.8.2"
"POETRY_VERSION": "1.8.3"
}
},
// Set *default* container specific settings.json values on container create.
Expand Down
12 changes: 6 additions & 6 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ repos:
- id: check-added-large-files

- repo: https://github.com/ambv/black
rev: 24.4.2
rev: 24.8.0
hooks:
- id: black
args: [--line-length, '130', --target-version, py38]
Expand All @@ -25,13 +25,13 @@ repos:
args: [--profile, black]

- repo: https://github.com/PyCQA/flake8
rev: 7.1.0
rev: 7.1.1
hooks:
- id: flake8
args: [--max-line-length, '130']

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
rev: v2.13.0
rev: v2.14.0
hooks:
- id: pretty-format-toml
args: [--autofix]
Expand All @@ -42,13 +42,13 @@ repos:
args: [--autofix]

- repo: https://github.com/igorshubovych/markdownlint-cli
rev: v0.41.0
rev: v0.42.0
hooks:
- id: markdownlint
args: [--fix]

- repo: https://github.com/python-poetry/poetry
rev: 1.8.2
rev: 1.8.3
hooks:
- id: poetry-check
- id: poetry-lock
Expand All @@ -57,7 +57,7 @@ repos:
args: [-f, requirements.txt, -o, requirements.txt, --without-hashes]

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.10.1
rev: v1.11.2
hooks:
- id: mypy
additional_dependencies: [types-requests]
1,072 changes: 580 additions & 492 deletions poetry.lock

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,19 @@ name = "tgtg-scanner"
packages = [{include = "tgtg_scanner"}]
readme = "README.md"
repository = "https://github.com/Der-Henning/tgtg"
version = "1.21.2"
version = "1.22.0"

[tool.poetry.dependencies]
apprise = "^1.4.0"
babel = "^2.16.0"
colorlog = "^6.7.0"
cron-descriptor = "^1.4.0"
discord = "^2.3.2"
googlemaps = "^4.10.0"
humanize = "^4.7.0"
packaging = "^24.0"
progress = "^1.6"
prometheus-client = "^0.20.0"
prometheus-client = "^0.21.0"
pycron = "^3.0.0"
python = ">=3.9,<3.13"
python-pushsafer = "^1.1"
Expand Down
42 changes: 22 additions & 20 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,41 +1,43 @@
aiohttp==3.9.5 ; python_version >= "3.9" and python_version < "3.13"
aiohappyeyeballs==2.4.2 ; python_version >= "3.9" and python_version < "3.13"
aiohttp==3.10.8 ; python_version >= "3.9" and python_version < "3.13"
aiosignal==1.3.1 ; python_version >= "3.9" and python_version < "3.13"
anyio==4.4.0 ; python_version >= "3.9" and python_version < "3.13"
apprise==1.8.0 ; python_version >= "3.9" and python_version < "3.13"
anyio==4.6.0 ; python_version >= "3.9" and python_version < "3.13"
apprise==1.9.0 ; python_version >= "3.9" and python_version < "3.13"
async-timeout==4.0.3 ; python_version >= "3.9" and python_version < "3.11"
attrs==23.2.0 ; python_version >= "3.9" and python_version < "3.13"
cachetools==5.3.3 ; python_version >= "3.9" and python_version < "3.13"
certifi==2024.7.4 ; python_version >= "3.9" and python_version < "3.13"
attrs==24.2.0 ; python_version >= "3.9" and python_version < "3.13"
babel==2.16.0 ; python_version >= "3.9" and python_version < "3.13"
cachetools==5.5.0 ; python_version >= "3.9" and python_version < "3.13"
certifi==2024.8.30 ; python_version >= "3.9" and python_version < "3.13"
charset-normalizer==3.3.2 ; python_version >= "3.9" and python_version < "3.13"
click==8.1.7 ; python_version >= "3.9" and python_version < "3.13"
colorama==0.4.6 ; python_version >= "3.9" and python_version < "3.13" and (sys_platform == "win32" or platform_system == "Windows")
colorlog==6.8.2 ; python_version >= "3.9" and python_version < "3.13"
cron-descriptor==1.4.3 ; python_version >= "3.9" and python_version < "3.13"
cron-descriptor==1.4.5 ; python_version >= "3.9" and python_version < "3.13"
discord-py==2.4.0 ; python_version >= "3.9" and python_version < "3.13"
discord==2.3.2 ; python_version >= "3.9" and python_version < "3.13"
exceptiongroup==1.2.1 ; python_version >= "3.9" and python_version < "3.11"
exceptiongroup==1.2.2 ; python_version >= "3.9" and python_version < "3.11"
frozenlist==1.4.1 ; python_version >= "3.9" and python_version < "3.13"
googlemaps==4.10.0 ; python_version >= "3.9" and python_version < "3.13"
h11==0.14.0 ; python_version >= "3.9" and python_version < "3.13"
httpcore==1.0.5 ; python_version >= "3.9" and python_version < "3.13"
httpx==0.27.0 ; python_version >= "3.9" and python_version < "3.13"
httpx==0.27.2 ; python_version >= "3.9" and python_version < "3.13"
humanize==4.10.0 ; python_version >= "3.9" and python_version < "3.13"
idna==3.7 ; python_version >= "3.9" and python_version < "3.13"
importlib-metadata==8.0.0 ; python_version >= "3.9" and python_version < "3.10"
markdown==3.6 ; python_version >= "3.9" and python_version < "3.13"
multidict==6.0.5 ; python_version >= "3.9" and python_version < "3.13"
idna==3.10 ; python_version >= "3.9" and python_version < "3.13"
importlib-metadata==8.5.0 ; python_version >= "3.9" and python_version < "3.10"
markdown==3.7 ; python_version >= "3.9" and python_version < "3.13"
multidict==6.1.0 ; python_version >= "3.9" and python_version < "3.13"
oauthlib==3.2.2 ; python_version >= "3.9" and python_version < "3.13"
packaging==24.1 ; python_version >= "3.9" and python_version < "3.13"
progress==1.6 ; python_version >= "3.9" and python_version < "3.13"
prometheus-client==0.20.0 ; python_version >= "3.9" and python_version < "3.13"
pycron==3.0.0 ; python_version >= "3.9" and python_version < "3.13"
prometheus-client==0.21.0 ; python_version >= "3.9" and python_version < "3.13"
pycron==3.1.1 ; python_version >= "3.9" and python_version < "3.13"
python-pushsafer==1.1 ; python_version >= "3.9" and python_version < "3.13"
python-telegram-bot[callback-data]==21.4 ; python_version >= "3.9" and python_version < "3.13"
pyyaml==6.0.1 ; python_version >= "3.9" and python_version < "3.13"
python-telegram-bot[callback-data]==21.6 ; python_version >= "3.9" and python_version < "3.13"
pyyaml==6.0.2 ; python_version >= "3.9" and python_version < "3.13"
requests-oauthlib==2.0.0 ; python_version >= "3.9" and python_version < "3.13"
requests==2.32.3 ; python_version >= "3.9" and python_version < "3.13"
sniffio==1.3.1 ; python_version >= "3.9" and python_version < "3.13"
typing-extensions==4.12.2 ; python_version >= "3.9" and python_version < "3.11"
urllib3==2.2.2 ; python_version >= "3.9" and python_version < "3.13"
yarl==1.9.4 ; python_version >= "3.9" and python_version < "3.13"
zipp==3.19.2 ; python_version >= "3.9" and python_version < "3.10"
urllib3==2.2.3 ; python_version >= "3.9" and python_version < "3.13"
yarl==1.13.1 ; python_version >= "3.9" and python_version < "3.13"
zipp==3.20.2 ; python_version >= "3.9" and python_version < "3.10"
4 changes: 2 additions & 2 deletions tests/test_item.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ def test_item(tgtg_item: dict, monkeypatch: pytest.MonkeyPatch):
assert item.item_category == tgtg_item.get("item", {}).get("item_category", "-")
assert item.description == tgtg_item.get("item", {}).get("description", "-")
assert item.link == "https://share.toogoodtogo.com/item/774625"
assert item.price == "3.00"
assert item.value == "9.00"
assert item.price == "3.00"
assert item.value == "9.00"
assert item.currency == "EUR"
assert item.store_name == tgtg_item.get("store", {}).get("store_name", "-")
assert item.item_logo == tgtg_item.get("item", {}).get("logo_picture", {}).get("current_url", "-")
Expand Down
4 changes: 2 additions & 2 deletions tests/test_notifiers.py
Original file line number Diff line number Diff line change
Expand Up @@ -262,8 +262,8 @@ def test_smtp(test_item: Item, reservations: Reservations, favorites: Favorites,
assert body[2] == "From: [email protected]"
assert body[3] == "To: [email protected]"
assert body[4] == "Subject: New Magic Bags"
assert body[7] == 'Content-Type: text/html; charset="utf-8"'
assert body[11] == f"<b>=C3=81 =C3=AA</b> </br>Amount: {test_item.items_available}"
assert body[8] == 'Content-Type: text/html; charset="utf-8"'
assert body[12] == f"<b>=C3=81 =C3=AA</b> </br>Amount: {test_item.items_available}"


@pytest.fixture
Expand Down
103 changes: 62 additions & 41 deletions tgtg_scanner/models/item.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
from http import HTTPStatus
from typing import Any, Union

import babel.numbers
import humanize
import requests

Expand Down Expand Up @@ -52,44 +53,64 @@
returns well formated data for notifications.
"""

def __init__(self, data: dict, location: Union[Location, None] = None):
self.items_available = data.get("items_available", 0)
self.display_name = data.get("display_name", "-")
self.favorite = "Yes" if data.get("favorite", False) else "No"
self.pickup_interval_start = data.get("pickup_interval", {}).get("start", None)
self.pickup_interval_end = data.get("pickup_interval", {}).get("end", None)
self.pickup_location = data.get("pickup_location", {}).get("address", {}).get("address_line", "-")

item = data.get("item", {})
self.item_id = item.get("item_id")
self.rating = item.get("average_overall_rating", {}).get("average_overall_rating", None)
self.rating = "-" if not self.rating else f"{self.rating:.1f}"
self.packaging_option = item.get("packaging_option", "-")
self.item_name = item.get("name", "-")
self.buffet = "Yes" if item.get("buffet", False) else "No"
self.item_category = item.get("item_category", "-")
self.description = item.get("description", "-")
item_price = item.get("item_price", {})
item_value = item.get("item_value", {})
price = item_price.get("minor_units", 0) / 10 ** item_price.get("decimals", 0)
value = item_value.get("minor_units", 0) / 10 ** item_value.get("decimals", 0)
self.price = f"{price:.2f}"
self.value = f"{value:.2f}"
self.currency = item_price.get("code", "-")
self.item_logo = item.get("logo_picture", {}).get(
def __init__(self, data: dict, location: Union[Location, None] = None, locale: str = "en_US"):
self.items_available: int = data.get("items_available", 0)
self.display_name: str = data.get("display_name", "-")
self.favorite: str = "Yes" if data.get("favorite", False) else "No"
self.pickup_interval_start: Union[str, None] = data.get("pickup_interval", {}).get("start", None)
self.pickup_interval_end: Union[str, None] = data.get("pickup_interval", {}).get("end", None)
self.pickup_location: str = data.get("pickup_location", {}).get("address", {}).get("address_line", "-")

item: dict = data.get("item", {})
self.item_id: str = item.get("item_id", None)
self._rating: Union[float, None] = item.get("average_overall_rating", {}).get("average_overall_rating", None)
self.packaging_option: str = item.get("packaging_option", "-")
self.item_name: str = item.get("name", "-")
self.buffet: str = "Yes" if item.get("buffet", False) else "No"
self.item_category: str = item.get("item_category", "-")
self.description: str = item.get("description", "-")
item_price: dict = item.get("item_price", {})
item_value: dict = item.get("item_value", {})
self._price: float = item_price.get("minor_units", 0) / 10 ** item_price.get("decimals", 0)
self._value: float = item_value.get("minor_units", 0) / 10 ** item_value.get("decimals", 0)
self.currency: str = item_price.get("code", "-")
self.item_logo: str = item.get("logo_picture", {}).get(
"current_url",
"https://tgtg-mkt-cms-prod.s3.eu-west-1.amazonaws.com/13512/TGTG_Icon_White_Cirle_1988x1988px_RGB.png",
)
self.item_cover = item.get("cover_picture", {}).get(
self.item_cover: str = item.get("cover_picture", {}).get(
"current_url",
"https://images.tgtg.ninja/standard_images/GENERAL/other1.jpg",
)

store = data.get("store", {})
self.store_name = store.get("store_name", "-")
store: dict = data.get("store", {})
self.store_name: str = store.get("store_name", "-")

self.scanned_on = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self.scanned_on: str = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
self.location = location
self.locale = locale

@property
def rating(self) -> str:
if self._rating is None:
return "-"

Check warning on line 96 in tgtg_scanner/models/item.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/item.py#L96

Added line #L96 was not covered by tests
return self._format_decimal(round(self._rating, 1))

@property
def price(self) -> str:
return self._format_currency(self._price)

@property
def value(self) -> str:
return self._format_currency(self._value)

def _format_decimal(self, number: float) -> str:
return babel.numbers.format_decimal(number, locale=self.locale)

def _format_currency(self, number: float) -> str:
if self.currency == "-":
return self._format_decimal(number)

Check warning on line 112 in tgtg_scanner/models/item.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/item.py#L112

Added line #L112 was not covered by tests
return babel.numbers.format_currency(number, self.currency, locale=self.locale)

@staticmethod
def _datetimeparse(datestr: str) -> datetime.datetime:
Expand Down Expand Up @@ -155,18 +176,18 @@
"""
Returns a well formated string, providing the pickup time range
"""
if self.pickup_interval_start and self.pickup_interval_end:
now = datetime.datetime.now()
pfr = self._datetimeparse(self.pickup_interval_start)
pto = self._datetimeparse(self.pickup_interval_end)
prange = f"{pfr.hour:02d}:{pfr.minute:02d} - {pto.hour:02d}:{pto.minute:02d}"
tommorow = now + datetime.timedelta(days=1)
if now.date() == pfr.date():
return f"{humanize.naturalday(now)}, {prange}"
if (pfr.date() - now.date()).days == 1:
return f"{humanize.naturalday(tommorow)}, {prange}"
return f"{pfr.day}/{pfr.month}, {prange}"
return "-"
if self.pickup_interval_start is None or self.pickup_interval_end is None:
return "-"

Check warning on line 180 in tgtg_scanner/models/item.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/item.py#L180

Added line #L180 was not covered by tests
now = datetime.datetime.now()
pfr = self._datetimeparse(self.pickup_interval_start)
pto = self._datetimeparse(self.pickup_interval_end)
prange = f"{pfr.hour:02d}:{pfr.minute:02d} - {pto.hour:02d}:{pto.minute:02d}"
tommorow = now + datetime.timedelta(days=1)
if now.date() == pfr.date():
return f"{humanize.naturalday(now)}, {prange}"

Check warning on line 187 in tgtg_scanner/models/item.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/item.py#L187

Added line #L187 was not covered by tests
if (pfr.date() - now.date()).days == 1:
return f"{humanize.naturalday(tommorow)}, {prange}"

Check warning on line 189 in tgtg_scanner/models/item.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/item.py#L189

Added line #L189 was not covered by tests
return f"{pfr.day}/{pfr.month}, {prange}"

def _get_distance_time(self, travel_mode: str) -> Union[DistanceTime, None]:
if self.location is None:
Expand Down
4 changes: 2 additions & 2 deletions tgtg_scanner/models/metrics.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
"""
try:
self.item_count.labels(item.item_id, item.display_name).set(item.items_available)
self.item_price.labels(item.item_id, item.display_name).set(float(item.price))
self.item_value.labels(item.item_id, item.display_name).set(float(item.value))
self.item_price.labels(item.item_id, item.display_name).set(item._price)
self.item_value.labels(item.item_id, item.display_name).set(item._value)

Check warning on line 44 in tgtg_scanner/models/metrics.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/models/metrics.py#L43-L44

Added lines #L43 - L44 were not covered by tests
except ValueError as err:
log.warning("Error updating metrics: %s", err)
2 changes: 1 addition & 1 deletion tgtg_scanner/notifiers/discord.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@
item = self.queue.get(block=False)
if item is None:
self.bot.dispatch("close")
await self.bot.close()

Check warning on line 72 in tgtg_scanner/notifiers/discord.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/notifiers/discord.py#L72

Added line #L72 was not covered by tests
return
log.debug("Sending %s Notification", self.name)
await self._send(item)
Expand All @@ -89,7 +90,6 @@
# Commands are handled separately, in case commands are not enabled
self._setup_commands()
asyncio.run(self.bot.start(self.token))
self.bot.http.connector.close()

def _setup_events(self):
@self.bot.event
Expand Down
6 changes: 4 additions & 2 deletions tgtg_scanner/notifiers/smtp.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import smtplib
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.utils import formatdate
from smtplib import SMTPException, SMTPServerDisconnected
from typing import Union

Expand Down Expand Up @@ -93,7 +94,7 @@ def _stay_connected(self) -> None:
if status != 250:
self._connect()

def _send_mail(self, subject: str, html: str, item_id: int) -> None:
def _send_mail(self, subject: str, html: str, item_id: str) -> None:
"""Sends mail with html body"""
if self.server is None:
self._connect()
Expand All @@ -104,10 +105,11 @@ def _send_mail(self, subject: str, html: str, item_id: int) -> None:

# Contains either the main recipient(s) or recipient(s) that should be
# notified for the specific item. First, initalize with main recipient(s)
recipients = self.item_recipients.get(str(item_id), self.recipients)
recipients = self.item_recipients.get(item_id, self.recipients)

message["To"] = ", ".join(recipients)
message["Subject"] = subject
message["Date"] = formatdate(localtime=True)
message.attach(MIMEText(html, "html", "utf-8"))
body = message.as_string()
self._stay_connected()
Expand Down
6 changes: 3 additions & 3 deletions tgtg_scanner/scanner.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@
return items[0]
items = sorted(
[
Item(item, self.location)
Item(item, self.location, self.config.locale)
for item in self.tgtg_client.get_items(favorites_only=False, latitude=53.5511, longitude=9.9937, radius=50)
],
key=lambda x: x.items_available,
Expand All @@ -100,7 +100,7 @@
try:
if item_id != "":
item_dict = self.tgtg_client.get_item(item_id)
items.append(Item(item_dict, self.location))
items.append(Item(item_dict, self.location, self.config.locale))

Check warning on line 103 in tgtg_scanner/scanner.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/scanner.py#L103

Added line #L103 was not covered by tests
except TgtgAPIError as err:
log.error(err)
items += self._get_favorites()
Expand Down Expand Up @@ -133,7 +133,7 @@
except TgtgAPIError as err:
log.error(err)
return []
return [Item(item, self.location) for item in items]
return [Item(item, self.location, self.config.locale) for item in items]

Check warning on line 136 in tgtg_scanner/scanner.py

View check run for this annotation

Codecov / codecov/patch

tgtg_scanner/scanner.py#L136

Added line #L136 was not covered by tests

def _check_item(self, item: Item) -> None:
"""
Expand Down
2 changes: 1 addition & 1 deletion wiki/Configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ You can combine multiple crons as semicolon separated list.
| ScheduleCron | SCHEDULE_CRON | run only on schedule | `* * * * *` |
| ItemIDs | ITEM_IDS | **Depreciated!** comma-separated list of additional (none favorite) items to scan | |
| Metrics | METRICS | enable Prometheus metrics HTTP server | `false` |
| MetricsPort | METRICS_PORTS | port for metrics server | `8000` |
| MetricsPort | METRICS_PORT | port for metrics server | `8000` |
| DisableTests | DISABLE_TESTS | disable test notifications on startup | `false` |
| Quiet | QUIET | minimal console output | `false` |
| Locale | LOCALE | localization | `en_US` |
Expand Down
Loading