- Update
cffi
version to allow python3.12 support, addsetuptools
to requirements
- Add metrics with count if running and dead processes
- Changed default export period to 0.5 to fix metrics gaps
- Lower cpu usage in main process
- Add memory usage metrics
- restore signals for successful termination of processes during flow.stop() if they were overridden
- Update setup requirements: add extras - aiohttp, numpy
- Add Priority queues
- Route task to a specific worker queue according to handle_condition
- Default start timeout in aiohttp integration
- Check timeout when try to put value to queue
- Fix deadlock on exit when input queue has items
- Add a new async method share_value_with_data to the BaseTask class
- add support type bytes or bytearray for share memory in BaseTask
- Change name
on_start_timeout
toon_start_wait
- Add ability to set different subprocess start methods and on_start_timeout for heavyweight models
- Add metric for task size in bytes
- Add ability to set custom logger
- Add missing imports to aiohttp integration and add return from observe_flows
- fixed bad upload to pypi.org
- fixed unfinished stopping on child process death
- fixed reraise asyncio.CancelledError if the cause of exception is external actor
- fixed batch gatharing (batch was not always accumulate properly due to MP queue inconsistent behaviour)
- fixed batch time measurement
- added dynamic batching strategy (used if batch_timeout = 0)
- added
pulemet
as external dependency
- fix CPU consumption in subprocesses
- fix multiprocessing start method setting, imports order
- fix shared memory reference counting
- fix aiohttp integration (monitoring task is not affecting unit tests now)
Flow.run()
now waiting for all child processes to initialize (allHandler.on_start()
methods guaranteed to be completed beforerun
releases)
- tests migrated from pytest-asyncio to pytest-aiohttp
- added integration with aiohttp
- removed redundant files
- added
python_requires
parameter insetup.py
- added
examples
directory - renamed
lib
directory toaqueduct
- added StatsDBuffer Protocol
- added metrics sending in the
maas
service intests/benchmarks/bench_service
- reduced time interval for queue size metrics collection
- added benchmark tests
- moved unit tests
- added batching metrics: batch size and batch creating time
- metrics refactoring
- input and output Worker queues moved from
loop
to__init__
method
- moved avio-specified metrics prefix from Flow to Exporter
- changed queue size metrics type from gauge to timers
- added metrics for tasks count on completed and failed
- added queue_size parameter
- fixed typing
- added multiprocess batching support
- added the ability to skip unsuitable for Worker tasks
- removed
async-timeout
requirement - added batching mechanism to the
Worker
class - changed
handle
method of theBaseTaskHandler
class for tasks batch handling - tests refactoring
- fixed segfault error
- added shared fields support in the BaseTask
- removed
to_dict
BaseTask method
- fixed segmentation fault problems due to task update
- source task now updating with result
- process() method returns True if work on task completed
need_collect_task_timers
Flow property changed to cached_property- added the ability to disable the collecting metrics
- fixed unnecessary shared memory warnings
- fixed asleep main process due to huge count of tasks
- changed perf tests
- renamed
SharedData
method fromget_from_data
tocreate_from_data
- fixed ref counter
- added shared memory tools
- python version raised to
3.8.6
- added child processes explicit killing
- changed Flow
start
method from async to sync - added Flow
is_running
property
- removed event loop stopping after Flow stopping
- added StopTask forwarding to all steps
- updated multiprocessing module
- added environment checks
- revised Aqueduct errors
- removed pytorch dependency
- revised Flow stopping
- change queues to multiprocessing.Queue
- add processes liveness checking
- fixed
_task_states
clearing
- add timeout_sec for process method
- spawner moved from Torch and tuned for logger integration like Sentry
- new few
__str__
and__repr__
for more readable exceptions context - add some graceful terminating
- change queue size metrics type from counter to gauge
- removed text-detector specified code from Flow
- not None _metrics_manager field in Flow
- change metrics package structure
- add MetricsManager to encapsulate collect and export logic
- add queue size metrics
- add metrics prefixes
- add metrics export config
- add step name to Worker name
- change TaskMetrics.timers from dict to list
- change timer function from time() to monotonic()
- add timer metrics for task transfer and task handling
- first version based on shared memory via pytorch multiprocessing modules