Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Extract CatalogClient from flytepropeller to flytestdlib #148

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

MorpheusXAUT
Copy link
Contributor

@MorpheusXAUT MorpheusXAUT commented Dec 15, 2022

TL;DR

The CatalogClient implementation has been extracted from flytepropeller's source code and moved to flytestdlib as flyteadmin also needs access to the functionality.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

The whole latest implementation of the catalog package was moved from flytepropeller to here since flyteadmin will need access to a client interacting with datacatalog for evicting cached executions.
Additionally, the new functionality for acquiring/releasing reservations via artifact tag as well as the deletion of artifacts was implemented for the datacatalog client.

The PR was created as a draft until the flyteidl and flyteplugins PRs have been merged and respective versions have been published.

Tracking Issue

flyteorg/flyte#2867

Follow-up issue

NA

@codecov
Copy link

codecov bot commented Dec 15, 2022

Codecov Report

Merging #148 (d6ec6db) into master (80d1624) will decrease coverage by 0.49%.
The diff coverage is 64.50%.

@@            Coverage Diff             @@
##           master     #148      +/-   ##
==========================================
- Coverage   67.94%   67.45%   -0.50%     
==========================================
  Files          69       75       +6     
  Lines        4077     4756     +679     
==========================================
+ Hits         2770     3208     +438     
- Misses       1148     1348     +200     
- Partials      159      200      +41     
Flag Coverage Δ
unittests 67.45% <64.50%> (-0.50%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
catalog/client.go 0.00% <0.00%> (ø)
catalog/config.go 0.00% <0.00%> (ø)
catalog/noop_catalog.go 0.00% <0.00%> (ø)
catalog/config_flags.go 30.00% <30.00%> (ø)
catalog/datacatalog/datacatalog.go 63.35% <63.35%> (ø)
catalog/datacatalog/transformer.go 80.71% <80.71%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Implemented new datacatalog functionality required for cache eviction
Updated to latest unreleased version of flyteidl and flyteplugins

Signed-off-by: Nick Müller <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant