DVC

[Alazaren]

Hi,
I have a question about the DVC integration. AFAIU the DVC is working under the hood of the datum project.
I did two commits to the project but if I run the "dvc status" I have
There are no data or pipelines tracked in this project yet.
See https://dvc.org/doc/start to get started!
I want to add the remote repository to track the dataset versioning. So I guess remote should be added to the git and dvc in the root of the datumaro project. Can't find any doc on this. Or maybe remote is not supported and I need to create a separate folder with dvc initialized and kind of exporting some dataset revisions using Datumaro export.

[vinnamkim]

Hi @Alazaren, Thanks for your interests on our project!

I did some trials for this problem in my local. I tried to create a Datumaro project, add a dataset to the project as follows, and commit the project status.

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum$ datum project create -o test-proj
cd test-proj2023-08-21 13:53:02,906 INFO: Creating project at '/home/vinnamki/datumaro/ws_datum/test-proj'

2023-08-21 13:53:03,409 INFO: Project has been created at '/home/vinnamki/datumaro/ws_datum/test-proj'

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ datum project import -f imagenet_txt ../../tfdseurosat-imagenet_txt.1/
2023-08-21 13:54:34,279 INFO: Checking the source...
2023-08-21 13:54:34,819 INFO: Source 'source-1' with format 'imagenet_txt' has been added to the project

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ datum project commit -m "add eurosat"
100% Adding...|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|1/1 [00:10, 10.83s/file]
100% Adding...|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|1/1 [00:29, 29.93s/file]

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ ls .dvc
cache  config  tmp

I could find that the .dvc directory was created for dataset versioning with datum project. Thus, I tried to checkout the Datumaro project version using datum project checkout command:

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ datum project log
55db0644a4bfc214af079194146bbac044106ca9 add eurosat
d476d0b7310a7b85e99bf6b8bd5b626d069206be Initial commit

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ datum project checkout d476d0b7310a7b85e99bf6b8bd5b626d069206be
D       source-1/                     
(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ ls

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ datum project checkout 55db0644a4bfc214af079194146bbac044106ca9
A       source-1/  
(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ ls                                                                                                                         
source-1            

It correctly behaved as I expect, but I also found that dvc status is not compatible for the Datumaro project:

(datumaro) vinnamki@vinnamki:~/datumaro/ws_datum/test-proj$ dvc status
There are no data or pipelines tracked in this project yet.           
See <https://dvc.org/doc/start> to get started!

Consequently, please use Datumaro project command for project (dataset) versioning. We will try to look further why dvc status command is not compatible with Datumaro project command. Thanks!

[vinnamkim]

I closely read again your question and recognized that you need the DVC remote integration to Datumaro. I think that it is not supported for now. Thanks for letting us this problem! We will add this feature to Datumaro as soon as possible.