api of package

[yonicd]

I was thinking to make the api more flexible than the regular data storage in a package. This limits to how much data can be contributed to the package over time.

I opened a public S3 bucket that doesnt need any keys (basically just a gateway to a bucket for storage)

• placed a dataset in there in json and csv format.
• wrote wrapper functions on the R side to GET data from the bucket
• wrote documentation example for dataset that can be invoked using regular >? notation.

The idea would be that data is fetched once from the bucket for a session then cached in an internal package env that would manage the data

This will give us a naturally scalable way to maintain and access data regardless of language/platform/output to allow for a variety of potential users

What do you think?

cc @danielinteractive @wlandau @chstock

[danielinteractive]

Thanks @yonicd !

I think it is a nice idea in general, because you don't need to store data in the package.

However, I would have a few questions:

• Would this imply that users or R CMD check that want to run e.g. mmrm examples using data from this package need to have internet connection? And succeed connecting with the S3 bucket?
• How many data sets do we expect in the near term to be available in this package?
• How many kb size would each data set have?

[wlandau]

Alternatively, a caching strategy in tools::R_user_dir() could help. You could cache in advance, either through an explicit function call or through memoization, and then use the same interface to get the data from wherever it lives.

[yonicd]

@wlandau that would work in the same way that mocking works?

[wlandau]

Maybe. I was thinking the user could call something like clindata::get_dataset("sleepstudy"), and behind the scenes it could get the data however it needs to, either reading directly from file.path(tools::R_user_dir(package = "clindata", which = "cache"), "sleepstudy.rds"), or downloading to that location if necessary. There could also be functions to download one or more datasets to this cache, as well as one or more functions to clear the cache.

[danielinteractive]

How about starting with the minimal set of requirements here, i.e. just the data saved in the package as usual? And then develop the other functionality afterwards?

[wlandau]

I like that idea. With an interface-level get_dataset()-like function, cloud storage and local caching can be added later if permanent local storage becomes a problem. The encapsulation lets us change the internal business logic back-compatibly.

[yonicd]

created a static version of the data in the package. started with generating bcva using {targets} for reproducibility in the data-raw directory

This way there is a visual graph attached to each data we create that shows how the data is created and a way for anyone to reproduce it programatically.

It will also make it simpler to maintain the package as it potentially scales to multiple datasets

[danielinteractive]

Not sure, for sure it is agnostic but if we now change it we need to rerun all the SAS comparisons. Not sure if it is worth it. Since we have the simulations as well

[yonicd]

We can still use this “snapshot” for sas sims and the reproducible one for real time runs in the packages. I think if we want to promote reproducibility as a best practice we should deploy it across the packages.

timing-wise this would be the juncture in the development of brms.mmrm to implement a reproducible data it depends on going fw.

[danielinteractive]

yeah if we can continue using the same old "snapshot" for tests then it would be ok. I just would like to avoid retyping all the test expectations ...

[yonicd]

I put the snapshot of fev_data as the internal data for the package, it is called fev_sim_data

We can discuss in another discussion issue what is the proper way to simulate the fev data

[yonicd]

@wlandau I put in the basic functionality of the api to get and list data from within the package.