Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
2a83e84
this was amazingly a 1 line flip to get parity in existing tests.
mstackhouse Jun 3, 2024
bc68a47
For the row vectors to write out per standard, they have to be unnamed
mstackhouse Jun 3, 2024
aeeab5b
Flip writer to yyjsonr
mstackhouse Jun 3, 2024
d31433c
Allow file to go directly to yyjsonr reader
mstackhouse Jun 3, 2024
35ac486
docs and deps
mstackhouse Jun 3, 2024
6ebd048
Increment yyjsonr version for CI/CD failures
mstackhouse Jun 3, 2024
b12bdd5
whoops
mstackhouse Jun 3, 2024
8306fd6
Typos galore
mstackhouse Jun 3, 2024
57ecddc
Update R/read_dataset_json.R
mstackhouse Jun 20, 2024
2c64948
remove suggested jsonlite
mstackhouse Jun 20, 2024
f033c3b
Merge branch 'yyjsonr_switch' of github.com:atorus-research/datasetjs…
mstackhouse Jun 20, 2024
922de64
Roll back R version add back jsonlite to suggests
mstackhouse Jun 20, 2024
e0a1f71
Merge pull request #33 from atorus-research/yyjsonr_switch
nicholas-masel Jun 21, 2024
cf269f0
Data dump from latest datasetjson repo
mstackhouse Sep 5, 2024
ece3c00
docs
mstackhouse Sep 5, 2024
572ad97
Update metadata Rds files
mstackhouse Sep 5, 2024
a55deda
Merge pull request #41 from atorus-research/gh_issue_40
nicholas-masel Sep 5, 2024
a32f53d
New schema
mstackhouse Sep 5, 2024
64c2987
save progress in reformatting dataset metadata
mstackhouse Sep 5, 2024
945b662
Modifications for new schema
mstackhouse Sep 5, 2024
3ace7ac
archive
mstackhouse Sep 5, 2024
70f4aa3
Update docs
mstackhouse Sep 5, 2024
836b803
Update R/datasetjson.R
mstackhouse Sep 5, 2024
2dc4c43
Merge pull request #45 from atorus-research/gh_issue_43
mstackhouse Sep 5, 2024
ab1df19
initial pass at v1.1.0 updates
nicholas-masel Sep 5, 2024
6de009c
Updates from discussion
mstackhouse Sep 5, 2024
7d4f4ba
docs and cleanup
mstackhouse Sep 5, 2024
2cca514
Merge pull request #47 from atorus-research/gh_issue_43
nicholas-masel Sep 5, 2024
bad3577
address comment from discussion
nicholas-masel Sep 5, 2024
13c056f
Merge pull request #48 from atorus-research/dev
nicholas-masel Sep 5, 2024
9cbe86e
fix object overwrite
nicholas-masel Sep 5, 2024
4a5ccad
write updates
nicholas-masel Sep 16, 2024
8f7adda
cleanup
nicholas-masel Sep 16, 2024
680111f
put ndjson in a separate function
nicholas-masel Sep 17, 2024
41ca676
add write_dataset_ndjson()
nicholas-masel Sep 17, 2024
a7f7b5a
add ndjson read
nicholas-masel Sep 17, 2024
a65ae77
stream as list instead of df since names are not on rows
nicholas-masel Sep 17, 2024
5702e30
Column metasdata arguments
mstackhouse Sep 19, 2024
a9a8555
remove ndjson for now since streaming isn't clear w/yyjsonr and this …
nicholas-masel Sep 26, 2024
8c3cd3a
Merge pull request #52 from atorus-research/gh_issue_43
nicholas-masel Sep 26, 2024
3c7d002
column metadata update for new schema
nicholas-masel Sep 26, 2024
9ffe789
fix column setter
nicholas-masel Sep 26, 2024
c542cf9
read/write updates to account for column metadata already being on th…
nicholas-masel Sep 26, 2024
9ff7f46
datajson testing
nicholas-masel Sep 26, 2024
5140f73
removed tests for data_metadata, dataset_metadata, time conversions, …
nicholas-masel Sep 26, 2024
9b8890b
minor update
nicholas-masel Jan 21, 2025
7eabb1b
Merge pull request #46 from atorus-research/42-pull-variable-level-me…
mstackhouse Jan 21, 2025
97660db
initial pass to remove isReferenceData and ITEMGROUPDATASEQ
nicholas-masel Jan 21, 2025
0e6e60e
Closes #53
mstackhouse Jan 21, 2025
cea1257
More data conversions for new format
mstackhouse Jan 21, 2025
3a0146d
Merge pull request #55 from atorus-research/gh_issue_53
nicholas-masel Jan 21, 2025
d0d4d31
No longer necessary
mstackhouse Jan 21, 2025
9858072
Updates to rewrite testdata metadata files based on 1.1 updates
mstackhouse Jan 21, 2025
a82645d
Test updates based on data updates
mstackhouse Jan 21, 2025
e74249d
Merge pull request #56 from atorus-research/mike_updates
nicholas-masel Jan 21, 2025
6b11425
Merge pull request #54 from atorus-research/remove-isReferenceData-an…
mstackhouse Jan 21, 2025
79bfa75
add date/time conversion function
nicholas-masel Jan 21, 2025
10cf78e
Obsolete
mstackhouse Jan 21, 2025
213df23
doc sweep
mstackhouse Jan 21, 2025
a9c055e
Failing example
mstackhouse Jan 21, 2025
b5ed87a
test updates for times and datetimes in reader
mstackhouse Jan 21, 2025
a1b81f4
Merge pull request #59 from atorus-research/doc_updates
mstackhouse Jan 22, 2025
6c42375
update write to convert numeric dates to character
nicholas-masel Jan 22, 2025
4a3d5df
Merge branch '57-update-date-datetime-and-time' of github.com:atorus-…
mstackhouse Jan 22, 2025
87191c8
Save updates
mstackhouse Jan 22, 2025
2c7ca26
Get datetime read/write working
mstackhouse Jan 22, 2025
16fb44f
Increment version
mstackhouse Jan 22, 2025
d3539a0
We need POSIXlt on right to check timezone attributes, so I want read…
mstackhouse Jan 22, 2025
93d74c0
Revamp and introduce error checking
mstackhouse Jan 22, 2025
bd66a5d
test updates and data prep corrections
mstackhouse Jan 22, 2025
32b6851
Fix types
mstackhouse Jan 22, 2025
227de21
Vignette progress
mstackhouse Jan 22, 2025
09088ca
Pull in creation time
mstackhouse Jan 23, 2025
b913ac2
Example XPT files
mstackhouse Jan 23, 2025
8c8ea93
Save more writing
mstackhouse Jan 23, 2025
32688b6
Merge branch 'dev' of github.com:atorus-research/datasetjson into 57-…
mstackhouse Jan 23, 2025
7297ccc
doc updates
mstackhouse Jan 23, 2025
88c550b
Add extra test for when optional attributes are missing
mstackhouse Jan 23, 2025
cccff2f
Finish writing vignette
mstackhouse Jan 23, 2025
cbdd144
pkgdown updates
mstackhouse Jan 23, 2025
a184609
catch display format
mstackhouse Jan 23, 2025
d16ded8
R CMD check fixes
mstackhouse Jan 23, 2025
77ff4fb
vignette updates and new helper functions
mstackhouse Jan 23, 2025
702ff1e
Trying to fix build issue
mstackhouse Jan 24, 2025
e34d1a1
Test more relevant OS's
mstackhouse Jan 24, 2025
25f3f3f
Update R/write_dataset_json.R
mstackhouse Jan 24, 2025
938dc8c
Update README.Rmd
mstackhouse Jan 24, 2025
e561cec
Add reference to Date class
mstackhouse Jan 24, 2025
4d69f47
POSIXlt to POSIXct
mstackhouse Jan 24, 2025
ff6cda7
update README
mstackhouse Jan 24, 2025
0ee1c14
minor vignette updates
nicholas-masel Jan 24, 2025
1b2d9f6
Merge branch '57-mike-updates' of https://github.com/atorus-research/…
nicholas-masel Jan 24, 2025
ce72401
add hms difftime and data.table ITime as supported time types
mstackhouse Jan 24, 2025
7e79204
Merge branch '57-mike-updates' of github.com:atorus-research/datasetj…
mstackhouse Jan 24, 2025
5477591
Reword the text
mstackhouse Jan 24, 2025
9d29df3
Merge pull request #60 from atorus-research/57-mike-updates
nicholas-masel Jan 24, 2025
51df523
testing added
nicholas-masel Jan 24, 2025
d094c10
Merge pull request #63 from atorus-research/62-get-code-coverage-back…
mstackhouse Jan 24, 2025
dbedb9a
urlchecker
mstackhouse Jan 24, 2025
44e4b62
news and cran comments
mstackhouse Jan 24, 2025
b697724
rhub workflow
mstackhouse Jan 24, 2025
5d5b474
decimal data type conversion from float
mstackhouse Jan 27, 2025
b361f48
testing fixes
mstackhouse Jan 27, 2025
5726b24
vignette explaining new updates
mstackhouse Jan 27, 2025
d431fd6
docs
mstackhouse Jan 27, 2025
e044616
docs for devtools::check() fixes
mstackhouse Jan 27, 2025
c68cf30
Make sure reader respects metadata properly for decimal types and add…
mstackhouse Jan 27, 2025
cf5646a
leftover option
mstackhouse Jan 28, 2025
e933777
Catch other numbers to convert
mstackhouse Jan 28, 2025
6e37de5
Merge pull request #66 from atorus-research/floats_as_decimals
nicholas-masel Jan 28, 2025
58a6cc9
Merge pull request #64 from atorus-research/news_updates
nicholas-masel Jan 28, 2025
386a5a9
Update code and tests to use hms instead of lubridate
mstackhouse Jan 29, 2025
49c3a7d
Catch vignette update
mstackhouse Jan 29, 2025
2162eda
hms as import only
nmasel Jan 29, 2025
5a0f980
If only this was a magical fix
mstackhouse Jan 29, 2025
02bb162
Maybe this?
mstackhouse Jan 29, 2025
6b7318e
Checking if this fixes pipeline
mstackhouse Jan 29, 2025
d3f15d9
Why.
mstackhouse Jan 29, 2025
4bfc01c
Sigh
mstackhouse Jan 29, 2025
a3019a1
Try pulling latest action from r-lib
mstackhouse Jan 29, 2025
660b5b5
Pull lubridate back out now that workflow is fixed
mstackhouse Jan 29, 2025
8cdddbe
Update DESCRIPTION
mstackhouse Jan 29, 2025
e1c4c40
I think we'll eventually get this right.
mstackhouse Jan 29, 2025
954c7a5
Merge pull request #68 from atorus-research/hms_update
mstackhouse Jan 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 22 additions & 48 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# For help debugging build failures open an issue on the RStudio community with the 'github-actions' tag.
# https://community.rstudio.com/new-topic?category=Package%20development&tags=github-actions
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches:
Expand All @@ -12,7 +12,9 @@ on:
- master
- dev

name: R-CMD-check
name: R-CMD-check.yaml

permissions: read-all

jobs:
R-CMD-check:
Expand All @@ -26,60 +28,32 @@ jobs:
config:
- {os: windows-latest, r: 'release'}
- {os: macOS-latest, r: 'release'}
- {os: ubuntu-20.04, r: 'release', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
- {os: ubuntu-20.04, r: 'devel', rspm: "https://packagemanager.rstudio.com/cran/__linux__/focal/latest"}
- {os: ubuntu-22.04, r: 'release', rspm: "https://packagemanager.posit.co/cran/__linux__/jammy/latest"}
- {os: ubuntu-22.04, r: 'devel', rspm: "https://packagemanager.posit.co/cran/__linux__/jammy/latest"}
- {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-latest, r: 'release'}

env:
R_REMOTES_NO_ERRORS_FROM_WARNINGS: true
RSPM: ${{ matrix.config.rspm }}
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes

steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4

- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}
use-public-rspm: true

- uses: r-lib/actions/setup-pandoc@v2

- name: Query dependencies
run: |
install.packages('remotes')
saveRDS(remotes::dev_package_deps(dependencies = TRUE), ".github/depends.Rds", version = 2)
writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version")
shell: Rscript {0}

- name: Cache R packages
if: runner.os != 'Windows'
uses: actions/cache@v2
- uses: r-lib/actions/setup-r-dependencies@v2
with:
path: ${{ env.R_LIBS_USER }}
key: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-${{ hashFiles('.github/depends.Rds') }}
restore-keys: ${{ runner.os }}-${{ hashFiles('.github/R-version') }}-1-

- name: Install system dependencies
if: runner.os == 'Linux'
run: |
while read -r cmd
do
eval sudo $cmd
done < <(Rscript -e 'writeLines(remotes::system_requirements("ubuntu", "20.04"))')

- name: Install dependencies
run: |
remotes::install_deps(dependencies = TRUE)
remotes::install_cran("rcmdcheck")
shell: Rscript {0}

- name: Check
env:
_R_CHECK_CRAN_INCOMING_REMOTE_: false
run: rcmdcheck::rcmdcheck(args = c("--no-manual", "--as-cran"), error_on = "warning", check_dir = "check")
shell: Rscript {0}
extra-packages: any::rcmdcheck
needs: check

- name: Upload check results
if: failure()
uses: actions/upload-artifact@main
- uses: r-lib/actions/check-r-package@v2
with:
name: ${{ runner.os }}-r${{ matrix.config.r }}-results
path: check
upload-snapshots: true
build_args: 'c("--no-manual","--compact-vignettes=gs+qpdf")'
95 changes: 95 additions & 0 deletions .github/workflows/rhub.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# R-hub's generic GitHub Actions workflow file. It's canonical location is at
# https://github.com/r-hub/actions/blob/v1/workflows/rhub.yaml
# You can update this file to a newer version using the rhub2 package:
#
# rhub::rhub_setup()
#
# It is unlikely that you need to modify this file manually.

name: R-hub
run-name: "${{ github.event.inputs.id }}: ${{ github.event.inputs.name || format('Manually run by {0}', github.triggering_actor) }}"

on:
workflow_dispatch:
inputs:
config:
description: 'A comma separated list of R-hub platforms to use.'
type: string
default: 'linux,windows,macos'
name:
description: 'Run name. You can leave this empty now.'
type: string
id:
description: 'Unique ID. You can leave this empty now.'
type: string

jobs:

setup:
runs-on: ubuntu-latest
outputs:
containers: ${{ steps.rhub-setup.outputs.containers }}
platforms: ${{ steps.rhub-setup.outputs.platforms }}

steps:
# NO NEED TO CHECKOUT HERE
- uses: r-hub/actions/setup@v1
with:
config: ${{ github.event.inputs.config }}
id: rhub-setup

linux-containers:
needs: setup
if: ${{ needs.setup.outputs.containers != '[]' }}
runs-on: ubuntu-latest
name: ${{ matrix.config.label }}
strategy:
fail-fast: false
matrix:
config: ${{ fromJson(needs.setup.outputs.containers) }}
container:
image: ${{ matrix.config.container }}

steps:
- uses: r-hub/actions/checkout@v1
- uses: r-hub/actions/platform-info@v1
with:
token: ${{ secrets.RHUB_TOKEN }}
job-config: ${{ matrix.config.job-config }}
- uses: r-hub/actions/setup-deps@v1
with:
token: ${{ secrets.RHUB_TOKEN }}
job-config: ${{ matrix.config.job-config }}
- uses: r-hub/actions/run-check@v1
with:
token: ${{ secrets.RHUB_TOKEN }}
job-config: ${{ matrix.config.job-config }}

other-platforms:
needs: setup
if: ${{ needs.setup.outputs.platforms != '[]' }}
runs-on: ${{ matrix.config.os }}
name: ${{ matrix.config.label }}
strategy:
fail-fast: false
matrix:
config: ${{ fromJson(needs.setup.outputs.platforms) }}

steps:
- uses: r-hub/actions/checkout@v1
- uses: r-hub/actions/setup-r@v1
with:
job-config: ${{ matrix.config.job-config }}
token: ${{ secrets.RHUB_TOKEN }}
- uses: r-hub/actions/platform-info@v1
with:
token: ${{ secrets.RHUB_TOKEN }}
job-config: ${{ matrix.config.job-config }}
- uses: r-hub/actions/setup-deps@v1
with:
job-config: ${{ matrix.config.job-config }}
token: ${{ secrets.RHUB_TOKEN }}
- uses: r-hub/actions/run-check@v1
with:
job-config: ${{ matrix.config.job-config }}
token: ${{ secrets.RHUB_TOKEN }}
25 changes: 16 additions & 9 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: datasetjson
Type: Package
Title: Read and Write CDISC Dataset JSON Files
Version: 0.2.0
Version: 0.3.0
Authors@R: c(
person(given = "Mike",
family = "Stackhouse",
Expand All @@ -13,24 +13,31 @@ Authors@R: c(
email = "[email protected]",
role = "aut")
)
Description: Read, construct and write CDISC (Clinical Data Interchange Standards Consortium) Dataset JSON (JavaScript Object Notation) files, while validating per the Dataset JSON schema file, as described in CDISC (2023) <https://www.cdisc.org/dataset-json>.
URL: https://github.com/atorus-research/datasetjson
BugReports: https://github.com/atorus-research/datasetjson/issues
Description: Read, construct and write CDISC (Clinical Data Interchange Standards Consortium) Dataset JSON (JavaScript Object Notation) files, while validating per the Dataset JSON schema file, as described in CDISC (2023) <https://www.cdisc.org/standards/data-exchange/dataset-json>.
URL: https://atorus-research.github.io/datasetjson/
BugReports: https://github.com/atorus-research/datasetjson/issues/
Encoding: UTF-8
Language: en-US
License: Apache License (>= 2)
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3
Depends: R (>= 3.5)
RoxygenNote: 7.3.2
Depends: R (>= 4.0)
Imports:
jsonlite (>= 1.8.0),
jsonvalidate (>= 1.3.1)
yyjsonr (>= 0.1.18),
jsonvalidate (>= 1.3.1),
hms
Suggests:
testthat (>= 2.1.0),
jsonlite (>= 1.8.0),
knitr,
haven,
rmarkdown,
withr
withr,
purrr,
tibble,
dplyr,
lubridate,
data.table
VignetteBuilder: knitr
Config/testthat/edition: 3
28 changes: 13 additions & 15 deletions NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,30 +1,28 @@
# Generated by roxygen2: do not edit by hand

S3method(set_metadata_ref,data_metadata)
S3method(set_metadata_ref,datasetjson)
S3method(set_metadata_ref,default)
S3method(set_metadata_version,data_metadata)
S3method(set_metadata_version,datasetjson)
S3method(set_metadata_version,default)
S3method(set_study_oid,data_metadata)
S3method(set_study_oid,datasetjson)
S3method(set_study_oid,default)
export(data_metadata)
export(dataset_json)
export(dataset_metadata)
export(file_metadata)
export(get_column_metadata)
export(read_dataset_json)
export(set_data_type)
export(set_dataset_label)
export(set_dataset_name)
export(set_file_oid)
export(set_item_oid)
export(set_last_modified)
export(set_metadata_ref)
export(set_metadata_version)
export(set_originator)
export(set_source_system)
export(set_study_oid)
export(set_variable_attributes)
export(validate_dataset_json)
export(write_dataset_json)
importFrom(jsonlite,fromJSON)
importFrom(jsonlite,toJSON)
importFrom(hms,as_hms)
importFrom(jsonvalidate,json_validate)
importFrom(tools,file_path_sans_ext)
importFrom(utils,tail)
importFrom(yyjsonr,opts_read_json)
importFrom(yyjsonr,opts_write_json)
importFrom(yyjsonr,read_json_file)
importFrom(yyjsonr,read_json_str)
importFrom(yyjsonr,write_json_file)
importFrom(yyjsonr,write_json_str)
11 changes: 11 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
# datasetjson 0.3.0

This release provides a significant overhaul of the package due to the updates
for Dataset JSON 1.1.0. Performance has also been significantly improved, as well
as the main object interface.

- Initial support for Dataset JSON v1.1.0 schema
- Flip JSON backend to {yyjsonr} (#32)
- Redesign of core objects
- New vignettes and helper functions

# datasetjson 0.2.0

- Remove schema validation on read and write (#26)
Expand Down
56 changes: 56 additions & 0 deletions R/archive/data_metadata.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#' Create the data metadata container for a Dataset JSON object
#'
#' @param study Study OID value
#' @param metadata_version Metadata version OID value
#' @param metadata_ref Metadata reference (i.e. path to Define.xml)
#'
#' @return data_metadata object
#' @export
#'
#' @examples
#' # Create object directly
#' data_meta <- data_metadata(
#' study = "SOMESTUDY",
#' metadata_version = "MDV.MSGv2.0.SDTMIG.3.3.SDTM.1.7",
#' metadata_ref = "some/define.xml"
#' )
#'
#' # Use setter functions
#' data_meta <- data_metadata()
#' data_meta_updated <- set_metadata_ref(data_meta, "some/define.xml")
#' data_meta_updated <- set_metadata_version(data_meta_updated, "MDV.MSGv2.0.SDTMIG.3.3.SDTM.1.7")
#' data_meta_updated <- set_study_oid(data_meta_updated, "SOMESTUDY")
#'
data_metadata <- function(study = NULL, metadata_version = NULL, metadata_ref = NULL) {

x <- list(
studyOID = study,
metaDataVersionOID = metadata_version,
metaDataRef = metadata_ref,
itemGroupData = NULL
)

structure(
x,
class = c("data_metadata", "list")
)
}

#' Set data metadata parameters
#'
#' This set of functions
#' @param x data metadata or datasetjson object
#' @param study Study OID value
#' @param ... Additional parameters
#'
#' @return A datasetjson or data_metadata object
#' @export
#'
#' @family Data metadata setters
#' @rdname data_metadata_setters
#'
#' @examples
#' data_meta <- data_metadata()
#' data_meta_updated <- set_metadata_ref(data_meta, "some/define.xml")
#' data_meta_updated <- set_metadata_version(data_meta_updated, "MDV.MSGv2.0.SDTMIG.3.3.SDTM.1.7")
#' data_meta_updated <- set_study_oid(data_meta_updated, "SOMESTUDY")
2 changes: 1 addition & 1 deletion R/dataset_metadata.R → R/archive/dataset_metadata.R
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ set_item_data <- function(x, .data, ...) {

# Insert into object in proper spots
x[[1]][['records']] <- records
x[[1]][['itemData']] <- item_data
x[[1]][['itemData']] <- unname(item_data)
x
}

Expand Down
Loading
Loading