hvtiRdatabuild

Builds analysis-ready clinical datasets for the HVTI CORR group, and verifies them against the legacy SAS datasets they replace.

Status

Slices S0 (Verify), S1 (Pull), and the analysis-set checkpoint from S4. snapshot_oracle() and compare_built() verify an R-built dataset against its SAS oracle. read_study_config(), dw_connect(), dw_modules(), dw_pull(), and print.pull_result() read a study's warehouse modules into R. write_analysis_set() materializes a declared selection from a built dataset, and read_analysis_set() reads a stale checkpoint only in a draft, with a note, and refuses it in a final render. The build and derivation stages (build_dataset() and derive_vars()) remain future work.

Installation

hvtiRdatabuild is part of the HVTI R family. Install it with the rest of the family through hvtiR:

# install.packages("pak")
pak::pak("ehrlinger/hvtiR")
hvtiR::install()

hvtiR::install() installs every member in one call, so the versions the members require of one another resolve together. Update later with hvtiR::update(), from a fresh R session: it will not overwrite a package that is already loaded.

To install hvtiRdatabuild on its own:

# pak::pak("ehrlinger/hvtiRdatabuild")
install.packages("hvtiRdatabuild", repos = c('https://ehrlinger.r-universe.dev', 'https://cloud.r-project.org'))

This package imports hvtiRutilities, which is not on CRAN. The Remotes: field in DESCRIPTION points pak at ehrlinger/hvtiRutilities, so either command above pulls it in. Installing with plain install.packages() will fail to resolve it.

arrow is an optional dependency required for oracle snapshots and analysis set parquet files.

Declaring an analysis set

Add the patient-identifier column (id), selected columns (vars), exclusions, and expected counts under analysis_sets: in _study.yml. From the study root, load the utilities configuration, then write the checkpoint once and read that exact selection in downstream jobs:

set_config <- hvtiRutilities::study_config()
write_analysis_set("eda", set_config)
eda <- read_analysis_set("eda", set_config)

The sidecar records the parent dataset, declaration hash, row counts, and per-rule attrition. If the parent or declaration changes, reading stops and names the write_analysis_set() call that refreshes it.

Pulling warehouse modules for a study

library(hvtiRdatabuild)

# What modules exist, and what each one requires.
dw_modules()

config <- read_study_config("study.yaml")
conn   <- dw_connect(server = "<DW-SERVER>", database = "<DW-DB>", dsn = "HVI_DW")
result <- dw_pull(config, conn)

result$tables    # named list of raw tables, keyed by each module's `output`
result$manifest  # module, output, n_rows, n_cols, pulled_at

result
#> Warehouse pull: 2 module(s)
#>
#>  module output n_rows n_cols           pulled_at
#>    base bdbase      3      3 2026-08-07 17:43:46
#>     fup    fup      3      3 2026-08-07 17:43:46

pulled_at is wall-clock time at pull, so it will differ on your machine. print.pull_result() (dispatched automatically for result) never prints row-level data, only the manifest shape, so there are no identifiers to redact.

dw_pull() is read-only and never writes to the warehouse. The cohort write-back the SAS templates perform (libsql) is deliberately not ported in this slice. The re-pull variants (snapshotpull, ccfpull), which take an existing built dataset and remap keys because masterid stopped being stable in April 2023, need that write path and arrive in a later slice.

Verifying an R build against SAS

library(hvtiRdatabuild)

# Freeze the SAS-built dataset once.
snapshot_oracle(
  "/studies/st1234/datasets/built.sas7bdat",
  "oracle/st1234_built.parquet"
)

# Compare an R rebuild against it.
oracle <- arrow::read_parquet("oracle/st1234_built.parquet")
compare_built(oracle, my_rebuild, id = "ccfidu")

There is no overall pass/fail. Read the table, and record a resolution for every variable that is not identical.

compare_built() requires exactly one row per identifier on both sides. Of the five warehouse modules, only bdbase and bdstat are one row per patient and so are measurable against the SAS oracle today. echo, fup, and bdevents are one row per event (echocardiogram, follow-up visit, reoperation), where a given patid legitimately repeats, so a direct per-identifier comparison errors on them by design. Verifying those three needs a composite-key comparison, deferred to a later slice.

Local integration testing

The committed fixtures are synthetic. To exercise the read path against real SAS output, point HVTI_ORACLE_DIR at a study directory:

HVTI_ORACLE_DIR=/studies/st1234/datasets Rscript -e 'devtools::test()'

That directory holds PHI. It must live outside this repository, and nothing in the test suite copies from it. With the variable unset (the default, and what CI sees), these tests skip.

print() never emits identifiers unless show_ids = TRUE, because ccfidu is a medical record number combined with a date of surgery. Do not enable it in a session whose output is logged or shared.

Documentation

Design

See dev/specs/2026-08-04-hvtiRdatasets-design.md.