Package 'hvtiRutilities'

Description

hvtirutilities is a utility package for using SAS dataset in R at the Cleveland Clinic Heart, Vascular and Thoracic Institute (HVTI) Clinical Outcome Registries and Research (CORR) department.

Description

Writes one entry to the abbreviations: mapping in the study's _study.yml, replacing any entry for the same phrase, compared ignoring case. abbreviation = NULL writes a removal: the study then drops that phrase from the group default list.

Usage

add_abbreviation(phrase, abbreviation, start = getwd())

Arguments

Details

The entry is checked against the whole merged list, as study_abbreviations would check it, before anything is written; a refused entry leaves the file as it was. The file is rewritten whole, as register_data rewrites it, so comments in _study.yml are not kept.

Value

The study's abbreviation list after the change, invisibly, as a named list.

See Also

study_abbreviations.

Examples

root <- file.path(tempdir(), "add-abbrev-example")
study_setup(root, "Abbreviation example", 1L, renv = FALSE)
add_abbreviation("Surgical procedure", "SP", start = root)
add_abbreviation("Ejection fraction", NULL, start = root)
unlink(root, recursive = TRUE)

Description

Registers new labels in an existing label map, or applies labels directly to a data frame's variable attributes. This is the recommended way to label derived variables (e.g., ratios, binned groups, computed indices) that were not present in the original imported dataset.

When label_map_df is a label map (a data frame with key and label columns), the map is updated and returned. When label_map_df is a regular data frame (any data frame without the label map structure), labels are applied directly to the data using labelled::var_label(), which is the preferred approach because labels travel with the data through dplyr operations.

Usage

add_labels(label_map_df, new_labels)

Arguments

Value

When given a label map: the updated label map data frame. When given a data frame: the data frame with labels applied via labelled::var_label().

See Also

label_map to extract a label map from data, apply_label_overrides for bulk overrides from YAML.

Examples

# --- Method 1: Update a label map (for reporting) ---
dta <- generate_survival_data(n = 50, seed = 42)
lmap <- label_map(dta)

# Add labels for derived variables
lmap <- add_labels(lmap, c(
  age_group  = "Age Group (<40, 40-60, >60)",
  bsa_ratio  = "BSA Ratio",
  risk_score = "Composite Risk Score"
))
tail(lmap, 4)

# --- Method 2: Label a data frame directly (preferred) ---
dta$age_group <- cut(dta$age, breaks = c(0, 40, 60, Inf),
                     labels = c("<40", "40-60", ">60"))
dta <- add_labels(dta, c(age_group = "Age Group (<40, 40-60, >60)"))
labelled::var_label(dta$age_group)

Description

Deprecated. Catalog releases are retired; use update_manifest.

Repeats the candidate review and replaces _study.yml and manifest.yaml as one recoverable pair. The old dated release and its cache files remain on disk. Adoption does not make a Git commit.

The candidate must be named by its exact release ID. The value "latest" is never accepted, because the reviewed release and the adopted release must be the same object.

Usage

adopt_data_update(cfg = study_config(), dataset = "study", release_id)

Arguments

Value

The updated study_status object, returned visibly.

See Also

check_data_updates, review_data_update, register_data

Examples

root <- file.path(tempdir(), "adopt-data-update-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

# Deprecated: it warns and names update_manifest() instead, then stops,
# because this study is not pinned to a catalog release.
try(adopt_data_update(cfg, release_id = "surgery_cohort-20260921-r1"))
unlink(root, recursive = TRUE)

Description

Reads label overrides from a YAML file and applies them to a label map or directly to a data frame. This allows study-specific label replacements (e.g., abbreviations, corrections) to be configured externally rather than hard-coded in analysis scripts.

The YAML file should contain a simple mapping of variable names to labels. If the file does not exist, the input is returned unchanged — making it safe to call unconditionally in shared code.

When data is a label map (a data frame with key and label columns), the overrides are applied to the map. When data is any other data frame, labels are applied directly to the data via add_labels, which is the preferred data-first workflow.

Usage

apply_label_overrides(data, overrides_file = "labels_overrides.yml")

Arguments

Details

The YAML file format is a simple mapping of variable names to labels:

age_binned: "Age Group"
bsa_ratio: "BSA Ratio"
cavv_area: "Common AVV Area"

This design keeps study-specific label customizations in configuration rather than code. Each study gets its own labels_overrides.yml alongside its config.yml, and shared helper functions never contain hard-coded replacements.

Value

When given a label map: the updated label map data frame. When given a data frame: the data frame with labels applied via labelled::var_label(). In both cases, variables not mentioned in the YAML file are left unchanged.

See Also

add_labels for programmatic label updates, label_map for extracting labels from data.

Examples

# Create a temporary YAML overrides file
tmp <- tempfile(fileext = ".yml")
writeLines(c(
  "age: 'Patient Age (years)'",
  "bsa_ratio: 'Body Surface Area Ratio'"
), tmp)

# --- On a label map ---
dta <- generate_survival_data(n = 50, seed = 42)
lmap <- label_map(dta)
lmap <- apply_label_overrides(lmap, overrides_file = tmp)
lmap[lmap$key == "age", ]

# --- Directly on data (preferred) ---
dta <- apply_label_overrides(dta, overrides_file = tmp)
labelled::var_label(dta$age)

unlink(tmp)

Description

Reads a study's value_labels.yml and attaches the declared code-to-text mappings to the matching columns as value labels, via labelled::val_labels(). It declares; it does not convert. Pass the result to r_data_types with use_value_labels = TRUE to turn the declarations into factors with text levels.

A .sas7bdat file stores a format's name (YESNOF.); the code-to-text mapping lives in a .sas7bcat catalogue. The study corpus has no catalogues, so this file is not a supplement to one - it is the only place the mapping can come from. Because the declaration is written into the same slot a catalogue would have filled, everything downstream reads it without a second code path.

The function is safe to call unconditionally: a study with no value_labels.yml gets its data back unchanged.

Usage

apply_value_labels(data, value_labels_file = "value_labels.yml")

Arguments

Details

The file maps each variable to its codes, and each code to the text that code means:

disp:
  1: Home
  2: Rehab
  3: SNF
approach:
  1: Ascending aorta only
  2: Ascending aorta plus arch

This is the home for enumerated level definitions, and it is deliberately not the display label. Level definitions crammed into a variable label are what forced them to be reconstructed by hand in every table, and a label carrying eight mutually exclusive options cannot survive label_max (see label_map). Declaring them here keeps the two jobs apart: the converter reads this file, and a table renderer reads the label.

Two rules worth knowing before relying on the result:

• Value labels already on the column win. A column that arrives labelled - from a catalogue, or from an earlier call - is left alone and the variable is named in a warning, so a declaration can never silently overwrite a mapping read from the source.

• A variable named in the file but absent from the data is reported. A typo that quietly declares nothing is the failure this file is least able to notice.

The ordered key is reserved for the ordinal declaration and is refused rather than partly honoured. Ordinality is blocked on a statistical decision; reserving the key now keeps one home for both declarations without pre-empting it.

Value

data, with value labels attached to the declared columns via labelled::val_labels(). Columns not named in the file, and columns that already carry value labels, are returned unchanged.

See Also

r_data_types for converting declarations to factors, apply_label_overrides for the same pattern one level up - declaring variable labels rather than value labels, read_clinical_data for the catalog_file argument that would supply these mappings if a catalogue existed.

Examples

tmp <- tempfile(fileext = ".yml")
writeLines(c("disp:", "  1: Home", "  2: Rehab", "  3: SNF"), tmp)

dta <- data.frame(disp = c(1, 2, 1, 3))
dta <- apply_value_labels(dta, tmp)
labelled::val_labels(dta$disp)

# Declare, then convert
r_data_types(dta, use_value_labels = TRUE)$disp

unlink(tmp)

Description

Compares cohort_counts against explicitly supplied expected counts and errors on any disagreement. Call it before any analysis that would otherwise run happily on an unreconciled cohort.

Usage

assert_cohort(d, expected, event, time)

Arguments

Value

invisible(TRUE) on success; otherwise an error.

See Also

cohort_counts

Examples

d <- data.frame(dead = c(1, 1, 0, 0, 0), iv_dead = 1:5)
expected <- list(n = 5L, n_events = 2L, n_censored = 3L)
assert_cohort(d, expected, event = "dead", time = "iv_dead")

Description

Returns a one-row data frame identifying the built dataset by name, size, modification time and SHA-256. This is the record that lets a later reader tell whether two results were produced from the same data.

Usage

built_manifest(cfg = study_config(), dataset = "study")

Arguments

Value

A one-row data frame with columns file, size_bytes, mtime and sha256.

See Also

built_path, record_provenance

Examples

root <- file.path(tempdir(), "built-manifest-example")
dir.create(file.path(root, "datasets"), recursive = TRUE,
           showWarnings = FALSE)
yaml::write_yaml(
  list(study = "Example", built = "example.csv"),
  file.path(root, "_study.yml")
)
write.csv(data.frame(dead = c(1, 0, 0), iv_dead = 1:3),
          file.path(root, "datasets", "example.csv"), row.names = FALSE)
built_manifest(study_config(root))
unlink(root, recursive = TRUE)

Description

Resolves the registered filename beneath the study's logical datasets directory. The path is not checked for existence.

Usage

built_path(cfg = study_config(), dataset = "study")

Arguments

Value

Character(1). The path to the built dataset.

See Also

built_manifest, read_built

Examples

root <- file.path(tempdir(), "built-path-example")
dir.create(file.path(root, "datasets"), recursive = TRUE,
           showWarnings = FALSE)
yaml::write_yaml(
  list(study = "Example", built = "example.sas7bdat"),
  file.path(root, "_study.yml")
)
built_path(study_config(root))
unlink(root, recursive = TRUE)

Description

Runs an expensive computation (a random forest, a varPro fit, an imputation, a hazard model, a partial dependence grid) once, saves the result, and on later calls loads it instead of recomputing. The saved result is used only if nothing it was built from has changed: the code, every outside value the code reads, the versions of the packages its functions come from, and the seed. If anything changed, cache_fit() stops and lists what changed. It never returns a stale result.

Usage

cache_fit(
  name,
  code,
  seed = NULL,
  dir = study_dir("estimates"),
  refit = FALSE,
  packages = NULL
)

Arguments

Details

code is not evaluated on a cache hit. It can be written inline, as a single call or a {} block of several steps, or passed as a variable holding a quoted call built beforehand. Both forms give the same key for the same computation. Passing an object that has already been computed is an error, because the cache could never skip it.

code must be written in the call to cache_fit() itself. A wrapper function that forwards its own argument, e.g. function(nm, expr) cache_fit(nm, expr), forces that argument's promise when it resolves code, so the value is already computed and the call is refused as already-computed. A wrapper should instead build and pass a quoted call, for example cache_fit(nm, bquote(...)) or by passing quote(...) through.

The code runs in a new environment whose parent is the caller's, so names assigned inside a block do not appear in the caller.

The key digests each free variable of the code: a name bound by an assignment, a function formal, or a for index is local within that binding's scope, so a name that is also read free elsewhere in the code still counts as an input. Values read indirectly (inside the body of a function it calls, or through get()) are not seen; pass such values into the code as arguments. The exception is a function defined in the global environment (as in a shared _common.R): its own body is part of the key, so editing it invalidates a cache that calls it, even though values it in turn reads indirectly are still not seen. This reaches only one level deep: a global helper is part of the key only through its own body, not through the bodies of further helpers it in turn calls, so editing a callee of a helper does not invalidate a cache that calls the helper. For example, if hA <- function(z) hB(z) is defined globally and hB's body is later edited, the key computed for code that calls hA is unchanged.

The result is returned with its key attached as an attribute, and that attached key, not the object's own content, is what a downstream cache_fit() digests when this result is passed to it as an input. Mutating the returned object afterwards, for example fit$coefficients <- ..., is invisible to that downstream cache, because the attached key does not change to reflect the mutation. If a cached object is modified in place, strip its hvtiRutilities_cache_key attribute or re-wrap the result so a downstream cache is not fooled into treating it as unchanged.

The attached-key short cut above applies only to a cached object passed directly as an input. A cached object wrapped inside a list, or a model object built outside cache_fit(), carries no such key and is digested whole instead; such objects can digest differently across sessions or platforms even when nothing meaningful about them changed. This shows up as a loud, classed hvtiRutilities_stale_cache error rather than a silently wrong cache hit, which is the safer failure mode, but is worth knowing about when a downstream cache goes stale for no apparent reason.

The key records the version of the package that owns each function called in code, and no other. A wrapper is therefore keyed on itself only: gg_partial_rfsrc(forest, ...) records ggRandomForests, though randomForestSRC computes the partial dependence when it runs. An upgrade of randomForestSRC reaches that key only through forest's own key, which holds the version at fit time, not the version that runs now. Name such a package in packages, e.g. packages = "randomForestSRC", and its version joins the key, so an upgrade makes the cache stale. Adding packages to an existing call makes its cache stale once, with the package reported as (absent) in the stored key. Base-priority packages are ignored, as they are for functions called in code.

The computation must return a value; a result of NULL is an error, because there would be nothing to cache.

When seed is given, the code runs inside with_seed, which leaves the global random number stream as it was. When seed is NULL and the code consumes random numbers, the result is still saved, with a warning of class hvtiRutilities_nonreproducible_fit, and the key records reproducible = FALSE.

A stale file stops with an error of class hvtiRutilities_stale_cache; a file not written by cache_fit() stops with class hvtiRutilities_unkeyed_cache. Both are recomputed with refit = TRUE.

The result is written to a temporary file and renamed into place, so an interrupted run never leaves a partial file. When dir lies inside a study, a provenance sidecar carrying the key and the completed cache file's hash is published with capture_provenance and publish_provenance. The cache records an explicit empty data list rather than inferring that the currently registered dataset was an input to the computation.

Value

The result of code, computed or loaded, with its key attached as attribute hvtiRutilities_cache_key.

See Also

capture_provenance, publish_provenance, study_dir

Examples

dir <- file.path(tempdir(), "cache-fit-example")
dir.create(dir, showWarnings = FALSE)
x <- c(3, 1, 2)
first <- cache_fit("sorted", sort(x), dir = dir)
again <- cache_fit("sorted", sort(x), dir = dir)
identical(first, again)
unlink(dir, recursive = TRUE)

Description

Freezes the study identity, executing R session, loaded packages, lockfile, and explicit data and artifact records. Each loaded package is recorded with its version and source, and a package installed from a git remote also records the commit it was built from (sha), because one version number can cover more than one build. Capture never resolves a current dataset implicitly. Pass list() deliberately when a job has no known direct data input.

Capture does not write a sidecar or inspect a rendered output. Use publish_provenance only after the completed output exists.

Usage

capture_provenance(
  job,
  data,
  artifacts = list(),
  extra = list(),
  cfg = study_config()
)

Arguments

Value

A captured provenance payload as a plain list.

See Also

publish_provenance, record_provenance

Examples

root <- file.path(tempdir(), "capture-provenance-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

payload <- capture_provenance("death-hz-ac",
                              data = list(provenance_data(cfg = cfg)),
                              cfg = cfg)
names(payload)
unlink(root, recursive = TRUE)

Description

Deprecated. Catalog releases are retired; use study_status or verify_manifest.

Compares each release-aware dataset contract with its producer-owned catalog. The check verifies the pinned release and every later published candidate without changing _study.yml, manifest.yaml, or any cache file. Legacy dataset contracts have no update rows.

Usage

check_data_updates(cfg = study_config(), dataset = NULL)

Arguments

Value

An object of class "data_update_report": a data frame with columns dataset, scope, pinned_release_id, candidate_release_id, sequence, file, status, is_latest, and detail. Status is one of "CURRENT", "UPDATE AVAILABLE", "UPDATE STATUS UNKNOWN", "WITHDRAWN", or "FAIL".

See Also

review_data_update, adopt_data_update, read_built

Examples

root <- file.path(tempdir(), "check-data-updates-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

# Deprecated: it warns and names study_status() instead. A study registered
# without a catalog release has nothing to report.
check_data_updates(cfg)
study_status(root)
unlink(root, recursive = TRUE)

Description

Deprecated. Use apply_label_overrides() instead. clean_labels() has been renamed for clarity. This function is kept as an alias for backward compatibility.

Usage

clean_labels(label_map_df, overrides_file = "labels_overrides.yml")

Arguments

Value

When given a label map: the updated label map data frame. When given a data frame: the data frame with labels applied via labelled::var_label(). In both cases, variables not mentioned in the YAML file are left unchanged.

Examples

# Use apply_label_overrides() instead
tmp <- tempfile(fileext = ".yml")
writeLines("age: 'Patient Age (years)'", tmp)

library(labelled)
dta <- data.frame(age = c(25, 30, 35))
var_label(dta$age) <- "Patient Age"
lmap <- label_map(dta)

lmap <- clean_labels(lmap, overrides_file = tmp)
unlink(tmp)

Description

Counts rows for which both explicitly supplied event and time columns are present, and the events among them.

The event column must be logical or numeric binary coding: FALSE/TRUE or 0/1. Rows missing either column are excluded from the analysable cohort.

Usage

cohort_counts(d, event, time)

Arguments

Value

A list with integer elements n, n_events and n_censored.

See Also

assert_cohort

Examples

d <- data.frame(dead = c(1, 1, 0, 0, 0), iv_dead = 1:5)
cohort_counts(d, event = "dead", time = "iv_dead")

Description

Summarises the differences between two data pulls of the same dataset: added or dropped columns, row count changes, type changes, and label changes. This is useful for auditing data drift when a new extract arrives, and pairs naturally with the manifest system.

Usage

compare_datasets(old, new)

Arguments

Value

A list with the following elements:

rows_old

Number of rows in old

rows_new

Number of rows in new

cols_added

Character vector of column names present in new but not old

cols_dropped

Character vector of column names present in old but not new

type_changes

Data frame with columns variable, old_class, new_class for shared columns whose primary class changed

label_changes

Data frame with columns variable, old_label, new_label for shared columns whose label changed

See Also

update_manifest, verify_manifest

Examples

# Simulate two data pulls
v1 <- generate_survival_data(n = 100, seed = 1)
v2 <- generate_survival_data(n = 120, seed = 2)

# Add a column to v2 and drop one
v2$new_var <- rnorm(120)
v2$dead <- NULL

diff <- compare_datasets(v1, v2)
diff$rows_old
diff$rows_new
diff$cols_added
diff$cols_dropped

Description

Errors – never warns, never skips – when the quantity is absent on either side. A comparison that cannot fail is worse than no comparison.

Usage

compare_parity(quantity, r, sas, class, source = "lst", digits = NA_integer_)

Arguments

Details

The outcome is three-state. R_BETTER fires only for a log-likelihood that exceeds SAS's beyond tolerance: a multi-start optimizer regularly beats a single-start one, and recording that as a failure would train the reader to distrust a real improvement.

Value

A one-row data frame.

Relative discrepancy at a zero reference

When the SAS value is zero there is no relative scale. The reported rel_diff is 0 if the two agree exactly and Inf otherwise – never NA, which parity_headline would drop, reporting a real failure as no discrepancy at all.

Examples

compare_parity("log_likelihood", r = -239.1941, sas = -239.194,
               class = "loglik")

Description

One row per candidate: which concept it belongs to, and whether it is that concept's representative. Returned rather than applied, so a caller can put the map in its report before anything is discarded.

Usage

concept_map(pool, prefer = character(0), aliases = character(0), ...)

Arguments

Details

prefer overrides which form represents a concept, as c(concept = variable). The default is a modelling assumption, not a neutral choice: keeping the plain form asserts the effect is linear in it. Measured on one study, ln_crcl selected in 43.8% of replicates against 18.0% for crcl_pr, so defaulting to the plain form would have lowered a genuine risk factor's frequency. Override where the study has a view; the map goes into the report either way, so the choice is visible rather than implied.

Value

A data frame with columns variable, concept and representative, ordered by concept with the representative first.

See Also

concept_of, prune_to_one_form

Examples

concept_map(c("age", "ln_age", "age2", "agee"))

Description

A SAS candidate pool offers every transformation of a variable as a separate candidate: age, ln_age, in_age, age2 and agee all compete. A stepwise screen then spends its steps on them, and a per-variable selection frequency reports each form separately. On one real screen five forms of age each cleared 80% while the retained set collapsed to four distinct clinical concepts – and the paper the job comes from reports concepts.

Usage

concept_of(
  v,
  pool,
  affixes = POOL_AFFIXES,
  plain_suffix = POOL_PLAIN_SUFFIX,
  aliases = character(0),
  min_stem = POOL_MIN_STEM
)

Arguments

Details

The rule is deliberately conservative. A variable joins a concept only when stripping one known affix yields a name that is itself in the pool (or that name plus plain_suffix). So zexp2 groups under zexp because zexp is a candidate, and ln_crcl groups under crcl_pr because that is the plain form present. A name whose stem is absent stays its own concept: agee does not join age, because no rule strips a trailing e.

Grouping too little costs some pruning. Grouping too much silently merges two clinical concepts and discards a real candidate, which is the error that cannot be seen in the output. The rule errs the safe way, and callers are expected to report the map so a human can check it.

Two mechanisms are needed, and only one can be mechanical. vars.sas truncates the stem when it names a derivative, so stripping the affix yields a name absent by exact match:

• Truncation – effi is a prefix of effic, so it can be found. Handled automatically, but only when the stem prefixes exactly one pool member that is not itself a derived form, and only when the stem is at least min_stem characters. Ambiguity declines to group rather than guessing.

• Contraction – arin is not a prefix of area_int and no rule can find it. Declare it via aliases, so the judgement is recorded in the job rather than inferred by the function.

Left ungrouped on a real screen, effic at 86.4% and ln_effi at 60.8% were both reported as retained risk factors at r = 0.9988, and in_arin at 70.6% hid a concept whose union across its forms was 86.4%.

Value

A character scalar: the concept, or v itself when no rule applies.

A permanent compatibility layer, not a temporary shim

Stem truncation exists because SAS capped variable names at 8 characters. A modern builder has no such cap and a pool built that way needs none of this, because ln_efficiency strips cleanly to efficiency.

But the two regimes coexist indefinitely: reproducing a legacy SAS job in R is routine work, not a migration with an end date, so truncated pools keep arriving from built*.sas7bdat extracts for as long as old jobs are re-run. Do not delete the prefix rule or the alias table when a new builder ships. They are how a legacy pool is read. The right expectation is that new datasets simply never need them.

See Also

concept_map, prune_to_one_form, selection_crowding

Examples

pool <- c("age", "ln_age", "age2", "agee")
vapply(pool, concept_of, character(1), pool = pool)

Description

Reports, one row per variable, what a column is and what using it as a model covariate would do to it – so a job can report the decision rather than make it silently.

Usage

covariate_audit(data, vars, max_levels = 12L)

Arguments

Details

The problem this exists for. vars.sas is commonly called as %vars(missing=1, impute=1), which mean-imputes missing values and adds a paired ms_* missing indicator. So a 0/1 clinical variable arrives carrying three distinct values: 0, 1, and the cohort mean. One real example: hx_htn's third value is 0.714047, the prevalence of hypertension, not any patient's hypertension status.

In SAS these columns are numeric and enter a model linearly: an imputed row contributes the mean, and the ms_* indicator carries whatever the missingness itself is worth. That is the design the .sas job specifies.

Read into R they arrive as factors. Put a factor in a model formula and model.matrix() builds dummies from it, which makes "this value was imputed" its own category with its own coefficient. That is a different model, with a different parameter count, and nothing in the output says so. Multi-level counts have the same problem for a different reason: a surg_num with levels 1, 2, 3, 4 is a number of previous operations, not four unordered categories.

max_levels bounds where an imputed value is looked for at all. On a continuous covariate every value is its own level and most are non-integers, so an unbounded search returns the whole column.

That bound is also an honest statement of the limit. Mean imputation is detectable in a discrete column, because the mean is not a value the column can legitimately take. In a continuous column it is invisible by construction – an imputed mean looks exactly like a measured value – and the paired ms_* indicator is the only record that it happened. So NA in noninteger_levels means "not knowable here", not "none".

Value

A data frame with columns variable, storage, n_levels, noninteger_levels and action. noninteger_levels reports value (rows) per level: per level rather than as a total because the total cannot distinguish a mean-imputed binary (one such level, in a minority of rows) from an inverse transformation whose several non-integer levels are ordinary data. A blank means "no such level"; NA means "not looked for". action is what covariates_to_numeric would do, and an action beginning ERROR marks a variable that must not be fitted.

See Also

covariates_to_numeric, imputed_levels

Examples

d <- data.frame(hx_htn = factor(c("0", "1", "0.714047")),
                age = c(60, 71, 55))
covariate_audit(d, c("hx_htn", "age"))

Description

Factors whose levels are all numeric become numeric, so they enter a model linearly the way the SAS job specifies.

A factor with a genuinely non-numeric level is left alone and will be dummy-coded – the right outcome for a real categorical, and the wrong one for a mean-imputed binary. covariate_audit's action column is where a reader sees which of the two happened; check it before reading any coefficient.

Usage

covariates_to_numeric(data, vars)

Arguments

Value

data, with the convertible columns coerced to numeric.

See Also

covariate_audit

Examples

d <- data.frame(hx_htn = factor(c("0", "1", "0.714047")))
str(covariates_to_numeric(d, "hx_htn"))

Description

Creates a summary data frame describing every variable in a dataset: name, label, R class, number of unique values, percent missing, and a compact range or level summary.

This is the single most useful artifact for documenting a clinical dataset. It can be written directly to CSV or included in a Quarto document as a table.

Usage

data_dictionary(data)

Arguments

Value

A data frame with one row per variable and the following columns:

variable

Column name

label

Descriptive label (falls back to variable name if unlabeled)

class

Primary R class of the column

n_unique

Number of unique non-NA values

pct_missing

Percentage of rows that are NA (0-100)

summary

Compact summary: min / median / max for numeric, level counts for factor/character, TRUE percentage for logical

See Also

proc_contents for the full SAS-style variable table, proc_means for numeric summaries, label_map for extracting labels only, r_data_types for automatic type conversion before building the dictionary.

Examples

# Quick dictionary from synthetic clinical data
dta <- generate_survival_data(n = 100, seed = 42)
dict <- data_dictionary(dta)
head(dict, 10)

# After type conversion
dta_clean <- r_data_types(dta,
  factor_size = 5,
  skip_vars = c("ccfid", "iv_dead", "iv_reop", "iv_opyrs")
)
dict_clean <- data_dictionary(dta_clean)
head(dict_clean, 10)

# Write to CSV for documentation
out <- tempfile(fileext = ".csv")
utils::write.csv(data_dictionary(dta), out, row.names = FALSE)
unlink(out)

Description

One row per column: creation position, name, R class, SAS two-valued type, format.sas and label. This is the durable description of a source dataset — what a schema sidecar records, and what a later read is compared against.

Usage

dataset_schema(data)

Arguments

Details

label and format are read from the column attributes directly and are NA when the source carries none. This is the difference from proc_contents, which fills an absent label with the variable's own name: right for a printed listing, wrong for a record that outlives the source dataset.

Nothing here describes the data, only its shape. Two reads of an unchanged file produce an identical schema, which is what makes the sidecar's hash meaningful.

Value

A data frame with columns num (creation position, integer), variable, class (first R class), type ("Num"/"Char"), format (SAS format or NA) and label (or NA).

See Also

proc_contents for a printable listing that also summarises completeness.

Examples

d <- data.frame(x = 1:3, y = letters[1:3])
attr(d$x, "label") <- "An identifier"
dataset_schema(d)

Description

Summarises an event indicator and its follow-up intervals the way the dc-gfup template reports them: counts for the whole cohort and its event and censored subsets, the missing, negative and zero intervals, a proc_means table for each subset, and the rows worth a second look. It checks recorded follow-up; it does not establish completeness against a close date.

Usage

followup_check(
  data,
  event,
  followup,
  identifier = NULL,
  max_rows = 25L,
  stats = c("n", "nmiss", "mean", "std", "min", "p15", "median", "p85", "max")
)

Arguments

Details

Every argument is checked before anything is computed, and every column that is not in data is named in one error, so a job with three misspellings says so once.

A row with a missing event is counted in missing_event and belongs to neither the event nor the censored subset. A row is suspicious when its event is missing or any of its intervals is missing, negative or zero.

The quartiles in intervals use R's default interpolated quantiles (type = 7), while proc_means uses the SAS QNTLDEF=5 estimator, so a median in the two tables can disagree on a small subset. The difference is deliberate: intervals is a quick screen, and means is the table to compare against SAS.

means reports the 15th and 85th percentiles rather than the quartiles, because those are what the group's descriptive summaries show. SAS PROC MEANS prints no percentiles unless asked, so a parity comparison names the same statistics on the SAS side, or passes the SAS job's list to stats.

Identifiers are never included unless named in identifier. Check the result before sharing a report that shows review with one.

Value

An object of class followup_check, a list of four components, three data frames and a list of three:

cohort

One row: full, event, censored and missing_event counts.

intervals

One row per interval: missing, negative and zero counts, then min, q1, median, q3, mean, sd and max of the observed values.

means

A named list, full, event and censored, of proc_means tables over the intervals, one column per statistic in stats.

review

Up to max_rows suspicious rows, with the event, the intervals and any identifier.

See Also

proc_means, cohort_counts, which counts the analysable cohort rather than checking follow-up.

Examples

d <- data.frame(dead = c(1, 0, 0, NA, 1), iv_dead = c(2.5, 4, 0, 1.2, NA))
fc <- followup_check(d, event = "dead", followup = "iv_dead")
fc
fc$cohort
fc$review

Description

Creates a simulated dataset that mimics a cardiac surgery cohort, suitable for testing survival analysis workflows in ‘rf.survival.study.qmd’. Variable labels are attached via labelled::var_label() so they are compatible with haven and hvtiRutilities::label_map().

Usage

generate_survival_data(n = 500, seed = 1024)

Arguments

Details

Survival times are drawn from a Weibull distribution (shape = 1.5, ~8-year median follow-up) with a linear predictor built from LVEF, age, baseline hemoglobin, NYHA class, and eGFR. Administrative censoring is applied uniformly between 1 and 15 years. Reoperation probability is modelled via a logistic function of bypass time and NYHA class.

The returned data frame includes the five columns excluded from model_data in the template (ccfid, reop, iv_reop, origin_year, iv_opyrs), plus the survival outcome (iv_dead, dead) and 17 clinical predictors spanning demographics, pre-operative labs, cardiac function, and surgical variables.

The global RNG state is saved and restored on exit, so calling this function does not affect subsequent random number generation in the session.

Value

A data.frame with n rows and 24 columns:

ccfid

Character. Patient identifier ("PT00001", …).

origin_year

Integer. Calendar year corresponding to iv_opyrs = 0 (randomly sampled between 1998 and 2018).

iv_opyrs

Numeric. Observation interval length in years measured from origin_year; always greater than or equal to iv_dead.

iv_dead

Numeric. Observed follow-up time in years (min of true survival time and administrative censoring time).

dead

Integer. Event indicator: 1 = death, 0 = censored.

reop

Integer. Reoperation indicator: 1 = yes, 0 = no.

iv_reop

Numeric. Time to reoperation (years); NA if no reoperation.

age

Numeric. Age at surgery in years (range 1–85).

sex

Factor. "Male" / "Female" (55 / 45 %).

bmi

Numeric. Body mass index in kg/m² (range 15–50).

hgb_bs

Numeric. Baseline hemoglobin in g/dL (range 6–18).

wbc_bs

Numeric. Baseline WBC count in K/µL (range 1.5–20).

plate_bs

Numeric. Baseline platelet count in K/µL (range 50–500).

gfr_bs

Numeric. Baseline eGFR in mL/min/1.73 m² (range 10–120).

lvefvs_b

Numeric. Baseline LV ejection fraction in % (range 15–75).

lvmass_b

Numeric. Baseline LV mass in g (range 60–400).

lvmsi_b

Numeric. Baseline LV mass index in g/m² (range 40–220); derived as lvmass_b / (bmi * 0.5).

stvoli_b

Numeric. Baseline systolic stroke-volume index in mL/m² (range 20–110).

stvold_b

Numeric. Baseline diastolic stroke-volume index in mL/m² (range 40–180).

bypass_time

Numeric. Cardiopulmonary bypass time in minutes (range 20–240).

xclamp_time

Numeric. Aortic cross-clamp time in minutes; sampled as 50–75 % of bypass_time.

nyha_class

Ordered factor. NYHA functional class I–IV.

diabetes

Factor. Diabetes mellitus: "No" / "Yes" (75 / 25 %).

hypertension

Factor. Hypertension: "No" / "Yes" (55 / 45 %).

See Also

rfsrc, Surv, read_sas

Examples

dta <- generate_survival_data(n = 200, seed = 42)
str(dta)
table(dta$dead)          # event rate
summary(dta$iv_dead)     # follow-up distribution

# Drop admin columns to match model_data in the Quarto template
model_data <- dta[, !names(dta) %in% c("ccfid", "reop", "iv_reop",
  "origin_year", "iv_opyrs")]

Description

Returns the descriptive label for one variable name from a label map. This is a safer alternative to the manual match() pattern, providing clear errors on typos and missing variables.

Usage

get_label(label_map_df, variable)

Arguments

Value

A single character string: the label for the requested variable.

Examples

dta <- generate_survival_data(n = 50, seed = 42)
lmap <- label_map(dta)

get_label(lmap, "age")
get_label(lmap, "hgb_bs")

# Use in plot titles
var <- "lvefvs_b"
plot(dta[[var]], main = get_label(lmap, var), ylab = get_label(lmap, var))

Description

A vectorized companion to get_label. Returns a named character vector of labels for one or more variable names, making it convenient to label axes, table columns, or multi-panel plots in a single call.

Variables not found in the label map cause an error (just like get_label), so typos are caught immediately.

Usage

get_labels(label_map_df, variables)

Arguments

Value

A named character vector with names equal to variables and values equal to the corresponding labels.

See Also

get_label for single-variable lookup, label_map to extract a label map from data.

Examples

dta <- generate_survival_data(n = 50, seed = 42)
lmap <- label_map(dta)

# Look up several labels at once
get_labels(lmap, c("age", "bmi", "hgb_bs"))

# Useful for table column headers
vars <- c("age", "bmi", "lvefvs_b")
headers <- get_labels(lmap, vars)
print(headers)

Description

Stops before renv::snapshot() when the R running this session has a different major.minor version than the R recorded in the study's renv.lock. A snapshot from the wrong R rewrites the lockfile's R version and drops every locked package the session's library lacks. On 2026-10-22, R 4.6.1 snapshotted a study locked to R 4.5.3, and boostmtree and ggBoostedTrees fell out of its lockfile.

A patch difference (4.5.1 against 4.5.3) passes, as does a study with no renv.lock.

Use r_upgrade = TRUE only when you are deliberately moving the study to a new R. The guard then still stops if any package in the lockfile is not installed in this session's library, because the snapshot would drop it; restore those first with renv::restore().

study_need and study_setup call this before they snapshot, so you need to call it yourself only before a snapshot of your own.

Usage

guard_snapshot(
  root,
  r_upgrade = FALSE,
  session = .session_r(),
  installed = .installed_version
)

Arguments

Value

TRUE, invisibly, when the snapshot may go ahead. Otherwise an error that names both R versions and the R to start.

Examples

root <- tempfile("study")
dir.create(root)
# A study with no renv.lock has nothing to protect:
guard_snapshot(root)

# A lock written by R 4.5.3, checked from a session running R 4.6.1:
writeLines('{"R": {"Version": "4.5.3"}, "Packages": {}}',
           file.path(root, "renv.lock"))
try(guard_snapshot(root, session = numeric_version("4.6.1")))
unlink(root, recursive = TRUE)

Description

Some file names lead with a utility name rather than an analysis prefix — plots, PPTs. They are listed here so the test suite can tell "not a prefix" apart from "a prefix nobody documented". Without this distinction the taxonomy either fills with non-prefixes or stops catching real omissions.

Usage

hvti_non_prefixes()

Value

A character vector.

Examples

hvti_non_prefixes()

Description

A legacy prefix that names the same job as a prefix in hvti_taxonomy, so a census counts its files under that prefix. pm folds into lm: a review of the job catalog on 2026-09-11 found that the propensity work filed as pm belongs with the logistic models in lm.

Usage

hvti_prefix_folds()

Details

The map is stored, never derived. A fold is a judgement about what a legacy name means, and a rule that inferred it from spelling would also fold names that merely look alike. Every name is absent from hvti_taxonomy() and every value is present in it; the tests pin both.

Value

A named character vector, c(legacy = prefix).

See Also

job_files, which applies it.

Examples

hvti_prefix_folds()

Description

The prefix system inherited from the original CORR analysis binder. The prefix encodes both the type of analysis and the folder the job belongs in.

Usage

hvti_taxonomy()

Details

This is data rather than documentation on purpose. The same table lived in a README and drifted from the files it described; as a function it is checked by the test suite against the templates actually present.

Folder names are load-bearing. Row order is not, folder order included, as of 2026-09-03.

A template directory and a numbered study directory are both spelled <NN>_<folder>, as in 20_distributions; a legacy study keeps the bare name, as in distributions, and study_dir resolves either. The name after the digits, or the bare name, must be a folder in this table. hvtiRtemplates tests that every template directory names one, so renaming or removing a folder that holds a template turns its CI red. The digits are assigned, not derived from this table. They are hardcoded in study_dir's layout map here and in the template directory names there, and estimates is 90 though it is the table's fifth folder, because it holds saved output rather than jobs. Reordering rows, or folders, changes nothing downstream.

Adding or renaming a folder is not free on the study side. study_dir knows only the folders in its own layout map, and a test compares that map's names with unique(hvti_taxonomy()$folder), so a folder added here and not there turns this package's CI red rather than becoming one study_dir() refuses as unknown. The test compares names only; the new folder's digits still have to be assigned in the map.

Templates carry no ordinal. They were once named <NN>.<MM>-<prefix>, with NN taken from this table's folder order through a hardcoded map in hvtiRtemplates; the ordinal and that map were both retired on 2026-09-03 (hvtiRtemplates:dev/specs/2026-09-03-template-identity-design.md). A template is now <prefix>[-<qualifier>].qmd, and the catalog that lists them is hvtiRtemplates' own inst/extdata/templates.json.

History, kept because it is why the digits are assigned: moving hs out of analyses in 1.1.6 shifted bh from sixth to fifth while its shipped filename stayed 04.06, and nothing caught it. bh was renumbered 04.05. Identity derived from a row position breaks when an upstream row moves. A job named 04.06-bh, from hvtiRtemplates before its 1.1.0, is the same template as 04.05-bh.

hs is filed under graphs, which reads oddly for a job named "setup". It was moved there from analyses on 2026-08-29 on the corpus rather than on the name: all ten tp.hs.* templates in the SAS library and ten of the eleven R hs jobs in /studies sit in graphs/. It computes patient-level predictions that the plotting jobs beside it consume, in the setup / uses_setup pairing the corpus uses throughout. See hvtiRtemplates:dev/specs/2026-08-29-hs-template-design.md.

si and mi are two prefixes, not one, and that is the point. 223 studies call single mean imputation, 326 call multiple imputation, and 18 call both. They are different methods with different inferential properties, and 18 studies running both means one prefix could not label them unambiguously. mi is safe here only because it is paired: standing alone it reads as multiple imputation to a statistician, so using it for the single-imputation job would misname exactly the thing the split exists to distinguish.

Neither replaces vars, whose description still says "imputations" on purpose. vars is the job that enhances a dataset with temporary variables, imputations and propensity terms together – porting vars.sas found mean imputation across 394 variables inside it. si and mi name jobs whose whole purpose is imputation. A job that imputes as one step among several is vars; a job that exists to impute is si or mi.

folder names two different things. For most rows it is the analysis type's home folder, matched to a job prefix. One row, estimates, is an artifact kind rather than a job type: it holds serialized fits and cached results written by one job and read by a later one in the same set, and no analysis produces it directly. That row's prefix is NA, not a string, because there is no prefix to assign it.

The random forest family splits on the OUTCOME axis, not the package axis. rfs, rfc and rfr name the survival, classification and regression outcomes, which is the split bh/bl/bc/bn/bq/br and pm/rm/cm already use. It is a rename of a distinction the corpus draws already, not a new one imposed: the jobs carry the outcome in field two today, as tp.rfsrc.survival.R and its siblings.

rf and rfsrc are retained as legacy umbrella rows. They named the same set as each other, on the package axis, and neither is templated. They stay in the table so a census can still resolve the corpus that uses them, which is not small: rfsrc measures 131 studies and 2,295 jobs, more than rf (47), rfs (25) and rfc (19) combined, twice over. Deleting the rows was rejected on two grounds. hvti_prefix_folds cannot express the demotion, because a fold is one-to-one and rfsrc spans all three outcomes, so there is no single prefix to fold it into. And an absent prefix reads to the census as one nobody documented, which is the opposite of what a superseded name means. Demoted here means never templated, and since 2026-09-18 it is machine-readable as well as worded: the umbrella column is TRUE on exactly these two rows, so a consumer can exempt them without hard-coding a list. The two rows keep distinct descriptions so a census hit on a legacy job can still say which spelling it used.

The demotion is recorded here, and only here. Until 2026-09-19 hvtiR's job catalog also marked rf and rfsrc with a retire disposition, which coincided with this demotion by decision. That catalog has moved to hvtiRtemplates as its template catalog, read by hvtiRtemplates::template_catalog(), where every row is a template owed in that package. It has no retire disposition, and rf and rfsrc have no row at all; its guard that every taxonomy prefix has a catalog row exempts them by reading the umbrella column above. hvtiR::jobs() was removed in hvtiR 1.2.0.

sid and vt are filed under analyses with the rest of the family, and compose on the rf* jobs rather than replacing them. The rfc/rfs folder contradiction is deliberately NOT settled here: the taxonomy says analyses while the SAS library files them under graphs and documents, and the 2026-08-27 census found the same contradiction corpus-wide for ac/hz. It is a systemic open question with its own spec owed, and the ML batch is not blocked waiting on it. See hvtiRtemplates:dev/specs/2026-09-17-ml-family-roadmap-design.md.

Value

A data frame with columns prefix, name, folder, description and umbrella. prefix is NA for the one row that names an artifact kind rather than an analysis type. umbrella is logical: TRUE for the legacy umbrella prefixes rf and rfsrc, for which no template is owed; FALSE for every other prefix; NA for the artifact row.

Examples

head(hvti_taxonomy())

Description

Display the NEWS.md file for the hvtiRutilities package showing release notes and change history.

Usage

hvtiRutilities.news()

Value

Invisibly returns NULL. Called for side effect of displaying NEWS.

Examples

# Opens a pager, so only in an interactive session.
if (interactive()) hvtiRutilities.news()

Description

On a mean-imputed column these are the imputed value, which is the signature that imputation happened at all: a genuine 0/1 indicator has no such level.

Usage

imputed_levels(x)

Arguments

Value

A character vector of the offending levels, possibly empty.

See Also

covariate_audit

Examples

imputed_levels(c(0, 1, 0.714047, 1))

Description

Answers the question the template roadmap keeps needing: for each job prefix, which studies have run it and how many jobs each.

Usage

job_census(x)

## S3 method for class 'hvti_job_census'
print(x, ...)

Arguments

Details

Two count columns, deliberately. n_jobs counts distinct stems and is the honest unit – the .lst, .sas and .log of one job are one job, and an editor backup does not create a second. n_files counts rows, and exists because the hand-count this replaces counted files; keeping both means the new output can be reconciled against the table that already drove a decision.

A template is not a job, by either marker. is_template is strictly the legacy tp. prefix (spec section 4); a file written in the template naming convention (section 4.2, e.g. 03.01-ac.qmd) is also not a job, but with is_template = FALSE. is_template_naming carries that second signal, and the roll-up keys on both, so a template-convention file never shares a row with a genuine job at the same (study, prefix, folder) – if it did, that row could outlive the genuine job and still read as an attestation the distinct-studies gate (a prefix needs a second study running the real job before it is templatable) would wrongly count.

The job_files() rows are kept on the result as the "files" attribute, so the accounting – unplaced files, unknown prefixes, misfiled jobs – stays reachable from the summary rather than being computed and thrown away.

The roll-up cannot key every row, and says so rather than shrinking quietly: a file with no study or no prefix has nothing to group by. Those rows are attached as the "not_rolled_up" attribute and counted on the first line of the print, because what was dropped is the union of the unplaced and the unparsed and a reader cannot compute a union from two separate totals.

Subsetting. A row subset (x[i, ]) keeps both attributes. A column subset (x[, j]) drops them while leaving the class in place, so the result still dispatches to this print method with nothing to print from; print() detects that and errors rather than reporting zeros. The printed coverage line ("Rolled up N of M files") always describes the full original sweep, even from a row-subset x: its totals are fixed as an attribute when job_census() builds x, not recomputed from x$n_files at print time.

Value

A data frame of class hvti_job_census with one row per (study, prefix, folder, is_template, is_template_naming) and columns n_jobs and n_files. is_template_naming is TRUE when the row's files use the template naming convention (section 4.2) – distinct from is_template, which is strictly the legacy tp. marker. The originating job_files() rows are attached as the "files" attribute, and the rows that could not be rolled up – those with no study or no prefix – as the "not_rolled_up" attribute.

See Also

job_files

Examples

d <- file.path(tempdir(), "job-census-example")
dir.create(file.path(d, "alpha", "distributions"), recursive = TRUE)
file.create(file.path(d, "alpha", "distributions", "hz.dead.lst"))
file.create(file.path(d, "alpha", "distributions", "hz.dead.sas"))
job_census(d)
unlink(d, recursive = TRUE)

Description

Walks each root and returns one row per file. This is a filename-only sweep: no file is opened, nothing is parsed, and there is no TemporalHazard dependency.

Usage

job_files(roots)

Arguments

Details

Nothing is filtered out. Placement and classification are columns, not reasons to drop a row, so a file this sweep cannot classify stays findable. A sweep that reports only what it kept makes a missing job indistinguishable from a job that does not exist. The one exception is tooling: a path with a .git or .checkpoint component holds version-control internals or a study checkpoint's mirror, not study files, and is not listed.

A scaffolded name, <subject>-<type>-<prefix>[-<qualifier>].qmd as written by hvtiRtemplates::add_job(), is parsed with its qualifier. So is the template-first form, <prefix>[.<qualifier>].<subject>.<type>.qmd. The -runner.R (or .runner.R) that add_job() writes beside some reports takes the report's stem, so it counts as a file of that job rather than as a second job.

A legacy prefix listed in hvti_prefix_folds is reported as the prefix it folds into, so a pm.* file counts as lm. stem keeps the name as written, so a folded file stays findable.

There is deliberately no extension allowlist. See vignette-adjacent design note dev/specs/2026-08-26-job-type-inventory-design.md, section 4.4: a plausible default tuned on the hazard prefixes would have dropped every R-side job in the corpus.

folder_ok is folder == folder_expected, and is therefore NA – not FALSE – whenever either side is unknown. An unplaced file has no folder and an unparsed name has no expected folder; neither is a misfiled job, and !folder_ok must not select them.

Run this server-side. It stats every file beneath roots, which is metadata-latency-bound over an SMB mount – a 40-file scan has timed out at two minutes over the share.

Value

A data frame with one row per file and the columns path, study, folder, status, depth, naming, prefix, is_template, qualifier1, qualifiers, n_qualifiers, stem, ext, prefix_class, folder_expected and folder_ok. The three qualifier columns carry the dot-fields of a legacy name that lie between the prefix and the extension, or the trailing qualifier field of a scaffolded name, and are NA (count zero) for every other convention. They are named by position because the second field means a different thing in each taxonomy folder; see hvti_taxonomy and the design note referenced in ‘R/job_names.R’. Zero rows if the roots hold no files.

See Also

job_census, hvti_taxonomy

Examples

d <- file.path(tempdir(), "job-files-example")
dir.create(file.path(d, "alpha", "distributions"), recursive = TRUE)
file.create(file.path(d, "alpha", "distributions", "hz.dead.lst"))
job_files(d)[, c("study", "folder", "prefix", "status")]
unlink(d, recursive = TRUE)

Description

Extracts variable labels from a labeled dataset and returns them as a data frame with variable names (keys) and their corresponding labels. This is particularly useful when working with SAS datasets that include variable labels, or any dataset labeled with the labelled package.

A warning is issued when more than 50% of columns carry no label at all. This typically indicates the data was imported from a source without labels (e.g., plain CSV) and labels should be supplied via add_labels or a labels_overrides.yml file (see apply_label_overrides). A variable whose real label happens to equal its own name does not count towards the threshold: the absent label is read as NA rather than filled, so the two cases are distinguishable here even though proc_contents cannot tell them apart.

Labels longer than label_max are cut for display. The cut breaks on a word boundary and is marked with ..., and the source text is kept in label_full - truncation is a view, not a change to the data. dataset_schema deliberately has no such parameter: it records the label as the source carried it, because a schema sidecar outlives the source dataset.

The variable-name fallback is exempt from the cap. A name standing in for a missing label passes through whole, however long, and truncated is FALSE. A truncated name would match nothing in the data and would read as a deliberately short label rather than as a missing one, destroying the signal the fallback exists to give.

Labels that differ stay different. Cutting each label on its own can make two labels identical once their distinguishing ends are cut off. Within one map, labels that differ in label_full always differ in label. Each step below runs only on labels still over the cap or still colliding:

1. A heading, the text before the first ": ", " - " or "; ", shared by two or more labels, one of them over the cap, is abbreviated in every label that carries it: to its entry in abbreviations, or else to the initials of its words, skipping small words such as "of" and "and". "Surgical procedure" becomes "SP". A one-word heading is left alone, and two headings with the same initials are neither abbreviated.

2. Any phrase in abbreviations is applied, whole words and ignoring case, to a label still over the cap. A label that fits is never abbreviated this way.

3. The label is cut on a word boundary and marked.

4. Cut labels that still collide keep both ends, "Ascending aorta ... plus arch".

5. A label that still collides is shown whole, over the cap, and over_cap marks it: a long label is a layout problem a reader can see, two identical labels a wrong figure nobody can.

The abbreviations actually shown come back as the abbreviations attribute, to print as a key beneath a figure or table, and per variable as the abbreviations_by_key attribute (key, abbreviation, expansion), so a key under one section can list only what that section's labels use.

Usage

label_map(data, label_max = 40, abbreviations = NULL)

Arguments

Value

A data frame with five columns, and an abbreviations attribute: a data frame of abbreviation and expansion for every abbreviation the labels show, with no rows when there are none.

key

Character vector of variable names from the input dataset

label

Character vector of labels fit to print: the variable label cut to label_max, or the variable's own name where the source carries no label

label_full

The label as the source carried it, never truncated, or the variable name where there is none

truncated

Logical: TRUE where label was cut from label_full. Always FALSE for a filled variable name. subset(x, truncated) is the report of what was cut

over_cap

Logical: TRUE where label is longer than label_max because every shorter form collided with another label. subset(x, over_cap) is the report

See Also

get_label for looking up a single label, add_labels for registering labels for derived variables, apply_label_overrides for applying study-specific overrides from a YAML file.

Examples

# Generate labeled survival data
dta <- generate_survival_data(n = 50, seed = 42)
lmap <- label_map(dta)
head(lmap)

# Use for publication-ready tables
summary_vars <- c("age", "bmi", "hgb_bs")
tbl <- data.frame(
  variable = summary_vars,
  description = lmap$label[match(summary_vars, lmap$key)],
  mean = sapply(dta[summary_vars], mean)
)
print(tbl)

# With sample data (has labels)
dta <- sample_data(n = 20)
label_map(dta)

# Which labels were cut, and what they were
subset(label_map(dta, label_max = 20), truncated)

# Keep the source text
label_map(dta, label_max = Inf)

# Labels sharing a heading keep what tells them apart, and the key says why
procs <- data.frame(avr = 1, mvr = 1)
attr(procs$avr, "label") <- "Surgical procedure: aortic valve replacement with root enlargement"
attr(procs$mvr, "label") <- "Surgical procedure: mitral valve repair"
lmap <- label_map(procs)
lmap$label
attr(lmap, "abbreviations")

Description

One row per level of each discrete variable, giving the text as stored and the text that would be displayed once a leading code prefix is removed. Like strip_level_prefix it reports; it changes nothing.

Usage

level_map(data, vars = NULL, max_levels = 20L)

Arguments

Details

The shape deliberately mirrors label_map one level down: level, level_full and stripped read the same way as label, label_full and truncated. So subset(level_map(data), stripped) answers "which levels will print differently", exactly as subset(label_map(data), truncated) answers the question for labels.

Where the codes come from depends on what the column carries:

• a column with value labels - from value_labels.yml via apply_value_labels, or from a SAS format catalog - gives the declared codes;

• a factor gives its integer level positions, which is what the column actually stores;

• a character column has no codes, and code is NA.

Plain numeric and logical columns contribute no rows: they have no level text for a prefix to sit in front of.

High-cardinality columns are skipped, not expanded. A character column of identifiers would otherwise contribute thousands of rows to a report meant to be read. Columns with more than max_levels distinct values are omitted and named in a warning, so a skip is never silent.

Value

A data frame with one row per reported level and the columns

key

The variable name

code

The code the level maps to, as character, or NA

level

The text to display, with a prefix stripped where the rule in strip_level_prefix allows it

level_full

The level text as stored, never modified

stripped

Logical: TRUE where level differs from level_full

Zero rows, with those columns, when nothing is reportable.

See Also

strip_level_prefix for the rule itself, label_map for the same idea applied to variable labels, apply_value_labels for declaring level text outright.

Examples

dta <- data.frame(
  disp = factor(c("1. Home", "2. Rehab", "1. Home")),
  vess = factor(c("1 vessel", "2 vessel", "1 vessel"))
)
level_map(dta)

# Which levels will print differently
subset(level_map(dta), stripped)

Description

The headline, not the badge, is the claim to report. It is falsifiable and independent of whatever thresholds were chosen. A maximum relative discrepancy of exactly zero across many quantities is not a triumph – it means nothing was really compared – so it is flagged rather than celebrated.

Usage

parity_headline(df)

Arguments

Value

A single string.

Examples

parity_headline(compare_parity("a", 1.0001, 1, class = "mle_printed"))

Description

Tolerances are derived from what limits agreement for each quantity, not tuned until things pass. Three regimes:

Usage

parity_tolerance(class)

Arguments

Details

• Printed references are intervals, not numbers. When SAS prints -239.194, the value that produced it lies in [-239.1945, -239.1935). Half a unit in the last printed place is a floor that is derived.

• Stored references carry machine precision, so that floor does not apply. What remains is that two implementations run different optimizers on the same likelihood and converge to different points.

• Counts are exact, or there is a bug.

Value

A list with rtol and atol.

Examples

parity_tolerance("loglik")

Description

Defaults for concept_of, concept_map and friends, describing the naming conventions a legacy vars.sas uses: ln_ (log), in_ (inverse), in2 (inverse squared) and a trailing 2 (squared).

POOL_PLAIN_SUFFIX is the suffix untransformed measurements carry (crcl_pr, bun_pr, hct_pr). POOL_MIN_STEM guards the prefix rule: two characters would match half a pool by accident.

These are defaults, not constants – pass your study's own conventions through the affixes, plain_suffix and min_stem arguments where they differ, rather than editing a copy of this package.

Usage

POOL_AFFIXES

POOL_PLAIN_SUFFIX

POOL_MIN_STEM

Format

Character vectors, and an integer for POOL_MIN_STEM.

Examples

POOL_AFFIXES
pool <- c("age", "age2", "crcl_pr", "ln_crcl_pr")
concept_map(pool)
# A study whose trailing 2 does not mean "squared" drops that affix.
concept_map(pool, affixes = setdiff(POOL_AFFIXES, "2$"))

Description

What this catches that pruning cannot. concept_map groups transformations, by stripping a known affix and finding the parent. Two candidates that are numerically the same information but share no affix relationship are invisible to it.

male and female are the case that got through a real screen: exact complements, both offered, and 101 of 500 replicates selected both. With a free log_mu a design holding both is singular, so those replicates were fitting a rank-deficient model and nothing in the output said so.

Usage

pool_collinear_pairs(data, pool, threshold = 0.99)

Arguments

Details

Run it on the pool actually screened, after pruning. On an unpruned pool every ln_x correlates with its own x at ~1 and the report is a wall of expected pairs. After pruning, anything flagged is something pruning could not catch, which is exactly the set a human needs to look at.

complement separates the two cases a reader treats differently: an exact complement (x + y == 1) is a coding duplicate and one of the pair should simply not be a candidate, while a high correlation between genuinely different measurements is a judgement call about what the model can identify.

Value

A data frame with columns var1, var2, r and complement, ordered by decreasing abs(r). Empty when the pool is clean, so the caller decides whether to warn, fail, or print it.

See Also

concept_map, prune_to_one_form

Examples

d <- data.frame(male = c(1, 0, 1, 0), female = c(0, 1, 0, 1),
                age = c(60, 71, 55, 68))
pool_collinear_pairs(d, c("male", "female", "age"))

Description

Reports the version of every package a hazard-family analysis depends on. Run it on the machine that will do the work, before anything else.

Usage

preflight_report(extra = character(0))

Arguments

Details

numDeriv is only a Suggests of TemporalHazard, so install_github() does not pull it. Its absence silently costs standard errors on any interval- or left-censored multiphase fit: the analytic multiphase Hessian declines for those statuses by design and the optimizer falls back to numDeriv; with it missing there is no third option, and vcov() returns a bare logical while rcond and pd come back NA with nothing naming the cause.

R itself is reported from R.version, not from packageVersion(), so it is dropped from the package list: a caller who names it in extra would otherwise get a second R row reporting it as not found.

Value

A data frame with columns component, found, version and notes.

Examples

preflight_report()

Description

Reports the variable-level metadata that SAS PROC CONTENTS prints: creation position, name, SAS type, format, and label, alongside the R class and simple completeness counts.

Usage

proc_contents(data, order = c("alpha", "varnum"))

Arguments

Details

Reading a .sas7bdat through haven is lossy. Variable name, label, format.sas, and creation order survive; SAS storage LENGTH, POS (position within the observation), informat, and the dataset's created/modified timestamps do not. Those fields are omitted rather than inferred, because inferred values would look authoritative and would disagree with the source dataset whenever its LENGTH statement differed from the default.

"Survive" means preserved when the source carries them, not present on every variable. A SAS variable need not have a label or a format, and many do not: in one 879-variable clinical build, 865 variables carried a label and 395 carried a format.sas. An NA in format therefore reports that the source variable had no format, not that one was lost in reading. label does not behave this way — see Value.

The SAS Type column is two-valued, so the mapping from R is deliberately lossy: character and factor become "Char"; everything else, including logical, Date, and POSIXct, becomes "Num". A SAS date is a number carrying a date format, so type reports "Num" while class reports "Date" — keeping the disagreement visible rather than hiding it.

For a data frame with columns but zero rows — a fully filtered cohort, say — pct_missing is NaN rather than a number in 0-100, because mean(is.na(x)) on an empty vector is 0/0. The proportion of missing values among no values is genuinely undefined, and reporting 0 would assert that nothing is missing. Callers formatting this column for display should handle NaN explicitly.

Value

An object of class proc_contents: a list with two elements.

header

One-row data frame with observations (row count), variables (column count), and label (dataset label, or NA)

variables

Data frame with one row per column and the columns num (creation position), variable, type ("Num"/"Char"), format (SAS format, or NA), label, class (R class), n_unique (distinct non-NA values), and pct_missing (0-100, 1 decimal place, or NaN when data has zero rows)

label is never NA: it falls back to the variable's own name when the source carries no label, via labelled::var_label(null_action = "fill"). An unlabelled variable is therefore indistinguishable here from one whose label equals its name, and both are common. Callers recording this output as a durable description of a source dataset — a schema sidecar, a data dictionary kept after the source is retired — should read labels from the source attributes directly rather than from this column, or they will record a fallback as though it were the dataset's own metadata.

See Also

proc_means for numeric summaries, data_dictionary for a flattened documentation table.

Examples

dta <- sample_data(n = 50)
pc <- proc_contents(dta)
pc$header
pc$variables

# Creation order rather than alphabetical
proc_contents(dta, order = "varnum")$variables

Description

Produces the table SAS PROC FREQ prints for a TABLES statement: one row per observed combination of levels, with frequencies and percentages. One call is one table; a SAS step with several TABLES statements is several calls.

Usage

proc_freq(data, tables, missing = FALSE, list = FALSE, weights = NULL)

Arguments

Details

The last name in tables is the column variable, the one before it the row variable, and any earlier names are strata, as in SAS TABLES a*b*c.

Percentages depend on list, not only the columns shown. For a crosstab of three or more variables, SAS prints one row-by-column table per stratum and computes Percent within that stratum. With list = TRUE, Percent is of the grand total. The two agree for one- and two-way tables and disagree from three variables on.

Missing values sort first. SAS treats missing as the smallest value, so under missing = TRUE the missing level heads the table and the cumulative columns count it first. dplyr::count() places NA last, which moves every cumulative value.

Rows follow SAS ORDER=INTERNAL: factors in declared level order, numbers by value, character values by byte value (the C locale, so "B" sorts before "a"). Unused factor levels are omitted, as SAS omits zero-count levels without SPARSE.

A haven_labelled variable is grouped on its value label, as SAS forms levels from formatted values: stored codes sharing a label form one row, shown as the smallest of those codes, and a <var>_label column holding the label follows it. A value with no label is its own row. Variable labels on the tables columns are kept.

What counts as missing. As in SAS, a blank or whitespace-only character value is missing, and so is NaN. SAS special missing values, read by haven as tagged_na, are separate missing levels in SAS order: ._, then ., then .A to .Z.

Where this differs from SAS. Unlabelled doubles are grouped on their exact values, whereas SAS groups on the printed (BEST12.) value, so values that differ past about 12 significant digits stay separate rows here. Non-positive weights are an error, whereas SAS drops zero weights and ignores negative ones; proc_means makes the same deliberate choice.

Percentages are not rounded. SAS rounds only for display, and a rounded result would fail compare_parity against SAS output.

Tests of association (CHISQ, FISHER, MEASURES and the rest) are deliberately absent, for the same reason proc_means has no inference statistics.

Value

A data frame with one column per tables variable (each followed by a <var>_label column when the variable carries value labels), then:

• for a one-way table, or list = TRUE: Frequency, Percent, Cum_Frequency, Cum_Percent;

• for an n-way crosstab: Frequency, Percent (within stratum), Row_Percent, Col_Percent.

Frequency is integer when unweighted and double when weighted. The attribute frequency_missing holds the count (or weight) of excluded missing rows, and is 0 when none were excluded. A table with no remaining rows is returned with zero rows, not an error.

See Also

proc_means, proc_contents

Examples

dta <- generate_survival_data(n = 200, seed = 42)

# proc freq; table dead / missing;
proc_freq(dta, "dead", missing = TRUE)

# proc freq; table sex * dead;
proc_freq(dta, c("sex", "dead"))

# proc freq; table sex * dead / list;
proc_freq(dta, c("sex", "dead"), list = TRUE)

Description

Produces the table SAS PROC MEANS prints: one row per analysis variable, one column per requested statistic, in the order requested.

The default statistics are not SAS's. PROC MEANS with no statistic list prints N, mean, standard deviation, minimum and maximum, and no percentiles. The HVTI group reports the 15th and 85th percentiles alongside the median in its descriptive summaries, rather than the quartiles, so the default here adds p15, median and p85. A SAS parity comparison must therefore name the same statistics on both sides: pass the SAS job's list to stats, or add P15 MEDIAN P85 to the SAS PROC MEANS statement.

Usage

proc_means(
  data,
  vars = NULL,
  class = NULL,
  stats = c("n", "mean", "std", "min", "p15", "median", "p85", "max"),
  weights = NULL
)

Arguments

Details

Quantiles use stats::quantile(type = 2), which is the R equivalent of SAS's default QNTLDEF=5. This matters: R's own default is type = 7, a different estimator that disagrees with SAS on the small and even-numbered samples clinical subgroups produce. For c(1, 2, 3, 4), the first quartile is 1.5 in SAS and 1.75 under R's default. The median agrees, which is why the discrepancy hides.

With vars = NULL all numeric columns are analysed, matching SAS's behaviour when the VAR statement is omitted. Logical columns are not is.numeric() in R and so are excluded from that default set; naming one in vars coerces it to 0/1, making mean a proportion as PROC MEANS would give.

Rows are ordered by analysis variable first, then by class level. A factor class variable orders by its declared level order rather than alphabetically, matching SAS's default ORDER=INTERNAL; this keeps ordered clinical scales such as NYHA class in their clinical sequence.

Weights do not apply uniformly, and this follows PROC MEANS rather than being a simplification. Weighted: mean, std, var, cv, stderr, sum, uss, css, skewness, kurtosis, sumwgt. Unweighted: n, nmiss, nobs, min, max, range, mode, and every quantile – median, q1, q3, pNN and qrange. PROC MEANS does not compute weighted quantiles at all; that is PROC UNIVARIATE. So proc_means(d, stats = "median", weights = "wt") returns the unweighted median.

mode returns the smallest value among tied modes, and NA when no value repeats, except that a single observation is its own mode; all three match SAS. Weighted stderr divides the weighted standard deviation by the square root of the sum of the weights, as SAS does, not by the square root of the count. skewness and kurtosis are the adjusted Fisher-Pearson forms SAS uses, not R's naive moment ratios, and are NA for a constant column rather than NaN.

The PROC UNIVARIATE inference statistics (NORMAL, PROBN, T, PROBT, MSIGN, PROBM, SIGNRANK, PROBS) are deliberately absent: they would make this a hypothesis-testing function rather than a summary one. CLM, the confidence limits of the mean, is absent for the same reason.

cv is NA when the mean is zero, matching SAS; R's arithmetic would give Inf.

Value

A data frame with one row per analysis variable per class combination. Columns are the class variables (when supplied), then variable, label, then one column per requested statistic in the order given by stats. Count statistics (n, nmiss) are integer; all others are numeric.

See Also

proc_contents for variable metadata.

Examples

dta <- generate_survival_data(n = 200, seed = 42)

# The group's default statistics over every numeric variable
head(proc_means(dta))

# Named variables and an explicit statistic list
proc_means(dta, vars = c("age", "bmi"),
           stats = c("n", "mean", "median", "p15"))

# Weighted statistics
dta <- data.frame(age = c(51, 63, 47, 72), wt = c(1, 2, 1, 4))
proc_means(dta, vars = "age", stats = c("n", "sumwgt", "mean"),
           weights = "wt")

Description

Produces the statistics data set SAS PROC UNIVARIATE writes with OUTPUT OUT=: one row per analysis variable (per class level), one column per requested statistic in the order requested, then one column per PCTLPTS= percentile. Beyond proc_means it adds weighted and decimal percentiles, VARDEF=, and the tests for location and normality.

Usage

proc_univariate(
  data,
  vars = NULL,
  class = NULL,
  stats = c("n", "median", "mean", "std", "cv", "min", "max"),
  weights = NULL,
  pctlpts = NULL,
  pctlpre = "p",
  mu0 = 0,
  vardef = c("df", "n", "wdf", "weight")
)

Arguments

Details

Weighting. Weights apply as in proc_means, except that the quantiles are weighted: median, q1, q3, qrange, pNN and the pctlpts columns. The weighted percentile sorts the values with their cumulative weights $S_i$; the 0th percentile is the minimum and the 100th the maximum; otherwise, when $S_i = pW$ the result is the mean of $x_i$ and $x_{i+1}$, else the first $x_i$ with $S_i > pW$. Unweighted quantiles use stats::quantile(type = 2), SAS's default PCTLDEF=5. mode stays unweighted, as in SAS. stderr and stdmean divide the standard deviation by the square root of the sum of the weights. A zero or negative weight is an error; SAS instead treats a negative weight as zero and still counts the observation.

VARDEF. The variance divisor is $n - 1$ ("df"), $n$ ("n"), $W - 1$ ("wdf") or $W$ ("weight"), where $W$ is the sum of the weights ($n$ without weights). It changes var, std and cv. Under "n", skewness and kurtosis are the moment forms rather than the adjusted ones.

Statistics SAS does not compute are NA, as SAS writes missing:

• msign, probm, signrank, probs, normal, probn when weights is given;

• t, probt, stderr and stdmean when vardef is not "df";

• skewness and kurtosis when vardef is "wdf" or "weight";

• normal and probn above 2000 observations, with one warning per call: SAS switches to a Kolmogorov D test there, which is not ported;

• std, var, cv, stderr, stdmean, t, probt at one observation, skewness below three and kurtosis below four;

• t, probt, skewness, kurtosis, normal, probn when every value is equal (cv is then 0 unless the mean is zero, where it is NA);

• msign, probm, signrank, probs when no value differs from mu0;

• mode when no value repeats among two or more values.

Tests. t is Student's t for the mean against mu0; msign the sign statistic; signrank the Wilcoxon signed rank statistic, with an exact p-value for 20 or fewer non-zero differences and SAS's t approximation above that. normal is the Shapiro-Wilk $W$, from stats::shapiro.test(), which implements Royston's 1995 algorithm; SAS implements Royston's 1992 one, so normal and probn agree with SAS to about $10^{-8}$, not to machine precision. At two observations both are 1, as SAS reports.

Value

A data frame with one row per analysis variable per class combination. Columns are the class variables (when supplied), then variable, label, then one column per stats keyword in the order given, then one column per pctlpts point in the order given. n, nmiss and nobs are integer; all others are numeric.

See Also

proc_means for the descriptive statistics alone.

Examples

dta <- generate_survival_data(n = 200, seed = 42)

# SAS unistats default statistics over every numeric variable
head(proc_univariate(dta))

# Decimal percentiles, as for a bootstrap interval
proc_univariate(dta, vars = "age", stats = "n",
                pctlpts = c(2.5, 50, 97.5))

# Tests for location and normality
proc_univariate(dta, vars = "bmi",
                stats = c("mean", "t", "probt", "normal", "probn"),
                mu0 = 25)

Description

Records an existing artifact beneath the study root by canonical study-relative path, role, byte count, modification time, and SHA-256 hash.

Usage

provenance_artifact(path, role = "input", cfg = study_config())

Arguments

Value

A plain list containing one immutable-by-convention artifact record.

See Also

provenance_data, capture_provenance

Examples

root <- file.path(tempdir(), "provenance-artifact-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

writeLines("x <- 1", file.path(root, "30_analyses", "fit.R"))
str(provenance_artifact("30_analyses/fit.R", role = "code", cfg = cfg))
unlink(root, recursive = TRUE)

Description

Records the authoritative physical file selected by dataset. For a manifest entry with role: "primary", this is the promoted Parquet file rather than the retired source, and for a dataset registered with register_data it is the registered version, the dated Parquet file, rather than the source it was converted from. The returned plain list contains the logical dataset name, a canonical study-relative path, its role, byte count, modification time, and SHA-256 hash.

Usage

provenance_data(
  dataset = "study",
  cfg = study_config(),
  role = "analysis",
  working = .hvti_working_default()
)

Arguments

Value

A plain list containing one immutable-by-convention data record.

See Also

provenance_artifact, capture_provenance

Examples

root <- file.path(tempdir(), "provenance-data-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

# The registered Parquet file, not the CSV it was converted from.
str(provenance_data("study", cfg = cfg))
unlink(root, recursive = TRUE)

Description

Returns the path of the sidecar belonging to a rendered output: the output path with its extension replaced by .provenance.json.

Usage

provenance_path(path)

Arguments

Value

Character(1). The sidecar path.

See Also

publish_provenance, record_provenance

Examples

provenance_path("_output/death-hz-ac.html")

Description

Prune a candidate pool to one form per concept

Usage

prune_to_one_form(pool, ...)

Arguments

Value

The kept names, in the pool's original order, so the scope handed to a screen stays stable across runs.

See Also

concept_map, pool_collinear_pairs

Examples

prune_to_one_form(c("age", "ln_age", "age2", "bsa"))

Description

Requires an existing regular output, binds its byte count and SHA-256 hash to a previously captured payload, and writes the JSON sidecar through a same-directory temporary file and rename. Capture-session facts are not recomputed during publication.

Usage

publish_provenance(path, payload)

Arguments

Value

Invisibly, the published record as a list.

See Also

capture_provenance, record_provenance, provenance_path

Examples

root <- file.path(tempdir(), "publish-provenance-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

payload <- capture_provenance("death-hz-ac",
                              data = list(provenance_data(cfg = cfg)),
                              cfg = cfg)
# Publish only once the output exists.
out <- file.path(root, "40_graphs", "death-hz-ac.html")
writeLines("<p>report</p>", out)
invisible(publish_provenance(out, payload))
file.exists(provenance_path(out))
unlink(root, recursive = TRUE)

Description

Intelligently converts column types in a dataset based on their content. Handles character-to-factor conversion, binary numeric variables, and various NA representations. Preserves variable labels from SAS/labelled data.

Usage

r_data_types(
  dataset,
  factor_size = 10,
  skip_vars = NULL,
  binary_factor = FALSE,
  use_value_labels = FALSE
)

Arguments

Details

Each column is converted by the first rule that matches, in this order:

1. When use_value_labels = TRUE and the column carries value labels, they become the factor levels. This runs ahead of every rule below: a declared type is not subject to a threshold on distinct values.

2. Character strings "NA", "na", "Na" and "nA" become NA.

3. Numeric or integer columns with exactly 2 distinct values become logical, or factors when binary_factor = TRUE – but only when the two values are 0 and 1. Any other 2-distinct-value coding (a SAS-style 1/2, say) becomes a factor outright, regardless of binary_factor: as.logical() maps every nonzero code to TRUE, so a 1/2 column would otherwise become TRUE for every row and silently lose one of its two categories.

4. Remaining character columns become factors.

5. Numeric columns with 3 to factor_size distinct values become factors.

6. Logical columns become factors when binary_factor = TRUE.

Date, POSIXct, and POSIXlt columns are never altered by type conversion.

Which rule fired on which column is recorded on the result and read back with type_conversion_report.

Value

An object of the same class as dataset with columns converted according to the function's rules. Variable labels are preserved. The result carries a per-column conversion report, read with type_conversion_report.

See Also

type_conversion_report for the record of which rule fired on which column.

Examples

# Basic usage with sample data
dta <- sample_data(n = 100)
str(dta)  # Original types
dta_converted <- r_data_types(dta, use_value_labels = FALSE)
str(dta_converted)  # Converted types

# Real data example with mtcars
str(datasets::mtcars$vs)  # numeric (0/1)
mtcars_converted <- r_data_types(datasets::mtcars,
                                 use_value_labels = FALSE)
str(mtcars_converted$vs)  # logical (FALSE/TRUE)

# Skip specific columns
mtcars_partial <- r_data_types(datasets::mtcars,
                               skip_vars = c("vs", "am"),
                               use_value_labels = FALSE)
str(mtcars_partial$vs)  # Still numeric (unchanged)

# Control factor creation threshold
mtcars_strict <- r_data_types(datasets::mtcars, factor_size = 5,
                              use_value_labels = FALSE)

# Keep binary variables as factors
mtcars_factors <- r_data_types(datasets::mtcars, binary_factor = TRUE,
                               use_value_labels = FALSE)
str(mtcars_factors$vs)  # Factor instead of logical

# Keep the level text of a SAS formatted variable
disp <- haven::labelled(c(1, 2, 1, 3),
                        labels = c(Home = 1, Rehab = 2, SNF = 3),
                        label  = "Discharge disposition")
converted <- r_data_types(data.frame(disp = disp), use_value_labels = TRUE)
levels(converted$disp)
type_conversion_report(converted)

Description

Parses every .R file in dir and reports any top-level expression that is not an assignment. Function definitions and constants pass; calls do not.

Use this on any directory that is sourced wholesale, where a stray call would execute on every render.

Usage

r_dir_impurities(dir)

Arguments

Value

A character vector of complaints, one per offending expression, empty when the directory is clean. The caller decides whether to warn or to fail.

Examples

d <- file.path(tempdir(), "purity-example")
dir.create(d, showWarnings = FALSE)
writeLines(c("f <- function(x) x + 1", "print(f(1))"),
           file.path(d, "example.R"))
r_dir_impurities(d)
unlink(d, recursive = TRUE)

Description

Reads the dataset named in _study.yml and normalises its types so that both available read paths deliver the same frame.

For a dataset registered with register_data, the registered version (a dated parquet) is read, after its checksum is checked. If the source file has been rebuilt since, the registered version is still read and a message of class hvtiRutilities_source_changed says so and names update_manifest().

A combined dataset (see register_data) is still read when a dataset it was built from has been updated since. A message of class hvtiRutilities_parent_changed, which inherits hvtiRutilities_out_of_date, names the parents' recorded and current versions and the commands that update it. A combined dataset with no recorded parent versions is reported the same way, as unrecorded, until it is rebuilt and registered again.

Only a dated version is registered data. Any other read is a working read: it is allowed only in the interactive console and in jobs in the study's datasets or descriptive folder, and it signals a message of class hvtiRutilities_working_read. Review a rebuilt source with review_built() and register it with update_manifest(). The release branch below is deprecated.

For a release-aware contract, the pinned release is verified before cache access. A later valid release emits a message of class hvtiRutilities_update_available; an unavailable catalog or invalid candidate emits hvtiRutilities_update_status_unknown and the pinned data are still read. A changed pinned release errors with class hvtiRutilities_release_integrity. A withdrawn pin errors with class hvtiRutilities_withdrawn_release unless allow_withdrawn is TRUE.

The normalisation is not cosmetic. read_clinical_data converts SAS 0/1 numerics to logical while haven::read_sas() leaves them numeric, and downstream modelling code rejects a logical status vector outright — so without this the same document would run under one read path and fail under the other. Labelled vectors are likewise reduced to plain vectors, keeping the SAS variable label as an attribute because listings print labels rather than names.

Usage

read_built(
  cfg = study_config(),
  refresh = FALSE,
  dataset = "study",
  allow_withdrawn = FALSE,
  working = .hvti_working_default()
)

Arguments

Value

A data frame with lower-cased names, no logical columns and no haven_labelled columns.

See Also

built_manifest, assert_cohort

Examples

if (requireNamespace("arrow", quietly = TRUE)) {
  root <- file.path(tempdir(), "read-built-example")
  suppressMessages(study_setup(root, "Example", 1L, renv = FALSE))
  write.csv(data.frame(DEAD = c(1, 0, 0), IV_DEAD = 1:3),
            file.path(study_dir("datasets", root), "example.csv"), row.names = FALSE)
  suppressMessages(register_data(root, "example.csv"))
  names(read_built(study_config(root)))
  unlink(root, recursive = TRUE)
}

Description

A convenience wrapper that detects the file type, reads the data with the appropriate reader, and optionally runs r_data_types to convert column types. This saves novice users from having to remember which package reads which format and ensures labels are preserved.

Supported formats:

.sas7bdat

SAS datasets via haven::read_sas()

.csv

Comma-separated files via utils::read.csv() with check.names = FALSE, so column names are preserved exactly as written in the file (spaces, hyphens, and special characters are not silently converted to .).

.xlsx, .xls

Excel workbooks via readxl::read_excel()

.rds

R serialized objects via readRDS()

.parquet

Parquet files via arrow::read_parquet(); needs the arrow package

Usage

read_clinical_data(file, convert_types = FALSE, ..., catalog_file = NULL)

Arguments

Details

SAS format catalogs. A .sas7bdat file stores a format's name – YESNOF. – and not its values. The code-to-text mapping lives in a separate .sas7bcat catalog. Without one, reading a SAS dataset yields numeric codes and no value labels, however the read is written, and no amount of downstream work can recover text that never arrived. Pass catalog_file to read the two together.

A catalog on its own is only half the journey. With convert_types = TRUE, r_data_types discards value labels unless use_value_labels = TRUE is passed through ..., so supplying a catalog without it warns.

Value

A data frame with labels preserved and (optionally) types converted.

See Also

r_data_types for details on type conversion, label_map to extract labels after reading.

Examples

# Read a CSV
tmp <- tempfile(fileext = ".csv")
write.csv(mtcars, tmp, row.names = FALSE)
dta <- read_clinical_data(tmp, convert_types = FALSE)
str(dta[, 1:5])
unlink(tmp)

# Read without type conversion
tmp <- tempfile(fileext = ".csv")
write.csv(mtcars, tmp, row.names = FALSE)
dta_raw <- read_clinical_data(tmp, convert_types = FALSE)
str(dta_raw[, 1:5])
unlink(tmp)

# Read an RDS file
tmp <- tempfile(fileext = ".rds")
saveRDS(iris, tmp)
dta <- read_clinical_data(tmp, convert_types = TRUE, factor_size = 5)
str(dta)
unlink(tmp)

Description

Convenience wrapper for an output that already exists. It captures an explicit provenance payload and immediately publishes it beside the output.

Usage

record_provenance(
  path,
  data,
  artifacts = list(),
  extra = list(),
  cfg = study_config()
)

Arguments

Value

Invisibly, the published record as a list.

See Also

capture_provenance, publish_provenance, provenance_path

Examples

root <- file.path(tempdir(), "record-provenance-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

out <- file.path(root, "40_graphs", "death-hz-ac.html")
writeLines("<p>report</p>", out)
rec <- record_provenance(out, data = list(provenance_data(cfg = cfg)),
                         cfg = cfg)
rec$output$file
file.exists(provenance_path(out))
unlink(root, recursive = TRUE)

Description

Completes the default study data contract or adds one distinctly named dataset. The function derives row and column counts from the file and replaces _study.yml and manifest.yaml only after both updated files have been prepared successfully.

Registration converts the file once to a dated parquet in the same folder, <name>_YYYYMMDD.parquet, with its column record beside it as <name>_YYYYMMDD.schema.csv. That parquet is what read_built and the job templates read, so the source file may be rebuilt freely; run update_manifest() to register a rebuilt file as a new version. The date is the file's modification date unless extract_date is given. Conversion needs the arrow package. A release-aware registration (catalog_dataset and release_id) records the catalog's file as it is and converts nothing.

A dataset is registered once: registering it again stops with "already registered". So a study whose datasets are already registered adds kind, key or parents by editing that dataset's entry in _study.yml, then running update_manifest(). The key is checked when the next version is registered. A combined dataset records its parents' versions only once its source has been rebuilt, and reads as out of date until then. A release-aware dataset is skipped by update_manifest(): its parents' versions are recorded when a release is adopted with adopt_data_update(), and its key is checked only at registration.

Usage

register_data(
  root = getwd(),
  built,
  dataset = "study",
  role = c("study", "named"),
  population = NULL,
  source = NULL,
  extract_date = NULL,
  catalog_dataset = NULL,
  release_id = NULL,
  kind = NULL,
  key = NULL,
  parents = NULL
)

Arguments

Value

An object of class "study_status", returned visibly.

Where a subset goes

A subset of the study dataset has three homes, and which to pick depends on who needs it.

• A WHERE condition in one job writes no file. The job's hvtiRtemplates::read_job_data() applies the condition and records it. Use this for a subset that only one job analyses.

• An analysis set from hvtiRdatabuild::write_analysis_set() is saved as 00_datasets/<name>.parquet with <name>.set.yml beside it, and is derived from the study dataset only. A job reads it with DATASET <- "built" and ANALYSIS_SET <- "<name>". Use this for a subset of the study dataset that several jobs share.

• A separately registered dataset, role = "named" with kind = "subset", is for a subset built some other way, such as from a second source or by a script outside the study's build.

Exclusions that define the cohort itself belong in the build job (hvtiRtemplates::build_cohort()), not in a subset.

See Also

study_setup, study_config, update_manifest

Examples

root <- file.path(tempdir(), "register-data-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
register_data(root, built = "built.csv", key = "ccfid")
# The CSV is kept; the registered version is a dated Parquet file.
list.files(file.path(root, "00_datasets"))
unlink(root, recursive = TRUE)

# An additional dataset is registered with role = "named", under a name of
# its own, and read back by that name.
root <- file.path(tempdir(), "register-data-named-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv", key = "ccfid"))
utils::write.csv(
  data.frame(ccfid = c(1, 1, 2), echo_date = c("2020-01-05", "2020-06-01", "2020-02-10"),
             lvef = c(55, 50, 60)),
  file.path(root, "00_datasets", "echo.csv"),
  row.names = FALSE
)
invisible(register_data(root, built = "echo.csv", dataset = "echo", role = "named",
                        kind = "ancillary", key = c("ccfid", "echo_date")))
read_built(study_config(root), dataset = "echo")
unlink(root, recursive = TRUE)

Description

Compares the working dataset, as it stands in the study's datasets folder, with its registered version, the dated parquet jobs read. It reads both and writes nothing. Run it after a rebuild and before update_manifest(), which registers the rebuilt file as the next version. It describes how the data changed; it does not certify that the new file is analytically or clinically correct.

Usage

review_built(cfg = study_config(), dataset = "study")

Arguments

Value

An object of class "built_review" with

dataset

The logical dataset name.

registered

The current version record from manifest.yaml.

working

A list with the source file's sha256, n_rows and n_cols.

changed

TRUE when the source differs from the registered version.

comparison

A compare_datasets result, or NULL when unchanged.

See Also

update_manifest, read_built

Examples

if (requireNamespace("arrow", quietly = TRUE)) {
  root <- file.path(tempdir(), "review-built-example")
  suppressMessages(study_setup(root, "Example", 1L, renv = FALSE))
  path <- file.path(study_dir("datasets", root), "example.csv")
  write.csv(data.frame(DEAD = c(1, 0, 0), IV_DEAD = 1:3), path, row.names = FALSE)
  suppressMessages(register_data(root, "example.csv"))
  write.csv(data.frame(DEAD = c(1, 0, 0, 1), IV_DEAD = 1:4), path, row.names = FALSE)
  review_built(study_config(root))
  unlink(root, recursive = TRUE)
}

Description

Deprecated. Catalog releases are retired; use review_built.

Verifies the pinned release and one exact, newer candidate, then compares their structure. Review reads the source files directly; it does not write caches or change either study manifest. It describes data drift, but it does not certify that a candidate is analytically or clinically correct.

Usage

review_data_update(cfg = study_config(), dataset = "study", release_id)

Arguments

Value

An object of class "data_update_review" with

dataset

The logical study dataset name.

pinned,candidate

The catalog records for both releases.

comparison

A compare_datasets result.

See Also

check_data_updates, adopt_data_update

Examples

root <- file.path(tempdir(), "review-data-update-example")
invisible(study_setup(root, study = "Example", study_tracker_id = 1L, renv = FALSE))
utils::write.csv(data.frame(ccfid = 1:3, age = c(61, 70, 55)),
                 file.path(root, "00_datasets", "built.csv"),
                 row.names = FALSE)
invisible(register_data(root, built = "built.csv"))
cfg <- study_config(root)

# Deprecated: it warns and names review_built() instead, then stops,
# because this study is not pinned to a catalog release.
try(review_data_update(cfg, release_id = "surgery_cohort-20260921-r1"))
unlink(root, recursive = TRUE)

Description

Creates a data frame with labeled columns of various types, suitable for demonstrating r_data_types and label_map.

Usage

sample_data(n = 100)

Arguments

Value

A data frame with n rows and 7 labeled columns:

id

Integer sequence (Patient Identifier)

boolean

Integer 1/2 (Binary Indicator)

logical

Character "F"/"T" (Logical Status)

f_real

Uniform random values (Random Uniform Value)

float

Normal random values (Random Normal Value)

char

Character "male"/"female" (Gender)

factor

Factor C1-C5 (Category Group)

Examples

# Create and inspect labeled data
dta <- sample_data(n = 20)
str(dta)
label_map(dta)

# Full workflow: generate, convert types, extract labels
dta <- sample_data(n = 100)
dta_clean <- r_data_types(dta, skip_vars = "id")
lmap <- label_map(dta_clean)
print(lmap)

Description

Scans a '.sas' file and returns one row for each ' contains. SAS files in the legacy CORR library are macro *collections*, not single macros, so a file routinely defines several.

Usage

sas_macro_defs(file)

Arguments

Details

Matching is case-insensitive: SAS macro names are case-insensitive and the legacy corpus mixes ' definitions are handled by depth counting, so an inner ' terminate an outer macro.

The body hash is a SHA-256 digest of the definition with case folded and whitespace runs collapsed. Two definitions with the same hash are semantically identical up to formatting.

An unmatched ' containing no '

Value

A data.frame with one row per macro definition and columns file, macro, params, body_hash, line_start, and line_end. Macro and parameter names are lowercased.

Examples

f <- system.file("extdata", "example_macro.sas", package = "hvtiRutilities")
if (nzchar(f)) sas_macro_defs(f)

Description

Legacy CORR macros carry a structured comment block naming the macro, its purpose, its documented call signature, and a modification log. This function extracts those fields. They are used as evidence during human review of divergent macro redefinitions, and the documented call is a direct input to the Phase 1 SAS harness.

Usage

sas_macro_signature(file)

Arguments

Details

All fields are optional. A file with no header block yields a row of NAs rather than an error, because an undocumented macro is a normal occurrence in this corpus and not a defect.

When several MODIFIED BY lines are present, the last date is returned. Note that this date is advisory evidence only; sas_triage() never uses it to choose between competing macro definitions.

Value

A one-row data.frame with columns file, macro_name, short_desc, created_on, modified_on, and documented_call. Absent fields are NA_character_. macro_name is lowercased.

Examples

f <- system.file("extdata", "example_macro.sas", package = "hvtiRutilities")
if (nzchar(f)) sas_macro_signature(f)

Description

Joins its arguments onto the study root. A first component naming a logical study directory is resolved through the study's numbered or legacy layout. Other components are passed through unchanged. Use this instead of any literal path: the same study is mounted at different absolute paths on the analysis server and on a laptop.

Usage

sas_path(..., start = getwd())

Arguments

Value

Character(1). The joined path. It is not checked for existence.

See Also

study_root

Examples

root <- file.path(tempdir(), "sas-path-example")
dir.create(file.path(root, "datasets"), recursive = TRUE,
           showWarnings = FALSE)
yaml::write_yaml(
  list(study = "Example", built = "example.sas7bdat"),
  file.path(root, "_study.yml")
)
sas_path("datasets", start = root)
unlink(root, recursive = TRUE)

Description

Applies an ordered rule ladder to every SAS source file in dir and returns one row per macro definition, each carrying a decision and the evidence that justifies it.

Usage

sas_triage(dir, overrides = NULL)

Arguments

Details

Only the top level of dir is scanned; subdirectories are ignored.

Candidate files are selected by excluding known non-source suffixes (logs, listings, documents, binary data sets, numbered RCS backups) rather than by matching .sas. The library names files with dots as word separators – deciles.hazard and lm.cprobs are names, not stems with extensions – and many macro files carry no extension at all, so a .sas-only pattern silently omits them.

File-level rules, applied first, in order:

1. Copy of * – drop, as a filename-prefix duplicate.

2. *.sas~ – drop, as an editor backup superseded by construction.

3. Fails .sas_lint() – drop, citing the specific lint failure.

Definition-level rules, applied per macro name across surviving files:

1. [4.] Defined in exactly one file – canonical.

2. [5.] Defined in several files, all bodies hashing identically – canonical, recording every defining file.

3. [6.] Defined in several files with two or more distinct bodies – ambiguous. No canonical file is chosen.

Rule 6 never auto-resolves. Modification dates and visibility are attached as evidence for a human, never consumed as a tiebreaker, because a tool that silently picks a winner launders a coin-flip into a committed artifact. Filesystem mtime is deliberately never consulted: the corpus froze in 2019 and lives on a network volume where timestamps do not survive copies.

Ambiguity is resolved by a human writing an entry into overrides and re-running. This makes the result reproducible and auditable.

Value

A data.frame with one row per macro definition and columns file, macro, params, body_hash, line_start, line_end, visibility, decision, rule, and evidence. Rows are sorted by macro then file, so the result is deterministic.

Examples

d <- system.file("extdata", "macros", package = "hvtiRutilities")
if (nzchar(d)) sas_triage(d)

Description

The SAS jobs this package's callers reproduce specify their variables as blocks in the .sas: a phase name on its own line, then comma-separated names, terminated by a bare ;.