hvti_taxonomy() gains vp, a VarPro variable screen with coverage, noise and stability checks, for the vp-fit and vp-explain templates in hvtiRtemplates.study_update_identity() records a verified Study Tracker identity in an existing _study.yml: it writes the record's fields, sets identity_source: tracker and identity_verified: true, keeps every other key (the data contract included), refuses a record for another Study Tracker ID, and returns the keys it changed. It is the write step of study-setup --verify, which reads the Tracker and compares the record with the typed identity first.print() of a study_status() no longer fails with "subscript out of bounds" on a study whose identity is UNVERIFIED (a manual identity); the row prints as [~].study_setup(renv = TRUE) now sets use.cache to FALSE in the study's renv/settings.json, so the shared study library holds real copies of its packages and not links into one person's private renv cache, which teammates cannot read and which made their installs fail.study_need() no longer records already-installed packages in the study DESCRIPTION when the user declines installing a missing one, or the install fails: a refusal leaves the study as it was.The register_data() help page now says what role and dataset mean and how they go together: role takes one value, "study" for the study's single main dataset (the default) or "named" for another dataset under the name given in dataset. It also gains a "Where a subset goes" section comparing a job WHERE, an analysis set and a registered subset, and an example that registers a named ancillary dataset and reads it back.
study_setup() now cleans up template copies whenever it adopts a study (cleanup defaults to adopt,
including an adoption accepted at the prompt) and sets up renv by default (renv = TRUE). Adopting a study
where the template corpus is absent now stops unless template_dir is given or cleanup = FALSE.
renv environments are created in explicit mode: renv.lock is built from the study's DESCRIPTION, never by
scanning its code, so stale template copies cannot add packages to it. .renvignore also covers tp.* and
old_RF_templates*/ for studies set up before.
New study_declare() records the packages a study needs in its DESCRIPTION, and keeps every other field and
the existing entries as written.
study_setup(renv = TRUE) now stops before writing anything when renv is not installed or the root has a
package DESCRIPTION; renv = FALSE is the way out.
New study_need() installs a missing package such as arrow into the study's library and records it, instead
of stopping with an install.packages() line. Outside a renv study it names hvtiR::setup().
The README's Installation section now leads with hvtiR::install(), which
installs the whole HVTI R family in one call, and then gives
pak::pak("ehrlinger/hvtiRutilities") for this package alone.
study_setup() has a help-page example: it sets up a throwaway study under
tempdir(), adopts it again, and runs cleanup = TRUE against a throwaway
template corpus, then removes both.
Help-page examples for register_data(), the provenance functions
(provenance_data(), provenance_artifact(), capture_provenance(),
publish_provenance(), record_provenance()), study_checkpoint_push() and
the pool conventions (POOL_AFFIXES and friends). Each runs on a throwaway
study under tempdir() and removes it; the ones that register data run only
when arrow is installed.
Seven help pages lose their \dontrun{} examples for ones that run:
update_manifest() and verify_manifest() register and check a CSV under
tempdir(), data_dictionary() writes its CSV to a temporary file, the
deprecated check_data_updates(), review_data_update() and
adopt_data_update() show their deprecation on a throwaway study, and
hvtiRutilities.news() runs only in an interactive session.
packageDescription() returns NA for such a package rather than failing,
and the package entry then failed on $, so cache_fit() discarded its
computed result. The entry now records NA for the package's source and
version and no commit.A _study.yml that is not valid YAML now stops with a message that names
the function, the file, the line and the usual cause, instead of the YAML
parser's bare "Scanner error" from yaml.load. A hand-edited value holding
": ", such as a long study title, reads as a nested mapping unless it is
quoted; the message says so (#213).
Registered versions record how their data were produced. manifest.yaml gains a
provenance: map, keyed by parquet name: origin: bd with a pointer to the build record
when a build left a <stem>.build.yml describing exactly the file registered, and
origin: legacy otherwise. verify_manifest() checks each build record.
read_built(working = TRUE), or HVTI_DATA=working, reads the working dataset before it is
registered. Any read that is not of a dated version is a working read: it is allowed in the
interactive console and in datasets and descriptive jobs, and signals
hvtiRutilities_working_read. A dataset pinned to a catalog release is exempt until the release
path is removed. A study still on a legacy registration stops in every other job until
update_manifest() converts it.
New review_built() compares a rebuilt dataset with its registered version before
update_manifest() registers it.
The catalog release path is deprecated: check_data_updates(), review_data_update(),
adopt_data_update(), register_data(catalog_dataset =, release_id =) and
read_built(allow_withdrawn =) warn, and study_config() warns once on a release:
contract. They will be removed in a later release.
update_manifest() in a study registers a dataset that _study.yml names but manifest.yaml
does not list (or a study with no manifest.yaml) as its first dated version, so a
hand-written built: no longer leaves read_built() and register_data() refusing each other.
The working-read gate stops any job outside the datasets and descriptive folders: a job in any
other stage folder, a job at the study root, a script outside the study, and a non-interactive
session with no job file, such as Rscript -e, targets or callr. HVTI_DATA must be unset or
"working"; any other value stops every read_built() and provenance_data(). A dataset with
no dated version stops those jobs and the message names update_manifest(), the only way out
once _study.yml names the dataset. A job directly in the study root is named as outside any
stage folder.
provenance_data(working = ) is new. It records the working source file, as
read_built(working = TRUE) reads it, and is validated the same way.
update_manifest() registers a version with its build record. It renames <stem>.build.yml beside
the version, and stops on a malformed record with schema: 1 or one that says
derived_after_overlay: false, or whose job is an absolute path or climbs out of the study. A
record without schema: 1, such as the one hvtiRtemplates' bd.qmd writes, is left where it is
and the version is recorded as legacy, so removing a retired release: block does not stop
update_manifest().
update_manifest() migrates a promoted entry (role: primary, no parquet:) whose source was
retired, by renaming its parquet to a dated version, and returns a new action, "missing", for a
dataset whose source and registered version are both gone. A promoted entry whose source is still
on disk is compared through source_sha256: an unchanged source registers the promoted parquet
itself, and a rebuilt one keeps it as the earlier version. Neither is reported as overwritten,
and a combined dataset is no longer recorded as rebuilt when only its format changed.
update_manifest() prints its reminder to commit manifest.yaml after a rewrite that only
re-recorded a source stamp, and its documentation says that a key violation on one dataset stops
the whole run and leaves every dataset as it was.
study_status() adds a dated_version:<dataset> row, PENDING, for each dataset with no dated
version, so a study no longer reads as all OK while every analysis job stops.
The message that says a source changed since registration names review_built(), which compares
the rebuild with the registered version, before update_manifest().
study_setup() no longer needs root. When it is omitted, the study that
encloses the working directory is used, as study_root() finds it, or the
working directory itself when no _study.yml lies above it; an empty
working directory is set up as a new study. Run from a subfolder of an
existing study, it therefore works on that study rather than creating one
nested inside it. When the root already holds a _study.yml for the same
Study Tracker ID, an interactive session is asked whether to adopt it
instead of stopping with "use adopt = TRUE"; a batch session still stops,
and a folder with other files but no _study.yml still needs
adopt = TRUE.
hvti_taxonomy() names the lg prefix "Logit calibration", not "Logit trends".
The description, "variable transformation and linearity checks", is unchanged:
a survey of 847 SAS lg jobs found 97% calibrate a covariate against an event
to choose its scale, and only a dozen model a calendar-year trend.
hvti_taxonomy() describes nd as a non-linear mixed model of a repeated
measure over temporal phases, and np as the figures of an nd fit (average
curve, bootstrap bands, observed means). The old text, "distribution estimates
stratified by group" and "non-linear distribution figures", matched none of the
243 nd and 234 np studies a survey of the corpus read.
study_setup() gains cleanup, renv and template_dir, so one call,
study_setup(..., adopt = TRUE, cleanup = TRUE, renv = TRUE), leaves a
legacy study ready to work in. cleanup = TRUE deletes the template copies
a study was given when it was created: every tp.* file, every
binder.summaries.doc, and every other file whose name and exact content
match the template corpus at template_dir (default
/programs/development/template). A study's own file that shares a name
with a corpus file but differs from it is kept, as are .renvignore and
.gitignore, and folders the deletions leave empty are removed. Folders
named .git, .checkpoint or renv are never entered, symbolic links are
never followed or deleted, and a root that overlaps template_dir is
refused. No backup is made; the corpus is the backup. renv = TRUE then
initializes and snapshots the study's renv environment in a separate R
process, loading the study's project explicitly; a complete, valid
environment is kept and one interrupted after initialization is resumed.
Both default to FALSE, so existing calls behave as before.
manifest.yaml that records a registered dated version now starts its
datasets: list with a line that makes hvtiRutilities 1.4.x, and 1.5.0,
stop with an error instead of reading it. 1.4.x read such an entry as a
promoted dataset, reconverted the rebuilt source and overwrote the entry, so
a study pinned to 1.4.x was served unregistered data and 1.5 then reported
the intact registered version as edited. A manifest written by 1.5.0 is read
as it is and gains the line the next time update_manifest() runs. Code
that reads manifest.yaml directly must skip the line: it is a character
string, not an entry.Datasets have a kind and a key. register_data() gains kind
("built", "subset", "ancillary" or "combined"), key (the columns
that make each row unique, checked at registration and again when
update_manifest() registers a rebuilt source) and parents (for a
combined dataset). study_config() validates all three and rejects a cycle
among parents. A combined dataset records the versions of the datasets it
was built from, when it is registered. When one of them is updated,
read_built() still reads it and signals hvtiRutilities_parent_changed,
whose message gives the commands to update it, and study_status() and
update_manifest() list it as out of date. A combined dataset with no
recorded parent versions is reported as out of date until it is rebuilt and
re-registered. update_manifest() updates every parent before the combined
datasets built from it. If the out-of-date check in
study_status() cannot run, the out_of_date:<dataset> row is FAIL
with the error.
"built" is now a second name for the study dataset. Every function that
takes a dataset accepts it, and everything recorded (manifests,
provenance, status) still says "study", so records made under either name
compare equal. "built" is reserved: a named dataset may not use it, and a
study that already registered an additional dataset called built is asked
to rename it.
job_census() and job_files() read hvtiRtemplates' template-first job
names, <prefix>[.<qualifier>].<subject>.<type>.qmd and their
.runner.R, as scaffolded jobs. Before, the SAS-legacy parser claimed any
dotted name and counted them as SAS-era jobs.
Existing dotted job names of this shape are now read as scaffolded, and no
longer carry legacy qualifiers.
Registered versions are checked and protected more strictly. These fix problems found in review of the dated-parquet registration that shipped in 1.5.0.
verify_manifest() re-derives each registered version's row count from
its parquet footer and fails a mismatch. Under strict = TRUE it fails a
version whose count could not be re-derived, as it already did for other
files. A malformed history: entry, or a version with no recorded schema
checksum under strict = TRUE, now fails rather than being skipped.register_data() and adopt_data_update() refuse a dataset whose cache
name would be a registered version, and the read cache never writes over
one. Before, a dataset named like cohort_20260915.csv could overwrite
the registered version of cohort.csv, and migrating it could delete
that version.verify_manifest() and study_status() give the pending source and each
earlier version its own label, e.g. built.csv (source) and
built.csv (built_20260915.parquet). A checkpoint taken while a rebuilt
source was pending wrote duplicate keys to CHECKPOINT.yml, which could
not then be read.extract_date when given, otherwise by
the source's local modification date.study_status() counts datasets rather than version files, and reports
a dataset whose source is gone as present when its registered version is
there, since that is what read_built() serves. update_manifest() on a
study with no manifest.yaml names register_data().update_manifest() with no file reported "found none" for any error reading
_study.yml, so a study that exists but needs fixing was told it had no
study. It now says that only when no _study.yml is found, and passes any
other error on unchanged.
register_data(catalog_dataset = , release_id = ) refuses a dataset
registered as dated versions, as update_manifest(file) already did. It
replaced the entry with a flat one, dropping every registered version from
the manifest and leaving their parquets unchecked.
A combined dataset registered on a parent that predates dated versions no
longer reads as out of date once update_manifest() converts that parent
without a change to its data. The parent's checksum, recorded at
registration, is recognised as the version it was converted to.
register_data(parents = ) and _study.yml accept "built" for the study
dataset, as every other argument naming a dataset does, and record it as
"study". A parent that is not a registered dataset is now named in the
error, which names register_data() rather than study_config() and comes
before the dataset is converted to parquet.
update_manifest(file) refuses a file that is another entry's registered
version, current or earlier, or its schema sidecar. Recording it replaced
nothing on disk but hid that its caller had just written over registered
data; every job then failed the checksum with no hint of why.
update_manifest() re-records the size and modification time of a
registered dataset's source that was rewritten with the same contents.
Until then every read, verify_manifest() and study_status() hashed the
source again to find it unchanged. On a file system that records whole
seconds only, such as a network share, a hash is still needed on each read,
as it is for the read cache.
registered_versions tests no longer print update_manifest()'s
closing reminder to commit manifest.yaml; it reached the test output five
times. expect_message() captures only the first matching message, so in
the five tests where a run registers or migrates a dataset the reminder that
follows the detail line escaped. Those calls are now wrapped in
suppressMessages(), and each still asserts its detail message.update_manifest() with no arguments registers a rebuilt one.
register_data() now writes <name>_YYYYMMDD.parquet and its column record
beside the source, and jobs read that parquet. The source file
(built.sas7bdat) may be rebuilt freely: read_built() keeps reading the
registered version and signals hvtiRutilities_source_changed, and
verify_manifest() reports the dataset as PENDING rather than failing.
Run update_manifest() from anywhere in the study to register the rebuild
as a new version; earlier versions are kept and still verified. A study
registered before this release is converted on its first
update_manifest(); if its source was already overwritten, the previous
version is recovered from the read cache where it still matches.
verify_manifest() now defaults to the study's manifest when run inside a
study, and a checksum mismatch names update_manifest(). Registration and
update need the arrow package. read_clinical_data() reads .parquet
files, so a parquet source registers like any other format.
provenance_data() records the dated parquet as the file a job read, so a
report names its own data version.cache_fit() gains packages =, a character vector of package names whose
versions join the cache key. The key records only the package that owns each
function called in code, so a wrapper is keyed on itself:
gg_partial_rfsrc(forest, ...) recorded ggRandomForests and not
randomForestSRC, which computes the result. After a randomForestSRC upgrade
that cache stayed valid, and refit = TRUE kept it, because refit only
recomputes a cache that is stale. With packages = "randomForestSRC" the
upgrade makes the cache stale. Without packages the key is unchanged, so
existing caches stay valid; adding it to a call makes that cache stale once.Articles on the pkgdown site put the table of contents on the left and use
the full width of the window, through pkgdown/extra.css. The installed
vignettes are unchanged: the Quarto vignette engine renders them in its own
minimal format, which has no sidebar layout.
DESCRIPTION now declares the Quarto command line tool in
SystemRequirements. The vignettes have always needed it to build; the
field makes that visible to installers and to R CMD check.
Now requires R 4.4.0 or newer, up from 4.1.0, to match the rest of the
HVTI family. hvtiR::install() installs the members together, and several
already required 4.4.0, so on an older R the install failed whatever this
package declared.
proc_means() now defaults to c("n", "mean", "std", "min", "p15", "median", "p85", "max") rather than SAS's own five. The group reports the
15th and 85th percentiles alongside the median in its descriptive summaries,
not the quartiles, so the default says what a report shows. It departs from
SAS PROC MEANS, which prints no percentiles unless asked: a parity
comparison must request the same statistics on both sides, either by adding
P15 MEDIAN P85 to the SAS statement or by passing the SAS job's list to
stats. A call that relied on the old default and compares column for
column needs stats = c("n", "mean", "std", "min", "max").
followup_check()'s means tables report p15 and p85 in place of
p25 and p75, for the same reason. A new stats argument chooses the
statistics, defaulting to c("n", "nmiss", "mean", "std", "min", "p15", "median", "p85", "max"), and is validated exactly as proc_means()
validates its own. Code reading $means$full$p25 or $p75 now gets NULL;
pass stats with "p25" and "p75" to keep them.
sha field holding the commit it was built from (RemoteSha); CRAN and
base packages are recorded as before. A version number alone can cover more
than one build: TemporalHazard changed what se.fit means on its survival
path inside version 1.2.11, so two jobs recording "TemporalHazard 1.2.11"
could hold different quantities with nothing to tell them apart.
publish_provenance() accepts entries with or without sha, so sidecars
written before this change still validate, and rejects a sha that is not
a hexadecimal commit.job_files(), and so job_census(), now reads the job names
hvtiRtemplates::add_job() writes, <subject>-<type>-<prefix>[-<qualifier>].qmd,
as naming = "scaffolded" with the qualifier carried. They came back with
prefix = NA before, so the census undercounted every scaffolded job and
would in time have reported a well-used prefix as unused. The -runner.R
that add_job() writes beside some reports shares its report's stem and
counts as a file of that job, not a second job (#170).
A checkpoint delivery stuck on "is not on main" now repairs itself
(#152). If a session died after replaying unpushed snapshots onto the
remote's main but before writing the log, the entry kept naming the
pre-replay commit, stayed pending on every retry, and held back every
later entry until .checkpoint/log.yml was edited by hand. Delivery now
finds the replayed commit, records the old commit as replayed_from, and
delivers as normal. It looks in three places: another log entry already
mapping the same old commit, the one commit on main carrying the entry's
id, or the one carrying the id in the old commit's own message. The last
covers a reopening, which tags an existing commit. When none finds exactly
one commit, the entry stays pending with the manual-repair warning, as
before. A hand repair should set replayed_from as well as git_commit,
so later entries can heal from it.
Two sessions can no longer both take over the same stale checkpoint lock.
A lock more than 6 hours old was taken over by deleting and recreating
.checkpoint/lock, so two sessions judging it stale at the same moment
could both recreate it and both believe they held it, letting their outbox
writes interleave. The takeover now renames the stale lock aside, which
only one session can do; the other stops with the usual "retry when it
has finished" error. A session that finds it renamed a lock another
session had just taken puts it back and stops with the same error. The
refresh a long call makes between phases had the same race and could
write its old holder back over a session that had just taken over. The
holder now refreshes a lease file named for its own token, in one step,
and a session whose lock was taken over stops with an error saying so
instead of carrying on. A lock whose setup or put-back fails part way no
longer strands an empty .checkpoint/lock that refused every session for
six hours (#154).
study_close(), study_reopen() and study_checkpoint() now see a
closure made from another copy of the study (#153). Before deciding whether
the study is closed, each one syncs with the remote: it delivers pending
entries, or fetches and fast-forwards when nothing is pending. Closure state
is the latest closed-* or reopened-* event on main, counting local and
remote tags once per commit, rather than a count of local tags. The count
read a study as closed after it was reopened once two copies had each
closed it. When the remote cannot be reached, a message says that closure
state is from this copy's last fetch and may be out of date, in either
direction. When a closure or reopening tag in this copy is not on its
main, which an interrupted delivery can leave behind, the state is
unknown: study_close() and study_reopen() stop and point to
study_checkpoint_push(), and study_checkpoint() warns and records.
study_status() never syncs. Its closure row says "as of the last fetch"
when a remote is configured, and shows UNKNOWN in that case.
verify_manifest(stop_on_error = FALSE), which reports failures as a
warning by design; those tests now assert it with expect_warning(). The
other two were the once-per-session convert_types and use_value_labels
deprecation notices, which fired in whichever test first omitted the
argument. A setup file now marks both as spent; the dedicated tests that
assert each notice still reset the flag and see it.The group default abbreviation list (inst/extdata/abbreviations.yml) now
holds a starter list of 39 entries, drawn from a census of the labels in 705
studies' built datasets. It is the group level of study_abbreviations():
a job that passes that function's result to label_map(abbreviations = ),
as the hvtiRtemplates EDA templates do, uses it on its next render.
label_map() itself still applies no list unless given one. A study
overrides or drops an entry in _study.yml. The entries:
LV, RV, LA, PA, RCA, IMA;AV, MV, TV, PV, and R for replacement and r for
repair (AVR, AVr, MVR, MVr, TVR, TVr, PVr). PVR is left
out: in cardiology it usually means pulmonary vascular resistance;CABG, PCI, CPB, IABP, CEA;CAD, CHD, HOCM, AF, CVA, TIA, COPD, DSWI,
NYHA;EF, LVIDd, LVIDs, RWT;FFP, RBC, LOS, Preop.Five entries also list a second spelling: "Coronary artery bypass grafting", "Cerebrovascular accident", "Red blood cells", and "Left ventricular internal diameter" in diastole and systole.
An abbreviation list entry may name several spellings of one term,
separated by |: Red blood cell | Red blood cells: RBC. The spellings
share the abbreviation, and a key prints the first. study_abbreviations()
returns one element per spelling, with an expansion attribute naming the
term, which label_map() uses for its key.
label_map() records which abbreviations each variable's label shows, as
an abbreviations_by_key attribute (key, abbreviation, expansion), so
a key printed under one section of a report lists only what that section's
labels use. A text search of the labels could not tell a label that says
"SP" in its own words from one the list shortened.
label_map() reads overlapping phrases left to right. Phrases were applied
longest first, so in "Right coronary artery bypass graft" the longer
"Coronary artery bypass graft" won the overlap and the label read
"Right CABG". The leftmost phrase now wins, and the longest one where two
start together: "RCA bypass graft".label_map() no longer abbreviates a shared heading to initials that the
abbreviation list already gives another phrase. "Aortic valve reoperation"
became "AVR", the list's aortic valve replacement, so one key listed AVR
twice with two meanings. Such a heading now falls through to the later
steps, and the supplied list keeps its meaning.study_abbreviations() still refuses two entries whose abbreviations differ
only in case, with one exception: the house style's R for replacement and
r for repair, so AVR and AVr coexist while Pv beside PV is refused.label_map() writes a word-like abbreviation such as Preop in the case of
the text it replaces: "Days from preoperative echocardiogram" becomes "Days
from preop echocardiogram", not "Days from Preop". Initialisms (LV, AVr,
LVIDd) keep the list's spelling.label_map() keeps labels that differ distinct. Cutting each label to
label_max on its own could make two labels identical once their
distinguishing ends were cut off; a figure of Surgical procedure: ...
panels became unreadable, and nothing warned. Within one map, labels that
differ in label_full now always differ in label. A heading shared by
labels over the cap is abbreviated in all of them (Surgical procedure
becomes SP); cut labels that still collide keep both ends
(Ascending aorta ... plus arch); and a label that collides whatever is
done is shown whole, flagged in a new over_cap column. A new
abbreviations argument supplies a study's own abbreviations, and the
abbreviations the labels show come back as an abbreviations attribute, a
key to print under the figure. add_labels() and apply_label_overrides()
rebuild the whole map, so an added label keeps the guarantee. Design:
dev/specs/2026-09-02-label-length-and-fallback-design.md section 4.2.
study_abbreviations() merges the three abbreviation lists a job's labels
draw on: a job's own entries (extra), the study's abbreviations: mapping
in _study.yml, and a group default shipped in
inst/extdata/abbreviations.yml. A higher level replaces a lower level's
entry for the same phrase, ignoring case, and null removes one. The result
carries a source attribute naming each entry's level, for a job's
provenance. Two phrases sharing an abbreviation are an error naming both and
their levels, and every bad entry is reported in one error.
add_abbreviation() writes one entry to a study's list, after checking it
against the merged list; like register_data(), it rewrites _study.yml
whole, so comments there are not kept. study_config() now validates the
abbreviations: block. The group default ships empty: its starter list
comes from a census of labels across studies. Design:
dev/specs/2026-09-25-study-abbreviations-design.md.
followup_check() checks an event indicator and its follow-up intervals
before a time-related analysis: cohort, event, censored and missing-event
counts; the missing, negative and zero intervals; a proc_means() table
for each subset; and the suspicious rows, with identifiers only on request.
Every missing column is named in one error. It is the dc-gfup template's
checks as a function, so that job and the EDA report compute them once.
study_setup() gains identity_source = c("tracker", "manual"). A new
_study.yml records identity_source and identity_verified, which is
false for an identity typed by hand while Study Tracker was
unavailable. study_status() reports such an identity as
"UNVERIFIED" on the _study.yml row, so study_checklist() leaves its
box unticked. A manifest written before these fields existed is a
Tracker identity and is still reported "OK". study_config() rejects
any other value of either field.
study_checkpoint() commits an allow-listed snapshot of a study (its code,
identity and reproducibility files) to a private repository in
.checkpoint/repo/, tags it with a StudyTracker checkpoint kind such as
manuscript_submitted-1, records it in the outbox .checkpoint/log.yml,
and pushes it when _study.yml names a checkpoint: remote:. Known data
formats, credentials and symbolic links are never committed, even through
include: patterns, and neither is anything under 00_datasets/ or
90_estimates/. Files in
50_documents/ other than .qmd and .bib sources are not committed
either: CHECKPOINT.yml records each one's size and checksum, so a tag
still names the exact document that was submitted. A checkpoint is
committed locally first, so an unreachable remote never loses one;
study_checkpoint_push() retries. A study whose identity is unverified is
committed but not pushed. Two sessions cannot interleave on one study:
each writing call takes .checkpoint/lock and refreshes it between
phases, and a lock left by a crashed session is taken over after six hours.
study_close() closes a study as published, not published, superseded or
abandoned, with a final snapshot tagged closed-<outcome>-<n>;
study_reopen() reopens it. study_status() reports checkpoints, pending
deliveries and closure.
job_files(), and so job_census(), skip any path with a .git or
.checkpoint component, so a checkpointed study's .checkpoint/repo
mirror is no longer counted as a second study.
print.study_status() visibility test now captures its printed report,
so the study_status test log no longer carries a stray status block..cache_heads() test quoted pkg::fn(x) as a fixture. R CMD check
read pkg as an undeclared test dependency and fetched the CRAN and
Bioconductor indexes to look it up, so a Bioconductor outage failed the
check with a WARNING. The fixture now uses digest::digest(x), a declared
import, and the check no longer reaches the network.Registration is endpoint-neutral. Jobs supply cohort definitions explicitly,
and status, provenance, and release adoption no longer require a study-wide
cohort. study_config() now identifies the malformed population field and
its named dataset directly.
Provenance capture now requires explicit data records. provenance_data()
and provenance_artifact() freeze the files a job actually reads;
capture_provenance() freezes those records and the executing R session;
and publish_provenance() binds that payload to an existing completed
output through an atomic sidecar replacement. record_provenance() remains
an existing-output convenience, but no longer accepts dataset= or infers
the currently registered data. Promoted role: primary datasets snapshot
their authoritative Parquet file, publication deeply validates transported
capture payloads, and failed or interrupted cache replacement attempts to
restore the prior cache and sidecar as one pair, warning with the path of
any backup that cannot be restored.
r_data_types() no longer corrupts a 2-distinct-value numeric column coded
something other than 0/1 -- a SAS-style 1/2 sex code, say -- into a single
logical value. as.logical() maps every nonzero code to TRUE, so both
categories silently became TRUE and one was lost outright, with no error
or warning. Only a column whose two values are actually 0 and 1 converts to
logical (or a 2-level factor when binary_factor = TRUE); any other
2-distinct-value coding now becomes a factor unconditionally, preserving
both categories. The 0/1 check is exact, not all.equal()-with-tolerance,
which read a value merely close to 0 (1e-9, say) as 0 and would still
have collapsed two distinct categories.
r_data_types() no longer turns NaN into a nonmissing "NaN" factor
level. is.na() already treats NaN as missing, but factor()'s own
exclude matching does not recognise it as equal to NA, so a raw NaN
survived as its own category instead of being dropped like the NA next
to it.
r_data_types() no longer merges two distinct numeric values into one
factor level when they happen to print identically (adjacent representable
doubles differing by a few ULPs, say). factor()'s default numeric
handling formats levels via as.character() and only then deduplicates;
real SAS-coded categories are small whole numbers this never touches, so
ordinary labels are kept for that case, with a higher-precision fallback
only where a collision is actually detected.
check_data_updates(), review_data_update() and adopt_data_update() add
the consumer side of immutable dataset releases. A release-aware
read_built() verifies the pinned bytes before cache access, reports a later
candidate without advancing the study, and stops on a changed or withdrawn
pin; allow_withdrawn = TRUE permits a deliberate historical revision run.
study_status() adds one update row per release-aware dataset. Adoption
replaces _study.yml and manifest.yaml as a recoverable pair and leaves
the old dated file in place. Legacy study contracts keep their existing read
and status behavior. A published file changed in place now stops before a
cache refresh can record the changed bytes, including when the producer
catalog is missing or malformed. Review, adoption, and registration
reconcile the release catalog with the study manifest, and same-day catalog
revisions must be contiguous. Promoted role: primary datasets verify their
authoritative Parquet while retaining the catalog's source checksum, and
release review re-verifies both files after reading them.hvti_taxonomy()'s note on the job catalog is brought up to date. The
catalog left hvtiR for hvtiRtemplates (template_catalog()), where it
has no retire disposition and no rows for rf or rfsrc, and
hvtiR::jobs() was removed in hvtiR 1.2.0. The demotion of rf and
rfsrc is now recorded only in this table, through the umbrella column.
hvti_taxonomy()'s documentation no longer describes the template ordinal
and the FOLDER_ORDINAL guard, both retired in hvtiRtemplates on
2026-09-03. It now says what does depend on this table: the folder names,
which every numbered study and template directory must use. Row and folder
order are free, because the directory digits are assigned rather than
derived from position.
study_dir()'s layout map with
unique(hvti_taxonomy()$folder). Drift in either direction used to pass
unnoticed and now fails the test suite. A folder added to the taxonomy and
not to the map is one study_dir() and hvtiRtemplates::add_job() refuse
as unknown. A folder added to the map and not to the taxonomy is the
reverse: study_dir() resolves it, but it names no taxonomy folder, so the
job census reports a job filed there as unplaced. The test compares names
only, because the directory digits are
assigned rather than derived from row order. hvti_taxonomy()'s
documentation, which said no such test existed, is updated to match.hvti_taxonomy() gains a logical umbrella column. It is TRUE for
rf and rfsrc, the legacy umbrella prefixes no template is owed for,
FALSE for every other prefix, and NA for the artifact row that has no
prefix. The demotion was recorded only in those rows' wording; a consumer
can now exempt them without hard-coding a list. ⚠️ This changes the
function's output shape: code that asserts its exact column set, or indexes
columns by position, needs updating. The template catalog in
hvtiRtemplates reads it (hvtiRtemplates:dev/specs/2026-09-18-template-catalog-design.md).hvti_taxonomy() gains rfr, sid and vt, and the random forest
family splits on the outcome axis. rfs, rfc and rfr name the
survival, classification and regression outcomes, matching the split
bh/bl/bc/bn/bq/br and pm/rm/cm already use; rfc and
rfs no longer describe themselves as "reporting", which tangled the
fit-versus-report axis into a family that splits on outcome. sid names
unsupervised sidClustering with PAM over K, and vt names virtual twins.
rf and rfsrc are demoted to legacy umbrella rows: they described the
same set as each other on the package axis, no template is written for
either, and they stay in the table so a census can still resolve the corpus
that uses them, which is 131 studies and 2,295 jobs for rfsrc alone. They
are deliberately not added to hvti_prefix_folds(), because a fold is
one-to-one and rfsrc spans all three outcomes. The rfc/rfs folder
contradiction is untouched and still owed its own spec. This unblocks the
rf* template batch in hvtiRtemplates, whose job catalog rows for the
three new prefixes were waiting on this change.
cache_fit() caches expensive computations strictly. A random forest,
varPro fit, imputation, hazard model or partial dependence grid runs once and
is reloaded on later renders only while its code, every outside value it
reads, its packages' versions and its seed are unchanged. Anything else stops
with an error of class hvtiRutilities_stale_cache that lists what changed;
refit = TRUE recomputes. Code is written inline, as a {} block, or passed
as a quoted call. seed = runs the computation under withr::with_seed();
random number use without a seed is cached with a warning and recorded as not
reproducible. Writes are atomic, and inside a study a provenance sidecar
carries the key. withr moves from Suggests to Imports.
The lint CI job could not fail, and now can. lintr::lint_package()
returns a lints object and prints it, which is not the same as failing on
it, so the step exited 0 while reporting lints and every lint.yaml run in
the repo's history concluded success. The package also had no .lintr, so
lintr fell back to its 80-character default and reported 290 lints against a
green check. lint.yaml now sets LINTR_ERROR_ON_LINT, which makes lintr
exit 31 on a non-empty result.
A .lintr records the line width as a decision rather than a default.
135 characters, because hvti_taxonomy() is a data table written as code
whose column alignment is the only thing making 45 of its rows readable, and
the widest line in the package is 132. Two linters are switched off with
their reasoning in the file: commented_code_linter, which reads
# WBC (K/uL) as a call and the expected-value arithmetic in the weighted
proc_means tests as commented-out code, and object_name_linter, which
cannot be satisfied for the exported hvtiRutilities.news() or for R's own
.Random.seed. There are no path exclusions.
67 of the remaining 69 lints were fixed rather than excluded, so the gate
is green on its merits. 48 were indentation, and the rest were small: braces
around two multi-line function bodies and an if/else with one bare
branch, spaces after commas and around named-argument =, one trailing
space, and testthat:: qualified in the two test helper closures that
object_usage_linter inspects. The only name change is the function-local
.MTIME_ROUNDTRIP_TOLERANCE_SECONDS, shortened to .MTIME_ROUNDTRIP_TOL_SECONDS
to fit the 30-character limit. No exported object, signature or behaviour
changed.
brace_linter gains allow_single_line = TRUE, which accounts for the
last two lints. test-cache_key.R tests that a standalone { a <- 1 }
nested in another block binds a for what follows, so the block is the
subject of the test; the linter's default requires an opening brace never to
start a line, which a standalone block cannot do in any layout. Measured:
2 lints single-line, 1 multi-line, 0 only with this option. Every other
brace rule stays in force and the test is unchanged.
hvti_taxonomy()'s note on the job catalog's disposition: retire is
brought up to date. It said retire could not mark the demoted rf and
rfsrc because rfs, rfc and rfr carried it too; those three moved
to scaffold in hvtiR's catalog, so retire now holds exactly the two
demoted rows. The note says they coincide by decision, not by
construction.
hvti_taxonomy()'s documentation no longer says the job catalog's
disposition: retire "says nothing about whether a template is owed", nor
that rfr is templated. hvtiR::jobs() defines retire as a function
that already exists with no template owed, and whether rfs, rfc and
rfr get templates is still open. The point that stands is narrower:
retire does not mark the demoted rf and rfsrc, because the three
outcome rows carry it too.
proc_univariate() ports the SAS PROC UNIVARIATE output statistics.
Weighted and decimal percentiles (pctlpts, pctlpre), vardef, mu0,
stdmean, and the tests for location (t, msign, signrank) and
normality (Shapiro-Wilk), checked against SAS 9.4 output. With weights
the quantiles are weighted, unlike proc_means(). Statistics SAS leaves
missing are NA. Normality above 2000 observations is NA with a warning,
because SAS switches to a Kolmogorov D test there, which is not ported.
Non-positive weights are an error. proc_means() now shares its statistic
engine with proc_univariate(); its results are unchanged.
proc_freq() ports SAS PROC FREQ frequency tables. One-way and n-way
tables with the MISSING and LIST options, cell, row and column
percentages, cumulative columns, and WEIGHT. Missing values sort first and
character levels by byte value, as in SAS. From three variables on, a
crosstab computes Percent within stratum and a list table out of the grand
total, so list changes the numbers, not only the layout. Excluded missing
rows are counted in attr(, "frequency_missing"). Tests of association are
not ported. Blank character values and NaN count as missing, SAS special
missing values stay distinct, and codes sharing a value label form one row.
study_setup() writes <study>.Rproj when the root has no R project, so
opening the project, here::here() and study_root() name the same
directory. An existing project is left alone.
proc_means() now counts missing-weight rows in nobs, as SAS does.
With weights, SAS PROC MEANS and PROC UNIVARIATE compute NOBS as
N + NMISS plus the observations excluded for a missing weight.
proc_means() dropped those rows before counting, so nobs was too low.
A class level whose every weight is missing now keeps its row, with n of
0, as SAS PROC MEANS prints it. Every other statistic is unchanged.
proc_means() weighted stderr now divides by the square root of the
sum of the weights, as SAS does. It divided by the square root of the
count, so every weighted stderr was wrong unless the weights summed to the
count. Confirmed against SAS 9.4 PROC MEANS and PROC UNIVARIATE.
proc_means() returns a single observation as its own mode. It
returned NA, as for any sample with no repeated value; SAS reports the
value.
proc_freq() no longer returns NA frequencies and percentages when
integer weights sum past .Machine$integer.max. Weights are summed as
doubles, so a weighted frequency_missing is double for integer weights
too.
Study setup now follows the study's real sequence. study_setup()
records Tracker identity and creates numbered folders before data exist;
register_data() later adds the default study dataset or a distinctly named
subset or ancillary file. Existing bare-folder studies can be adopted
without moving their files, and all data and provenance helpers accept a
named dataset. The unused study_init() entry point has been removed.
hvti_prefix_folds() records legacy prefixes that fold into another,
and job_files() applies it: a folded file is reported under the prefix it
folds into and classed known, while stem keeps the name as written. The
first fold is pm into lm. A review of the job catalog on 2026-09-11 found
that propensity work filed as pm belongs with the logistic models in lm,
so pm leaves hvti_taxonomy() and future censuses count its files as
lm.
A missing _study.yml now points to the explicit recovery command.
study_config() tells the user to run study-setup --recover, with the
Tracker-ID form when the repository cannot identify the study.
hvti_taxonomy() relabels five prefixes after a biostatistician's
review of the job catalog, 2026-09-11. bn, nd, nm and np are
non-linear, not non-parametric: "Bootstrap non-linear", "Non-linear
distributions", "Non-linear model" and "Non-linear plot", and the three
descriptions that said "nonparametric" now say "non-linear". nb is
"Boosting", not "Notebook", and its description reads "boosting models
(Boostmtree, BoostMLR)". Names and descriptions change; no prefix, folder
or order moves.
hvti_taxonomy() gains two imputation prefixes, si and mi, both in
the datasets folder. They close §7 of the imputation package spec, which
had been open since 2026-09-03. 44 prefixes now, across 45 rows -- the extra
row is estimates, an artifact kind carrying prefix = NA.
Two prefixes, not one, and the split is the point. 223 studies call
single mean imputation, 326 call multiple imputation, and 18 call both. One
prefix could not label those 18 unambiguously. mi is safe only because it
is paired: standing alone it reads as multiple imputation to a statistician,
which would misname the single-imputation job.
vars keeps "imputations" in its description, deliberately. A job that
imputes as one step among several is vars -- porting vars.sas found mean
imputation across 394 variables inside it. A job whose whole purpose is
imputation is si or mi. Removing the word would have made the vars
description less accurate, not more.
strip_level_prefix() and level_map() take the code off level text.
Categorical levels arrive as 1. Yes / 0 = No / 01 - home, and the
prefix is stripped by hand in every table, which is where the typos come
from.
strip_level_prefix() does it by rule. It is a display helper: it returns
new text and never rewrites the levels of the object it was given, so a
level named 01 - home keeps its ordering prefix in the data while a table
prints home.
A separator is required — one of . : = ) - must follow the
leading integer. That restraint is the feature. A rule general enough to
turn 0 No into No also turns 1 vessel disease, 2 vessel and
3 vessel into two identical strings; nothing errors, a downstream
table() or model merges them, and the level count silently stops matching
the data dictionary. A missed strip is visible in the output and fixable by
declaring the text in value_labels.yml; a wrong strip is silent and
corrupts the level set. So 0 No and 1 Yes are deliberately not
stripped.
Two further refusals, both of which preserve the level set:
1-2 vessels matches leading-integer-then-hyphen and would otherwise
become 2 vessels, colliding with a real 2 vessels level. Ranges are
ordinary in this domain.level_map() reports what stripping would do across a dataset: one row per
level with key, code, level, level_full and stripped. The shape
mirrors label_map() one level down, so subset(level_map(d), stripped)
answers "which levels will print differently" exactly as
subset(label_map(d), truncated) answers it for labels. Codes come from
value labels where a column carries them, from integer level positions for a
plain factor, and are NA for a character column. High-cardinality columns
are skipped and named in a warning rather than expanding a readable report
into thousands of rows.
This set was judged minor-worthy when it landed (2026-09-03): it adds two
exported functions, apply_value_labels() and type_conversion_report(), and
changes label_map()'s return to four columns. It shipped as a patch anyway,
and the 1.2.0 name went to the SAS descriptive-procedure ports four versions
later. Recorded here because the version number alone understates what this
release contains.
read_clinical_data() can read a SAS format catalog. A .sas7bdat
stores a format's name — YESNOF. — not its values; the code-to-text
mapping lives in a separate .sas7bcat. Without one, reading a SAS dataset
yields numeric codes and no value labels however the read is written, and
nothing downstream can recover text that never arrived. This package had
never read a format catalog.
The new catalog_file argument passes one through to haven::read_sas().
It defaults to NULL — what was read before it existed — so no existing
call changes. It follows ... and so must be named, which keeps it from
capturing an argument that previously reached r_data_types()
positionally.
Supplying a catalog with a non-SAS file is an error rather than a silent
no-op, and supplying one alongside convert_types = TRUE without
use_value_labels = TRUE warns: decoding a mapping and then discarding it
is the loss the argument exists to end.
r_data_types() can keep the level text of a SAS formatted variable.
haven reads a numeric-plus-format variable as a haven_labelled vector
carrying its own code-to-text mapping. r_data_types() threw it away: the
column is is.numeric() and is not a factor, so the factor branch saw only
the codes and Home, Rehab, SNF were lost. Downstream, every table
reconstructed them by hand from the variable label — which is why REDCap
labels ended up enumerating eight options.
The new use_value_labels argument converts through
labelled::to_factor() instead. It runs ahead of every inference rule, so
a declared type beats both the binary-to-logical branch and the
factor_size threshold.
It defaults to FALSE and warns once per session when it is not supplied,
the same shape as the convert_types warning. The default becomes TRUE
in a later release; until then no existing caller changes behaviour.
type_conversion_report() says which rule fired on which column. One
row per column: the rule, whether the levels came from value labels or from
counting distinct values, how many there were, and the class in and out. A
column that became a factor because it was declared one and a column that
became a factor because it happened to have seven distinct values were
previously the same object.
The report records the transition a call made, not the state it arrived at,
so it is deliberately not idempotent: a second pass over already-converted
data reports unchanged throughout, which is the true answer for it.
label_map() caps a label for display, and keeps the original. Labels
were capped at 40 characters historically. When the constraint lifted, long
descriptive labels became normal, and a table or an axis has nowhere to put
them — so they get trimmed by hand, every time, which is where the typos
come from.
The new label_max argument, default 40, cuts the printed label on a word
boundary and marks the cut with .... The marker counts against the
budget, so the result is never longer than label_max. Truncation is a
view, not a change to the data: the source text stays in a new label_full
column and a new truncated column says what was cut, which makes the
report subset(lmap, truncated) rather than another function. Pass Inf
or NA to disable it.
label_max must be at least 4, so a cut always has room to be marked, and
TRUE/FALSE are refused rather than coerced — as.numeric(TRUE) is 1,
which would have meant "cap at one character". NA still disables
truncation.
A side effect worth knowing: the "most columns lack descriptive labels"
warning is now driven by whether a label is absent rather than by
label == key, so a variable whose real label happens to equal its own
name no longer counts towards the 50% threshold.
dataset_schema() deliberately does not gain the argument, and errors if
you pass it. A schema sidecar outlives the source dataset, and a truncated
label recorded there is unrecoverable.
The variable-name fallback is exempt. An unlabelled variable still
reports its own name, whole, however long, with truncated as FALSE. A
truncated name would match nothing in the data and would read as a
deliberately short label rather than as a missing one — destroying exactly
the signal the fallback exists to give. The cap and the fallback are each
sensible alone; composed without the exemption they fail silently and the
failure looks like success.
apply_value_labels() declares the code-to-text mapping for coded
variables. There are no .sas7bcat format catalogs in the study corpus,
so catalog_file — added above — is an argument nobody can exercise yet,
and the mapping has to be declared rather than read. That is why REDCap
labels enumerate eight mutually exclusive options: the label was the only
field that survived the read.
A study writes value_labels.yml beside its config.yml and
labels_overrides.yml:
disp:
1: Home
2: Rehab
3: SNF
apply_value_labels() writes those into labelled::val_labels() — the
same slot a catalog would have filled — so r_data_types(use_value_labels = TRUE) converts them to a factor with text levels without a second code
path, and nothing downstream needs to know which input supplied them. It
declares; it does not convert. A study with no such file gets its data back
unchanged, so the call is safe to make unconditionally.
This is the home for enumerated level definitions, and it is deliberately not the display label. Two things it will not do quietly: value labels already on a column win over the file and the variable is named in a warning, so a mapping read from a source is never overwritten; and a variable named in the file but absent from the data is reported rather than skipped, because a typo that declares nothing is the failure a declaration file is least able to notice.
The ordered: key is reserved for the ordinal declaration and is refused
with an error rather than partly honoured. Ordinality is still blocked on a
statistical decision, and an ordered: true that silently did nothing
would read in the file as though ordinality had been declared.
label_map() returns four columns, and shorter labels, in this
release. label_max defaults to 40 immediately rather than defaulting
off for a release the way convert_types and use_value_labels did. So
the label column now carries the trimmed text, and label_full and
truncated sit beside it. Code that reads key and label by name is
unaffected; code that assumes two columns, or that expects the untrimmed
text in label, is not — read label_full, or pass label_max = Inf.
hvtiPlotR and hvtiRtables both consume label maps and are the callers
to check.r_data_types() no longer errors on a two-valued SAS formatted
variable. as.logical() has no vctrs cast from haven_labelled, and
the binary branch called it before anything had dropped the labels — so
read_clinical_data(convert_types = TRUE) aborted on any SAS export
carrying a formatted yes/no variable. The labels are now dropped explicitly
before the numeric branches when use_value_labels = FALSE, making that
path a conversion rather than a crash.
job_files() no longer discards the part of a legacy job name that says
what the job does. The legacy parser captured the first dot-field as the
prefix and threw the rest away, so dp.trends, dp.gfup and
dp.spaghetti.echo all reduced to one bucket called dp. A bucket cannot
be templated, and the census that fed the hvtiRtemplates roadmap ordered
its work by the size of buckets like that one.
Three columns now carry those fields through: qualifier1 (the first),
qualifiers (all of them, dot-joined) and n_qualifiers (how many). They
are NA, count zero, for the set, template and r_transitional
conventions, whose grammars account for every field and so have no
qualifier slot.
The fields are named by POSITION, not by meaning. A census over all
2,240,554 corpus rows on 2026-09-02 measured what the second field holds,
and it differs by taxonomy folder: an outcome in analyses, a table type
in descriptive, a clinical variable in distributions, and a mix of plot
types and variables in graphs. Naming the column outcome or
refinement would bake one folder's reading into all of them, which is the
error being undone. Assigning meaning is a separate step that dispatches on
folder; see hvtiRtemplates
dev/specs/2026-09-02-per-folder-naming-parse-design.md.
A two-field name such as hzdead.sas7bdat has NO qualifier. The extension
separator and the field separator are the same character, so the qualifiers
are taken by dropping the first and last fields by position rather than by
a second regex. 426 corpus rows turn on that distinction.
job_files() therefore promises 16 columns rather than 13.
hvti_taxonomy(): row order within a folder is no longer load-bearing,
and folder order now has a real check behind it. hvtiRtemplates v1.0.16
(3e14345, #57) retired the test that asserted within-folder ordinal order
matches this table's row order, so rows may be inserted or reordered freely.
An ordinal is assigned once in that package's ledger and never recomputed
from position.
It did not simply delete the guard. test-roadmap.R now parses the
hardcoded FOLDER_ORDINAL map out of check-roadmap-counts.py and compares
it against unique(hvti_taxonomy()$folder). So folder order stays
load-bearing, but a folder change here turns that package's CI red naming
the drift, rather than silently validating every ordinal's major against the
wrong folder. The warning in the roxygen is now backed by a check instead of
by discipline.
This supersedes the 1.1.7 note, which said the constraint still held. It did at the time: 1.1.7 shipped while the retiring PR was still open.
No API change, and nothing breaking. No export is added, removed or
renamed; no signature changes; NAMESPACE is identical to 1.1.6. Downstream
packages need no action.
One behavioural change, scoped to a single function: write_collision_report()
now writes an extra ## Provenance section, so that report's bytes differ from
one produced by 1.1.6. Nothing else's output moves. A rendered-output diff
elsewhere is noise, not signal.
hvti_taxonomy()'s roxygen now states the row-order constraint
precisely. It said the ordinal is derived from row position. That is no
longer the mechanism -- hvtiRtemplates v1.0.15 (e71bef4) froze ordinals
into a ledger, assigned once and never recomputed, because an ordinal is an
identity rather than a position.
The constraint itself still holds, for a different reason. That package's
tests/testthat/test-taxonomy.R still asserts that within-folder ordinal
order matches the row order here, and it is live and unskipped at v1.0.15. So
reordering rows turns its CI red even though no number is recomputed. Two
guards in that package now disagree; until the test is retired, row order is
fixed. The check compares only prefixes with a template on disk, so inserting
a row for an untemplated prefix is safe -- what is not safe is changing the
relative order of two templated prefixes in one folder. Folder order is
load-bearing independently.
Worth the caution because the coupling already failed once, in this package's
direction: the 1.1.6 hs move shifted bh from sixth to fifth in analyses
while its shipped filename stayed 04.06, and nothing caught it -- the
ledger checks verify format, folder-major and uniqueness, never position.
bh was renumbered to 04.05 and 04.06 retired rather than freed, because
it had shipped in v1.0.13 and v1.0.14.
write_collision_report() now ends with a Provenance section derived from
the scanned corpus rather than from the clock: a fingerprint over every
(macro, file, body hash) triple and the excluded-directory list, the file,
definition and name counts, and the package version.
Neither the report nor write_macro_manifest() carries a generated-on
field, deliberately -- it would defeat the byte-for-byte reproducibility
that makes the artifact reviewable. But an artifact with no identity is
the opposite failure, and it has already cost something: the committed
collision_report.md and the canonicalization design disagree on skip
(10 files / 7 bodies against 13 / 10) and on four other rows, and nothing in
either file says which measurement is current. An input-derived stamp avoids
both failures, because it is a function of the inputs alone. Determinism is
over the corpus and the package version -- the version row is expected to
change between builds, since that is what attributes an artifact to one --
while the fingerprint depends on the corpus alone and stays comparable
across versions.
The fingerprint follows content, not location. sas_triage() records file
basenames, so re-scanning the same corpus from a different mount point
yields the same value -- a share remount must not invalidate every committed
report.
This is prospective. The committed artifact predates the stamp and gains one only when the corpus is next swept, which needs share access.
write_collision_report()'s help page was missing a clause. The roxygen
wrote %include unescaped, and % opens a comment in Rd, so everything
after it on that line was dropped: the rendered Description read "In SAS, the same macro means the second silently shadows the first".R CMD checkcannot catch this -- the Rd is valid, the text is simply commented out. Escaped, and the three markdown backticks in the same block converted to\code{}` per this package's Rd-not-markdown rule.hs moved from analyses to graphs, positioned immediately after
hp. The move follows the corpus rather than the name: all ten tp.hs.*
templates in the SAS library, and ten of the eleven R hs jobs across
/studies, sit in graphs/. hs computes patient-level predictions that
the plotting jobs beside it consume — the setup / uses_setup pairing the
corpus uses throughout.
⚠️ Row order within a folder is load-bearing. hvtiRtemplates derives a
template's ordinal from this table — folder position gives the major, row
position within the folder gives the minor — and asserts both. Placing hs
after hp is what makes its template graphs/06.02-hs.qmd; anywhere else in
the block gives a different ordinal. The insertion shifts the implied minor
of mp, lp, np, dp, fp, gp, cp, ce and rp by one, which costs
nothing today because hp is the only templated prefix in graphs and it
keeps 06.01. It will not stay free: once any of those nine ships, the same
insertion would renumber a live template.
Requested by hvtiRtemplates for its hs template, designed in
hvtiRtemplates:dev/specs/2026-08-29-hs-template-design.md.
dev/specs/, adopting the portfolio
convention settled in ehrlinger/house-style. Designs and their plans now
share a directory and a slug, with -design / -plan carrying the
distinction specs/ and specs/plans/ used to encode a second time. The
six paths quoted in NEWS.md, R/, man/ and the tests were repointed,
so nothing here links at a file that no longer exists.ROADMAP.md moved to the repository root. It says where the package is
going rather than what was decided on a date, so it is not a development
record. .Rbuildignore gained a matching line to keep the tarball clean.Candidate-pool preparation helpers, ported from a study's local R/ (#47).
They were written for one study and are needed by the hm and bh job
templates in hvtiRtemplates, which cannot depend on one study's private
helpers.
sas_variable_block() reads a variable list out of a .sas job rather
than transcribing it — a transcribed list drifts from the job it claims to
reproduce and nothing catches it. Comment handling is the whole difficulty,
and it is silent in both directions: a banner comment
(/***** Patient Variables *****/) contains *, so line-by-line stripping
leaves it intact and it swallows the first name on the next line
(female, afib_pr, plvidd and size were lost this way), while a
/* ... */ spanning two lines is never stripped and a commented-out
variable is read as live (avet_con entered a screen that way). Both are
regression-tested.
covariate_audit(), imputed_levels() and
covariates_to_numeric() report and apply what happens to a covariate on
its way into a model. %vars(missing=1, impute=1) mean-imputes and adds a
paired ms_* indicator, so a 0/1 clinical variable arrives with three
values — 0, 1, and the cohort mean (one real column's third value is 0.714,
the prevalence of hypertension, not any patient's status). In SAS those enter
linearly; read into R they arrive as factors and get dummy-coded, which
is a different model with a different parameter count and nothing in the
output says so.
NA in noninteger_levels means "not knowable here", not "none": mean
imputation is detectable in a discrete column but invisible by construction
in a continuous one.
concept_of(), concept_map(), prune_to_one_form() and
selection_crowding() group a candidate pool into clinical concepts. A
SAS pool offers every transformation as a separate candidate, so a stepwise
screen spends its steps on them and a per-variable frequency reports each
separately — five forms of age each cleared 80% on one screen while the
retained set collapsed to four concepts, and the paper reports concepts.
The rule is deliberately conservative, because grouping too little only costs some pruning while grouping too much silently merges two clinical concepts. Stem truncation is handled automatically but only when unambiguous; contractions need an explicit alias.
pool_collinear_pairs() catches what concept grouping cannot: candidates
that are the same information under unrelated names. male and female are
exact complements, both were offered to a screen, and 101 of 500 replicates
selected both — fitting a rank-deficient design with nothing in the
output saying so.
POOL_AFFIXES, POOL_PLAIN_SUFFIX and POOL_MIN_STEM carry the
legacy naming conventions as documented defaults, overridable per study
rather than edited in place.
The affix and alias machinery is a permanent compatibility layer, not a temporary shim: stem truncation exists because SAS capped names at 8 characters, and reproducing a legacy SAS job in R is routine work rather than a migration with an end date.
concept_frequencies() is deliberately not ported yet — it takes a bagging
object whose structure does not exist in this package. It belongs with the bh
work.
job_files() no longer aborts a sweep when a filename is invalid in the
session's encoding. A shared corpus does not guarantee valid names -- one
Latin-1 byte in a name created on Windows was enough to kill the sweep of a
841,042-file directory with input string 841042 is invalid. Invalid bytes
are now replaced with their <xx> escape, so the file appears in the output
with a legible, greppable name instead of taking the directory down with it.
Two of the three string operations involved failed silently rather than
loudly, which is the worse half of this bug: grepl() warns and returns
FALSE, leaving the prefix unclassified, and strsplit() warns and returns
NA, marking the file unplaced. Only substring() raised an error. Had the
error not fired first, the file would have been miscounted rather than
reported.
job_files() now credits a file under a symlinked subdirectory to the
study it was walked into. It normalised every path before stripping the
root prefix, and normalizePath() resolves symlinks -- so when a
subdirectory inside the corpus pointed somewhere outside the root, the
resolved path no longer carried that prefix, the strip sliced at the wrong
offset, and study came back mangled with no error raised.job_files() makes one filesystem pass per file instead of three. The
per-file normalizePath() is unnecessary -- list.files() prefixes its
results with the root exactly as given, and the root is already normalised
-- and the per-file dir.exists() was redundant, since
list.files(recursive = TRUE) never returns directories. Each removed pass
was a stat syscall; on a local disk that is invisible, but the corpus is
read over an SMB mount where every stat is a network round-trip and the two
passes dominated the walk. A partial corpus sweep over the share reached one
top-level directory in an hour before this change.job_files() and job_census() — a filename-only inventory of the job
corpus. job_files() returns one row per file with its study, taxonomy
folder, prefix and naming convention; job_census() rolls that up to
(study, prefix, folder, is_template) with n_jobs (distinct stems) and
n_files.
The print method leads with distinct-studies-per-prefix, which is the
lookup that says whether a job type can be templated yet.
Nothing is filtered: placement and classification are columns, so a file
the sweep cannot classify stays in the output rather than vanishing. There
is no extension allowlist, deliberately — see
dev/specs/2026-08-26-job-type-inventory-design.md §4.4.
hvti_taxonomy() and hvti_non_prefixes() — the analysis-prefix table,
moved here from hvtiRtemplates, which now imports them back. The table is
shared vocabulary rather than template machinery, and this package is the
lower layer.
read_clinical_data()'s convert_types argument now defaults to FALSE.
It defaulted to TRUE, applying r_data_types() to every column, which
converts any two-valued numeric column to logical — including 0/1 event and
censoring flags, which hzr_kaplan() and similar then reject. Reading and
type derivation are now separate steps. Callers who omit the argument get a
once-per-session warning; pass convert_types = TRUE to restore the old
behaviour.dataset_schema() — one row per column giving creation position, name, R
class, SAS type, format.sas and label. Labels and formats are read from
the column attributes directly, so an absent label is NA rather than the
variable's own name. This is the durable description of a source dataset:
it describes shape only, so two reads of an unchanged file produce an
identical schema.update_manifest() gains n_cols, schema_sha256 and role. n_cols is
recorded as part of the schema baseline, since a row count alone cannot
detect a dropped column; it is verify_manifest()'s comparison of
schema_sha256 against the sidecar on disk -- a hash of the schema
sidecar, making the manifest-to-sidecar link tamper-evident -- that
actually detects one. role is "source" or "primary" and
distinguishes a dataset SAS still builds from one whose
parquet has become authoritative.verify_manifest() checks schema_sha256 against the sidecar on disk, and
reports two entries whose file stems collide and would therefore claim the
same derived .parquet and .schema.csv paths.read_built() now caches its source as parquet on first read and uses that
cache while the source's size and modification time are unchanged. The
schema sidecar and manifest entry are written from the haven read, before
the parquet exists, so the recorded baseline is independent of the
conversion. Conversion is lazy: a dataset nobody reads is never converted.
arrow is a suggested package — without it, or with
options(hvtiRutilities.disable_parquet_cache = TRUE), reads behave exactly
as before.role
rather than a single size/mtime heuristic. A role: "primary" entry is
served from its parquet unconditionally — the source may have been retired
and need not exist. A role: "source" entry is valid when the source's
size and mtime both match the recorded stamp; size alone cannot be
trusted, since a .sas7bdat is page-aligned and a changed row count can
leave its size identical. Whether a matching mtime alone proves the
source is unchanged is a property of the filesystem, and it is measured
from the stat rather than assumed: a fractional mtime means sub-second
resolution and is trusted directly; a whole-second mtime means the
filesystem cannot see inside that second, and the recorded sha256 is
verified instead. Production runs on a local filesystem with nanosecond
mtime, so the fast path applies there; development over SMB may see
whole-second stamps and pay for a hash — neither is a special case in the
code. The source is stat'd before the read rather than after, so a source
rewritten mid-read is stamped conservatively stale instead of validating
forever, and if the source's size or mtime changes again between that
stat and the end of the read, the frame is discarded and the read errors
rather than caching a possibly torn result. (An earlier draft of this
validity check compared the source's mtime against a client-side
Sys.time() recorded at conversion; that compared two different clocks
whenever the files live on another host, so a client running even one
second fast made every entry look unambiguous forever. It never shipped in
a release.)read_built() gains refresh = TRUE, which forces a re-read from the
source and a reconversion regardless of role or the cached size/mtime
stamp — for changes a timestamp can't express, such as a rebuild that
preserves mtime or a correction applied out of band. refresh = TRUE
against a role: "primary" entry errors clearly, naming the role, rather
than failing inside haven, even with arrow absent or the cache disabled.
read_built() also now serves a role: "primary" entry whose source file
is missing, instead of erroring before the manifest is even consulted —
without this, promotion (retiring the source once its parquet is
authoritative) was unreachable. If a promoted entry's parquet is also
missing, the error now names the parquet as the missing copy rather than
the source, which was retired on purpose.verify_manifest() now hashes whichever file a manifest entry's role
makes authoritative — the source for role: "source", the parquet
(resolved the same way the cache names it) for role: "primary" — rather
than always hashing the entry's own file. file never changes on
promotion, so this cannot be inferred from the name. A missing source is
therefore no longer reported as a failure once an entry is promoted; it is
expected, since promotion means the source was retired.role: "primary") entry — its parquet lost
while the retired source happens to still be present — now reconverts the
parquet without rewriting the entry as an ordinary miss would: sha256 is
updated to describe the new parquet, and promoted_date and
source_sha256 are left untouched, rather than being silently dropped and
replaced with the source's own hash.role: "primary" entry, which has no second
chance to self-heal once the source is retired.update_manifest() gains reader, recording the package and version that
produced a derived file (e.g. "haven 2.5.5"). The parquet cache supplies
it automatically. Under role: "primary" this is part of the parquet's
provenance: a reader defect found later is otherwise unfindable once the
source is gone.read_built() now errors when lowercasing column names produces duplicates,
naming the colliding pair, instead of returning two identically named
columns where every downstream selection silently takes the first. SAS names
cannot collide this way; .csv, .xlsx and .rds sources can.verify_manifest()'s default data_dir now resolves each entry
individually instead of choosing datasets/ vs. the manifest's own
directory once for the whole manifest based on whether datasets/ exists.
Previously, a flat-layout study whose manifest referenced files beside
manifest.yaml would have every entry misdirected into datasets/ (and
fail with "File not found") merely because that directory happened to also
exist, empty or not. Each entry now prefers datasets/<file> only when that
file is actually there, and otherwise resolves beside the manifest. Callers
who pass data_dir explicitly are unaffected — an explicit data_dir is
still used exactly as given, with no nested search.read_built() for
that dataset. A role: "source" entry warns, naming the file, and falls
back to regenerating it from the source; a role: "primary" entry still
errors, naming it the only copy, since there is no source to regenerate
from.source_size recorded without source_mtime, from a hand edit or
an interrupted write. Any missing stamp field now reads as "not valid"
rather than reaching as.POSIXct(NULL) and throwing.update_manifest() and the internal stamp update), are now
atomic — a temporary file, then a rename — matching
.write_parquet_atomic()'s existing discipline, which is now shared by all
three through one helper. Two first reads racing on the same dataset could
previously leave one process hashing a sidecar the other was midway
through truncating, recording a schema_sha256 that matched nothing on
disk and putting the entry into permanent verification failure.utils is added to DESCRIPTION's Imports:. R/parquet_cache.R calls
utils::write.csv() and utils::packageVersion(), and utils was already
imported in NAMESPACE, but it was missing from DESCRIPTION — which
R CMD check flags.update_manifest() rebuilds a manifest entry from its own known fields
rather than merging into whatever is already there, so any field a study's
own tooling added to an entry is silently dropped the first time a cache
miss (or any other call to update_manifest()) rewrites it. The parquet
cache preserves extract_date, source and sort_key specifically,
because losing those would clobber values study_init() sets explicitly;
no other field is preserved. This is unchanged in this release — recorded
here so it's a known limitation rather than a surprise.proc_contents() now documents that its label column is never NA: it
falls back to the variable's own name via
labelled::var_label(null_action = "fill"), so an unlabelled variable is
indistinguishable from one labelled with its own name. Callers recording the
output as a durable description of a source dataset should read labels from
the source attributes directly. Measured on an 879-variable clinical build,
14 variables carry no label and would be recorded as labelled with their
names.proc_contents() also clarifies that name, label, format.sas and creation
order survive a haven read when the source carries them, rather than being
present on every variable. In the same build, 865 of 879 variables carry a
label and 395 carry a format.sas, so an NA format reports an unformatted
source variable rather than a lost attribute..Rbuildignore now excludes .remember, the session-tooling directory. It
was landing in the build and raising a "hidden files and directories" NOTE on
every branch, which masks real NOTEs during release gating.parity_tolerance(), compare_parity() and parity_headline() — the
comparison half of the SAS parity harness. compare_parity() errors when a
quantity is absent on either side, and reports PASS / DIFFERS /
R_BETTER.preflight_report() — environment audit naming every package a hazard-family
analysis depends on, including numDeriv.study_init() initializes a study for reproducible analysis: it writes the
_study.yml identity manifest that study_config() reads, and seeds a
manifest.yaml pinning the built dataset's SHA-256. The cohort counts are
derived from the dataset rather than supplied, because a hand-typed count is
a count that can disagree with the data. citation is written as an explicit
null, so a study that is later published has an obvious place to record its
reference.
study_status() audits a study without writing anything, reporting whether
it has a valid _study.yml, an renv.lock, a manifest.yaml whose
checksums still match, and a provenance sidecar for every .qmd or .Rmd
source. It never errors on an absent or malformed manifest — that is the
finding, not a failure — and it distinguishes a check that could not run
(MISSING) from one that ran and failed (FAIL).
study_checklist() renders a study_status() result as a markdown
checklist, ticked where the study already complies.
verify_manifest() gains strict = FALSE. Under strict = TRUE an entry
whose row count could not be re-derived is reported as "FAIL" rather than
passing on its checksum alone, and the message names which of the three
causes applies: the count is not recorded in the manifest, the file type
cannot be counted at all, or heavy row counting is left disabled. Use it
where the question is "did every check actually run" rather than "is the
data intact": a release gate, an archival gate, or a hand-off where a
downstream reader will read "OK" as fully verified.
The default is unchanged and stays permissive, because a check that cannot
run is not a check that failed. Under it "OK" can still mean "checksum
verified, row count not examined", and row_count_checked remains the
column that distinguishes the two.
verify_manifest() no longer reports FAIL for a .sas7bdat or Excel
entry whose SHA-256 matches. Re-deriving the row count of those formats
needs options(manifest.allow_heavy_rowcount = TRUE), and without it the
attempt errored and the error was reported as a failed entry. At the default
stop_on_error = TRUE that halted analyses whose data was intact. The count
is now skipped when it cannot be re-derived and the entry passes on its
checksum; a file that cannot be read at all is still FAIL.
verify_manifest() gains a row_count_checked column, TRUE when the row
count was re-derived from the file and compared with the manifest. The
entry's message notes a count that was not re-derived.
verify_manifest() no longer reports SHA-256 match (n = ) for an entry
whose manifest records no row count. An empty count field reads as a
verified count of nothing; the message now says no row count recorded.
Entries written by hand, or by a version that could not count rows, are the
ones affected.
verify_manifest() no longer reports an entry that records no SHA-256 as a
SHA-256 mismatch against a blank expected value. Such an entry still
fails, because the checksum is the whole of what the function verifies, but
the message now says none was recorded and names the algorithm the manifest
used instead. A manifest written by an md5-based writer is the case this
turns up on.
study_init() does not run renv::init(). A missing renv.lock is
reported as an open item instead: creating one restarts the R session and
rewrites .Rprofile, which a function that writes two YAML files has no
business doing.study_status() reports a manifest.yaml entry as verified when its
SHA-256 matches but its row count could not be re-derived, and says how many
counts were skipped. For a .sas7bdat that is the normal case.study_config() reads a study's _study.yml manifest, found by walking up
from a starting directory. It validates that every required key is present
and errors otherwise, naming the key: a study without a complete manifest
must not render, and a partial default would be a silent wrong answer.
record_provenance() writes a JSON sidecar beside a rendered output,
recording the study manifest checksum, the R version and platform, every
loaded package and its version, the renv.lock checksum, the input
dataset's SHA-256, and the cohort. renv.lock alone cannot say what
produced a particular result — it is project-scoped and re-snapshotted
through a study's life — so the record is job-scoped and lives with the
result. Failure to write it is an error, not a warning.
The study data contract moves in from the per-study R/ directories:
study_root(), sas_path(), built_path(), built_manifest(),
read_built(), cohort_counts() and assert_cohort(). All of them read
study-specific values from _study.yml rather than from constants, so no
study path, title, dataset name or cohort count appears in package code.
r_dir_impurities() reports top-level executable code in a directory that
is sourced wholesale, where a stray call would run on every render.
jsonlite, for the provenance sidecar.built_manifest() records sha256 rather than the md5 used by the
earlier per-study version, matching the provenance record. One hash
algorithm across the design, not two.sas_triage(): the excluded-subdirectory counts no longer include
dot-directories. A local clone carries .git, whose loose-object files carry
no extension and so matched none of the non-source suffixes, reporting 31
files of "excluded SAS source" that were git internals. These counts exist to
tell a human what triage did not look at, and .git is never that decision.
CVS/ is still reported: it is a legacy artifact committed into the corpus
itself, not infrastructure of the clone.proc_means() gains nine statistics from the SAS unistats vocabulary:
nobs, var, uss, css, skewness, kurtosis, sumwgt, qrange and
mode. skewness and kurtosis are the adjusted Fisher-Pearson forms SAS
uses, cross-validated in the test suite against e1071 with type = 2.
mode returns the smallest of tied modes and NA when no value repeats,
matching SAS.
proc_means() gains a weights argument naming a numeric column, mirroring
the SAS WEIGHT statement. Weights apply within each class level.
Weighting is not uniform, following PROC MEANS: mean, std, var, cv,
stderr, sum, uss, css, skewness, kurtosis and sumwgt respond to
weights; n, nmiss, nobs, min, max, range, mode and every
quantile do not. PROC MEANS computes no weighted quantiles; that is
PROC UNIVARIATE.
A zero or negative weight raises an error naming the offending rows. SAS's own handling of non-positive weights differs across procedures and versions, so this fails loudly rather than encode a guess.
The PROC UNIVARIATE inference statistics remain out of scope. See
dev/specs/2026-08-14-proc-means-unistats-design.md.
proc_means(): cv now returns NA when the mean is zero, matching SAS.
R's arithmetic gives Inf, which asserts an infinite coefficient of
variation where SAS reports the value as undefined.proc_means() statistics are dispatched through a .STATS registry that
declares, per statistic, whether it responds to weights and whether it is
integer-typed. The weighted set is asserted directly in the test suite.Phase 0 of the SAS macro canonicalization program: sas_triage(),
sas_macro_defs(), sas_macro_signature(), write_macro_manifest() and
write_collision_report(). Together they inventory a legacy SAS macro
library, classify each definition public or private, and report the name
collisions that make %include order-dependent. Pure R -- no SAS dependency,
so the inventory runs on the development workstation.
Two design points are worth recording, because the obvious implementations of both are wrong for this corpus.
File discovery excludes known non-source suffixes rather than matching
.sas. The library uses dots as word separators in filenames --
deciles.hazard, lm.cprobs and kaplan.int are names, not stems with
extensions -- and many macro files carry no extension at all. An
extension-anchored pattern omits both populations silently, including
unistats, whose statistic vocabulary is the closest analogue in the corpus
to proc_means().
Validity checking uses a stateful scanner, not pattern matching. It tracks
comment state (/* */, * ... ;), string state ('...', "...") and macro
quoting (%'), so comments and literals spanning lines, apostrophes inside
double-quoted strings, and the %STR(%') transpose idiom are not mistaken for
syntax errors. Every rule reads the scanned source, so a %macro written
inside a comment or a string literal is not counted as a definition. The check
reports a file that ends inside a string literal or inside a comment,
naming the line where it opened -- an unclosed /* is the more destructive of
the two, because every statement after it is inert while the file still looks
like usable source. There is deliberately no do/end balance check: textual
balance is not a validity property of macro source, since %do-guarded blocks
emit DO and END from separate branches.
A macro redefined inside one file is resolved, not escalated. SAS
compiles definitions in order and the later one replaces the earlier, so the
last definition in a file is what any %include of it actually gets. Rule 3
keeps that one and drops the rest, each carrying the line that supersedes it.
This is deliberately not a name collision: write_collision_report() lists
names found in more than one file, because those are the ones whose
behaviour depends on %include order. On the reference corpus 158 of the 816
definitions are shadowed this way, leaving 658 live.
On the reference corpus of 336 files this inventories 816 macro definitions
across 307 distinct names, rejecting 5 as genuinely defective:
bl_ord.norm.ci.sas, CR_compare_CP_test_AT.sas, rem.original, rem.uab
and repeated.sas.
Those figures replace an earlier measurement of 866 definitions across 250
names. The difference is not a change of policy but a correction: the earlier
run counted %macro statements that the scanner had already established were
comment or string content. The rejected set is the same size and different in
membership -- xmacro.sas leaves it, having been failed for an imbalance
whose second definition is commented out, and bl_ord.norm.ci.sas enters it,
whose * NOT COMPLETE header carries no semicolon and therefore swallows the
%MACRO BLORD statement on the line below. The file says as much itself.
New vignette, PROC CONTENTS and PROC MEANS in R, written for readers who
run both languages. It walks proc_contents() and proc_means() beside the
procedures they replace, covers the QNTLDEF=5 quantile difference that
changes quartiles without changing the median, and lists every behaviour
where the R version departs from the SAS original.
compare_datasets() is now demonstrated in Dataset Version Tracking
rather than only named in its index table. The example covers the four kinds
of drift it reports and why a label change is the one to watch.
proc_contents(): a port of SAS PROC CONTENTS. Returns a dataset header
(observations, variables, label) and a variables table carrying creation
position, name, SAS type, format, label, R class, distinct-value count, and
percent missing. Len, Pos, and Informat are deliberately omitted —
haven cannot recover them from a .sas7bdat, and inferred values would
disagree with the source dataset whenever its LENGTH statement differed
from the default.
proc_means(): a port of SAS PROC MEANS. Takes SAS statistic keywords
(n, nmiss, mean, std, min, max, sum, range, stderr, cv,
median, q1, q3, and any pNN), defaulting to SAS's own five, and
supports CLASS stratification with SAS's default handling of missing class
levels. Quantiles use type = 2, the R equivalent of SAS QNTLDEF=5;
R's default type = 7 disagrees with SAS on small and even-numbered samples.
proc_contents() now documents that pct_missing is NaN, not a value in
0-100, when the input has columns but zero rows: the proportion of missing
values among no values is undefined, and reporting 0 would assert that
nothing is missing. Behaviour is unchanged and is inherited from
data_dictionary(); only the documented contract is now accurate, and a test
pins it.data_dictionary() is now a thin wrapper over proc_contents() and the
shared statistic engine. Its signature and output are unchanged, pinned by
characterization tests added before the refactor.update_manifest() and verify_manifest() are now silent by default.
Both gained a verbose argument (default FALSE) that gates all
informational message() output: "Manifest entry added / updated" in
update_manifest(), and the per-entry "— SHA-256 match (n = N)" and
"Manifest contains no dataset entries." lines in verify_manifest().
Pass verbose = TRUE to restore the previous console output.
This removes unconditional chatter from packages that call these functions
in a loop (for example hvtiRdatasets::snapshot_oracle(), which called
update_manifest() once per study), and brings the package in line with
the CRAN policy against chatty message()/cat() in function bodies.
Failures are unaffected: a SHA-256 mismatch, a missing file, or a row-count
mismatch still raises stop() (or warning() when
stop_on_error = FALSE) regardless of verbose. The per-entry status text
is also still returned in the message column of verify_manifest()'s
data frame, so silencing the console loses no information.
verbose is appended last in both signatures, so existing positional calls
are unaffected.
[email protected]. The redundant
Maintainer: field was removed from DESCRIPTION — with Authors@R present
the maintainer is derived from the cre role, and having both declared
different addresses.https:// (the http:// form
301-redirected, failing urlchecker::url_check()).Date: refreshed to the 1.0.1 release date.read_clinical_data(): files with no extension now produce a clear error
("Cannot determine file type: … has no extension") instead of the
misleading Unsupported file type: '..' message. The unsupported-extension
error also now includes the full file path for easier diagnosis.test-read_clinical_data.R: strengthened the tibble-coercion assertion from
expect_true(is.data.frame(result)) (TRUE for tibbles) to
expect_equal(class(result), "data.frame") so the test actually protects
against as.data.frame() being removed or bypassed.vignettes/reproducible-seeds.qmd: section heading "Using the Seed with
varpro" and narrative references to the package now use the correct CRAN
casing varPro; function-name references (varpro(), unsupv.varpro())
remain lowercase as those are the exported function names.Suggests entry for varPro: package name on CRAN is varPro
(camelCase), not varpro (lowercase). The case mismatch caused pak
lockfile resolution to fail with "Can't find package called varpro".
Updated the corresponding varPro::varpro() call in
vignettes/reproducible-seeds.qmd to match..Rmd) to Quarto (.qmd). Added
quarto to Suggests.read_clinical_data(): CSV files are now read with check.names = FALSE so
column names containing spaces, hyphens, or special characters are preserved
exactly as written, preventing silent name mangling that could break
downstream label lookups.generate_survival_data(): generates a synthetic cardiac surgery
survival cohort with 22 clinical variables, Weibull-distributed survival
times, reoperation outcome, and variable labelsr_data_types() silently corrupting Date, POSIXct, and POSIXlt
columns that had exactly 2 unique values (they were converted to logical)r_data_types() incorrectly converting constant columns (1 unique
value) to logical; binary detection now requires exactly 2 unique valuesr_data_types() producing a cryptic "missing value where TRUE/FALSE
needed" error when factor_size = NaNr_data_types() giving a misleading "not found in dataset" error
when skip_vars was not a character vectorgenerate_survival_data() producing NaN in iv_reop for patients
with very short follow-up timesgenerate_survival_data() permanently altering the global RNG state;
the session's RNG is now saved and restored on exitr_data_types() now validates all inputs before doing any work, so errors
are raised immediately with clear messagesr_data_types() input validation now explicitly checks that dataset is a
data.frame, skip_vars is a character vector, and factor_size is not NaNgenerate_survival_data() now uses labelled::var_label() consistently
to attach labels instead of attr() directlyif (interactive()) development block from
generate_survival_data.Rsurvival-data demonstrating generate_survival_data()
and its integration with r_data_types() and label_map()generate_survival_data() covering structure, column
types, outcome validity, reproducibility, RNG side-effect safety, and
variable labelsskip_vars (the
previous test only verified the skip_vars workaround)skip_vars type-error test to match the improved error messager_data_types() where dplyr::na_if() was called with a vector instead of scalar valuessample_data() where sample.int() parameters were reversed, causing errors for small sample sizesr_data_types() when using skip_vars - columns now maintain original orderfactor_size > 50 was changed to 20 without user consent - now errors insteadlubridate dependency that was never actually used in the packageinvisible() from label_map() and r_data_types() functionshvtiRutilities.news() function that was referenced but didn't existcontext() calls)label_map() documentation with @examples tag== TRUE comparisons to simpler boolean checkslabel_map() documentation (was incomplete sentence, wrong return type)r_data_types() documentation with clearer parameter descriptionsr_data_types(), label_map(), sample_data()