--- title: "Adopt an existing study for descriptive and EDA work" author: "John Ehrlinger" vignette: > %\VignetteIndexEntry{Adopt a study for descriptive and EDA work} %\VignetteEngine{quarto::html} %\VignetteEncoding{UTF-8} format: html: toc: true embed-resources: true --- This tutorial starts with an existing study directory and an existing dataset. It takes the study through identity setup, data registration, descriptive tables and EDA plots. A later tutorial can then begin with a specific analysis type instead of repeating this setup. Adoption is additive. It creates the hvtiR study contract without renaming the legacy working directories or deleting old files. If the study does not exist yet and you are starting from a newly delivered dataset, follow [Start a new study](new-study.html) instead. The executable example uses one 40-row synthetic dataset. Its files stay in a temporary directory and disappear after the article renders. For translating existing SAS descriptive jobs, see [From SAS descriptive jobs to R](sas-to-r-descriptive.html). ## Open the study as an RStudio Project In RStudio, choose **File > New Project > Existing Directory** and select the study directory. Open that `.Rproj` whenever you work on the study. RStudio starts the session in the project directory, so there is no working-directory step to remember or repeat. Confirm the location before changing the study: ```r normalizePath(".") ``` When the Console is already at the study root, use `"."` anywhere a function asks for `root`. The executable example below uses an explicit temporary path only because it cannot open an RStudio Project while pkgdown renders the page. ### Select the R version If the study already has an `renv.lock`, its `R` `Version` entry records the R version to preserve. Select that version in RStudio and restart the session. Do not silently upgrade an already pinned study. If the study is not already pinned, select **R 4.6** instead of the older 4.4.1 default, restart the session and confirm the active version: ```r R.version.string ``` ### Load the study packages Install or update the hvtiR family, then check whether anything else is missing or out of date: ```r pak::pak("ehrlinger/hvtiR") hvtiR::status() ``` In the study session, attach the two packages used throughout this tutorial: ```r library(hvtiRutilities) library(hvtiRtemplates) ``` ```{r} #| label: synthetic-setup #| include: false knitr::opts_chunk$set(collapse = TRUE, comment = "#>") library(hvtiRutilities) library(hvtiRtemplates) workspace <- withr::local_tempdir(.local_envir = environment()) adopted_root <- file.path(workspace, "legacy-study") folders <- c("datasets", "descriptive", "distributions", "analyses", "graphs", "documents", "estimates") dir.create(adopted_root) for (folder in folders) dir.create(file.path(adopted_root, folder)) i <- seq_len(40L) built <- data.frame( ccfid = i, dead = i %% 2L, iv_dead = i / 10, iv_fup = i / 10 + 1, year = 2000L + i %% 10L, female = i %% 2L, race_grp = 1L + i %% 3L, repair = as.integer(i %% 3L == 0L), age = 40 + i, bmi = 20 + i / 10, hx_chf = as.integer(i %% 4L == 0L), lvmassi = 80 + i, iv_opyrs = i + 0.25 ) utils::write.csv( built, file.path(adopted_root, "datasets", "built.csv"), row.names = FALSE ) ``` ## Adopt the existing study `study_setup()` writes `_study.yml`, environment defaults and a project file when one is missing. With `adopt = TRUE`, it keeps the existing directory layout and every existing study file. Use the title and ID from Study Tracker: ```r study_setup( root = ".", study = "Study title from Study Tracker", study_tracker_id = 42L, adopt = TRUE ) study_root() study_status() ``` The current API requires `root` for this first call. Here `"."` is the project directory that RStudio established. Once `_study.yml` exists, `study_root()`, `open_job()` and `render_job()` find it from anywhere inside the project. The executable example uses its temporary directory instead: ```{r} #| label: adopt-study # A vignette builds a throwaway study; in a real study, run hvtiR::setup(). study_setup( adopted_root, study = "Synthetic legacy study", study_tracker_id = 42L, adopt = TRUE, cleanup = FALSE, renv = FALSE ) root <- adopted_root ``` ## Pin the package environment `study_status()` reports a missing `renv.lock` until the study initializes its package environment. For an unpinned study, do this after selecting R 4.6 and restarting RStudio. First open RStudio's Terminal tab at the study root and run: ```sh organize_templates.sh ``` This `qhsprograms` script creates `templates/` under each first-level study folder, moves that folder's `tp*` files into it and replaces the root `.renvignore` with the centrally managed copy used for dependency discovery. Before running it, make sure a destination `templates/` directory does not already contain a same-named file. Then return to the Console: ```r renv::init() # Install or update the packages used by the study, then record them. renv::snapshot() ``` Snapshot again when an accepted result needs a new reproducible package state. The temporary example does not run `renv::init()` because it changes the active R project while the article renders. ## Register the existing dataset Registration records the input filename, checksum and population. It does not declare a study-wide endpoint or cohort: an endpoint-free descriptive job and an event-time analysis can read the same dataset with different valid job definitions. ```r register_data( built = "built.csv", role = "study", population = "Patients meeting the study inclusion criteria" ) verify_manifest() ``` Do not invent an outcome or follow-up field during setup. Each analysis job declares its own outcome variables, coding, filters and cohort checks. The descriptive jobs below depend only on a registered study dataset. ```{r} #| label: register-data register_data( adopted_root, built = "built.csv", role = "study", population = "Synthetic full cohort" ) verify_manifest(file.path(adopted_root, "manifest.yaml")) cfg <- study_config(adopted_root) study_data <- read_built(cfg) nrow(study_data) ``` Compare the registered file and its manifest checksum with the data build before proceeding. A job that filters this data, or analyses an event and follow-up time, must derive and check its own reference counts. ### If EDA uses a subset The jobs can read the registered study dataset directly. If EDA needs a subset, create it with a stated rule, save it as a separate dataset and register that file too. For example: ```r study_data <- read_built(study_config()) eda <- subset(study_data, age >= 50) # Replace with the study's actual rule. utils::write.csv( eda, file.path(study_dir("datasets"), "eda.csv"), row.names = FALSE ) register_data( built = "eda.csv", dataset = "eda", role = "named", kind = "subset", population = "Patients age 50 or older" ) ``` In a job that uses this subset, set `DATASET <- "eda"` and `ANALYSIS_SET <- NULL`. When the whole registered study dataset is the intended population, use `DATASET <- "built"` and `ANALYSIS_SET <- NULL` instead. For a subset of the study dataset that several jobs share, an analysis set is the alternative: write it with `hvtiRdatabuild::write_analysis_set()` and read it with `DATASET <- "built"` and the set's name in `ANALYSIS_SET`. ## Create the descriptive and EDA jobs `open_job()` creates a missing job and opens it in RStudio. If the job already exists, it opens the study-authored file without replacing any edits. ### Understand the name Each call supplies four fields. `prefix` and `qualifier` select a template; the other two fields keep related outputs together: - `dc` means descriptive: the checks, tables and EDA plots. Legacy studies also hold `dp` (descriptive plot) jobs, which these templates replace. - Qualifiers such as `general`, `tables` and `eda` select one template in that family. - During EDA there may be no modeled endpoint or analysis method. `cohort` names the subject being described and `eda` names the stage. These are organizational labels, not invented analysis choices. For example, `open_job("dc", "cohort", "eda", qualifier = "tables")` creates `descriptive/dc.tables.cohort.eda.qmd` in an adopted legacy study. Its HTML report stays beside the QMD, while its Word table goes under `documents/cohort-eda/`. The trends and EDA report jobs sit in `descriptive/` too, and their plot PNGs go under `graphs/cohort-eda/`. ### Start with the generally applicable jobs Create general checks and descriptive tables for every registered study: ```r general <- open_job("dc", "cohort", "eda", qualifier = "general") tables <- open_job("dc", "cohort", "eda", qualifier = "tables") ``` The EDA report template is also part of this stage, but it requires a meaningful `X_VAR` for the horizontal axis. Create it when the dataset has an appropriate reference date, calendar year or follow-up measure: ```r eda <- open_job("dc", "cohort", "eda", qualifier = "eda") ``` Its follow-up section needs the death and follow-up fields; take `"followup"` out of `SECTIONS` when the data do not have them; `SECTIONS <- c("continuous", "percent", "count")` draws the variable sections alone. Two additional jobs are conditional. Add `dc-gfup` only when the registered event and follow-up fields answer a real completeness question. Add `dc-trends` only when the data have a meaningful calendar or operation-time field: ```r gfup <- open_job("dc", "cohort", "eda", qualifier = "gfup") trends <- open_job("dc", "cohort", "eda", qualifier = "trends") ``` The synthetic data have all of those fields, so the executable example scaffolds all five jobs without opening editor windows: ```{r} #| label: scaffold-example-jobs study_jobs <- c( general = add_job("dc", "cohort", "eda", adopted_root, "general"), tables = add_job("dc", "cohort", "eda", adopted_root, "tables"), gfup = add_job("dc", "cohort", "eda", adopted_root, "gfup"), trends = add_job("dc", "cohort", "eda", adopted_root, "trends"), eda = add_job("dc", "cohort", "eda", adopted_root, "eda") ) study_jobs ``` ## Work the jobs and generate output Each new job contains `EDIT:` markers for decisions that cannot come from the template: variables, labels, groups, units and output choices. `open_job()` opens the QMD in RStudio's file editor. Edit the file there, working from the top marker downward, and **Render** to see what your choices do. A draft render carries a DRAFT banner while any marker remains. Always render; do not run the chunks by hand with **Run All**. Rendering starts at the top in a clean session, much like running a SAS job from beginning to end, so the report comes from this job alone. Run All works in your current session, where an object or a setting left by another job can stand in for one this job does not make. Render every edited job before leaving the study so the saved QMD is still in an executable state. If you want the same final check from the Console, run `render_job(tables, final = TRUE)`, substituting the job you edited. The HTML report stays beside its QMD in `descriptive/`. The descriptive Word table is filed under `documents/cohort-eda/`; trend and EDA PNGs are filed under `graphs/cohort-eda/`. Open those artifacts and check the population, variables, labels, counts, summaries and axes—not merely that the files exist. If the study has older SAS jobs, continue with [From SAS descriptive jobs to R](sas-to-r-descriptive.html). That article owns the detailed source mapping and migration workflow so adoption itself remains short and non-destructive. ### When registered data change Registration is endpoint-neutral: it records the dataset file, its population description and manifest facts, but does not declare a study-wide endpoint or cohort. After the study dataset is rebuilt, review the rebuilt file before registering it. `hvtiRutilities::review_built()` compares the working file in the datasets folder with its registered version and describes how the data changed, without writing anything. It does not certify that the new file is analytically or clinically correct; that judgment stays with you. When the review reads as expected, `hvtiRutilities::update_manifest(dataset = "study")` registers the rebuilt file as the next dated version, keeps every earlier version and records the change in `manifest.yaml`. Jobs read the new version from then on, so review dependent analysis sets and rerender their jobs.