sdtm validation sas cdisc disposition regulatorysubmissions Interim SDTM Without Invented Endpoints: Open Queries, Snapshot Cutoffs, and Honest DS Records July 28, 2026 ClinPilot Insights An interim SDTM delivery creates an awkward programming problem: the extract has stopped, but the study hasn't. That distinction sounds obvious until the DS program reaches subjects who are active, have unresolved disposition queries, or haven't yet reached the protocol milestone expected by downstream checks. Then someone asks whether we can add an end-of-study record “so validation passes.” No. The cutoff is an operational boundary. It isn't a clinical event. A clean interim package should say what was known at the cutoff, retain legitimate partial dates, and explain structural incompleteness without manufacturing outcomes. That gives reviewers something they can actually trust. It also prevents a nasty reconciliation exercise when the next snapshot arrives and those invented records have to disappear. ## A Data Cut Is Not a Disposition Event SDTM's DS domain represents protocol milestones, disposition events, and other events relevant to subject disposition. It doesn't exist to certify that an extract completed successfully. These are different facts: - The database extract ran through June 30. - Subject 1045 completed treatment on June 24. - Subject 1045 remains in survival follow-up. - Subject 1045 completed the study. Only the last three statements concern the subject. And they're not interchangeable. If a subject is active at the cutoff, the absence of an end-of-study disposition can be correct. Creating `DSDECOD="COMPLETED"` merely because all available pages were processed turns an operational fact into a false clinical statement. The same problem appears with pseudo-status values such as `ONGOING`, `PENDING`, or `NOT YET COMPLETED`. Before assigning one of those to `DSDECOD`, ask a blunt question: was that an actual collected or otherwise defensibly derived disposition event, and is the value valid for the applicable codelist? Usually, “ONGOING” is just the programmer trying to fill silence. Don't do that. ## Freeze the Snapshot Before Mapping It The first control should happen upstream of SDTM. Define one reproducible cutoff and apply it consistently to all source inputs. A weak process grabs the latest AE export on Monday, the latest DS export on Tuesday, and a coding refresh on Thursday. Every file is individually current. Together, they're incoherent. A defensible snapshot records at least: - Extract timestamp and time zone. - EDC transaction or freeze identifier, where available. - External vendor transfer versions. - Medical coding dictionary version and coding cutoff. - Query-state extract timestamp. - Protocol, CRF, SDTM specification, controlled terminology, and validation-rule versions. - Known source systems that weren't included. Keep this in a machine-readable manifest. A spreadsheet can work, but a CSV or JSON file under version control is less prone to quiet edits. ```text source,extract_timestamp,timezone,transfer_id,included_through EDC,2026-06-30T23:59:59,UTC,EDC_20260701_001,2026-06-30 IRT,2026-07-01T02:15:00,UTC,IRT_20260701,2026-06-30 CENTRAL_LAB,2026-07-02T06:00:00,UTC,LAB_20260702,2026-06-30 CODING,2026-07-03T12:00:00,UTC,MEDDRA_29_0_CUT1,2026-06-30 ``` The point isn't administrative neatness. It's repeatability. When a reviewer asks why an AE exists but its later resolution doesn't, you should know whether the answer is clinical timing, source latency, or an inconsistent pull. ## Separate Three Kinds of Missingness Programmers get into trouble when every blank is treated as the same blank. For an ongoing study, it helps to classify missing information before SDTM derivation. | State | Meaning | SDTM treatment | |---|---|---| | Not expected yet | The event or milestone hasn't occurred by cutoff | Do not create a placeholder record | | Expected but unresolved | Source indicates an event, but a query or coding issue remains | Preserve supported source content; don't guess | | Collected as partial | The source intentionally contains an incomplete date | Retain valid ISO 8601 precision | | Source unavailable | A scheduled transfer or form wasn't present in the snapshot | Document the omission and assess affected domains | | Structurally absent | No records exist because no qualifying observations occurred | Represent according to the standard and metadata, not with dummy rows | That first row saves a lot of grief. No end-of-study record for an active subject isn't necessarily missing data. It may be an event that hasn't happened. ## Build DS From Evidence, Not Expected Milestones A practical DS program should start with qualifying source events and apply an explicit mapping hierarchy. For example: - Map collected protocol milestones such as informed consent and randomization. - Map actual treatment and study disposition events. - Retain the verbatim source value in `DSTERM`. - Map `DSDECOD` according to `DSCAT` and the applicable controlled terminology. - Do not infer completion from the absence of later visits. - Do not derive lost to follow-up merely because contact data are stale. - Do not create study completion at the snapshot date. `DSDECOD` can draw from different codelists depending on the record context. For SDTMIG v3.3 implementations, protocol milestones and disposition events don't use one undifferentiated value list. That means a mapping that ignores `DSCAT` can produce technically polished nonsense. Here's a stripped-down SAS pattern. ```sas data ds_pre; set src_disposition; length studyid $20 domain $2 usubjid $40 dsterm $200 dsdecod $80 dscat $40 dsstdtc $19 epoch $40; studyid = "&studyid"; domain = "DS"; usubjid = catx("-", studyid, strip(subject)); dsterm = strip(source_term); select (upcase(strip(event_group))); when ("PROTOCOL MILESTONE") do; dscat = "PROTOCOL MILESTONE"; select (upcase(strip(source_term))); when ("INFORMED CONSENT SIGNED") dsdecod = "INFORMED CONSENT OBTAINED"; when ("RANDOMISED", "RANDOMIZED") dsdecod = "RANDOMIZED"; otherwise call missing(dsdecod); end; end; when ("STUDY DISPOSITION") do; dscat = "DISPOSITION EVENT"; select (upcase(strip(source_term))); when ("COMPLETED STUDY") dsdecod = "COMPLETED"; when ("WITHDREW CONSENT") dsdecod = "WITHDRAWAL BY SUBJECT"; when ("LOST TO FOLLOW UP", "LOST TO FOLLOW-UP") dsdecod = "LOST TO FOLLOW-UP"; when ("DEATH") dsdecod = "DEATH"; otherwise call missing(dsdecod); end; end; otherwise do; dscat = "OTHER EVENT"; call missing(dsdecod); end; end; dsstdtc = strip(source_event_dtc); if not missing(dsterm); run; ``` The unmapped branches are deliberate. They should feed an exception listing, not a default assignment. ```sas data ds_mapping_exceptions; set ds_pre; if missing(dsdecod); length issue $120; issue = "No approved DSDECOD mapping for collected source term"; run; ``` A blanket fallback such as `dsdecod=upcase(dsterm);` feels convenient right up to the validation review. Then you're sorting through spelling variants, nonstandard terms, and values assigned under the wrong codelist. Been there. It's tedious and avoidable. ## Partial Dates Aren't Permission to Impute SDTM SDTM date/time variables use ISO 8601 character representation and can preserve reduced precision. If source collection supports only a year and month, retain that precision. ```sas data ds_dates; set ds_pre; if not missing(event_date_full) then dsstdtc = put(event_date_full, e8601da.); else if not missing(event_year) and not missing(event_month) then dsstdtc = cats( put(event_year, 4.), "-", put(event_month, z2.) ); else if not missing(event_year) then dsstdtc = put(event_year, 4.); run; ``` So a source value known only as March 2026 becomes: ```text 2026-03 ``` It doesn't become March 1, March 15, or March 31 in SDTM just because another derivation wants a complete date. Imputation belongs in analysis data when the analysis rules call for it, with traceability and flags. SDTM should preserve the collected precision. Quietly completing a DS date can change event ordering, study-day calculation, and `EPOCH` assignment. That's not cosmetic. When the date is too partial to support a reliable study day, leave `DSSTDY` missing. When it can't support a defensible epoch lookup, don't force `EPOCH`. ## Derive EPOCH Only When the Timing Supports It Under SDTMIG v3.3 guidance, `EPOCH` in DS is based on `DSSTDTC`. For complete dates or date-times, a subject-element lookup can be straightforward. Partial dates can overlap multiple elements, though. That's where careless code starts inventing precision. ```sas proc sql; create table ds_epoch as select a.*, case when lengthn(a.dsstdtc) >= 10 and input(substr(a.dsstdtc, 1, 10), yymmdd10.) between input(substr(b.sestdtc, 1, 10), yymmdd10.) and input(substr(b.seendtc, 1, 10), yymmdd10.) then b.epoch else "" end as epoch length=40 from ds_dates as a left join se as b on a.studyid = b.studyid and a.usubjid = b.usubjid; quit; ``` Production code needs better handling for open-ended elements, date-times, overlapping intervals, and partial SE dates. But the principle holds: derive only when the timing proves the assignment. A warning for missing `EPOCH` with a partial `DSSTDTC` can be explained. A confident but wrong epoch is harder to defend. ## Keep Open Queries Out of Published Clinical Values Query status is operational metadata. It shouldn't normally be inserted into clinical variables just to advertise uncertainty. Suppose the EDC contains: - Reported reason: “subject moved away” - Data-management query: confirm whether contact attempts meet the protocol definition of lost to follow-up - Query state at cutoff: open The DS output shouldn't automatically say `LOST TO FOLLOW-UP`. Nor should `DSTERM` become `OPEN QUERY`. Use the supported source statement if it qualifies as a disposition event, hold the standardized mapping as an exception if the answer is unresolved, or omit the event if the source hasn't established that an event occurred. The right treatment depends on the collection design and mapping specification. Keep a separate query-impact report for internal review: ```sas proc sql; create table query_impact as select q.subject, q.form, q.field, q.query_status, q.query_text, m.domain, m.target_variable from open_queries as q left join mapping_inventory as m on upcase(q.form) = upcase(m.source_form) and upcase(q.field) = upcase(m.source_field) where upcase(q.query_status) = "OPEN"; quit; ``` That report answers the useful question: which submission variables might change after query resolution? It also gives QC something better than a raw dump of 4,000 open queries, most of which may have no effect on the delivered domains. ## Validate the Snapshot as a Snapshot Run the same structural, metadata, terminology, and regulatory checks used for a final package. Don't disable rules globally because the study is ongoing. Instead, classify findings. | Finding type | Typical action | |---|---| | Actual mapping defect | Fix before delivery | | Invalid controlled term | Correct the mapping or terminology | | Missing event that hasn't occurred | No fabricated record; explain if the check assumes completion | | Expected variable unavailable due to source latency | Assess reviewer impact and document | | Partial date causing timing limitation | Retain source precision and explain downstream blanks | | Cross-domain inconsistency | Determine whether extracts were misaligned or the clinical data are genuinely unresolved | Validation tools don't know your data-cut intent unless the data and documentation make it visible. A clean report is nice. An honest report with a few well-explained, non-correctable findings is better than a clean report produced by false records. And yes, seeing zero unexplained high-severity findings still feels good. Some habits never leave you. ## Put the Cutoff Story in the cSDRG The cSDRG is where the reviewer should learn what this package represents. Don't bury the explanation in a validation spreadsheet with cryptic comments like “study ongoing.” Document: - The clinical cutoff date and extract timestamps. - That the study was ongoing when the snapshot was created. - Which domains or vendor transfers were incomplete or delayed. - Whether open queries remained and how their possible SDTM impact was assessed. - That no placeholder DS records were created for milestones not reached by cutoff. - How partial dates were represented. - Which validation findings result from legitimate interim incompleteness. - Whether later submissions will replace the snapshot with a refreshed full package. Be exact. “Data may be incomplete” tells the reviewer almost nothing. A stronger statement reads like this: > The submitted SDTM datasets represent data available through 30 June 2026. Subjects active on that date do not have an end-of-study DS record unless a qualifying disposition event had been collected. Open queries affecting disposition terminology were not resolved by inference; affected records are identified in the data-conformance discussion. That's useful. It explains both the limitation and the programming decision. ## Add Snapshot-Specific QC Standard double programming won't catch every interim defect. Add checks that compare clinical facts with the cutoff policy. ```sas /* Flag end-of-study completion dated after the declared cutoff */ data qc_ds_after_cutoff; set ds; cutoff = input("&cutoff_date", yymmdd10.); if lengthn(dsstdtc) >= 10 then do; dsdt = input(substr(dsstdtc, 1, 10), yymmdd10.); if dscat = "DISPOSITION EVENT" and dsdecod = "COMPLETED" and dsdt > cutoff then output; end; format cutoff dsdt yymmdd10.; run; ``` Also check for: - Subjects marked completed in DS who have later on-treatment or follow-up records that contradict the mapping. - End-of-study dates copied from the snapshot cutoff. - Identical completion dates across an implausibly large number of active subjects. - `RFENDTC` populated from the extract date rather than a subject-level reference-period event. - DS terms mapped differently between snapshots without a source or terminology reason. - Records that disappear between cuts without documented source deletion or correction. That last check matters. Interim data should change, but every material change should be traceable. ## Treat Each Cut as a Reproducible Release A workable rolling-submission setup retains: - Immutable source extracts. - The snapshot manifest. - Mapping specifications and controlled terminology version. - Program commit or release tag. - Validation engine and rule versions. - Generated XPT files and Define-XML. - Validation results and dispositions. - A record-level delta against the prior cut. Don't overwrite the previous package with `final_v2_really_final`. We've all inherited that folder. A subject's DS record may legitimately change after query resolution. The audit trail should show the earlier source, the later source, the mapping rule applied at each cut, and why the output changed. That turns a moving study into a series of explainable states rather than a pile of mutable extracts. ## The Rule That Keeps the Package Honest For interim SDTM, absence can be accurate. An active subject doesn't need an invented completion record. A partial date doesn't need an imputed day. An open query doesn't need a guessed decode. A validation warning doesn't need to be erased at any cost. Freeze the source boundary, map only supported clinical facts, retain uncertainty where the source contains it, and explain the rest in the cSDRG. The next snapshot will still bring changes. At least you won't spend the refresh removing things that never happened. Recommended Insights sdtm define-xml validation Stop Hand-Coding VLM: Compile CDISC Dataset Specializations into Study Metadata Compile CDISC Dataset Specializations into governed study VLM before test codes, units, origins, and codelists drift apart. Jul 28, 2026 Read More sdtm clinicalprogramming sas RELREC That Reviewers Can Follow: Linking PR, TU, TR, and RS for RECIST 1.1 Build reviewer-readable links across scans, lesions, measurements, and RECIST 1.1 responses without relying on dates or sequence numbers. Jul 28, 2026 Read More
sdtm define-xml validation Stop Hand-Coding VLM: Compile CDISC Dataset Specializations into Study Metadata Compile CDISC Dataset Specializations into governed study VLM before test codes, units, origins, and codelists drift apart. Jul 28, 2026 Read More
sdtm clinicalprogramming sas RELREC That Reviewers Can Follow: Linking PR, TU, TR, and RS for RECIST 1.1 Build reviewer-readable links across scans, lesions, measurements, and RECIST 1.1 responses without relying on dates or sequence numbers. Jul 28, 2026 Read More
