sdtm define-xml validation cdisc biomedicalconcepts metadata Stop Hand-Coding VLM: Compile CDISC Dataset Specializations into Study Metadata July 28, 2026 ClinPilot Insights Value-level metadata usually starts life as a reasonable spreadsheet. Then study setup gets busy. One standards programmer adds `WBC`. Another copies the row from an older oncology study. The central-lab transfer changes the reported unit. Someone fixes `LBSTRESU` in the SDTM program but not in the Define-XML workbook. Six months later, the test code, label, specimen, method, origin and codelist are being maintained in five places. All for one laboratory concept. That’s the manual mapping pain CDISC Biomedical Concepts and SDTM Dataset Specializations are meant to reduce. CDISC describes Biomedical Concepts as standards-agnostic semantic definitions, largely tied to NCI Thesaurus concepts. Dataset Specializations are the SDTM implementation layer: value-level definitions showing how a concept is represented in a particular domain. The useful part for programmers is straightforward. A Dataset Specialization can act as the upstream package from which we compile study metadata. Not copy. Compile. That distinction matters. ## The BC Isn’t Your Define-XML Row A Biomedical Concept answers the semantic question: what clinical concept are we talking about? A Dataset Specialization answers the SDTM question: how is that concept represented in a dataset? Study metadata answers the operational question: how will this study implement it? | Metadata layer | What it should tell you | |---|---| | Biomedical Concept | The unambiguous clinical meaning and associated data elements | | SDTM Dataset Specialization | Domain placement, variables, assigned terms, relationships and value-level constraints | | Sponsor standards library | Approved local defaults, mappings, naming conventions and governance status | | Study metadata | The selected concepts, study-specific origin, collection detail, units and permitted exceptions | | Define-XML | The submitted metadata describing the datasets that were actually delivered | These layers overlap, but they aren’t interchangeable. A Dataset Specialization may tell you that a concept belongs in LB and constrain `LBTESTCD`, `LBTEST`, specimen or units. It doesn’t know whether your study collects that result on a CRF, receives it from a central laboratory, derives it from two source values, or doesn’t collect one of the optional data elements at all. So don’t point a Define-XML generator straight at CDISC Library and call the job finished. That skips the study. ## Start with a Versioned Source Package CDISC Library exposes standards metadata through a REST API and requires an API key. CDISC also publishes current Biomedical Concept and Dataset Specialization exports through the COSMoS materials. Either route works, but the input must be frozen. Store at least: - The retrieval date - The source URL or API endpoint - The BC and Dataset Specialization package identifier or version - The SDTM and SDTMIG versions targeted by the study - The Controlled Terminology package date - The raw JSON or workbook checksum - The local compiler version Never compile against a floating “latest” endpoint during production. If CDISC publishes a revised specialization on Tuesday and your Define-XML changes on Wednesday without a standards decision, you don’t have reproducibility. You’ve got a surprise. A simple retrieval wrapper might look like this: ```python import hashlib import json import os from datetime import datetime, timezone from pathlib import Path import requests BASE_URL = "https://library.cdisc.org/api" ENDPOINT = "/mdr/bc/specializations/sdtm/latest" OUT = Path("metadata/source") headers = { "api-key": os.environ["CDISC_API_KEY"], "Accept": "application/json" } response = requests.get( f"{BASE_URL}{ENDPOINT}", headers=headers, timeout=60 ) response.raise_for_status() raw = response.content sha256 = hashlib.sha256(raw).hexdigest() OUT.mkdir(parents=True, exist_ok=True) (OUT / "sdtm_specializations.json").write_bytes(raw) manifest = { "endpoint": ENDPOINT, "retrieved_utc": datetime.now(timezone.utc).isoformat(), "sha256": sha256, "compiler_release": "vlm-compiler-1.2.0" } (OUT / "manifest.json").write_text( json.dumps(manifest, indent=2), encoding="utf-8" ) ``` The precise API path can change with the versioned CDISC Library API, so treat the endpoint as configuration and verify it in the developer portal. Don’t bury it inside a transformation function. And don’t log the API key. Obvious, yes. Still worth saying. ## Build a Study Concept Manifest The compiler needs an explicit list of concepts selected for the study. Otherwise, it either imports thousands of irrelevant definitions or guesses from the data after programming has started. Neither is good. Use a small, reviewed manifest: ```yaml study_id: ABC123 standards: sdtm: "2.0" sdtmig: "3.4" controlled_terminology: "2026-03-27" define_xml: "2.1" concepts: - specialization_id: "example-white-blood-cell-count" target_domain: "LB" status: "approved" overrides: origin_type: "Collected" origin_source: "Vendor" collected_units: - "10^9/L" - specialization_id: "example-systolic-blood-pressure" target_domain: "VS" status: "approved" overrides: origin_type: "Collected" origin_source: "Investigator" collected_units: - "mmHg" ``` The identifiers above are illustrative. Use the identifiers supplied by the frozen CDISC package, not labels typed from memory. Why a manifest? Because concept selection is a standards decision. It should survive code regeneration, be diffable in Git, and have an approval trail. An Excel tick mark in somebody’s Downloads folder doesn’t meet that bar. ## Normalize Before You Generate Anything Raw standards metadata and study metadata won’t have matching shapes. Put a canonical layer between them. For Findings domains, a practical normalized table may contain: - Study and source-package identifiers - BC identifier - Dataset Specialization identifier - Domain - Topic variable - Topic code and decode - Variable name - Assigned term or permitted value set - Data type and length policy - Codelist name and NCI identifier - Unit codelist and allowed unit subset - Specimen, method, position or fasting constraints - Origin type and source - Where-clause components - Sponsor override reason - Approval status This is the table your downstream jobs should read. SDTM specifications, terminology checks, Define-XML inputs and programming formats should all come from it. One source. Several outputs. Much less reconciliation. Here’s a trimmed Python pattern: ```python from copy import deepcopy def normalize_specialization(spec, study_cfg): rows = [] domain = spec["domain"] topic_var = spec["topicVariable"] specialization_id = spec["id"] overrides = study_cfg.get("overrides", {}) for variable in spec.get("variables", []): row = { "specialization_id": specialization_id, "domain": domain, "topic_variable": topic_var, "variable": variable["name"], "data_type": variable.get("dataType"), "assigned_value": variable.get("assignedTerm"), "codelist": variable.get("codelist"), "mandatory": variable.get("mandatory"), "origin_type": overrides.get("origin_type"), "origin_source": overrides.get("origin_source"), "override_reason": overrides.get("reason") } rows.append(deepcopy(row)) return rows ``` That isn’t a complete parser for the CDISC model. It’s the pattern: preserve the source semantics, normalize known properties, and keep local additions separate. Don’t flatten relationship metadata into a comma-delimited comment and throw the structure away. If the specialization says one variable qualifies another, retain that relationship in machine-readable form. You’ll need it when building where clauses and checking combinations. ## Treat Assigned Terms and Value Sets Differently This is where home-grown generators often wobble. An assigned term says the variable receives one fixed value for that specialization. A value set says several values are permitted. Those aren’t the same instruction. For a test specialization, the topic variable may be assigned: ```text LBTESTCD = WBC LBTEST = Leukocytes ``` A unit variable may allow a constrained subset: ```text LBORRESU in a study-approved set LBSTRESU = 10^9/L ``` The compiler should emit different metadata objects for those cases. A fixed topic code can become a where-clause discriminator. An allowed unit subset belongs in codelist metadata or a value-level codelist reference, depending on the study design and Define-XML representation. Don’t create a single-item codelist for every assigned term merely because the generator can. Define-XML guidance notes that where value-level metadata would produce a one-item codelist for each value, a codelist at the suitable level may be the better representation. Metadata volume isn’t the goal. Accurate description is. ## Compile the Where Clauses, Don’t Type Them Define-XML value-level metadata uses `ValueListDef`. Each value definition references an `ItemDef` and a `WhereClauseDef`, while the parent variable definition links to the value list. For a simple LB result definition, the logical output is: ```xml WBC ``` If specimen or method changes the metadata, the where clause may need more than `LBTESTCD`. That’s exactly why hand-coded VLM gets messy. `HCG` in serum may have a different unit, origin or datatype treatment from `HCG` in urine. A compiler can build a compound where-clause key from the specialization relationships and study selections. A person copying rows tends to notice after validation. Or after reviewer feedback. Worse. Use deterministic OIDs too. The same approved input should create the same OID every time. ## Add Compile-Time Checks Before Define-XML Validation Schema validation only tells you whether the XML is structurally acceptable. It won’t catch every bad study decision. Fail the metadata build when: - Two selected specializations assign different decodes to the same topic code. - A required specialization variable has no target SDTM variable. - A unit isn’t present in the frozen CT package or approved sponsor extension. - A codelist is referenced but missing from the compiled codelist table. - A study override has no reason or approval status. - Two value definitions produce the same where clause but conflicting metadata. - An origin is missing where study metadata requires it. - A selected concept targets a domain that isn’t included in the study specification. - Data contain a test code absent from the approved concept manifest. - The generated VLM describes a test code that can’t occur in the final dataset without a documented reason. SAS can handle the data-facing part cleanly: ```sas proc sql; create table work.unapproved_tests as select distinct a.domain, a.testcd from work.actual_findings_tests as a left join metadata.approved_topics as m on a.domain = m.domain and a.testcd = m.topic_code where missing(m.topic_code); create table work.metadata_without_data as select m.domain, m.topic_code from metadata.approved_topics as m left join work.actual_findings_tests as a on m.domain = a.domain and m.topic_code = a.testcd where missing(a.testcd); quit; ``` The second table isn’t automatically an error. Planned but uncollected tests can be valid study metadata depending on the agreed submission policy. It is, however, something a human should decide. The compiler shouldn’t quietly choose. ## Keep Sponsor Extensions Visible Dataset Specializations won’t cover every study concept on day one. CDISC Library listed 1,345 Biomedical Concepts and 1,326 SDTM Dataset Specializations in its April 2026 update, which is a serious body of content, not universal coverage. When no suitable specialization exists, create a sponsor extension using the same internal schema. Mark it clearly: ```yaml provenance: authority: sponsor status: approved-extension based_on_cdisc_specialization: null review_ticket: STD-4821 effective_date: "2026-06-15" ``` Don’t edit the downloaded CDISC object and save it under the same identifier. That destroys provenance. Keep the CDISC package immutable, store the extension separately, and merge them only in the compiled study layer. Same rule for overrides. Preserve the original value, changed value, reason, approver and effective date. Otherwise an impact assessment six months later becomes spreadsheet archaeology. ## VLM Is an Output, Not the Master The real change is architectural. Your VLM worksheet shouldn’t be where semantics are invented. It should be one rendered view of approved concept metadata. The same compiled layer can feed: - Study SDTM specifications - Define-XML value-level definitions - Codelist and unit subsets - Mapping configuration - SAS formats or R lookup objects - Data-versus-metadata checks - Annotated CRF support - Standards impact reports CDISC’s 360i work has already demonstrated the broader pattern: Biomedical Concepts and Dataset Specializations can feed CRFs, SDTM resources, Define-XML and dataset shells. You don’t need to build that whole chain on day one. Start with the ugly bit everybody already hates maintaining. The VLM workbook. Freeze the source. Select concepts explicitly. Normalize once. Apply governed study overrides. Generate the metadata. Then compare it against the actual datasets before validation. That’s how you get out of the copy-paste loop without replacing it with a black box nobody trusts. Recommended Insights sdtm clinicalprogramming sas RELREC That Reviewers Can Follow: Linking PR, TU, TR, and RS for RECIST 1.1 Build reviewer-readable links across scans, lesions, measurements, and RECIST 1.1 responses without relying on dates or sequence numbers. Jul 28, 2026 Read More sdtm adam validation Retiring --BLFL Without Breaking Baseline: Rebuilding SDTM Findings for --LOBXFL Replace --BLFL with --LOBXFL in SDTM while keeping protocol baseline logic where it belongs: traceable and reproducible in ADaM. Jul 28, 2026 Read More
sdtm clinicalprogramming sas RELREC That Reviewers Can Follow: Linking PR, TU, TR, and RS for RECIST 1.1 Build reviewer-readable links across scans, lesions, measurements, and RECIST 1.1 responses without relying on dates or sequence numbers. Jul 28, 2026 Read More
sdtm adam validation Retiring --BLFL Without Breaking Baseline: Rebuilding SDTM Findings for --LOBXFL Replace --BLFL with --LOBXFL in SDTM while keeping protocol baseline logic where it belongs: traceable and reproducible in ADaM. Jul 28, 2026 Read More
