Update README.md

README.md

@@ -1,3 +1,217 @@
----
-license: mit
----
+# TCGA & IMPACT Genomic Biomarker WSI Training Checkpoints
+
+This repository hosts the full set of 200th-epoch classification checkpoints
+used for genomic biomarker prediction across TCGA and IMPACT cohorts.
+
+Checkpoints are organized strictly by:
+
+- Dataset source (`TCGA` or `IMPACT`)
+- Tumor type (e.g., `HNSC`, `UCS`, `BRCA`)
+- Gene (e.g., `PIK3CA`, `FBXW7`, `BRAF`)
+- Encoder (e.g., `virchow`, `gigapath_ft`)
+- Data split index (`split_1`, `split_2`, ...)
+
+---
+
+## Repository Structure
+
+The exact directory layout in this Hugging Face repo is:
+
+```text
+TCGA_Genomic_Biomarker_WSI_Training/
+├── TCGA/
+│   └── checkpoints/
+│       └── <TUMOR>/
+│           └── <GENE>/
+│               └── TCGA_trained_<TUMOR>_<GENE>_<ENCODER>_gma_<SPLIT>_200.pth
+│
+└── IMPACT/
+    └── checkpoints/
+        └── <TUMOR>/
+            └── <GENE>/
+                └── IMPACT_trained_<TUMOR>_<GENE>_<ENCODER>_gma_<SPLIT>_200.pth
+```
+
+### Examples
+
+```text
+TCGA/checkpoints/HNSC/PIK3CA/
+    TCGA_trained_HNSC_PIK3CA_virchow_gma_1_200.pth
+    TCGA_trained_HNSC_PIK3CA_virchow_gma_2_200.pth
+    TCGA_trained_HNSC_PIK3CA_gigapath_ft_gma_1_200.pth
+
+IMPACT/checkpoints/UCS/FBXW7/
+    IMPACT_trained_UCS_FBXW7_virchow_gma_1_200.pth
+    IMPACT_trained_UCS_FBXW7_gigapath_ft_gma_2_200.pth
+```
+
+Each checkpoint filename is self-descriptive:
+
+```text
+<SOURCE>_trained_<TUMOR>_<GENE>_<ENCODER>_gma_<SPLIT>_200.pth
+```
+
+---
+
+## Downloading
+
+### 1. Clone with Git LFS (recommended)
+
+```bash
+git lfs install
+git clone https://huggingface.co/chadvanderbilt/TCGA_Genomic_Biomarker_WSI_Training
+cd TCGA_Genomic_Biomarker_WSI_Training
+```
+
+### 2. Download an individual checkpoint
+
+```python
+from huggingface_hub import hf_hub_download
+
+ckpt_path = hf_hub_download(
+    repo_id="chadvanderbilt/TCGA_Genomic_Biomarker_WSI_Training",
+    filename="TCGA/checkpoints/HNSC/PIK3CA/TCGA_trained_HNSC_PIK3CA_virchow_gma_1_200.pth"
+)
+print(ckpt_path)
+```
+
+---
+
+## Checksum Logs (SHA256)
+
+Each upload run writes a checksum log under:
+
+```text
+logs/checkpoint_checksums_YYYYMMDD_HHMMSS.json
+```
+
+Each entry in this JSON file includes:
+
+- `source` (`TCGA` or `IMPACT`)
+- `tumor`
+- `gene`
+- `encoder`
+- `split`
+- `remote_path` (path inside this repo)
+- `size_bytes`
+- `sha256`
+- `timestamp`
+
+These logs allow you to verify that your local copies of the checkpoints
+match the originals used at upload time.
+
+---
+
+## Verifying Checkpoints After Download
+
+This repo includes a helper script `verify_checkpoints.py` for checksum verification.
+
+### Usage
+
+From the root of the cloned repo:
+
+```bash
+python verify_checkpoints.py logs/checkpoint_checksums_YYYYMMDD_HHMMSS.json
+```
+
+The script will:
+
+1. Read the JSON log.
+2. For each record, look up the file at `remote_path` under the repo root.
+3. Recompute SHA256 and size.
+4. Compare with the logged `sha256` and `size_bytes`.
+
+Example output:
+
+```text
+OK       : 128
+MISMATCH : 0
+MISSING  : 0
+```
+
+- **OK** – file exists and matches checksum and size.
+- **MISMATCH** – file exists but checksum or size does not match the log.
+- **MISSING** – file listed in the log is not present on disk.
+
+The script exits with a non-zero status code if there are any mismatches or missing files.
+
+---
+
+## `verify_checkpoints.py`
+
+For convenience, the expected content of `verify_checkpoints.py` is:
+
+```python
+import json, hashlib, sys
+from pathlib import Path
+
+def sha256_file(path, buf=1024*1024):
+    h = hashlib.sha256()
+    with open(path, "rb") as f:
+        while True:
+            chunk = f.read(buf)
+            if not chunk:
+                break
+            h.update(chunk)
+    return h.hexdigest()
+
+def main(log_json: str):
+    log_file = Path(log_json)
+    if not log_file.is_file():
+        print(f"ERROR: log not found: {log_json}")
+        sys.exit(1)
+
+    with log_file.open() as f:
+        records = json.load(f)
+
+    repo_root = Path(__file__).resolve().parent
+
+    ok = mismatch = missing = 0
+
+    for rec in records:
+        remote_path = rec["remote_path"]
+        expected_sha = rec["sha256"]
+        expected_size = rec["size_bytes"]
+
+        local_path = repo_root / remote_path
+
+        if not local_path.exists():
+            print(f"[MISSING] {remote_path}")
+            missing += 1
+            continue
+
+        actual_size = local_path.stat().st_size
+        actual_sha = sha256_file(local_path)
+
+        if actual_sha == expected_sha and actual_size == expected_size:
+            ok += 1
+        else:
+            mismatch += 1
+            print(f"[MISMATCH] {remote_path}")
+            print(f"  expected sha : {expected_sha}")
+            print(f"  actual sha   : {actual_sha}")
+            print(f"  expected size: {expected_size}")
+            print(f"  actual size  : {actual_size}")
+
+    print()
+    print(f"OK       : {ok}")
+    print(f"MISMATCH : {mismatch}")
+    print(f"MISSING  : {missing}")
+
+    if mismatch or missing:
+        sys.exit(1)
+
+if __name__ == "__main__":
+    if len(sys.argv) != 2:
+        print("Usage: python verify_checkpoints.py logs/checkpoint_checksums_YYYYMMDD_HHMMSS.json")
+        sys.exit(1)
+    main(sys.argv[1])
+```
+
+You can either copy this script into your local clone, or use the version
+shipped directly in the repository (if present).
+
+
+---
+license: mit
+---