Automatically validate, clean, and archive incoming data files

flowchart TD trig(["scheduled check for new files"]):::trigtime s0[["Validate each file"]]:::mi s1["Email status summary"]:::svc trig --> s0 gx{"× does file pass validation"}:::gate s0 --> gx p00["Clean and reformat file"]:::task gx -->|"validation passed"| p00 p01["Save to GitHub"]:::task p00 --> p01 p10["Flag file as failed"]:::task gx -->|"validation failed"| p10 jn{"○ summarize results"}:::gate p01 --> jn p10 --> jn jn --> s1 out[/"cleaned files archived in GitHub"/]:::out pay{{"reliable auditable file processing"}}:::pay s1 --> out out --> pay s1 -. "failures recorded, run continues" .-> out classDef task fill:#e7f6fe,stroke:#34b8f0,color:#2c2a29 classDef svc fill:#f6f8fa,stroke:#7c8795,color:#2c2a29 classDef mi fill:#e7f6fe,stroke:#0079a8,color:#2c2a29,stroke-width:2px classDef human fill:#fff,stroke:#0079a8,color:#0079a8 classDef store fill:#f6f8fa,stroke:#0079a8,color:#2c2a29 classDef trig fill:#00a4eb,stroke:#0079a8,color:#fff,font-weight:bold classDef trigtime fill:#00a4eb,stroke:#0079a8,color:#fff,font-weight:bold classDef trigdata fill:#8ad4f5,stroke:#0079a8,color:#06314c,font-weight:bold classDef gate fill:#fff,stroke:#e8a23d,color:#6b4708,font-weight:bold classDef out fill:#1f9d6b,stroke:#167a53,color:#fff,font-weight:bold classDef pay fill:#06314c,stroke:#021f33,color:#fff