⚡ EVF Enrichment Pipeline Dashboard

14,680
Total Parts
63
Columns
12
Gates Passed
573
Families
115
Subgroups

12-Gate Enrichment Pipeline

flowchart TB
    subgraph EXTRACT["🔬 EXTRACTION LAYER"]
        direction LR
        G0["G0: Atomic PDF Extraction
904 PDFs → 11,581 data points"]
        G1["G1: Variable Frequency Report
454 variables cataloged"]
    end

    subgraph MERGE["🔗 MERGE LAYER"]
        direction LR
        G2["G2: Catsy Master Backfill
+11 columns, 14,610 matched"]
        G3["G3: Schema Enforcement
Connection + Finish + Material + UoM"]
    end

    subgraph RESOLVE["⚖️ CONFLICT RESOLUTION"]
        direction LR
        G4["G4: 5-Expert Conflict Counsel
2,640 conflicts → 0 remaining"]
    end

    subgraph CONSOLIDATE["🧹 CONSOLIDATION"]
        direction LR
        G5["G5: Material Consolidation
6 → 4 material columns"]
        G6["G6: Category Gap Fill
MAIN_CAT → 100%"]
    end

    subgraph CLEAN["✨ SCHEMA CLEAN"]
        direction LR
        G7["G7: Dimension UoM + Desc Cleanup
20,602 inch marks + 5,027 PN removed"]
        G8["G8: Size Integrity
332 non-sizes cleared, 560 zeros"]
    end

    subgraph ENRICH["🚀 ENRICHMENT"]
        direction LR
        G9["G9: Description Schema
9,396 standardized + N/A removed"]
        G10["G10: Connection Deep Fill
90.5% → 98.3%"]
    end

    subgraph FINAL["🏆 FINAL ASSEMBLY"]
        direction LR
        G11["G11: 5-Expert Final Counsel
15,181 cells filled + Subgroups"]
    end

    G0 --> G1
    G1 --> G2
    G2 --> G3
    G3 --> G4
    G4 --> G5
    G5 --> G6
    G6 --> G7
    G7 --> G8
    G8 --> G9
    G9 --> G10
    G10 --> G11

    G11 --> DB[("💾 EVF_TRUE_FAMILIES.csv
14,680 rows × 63 columns")]

    style EXTRACT fill:#1e1b4b,stroke:#6366f1,color:#e2e8f0
    style MERGE fill:#1a2332,stroke:#06b6d4,color:#e2e8f0
    style RESOLVE fill:#1a2332,stroke:#f59e0b,color:#e2e8f0
    style CONSOLIDATE fill:#1a2332,stroke:#8b5cf6,color:#e2e8f0
    style CLEAN fill:#1a2332,stroke:#10b981,color:#e2e8f0
    style ENRICH fill:#1a2332,stroke:#ec4899,color:#e2e8f0
    style FINAL fill:#064e3b,stroke:#10b981,color:#e2e8f0
    style DB fill:#10b981,stroke:#10b981,color:#0a0e1a

Column Coverage Heatmap — All 63 Columns

Gate Execution Timeline

Family Subgroup Tree — Top Categories

flowchart LR
    ROOT["EVF Catalog
14,680 parts
573 families"]

    CAT1["1-1 Water Supply
~2,400 parts"]
    CAT2["1-2 Tubular
~800 parts"]
    CAT3["1-3 Drainage
~600 parts"]
    CAT4["1-4 Gas Connectors
~300 parts"]
    CAT5["1-5 Plumbing Spec
~500 parts"]
    CAT6["1-6 Valves
~1,800 parts"]
    CAT7["1-7 Faucet and Bath
~400 parts"]
    CAT8["2-x Specialty
~1,200 parts"]
    CAT9["3-1 PEX Products
~1,500 parts"]
    CAT10["4-1 Copper Fittings
~2,000 parts"]
    CAT11["5-1 Threaded
~1,800 parts"]
    CAT12["6-1 PVC / CPVC
~1,200 parts"]

    ROOT --> CAT1
    ROOT --> CAT2
    ROOT --> CAT3
    ROOT --> CAT4
    ROOT --> CAT5
    ROOT --> CAT6
    ROOT --> CAT7
    ROOT --> CAT8
    ROOT --> CAT9
    ROOT --> CAT10
    ROOT --> CAT11
    ROOT --> CAT12

    SG1["Standard
12,178 parts"]
    SG2["Threaded | Copper
501 parts"]
    SG3["Threaded | Brass
204 parts"]
    SG4["PEX F1807 | PEX
144 parts"]
    SG5["OD | Silicone
130 parts"]
    SG6["Solvent | Galv Steel
128 parts"]
    SG7["Compression | LF Brass
124 parts"]
    SG8["Threaded | Mixed
111 parts"]
    SG9["Threaded | Chrome
94 parts"]
    SG10["Other Subgroups
1,064 parts"]

    CAT10 --> SG2
    CAT6 --> SG3
    CAT9 --> SG4
    CAT5 --> SG5
    CAT12 --> SG6
    CAT1 --> SG7
    CAT11 --> SG8
    CAT7 --> SG9
    CAT8 --> SG10

    style ROOT fill:#6366f1,stroke:#818cf8,color:white
    style SG1 fill:#10b981,stroke:#10b981,color:#0a0e1a
    style SG2 fill:#f59e0b,stroke:#f59e0b,color:#0a0e1a
    style SG3 fill:#f59e0b,stroke:#f59e0b,color:#0a0e1a
    style SG4 fill:#06b6d4,stroke:#06b6d4,color:#0a0e1a
    style SG5 fill:#ec4899,stroke:#ec4899,color:white
    style SG6 fill:#8b5cf6,stroke:#8b5cf6,color:white

Top Subgroups

RankSubgroupPartsType
1Standard (uniform families)12,178Standard
2Threaded | Copper501Multi
3Threaded | Brass204Multi
4PEX (F1807) | PEX144Multi
5OD | Silicone130Multi
6Solvent | Galvanized Steel128Multi
7Compression | Lead Free Brass124Multi
8Threaded | Mixed111Multi
9Threaded | Chrome Plated94Multi
10Threaded | PTFE93Multi
11Hub | Chrome Plated86Multi
12Threaded | Stainless Steel77Multi
13Mechanical | PEX73Multi
14Threaded | Lead Free Brass60Multi
15Press | Stainless Steel53Multi

Data Source Architecture

flowchart TB
    subgraph SOURCES["📥 DATA SOURCES"]
        PDF["904 Cached PDFs
Spec Sheets"]
        CATSY["Catsy PIM Master
40,000+ products"]
        CATS["EVF Categories Tree
139 categories"]
        PHOCUS["PHOCUS Categories
Priceline mapping"]
    end

    subgraph ENGINES["⚙️ PROCESSING ENGINES"]
        E1["Atomic Extractor
PDF → structured data"]
        E2["Catsy Merger
Empty-cell-only backfill"]
        E3["Schema Enforcer
5-agent normalization"]
        E4["Conflict Counsel
5-expert vote"]
        E5["Competing Swarms
2v2+judge pattern"]
        E6["5-Expert Final
Gap fill + subgroups"]
    end

    subgraph OUTPUT["📊 OUTPUT LAYER"]
        DB[("EVF_TRUE_FAMILIES.csv
14,680 × 63")]
        AUDIT["Audit Logs
category, size, connection"]
        REPORT["Subgroup Report
573 families × 115 groups"]
        GATES["Gates YAML
12 gates tracked"]
        SCHEMA["Schema Registry
Canonical vocabularies"]
    end

    PDF --> E1
    CATSY --> E2
    CATS --> E5
    PHOCUS --> E3

    E1 --> E3
    E2 --> E4
    E3 --> E4
    E4 --> E5
    E5 --> E6

    E6 --> DB
    E6 --> AUDIT
    E6 --> REPORT
    E5 --> GATES
    E3 --> SCHEMA

    style SOURCES fill:#1e1b4b,stroke:#6366f1,color:#e2e8f0
    style ENGINES fill:#1a2332,stroke:#06b6d4,color:#e2e8f0
    style OUTPUT fill:#064e3b,stroke:#10b981,color:#e2e8f0
    style DB fill:#10b981,stroke:#10b981,color:#0a0e1a