Executive summary
The project did not stop at “ontology + evidence bridge closure.” The later stages added evidence refactor, raw-image-aware linkage, exact-linked subset packaging, contract-first reinterpretation, and an explicit handoff into signal-first aesthetic–emotion inference planning. The backbone is structurally ready; the remaining gap is empirical inference execution and alias propagation cleanup.
14
updated critical milestones in this report
3101
final canonical core terms stabilized
30k
exact-linked experiment subset prepared
3
final architectural layers: ontology, evidence, inference
Global architecture at the current endpoint
Term/core-first FEL architecture
Updated through contract lock, linked subset, and inference readiness
Validated ontology backbone
Validated evidence backbone
Contract / substrate governance
Inference target layer
Milestone 1
DF1 normalization foundation
Purpose
Standardize DF1’s five benchmark-specific raw formats into one normalized source substrate with global join keys and ZIP/QC packaging.
Why decisive
Every later DF1 registry, common event table, and evidence refactor depended on this normalized root rather than raw benchmark-specific logic.
Core outputs
df1_fel_extract_and_qc_split.py, normalized benchmark tables, split-stable global tables, and the first DF1 term/evidence/event separation lineage.
Update note
Even by Stage 95 the conclusion is that Stage 1–3 do not need rerun; they remain the trusted substrate.
Milestone 2
FP normalization foundation
Purpose
Bring Fashionpedia into FEL-normalized format and separate term/evidence/event outputs compatible with DF1 processing.
Why decisive
Without normalized FP, the project would remain a DF1 engineering effort instead of a cross-dataset ontology system.
Core outputs
fp_fel_extract_and_qc.py, fp_term_registry.csv, fel_events_fp_common_v2.csv.gz, and FP extract outputs.
Update note
Later evidence refactor repeatedly confirms FP as the more structurally stable source and the stronger instance-support axis.
Milestone 3
Dataset-specific SSOT registries fixed
Purpose
Freeze DF1 and FP vocabularies into dataset-internal SSOT registries before any canonical convergence.
Why decisive
The project later concludes that FEL is term/core-first. That only works because dataset term SSOT was stabilized first.
Core outputs
DF1 and FP term registries, attr_type materialization, HP merge into fel_core_terms.v1_1.csv and fel_core_aliases.v1_1.csv.
Update note
Stage 89–96 later show that upstream merge/materialization recovered 3205 aliases successfully; the remaining issue is downstream propagation.
Milestone 4
Common event schema and merge substrate established
Purpose
Convert DF1 and FP into a shared common-v2 event language with split normalization and merge compatibility QC.
Why decisive
This was the first observational unification step before canonical ontology convergence.
Core outputs
DF1 common events, split-normalized DF1 events, FP common events, and DF1+FP merged FEL event substrate.
Update note
Later refactor stages use these event substrates as conceptual templates even when rebuilding final evidence from enriched inputs.
Milestone 5
Pairwise alignment abandoned for Core Hub
Purpose
Replace DF1↔FP direct alignment with a hub-and-spoke strategy around FEL Core.
Why decisive
This changed the project from pairwise reconciliation into scalable canonical convergence.
Core outputs
Core seed, review queues, human/auto merged alignment, core-aligned DF1 and FP events.
Update note
Later stages explicitly reaffirm: dataset normalized → dataset term SSOT → core hub → ontology final → evidence grounding → inference.
Most important architectural pivot: FP stopped being the implicit destination ontology. FEL Core became the actual hub.
Milestone 6
Core seed, selective HITL, and facet stabilization
Purpose
Consolidate core candidates, review only risky cases, repair unresolved facets, and lock the core semantic typing.
Why decisive
Without final facet completeness, later ontology edges, parent groups, and inference rules would remain unstable.
Core outputs
Review queues, consolidation candidates, core_terms.v1.repaired.filled_final.csv, and backbone-final materialization.
Update note
Stage 44 resolves the last 52 blank attribute facets and produces 3101 terms with blank facet = 0.
Milestone 7
Ontology final became an operational definition graph
Purpose
Move from backbone-only ontology into final node and edge tables with operational definitions.
Why decisive
The ontology stopped being only a typed registry and became a working graph with IN_FACET, ALIAS_OF, HAS_ATTRIBUTE, HAS_PART, and SUBCATEGORY_OF.
Core outputs
core_terms.final.csv, core_facets.final.csv, ontology_parent_terms.final.csv, term_alias.final.csv, ontology_edges.final.csv.
Update note
This is where ontology final becomes the semantic contract later used by evidence grounding and inference planning.
Milestone 8
Evidence final first closed structurally
Purpose
Build final image / instance / evidence-event tables and prove ontology–evidence structural closure.
Why decisive
This is where the project first proves that ontology final and evidence final can close together with FK integrity.
Core outputs
images.final.v4/v5, instances.final.v5, evidence_events.final.v4, joint QC and bridge QC.
Update note
Stage 61 confirms the first full bridge-level closure. Later stages refine richness, not basic structural validity.
First hard closure: image, instance, event, and core term bridges reach zero FK misses at this stage.
Milestone 9
Evidence refactor rebuilt DF1/FP from enriched contracts
Purpose
Stop over-relying on old common-event outputs and rebuild DF1 / FP evidence using normalized + enriched contracts.
Why decisive
This stage shifts the project from 'common events reused as final evidence' to 'final evidence rebuilt from stronger lineage'.
Core outputs
DF1 enrichment, DF1 extract v6 / v6_2, FP extract v3, and rebuilt images.final.v5.csv.
Update note
DF1 enrichment succeeds numerically and FP extract stays stable; the later bottleneck is support propagation, especially on the DF1 side.
Milestone 10
Instances and event backbone stabilized at v9_1 / v7_1
Purpose
Rebuild instances and evidence events after v7/v8/v9 regressions, preserving FP recovery while improving DF1 support logic.
Why decisive
This is the operational turning point where the later 'current evidence backbone' is actually formed.
Core outputs
instances.final.v9_1.csv, evidence_events.final.v7_1.csv, bridge QC v5, and later full-backbone validation inputs.
Update note
The backbone is stable, but DF1 anchor counts remain low relative to FP. This later gets reinterpreted as an alias-propagation bottleneck rather than a bridge-structure failure.
Milestone 11
Upstream merge bug fixed, downstream alias bottleneck isolated
Purpose
Diagnose why upstream core merge recovered 3205 aliases but downstream alias files remained stuck at 2748 rows.
Why decisive
This prevents a false diagnosis. The evidence backbone is not the current failure point; alias propagation is.
Core outputs
Fixed merge/materialization interpretation, alias bottleneck diagnosis, and explicit non-rerun decision.
Update note
The important result is negative but decisive: do not rerun the entire pipeline; target alias lineage and backbone rebuild contracts instead.
Critical correction: by the late stages, the main problem is no longer bridge integrity or evidence build logic; it is the propagation path from
fel_core_aliases.v1_1 to backbone_alias_edges to term_alias.v1/final.Milestone 12
Contract-first architecture was explicitly redefined
Purpose
Reinterpret recurring failures as a contract problem rather than isolated script bugs.
Why decisive
This is where the architecture is explicitly restated as term/core-first and layer-separated: ontology, evidence, inference.
Core outputs
Design requirement for fel_schema_contract.yaml, fel_column_aliases.yaml, fel_ontology_contract.yaml, fel_stage_manifest.yaml, and validators.
Update note
This reframing is central to the current project identity. It clarifies that image-first or evidence-first interpretations are inaccurate.
Milestone 13
Exact-linked 30k subset and raw image governance completed
Purpose
Move beyond production tables and verify raw-image-aware linkage, benchmark-aware image inventories, and a clean exact-linked experimental subset.
Why decisive
This transforms FEL from only a production backbone into an experiment-ready substrate with explicit lineage.
Core outputs
Raw image inventory, benchmark-aware image linkage, linked subset package, and the final 30k experimental substrate.
Update note
The linked subset is now explicitly defined as 14,049 images, 30,000 instances, 678,548 events, and 530,642 semantic events.
Milestone 14
Final endpoint: inference-ready, not inference-complete
Purpose
Formalize the next layer as signal-first aesthetic–emotion inference over the exact-linked substrate.
Why decisive
The project now has an explicit handoff into inference design, but not yet empirical inference execution.
Core outputs
Signal-first pipeline design, continuous aesthetic score vector, VADN affect state, auxiliary Hourglass, whitelist governance, and dual-track reporting for inference planning.
Current boundary
Backbone is ready; feature materialization, signal validation, BWS pilot, centroid stats, and aesthetic→affect execution remain open empirical work.
Most accurate final statement: FEL has reached an inference-ready substrate, not a completed aesthetic–emotion model. The design is ready; the empirical layer is still to be executed.
Closing summary
The updated pipeline is now best described as: dataset normalization → dataset term SSOT → core hub convergence → ontology final → evidence grounding → raw-image-aware exact-linked subset packaging → signal-first inference readiness. The strongest part of the system is the ontology–evidence substrate. The main unresolved engineering bottleneck is alias propagation contract lock. The main unresolved scientific step is empirical aesthetic–emotion inference execution.
Ontology final = canonical semantic substrate
Evidence final = structurally valid observational backbone
Exact-linked 30k = clean experimental substrate
Next real work = feature materialization + BWS + inference