[feixufeixu]

With KC_V2_ASPECTS on, pull rebuilds a model pushed with the second-generation types from its entries. It writes the model back as its model file, its profile files and its overview sidecars.

The mapping rules here were settled by Fei Xu with Dmitry Lychagin, the author of the mapping.

What changed

• Pull reads each model, entity, metric and relationship entry with every aspect push writes on it. A model whose semantic-model aspect has specVersion takes the new reader.
• The rebuilt model has its flavor, its profiles in stored order, its deployments, sources as written, keys, expressions with their dialect lists, descriptions, ai_context with custom members, and third-party blocks. Each named profile gets its sources, keys, field expressions, excluded fields and relationship columns from the entries. Its metrics_exclude is '*' when it excludes every metric, and otherwise a list sorted by name.
• Each relationship comes from its own entry. Pull reads no entry link.
• A metric's entity comes back as push stored it, declared or derived, and pull writes it as entity:.
• An entity's schema lists only the fields it declares. A field the entity only rebinds comes back from its sql-expressions records.
• Entities, metrics and relationships come back sorted by name, whatever order the catalog lists them in.
• Pull writes the model file, and one profile file per named profile of a Google-flavor model. A vanilla model keeps its profiles in its GOOGLE block. A rewritten profile file keeps the actions: it binds locally, since the catalog stores no profile's actions.
• Pull writes <entry-id>.overview.md for each entry whose overview holds content, a content type or links. Content whose first line is --- gets an empty frontmatter, so the next push reads the same content.
• A local profile file or overview sidecar that the pull did not write draws a warning, and --force-remove deletes it.
• When one read of an entry returns 100 aspects, pull reads the entry again once per aspect type.
• Pull passes KC_TYPE_PROJECT to the catalog reads. A flag-off pull now looks up relationship links in that project, where push writes them. Otherwise a flag-off pull is unchanged.

What now fails

entry '<id>' holds 100 or more <type> aspects, and Knowledge Catalog returns at most 100 in one read, so pull cannot read them all; pull aborted

New warning: local overview sidecar <path> is not in the catalog; delete it, or pull with --force-remove to delete it.

What to look at closely

1. readEntityV2 and withRebindings in kc_converter.ts, which rebuild the fields an entity declares and the ones it only rebinds.
2. How readModelV2 fills each named profile from the entity, relationship and metric entries.
3. The second-generation branch of pullSemanticModel in commands.ts and the file writer it calls, which writes the rebuilt files and sidecars and finds the stale ones.
4. The round-trip test in pull_kc.test.ts. It pushes a Google-flavor model with profile files and a vanilla model with GOOGLE blocks, pulls each from entries listed in reverse, writes the files, loads them and pushes again. It expects the same entries, links, and BigQuery and Spanner DDL.

[merr1988]

Thanks Fei, the reader mirrors the writer closely, the round-trip test is a great safety net, and the per-type re-read answers my question from #556. Approving, with one question and a small optional note inline.

[Question] metrics_exclude comes back as '*' when it lists every metric (kc_converter.ts L271)
Since push expands '*' and never stores it, pull can't tell '*' from a list that happens to name every metric. Writing '*' back changes meaning on the next edit: after a pull, adding a metric silently excludes it from that profile, though the author's list didn't. Writing the sorted list only changes notation for someone who wrote '*', and never excludes more than they asked. Was '*' the agreed choice?

[merr1988]

[Optional] pulledFileTarget accepts any .overview.md under <model>/, while push reads only <model>.overview.md and <model>/{entities,metrics,relationships}/*.overview.md. Matching that set would keep pull from writing a sidecar push then ignores.

[feixufeixu]

Done in c025863: with sidecars on, pull now accepts only <model>.overview.md and <model>/{entities,metrics,relationships}/<name>.overview.md, the paths push reads, and skips any other sidecar name with the usual warning.

[feixufeixu]

Thanks Miguel. This follows Dmitry's design in the 10/21 mapping exactly: pull derives the notation instead of storing it, and writes '*' when the profile excludes every metric the model declares, and a sorted list otherwise.