feat(input-mapping): AJDA-2714 read description from table definition - #533
Conversation
|
@kudj Chystaná změna input manifestu na komponentě - kvůli přidání description, současně přidává i schema v co nejpodobnějším tvaru jako je na output mappingu. |
zajca
left a comment
zajca
left a comment
There was a problem hiding this comment.
Přijde mě to cajk nenapadá mě kde by to mohlo zhučat, necháme sa překvapit.
|
LGTM ദ്ദി(•ᴗ•) |
keboola-pr-reviewer-bot
left a comment
keboola-pr-reviewer-bot
left a comment
There was a problem hiding this comment.
Verdict: needs_human (risk 3/5) · profile ajda
This PR makes an ungated, observable behavior change to platform-libraries (input-mapping), which AJDA policy treats as load-bearing.
Concerns:
libs/input-mapping/src/Helper/ManifestCreator.php: Ungated observable change in platform-libraries: every manifest now serializesschema, visible to all consumers.libs/input-mapping/src/Configuration/Table/Manifest.php: Newschemaanddescriptionnodes added to manifest config with no feature-flag gate.
|
Docs PR keboola/developers-docs#400 |
| ->prototype('array') | ||
| ->children() | ||
| ->scalarNode('name')->isRequired()->end() | ||
| ->variableNode('data_type')->end() |
There was a problem hiding this comment.
na OM to je trochu striktnější ten data_type kde validujeme i jeho obsah
There was a problem hiding this comment.
nj, to je pravda - ale to bych tu musel zavést, že existuje snowflake a bigquery a na to mě ta validace nepřijede natolik důležitá (jakože tady se ten manifest jen produkuje) kvůli tomu zanášet takovou závislost
5 participants
Link to issue
https://linear.app/keboola/issue/AJDA-2714/input-mapping-read-description-primarily-from-table-definition
Description
Input Mapping now exposes the table/column
descriptionfrom the native first-class definition field in the table manifest, and adds a new first-classschemanode mirroring the output-mapping schema shape.ManifestCreator::writeTableManifestadds two native-sourced fields (purely additive — the legacycolumns/column_metadata/metadatastructures are left untouched):description(top-level): taken fromdefinition.description; omitted when absent/empty.schema: one entry per selected column withname,data_type(base+ the table's actual backend type, includinglength/defaultwhen present),nullable,primary_key, and the nativedescription. Emitted for both typed and non-typed tables (non-typed columns simply omitdata_type/nullable); a table with no definition serializes an emptyschema: [].Key decisions:
KBC.descriptionmetadata is left intact and Connection backfills the native description into it, so consumers without native support keep working. This also avoids having to pick a winner among multiple multi-providerKBC.descriptionrecords.schema[].metadata— it would require collapsing multiple metadata records into a single map (lossy/ambiguous). Raw records stay in the untouchedcolumn_metadata.data_typereflects the table's actual backend only (base+ that one backend), matching the real state of the table.Justification
Native
descriptionis now a first-class field on table/column definitions in Storage. See the linked Linear issue.Plans for Customer Communication
None.
Impact Analysis
Additive change to the table manifest only (new
descriptionandschemakeys; every manifest now also carriesschema, empty when there is no definition). No existing manifest fields change, so current consumers are unaffected. Not gated by a feature flag.Deployment Plan
Release a tagged version.
Rollback Plan
Revert this PR.
Post-Release Support Plan
None.