Repository navigation
Feature/add rows updated and deleted to audit table - #202
Merged
fivetran-savage merged 5 commits intoOct 8, 2026
Merged
fivetran-savage merged 5 commits into
fivetran-savage merged 5 commits into
Conversation
fivetran-catfritz
requested changes
Oct 1, 2026
fivetran-catfritz
left a comment
fivetran-catfritz
left a comment
Contributor
There was a problem hiding this comment.
Left a few comments here and in the ticket!
Comment on lines
+16
to
+18
| max(case when event_subtype = 'sync_end' | ||
| then cast({{ fivetran_log.fivetran_log_json_parse(string='message_data', string_path=['status']) }} as {{ dbt.type_string() }}) | ||
| else null end) as sync_status |
Contributor
There was a problem hiding this comment.
max(status) picks alphabetically (my findings in the ticket).
Suggested fix: add a rank to log first (line 3 above):
then pick the latest sync_end here:
Suggested change
| max(case when event_subtype = 'sync_end' | |
| then cast({{ fivetran_log.fivetran_log_json_parse(string='message_data', string_path=['status']) }} as {{ dbt.type_string() }}) | |
| else null end) as sync_status | |
| max(case when event_subtype = 'sync_end' and nth_event_subtype_record = 1 | |
| then cast({{ fivetran_log.fivetran_log_json_parse(string='message_data', string_path=['status']) }} as {{ dbt.type_string() }}) | |
| else null end) as sync_status |
| table_name, | ||
| schema_name, | ||
| coalesce(write_to_table_start, backup_write_to_table_start) as write_to_table_start, | ||
| case when event_subtype = 'records_extracted' |
Contributor
There was a problem hiding this comment.
See the ticket for the query, but records_extracted's extra next_write_to_table_start fallback can resolve to a different write-cycle timestamp than records_modified, splitting rows_extracted/rows_loaded across two rows instead of merging.
| @@ -4,22 +4,59 @@ with log as ( | |||
Contributor
There was a problem hiding this comment.
Explanation a few rows down.
Suggested change
| select | |
| *, | |
| row_number() over (partition by sync_id, event_subtype order by created_at desc) as nth_event_subtype_record |
fivetran-catfritz
approved these changes
Oct 8, 2026
fivetran-catfritz
left a comment
fivetran-catfritz
left a comment
Contributor
There was a problem hiding this comment.
looks great! approved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
rows_loadedper table tofivetran_platform__audit_tabletotal_rows_extractedper connection tofivetran_platform__sync_metrics