Deserialization must fail if bytes remain after the last entry - #519
SavicStefan wants to merge 2 commits into
Conversation
Signed-off-by: Stefan Savić <stefan.savic@databricks.com>
Signed-off-by: Stefan Savić <stefan.savic@databricks.com>
proost
left a comment
There was a problem hiding this comment.
Technically, you are right. But it can change implicit contract in the API.
but like serialized format does not identify the summary type, so this is needed to detect that mismatch, wdyt? |
|
@SavicStefan And also some users will be relying on it, This change would break them. |
|
I agree with @proost on this. We can't assume that a given MemorySegment is dedicated to a sketch / Summary and nothing else. It is really up to the sketch and the user's code to figure out how many bytes to read. This is true of all our sketches that leverage MemorySegments. |
Require compact tuple sketch deserialization to consume the entire input buffer.
Adds a test for deserializing a wider summary as a narrower summary.