Skip to content

Make count_with more consistent for builtin types. - #318

Closed
kitlith wants to merge 3 commits into
jam1garner:masterfrom
kitlith:correct_count_with
Closed

Make count_with more consistent for builtin types.#318
kitlith wants to merge 3 commits into
jam1garner:masterfrom
kitlith:correct_count_with

Conversation

@kitlith

@kitlith kitlith commented Feb 22, 2025

Copy link
Copy Markdown
Collaborator

Splitting this off from #317, as there's more solution space to sort through there, and I think fixing this is more important.

While writing this PR description, I thought of a less contrived example and test where this would break: reading 24 bit integers, which we already provide a helper for. count_with(N, read_u24) will currently read N u32s, instead of N u24s, and I have updated the test to reflect this.

The main thing I'm uncertain about here is the last commit, where I modify the API of count to make it callable from Vec::read_options. That may be a semver breakage, I'm not sure.

Before merging, I figure I'll do an interactive rebase to squash all of the commits prefixed with fixup!.

@kitlith

kitlith commented Feb 22, 2025

Copy link
Copy Markdown
Collaborator Author

same minor UI test failure on nightly as before. I don't see a reason why this would be more likely to fail on 1.66 than the other, so I'm not bothering to re-run the cancelled checks.

@jam1garner

Copy link
Copy Markdown
Owner

I think that change is breaking for anyone using turbofish syntax for count at minimum? maybe in practice that's acceptable, unclear

@jam1garner

Copy link
Copy Markdown
Owner

Not sure I 100% understand your meaning about the u24 example. Did it previously read 4 bytes a piece?

@kitlith

kitlith commented Feb 24, 2025

Copy link
Copy Markdown
Collaborator Author

Did it previously read 4 bytes a piece?

Yes, it did, and the resulting "u24" (which we represent as a u32) can have it's high 8 bits set. So it's not a u24 at all, when going through count_with.

edit: if you check out the commit before i fix this, 6b43f69 , you can observe that the new test fails with the equivalent of "unexpected EOF"

@csnover

csnover commented Feb 24, 2025

Copy link
Copy Markdown
Collaborator

The main thing I'm uncertain about here is the last commit, where I modify the API of count to make it callable from Vec::read_options. That may be a semver breakage, I'm not sure.

I think this is fine. I’m not worried about the signature change at least. I thought I had done something similar to this in some uncommitted code change but I can’t find it in any of my stashes, so it must have been an experiment I abandoned.

This change makes sense to me and seems fine if you want to wrap it up. I’ve pushed a commit to master with updated outputs for the nightly UI tests so that will not fail any more on rebase.

@kitlith

kitlith commented Feb 24, 2025

Copy link
Copy Markdown
Collaborator Author

... i squashed one too many commits in the rebase. sec.

Test is written to "what a user would expect", this currently fails.
We should either move the optimization to the count helper, or fix
it in some other way.
count_with can't make any assumptions about the read function it's
been given, unless we find a way to specialize on "read function
is "T::read_options"

This fixes the test introduced in the previous commit.
Note: this changes the signature of binrw::helpers::count,
which may require a semver bump.
csnover pushed a commit to csnover/binrw that referenced this pull request Mar 12, 2025
`count_with` can't make any assumptions about the read function
it's been given, unless we find a way to specialize on "read
function is `T::read_options`". `count` does not expose the read
function, so can still specialize on `T`.

Closes jam1garner#318.
@csnover csnover closed this in 8739b52 Mar 12, 2025
@csnover

csnover commented Mar 12, 2025

Copy link
Copy Markdown
Collaborator

Thanks! I squashed this to a single commit to avoid future bisects to a commit with a known broken test, and since the change is small I didn’t see a lot of value in keeping the other stepwise commit.

Since, as you noted, this changed one of the helper function signatures anyway, I took the time to separately go through and normalise all of the helper functions’ generic parameters so they are consistently ordered, with the ones most likely to need explicit arguments toward at the beginning.

Separately, I adjusted the test for clarity, since I could see myself forgetting why it exists and misinterpreting it as an attempt to verify that read_u24 only reads 3 bytes at a time. Now it explicitly says that it’s making sure fake-specialisation doesn’t ignore the function. It could have been written to be even more explicit by using a custom function that sets a flag when it is called instead of using read_u24, but I didn’t feel like working that hard :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants