Skip to content

Conversation

@julik
Copy link
Owner

@julik julik commented Apr 18, 2025

In a few places strings in other encodings could slip through.

  • Make sure Writable#<< converts the strings it is given into binary if they are not already in binary. This fixes an issue where Heuristic would suddenly start forwarding strings as-is to downstream callees. There is a lot of spots where the string-to-write gets forwarded and converting in every single one will be quite wasteful, but it can be handy to do in a few key places.

  • Make sure WritableBuffer#<< converts the strings it is given into binary if they are not already in binary. This helps prevent an issue where the receiving object the buffer flushes to is in a different encoding than binary (and all of our use cases assume bytes anyway, except for filenames).

Fixes #23

julik added 3 commits April 18, 2025 19:15
It seems that there are quite a few places where data entering the `Writable` can be, say, in UTF-8. That's not a big deal, but it can lead to problems with appending strings in the WriteBuffer, so let's see what we can do about that.

Fixes #23
@julik julik force-pushed the ensure-writable-binarizes-strings branch from ac3852a to 3b4d6c5 Compare April 18, 2025 17:16
@julik julik merged commit eb177ad into main Apr 18, 2025
3 checks passed
@julik julik deleted the ensure-writable-binarizes-strings branch April 18, 2025 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heuristic sometimes rejects UTF-8 string after deciding stored/deflated mode

2 participants