Offer a way to free resources in uninterruptible way by default and opt-out ability to interrupt

[swamp-agr]

Follow-up to #34.

Since 0.5.0.0 the resource pool has changed its behaviour: resource-freeing function runs under regular interruptible mask_ instead of uninterruptibleMask_ as before. That leads to situations where asynchronous exceptions could interrupt free in the middle of a blocking cleanup (e.g. closing a handle/network socket) leading to resource never being released and the actual leakage.

We have tried to perform an upgrade of one of the resource-pool consumer packages and stumbled upon a few leakage leading to following situations:

• Hedis: Issue 1 with its fix.
• Hedis: Issue 2 with its fix.
• persistent: Issue, its fix, driver's fork fix.

See minimal reproducible cases attached to those issues (they are all similar).

Proposed solution would be to change free in such a way to pass releasing function as argument, that way consumer or end user would be reminded to handle resource destruction explicitly. Or extend pool configuration with a flag that indicates how to handle pool resource destroying. What do you think?

[arybczak]

While it's unfortunate that this caused issues in your prod environment (and I'm sure it wasn't fun to debug), ultimately it's client's job to ensure that resource creation and destruction is asynchronous exception safe. If a library has a latent bug like this, it's enough to use a bracket instead of this library to trigger the bug.

Hedis: informatikr/hedis#275 with its informatikr/hedis#276.

As I wrote in the comment there, this is unrelated to this change, it was always a problem because this library never ran resource creation within uninterruptibleMask.

I think the best we can do is to communicate loudly in a documentation that resource creation and destruction need to be asynchronous exception safe.

[arybczak]

extend pool configuration with a flag that indicates how to handle pool resource destroying

This doesn't help, because either

1. you set up pool internally and you might as well fix your functions so they work everywhere, instead only with pooling, or
2. you use the pattern from README and then the user controls whether resources leak or not, or you take the resulting pool and override this setting, at which point you can also fix your functions like in (1).