Recognize SIGTERM as well as SIGINT signal for stratisd shutdown

[mulkieran]

stratisd responds to SIGINT, but ignore SIGTERM, inviting SIGKILL. I believe it would be sensible for it to shut down in an orderly fashion with both SIGINT and SIGTERM. I believe that there may be some situations, e.g., container environments, where the systemd configuration which specifies SIGINT as the kill signal will be ignored, leading stratisd to fail to be shut down cleanly.

EDIT: ^orderly fashion with both^orderly fashion with both SIGINT and SIGTERM.

[jbaublitz]

@mulkieran One of the major problems with trying to handle these signals is that if there is something hanging, catching the signal and attempting to shut down gracefully prevents the daemon from exiting at all. I'm not sure about SIGTERM, but SIGKILL usually is a last resort and we likely shouldn't do anything that would compromise its ability to shut down stratisd, gracefully or otherwise.

[mulkieran]

@mulkieran One of the major problems with trying to handle these signals is that if there is something hanging, catching the signal and attempting to shut down gracefully prevents the daemon from exiting at all. I'm not sure about SIGTERM, but SIGKILL usually is a last resort and we likely shouldn't do anything that would compromise its ability to shut down stratisd, gracefully or otherwise.

@jbaublitz SIGKILL is a direction to the OS and it manages the process by just removing it between one instruction and the next. What I'm saying is that if there is a situation where an, e.g., container, does not read the systemd service file and find out that stratisd responds to SIGINT, it may instead try SIGTERM, which is a more standard stop signal and if SIGTERM fails then it may fall back on SIGKILL. It is undesirable to encourage use of SIGKILL on stratisd, so better to behave nicely on SIGTERM. The only other entity that uses SIGKILL for a systemd kill signal on my systems right now is udisks2, and that is probably because it started out as a gnome utility before it was adopted by storage.

On further inspection it looks like it would fall back to the OS default when the SIGTERM signal is ignored, which is kill. So definitely better that stratisd handles it by exiting gracefully on SIGTERM. I can't at this time see any downside to doing that.

[jbaublitz]

I was a little bit fuzzy on the details. It's been a while since I've worked with signal handlers. Since we don't have to worry about SIGKILL ceasing to work, I'm not opposed to the change, but I do see this as very low priority. We already define the exit signal in the systemd unit file as SIGINT and the only justification I can see for doing this is the container case you mentioned if systemd is not available. pkill by default sends SIGTERM, I believe, so this would only be relevant in that particular case which is a very specific scenario.

[mulkieran]

I was a little bit fuzzy on the details. It's been a while since I've worked with signal handlers. Since we don't have to worry about SIGKILL ceasing to work, I'm not opposed to the change, but I do see this as very low priority. We already define the exit signal in the systemd unit file as SIGINT and the only justification I can see for doing this is the container case you mentioned if systemd is not available. pkill by default sends SIGTERM, I believe, so this would only be relevant in that particular case which is a very specific scenario.

It's a bit more worrisome. The OS default, unless stratisd overrides, which it doesn't a present, is that, if it does not respond in a timely fashion to SIGTERM, it is just killed by the OS. So if someone w/ root privileges calls pkill (sending SIGTERM), stratisd is abruptly terminated by the OS. This is clear in the journal, stratisd never gets to log its shutdown actions, but is just abruptly disappeared.

[mulkieran]

I think sooner rather than later is best.

[mulkieran]

when stratisd initializes its devicemapper functionality this has the effect of installing a DM-specific handler as the only handler for stratisd. stratisd can delay adding its own handler to the last possible moment and clobber the DM handler, but we need to believe that the DM handler is also doing an important thing. We should plan to use the signal-hook or similar Rust crate to manage this better.

[mulkieran]

First step is to find out what precisely is clobbering our signal handler.