Question 1

Is this the same as a dead man's switch?

Accepted Answer

It is exactly that, applied to cron job monitoring. Everything else on this site goes and looks at something. This one waits to be told, and alerts on the silence, which is the only way to watch a job that has no address to check and no page to read.

Question 2

How late can my job be before you alert?

Accepted Answer

As late as you allow, in minutes. A job that runs every hour and takes anywhere from one to twenty minutes can be allowed thirty, so a call is expected between sixty and ninety minutes after the last one. Left blank it is a tenth of the period it runs on, and never less than a minute: two minutes late means nothing to a nightly backup and everything to a job that runs every five. The floor is there because a cron job firing a few seconds either side of the minute is normal and must never open an incident.

Question 3

How do I call it from a cron job or a shell script?

Accepted Answer

Fetch the URL at the end of the job, with curl or wget or anything else that speaks HTTP. GET, POST and HEAD are all accepted, because half the world reaches for one and half for another, and the answer is an empty 204 so that a nightly job does not mail a page of HTML to the operator.

Question 4

Should I call it at the start or at the end?

Accepted Answer

At the end, after the work succeeded and not before it starts. A call at the top of the script says the machine woke up, which is rarely the thing that goes wrong. Put it on the last line, behind whatever exit code check the script already has, and the silence then means the job failed rather than that the host is down.

Question 5

What if the URL leaks?

Accepted Answer

The URL is the credential, which is why it is 256 bits of randomness and why it is shown to you once. Anybody who has it can report your job as healthy, which is the failure worth worrying about rather than anybody reading anything. Regenerate it and the old one stops working immediately.

Question 6

Can a job that runs weekly use this?

Accepted Answer

Yes, and it is one of the better reasons to. This interval goes out to a month where the other check types stop at a day, because a weekly backup is an ordinary thing to own and a backup that quietly stopped running three weeks ago looks identical to one that ran perfectly, right up to the day somebody needs it.

Question 7

Why not have the job email me when it fails?

Accepted Answer

Because the failure you need to hear about is the one where the job does not run at all, and a job that did not run sends no email. That covers a crash inside the script, a machine that rebooted, a cron entry somebody commented out, a container that never started and a queue that stopped being drained. Catching errors in the job is still worth doing; it is a different failure from silence.

Cronとバックグラウンドジョブのモニタリング

用途

設定できる内容

質問

これはデッドマンズスイッチと同じですか？

アラートが送られるまで、ジョブはどれくらい遅れても大丈夫ですか？

Cronジョブやシェルスクリプトからどのように呼び出しますか？

開始時と終了時のどちらで呼び出すべきですか？

URLが漏洩した場合はどうなりますか？

週次で実行されるジョブでも使用できますか？

ジョブが失敗したときにメールを送るようにしないのはなぜですか？

他のチェック