GMC good medical practice states that patient safety may be affected if we are not able to provide care, but beyond our contracted hours we also have a duty to ensure we have enough rest to avoid a negative impact on our judgment and performance, a duty to our family or those who depend on us, and a duty to be trustworthy when it comes to commitments outside of work.
And your employer has a responsibility to you, to make sure the system as a whole functions to avoid problems- the decision to fill a rota gap should ideally be a rational one, not a moral one, and when others are not fulfilling their responsibilities you can certainly refuse to feel guilty about not seeing it as a moral responsibility for filling a rota gap.