Create/cancel an alert