What happens with my delegation if the validator goes to the unbonding/unbonded status?

You can always unbond your delegation from a validator. The only difference is that when the validator is in the active or unbonding stage, you have to wait for the unbonding period to return the delegation back to your balance; whereas if the validator is unbonded, your delegation will be transferred to your balance immediately.