In this case, it is only used in the singular form: there is no plural version of it.
Here 'relief' refers to a feeling of comfort when something frightening or worrying is about to happen but then doesn't. Perhaps you thought you had a serious illness but then found out you didn't - what a relief!
The same thing happens with the normally uncountable nouns 'pity' and 'shame'.
What a pity!
What a shame!
Listen and find out more about the word 'relief'.