delete removed category

I have been tidying our categories and sub cats up but when i dlete them they don't get removed they just go grey i don't want them left grey as i have moved the category somewhere else.

how do i permemantly remove these categories?