Because they're cheap, comfortable and don't need to be provided in so many sizes.
In my experience of British prisons, you wear what the prison provides in Categories A and B (the most high security ones). But in low security Category C, it is normal to be allowed to wear your own clothes unless you've been reduced to Basic level for bad behaviour. And whaddya know - everyone chooses to wear pretty much the same as prison kit. It's comfy. And it's not a good idea to advertise you can afford expensive clothes.
And, I would remind you, prisons are full of thieves for some reason, and stuff tends to go missing in the laundry. When I was allowed my own clothes, I took to buying washing powder and washing my own underpants and socks in the cell washbasin as they had a habit of going missing.
And I never saw it as a privilege because your own clothes have to be sent in from outside. If the prison provides all you need, life is much easier. I next moved to a prison without this "privilege", I think because some berk constructed it without a laundry, and that removed the worry of what's happening to my clothes. Obviously they had to get the laundry done somewhere else and couldn't be sure of getting it back the same day. Much easier for them to tell us to send our kit back in the morning and supply clean kit in the afternoon from the Clothing Exchange Store.