let's see the odds
in about 3 decades, this problem faced me: zero times
however, when we go to makkah, we go with very cheap sleepers, not fearing of being stolen, but that you can easily mix up your shoes. in makkah, at a normal day, you might find 100s of thousands of people. so it is so easy to forget where you put your shoe or someone else take it by mistake.