Madame has a point: in Japan, the animals are more likely to pose a threat than the people.
Then again, nature has one up on even the animals. If a massive quake hits and you're in the wrong place at the wrong time, your number is up.
However, assuming these pretty unlikely things don't happen, then in terms of actual crime, there are few safer places to be assuming you employ common sense. During the daytime while it's light, you could carry a Leica M with a Noctilux on it around central Tokyo and the chances of someone trying to take if off you are remote...and were it to happen, I'd bet my money on it being a non-Japanese. Even at night the chances of being attacked for your possessions are, when compared to most western countries (and many other Asian ones) slim.
However, if you are concerned about it, consider taking out an insurance policy on your camera in the event of theft.