He has something to hide, believe me. I have been on both sides of the fence.
For a few years I was the 'hider' in my marriage. It started out innocent enough, but as he became overly jealous of my online activities, it became easier to just hide it. Now, it did end up in my case leading to infidelity and I asked for a divorce.
I am now in a very good, true relationship and it's still hard to break old habits. My boyfriend knows EVERYTHING about my past, I didn't want any secrets. Therefore, we exchanged passwords to each and every account we have. I have to give up my 'secret' emails and we blocked addresses and senders of people, links we no longer wanted to hear from.
You are not being paranoid, you are following your gut... and continue to do so. You need to have a serious face to face, heart to heart with him. You may hear things you might not like, but being honest is key. If you can't get over and forgive if thats the case, honey, I say move on. You deserve someone that you can trust and someone that respects and trusts you...