Read through your speech a couple of times beforehand. Try to read it out loud at least once. Go away from other people while you do this. When it is your turn, put your notes on the lectern, look at someone and deliver the opening line straight to them. You can then read your speech but as you know much of it, you can look up now and then and when you do, look straight at someone and say the words to them. Pick people you know well or complete strangers, it doesn't matter. When you look at one person in the group, everyone gets the sense that you are looking at them.
Take your time. If you get emotional, pause, take a deep breath and try to carry on. Remember your brother has died. You knew him as no-one else did. Your speech will carry that sense of your relationship with him that is special to you. People will listen and appreciate what you are saying even if you read the entire speech without looking up but looking up occasionally and telling them directly will give your message clearly.