I really think that you two should get together and do some serious talk with each other. If you want to get back with your husband then you need to tell him. It seems like he still loves you and cares for you, but maybe he hurt too, to even face you to let you know that he was wrong himself. Maybe it is easy for him to come home, he is only making it harder than what it really is.
Don't let the little arguments tear both you and him apart. There's still something there. So, if you are totally in love with him, call him and arrange to see him. Apologize to him, he may apologize to you too. Do what you can to get him back and do what you can to repair the marriage. Your daughter needs the both of you. Don't let your daughter suffer because of what you two have done. Kids have feelings too.
Also, consider reading, "The Proper Care and Feeding of Marriage" and "The Proper Care and Feeding of Husbands".
Btw, how long have you two been married?