One of you needs to file SINGLE, the other HEAD of HOUSEHOLD.
Which one? The one who supports the household expenses more than 50% is head of household. This is usually, but not necessarily, the one who makes the most money, assuming he or she can claim at least one child, see below.
The natural parents get priority on the children claimed; if you both are the natural parents of the children, work it out between you.
The itemized deductions are usually the one who pays them. But if your finances are comingled, money is fungible (love that word) so whoever it benefits, probably the one who gets paid the most.
The thing about "common law" marriage which someone suggests, it is only in a few states, and you have to present yourself as if you were married. The question's wording indicates that you don't consider yourselves married, with is a prerequisite for common law marriage.
Hope this helps.