It's true that you're from New York and that you're from NJ; I say that as someone who was born in one particular state and raised in another state.
However, you tell me: is Bruce Willis a German? After all, he was born in West Germany though he grew up in South Jersey according to Wikipedia. In other words, is Bruce Willis (for example) a "German" according to a normal or vernacular use of such a word? No, Willis isn't a German, Suzanne Vega isn't a Californian (though born in Santa Monica), and you're not a New Yorker all *in the everyday sense* of the nouns in question. People are (virtually) never called "New Yorkers" unless they either live in New York or they've just moved away from New York.
So refer to yourself with the term "New Yorker" if you want, but be prepared to qualify that description. And I say that as a Californian who has seen what happens when I say to someone "I'm from California" or "I'm a Californian" and the person to whom I'm talking thinks that I'm just visiting the East Coast; they don't understand that I'm an East Coast taxpayer.