I live in St. Louis now but grew up in New York City and go back often. One other response mentioned the Cloisters, but I want to emphasize how special it is. It's one of the most romantic, elegant, and serene spots in Manhattan (which is not exactly known for its serenity). Plus, if you go to the Metropolitan Museum of Art and the Cloisters on the same day, you pay once for both museums because the Cloisters is a branch of the Met. See the Cloisters first, though, because it's smaller and the Met is open longer hours.
Take the M4 bus, which starts at Penn Station, shifts to Madison Avenue, and continues way uptown. The M4 takes you practically to the door of the Cloisters, its last stop. When you want to leave for the Met, take the M4 downtown. It ultimately heads to Fifth Avenue and will drop you off at the Met. These are long bus rides, but you'll see a lot of the city.
The Met has places to eat, but you might want to get off the M4 around 92nd Street or so, walk east to Madison Avenue, and visit Jackson Hole (91st and Madison), which has burgers, sandwiches, salads, and the like. The food is reasonably priced, tasty, and filling—and you're going to need your strength and stamina for the Met, which from Jackson Hole is about 10 blocks away on Fifth Avenue.
Have a wonderful honeymoon!
Here are the websites for everything I referenced: www.mta.info (bus and subway), www.metmuseum.org, and www.jacksonholeburgers.com.