This is what the Farmer's Almanac has to say:
Not at all -- and with the price of commercially grown saffron (about $70 an ounce), it may be the preferred method of getting your hands on this spice. Saffron is the dried, bright red stigmas of the flower Crocus sativus, which is a relatively easy-to-grow perennial. This crocus grows well in Zones 6 through 9. It lies dormant all summer, then pushes its purple blossoms up through the mulch just as other plants are succumbing to frost. Each blossom offers up to three scarlet stigmas. Plant the bulbs in summer and harvest the stigmas in fall. A starter supply of about 50 bulbs costs about $30 and will produce about a tablespoon of the spice the first year. However, each year more flowers will grow, and therefore you'll get more of the spice. Ultimately, your investment will pay off. Fresh saffron threads can be used immediately for cooking, or they can be dried and stored. To dry them, place on paper towels and leave for several days in a warm place. Then transfer them to an airtight container and keep in a cool, dry place. Saffron is a staple in the diets of Mennonite farmers in places such as Lancaster County, Pennsylvania, where gardeners have been growing it for centuries.