Many collectors like to add a first day cover of their stamps to their collection. Such an item is known as collateral material - something that amplifies the meaning of the stamp while adding interest to the collection.
While there is a sameness to the commercially available first day cachets for newly released stamps, there are also more limited privately printed or artist cachets that can add a bit more flair and uniqueness to your collection.
There are some first day covers that actually tell a story, like the censored Linn cacheted covers for the U.S. National Parks series of the Thirties that had a representation of the stamp in the cachet, which the USPOD felt the need to censor. Such an item can add another dimension to a set of Parks stamps, all of which appear the same from album to album.
First day covers have their own built-in collateral material to the stamp that franks as the cachet further explains why the stamp is important enough to exist. First day covers aren't the only items the collector may add to his stamps to further tell their stories. Other items are:
Postal History Covers - Depending on how involved you want to get in telling the story of what your stamps actually accomplished when they were placed on envelopes to travel through the mails, add these to show the routes that your stamps have traveled, issue dates, and postal rates. These concerns are the backbone of the postal history of your stamps and a few pieces of fine postal history placed within your collection can add real interest, while not busting your stamp budget as a collection full of postal history itself might do.
Souvenir Cards -- Issued by the Bureau of Engraving and Printing, souvenir cards often feature essays of classic stamps tied into an anniversary or an important philatelic event. Their attractive engraved vignettes, often of a much earlier time, also add a touch of class.
Meters -- Especially for topical exhibitors, there are many great pictorial meters. The wisdom today is "don't cut your cover up." Of course if there is nothing on the cover of any value to your collection or exhibit, you might want to just save the "2 x 4" the term for the cut rectangle that shows the meter stamp.
Seals and Labels -- There have always been interesting non-stamp labels attached to covers. Topics are as varied as flower shows, air meets, stamp shows, town anniversaries, any number of historical events, and on and on. By all means, exhibit the entire cover that contains such a label, as the cancellation and perhaps other elements will all be tied into the same subject.
FDC Programs -- First day cover programs not only give your stamp a sense of time, but are also one-stop shops for the technical details of your stamp. Also included is information about dignitaries and those of importance in the philatelic world.
Original stamp art - If your budget can bare it, you might pick up the original artist piece that the stamp was based on. But expect to pay big money for such a thing.
Proofs and essays - Still surprisingly affordable, essays are impressions of a rejected stamp design; proofs, the final design printed before the stamps go to production. So-called trial color proofs are favored by some collectors who wish to add variety to their collections. Classic U.S. stamp collecting is rich in proofs and essays and a specific group grew to accommodate collectors interested in these items: The Essay Proof Society, now no more, though the U.S. Philatelic Classics Society has continued in the area.
You must be careful to keep most of these collateral items as a side collection if you intend to exhibit your stamp collection someday. Some of the items are allowed in an exhibit to round out and explain legitimate stamps and covers in your collection, while many are not.
But in the end there is no reason to limit your stamp collection to just stamps. There is a great variety in collecting the stamps of the world. But even greater variety and interest may be found in those philatelic items slightly off the beaten track.