There Is Always Paperwork
Fill out the birth certificate. If you don't do this before you leave the hospital, your baby may end up labeled “Baby Smith” on this critical document, with the first name merely recorded as an amendment.
Order photographs, even if it's against your better judgment. Your newborn may be a blotchy-faced conehead, but his first picture is one worth having — particularly since hospital photographers seem to be able to catch the babies with their eyes open and their mouths shut. Don't go for the super-sized package, however. This will not be your child's best picture and you don't need a hundred copies of it.
You will also need to order your child's Social Security card from the hospital. Doing it at the hospital is much easier than figuring out where your local Social Security office is and trying to handle documents with a baby carrier and diaper bag in tow. Be forewarned: you'll need this number sooner than you'd think.