Ellen Herndon was born on August 30, 1837, in Culpepper Court House, Virginia. Her father, William Lewis Herndon, was in the navy and died in a hurricane when she was twenty. Nell, as she was called, met Arthur when she was on the way to visit her mother, Frances Hansbrough Herndon, after her father's death. On October 25, 1859, the two were married. Nell was a talented singer who performed around New York City where they lived. Together, they had two children live to maturity: Chester Alan Jr. and Ellen Herndon.

In 1880, Nell caught pneumonia after waiting outside in the cold. She drifted into a coma and died two days later on January 10, 1880. Arthur was away when she got sick and wasn't able to speak to her before her death, which greatly troubled him for the rest of his life. Arthur's sister Mary McElroy acted as his unofficial first lady during his term in office.