Just after the families returned to Boston from their 1906 vacation together, Joe invited Rose to a school dance, but her father refused to allow her to accompany him. John Fitzgerald believed that the son of a tavern owner was not good enough to marry his favorite daughter. Rose and Joe were determined to see each other despite her father's misgivings, and set out on a secret romance. They met at dances, parties, and the city library. John Fitzgerald soon became aware of the romance and intended to put a stop to it, but all of his efforts were unsuccessful.

After graduating from high school, Rose wanted to attend Wellesley, but her father, heavily influenced by Boston's Cardinal O'Connell, decided that a Catholic education was more suitable. He enrolled her at the Catholic school, Convent of the Sacred Heart, in Boston. She later attended Sacred Heart in New York and then spent one year at the school in Blumenthal, Prussia, where Fitzgerald was happy to have his daughter far from Joe. At Sacred Heart, Rose developed an intense commitment to the Catholic faith that would endure for the rest of her life.

In the meantime, Joe was determined to become a bigger success than his father and even Fitzgerald. He was accepted into Harvard University in 1908. He lived in Harvard Yard, where his Irish heritage in a privileged Brahmin world was no mean feat. Joe ignored the slurs and the name-calling. He paid more attention to his goal of climbing the social ladder. He attempted without success to join some of the most prestigious clubs and fraternities. After a lot of hard work, he was accepted into the less prestigious but well-regarded Hasty Pudding Dramatic Club.

After graduation, Joe decided to make his move into the banking business where he hoped to rub shoulders with the wealthy and powerful. To learn the trade, Joe, through the help of his father, secured a position as a state bank examiner. He excelled in the position and soon became known in the banking field for his quick and ambitious nature. In 1913, Boston Irish-owned Columbia Trust found itself threatened by a takeover from a rival Boston bank, but Joe managed to secure funding from Merchants National Bank. Columbia was saved, and Joe was rewarded for his effort. At the age of twenty-five, he became the bank president.

The year 1914 was full of ups and downs for the Fitzgeralds. John Fitzgerald ran for mayor of Boston but withdrew from the race when an opponent, James Michael Curley, indicated he would expose Fitzgerald's most recent extramarital affair with a cigarette girl named “Toodles” Ryan. After Fitzgerald's slip from prominence, Joe's status as a banker made him a somewhat more welcome addition to the family, and Fitzgerald finally gave his consent for Joe to marry his daughter. On September 20, 1914, the couple was married by Cardinal O'Connell.