The 40-year-old Williams was beaten 7-5, 6-7 (4/7) 6-1 by Tomljanovic in a gruelling 3-hour 5-minute battle.
Williams poured everything into one last stand in front of a ferociously partisan crowd at the Arthur Ashe Stadium.
But the veteran wilted visibly in the decisive third set, wearied by the demands of a heroic second set display that saw her level the match in a tie break.