The West Lothian outfit scored an 84th-minute penalty and then a 97th-minute winner right at the death to take all three points, just days after beating Athletic 1-0 at home in the first part of a top-of-the-table double-header.
Athletic midfielder Jonny Mitchell was sent off early on in the second half, but it was the hosts that went ahead at Bo’ness United’s Newtown Park through a Tiwi Daramola strike.
Athletic boss Willie Irvine said the weekend’s defeat hasn’t disheartened him.
“It’s by no means over,” he said. “It was a disappointing match to lose because even when we went down to ten men, we had most of the play and we went ahead.
“A red card made it harder, but we still did things well in the main. The way we conceded the goals were the annoying things for me.
“We lose a stupid penalty to give away and they score from it, and it was a foul we didn’t need to make. The guy was going away from goal and it was out of nothing.
“Then we lose a soft goal right at the death. We win the throw-in and hit it off our own guy and it goes out for a corner for them. It was just one of those things really.
“They were frightened of our pace up top, and you could see that because they sat back for most of it and they didn’t bombard us with chances even with an extra man.
“Nothing is finished yet. Whitburn have to go Armadale on Saturday.”