Goals by Thomas Orr and Adam Brown were enough to get Warriors over the line despite a late Annan onslaught which saw Robert McCartney reduce the deficit and ensure a nervy last few minutes.
"I think we've played a lot better and lost games, but the most important things is the win,” Swift told Warriors TV."We knew there would be an onslaught at the end with them throwing balls into the box.
"We would have liked to have dealt with it better because they got a goal with five minutes to go which makes it really nervy.
"If you look at our goals to go 2-0 up it was a scramble in the box and Adam got a deflection so it was two ugly goals in an ugly game - bit I'll take it. I'm delighted to get three points."
Swift admitted that Stenny allowed Annan to push them back after Brown put them two up, but says that was understandable given the pressure to break their league duck.
"Because they've not got that first win off their back, they're a wee bit nervy and drop deeper than they normally would.
"They don't manage the game as they usually would so we need to get better at that. Confidence brings that, winning games of football brings that so I'm pretty sure this will do us the world of good.
"We've played really well in games and come away with nothing so to actually not play so well and come away with three points is really encouraging.”
Swift was quick to praise young keeper David Wilson, who kept his place after his clean sheet in the SPFL Trust Trophy at Cove.
Swift said: “I thought he was excellent again, especially when you see that onslaught at the end."He's only a 20-year-old goalkeeper but he really showed a presence and came and punched a couple which I thought was terrific and shows he is the real deal.”
The win moved Stenhousemuir off the foot of the cinch League 2 table – next up on Saturday is a trip to Stranraer, one of two sides now below them.