Jamie Gaukroger's long-range effort handed the Pilgrims a 49th minute lead but Stamford equalised a minute later and won it with a quarter-of-an-hour to play.
United twice rattled the crossbar and both Taylor Gray and Gaukroger were both denied by last-ditch goalline clearances.