United trailed 2-0 and then 3-1, courtesy of two goals from Phil Marsh and a spectacular free-kick from Kristian Platt, but the Pilgrims were undeterred.
Netan Sansara bagged his first goal for the club while Ben Fairclough and Marc Newsham struck crucial second half goals to haul United level.
The result was arguably fair on the balance of play, but both sides had their chances to take all three points, with United keeper Lewis King in imperious form once again.
His opposite number Arron Jameson, on the other hand, had a complete nightmare and played a key role in two of United's goals.
Sansara had been deployed on the left-hand side of a United front three and headed the game's first chance over the bar following Ian Ross' lofted cross.
Sean McConville blasted wide from 20 yards as Celtic carved out their first chance and the hosts were soon in front.
MARSH rammed the ball home at the third attempt after Liam Marrs' initial clearance off the line and King's follow-up save following two McConville efforts.
King then made a flying save to keep out McConville's angled effort before United's afternoon took two further turns for the worse.
Ricky Miller limped off with a twisted ankle and from a hotly-disputed free-kick 30 yards from goal, where Newsham appeared to be fouled himself, PLATT stepped up to crash a stupendous set-piece high to King's right.
United were stunned and Carl Piergianni glanced just wide following Ross' corner, but the Pilgrims were soon back in contention.
Newsham's drive was spilled by Jameson and SANSARA walked the loose ball home.
Jameson booted the ball against Sansara moments later, but it bounced to safety as the Celtic keeper's afternoon continued to unravel.
Jameson did make a tremendous save to deny Sansara following Ross' free-kick but United were soon dealt a further blow.
King did brilliantly to deny Andy McWilliams and Rene Steer did equally as well to deny McConville a tap-in on the follow-up. The United wing-back received a gash to the face in the process and was unable to continue.
Fairclough was Steer's replacement from the bench and Sansara reverted to a more familiar role on the left-hand side.
Jameson held on to efforts from Fairclough and fellow sub Spencer Weir-Daley at the start of the second half, but he fumbled Ross' free-kick which allowed Sansara to blaze an effort into the side-netting.
The game appeared to be up for United on 66 minutes when Celtic broke rapidly and MARSH showed plenty of composure to jink around Scott Garner and keeper King before rolling the ball into an unguarded net.
Two more minutes had elapsed when FAIRCLOUGH blasted United firmly back into contention with a rising, left-footed 25-yarder nestling into the top corner leaving Jameson powerless.
King made a crucial save to deny McConville as Celtic threatened to kill the contest once and for all and United took full advantage.
Ross' dipping drive was spilled by Jameson and NEWSHAM applied a trademark predatory finish to draw United level.
Conor Marshall volleyed over the the bar following Marrs' knockdown before Piergianni had a header blocked on the line by Tom Buckley, who motored quickly to exchange passes with Marsh in a lightning quick break only to be penalised for offside as he rolled the ball past King.
Stalybridge (4-3-3): Arron Jameson; Kelvin Bleau, James Tunnicliffe, Kristian Platt, Andy McWilliams; Prince Haywood (Keith Briggs 59), Callum Warburton, Adam Pepper; Sean McConville, Phil Marsh, Paul Ennis (Tom Buckley 79). Subs (not used): Obi Anoruo, Conal Platt, Craig Ellison (gk).
Boston (3-4-3): Lewis King; Stefan Galinski (Jake Hall 78), Scott Garner, Carl Piergianni; Liam Marrs, Ian Ross, Conor Marshall, Rene Steer (Ben Fairclough 40); Marc Newsham, Ricky Miller (Spencer Weir-Daley 13), Netan Sansara. Subs (not used): Ben Milnes, Sam Vince (gk).
Referee: Steven Copeland
Attendance: 354