Ashley Hemmings had fired the Pilgrims in front from the penalty spot in the fourth minute but Harrogate were level two minutes later despite George Willis saving Joe Leesley's spot-kick, with the same player netting the rebound.
George Thomson handed Harrogate a half-time lead with a fine solo effort and Jordan Thewlis sealed the win two minutes short of the hour mark.