The Pilgrims struggled to find their rhythm against the Brakes, who took all three points courtesy of Ahmed Obeng's 53rd minute strike.
Both teams finished with 10 men, with Reece Thompson and Joe Magunda dismissed in separate incidents.
MATCH REPORT