Seen on reddit for anyone having performance issues with an Nvidia card to change the power management settings.
GLideN64 runs like absolute rubbish on Nvidia GPUs. Initially, I just put it down to bad coding. But while the plugin is indeed hardware intensive, the culprit is actually Nvidia's drivers.
More precisely, Nvidia's power management.
To fix the performance issues, add your N64 emulator's EXE to Nvidia's 3D settings and set Power management mode to "Prefer Maximum Performance". You could alternatively set this globally, but that's your decision.
This is a problem that affects Dolphin, too. Or at least it did last time I checked.