Speedo cable with Columbia overdrive

[Kube]

1stford,
I have attached a scan of the 1940 Ford Columbia instructions. Although 1940 Ford, it is very similar to your '41 Merc installation.
Note also the speedometer changer is always engaged whether in OD or not. Think of it as a small transmission. There are gears in there
While I suppose it's possible the issue stems from that changer, it is highly unlikely. The gears are fiber and when on rare occasion they jamb up, the tiny teeth sheer.
I might suggest you remove both speedometer cable assemblies, remove the cable from each, wipe it down with mineral spirits, coat with a very light oil and reassemble.
then you can be reasonably sure the cables are not the issue.
You can, if you like, bypass the changer temporarily and route the lower cable assembly directly to your speedometer. If it works well at that time, the issue rests in the changer and / or the upper cable.

The speedometer drive gear is what many call the "turtle". It is the piece on the torque tube that the lower cable assembly attaches to.
That drive gear assembly can get gummed up. Easy enough to remove it, clean it out very well and reinstall. That gear within the housing should turn freely with your fingertip.