Would it be OK to use 3/16" (brake pipe) as the fuel return on my swirl pot setup?

There wouldn't be issues with pressure build up would there?

I know it would be good to have a bit of back pressure so the lift pump isn't flat out all the time, is it an idea to have restrictor in line?
if so what size?

I have a restrictor ( about 2mm IIRC) in the tank return from my swirlpot together with a "Fuel King" filter/regulator just after the LP pump which is set at 2psi.
You do want reasonable flow back to the tank to help prevent temperature build up. I put a small copper coil cooler in the line returning from the fuel rail to the pot. Every little helps.

Your 3/16 pipe might be restriction enough. The sound of your LP pump should give you a clue. You could measure the flow before and after the 3/16 pipe.

Was it a coil you made or a proper cooler/coil?
Have to got any images of it?
Is it in cool air flow?