When doing a speed test however, speed was fine.
The issue was MTU.
To fix this, on my VLAN that I assigned to my clients I needed to run the following command to stop the packets fragmenting:
ip tcp adjust-mss 1452
Make sure you put this on the VLAN itself, not the Dialer Interface.