The version of Python 3 installed via Xcode or Command Line Tools (CLT) can probably not be removed specifically, unless you delete Xcode or the CLT as a whole. However, you can install Python 3 yourself and it can coexist with Apple’s version. For example, the installer package from
python.org will install the binaries or symlinks thereof in /usr/local/bin. That location is already in $PATH.