When it comes to bit-ness of an application, there's generally
only 2 things you need to worry about:
Am I running the right version for the OS?
Are all the components compatible with the same version?
I don't think I've come across any instances of configuration
settings being applied differently to different versions of the
same application. But PowerShell is different.
When managing environment settings such as Execution Policies in
PowerShell, the settings actually need to be changed for the right
version, and the right version depends on the application calling
the runtime. Shown below are the 2 different versions of the
PowerShell shell which comes with Windows, after running
Get-ExecutionPolicy at the same time:

Annoyingly, if these do not match up, and you run an application
built for 32 bit rather than 64 bit without realising it, you will
get a CmdletInvocationException:

Hope this helps someone.