If DoH/DoT is configured on the Router, is it also needed to configure DoT/DoH in the Browser or OS?
No, but check if the browser isn’t using other DoH provider. If so just disable it.
Note that it’s obviously a different story for mobile devices. If you connect to different networks, you might want to leave DoH on on your device
Makes sense, is their any downside if I set the same DNS in Android and Router?
No - in either case a https connection will be established and DNS will be available via that connection. The only exception is the very first connection of the browser/system, where the domain of the DoH provider needs to be resolved first (e.g. dns.quad9.net -> 9.9.9.9).
you can check via this website
Depends on your security model IMHO. If unencrypted dns traffic on your network, or your router being a possible aggretion point for dns requests from devices on your network is fine, then it is a great way to simplify using it for your network.
I imagine it’s probably good for 98% of people.