Chromecast work with xbox one?

Does the chrome cast work with the HDMI in port on the xbox one. Im not sure whether it works like switching the source on TV.

For example if I say "Xbox, watch TV" it just switches to whatever I have plugged into the HDMI in on the console, be it a chromecast, a 360 or a ps4.

Does it work like this, or is it set to work with only cable boxes?

It would be great to hear from someone that has tried this, or someone that is willing to.