Android updates for stock HTC One and S4

Usually on any Nexus device, as soon as the new version of Android is pushed, it is available for update.

With the two new devices HTC One and S4, Hugo Barra specifically mentioned that they came with "Nexus user experience" - of course with some of the hardware in these devices, 10% would be different like Boomsound and ultrapixed for HTC. So these are NOT really Nexus devices.

So when 4.3 comes along, all Nexus devices will get the update immediately. But I fear these two phones will not get an update from Google directly. Users will have to wait for the HTC and Sammy to update them. Fortunately there is no carrier approval required so that might make it faster. But still there is a wait and it could vary from company to company again. So users will be a hostage to HTC or Samsungs priorities?

Are my fears completely unbased?