Mac Pro Pricing - Am I missing something?

I'll be quick.

I've seen some uproar regarding the Mac Pro's pricing - some people calling it another overpriced Apple product. But compared to its competitors, it actually seems quite reasonably priced! I'll give an example:

The 6-core, 16GB model is listed on Apple's website as setting you back US$3,999 with dual D500 GPUs and 256GB high-speed SSD storage.

Look for HP's Z420 workstation, product ID F1K03UT. It's also US$3,999, and it's got a 6-core Xeon at 3.5GHz. It comes with twice the RAM and SSD storage, but a single GPU (I don't have performance numbers on a Quadro K4000 vs. an AMD D500).

Adding an extra GPU to the HP model would probably cost a lot more than adding extra RAM and storage on the Mac Pro; point being, they're cost-equivalent when you sum it all up. I know upgradeability is a serious factor for the Mac Pro, as well as the possibility of expanding it with the current state of Thunderbolt accessories. But I don't feel that price itself should be among the cited shortcomings of the Mac Pro. In fact, depending on how good (and thus expensive) these GPUs are, it might even be aggressively priced compared to its competitors.

Am I missing something here? Or are people referring to the fact that it's not upgradeable when they talk about it being expensive? As in, it'll be more expensive down the road because I'll have to replace it instead of upgrading it. Because quite frankly, the upfront price doesn't seem bad at all.