Of course it is. Any destroyer at best makes it more difficult for US surface forces to operate west of Guam. A very silent nuclear submarine armed with cruise missiles can potentially threaten the US assets and interests on land and sea everywhere and force a massively capability reducing dispersal of American surface and submarine forces to every part of the pacific and Indian oceans.
The key point here is "very silent".
So far the Chinese have simply not been able to produce a new SSN that is within a generation or two of the vessels like the Virginia and Astute, etc.
When they do, and if it is a time frame where it begins to really compete with the latest generation US vessels, then what you say will certainly be true.
We just have not seen it yet.