But the Han and Tang expanded to their sizes precisely because they gradually absorbed and sinicized the areas further south. Northern Vietnam was a province of China for a long time. Could it have become another Yunnan or Guangdong if it had been held and sinicized further?
On the subject of the Han and Tang's intentions:
The Han conquest of the south was almost entirely a matter of taking over the previous Qin commanderies (and Vietnam, which got included because Qin general-turned-warlord Zhao Tuo annexed it during the Qin collapse to reinforce his personal base. Yunnan's kingdoms submitted voluntarily). Han conquests in the east were based on taking over Wiman Joseon, which was led by renegade Yan general Wei Man and controlled the ancient Yan territory of Liaodong. The Han was focused on eliminating Chinese rivals, and barbarians only got included if they happened to be serving those Chinese rivals. Only in the north and west, the Han went all out to recruit Xiongnu subjects as allies against the existential Xiongnu threat, but that is why Southern Xiongnu became a vassal, and the Western Regions were called "Protectorate" 都护府, not province 州 or commandery, county 郡县.
The Tang based their conquests on the same Han borders (not all successfully), with the same exception of the northern and western protectorates. In fact, the Tang didn't even bother to go further south then what the Southern Dynasties had been hanging onto since Han times, and even redesignated Vietnam as a protectorate to exempt their chiefs from the normal bureaucratic system. This exemption was inevitable because the Southern Dynasties had already tried and failed to implement central government control in Vietnam, causing many rebellions (Catherine Churchman,
The People between the Rivers).
From the Song onwards, no Chinese dynasty (with the exception of the Mongols of Yuan who believed that Tengri wanted them to literally conquer every inch of the earth) would ever again try to annex previously unclaimed territories. The Ming only started to take an interest in Taiwan because of the subjugation of Chinese warlord Zheng Zhilong who already had bases on Taiwan, and then the Qing had to take it because the Zheng family entrenched themselves there. Even the Qianlong emperor had to argue with his officials that the Western Regions belongs to China because of the Han and Tang, and therefore they should stop objecting to his annexation of it.
The one and only truly expansionist Chinese emperor was actually Qin Shihuang, who was also very unusual in many other ways. Let's not fuel this Sinophobic myth that China has always wanted to conquer the world. The tributary relations that China always preferred was a two-way, mutually-beneficial arrangement, not some semi-colonial scheme.
On the question of holding on to Vietnam:
After Tang, no dynasty had the strength to recover Vietnam until the Ming. Song was weak. Yuan's Mongol casteism made them so unpopular that ethnic Han went to Vietnam to help them fight Mongols. Even the Ming decided to give up after 20 years because Vietnam was economically useless and the Mongols were a more pressing issue to focus on. AND most importantly: After the 20 years of Ming sinicization of Vietnam, Vietnam centralised and professionalised its government (the previous Tran dynasty was mostly governed by imperial relatives in each province), became more powerful than ever and conquered from the Tonkin Gulf to the Mekong Delta. After that, reconquest was impossible.