I would imagine that important reasons to avoid swing wing aircraft were the costs of production and maintenance and also the weight of the structure. The replacement is the vortex generator in its many shapes. Early large vortex generators are the front of the wing of Concorde and the SAAB Draken, later follow the LEX of F-5E and its relations and we see the full development in the Flankers. Don't forget the canard surfaces on SAAB Viggen, several Mirages and J-20.
Even earlier vortex generators were the wing fences as seen in MiG-15 and still on H-6. A special case is the Gloster Javelin tailed delta which had three rows of small vortex generators, one just in front of the ailerons, a second in front of the first and the third near the thickest place of the wing in front of the others. IIRC these must tame the airflow at trans sonic speed.
Books have no doubt been written about this important subject, but I didn't read them.
PS. I remember visiting the zoo in Rotterdam with my wife and together admiring the vortex generator on each arm of a turtle swimming there.