Yes, the Japanese sunk a total of five fleet carriers, one light carrier, and five escort carriers in World War II. Some by submarines, one by gunfire, and the rest by aerial attack. It was an all out naval war at sea in World War II in the Pacific. One of those fleet carriers was the Royal Navy Hermes as you say, the rest were all US carriers.
In addition, the Germans sank four Royal Navy Fleet carriers, three Royal Navy Escort carriers, and one US Escort carrier in World War II. Except for one, all of the German losses were due to German U-boats. The one exception was the Fleet Carrier Glorious, sunk by gunfire from German battleships Scharnhorst and Gneisenau.
All in all, the allies lost nineteen carriers in World War II. Nine fleet carriers, one light carrier, and nine escort carriers. The Japanese sunk a total of eleven and the Germans sunk a total of eight.
In return, the Japanese lost a total of ten fleet carriers, six light carriers, and four escort carriers in the war. A total of twenty carries in all.
Hard to imagine the level of naval combat and losses over a five year period. Thirty nine aircraft carriers sunk in total for the US, the UK, and Japan. Now there were three other carriers (two Italian and one German) that were also lost...but they were never completed and never saw action before being destroyed.