The Tide of War and Rebellion

It should not be thought that Gower was firmly held by the Normans from the arrival of Henry Beaumont in about 1106. They were probably driven right out several times. In 1135 a large Norman force seems to have been wiped out by the army of Hywel ap Maredudd, in a battle quite likely on Garngoch Common, between Garngoch and Gorseinon. In 1189 Lord Rhys of Deheubarth plundered Gower and in 1192 besieged Swansea for ten weeks. In 1257 Llewelyn ap Gruffydd, prince of Gwynedd "laid waste all Gower". Between 1403 and 1405 the whole lordship was probably controlled, like most of the rest of Wales, by Owain Glyndwr.

In all these attacks, independent Welsh princes were aided by the Welsh of Gower, and Oystermouth castle very likely suffered, though the chronicles that have survived do not mention it specifically. On other occasions its fate is recorded:

About 1116, Gruffydd ap Rhys ap Tewdr invaded Gower and burnt Oystermouth Castle.
"William de Londres for fear of him left his castle and all his cattle and all his precious wealth"
In 1215 Rhys Grug and Rhys Ieuanc, allies of Llewelyn ap Iorwerth, the greatest prince of Gwynedd, attacked Swansea and then:
"started for the castle of Ystum Llwynarth (Oystermouth) and encamped around it that night, and the next day he took the castle and the town"
In 1287, despite Edward I's defeat of Gwynedd, Rhys ap Maredudd joined with the Welsh of Gower
"and by the advice of these men, especially one Einon ap Hywel, attacked and burnt Swansea and took Oystermouth"

What the Castle Looked Like