The 'Long Man' of Wilmington, in Sussex
The original inhabitants of the island of Albion - or Merlin's Enclosure, as it was first called, Merlin being the presiding deity - were a race of Giants. Indeed, the name of Albion comes from their king.
When the island was first settled by normal-sized Men (Brutus the Trojan and his followers - great grandson of Aeneas) - it was necessary to defeat resident Giant population. Gogmagog was the most famous of these - apparently a small and weak example of the race. However, a remnant of Giants continued as an occasional menace for a very long time afterwards.
Thus Geoffrey of Monmouth - Albion's primary mythographer.
Giants crop-up in many historical, religious, legendary and mythical sources, from all over the world; so there is no good reason to doubt their reality - except that we don't seem to have any nowadays. Much the same applies to the races of elves/ fairies and to dwarves - there is ample evidence for their existence in earlier time; far more evidence than for most supposed facts of history.
But of course, that does not mean that Modern Man would be able to perceive Giants, fairies or dwarfs, even if they were present - since we are self-blinded to much of the primary reality of this world; and furthermore treat as dogmatically-real many things which are imperceptible and undetectable (except by long chains of insecure and labile inferences).
Giants seem to be a member of the human genus - and perhaps even the same species, since there are many accounts of breeding between us and them - with fertile offspring. Presumably therefore, in some places (perhaps especially some parts of Britain?) some modern Men therefore have gigantic ancestry...
There is no reason why all Giants, everywhere and at all times, should be identical; but it seems that at least some were intelligent as well as strong. The great feats of building attributed to them could not have been accomplished solely by muscle-power.
The idea that giants were stupid seems like a modern slander, since they had their own long-lasting civilisations all over the place - but I suspect that were were easily tricked, and outwitted, by the normal-sized Men who replaced them. Perhaps childlike, naive and instinctive rather than idiotic.
Or perhaps some Giants were magical, and used that rather than 'technology' to perform their feats - indeed, since early man was certainly able to do magic, it seems likely that Giants were too.
How gigantic were Giants? My guess is about 12-18 feet, full-sized and full-grown - big enough to be qualitatively different from us; but not so big as to be just-plain-silly...
Giants don't, for me, have the fascination of fairies, nor the common sense imaginability of dwarves; yet perhaps they ought-to. It just needs a great storyteller or two, to help us picture and empathise-with them.
The Cerne Abbas Giant, Dorset - another impressively 'long' man! - albeit, he gained some extra 'length' a century or two ago when his umbilicus was accidentally incorporated into his phallus by over-enthusiastic turf-cutters...
(Note: Geoffrey of Monmouth had access to a now-lost book of the deep and mythic history of Albion - a book which, I imagine, would have provided almost exactly what JRR Tolkien felt was missing from his country's culture - 'a mythology for England'. Our modern Geoffrey - Geoffrey Ashe, the Greatest Living Englishman - made a really excellent job of filling this gap with his Mythology of the British Isles of 1990. Ashe uses Monmouth's book as a skeleton, fleshed out with all other available and relevant sources to provide a clear and concise mythical account at the start of each chapter, followed by scholarly commentary and footnotes. If someone extracted the mythical sections, and arranged them sequentially; when imaginatively-illustrated and published as a unity, this could make a wonderful Child's History of Albion.)