I have 4 ram cichlids.. 2 blue rams and 2 golden rams.. only the blue ones are quite often facing each other and try and peck each other.. peck isn't really the right word for fish but you know. I suppose this is territorial? I haven't really seen the golden rams do this, and I've had them for longer.

I have an 80 litre tank, and I've read somewhere this is the minimum size really, do I have too many cichlids?

Yeah, you should really only have one pair of cichlids in that size tank and even then it is on the small side.

Yes to be honest you've 2 too many. They may get on okay at the moment but if they do decice to pair things will become awkward and if spawning enters their minds downright nasty.
Possibly resulting in some badly damaged/dead fish.
Sorry.

are both the blue ones male or is one female? male rams should have about 2foot or floor space each for their own territory, and they should be kept in male female pairs.