It is definitely fin rot and the infection looks to be very well established which means that any antibacterial remedy is going to struggle to cure it.
The best course of treatment would be antibiotic injections under the guidance of a vet (which is likely to be expensive). Next you could try treating topically with povodine iodine. This should require just a single treatment and if that had no effect then repeating it is unlikely to work.
Along with the treatment you will need to find the underlying cause and put that right or no treatment will work for long. You mention that you realise the fish aren't in exactly ideal conditions and I agree and this really needs addressing urgently.
beach sand would need to be very well cleaned because there is in most cases a lot of organic material in it especially if its a n area where seaweed gets washed up. It is also likely to contain a lot of calcareous material like shell fragments meaning it would be of no use where soft acidic water was required.
But you can in some cases take it quite legally unless there are local bylaws preventing this i.e if a beach is owned by the National Trust (as the majority seem to be these days) then absolutely nothing can be taken, not even a small shell. So check first.
Using an external filter might not be a good idea because unless you are prepared to replace the media at very short intervals it will become a nitrate factory. Most of the rubbish is removed by skimming and what is left is dealt with by micro organisms living in the live rock provided that you have enough water circulation and no dead spots.
The leporinus wasn't a fish I would have chosen to keep but it was brought in to my lfs when I was there. This particular lfs is really a general pet shop with a back room with about 40 x 60 litre tanks. This fish was simply to big for a 60 litre tank and so having a 450 litre tank at home which wasn't doing very much I decided to offer to take it.
The redhooks are related to piranha but they are 100% vegetarian. No real plant is safe with them. I feed them with all my prunings which can be considerable i.e a carrier bag full of aquatic plant prunings will vanish in 24 hrs.
My shark is entirely peaceful but he/she is in a 250 litre planted tank which is quite sparsely populated probably only stocked at 1/3rd the usual. Interestingly I have never seen it take fish food of any kind - ever. It just goes around grazing on the leaf surfaces apparently for algae and diatoms. I've had it for years and it is quite plump so it is obviously getting enough of what ever it is eating.
The shovelnose is interesting because it periodically sheds its skin like a reptile and then proceeds to eat it.
I won't go through all the usual questions because I've seen your posts on here and I doubt anything in the fish's environment has caused this. You probably realise how serious this is too.
Osmoregulation is the process where the fish maintains the right ionic balance for salts/fluids within its body. Fish have very waterproof bodies except for their gills, almost all gas exchange, salt uptake and fluid uptake occurs at the gills and the gills are controlled by the endocrine system (hormones). The kidneys along with other glands control via hormones how much fluid the fish takes in or expels to maintain the narrow band of an acceptable ionic balance.
Quite often infection is blamed for dropsy and while it is possible it represents a tiny minority of cases and in a well maintained tank of healthy fish where has been no recent additions then infection can be pretty much ruled out.
Organ failure is a much more likely cause, but as you can see this is a very complex system and it is impossible for any hobbyist to properly diagnose the precise cause and even if the cause were known for certain I for one would have no idea how one would treat kidney failure in a fish for example.
There are lots of suggestions out there about adding salt or adding epsom salt and so on but...
The excess fluid that the fish has taken on isn't sat in a reservoir in the gut, it is actually within the cell walls of each cell, this makes the cells swell and it is this which makes the scales pinecone which doesn't happen when the fish eats to much for example. Obviously when this happens not only to the skin but to the internal organs too it can do a lot of damage to those organs including the already failing organ. This is why there is such a poor prognosis for a fish with dropsy.
Hopefully this doesn't sound to harsh but if you intend to try treating the fish would it be possible to take some really clear photos, the reason I ask is that fish health photos are hard to come by and if by some good fortune the fish recovers those photos would be invaluable for an article on the subject and even if the fish succumbs fish health photos are always useful for illustration and helping others which is obviously something dear to you and it would mean something good coming out of this.
If there is no improvement quite quickly I think ending the suffering should be considered, sorry I can't offer anything more useful than this.