Several explanations have been offerred, none are very convincing to me. One tack is to lay the explanation at the feet (flippers?) of the marine reptiles themselves. Without a set of pharyngeal muscles to seal the throat from the mouth (thus preventing ingestion of large quantities of seawater), or a well-developed set of lip muscles around the mouth to prevent lateral escape of water and food particles or a fast swallowing reflex sauropsids were ill-suited to evolving suspension feeding. In contrast these specialisations were present in mammals before they became aquatic, pre-adapting them to become suspension feeders. This idea is explained more fully in Collin and Janis (1997). I find this explanation rather weak because: a) crocodiles have evolved a mechanism to seal the throat from the mouth despite being sauropids, b) turtles have evolved aquatic suction feeding (one of the mechanisms of suspension feeding) without any of the modifications seen in mammals and c) Mesozoic marine reptiles had to have had mechanisms to prevent engulfing too much seawater whilst feeding underwater in anycase. There can be little doubt that big-mouthed ichthyosaurs, mosasaurs and pliosaurs all engulfed their prey while underwater.
So if intrinsic factors relating to the sauropsids themselves are not convincing then the explanation must lie in some extrinsic factor relating to the environment of the Mesozoic seas.
It has been suggested that the warm waters that predominated for most of the Mesozoic couldn't host large masses of zooplankton to make bulk filter-feeding viable. But, as Heteromeles pointed out, although there aren't the huge clouds of krill that you get in cooler upwelling zones, there is enough suspended plankton in tropical seas to support numerous filter feeders such as manta rays, whale sharks and minke whales.
I don't know the answer to this conundrum but I have an idea that doesn't seem to be mentioned elsewhere. I think the failure of Mesozoic marine reptiles to develop a large suspension feeder is because the niche was well and truly sown-up by another mega-diverse clade. I'm not talking about pachycormid fish. These guys certainly were a long lasting clade of whale-sized suspension feeders in the Mesozoic but they do not appear to have been prolific and wide-ranging enough to have completely excluded all sauropsids from the niche. No I think there is a simple answer, something that was virtually ubiquitous throughout all Mesozoic seas and oceans that is completely absent in the Cenozoic: ammonites.
Yes those (mostly) coiled shelly cephalopds that are virtually synonymous with the popular conception of the word 'fossil'. We don't know much about the soft-tissues of ammonites. They are usually reconstructed as having robust nautilus or even squid-like tentacles and macropredaceous habits. I've even seen more than one reconstruction of the giant ammonite Pachydiscus attacking and killing an ichthyosaur!
Nevertheless there are good reasons to suppose that ammonites were not like squids or cuttlefish in shells. First of their sheer abundance.
Scenes like this are not at all unusual in the fossil record. Indeed if you find ammonites at all, more likely than not, the rock will be stuffed with them. What kind of upper level macropredator is this abundant, or species-rich for that matter?
Ammonites, ammonites, everywhere!
But we don't have to hand wave about the likely ecological role for ammonites. At least for some species we now know that they were indeed micropredators feeding on tiny crustaceans that were likely part of the zooplankton. This is based on synchrotron scanning work that revealed small-scale anatomy of the buccal mass of some very well-preserved ammonites. Published back in 2011 by Kruta et al., these authors suggest that at least aptychophoran ammonites (that is ammonites that had a specialised jaw apparatus called an aptychus) were the major suspension feeders in the Mesozoic oceans. I just go one step further and suggest that it was the swarms of countless millions of ammonites that dominated the suspension feeding niche and prevented the rise of any suspension feeding marine sauropsid. Go molluscs!
In effect the role of a single baleen whale was handled by swarms of ammonites.
References
Collin, R. and Janis, C.M. 1997. Phylogenetic constraints on feeding adaptations: Why there were no suspension- feeding marine reptiles? E. Nichols and J. Calloway eds. Ancient Marine Reptiles. Society of Vertebrate Paleontology symposium on Mesozoic marine reptiles. Academic Press.
Kruta, I., Landman, N., Rouget, I., Cecca, F., Tafforeau, P. 2011. The role of ammonites in the Mesozoic marine food web revealed by jaw preservation. Science 331: 70-72.
