There are slavers who would do so, and disasters which happen and limit food resources of course.

But there are numerous species in the galaxy who have different dietary needs, and some which are perfect for one would be lethal to another. Of course, some planets have their own quirks in terms of have can be farmed or not, but generally some type of trade is available. So people eat the diet which works best for them.