Enforced monogamy for over 100 generations resulted in limited cognition in male fruit flies.
Sexual selection is responsible for the evolution of male ornaments and armaments, but its role in the evolution of cognition—the ability to process, retain and use information—is largely unexplored. Because successful courtship is likely to involve processing information in complex, competitive sexual environments, we hypothesized that sexual selection contributes to the evolution and maintenance of cognitive abilities in males. To test this, we removed mate choice and mate competition from experimental populations of Drosophila melanogaster by enforcing monogamy for over 100 generations. Males evolved under monogamy became less proficient than polygamous control males at relatively complex cognitive tasks. [emphasis mine]
Too bad the researchers couldn't even imagine fruit fly "free love" for a control group. Flies aren't smart enough for polyamory, which requires negotiation.
The full article is behind a paywall -- arrgh!
Still, "remov[ing] mate choice and mate competition" (presumably by just pairing up males and females) isn't the same as only "enforcing monogamy" (which would require watching all the fruit flies 24/7, letting them choose mates on their own, then immediately isolating each pair so they're stuck with each other).
In polyamory (which, as you said, requires higher-level cognition!), partner choice can be an active concern over more of one's life, but it can be for different roles and types of relationships at different times. Significantly, a poly person looking for a committed "primary" partner knows that that person won't be "Everything" to them.
Interesting reading. It seems to go with another study, for which I cannot find the link, indicates people who are into non-vanilla sexual activities tend to be more intelligent in terms of IQ than those who engage only in vanilla activities.
It's no longer behind a paywall.