I found this story in Pamela Horn's historical academic text Ladies of the Manor.
Aristocrats in the Victorian era always lived by slightly different rules when it came to their marriages. While adultery certainly wasn't encouraged, it was often tolerated by spouses, relatives, and society in general, provided that one was discreet. Wives in particular tended to be far more ostrasized, not for their infidelity but rather for embarrassing their husband publicly.
The general rules were:
1.) For wives, make sure your husband has a legitimate heir and a spare before you run any possible risk of giving him an illegitimate one. If illegitimate children follow after his heir and a spare, everyone will probably just pretend they're his kids and move on.
2.) Be discreet in your own relations. Flaunting your infidelity is NOT okay. And while it's fine for most of the aristocracy to know, once it gets into the papers and becomes the discussion of the middle and lower classes, you are truly screwed.
3.) If you see someone from your social circle clearly having an affair (meeting at a hotel with someone not their spouse, for example), but they are at least attempting discretion, do not call them out on it. If you run into each other, pretend you are strangers, and never speak of it to anyone.
4.) Don't ruin the family finances with gifts for mistresses and lovers. Don't insult your spouse's family or your political party. Basically don't do anything that could jeopardize the socio-economic stability of your dynasty.
However, these rules were not hard and fast. As the century went on, the idea of marrying for romantic love (even as you are attempting to make a 'good match' for your family) became more and more prevalent. Depending on what social set you belonged to in the aristocracy, these rules might or might not apply.
In 1875, the Marquis of Londonderry married Lady Theresa Susey Helen Talbot.
Here is the Marquis. His pantaloons are very disapproving.
Here is the Marchioness with their eldest son. She is clearly trying to fill the void in her heart with a surfeit of necklaces. Oh, honey, jewelry will only fill that void so much. Have you learned nothing from Liz taylor?
Anyway, at some point during their marriage, Lady Londonderry started a little slap and tickle with THIS GUY:
I find his disheveled nonchalance highly erotic. Good on ya, Lady L.
This dude was "Harry Cust, younger brother of Lord Brownlow and a norotrious philanderer. Some of her letters fell into the hands of another of Cust's mistresses and she passed them to Lord Londonderry.
"His reaction was to return them to his wife with a note attached: 'Henceforth we do not speak'.
"He was as good as his word, communicating with her always through a third party. Later he became Viceroy in Ireland and held various official positions, while Lady Londonderry herself became a leading political hostess for the Conservative Party. But they always received their guests standing a little apart, and he seemingly never spoke to her" (166).
I have no idea if the Marquis and Marchioness had a romantic relationship at the start of their marriage or not, and if his decades-long silent treatment was the result of a broken heart or merely a bruised ego. Regardless, what probably did not help the matter was that Lady Londonderry was so sloppy in her affair.
She presumably signed her love letters (a ROOKIE mistake) or at least made no attempt to disguise her handwriting. Cust was careless, not destroying letters, having multiple women on the go, and letting one of those lovers root around his personal belongings and find incriminating evidence. Considering the women he slept with, he really should have known to be more careful.
The double standard in all of this is that Lady Londonderry would be held accountable for Cust's mistakes, because women are ALWAYS punished more for affairs than men are. The logic was that she took up with a known rake, which shows her lack of taste in choosing a suitable lover.