Astrological factors responsible for extra marital affair are
Venus --- significator of marriage for male, rules emotional love, sensual pleasures and worldly desires.
Jupiter --- significator of marriage for female
Mars --- rules intense sexual expression, daring and passion for carnal pleasures
Rahu --- indicates extreme mental tendencies, unconventional thing of life
Moon --- mental tendencies, harmony with others, emotional stability
5th house --- stands for love, romance, and scandals and is the primary indicator of affairs.
7th house --- indicates marriage, harmony in love
8th house --- house indicates sexual attitudes, orgasm, masturbation, secrecy of life etc. The 8th house indicates whether the affair will be physical or not.
12 th house/lord --- Bed pleasure