ِ‘Abdullāh ibn Mas‘ūd (may Allah be pleased with him) reported that the Prophet (may Allah's peace and blessings be upon him) said: "If you are three, then two should not converse privately with the exclusion of the third, until more people join you, because that would grieve him."
Islam encourages empathy as well as good manners and speech in all gatherings. Islam also forbids everything which may harm, frighten, or cause negative or evil thoughts. Two people privately conversing in the presence of a third hurts the feelings of the third person and makes him feel sad and undeserving of taking part in their conversation. It also makes him feel ignored and left out. Hence, Islam forbids this kind of private conversation.