Anas ibn Malik said, "A man said, 'Messenger of Allah, we were in a house and there were a large number of us and we had a lot of property in it. Then we moved to another house and our numbers decreased and our property became less in it (the new house).' The Messenger of Allah, may Allah bless him and grant him peace, said, "Go back to it (the first house) or leave it (the second house) - it is bad.'"
Source: sunnah.com
Other translations
Arabic
حَدَّثَنَا عُبَيْدُ اللهِ بْنُ سَعِيدٍ يَعْنِي أَبَا قُدَامَةَ، قَالَ: حَدَّثَنَا بِشْرُ بْنُ عُمَرَ الزَّهْرَانِيُّ، قَالَ: حَدَّثَنَا عِكْرِمَةُ بْنُ عَمَّارٍ، عَنْ إِسْحَاقَ بْنِ عَبْدِ اللهِ، عَنْ أَنَسِ بْنِ مَالِكٍ قَالَ: قَالَ رَجُلٌ: يَا رَسُولَ اللهِ، إِنَّا كُنَّا فِي دَارٍ كَثُرَ فِيهَا عَدَدُنَا، وَكَثُرَ فِيهَا أَمْوَالُنَا، فَتَحَوَّلْنَا إِلَى دَارٍ أُخْرَى، فَقَلَّ فِيهَا عَدَدُنَا، وَقَلَّتْ فِيهَا أَمْوَالُنَا؟ قَالَ رَسُولُ اللهِ صلى الله عليه وسلم: رُدَّهَا، أَوْ دَعُوهَا، وَهِيَ ذَمِيمَةٌ.
sunnah.com