What to do about friend who thinks me & my brother are abused/neglected?

Im 15, my brother is 16, we are very close. we have 2 older twin brothers who are turning 18 very soon, but neither of us get on very well with them. our mother died when i was very young, we live with our father, the house is pretty small & the 4 of us share a room. My brothers best friend, who I am also very good friends with, thinks its weird since I'm a girl sharing with my 3 brothers, but there is nowhere else cud I sleep. our father works but we dont have much money. we get some allowance every week, but not much & we normally buy our own clothes. recently i completely messed up a pair of jeans & only had one other pair & was outta money, so i just wore my brothers. he is like 6”2 & I'm 5”4, but the ends were torn so I just kinda pinned them up, I thought they were ok cos I'm into metalhead kind of style lol. but our friend basically said (only not so bluntly) that we were poor & this was a form of neglect. I do very well in school, and so does my brother, but he sometimes drinks with his friends. My father has caught him a few times & ed him with his belt, & he has skipped school a few times & once I did too & we were both caught & the same thing happened. I hated it but we did kinda deserve it, & he has never hit us in any way like punching or kicking for no reason. I never did it again, but my brother didn’t really care, he was just annoyed he got caught & was saying to our friend, but he said that we were getting abused, even though lots of people we know get ed. Now I'm worried he’ll call social services or something. Cud we be put in a home for something like this