You say that XMPP is much lighter. But I think that is mostly due to Synapse not being very efficient. Other implementations are fairly light. Even then my Synapse is using fairly small amounts of resources. You should also check that you are making an apples-to-apples comparison with large rooms, media and message history like you would typically see in a common Matrix server.
I have a Prosody server running with about 10 concurrent users (friends/family). I just checked and it’s using 32M of RAM, local storage is in the megabytes. The database I’m using as a backend for message history and such is about 70MB. The only other data is temporary cache for uploaded media, which varies depending on what’s uploaded. How does that compare with a typical Matrix server for friends and family?
Looks like stats are posted here: https://bsky.jazco.dev/stats