In the sign up process, it discourages using the primary srever. In the docs it talks about performance limitations. In contast Mastadon does not do this. Worse yet, most people think of a single centralized server. And this server only has some 500+ active users a month. Maybe 10 times that many lurkers?
While I love the Lemmy design, and the community looks great, I fear that it does not scale. Worse yet I suspect that the Activity Pub protocol is grossly inefficient for Reddit style applications. Maybe a publish subscribe message broker, like MQTT would be much more efficient.
So what is gong on here. Is this an issue? Am I correct to be scared about Lemmy scalability.
And just for context, I think you are totally on the right track, way ahead of all ofther reddit clones, but maybe just a little off course. Can you crank up performance?
So far there are no performance problems at all. Lemmy.ml runs on a dual core vps and only uses like 5-10% of each core (load average around 0.5). The reason we want to discourage use of lemmy.ml is that it makes for an unhealthy ecosystem if the majority of users are all on a single instance. I also dont think that federation would be a performance problem unless you have a really huge instance. If we run into any performance issues, then Rust will give us a lot of ways to optimize that.
Is the programming language really the bottleneck when dealing with web servers like that? The last time Lemmy became slow it was because of the way the database was used and not because of CPU/RAM bottleneck with Rust if I remember correctly. Good job by the way on fixing it, Lemmy is really snappy and I like it!
I guess Lemmy will struggle when there will be posts with thousands of comments but we are not there yet. In this case Rust wont help much and only a better API with pagination will (if it’s not already implemented), though I guess Rust certainly helps a bit.
Rust is just a really nice language (no null pointers or anything similar). We arent using it for performance reasons, thats just a nice bonus.
Once this is implemented, large threads can be handled much more efficiently.
Yep, I find rust amazing too. I just think that the ecosystem has some maturation to do, there are still many major crates not having reached version 1.0, but we’re getting there, and some compiler features (GAT, const generics) will soon make some of my grievances go away.
Glad to see you’re already working on thread pagination. Pagination for something with a non trivial sorting order doesn’t seem easy to solve. Lemmy is amazing thanks for everything!
Mastodon will struggle to live on a 1GB RAM system, like most cheap VPS. I host one instance (single-user though) on a ROCK64 with 4GB of RAM, and it takes around 1GB. Lemmy on the other hand takes right around 100MB of RAM, also single-user instance. I guess Lemmy’s developers prefer to stay on the safe side and encourage people to make use of federation as much as possible.
Lemmy is written in Rust while Mastodon is written in Ruby, this likely explains the differences in RAM usage.
Maybe CPU/Disk will scale worse on lemmy because each page loads a lot of comments, while on Mastodon you rarely load more than a few toots at once.
I think you’re right, Mastodon’s CPU/RAM usage seems to be always the same. On Lemmy I can’t really try yet, as federation doesn’t seem to work for me. Once I can get content from other instances I’ll have more information.
I used to host a mastodon instance for just one user, me. It really struggled unfortunately in both speed and space requirements.
I wish they had an option to block all bots from the platform. I can block individuals and instances, but bots would still pop up like crazy.
From lemmy.ca:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMANDlemmy 17645 0.3 0.3 672368 98876 ? Ssl Aug03 154:51 node dist/js/server.jslemmy 20959 0.4 0.6 4127676 210048 ? Ssl Aug03 169:59 /home/lemmy/lemmy/target/release/lemmy_serverpictrs 32164 0.0 11.4 10599856 3769764 ? Ssl Jul28 18:50 /home/lemmy/pictrs/pict-rs/pict-rsiframely 20977 0.0 0.1 901704 55368 ? Ssl Jul08 3:25 iframelyMake of it what you will. :-)
WOW. 100MB is tiny. but Lemmy requires postgress, and a quick google search says Postgres requires minimum of 2 GB or RAM.
I must say that for now there is no content on it, it’s a fresh instance, with federation enabled but still not getting content from others. Once I can start to sync with others, I’ll see if there is any change.
You can use RDS and not host it within the same server.
