Hello all and Happy Thanksgiving!

A few people have asked how the server is doing and I’ve been meaning to post some metrics for a while.

We’re running on a “Advance-1 Gen 2” server with OVH hosted in Beauharnois, Quebec. This is a 6 core xeon 2386G with 32gb of ram and 2x 512gb nvme ssd’s (raid 1). It’s also ended up being pretty overkill for what we need so we may want to consider downgrading in the future, but it’s relatively inexpensive ($130/mo) for what it is.

We store image uploads on OVH Object Storage, consuming about 430gb.

CPU peaks at barely 10%:

Similarly most of our RAM just gets used for filesystem caching. The pattern you see is due to a cronjob I have in place that restarts lemmy each night at 3am PT, since it seems to leak memory for us.

A few weeks ago we migrated our pict-rs over to OVH’s object storage platform, so our disk needs are pretty minimal.

and with postgres basically just keeping the working DB in memory, disk IO is mostly just writes:

Database throughput shows some interesting activity happening for the past few days, but I just noticed and haven’t dug into who/what might be causing this.

but still performance is fine with most queries returning in under 100ms

Lemmy tends to return 4xx errors hence the high error rates from nginx:

Most of our traffic is still ipv4:

Lastly, we use cloudflare as a caching proxy and ddos protection layer in front of our server. They absorb about 64% of our bandwidth usage:

Let me know if there’s anything else you’re curious about!

  • cygnus
    link
    fedilink
    English
    arrow-up
    8
    ·
    8 months ago

    Thanks for sharing, this is super cool. I expected it to be much more resource-intensive than it is.

  • Greg Clarke
    link
    fedilink
    English
    arrow-up
    8
    ·
    8 months ago

    Thanks for sharing! Out of interest, why did you decide on dedicated hardware instead of a virtual machine?

    • ShadowOPMA
      link
      fedilink
      English
      arrow-up
      15
      ·
      edit-2
      8 months ago

      VMs are good for scaling and resiliency to hardware failure, but they’re generally more expensive and with slower IO.

      Hardware is the best bang for the buck if you’re not concerned about a few hours of downtime if the hardware fails.

      Long term I could see us moving to container based hosting (k8s behind the scenes) like some other sites are doing.

  • Anony Moose
    link
    fedilink
    English
    arrow-up
    3
    ·
    8 months ago

    Very cool, thanks for sharing these, they are always interesting.

    You probably mentioned this earlier, but have we started testing out the lemmy 0.19 release?

  • lazylion_ca
    link
    fedilink
    English
    arrow-up
    2
    ·
    8 months ago

    Where are we donating these days? Do I need to update Librepay before the next renewal?

    • smorksMA
      link
      fedilink
      English
      arrow-up
      2
      ·
      8 months ago

      this is still being worked on, as far as i know. i’ve disabled new donations at liberapay, not sure what will happen with renewals.

      • GreyEyedGhost
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 months ago

        I’m pretty happy here and would like to contribute. It sounds like a couple bucks a month from some portion of the community would be enough for this instance to stay afloat (perhaps more to support the admins doing work I don’t want to), and I’d much rather do that than have to contemplate dealing with Reddit on a regular basis. Please, admits, reach some consensus so we can help support this effort.