Hello everyone!

I’ll be working on migrating our image uploads from local files over to object store this Sunday starting at about 11am PT.

This may require extended lemmy.ca downtime as pict-rs has to be stopped while the migration happens, but I’m hoping I can keep things running as read-only and run the migration off a second instance.

Updates will be posted here - https://status.lemmy.ca/maintenance/257501

  • ShadowOPMAEnglish
    12·
    10 months ago

    lemmy-ui will crash and burn if it can’t load the site logo

    lol yeah I discovered this one a while ago. Thanks for the tip on setting it to null.

    pitcrs is not actually stateless. You MUST save that weird sled database

    Yeah I was assuming so, didn’t think it was stateless.

    pictrs migration code is a bit on the shit side.

    Awesome.

    migration is dogshit slow.

    Even more awesome. I do know there have been performance improvements in recent patches, so if you did your migration awhile ago then we should hopefully be faster. We do have 350gb to migrate though.

    My current thought is:

    1. Block image uploads at the nginx level to make pictrs read-only-ish
    2. Clone the database for a second pictrs container
    3. Start the migration in that second container
    4. Leave the primary container up and serving images, but without any uploads.
    • Illecors@lemmy.cafeEnglish
      5·
      10 months ago

      I did the migration last week. London DigitalOcean to Amsterdam Backblaze. Sticking to “stable” pictrs, so 0.4.2. I think the biggest issue for the migration is that it’s single threaded. If there’s a way to tell it to do many items at once - that would help a great deal. Might be worth looking into 0.5.x alpha releases.

      The problem I see with your plan is running pictrs during migration - you will still get thumbnails generated during that time. All this will lead to split brain. Not too big of a deal as you simply won’t have thumbnails for recent sites, but, technically speaking, you will still have data loss in terms of imaging. Might as well avoid the overhead of making a copy of the db.

      $0.02

      • ShadowOPMAEnglish
        5·
        10 months ago

        The problem I see with your plan is running pictrs during migration - you will still get thumbnails generated during that time. All this will lead to split brain. Not too big of a deal as you simply won’t have thumbnails for recent sites, but, technically speaking, you will still have data loss in terms of imaging. Might as well avoid the overhead of making a copy of the db.

        Yep but I was thinking that losing some thumbnails was better than having image delivery completely off for hours…