• PerogiBoi
    link
    fedilink
    English
    arrow-up
    92
    ·
    5 months ago

    Also check out LLM Studio and GPT4all. Both of these let you run private ChatGPT alternatives from Hugging Face and run them off your ram and processor (can also offload to GPU).

    • Just_Pizza_Crust@lemmy.world
      link
      fedilink
      English
      arrow-up
      26
      ·
      5 months ago

      I’d also recommend Oobabooga if you’re already familiar with Automatic1111 for Stable diffusion. I have found being able to write the first part of the bots response gets much better results and seems to make up false info much less.

      • PerogiBoi
        link
        fedilink
        English
        arrow-up
        43
        arrow-down
        1
        ·
        edit-2
        5 months ago

        Mistral is thought to be almost as good. I’ve used the latest version of mistral and found it more or less identical in quality of output.

        It’s not as fast though as I am running it off of 16gb of ram and an old GTX 1060 card.

        If you use LLM Studio I’d say it’s actually better because you can give it a pre-prompt so that all of its answers are within predefined guardrails (ex: you are glorb the cheese pirate and you have a passion for mink fur coats).

        There’s also the benefit of being able to load in uncensored models if you would like questionable content created (erotica, sketchy instructions on how to synthesize crystal meth, etc).

          • PerogiBoi
            link
            fedilink
            English
            arrow-up
            5
            ·
            5 months ago

            Absolutely. Synthesizing hard drugs is time consuming and a lot of hard work. Only I get to enjoy it.

                • tsonfeir@lemm.ee
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  ·
                  5 months ago

                  I just buy my substrate online. I’m far less experimental than most. I just want it to work in a consistent way that yields an amount I can predict.

                  What I really want to grow is Peyote or San Pedro, but the slow growth and lack of sun in my location would make that difficult.

                  • PerogiBoi
                    link
                    fedilink
                    English
                    arrow-up
                    1
                    ·
                    5 months ago

                    Precolonized or just substrate? I wonder if a grow lamp would work. Pretty pricey though for electricity.

          • PerogiBoi
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            1
            ·
            5 months ago

            They’re the first results on all major search engines.

          • Black_Gulaman@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            3
            ·
            5 months ago

            You can search inside LM studio for uncensored or roleplay. Select the size you want then it’s all good from there.

      • Hestia@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        5 months ago

        Depends on your use case. If you want uncensored output then running locally is about the only game in town.

    • webghost0101@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      7
      ·
      5 months ago

      Something i am really missing is a breakdown of How good these models actually are compared to eachother.

      A demo on hugging face couldnt tell me the boiling point of water while the authors own example prompt asked the boiling point for some chemical.

        • webghost0101@sopuli.xyz
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          5 months ago

          I asked H2O first but no proper answer.

          i heard dihydrogen monoxide has a melting point below room temperature and they seem to find it everywhere causing huge oxidation damage to our infrastructure, its even found inside our crops.

          Truly scary stuff.

    • M500@lemmy.ml
      link
      fedilink
      English
      arrow-up
      5
      ·
      5 months ago

      I can’t find a way to run any of these on my homeserver and access it over http. It looks like it is possible but you need a gui to install it in the first place.

        • Scipitie@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          5 months ago

          (edit: here was wrong information - I apologize to the OP!)

          Plus a GUI install is not exactly the best for reproducability which at least I aim for with my server infrastructure.

          • Emma_Gold_Man@lemmy.dbzer0.com
            link
            fedilink
            English
            arrow-up
            2
            ·
            5 months ago

            You don’t need to run an X server on the headless server. As long as the libraries are compiled in to the client software (the GUI app), it will work. No GUI would need to be installed on the headless server, and the libraries are present in any common Linux distro already (and support would be compiled into a GUI-only app unless it was Wayland-only).

            I agree that a GUI-only installer is a bad thing, but the parent was saying they didn’t know how it could be done. “ssh -X” (or -Y) is how.

            • Scipitie@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              5 months ago

              That’s a huge today-I-learned for me, thank you! I took ill throw xeyes on it just to use ssh - C for the first time in my life. I actually assumed wrong.

              I’ll edit my post accordingly!