DeepSeek's R1-0528 model is the most powerful open-source model. .Learn to run the model and Qwen3-8B distill with Unsloth 1.78-bit Dynamic quants.
  • besseljEnglish
    2·
    17 days ago

    180GB of combined system RAM and VRAM recommended :(

    Although it looks like the minimum of 60GB ram might be achievable for some

    • fubarx@lemmy.worldEnglish
      1·
      17 days ago

      M3 Max and M4 Max go up to 128GB combined memory. Only Macs that go higher are the Ultras that go to 192GB.

      That limits how many users can run a local server on Apple Silicon. Even if they max out an Ultra, that would be 2 tokens/second. Wonder how much electricity that would burn?