Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

llm

  1. AndreeOnline

    MP 7,1 Local LLM on Intel and W6800X Duos

    On a whim, I searched if there was some work being done to leverage the W6800X Duos for AI work. Felt like a long shot, but I was surprised to see that there is a project that shows some promise. https://github.com/Basten7/iRon-Llama Seems relatively straightforward to setup, but I haven't...
  2. devon807

    MP 7,1 7,1 Rack AI-Server Build

    Hey everyone!! I hope you are having a wonderful day! Just wanted to share my Mac Pro project. This AI-Focued build is going to replace my old DL380 G10 with two Tesla P100s: The old G10 ran great for a while, but I needed better GPUs for fine tuning. The setup ran great for inference. For...
    • devon807
    • Thread
    • artifficalintelligence llm mac pro (2019) rack rtx 3090
    • Replies: 14
    • Forum: Mac Pro
  3. D

    Seeking M5 Max/Ultra telemetry: does higher bandwidth improve Tokens/Joule or just raw throughput?

    I built an open-source LLM inference telemetry suite that measures Tokens Per Joule — the energy efficiency metric, not just raw speed. Current baseline is on an M1 Pro (32GB UMA): 2.42 Tokens/Joule on Qwen-3B Q4_K_M 22 t/s on Llama-3.1-8B Q8_0 at 8192 context (13.7GB workload) at 35W...
    • dilber
    • Thread
    • apple silicon benchmarking energy efficiency inference llama.cpp llm m1 pro m5 max tokens per joule unified memory
    • Replies: 0
    • Forum: Apple Silicon (Arm) Macs
  4. Populus

    Which M5 SoC should I choose for local LLMs such as Gemma4?

    …without breaking the bank. With the release of Gemma 4 and everyone going crazy with the 31B and 26B models, my hype for local LLM experimentation has rocketed again (after a frustrating experience with LM Studio on a 16GB base M5 weeks ago). Seeing that local models keep getting better and...
  5. B

    M-Courtyard – local LLM fine-tuning app for Apple Silicon

    Hi MacRumors folks, I’m the developer of M-Courtyard, an open-source macOS app for fine‑tuning LLMs locally on Apple Silicon. I’m sharing it here to get feedback from Mac users. It wraps the full pipeline in a GUI: docs → training data → LoRA fine-tune → test → export to Ollama. GitHub...
  6. B

    Gemini on powerpc

    There is a command-line app for Gemini which runs on powerpc: gemini-cli Installable from PPCPorts with usual `sudo port install`. (Should work on other archs too.)
  7. orca0812

    iPhone AI calendar App that works with iCloud & Reminders

    Hey folks — If you use iCloud Calendar or Apple Reminders, but often skip planning because it’s too much work, I get it. I built Trace, an AI-powered calendar app, because I wanted something that would make scheduling feel like texting — not filling out forms. Available on iOS, iPadOS, and...
  8. S

    MP 1,1-5,1 cMP 5,1 options for local AI/LLM?

    IM just genuinely curious - trying to read up and learn.. I want to fool around with a few things, but mostly i want to teach myself about things like - hosting and training a local - offline AI. What ive learned so far (big picture atleast) is that AI loves nvidia, or modern nvidia, etc...
  9. joephish

    Substage: I made a new kind of command bar for Finder that uses natural language!

    Hey folks! I wanted to share a side project that I've been working on! It's a little Mac app called Substage. It's a command bar that sits below Finder windows and lets you interact with your files using natural language: (If you want to give it a try you can download it here!) During my...
    • joephish
    • Thread
    • apps automation finder llm productivity
    • Replies: 0
    • Forum: Mac Apps
  10. E

    Will Apple Intelligence ever allow models other than ChatGPT to be integrated?

    I'm overall disappointed with Apple plugging in a 3rd party solution, but it begs the question on whether we'll be able to use other LLMs like Grok, Gemini, or whatever else you use, with Siri instead of ChatGPT.
  11. K

    Predictive Code Completion Model Xcode 16

    So i tested newest Predictive Code Completion Model provided by Apple. It seems to be small language model (2.2 gigabytes in memory) and it runs on GPU (at least on M1 Ultra) predictions are pretty bad compared to LLAMA 2/3 or GPT 3.5. Here some short video of some minimal swift project with...
  12. ReexpressAI

    Announcing Reexpress one: AI/LLM-powered Data Analysis for Mac

    We're excited to announce the availability of our startup's first software application, Reexpress one, on the US Mac App Store: https://apple.co/40XXpe9 Reexpress one brings reliable, uncertainty-aware document classification and semantic search directly to your Mac, leveraging the power of...
    • ReexpressAI
    • Thread
    • app apple silicon for mac llm macos sonoma nlp semanticsearch
    • Replies: 0
    • Forum: Mac Apps
Top Bottom
Back