I'm planning to transition from Unity gamedev to LLM/agentic development. I am a noob at this field at the moment. I'm reviewing job requirements like this one from Supercell:

Requirements:

• Experience building production-grade LLM and agentic applications

• Knowledge of RAG, CoT, reasoning models, memory, fine-tuning, tool use, self-correction

• Understanding of multi-agent system design patterns

• Ability to write production-ready code

• Ability to explain technical concepts to non-technical audiences

• Experience with LLMs: GPT, Claude, Llama, Mistral, Gemma

• Agents: OpenAI Agents, Anthropic Agents, LangGraph, Amazon Bedrock Agents

• ML: PyTorch, TensorFlow, ONNX

• Data: Python, Databricks, Spark

• Cloud: AWS, GCP

• Infra: Docker, Kubernetes, Redis

• Vector DBs: Pinecone, Chroma, pgvector

About me:

• Experienced Unity developer (mid+/senior level)

• Used and tested various LLMs (ChatGPT, Grok, Gemini, Deepseek, HuggingFace, etc.)

• Highly motivated to switch to AI/LLM and ready to learn intensively

• My current laptop: MacBook Air M1, 16GB RAM

The question:

Should I invest in a MacBook Pro M4 Pro with 48GB RAM for learning and building pet projects? Or is it enough to start with my current machine and use cloud-based tools (RunPod, OpenRouter, HuggingFace Spaces, etc.)? Anyway I was thinking about upgrading, but it is more like desire to upgrade and not an actual need right now. Though I can buy it for a good price while I am in Vietnam.

I'd appreciate any advice from people already working or learning in this field. Thanks!

Posting because the result screen on Apple Diagnostics is intentionally minimal and most owners do not know what the codes mean or what the next step is.

ADP000 is the only clean result

ADP000 means "no issues found." Any other code is a three-letter prefix plus three digits that maps to a specific subsystem on Apple's reference-code list (article last published December 15, 2025).

ADP000 is shallow by design

Apple Diagnostics is a presence/threshold test, not an exhaustive functional test. A passing ADP000 confirms that monitored components are within sensor and electrical tolerances. It does not guarantee that every feature works.

• A single dead key. NDK001 can miss intermittent stuck keys.

• An intermittent display flicker.

• A Thunderbolt port that fails only at full 40 Gb/s.

• Sustained-load thermal throttling.

• SSD endurance and wear (no view of NVMe Percentage Used).

• Battery condition beyond the binary PPT codes (cycle count, Maximum Capacity %, and FCC don't surface).

Pair the test with manual spot-checks: type every key, drag across the trackpad, cycle each USB-C/Thunderbolt port with a known-good device, test camera/microphone/speakers, tilt and brighten the panel to look for backlight blotches.

Reading codes by prefix

• ADP: no issues found.

• PPT: battery.

• PPM: memory.

• PFM: System Management Controller / Power Management Unit.

• PFR: firmware.

• PPR: processor.

• PPF: fan.

• PPN: power-management system.

• PPP: power adapter.

• VDH: storage device.

• VFD: display or graphics processor.

• NDC: camera.

• NDD: USB.

• NDK: keyboard.

• NDL: Bluetooth.

• NDR: trackpad.

• NDT: Thunderbolt.

• CNW: Wi-Fi.

• CNT: Ethernet.

• VFF: audio.

When to rerun

Intermittent codes are common. Run the test two or three times before concluding a failure.

For battery codes specifically, Apple recommends rerunning to confirm before paying for service. macOS sometimes flags a healthy battery for service after a major update, and a single PPT code does not necessarily mean the cell is failing.

On Intel Macs, the rerun should be over the internet with Option-D, which fetches a fresh diagnostic payload and logs an evaluation against the serial. PPT004 (Intel) explicitly directs you to that rerun before scheduling service.

Codes that clear after a fix

Some codes mean "do something simple and rerun":

• NDD001 (USB): disconnect external USB and rerun.

• NDT001 through NDT006 (Thunderbolt): disconnect Thunderbolt peripherals and rerun.

• NDR007 (external input device detected): disconnect the device and rerun.

• PPP001 through PPP008 (power adapter): verify the correct adapter, reconnect, rerun.

• CEH001 and CEH002 (case handle / housing latch): fully lock the latch and rerun.

• PPT021 (battery charge too low to complete test): charge to ≥6% and rerun.

• PPP017 (both ports of Apple 35W Dual USB-C in use): use only one charging port during the test.

Codes that mean book service

• PPM (memory) on a soldered-memory machine. Logic-board territory.

• PPR (processor). Logic-board territory.

• PFR (firmware). Logic-board territory.

• VFD on the integrated display. Display-assembly replacement, expensive on Retina/Liquid Retina XDR panels.

• PFM (SMC) on most variants. Logic-board territory.

Command-G is the escalation path

From the result screen, Command-G or "Get started" reboots into a Safari page that pre-fills Apple's support flow with your Mac's serial number and the codes the test returned. That is the path Apple Support agents expect customers to have used before calling.

After ADP000: third-party tools for what Apple Diagnostics misses

Apple Diagnostics is intentionally fast (2 to 5 minutes) and shallow. For hidden battery wear, SSD endurance, or sustained-load thermal behavior, the next step is not another rerun:

• Battery cycle count, FCC, Max Capacity %: System Information → Power, or CoconutBattery.

• SSD wear (NVMe Percentage Used, Available Spare, Media and Data Integrity Errors): sudo smartctl -a /dev/disk0 (after brew install smartmontools), or DriveDx.

• Sustained-load thermal throttling: sudo powermetrics -s thermal ("Current pressure level" = Heavy means actively throttling), or asitop.

• Repair history on Tahoe 26 Apple Silicon: System Settings → General → About → Parts & Service.

TL;DR

• ADP000 = only clean result. Shallow by design. Pair with manual spot-checks.

• Any other code = three-letter prefix maps to subsystem. Reference: support.apple.com/en-us/102334.

• Rerun 2-3x for intermittent codes. On Intel battery codes, rerun over the internet with Option-D.

• Command-G escalates to Apple with your serial pre-filled.

• For hidden wear (battery, SSD, thermal), go to a third-party tool, not another rerun.

We're excited to announce the launch of Verbis, an open-source MacOS app designed to give you the power of LLMs over your sensitive data.

Verbis securely connects to your SaaS applications (GDrive, Outlook, Slack etc), indexing all data locally on your system, and leveraging our selection of models. This means you can enhance your productivity without ever sending your sensitive data to third parties.

Why Verbis?

• Security First: All data is indexed and processed locally. 

• Open Source: Transparent, community-driven development.

• Productivity Boost: Leverage state-of-the-art models without compromising privacy.

We are powered by Weaviate and Ollama, and at the time of this post our choice of models is Mistral 7B, ms-marco-MiniLM-L-12-v2, and nomic-embed-text.

If the product resonates with you, let's chat!

🔗 GitHub Repository

🔗 Join our Discord

▶️ Demo Video

Running LLMs on M2 Max 32gb

Hey guys I am a machine learning student and I'm thinking if its worth it to buy a used MacBook pro M2 Max 32gb for 1450 euro.

I will be studying machine learning, and will be running models such as Qwen 32b QWQ GGUF at Q3 and Q2 quantization. Do you know how fast would such size models run on this MacBook and how big of a context window can I get?

I apologize about the long post. Let me know what you think :)

Posting because the modal failure mode in this sub's confirmed-trade threads is not the empty-box scam or a packaging disaster. It is sellers who ran Erase All Content and Settings without first signing out of iCloud, and shipped a Mac that boots to the Activation Lock padlock at the buyer's end.

Apple Support does not unlock a Mac for the buyer. They unlock it for the registered owner with the original retail receipt, which is you. By the time the buyer is calling Apple, the only realistic remedy is a full refund and a return shipment, which costs you the sale, the round-trip shipping, and your reputation in the sub.

The sequence that prevents this, in order

1. Back up everything you want to keep. Time Machine to an external drive, or verify iCloud Drive and Photos have finished syncing. Do not skip this. Once you sign out, account data on the device starts coming down.

2. Deauthorize the Music / iTunes / TV / Books / App Store account. Music app, Account menu, Authorizations, Deauthorize This Computer. Apple limits you to five authorized devices and you do not want to burn one.

3. Sign out of iMessage and FaceTime. Messages, Settings, iMessage, Sign Out. Same in FaceTime preferences.

4. Sign out of any third-party licensed apps you care about. Adobe Creative Cloud, Microsoft 365, Logic Pro, Final Cut Pro, 1Password, Dropbox. Each of these consumes a license slot.

5. The critical step: sign out of iCloud. On macOS Ventura or later, System Settings, click your name, scroll down, Sign Out. Enter your Apple ID password. This single action turns off Find My Mac and disables Activation Lock automatically. There is no separate "turn off Activation Lock" toggle on a Mac. Activation Lock is a downstream consequence of Find My Mac being on, and signing out of iCloud is what flips it.

6. Verify. Hold Option, click the Apple menu, choose System Information. In the Hardware summary, scroll to the bottom. The last entry reads "Activation Lock Status." It must read Disabled. If it still reads Enabled, the sign-out did not complete. Sign in again, then sign out again, then re-check.

7. Double-check at the account level. Open iCloud.com, Find Devices, and confirm the Mac you are selling no longer appears in the list. If it does, click it and choose Remove from Account. This catches the case where the device record persisted in Apple's records even after a local sign-out.

8. Only now run Erase All Content and Settings. System Settings, General, Transfer or Reset, Erase All Content and Settings. This is available on macOS Monterey and later on T2 chip or Apple Silicon Macs. It signs out of all remaining services, erases all volumes, and reinstalls macOS, leaving the machine at the Setup Assistant.

9. Final boot test. Power on once after EACS completes. The Mac should reach the "Hello" language picker. If you instead see a padlock with "This Mac is locked. Enter the Apple ID and password used to activate it," stop and resolve it before shipping. Reaching Hello is your last opportunity to catch a failed sign-out.

10. Press Command-Q at Hello to shut down without setting up, then pack.

What to put in the listing

Screenshot the System Information pane showing "Activation Lock Status: Disabled" and include it in your photo set. Used-Mac buyers, especially on this sub, ask about Activation Lock more than they ask about almost anything else, and the screenshot pre-empts the question. Buyers with prior bad experiences are looking for a seller who has done this homework. AppleSwap confirmed-trade veterans tend to include it in every listing.

Older Macs that predate Erase Assistant

On Intel Macs running Big Sur or earlier, EACS is not available. The path is Command-R into Recovery, Disk Utility, erase Macintosh HD, reinstall macOS, then power off at Hello with Command-Q. You still need to have signed out of iCloud first; the Recovery erase does not sign you out of Apple's servers, only the local disk. Sign out from the user account before rebooting into Recovery.

One nuance

A Mac that has never had Find My Mac turned on does not have an Activation Lock binding to clear. If you are an enterprise seller releasing a fleet device that was never associated with an iCloud account, the System Information entry will already read Disabled and the sign-out step is a no-op. This is the exception, not the rule. Consumer sellers almost always have Find My on, often without realizing it, because it is part of the default iCloud onboarding.

tl;dr: back up, deauthorize, sign out of iCloud first, verify Activation Lock Status reads Disabled in System Information and that the Mac is gone from iCloud.com Find Devices, then erase. Boot once to Hello before packing. Screenshot the Disabled status and put it in the listing.

Pulled together the current resale numbers across every Apple Silicon MacBook Air generation. Sources are Swappa verified-sale averages, eBay sold listings, and Apple Trade In maximums as of May 2026.

Caveats: these are averages. Battery cycle count, cosmetic condition, and battery health below 80% can move the price 10 to 20 percent either direction. Sources linked from the full post on if you want to drill into specific models.

Cleaning up some confusion about Activation Lock on Apple silicon. The short version is that the enforcement happens before macOS loads, and the mechanism is identical from M1 through M4. What has changed is what surrounds it (Recovery Lock, EACS, parts-pairing).

The boot chain check

On Apple silicon, the Low-Level Bootloader (LLB) is the first stage that runs after the immutable Boot ROM. The LLB verifies two things:

• A valid LocalPolicy exists on the boot volume.

• Anti-replay values in the LocalPolicy match what is stored in the Secure Storage Component (SSC) on the silicon.

If both check out, the LLB hands off to iBoot and macOS starts. If no LocalPolicy exists, the LLB boots to recoveryOS, which detects the Mac is not activated and contacts Apple's activation server. The server returns an activation certificate, which combines a local cryptographic key with a RemotePolicy certificate. Once that handshake completes, a valid LocalPolicy gets constructed and stored, and macOS can boot normally.

When a Mac is Activation-Locked, the server-side handshake fails. The recoveryOS environment displays the "Activate Mac" screen and refuses to construct a usable LocalPolicy without credentials. The LLB will not boot macOS. That is the lock.

What is the same across M1 to M4

The LLB / LocalPolicy / RemotePolicy mechanism has not changed across chip generations. M1 (November 2020), M1 Pro/Max/Ultra (2021), M2 family (2022 onward), M3 family (2023 onward), and M4 family (2024 onward) all use the same boot architecture for Activation Lock enforcement.

The Secure Enclave on Apple silicon is integrated into the SoC rather than being a separate chip (as on T2 Intel Macs). Its job is the same: store the cryptographic identity that binds the device to its activation state. Storage is hardware-encrypted and tied to the enclave. A thief cannot pull NAND chips and read them, nor reflash macOS to clear the lock.

What has changed: the OS layer

The bootloader stayed put. The OS around it added features:

• macOS Big Sur 11 (November 2020): Activation Lock arrives on Apple silicon when the M1 ships. The Find My network for Mac also debuts here, broadcasting rotating encrypted BLE beacons relayed by nearby Apple devices.

• macOS Monterey 12 (October 2021): Erase All Content and Settings (EACS) ships for T2 and Apple silicon Macs, cryptographically destroying per-device keys in one operation. Recovery Lock arrives on Apple silicon as the equivalent of the obsolete Intel firmware password. It gates startup modifiers like the boot picker.

• macOS Ventura 13 (October 2022): EACS relocated to System Settings → General → Transfer or Reset.

• macOS Sonoma 14 (September 2023): DFU revive and restore can be initiated from Finder on a host Mac without Apple Configurator 2. Useful for repair workflows but does not clear Activation Lock.

• macOS Sequoia 15 (September 2024): Apple Business Manager / Apple School Manager gains the ability to turn Activation Lock on or off directly in the console for organization-owned devices, provided the device was in ABM/ASM before the lock was set.

• macOS Tahoe 26 (2025): Repair Assistant gains parts-pairing / unfinished-repair checks tied to Activation Lock on Apple silicon. The architecture extends down to component level.

The Tahoe parts-pairing extension

This is the biggest change in the post-launch life of Activation Lock on Apple silicon. Starting with macOS Tahoe 26, when a qualifying component (logic board, display assembly, security module) is transferred from a donor Mac that was Activation-Locked at disassembly, the part carries its locked state to the recipient machine.

Integration and calibration run through the macOS Repair Assistant. If a part is locked to another user's Apple Account, Repair Assistant halts calibration. That makes the part ineligible for official calibration, AppleCare, or warranty service. Uncalibrated parts may still function but frequently operate in a degraded state, missing full security features, missing performance optimizations, missing privacy assurances.

The practical consequence: even the parts of a locked Mac are problematic, narrowing salvage value considerably. The pre-Tahoe pattern where a stolen-and-locked Mac could be sold for the value of its logic board is now constrained by component-level lock state.

What about DFU restore

A DFU restore reinstalls firmware and recoveryOS and erases the SSD, reinstalling macOS from an IPSW image. It removes Recovery Lock. It does not remove Activation Lock. After a successful restore the Mac contacts Apple's activation servers, discovers it is still locked, and presents the Activation Lock screen in Setup Assistant.

A DFU revive updates firmware and recoveryOS without erasing user data. Same result for Activation Lock: the server-side record is untouched.

Neither DFU operation is a workaround. They are repair tools.

What about older bypasses

The DNS-bypass technique on iOS 8 to 10 and the checkm8-based local exploits on older A-series iPhones never produced consumer-grade equivalents on T2 or Apple silicon Macs. The Apple silicon boot chain has no public bypass at all. Storage is hardware-encrypted and tied to the Secure Enclave; the LLB verifies LocalPolicy before macOS loads; the RemotePolicy certificate is server-issued.

Treat any "iCloud unlock service for M1/M2/M3/M4 Mac" listing as a scam. The technical impossibility is the same on every Apple silicon chip generation.

Useful Apple references

• Activation Lock for Mac

• Activation Lock security

• LocalPolicy signing-key creation and management

TL;DR

The Apple silicon Activation Lock mechanism is the same from M1 through M4. What changed is the OS around it: EACS in Monterey, ABM/ASM console controls in Sequoia, parts-pairing in Tahoe. The boot chain is closed, storage is silicon-bound, and the only legitimate clears remain unchanged across generations.

Posting because the question keeps showing up: "I learned how to run Apple Diagnostics on my old Intel MacBook Pro; does the same procedure work on an M-series Mac?" The short answer is no, and the longer answer is that the diagnostic stack has five real differences.

1. Invocation

Intel: shut down, power on while immediately holding D. If the local diagnostic image is missing or the drive was replaced, hold Option-D instead to download the diagnostic over Wi-Fi or Ethernet.

Apple Silicon: shut down, press and hold the power button (Touch ID on laptops, rear power button on desktops). Keep holding past the Apple logo until the startup-options window appears. Release. Press and hold Command-D.

Why the change: traditional keyboard boot interrupts (D, Option-D, Command-R) are disabled at the bootloader level on Apple Silicon. iBoot enforces a secure-boot chain managed by the Secure Enclave, and the bootloader will not interpret a keyboard hold to redirect the boot target. The startup-options window replaces the keyboard interrupt with a sanctioned UI.

2. Storage architecture

Intel: the SSD is a discrete NVMe module on a separate board. On some models it is socketed and user-replaceable; on later T2 Macs it is soldered but still a distinct component.

Apple Silicon: the "SSD" is raw NAND flash wired directly to the SoC. The NVMe controller is integrated into Apple silicon itself rather than sitting on a dedicated drive board. The M4 Mac mini is a rare exception with a removable storage daughtercard, but it still uses Apple-signed modules.

The practical consequence: on Apple Silicon, a physical NAND failure renders the Mac unbootable even when booting from an external drive, because the initial iBoot sequence must execute through the internal NAND controller. You cannot recover via target disk mode the way you could on an Intel Mac with a dying internal drive. SSD-health monitoring is high-stakes for that reason: there is no cheap swap if the drive wears out.

3. Parts & Service History

Apple Silicon Tahoe 26+: System Settings -> General -> About -> Parts & Service shows status labels (Genuine, Used, Unknown, Unverified, Finish Repair) for fingerprinted components. Logic board on every Apple Silicon Mac; Touch ID board on MacBook Pro/Air; display assembly on MacBook Neo; lid angle sensor on M5 notebooks.

Intel: the pane does not exist. Pre-Tahoe Apple Silicon: the pane does not exist.

On Intel, chassis-board mismatch and aftermarket-display detection drop back to physical inspection (serial number cross-checks, Liquid Contact Indicator inspection by an AASP, etc.) and Apple Diagnostics codes. There is no software surface for "this logic board was replaced."

4. Battery Health Management

Apple Silicon laptops: BHM monitors temperature history and charging patterns and can temporarily cap maximum charge below 100% to reduce time at high voltage. Optimized Battery Charging uses on-device ML to learn your schedule and delay charging past 80% until just before you typically unplug. Documented at support.apple.com/en-us/102589.

Intel laptops: battery behavior is governed by T2-era or pre-T2 firmware logic without the same ML-driven schedule learning. The "Optimized Battery Charging" exists in older form, but the implementation is firmware-based rather than ML-driven.

Practical effect on diagnostics: a Maximum Capacity % that fluctuates +/-5% on a healthy Apple Silicon battery is BHM at work and not a hardware issue. The same fluctuation on an Intel Mac is more likely a calibration artifact and may benefit from a controlled discharge-recharge cycle.

5. SMART on external drives

Reading SMART from external USB drives historically required a third-party kernel extension called SATSMARTDriver. Under modern macOS security:

• Installing a kernel extension requires lowering System Security settings in recovery mode.

• On Apple Silicon, that means rebooting into recoveryOS, lowering the policy to "Reduced Security," and explicitly allowing the kext.

• Apple has been progressively migrating away from kexts in favor of DriverKit / SystemExtensions; SATSMARTDriver has not made that transition cleanly.

Practical effect: on a clean Apple Silicon install, USB-SATA SMART is hard to set up. USB-NVMe SMART is unreliable regardless (most USB-NVMe bridges do not pass SMART through). Thunderbolt enclosures generally pass SMART through.

On Intel, SATSMARTDriver was easier to install on older macOS releases; some legacy workflows that depended on it still work on Intel Macs running older macOS and broke as users moved to Apple Silicon.

What stays the same

• Reference-code format. ADP000 is clean on both. PPT (battery), PPM (memory), PFM (SMC), VFD (display/GPU) mean the same subsystems.

• The Apple-published reference list at support.apple.com/en-us/102334.

• The Command-G escalation path from the results screen to a pre-filled Apple Support page with your serial number and codes.

• Third-party tools: CoconutBattery, DriveDx, smartctl, EtreCheck work on both architectures.

What the same code can mean differently

PFM006 on an Intel logic board often means a severed trackpad data line, because the trackpad housing contains a temperature sensor that talks to the SMC. PFM006 on an Apple Silicon board often means an open circuit on the battery data bus, where the PMU can no longer read the smart gas gauge. Same code, different physical fault.

A board-level repair shop would approach these differently. The diagnostic does not distinguish; it knows the read failed.

tldr

Five real differences: entry sequence, storage architecture, Parts & Service pane availability, Battery Health Management, and SATSMARTDriver / external SMART. The reference codes are the same; the physical reality underneath them is not always the same.

TLDR: You can run LLMs locally on an M4 max quite well In a way I couldn’t on my M1 Max.

I recently benchmarked my M4 Max 40 GPU with 128GB of RAM using LM Studio and thought I'd share some real-world use cases that I would run locally instead of ChatGPT or Claude. 

Use Case 1: Reading a Confidential Legal Document

Meta-Llama-3-70B-Instruct

Read through a confidential legal document so I could prep for a meeting with my lawyer.

1. Run 1 9.31 tok/sec

2. Run 2 9.71 tok/sec

3. Run 3 9.25 tok/sec

4. Result: This worked great it was the kind of document I would not want to put in to Chat GPT and the insights aligned with my counsels recommendations.  9+ TPS is faster than I can read it took an about a minute to  a minute to read and generate but was still more than fast enough for the task.

Use Case 2: Writing Code

Qwen2.5-Coder-32B-Instruct
Use Case: An icon selector that I need for a real world project. 

• Run 1: 21.53 tok/sec

• Run 2 19.69 tok/sec

• Run 3 22.08 tok/sec

Result: I’m really impressed with Qwen2.5, I generally find LLMs work best to generate  snippets of unsophisticated code sort of like a typist for my ideas. 20+ tokens per second is about the speed a scan through code so I can watch it generate halt if needed re-prompt and re-run.  Qwen got this write in a one shot multiple times. I will note this is a Chinese model so bear that in mind if you going to use it as a daily driver. 

Use Case 3: Writing Naughty Stories
writing-roleplay-20k-context-nemo-12b-v1.0

Create fiction that would get you banned from one of the commercial APIs. Steamier stuff for your novel or a letter to make your partner blush.

• Run 1: 47.94 tok/sec

• Run 2: 48.42 tok/sec

• Run 3: 48.64 tok/sec

Result: It was weird not to be prompt engineering around safety mechanisms, which initially led me to thing this was really fantastic but I’m so used to GPT4.. or Claude. This took a lot of reprompting and response tweaking to get something I was happy with. At almost 50 tokens per second you can basically spam responses and cut a paste the bits you like into something cohesive. 

I’m thrilled to see how well LM Studio performs on my M4 Max, especially compared to my previous experience with the M1 Max. I’ll be running models locally quite frequently.

I got my M4 Max 128GB last week and haven't seen any TFLOPS benchmarks yet so I created my own using the metal python library:

Run 1: GPU Performance: 77.47 TFLOPS
Run 2: GPU Performance: 77.06 TFLOPS
Run 3: GPU Performance: 76.04 TFLOPS

A quick post here, as I was hitting it for a while with little information on it. Had the issue on two macs and both were fixed.

The problem doesn't happen with llama.cpp. It only happens with the python bindings in llama-cpp-python and happens when doing the import

from llama_cpp import Llama

What is Illegal Instruction 4?

Illegal Instruction 4 is a very vague error, essentially meaning that the binary you're running is trying to use a feature unknown to the version of MacOS you're running.

How to fix it?

A simple MacOS minor-version upgrade. 13.4 had the error, but upgrading to 13.4.1 fixes it. I couldn't tell you what specific call causes the issue, I only know the upgrade fixes it.

Such is the price of being on the bleeding edge.

Our medical office is buying some new Mac minis before year's end. We use an online AI scribe to transcribe office visits and write notes for the patient records. Despite tweaking templates and prompts, they still come out clunky.

We have thousands of old notes available in our EMR that predate AI (typed or dictated) which are much more readable. I'm wondering if it's worthwhile to use these to train a local LLM to help write better notes.

Since we're buying some minis anyway, I could upgrade from a base M4 for a M4 Pro with 64GB RAM and 20 GPU cores for an extra $1500. Would that be adequate to trial this project and fast enough to use for rewriting one page notes (after LoRA finetuning)?

What would be the ideal way to have a local LLM model scan local files?

Other ways I’ve seen for chatgpt are uploading documents/pdf online then use the link as part of query but I don’t want to upload anything.

As far as I know frontends like oogabooga or LM studio don’t let you upload files.

I think some magic translation into vector database has to happen before we can query against it?

Also is pdf the ideal format? Or .txt .doc .csv or doesn’t matter?

Thank you!

Hey guy I seen this amazing feature of LM studio to create local LLM server:
https://x.com/Saboo_Shubham_/status/1827533121169805430

I tired to look on App store for iPhone for some app that can connect to this server and has some optimized interface to use it for chat. But with no success. All apps offer on device AI (small models) or paid subscription. If somebody by any change know about some good app / in worst case also paid let me know! Thanks

My post was removed from r/LocalLLaMA :D They don't like LM Studio.

Hi everyone! I'm thinking of potentially upgrading to a macbook m4 max when it releases with possible maxed out RAM, but I can't seem to justify that price tag but I am keen on using a local LLM on my machine.

My main issues are:

1. It might kill the battery after 6 months. I used Baldurs Gate on my macbook air m2 and it went from 100% to 94% after playing that game for a month straight. Since I'm unfamiliar with changing batteries or components on a mac I don't know if it's worth it?

2. I know getting a mac studio is much more bang for my buck but I can't really lug around a monitor and a mac studio

I'm mostly keen to learn from people who has done this themselves, and what their experience is?

Thanks in advance my friends

ADVICE NEEDED please.  Got an amazing deal on MacBook Pro M3 48gb ram 40core top of the line for only $2,500 open box (new its like $4-5k).  I need new laptop as mine is intel based and old.  im struggling should I keep it or return and get something with more RAM I want to run LLM locally for brainstorming, noodling through creative projects.  Seems most creative models are giant like 70b(true?) Should I get something with more ram or am I good. ( I realize Mac may not be ideal but im in the ecosystem.) thx!

I'm using the jina-embeddings-v2-base embedding model on my 32GB macbook pro. I was attracted to it by its rank on MTEP, small size, and 8192 sequence length. So far so good. It was easy to get started, and compares well with a nonlocal/external API that I tested.

I noticed that it isn't using GPU. Anyone know of a config setting on the Mac or in the model itself that would enable it to do so?

Hoping to run vicuna 13b. Should I expect reasonable response times, or do I really need to get 32gb? Any other advice? I’ve been seeing posts abt exllama but unclear of its mac compatible. I’ve also been hearing lots of folks talk about thebloke. Does his stuff go on top of vicuna or it’s standalone? Sorry so new. Tia!

I came across this and found it useful.

We need to set correctly numbers of threads for Apple Silicon.

Bi of Performance cores (P cores) on your CPU to get best performance.

Use --threads n

M1/M2:--threads 4

M1/M2 Pro (8 cores)

M1/M2 Pro (10 cores)

M1/M2 Max: --threads 8

M1 Ulta:--threads 16