Offline AI assistant on a Mac: local models, no internet

What a local session does without the network

A local session runs both halves of the assistant on the Mac. Whisper turns what the microphone hears into a transcript, and Qwen writes the answers, or the translation when translation is on. No cloud model is involved, so the work itself does not depend on the network: not on hotel Wi-Fi that drops, not on a train between stations, not in a room where devices have no network at all.

This guide is about working without a connection. If your reason for going local is privacy instead — a conversation whose audio should not leave the room — the guide on private transcription on a Mac covers that side.

In a local session you can:

• Transcribe: a live transcript of what the microphone hears.
• Answer: the assistant's replies, as text.
• Translate: one direction at a time, from the From language into the To language, as text.
• Keep it: the session is stored on the Mac, like any other.

What does not work offline

Attachments are not available with the local answer model, so a question about a photo or an image has to wait until you are back online and in a cloud session.

None of this exists on iPhone or iPad. There, sessions run in the cloud and need an internet connection; there is no offline mode to fall back on. If you need help on a phone somewhere without signal, plan for that before you go.

The local models also need a Mac with Apple silicon and macOS 26 or later. That is the same requirement as the Mac app itself.

Before you leave: download the models while online

The models are not part of the app download. You fetch them once, from Hugging Face, in the app's Settings, and they stay on the Mac. Together they take several gigabytes, so do it at home or at the office on a good connection, not on airport Wi-Fi twenty minutes before boarding.

1. Open the app's Settings on the Mac and download Whisper Large v3 Turbo, for transcription, and Qwen 3 8B, for answers.
2. Leave the Mac awake and connected until both downloads have finished.
3. Choose the local models for a session — Whisper to listen, Qwen to answer — before you start it.
4. Click Start Session, say a few sentences and ask a question, and check that the transcript and an answer appear.
5. If you will need translation, click Translate, set From and To for the direction you expect, and check one sentence.
6. Click End. Now you know the models work; the next step is to know they work with no connection.

Try it once with Wi-Fi off

Whether the app opens and starts a session with no connection at all is something to check for yourself, on your own Mac, before it matters. It takes five minutes, and it is far better to find out at your desk than at your seat on the plane.

1. Turn Wi-Fi off and unplug any network cable.
2. Quit the app completely and open it again, so nothing is carried over from the online session.
3. Start a local session, speak for a minute, and ask a question.
4. Check that the transcript and the answer both appear, and, if you need it, that a sentence is translated.
5. End the session. If anything along the way asked for a connection, you know now, while you can still plan around it.

Good uses on a plane, a train or a closed room

A model that runs on a laptop works best with what you give it. Offline, that is what you have: your own words, and the other person's if you are talking with someone.

• Think out loud through a plan, a proposal or a difficult email, and ask for it back as a structured outline.
• Rehearse the opening of a presentation you will give on arrival, and ask what an audience would want explained.
• Translate a few sentences as text for the conversation waiting at the other end, one direction at a time.
• Keep a live transcript of a conversation in a room where devices have no network, with everyone told first.
• Work through a problem step by step when there is nothing to look up.

What a laptop-sized model is good at, and what it is not

Qwen 3 8B is small enough to run on a laptop, which is the point, and it is noticeably less capable than a large cloud model. It is good at rewording, structuring, pulling out what was decided when you ask, explaining common ideas and translating plain sentences. It is weaker at specialised knowledge and long chains of reasoning.

Offline, it also has no way to check itself. Treat names, figures, dates and quotations in its answers as things to verify once you are back online, and do not rely on it for medical, legal or financial decisions.

Running the models takes real memory and processing power, so the Mac works harder than it does for ordinary tasks and the battery runs down faster. Plug in when you can, especially for a long session.

Credits, privacy and what stays on the Mac

Local sessions use no credits and no cloud. The app still needs its subscription to run; local sessions simply do not draw on the monthly credits, so a long flight of talking to the assistant costs nothing extra.

Sessions are stored on the Mac, with no cloud sync, so what you do offline does not appear on your iPhone later. Back online, cloud sessions work as before, attachments included.

If you transcribe other people, even in a room with no network, tell them first. The guide on recording and transcribing other people covers what to say.