How to Get Meeting Notes Using Local Whisper AI

Whisper is OpenAI’s speech recognition model, released as open source in 2022. It’s remarkably accurate — competitive with commercial transcription APIs — and it runs entirely on your own hardware. That makes it a compelling option for anyone who wants meeting transcription without cloud audio.

Here’s how it works, and how you can use it today.

What is Whisper?

Whisper is a neural network trained on 680,000 hours of multilingual audio. OpenAI released the model weights publicly, which means anyone can run it without paying for API access or sending audio to OpenAI’s servers.

The catch: the original Python implementation is slow on CPU, and requires a Python environment to set up. Not ideal for a tool you’d use every day.

whisper.cpp: Whisper for the real world

whisper.cpp is a C++ port of Whisper by Georgi Gerganov. It’s significantly faster than the Python original, supports Apple Silicon acceleration via Metal, and can run on NVIDIA and AMD GPUs too.

More importantly, it compiles to a standalone binary. No Python, no conda environments, no dependency management. You download the binary, download a model file, and run it.

whisper.cpp supports all of Whisper’s model sizes — from tiny (fast, lower accuracy) to large-v3 (slow, highest accuracy). For meeting transcription, medium or large typically gives the best results.

The problem with running it yourself

The technical setup is manageable for developers, but it’s friction. You need to:

1. Download and compile whisper.cpp (or find a pre-built binary)
2. Download the model weights (~1.5GB for large-v3)
3. Record your meeting as a WAV file
4. Run the transcription manually after each meeting
5. Do something useful with the raw transcript output

For a tool you’d use several times a day, this pipeline breaks down quickly.

How Harknotes makes this practical

Harknotes bundles whisper.cpp as a compiled binary and handles the full pipeline for you:

1. Recording — click record in the app to start capturing
2. Capture — mic and system audio are captured as separate streams (important for speaker separation)
3. Transcription — when you stop, whisper.cpp runs on the audio locally, with no internet required
4. Notes — the transcript is structured into a summary, action items, decisions, and full text, then saved as a Markdown file

The Whisper model is downloaded once on first run. After that, transcription is fully offline.

Accuracy and model selection

Harknotes lets you pick the model that fits your hardware — from tiny for older laptops up to the large models for maximum accuracy. Because transcription happens live in small chunks during the meeting, even modest hardware keeps up comfortably with the smaller models.

For most meeting audio — two to eight speakers, typical background noise — Whisper produces transcripts that are accurate enough to be genuinely useful without editing.

Speaker identification

Raw Whisper output is a single stream of text. Harknotes captures your microphone and system audio as separate streams, so your voice (labelled “User”) is always distinguished from other participants (labelled “Meeting”). You can rename speakers and edit any transcript line in the app after the meeting.

Full per-person diarisation — telling apart individual voices within the meeting — is on the roadmap.

What goes to the cloud

Nothing, by default. Transcription is entirely local, and summaries can run against a local model. If you opt in to a cloud provider with your own API key, only the completed text transcript is sent — directly to your provider. No audio, no recording, no raw data, no middleman server.

If you prefer, you can use the local transcript without generating an AI summary at all — it’s stored locally either way, and exports to Markdown whenever you want.

Download Harknotes →