Learning from the Experts: How to Use a Local AI for Small Coding Tasks

Even a top AI maintainer keeps it simple - and so can you.

What it is

Georgi Gerganov shared in a Hacker News comment that he uses a local AI model called Qwen3.6-27B almost daily for small, mundane coding tasks in his work as a maintainer at ggml-org. He runs it locally - either on an M2 Ultra or an RTX 5090 - with a very lightweight harness ('pi agent with everything stripped, pi -nc --offline') and a short system prompt to align the output with his personal style. This shows that even people doing serious AI work use it in simple, focused ways for everyday maintenance tasks.

Why it matters

You don't need an impressive or complex setup to get real value from AI tools. Gerganov's example shows that using a local model for small, repetitive coding tasks adds up to genuine help over time. The key transferable skill is giving an AI a short, clear instruction that anchors its style before you hand it a task - something anyone can practice.

How to use it

  1. 1Pick one small, repetitive coding or text task you do regularly.
  2. 2Write a short style instruction that describes how you want the output to sound or behave - this is the 'system prompt' idea from Gerganov's setup.
  3. 3Combine that style instruction with a specific, focused task in a single prompt.
  4. 4Review the output, adjust the style instruction if needed, and note how it changes the result.
  5. 5Repeat with the same type of task over several days to refine your approach.

Example

Gerganov uses a short system prompt to align the AI's output with his personal coding style before asking it to help with maintainer tasks at ggml-org. The practical idea - leading with a brief style anchor before giving a task - is something you can try in any AI tool you use, even if your context differs from his local, offline setup.

Practice it

Open ChatGPT or Claude and try this two-part prompt structure: first write one sentence describing the style or tone you want ('Write in terse, direct code-comment style'), then give a small concrete task ('Rewrite this function description to match that style: [paste a short piece of text or code]'). Run it once with the style instruction and once without, and compare the difference. This trains the habit of anchoring AI output before asking for help.

Source