Join a video meeting as an AI bot with voice, avatar, and screenshare across four operating modes.
by johnpatternai21 installs8 stars
Design & media
edge-cpu-gguf-tuner
Try itEvidence-first, offline tuning of llama.cpp GGUF inference on CPU and constrained edge hosts. Inspects CPU topology, renders compatible benchmark plans, optionally runs an explicitly supplied local llama-bench, ranks pp/tg/pg results with variance-aware confidence, verifies output gates, and renders (never executes) a measured command. It does not install llama.cpp, download models, call APIs, or claim universal defaults.
What it does
**Max tokens/sec for llama.cpp on CPU-only / constrained machines** (VPS, containers, sandboxes, Raspberry Pi). GPU tuning guides (e.g. ) actively mislead on CPU — this skill is the measured CPU counterpart.
The skill document
edge-cpu-gguf-tuner 🧮⚡
Max tokens/sec for llama.cpp on CPU-only / constrained machines (VPS, containers, sandboxes, Raspberry Pi).
GPU tuning guides (e.g. llama-params-optimizer) actively mislead on CPU — this skill is the measured CPU counterpart.
TL;DR — measured results (2 vCPU / 2 GB RAM, llama.cpp build 1a064ab, CPU build, r=3–5)
| Param | Best on CPU | Why (measured) |
|---|---|---|
--threads | = physical cores | 1→2 threads: tg 16.9→30.7 t/s (+82%, perfect 1.9× scaling) |
--flash-attn | on | pp2048 +9%, tg128 +11% even at short ctx; bit-identical output (temp 0.1, FA on/off) |
--cache-type-k/v | f16 (default) | q8_0: pp −11…19% — dequant overhead > bandwidth win at short ctx 🌀 |
-b/--batch-size | default 2048, don't bother | 512/1024/2048 spread = ±2.4% — pure noise on CPU+small models |
| quant choice | newer-arch Q4_K_M > older smaller Q5_K_M | Qwen3-0.6B Q4_K_M beat Qwen2.5-0.5B Q5_K_M everywhere (pp 152 vs 77 t/s, tg 33 vs 30) |
| mmap | keep default (on) | lets page cache absorb models > free RAM (1.1 GB model ran fine in 1.4 GB available) |
Workflow (30 min, control-variable)
- Build bench tool (~2 min cached):
cd llama.cpp && cmake --build build --config Release --target llama-bench llama-completion -j $(nproc) - Baseline (root of all truth):
llama-bench -m model.gguf -p 512,1024,2048 -n 128,256 -t -o md - Sweep ONE variable per invocation — never chain:
-t 1,2/-b 512,1024,2048/-fa off,on/-ctk f16,q8_0 -ctv f16,q8_0(bench cross-multiplies),-r 3is enough during sweep. - Validate winner head-to-head on every model baseline vs tuned).
- Quality gate: same prompt,
llama-completion --temp 0.1, FA on vs off — outputs must be identical. - E2E: real generation, read
common_perf_printtokens/s; expect within ±5–8% of bench (shared-box noise).
Counterintuitive log (the valuable part) 🌀
- KV q8_0 slows CPU (−11…19% pp, −1…2% tg). The GPU rule inverts: at ≤4K ctx the KV fits caches anyway, dequant math is pure overhead. Only quantize KV when RAM-starved.
- FA helps even at short ctx on ≤0.7B models; neutral on 1.5B — never hurts → default ON on CPU.
- Bigger+newer-arch Q4 beats smaller-older Q5. Architecture generation ≺ quant type for CPU speed; don't pick models by param count alone.
- Batch size is a no-op for CPU prompt processing of small models — ignore the GPU-era +67% claims.
- No memory sweet-spot cliff on CPU (that's a VRAM-bank artifact). Just keep model+KV+compute inside available RAM; 90–95% rules don't apply.
Pitfalls
- llama-bench default
-r 5— long runs; pass-r 3for sweeps. - New llama.cpp renamed the CLI: monolithic
llamaneeds server libs (fails withLLAMA_BUILD_SERVER=OFF); usellama-completion— full common-params (-fa,-ctk, templated chat, non-interactive stdin-EOF exit). - Classic
llama-simpleaccepts ONLY-m -n prompt— no tuning flags reach it. - Rerun after any environment wipe; binary targets:
llama-bench llama-completion(+optionalllama-simple).
Deploy the tuned answer
Related skills
Stores durable facts in a categorized, plain-markdown vault on disk, alongside your agent's built-in memory.
by Iván555 installs18 stars
Write, debug, and tune Playwright specs with locator strategy, trace diagnosis, and CI-aware timeouts.
by ivangdavila1.3k installs129 stars
Save, search, and manage personal notes and knowledge bases in Get笔记 on explicit request.
by iswalle763 installs66 stars
Generate and edit Draw.io, Mermaid, and Excalidraw diagrams from natural language using a structured JSON spec.
by nssa.io1.0k installs47 stars
Query and manage Linear issues, projects, teams, cycles, labels, and comments through a managed OAuth GraphQL endpoint.
by byungkyu518 installs18 stars
More from orionshaowswmw
Browse all skillsDetect and repair partially wiped agent workspaces with integrity checks, signed manifests, guarded restore recipes, bounded local recovery state, and explicit off-box sync. Use when files, scripts, trees, models, or build outputs disappear or lose integrity between turns.
by orionshaowswmw1 installs1 stars
Seven offline mechanisms against slow/stale/zombie/sycophantic agent turns: prompt compaction, request fencing, zombie detection, CAPTCHA triage, anti-sycophancy spine, delivery register, invention quarry. Use when chat feels laggy, reconnects surface old answers, long chats degrade, or the agent caves under contradiction. JSON contracts; state per-agent under ~/.arena_turn; no network, no sudo.
by orionshaowswmw2 installs
Quota-aware LLM router that squeezes maximum usable AI out of free-tier API keys across Gemini, Mistral, OpenRouter, Kilo and Cerebras plus any OpenAI-compatible endpoint (including local Ollama/llama.cpp/vLLM). Probes every model on every key, measures real quality and real published rate limits, then routes each request to the cheapest model that can do the job — spending abundant capacity first and reserving scarce daily quota for when it is actually needed. Persists cooldowns to disk so a 429 discovered in one process is respected by the next. Use when an agent must make many LLM calls on free keys without hitting rate limits, when "all models failed", or when deciding which of several provider keys to use for a task.
by orionshaowswmw2 installs
Opt-in, model-neutral guidance for evidence-aware, dignified AI communication, with a compact response contract and offline deterministic text audit. It never injects prompts, edits host configuration, calls networks, reads secrets, or treats heuristics as truth.
by orionshaowswmw2 installs
Iran Chemical Database — live, dated, auditable, BEST-EFFORT index of chemical offerings in configured public Iranian supplier catalogues (websites + public Telegram channels). HTTrack/WooCommerce-REST/Telegram mirroring → local-only parsing → RDKit/PubChem/CAS-validated PostgreSQL with FastAPI + Streamlit. Fail-closed Iranian-suppliers-only country gate; coverage measured and published, never claimed complete. Installation = software + queued crawl, not a populated dataset. Ships a 1399-molecule CID-unique confirmed-organic seed baseline (v2.22, 2026-08-27: v2.19 primary + live Telegram/WooCommerce/sitemap crawl + 5-model fleet normalization, every new identity PubChem-confirmed). For academic procurement research.
by orionshaowswmw3 installs
Model-agnostic, agent-agnostic fidelity-first pipeline converting operator-authorized Persian/English RTL lecture PDFs into offline HTML study guides — recall-first dual OCR (PyMuPDF + Tesseract fas+eng PSM ensemble), rendered-page evidence, multi-model correction, session-grounded enrichment (tables/flashcards/quizzes/mnemonics/summaries/scenarios), measured fidelity, QA gates, verified ZIP. v1.5.0 runs on ANY model family through 8 API dialects (OpenAI, Responses, Gemini, Anthropic, Cohere, Ollama, HuggingFace, offline mock) or with no model at all, auto-discovers providers from the host agent's environment, self-heals provider quirks and model retirements, and exposes one deterministic CLI/MCP entrypoint plus cross-model consensus so different agents reproduce the same intended result.
by orionshaowswmw2 installs