Skip to main content
Updated · 1h ago
READ · choose how deep
TECH ValidatedⓘEvidence tiers: Early signal = 1 uncorroborated request — treat as radar, not a validated opportunity · Corroborated = 2-3 requests · Validated = 4+ requests from 2+ different people. Every count appears exactly once, in the fact bar below — each traces to a real source post.

Local LLM Model Selection

22
builder posts asking
latest ask seen 1d ago
+40 product/news posts tracked · 1 event tracked
see the posts →
22
different people asking
across 1 platform
who →
15
rivals shipping · web scan
how verified →

In their words

"I'm just looking for real performance numbers: * tok/s at 256K / 262K * stack used: llama.cpp / vLLM * ROCm version / patches, if relevant I’ve seen \~10 tok/s at 256K for AWQ/FP8, but I’m trying to know whether Q6 + MTP changes that" — a builder, verbatim · sign in free to trace the source ↗

What they tried & dropped

Quoted from the posts below — each quote is checked word-for-word against the stored post.

nomic 1.5 — "results weren't all that satisfactory" verbatimsign in to trace ↗
llama.cpp — "stick with llama.cpp for running it or not" verbatimsign in to trace ↗
Llama.cpp Tensor Parallelism RPC — "scaling down to about 17 tps by 100k context" verbatimsign in to trace ↗
RTX 3090 — "a RTX 3090 or 2 RTX 3060" verbatimsign in to trace ↗
llama.cpp/GGUF — "using llama.cpp/GGUF" verbatimsign in to trace ↗

Pain, in numbers

Only where a builder stated a figure — quoted word-for-word from the post.

“roughly 80 thousand of them”80 thousand stories

“I get about 28 tps on a fresh session”28 tps on a fresh session

“scaling down to about 17 tps by 100k context”17 tps by 100k context

“I get about 24-26 tps on the single 5090”24-26 tps on the single 5090

Hard requirements

Summarised from the posts below. Miss one and they will not switch.

8GB VRAMsmall to medium sizerun comfortably across a wide range of systemssmaller modelGPU VRAM constraintGPU RAM constraint

Competitor pricing

Named rivals & real prices 🔒
Rivals charge $697/mo · names — log in ↗

How we checked

Where we searched
3 sources · GitHub (open-source check) · App Store · SaaS marketplaces
Verification funnel
86 raw matches scanned, then AI-de-duped · see the breakdown below
Last scan
6d ago · auto-refreshed every 30 days

Sign in to see the full opportunity

Every source post · named rivals with real prices · what builders tried & dropped · quantified pain — all traceable to the original posts

Sign up free →

What the numbers suggest

15 rivals shipping (fact bar) how verified → market is crowded; differentiate on the unmet requirements above, don't compete on breadth.
22 builder posts asking · 22 different people (fact bar) see posts → demand is independently reported, not one person repeating themselves.
12 new tracked posts (all types) in the last 30 days vs 14 the month before see the curve → demand is cooling.
22 demand signals vs 15 shipping rivals → demand and supply are balanced — differentiation decides.
0 willingness-to-pay quotes yet → no one has named a price — validate pricing before building.
last new signal 1 day ago → conversation is live now.

Signal history

62 tracked posts (22 asks · 40 product/news) first post Apr 2026 ▲ 12 in the last 30 days
2026-04-23 · +1 post · 1 on record2026-04-27 · +1 post · 2 on record2026-05-04 · +1 post · 3 on record2026-06-02 · +1 post · 4 on record2026-06-06 · +1 post · 5 on record2026-06-13 · +2 posts · 7 on record2026-06-14 · +1 post · 8 on record2026-06-17 · +1 post · 9 on record2026-06-21 · +1 post · 10 on record2026-06-29 · +1 post · 11 on record2026-07-02 · +2 posts · 13 on record2026-07-27 · +3 posts · 16 on record2026-07-29 · +1 post · 17 on record2026-08-02 · +3 posts · 20 on record2026-08-09 · +1 post · 21 on record2026-08-14 · +1 post · 22 on record2026-08-15 · +3 posts · 25 on record2026-08-19 · +1 post · 26 on record2026-08-21 · +1 post · 27 on record2026-08-24 · +1 post · 28 on record2026-08-27 · +2 posts · 30 on record2026-08-28 · +2 posts · 32 on record2026-08-29 · +1 post · 33 on record2026-08-31 · +1 post · 34 on record2026-09-02 · +1 post · 35 on record2026-09-04 · +1 post · 36 on record2026-09-07 · +2 posts · 38 on record2026-09-08 · +2 posts · 40 on record2026-09-10 · +1 post · 41 on record2026-09-19 · +1 post · 42 on record2026-09-20 · +1 post · 43 on record2026-09-22 · +1 post · 44 on record2026-09-27 · +1 post · 45 on record2026-09-28 · +1 post · 46 on record2026-09-30 · +1 post · 47 on record2026-10-01 · +3 posts · 50 on record2026-10-06 · +1 post · 51 on record2026-10-11 · +1 post · 52 on record2026-10-12 · +1 post · 53 on record2026-10-14 · +1 post · 54 on record2026-10-19 · +1 post · 55 on record2026-10-26 · +2 posts · 57 on record2026-10-27 · +2 posts · 59 on record2026-10-31 · +2 posts · 61 on record2026-11-02 · +1 post · 62 on record 62 0 Apr 2026 today

Cumulative tracked posts (asks + products + news) by original post date · GapMine tracked communities — not search volume · every step traces to a real post in the evidence record (sign in to read them).

Evidence pool

Top 10 of the 62 tracked posts, picked as evidence — each insight links back to a real one.

10 reddit

Log in free to read every original post ↗

Sign up to save

More in TECH