Qwen3.8 Flash Next FP8 on 4x CMP 170HX: PP3 transformer stages with the 51B PLE table on GPU 3
-
Updated
Aug 30, 2026 - Python
#
gpu-offload
Here are 7 public repositories matching this topic...
用消费级显卡在本地跑动 MoE 架构 30B~35B 大模型：显存与内存混合承担（非专家权重上显存、占大头的专家权重留内存），速度远超纯 CPU；显存不够或没有独显时纯 CPU 兜底。免装 CUDA/Python/任何运行库，模型可替换。
windows amd nvidia moe quantization avx512 non-commercial expo memory-bandwidth ryzen ddr5 cpu-inference llama-cpp local-llm gguf openai-compatible qwen3 llm-tuning zen5 gpu-offload
-
Updated
Oct 16, 2026 - C++
Measured report + drop-in harness for running llama.cpp with the Vulkan backend on Android/Adreno (Turnip). GPU offload cuts CPU time 6.6x while token generation stays flat. Raw benchmark data, offload ladder, and a corrections log.
android benchmark vulkan gpu-acceleration termux turnip agents snapdragon qualcomm adreno mesa edge-ai on-device-ai llm llama-cpp ggml local-llm llm-inference qwen gpu-offload
-
Updated
Oct 1, 2026 - Shell
Experimental portable GGML GPU offload for diffuse-cpp across Vulkan, Metal, and CUDA
-
Updated
Aug 18, 2026 - C++
Run LLM2Vec (Llama-3-8B) on an 8 GB GPU via RAM offload, and batch it without changing the embeddings (padding shifts them: cosine 0.76 vs 0.999956)
pytorch batching padding inference-optimization huggingface text-embeddings llama-3 llm2vec low-vram gpu-offload
-
Updated
Aug 30, 2026 - Python
Proof repo: run a real LLM fully on-device in React Native (Expo + llama.rn). Offline, no cloud, sub-2s cold start.
react-native flutter rag npu edge-ai on-device-ai mobile-ai llm llama-cpp genai offline-ai private-ai gpu-offload
-
Updated
Jul 25, 2026 - TypeScript
Add this topic to your repo
To associate your repository with the gpu-offload topic, visit your repo's landing page and select "manage topics."