InferenceKey

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

Joined on Oct 14, 2026 https://inferencekey.com

InferenceKey for InferenceKey

Nov 2

Shipping Ming-Image Design: size the card by its files, and check the alpha in code

#ai #machinelearning #python #gpu

4 min read

Want to connect with InferenceKey?

Create an account to connect with InferenceKey. You can also sign in below to proceed if you already have an account.

Create Account

Already have an account? Sign in

InferenceKey for InferenceKey

Nov 1

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

#ai #llm #machinelearning #gpu

4 min read

InferenceKey for InferenceKey

Oct 31

Fitting Flint's coding agent on a 24 GB RTX 4090: context is the knob that decides

#ai #llm #gpu #selfhosted

5 min read

InferenceKey for InferenceKey

Oct 30

Serving MOSS-Image: step count, memory mode and the traps that change the words you ship

#ai #machinelearning #gpu #python

4 min read

InferenceKey for InferenceKey

Oct 29

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

#ai #llm #gpu #opensource

5 min read

InferenceKey for InferenceKey

Oct 28

Price your coding agent from its own logs before you self-host DeepInfer

#ai #llm #devops #gpu

5 min read

InferenceKey for InferenceKey

Oct 27

Serving LFM2.5-24B-A2B on one 48 GB GPU: the flags, the traps and what to measure

#ai #llm #gpu #selfhosted

4 min read

InferenceKey for InferenceKey

Oct 26

Building a speech editing API on AuK-Flash: the knobs it ignores and the wording it doesn't

#ai #machinelearning #python #opensource

5 min read

InferenceKey for InferenceKey

Oct 25

Serving a 6 GB ternary 27B model: the settings that decide whether it works

#llm #ai #gpu #selfhosted

4 min read

InferenceKey for InferenceKey

Oct 24

Muse Spark Open vs DeepSeek V4.1 Flash in Claude Code: what to measure before you pick

#ai #llm #opensource #machinelearning

4 min read

InferenceKey for InferenceKey

Oct 23

Self-hosting AstaBrief-8B for cited reports: check every bracket before anyone reads it

#ai #llm #python #opensource

4 min read

InferenceKey for InferenceKey

Oct 22

Three settings that decide whether EmbeddingGemma 2 search works on your GPU

#ai #machinelearning #python #opensource

4 min read

InferenceKey for InferenceKey

Oct 21

Fifty API keys, one queue, ten GPU workers: the parts of a shared LLM that are not the model

#ai #llm #devops #selfhosted

4 min read

InferenceKey for InferenceKey

Oct 20

Fine-tuning Pocket TTS for Castilian Spanish: stop at step 600 and judge the accent by ear

#ai #machinelearning #python #opensource

4 min read

InferenceKey for InferenceKey

Oct 20

Running DeepSeek-V4.1-Flash (763B) on one 96 GB GPU: the three settings that decide your speed

#ai #llm #gpu #opensource

3 min read

InferenceKey for InferenceKey

Oct 19

Serving Iris-3B Turbo: the 42 GB repo, the 21 GB of VRAM and the 8 MB payload limit

#ai #machinelearning #python #opensource

5 min read

InferenceKey for InferenceKey

Oct 17

Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned

#ai #llm #gpu #opensource

3 min read

InferenceKey for InferenceKey

Oct 17

Kandinsky 6.0 in a pipeline: offload, the untimed default and what to measure

#ai #machinelearning #opensource #gpu

4 min read

InferenceKey for InferenceKey

Oct 16

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

#llm #gpu #machinelearning #selfhosted

4 min read

InferenceKey for InferenceKey

Oct 15

Same config.json, different serve command: what to check before serving AREX-2

#ai #llm #machinelearning #gpu

5 min read

InferenceKey for InferenceKey

Oct 14

Three things that break when you point Claude Code at your own model

#ai #llm #selfhosted #opensource

4 min read

DEV Community

InferenceKey

Badges

1 Week Community Wellness Streak

Writing Debut

Organizations

InferenceKey

Shipping Ming-Image Design: size the card by its files, and check the alpha in code

Want to connect with InferenceKey?

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

Fitting Flint's coding agent on a 24 GB RTX 4090: context is the knob that decides

Serving MOSS-Image: step count, memory mode and the traps that change the words you ship

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

Price your coding agent from its own logs before you self-host DeepInfer

Serving LFM2.5-24B-A2B on one 48 GB GPU: the flags, the traps and what to measure

Building a speech editing API on AuK-Flash: the knobs it ignores and the wording it doesn't

Serving a 6 GB ternary 27B model: the settings that decide whether it works

Muse Spark Open vs DeepSeek V4.1 Flash in Claude Code: what to measure before you pick

Self-hosting AstaBrief-8B for cited reports: check every bracket before anyone reads it

Three settings that decide whether EmbeddingGemma 2 search works on your GPU

Fifty API keys, one queue, ten GPU workers: the parts of a shared LLM that are not the model

Fine-tuning Pocket TTS for Castilian Spanish: stop at step 600 and judge the accent by ear

Running DeepSeek-V4.1-Flash (763B) on one 96 GB GPU: the three settings that decide your speed

Serving Iris-3B Turbo: the 42 GB repo, the 21 GB of VRAM and the 8 MB payload limit

Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned

Kandinsky 6.0 in a pipeline: offload, the untimed default and what to measure

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

Same config.json, different serve command: what to check before serving AREX-2

Three things that break when you point Claude Code at your own model