Moderation

InferenceKey

Nov 1

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

#ai #llm #machinelearning #gpu

4 min read

InferenceKey

Oct 31

Fitting Flint's coding agent on a 24 GB RTX 4090: context is the knob that decides

#ai #llm #gpu #selfhosted

5 min read

InferenceKey

Oct 30

Serving MOSS-Image: step count, memory mode and the traps that change the words you ship

#ai #machinelearning #gpu #python

1

11

4 min read

InferenceKey

Oct 29

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

#ai #llm #gpu #opensource

2

5 min read

InferenceKey

Oct 28

Price your coding agent from its own logs before you self-host DeepInfer

#ai #llm #devops #gpu

5 min read

InferenceKey

Oct 27

Serving LFM2.5-24B-A2B on one 48 GB GPU: the flags, the traps and what to measure

#ai #llm #gpu #selfhosted

4 min read

InferenceKey

Oct 26

Building a speech editing API on AuK-Flash: the knobs it ignores and the wording it doesn't

#ai #machinelearning #python #opensource

5 min read

InferenceKey

Oct 25

Serving a 6 GB ternary 27B model: the settings that decide whether it works

#llm #ai #gpu #selfhosted

2

4 min read

DEV Community

InferenceKey

Meet the team

Serving a 1B math model on vLLM: the memory cap sets how many answers fit, not how fast

Fitting Flint's coding agent on a 24 GB RTX 4090: context is the knob that decides

Serving MOSS-Image: step count, memory mode and the traps that change the words you ship

Nemotron 3.5 Bolt on one GPU: what the 24 GB covers, the drafter flag and the llama.cpp trap

Price your coding agent from its own logs before you self-host DeepInfer

Serving LFM2.5-24B-A2B on one 48 GB GPU: the flags, the traps and what to measure

Building a speech editing API on AuK-Flash: the knobs it ignores and the wording it doesn't

Serving a 6 GB ternary 27B model: the settings that decide whether it works