Complete Guide: Inference - Complete Overview

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Sachin Kr. Rajput

Nov 2

The Roofline Model for LLM Inference

#llm #machinelearning #inference #performance

9 min read

Nov 2

Should You Rent AI or Run Your Own?

#machinelearning #inference #cost

6 min read

云微

Nov 1

Can a language model run in Linux eBPF?

#ebpf #llm #kernel #inference

6 min read

BytePenny

Nov 2

Scaling local AI: from one server to a cluster

#llm #machinelearning #inference #ai

5 min read

Vikas Srivastava

Oct 26

Why Trillion-Parameter Models Need a New Serving Architecture

#inference #cerebras #modelarchitecture

7 min read

amrit

Oct 21

How DeepInfer Slashed Latency by 2.4 and Cut Costs by 45% – The Future of AI on Edge

#ai #inference #deeplearning

5 min read

TechPulse

Oct 21

DeepSeek Open-Sources DeepInfer, Its Production Inference Engine

#deepseek #ai #inference #opensource

2 min read

jamilxt

Oct 20

DeepInfer vs vLLM: DeepSeek's New Engine Is Not a Drop-In Replacement

#ai #inference #devtools #programming

7 min read

Raff Martin (esatapedico)

Oct 28

NVFP4 tiers, triaged: which file for your card, what it costs, what breaks first

#ai #inference #opensource #devto

3 min read

ai maya

Oct 20

Can two Mac Studios really run an 816B model? A bandwidth roofline check on Muse Spark Open

#ai #machinelearning #llm #inference

12 min read

EME GUG

Oct 20

Why LLM Inference Engines Are Fast: Continuous Batching, PagedAttention and Prefix Caching Explained with vLLM

#llm #vllm #inference #performance

7 min read

Reno Lu

Oct 20

Strata runs a 125B model on a 12 GB gaming GPU by leaning on system RAM, and the README spells out the cost

#llm #localai #inference #ai

3 min read

ai maya

Oct 19

Ling-3.1-flash and the 7:1 hybrid: how much KV cache does a 560B model need at 1M tokens?

#ai #machinelearning #llm #inference

10 min read

Hacks.gr

Oct 19

Tenstorrent raises $1,5 billion and doubles its value

#tenstorrent #ai #hardware #inference

1 min read

NIA

Oct 12

The Rounding Error That Stops the Machine

#performance #llm #tokenization #inference

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.