Mastering: Inference - Complete Overview

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

云微

Nov 1

Can a language model run in Linux eBPF?

#ebpf #llm #kernel #inference

6 min read

Vikas Srivastava

Oct 26

Why Trillion-Parameter Models Need a New Serving Architecture

#inference #cerebras #modelarchitecture

7 min read

amrit

Oct 21

How DeepInfer Slashed Latency by 2.4 and Cut Costs by 45% – The Future of AI on Edge

#ai #inference #deeplearning

5 min read

TechPulse

Oct 21

DeepSeek Open-Sources DeepInfer, Its Production Inference Engine

#deepseek #ai #inference #opensource

2 min read

jamilxt

Oct 20

DeepInfer vs vLLM: DeepSeek's New Engine Is Not a Drop-In Replacement

#ai #inference #devtools #programming

7 min read

Raff Martin (esatapedico)

Oct 28

NVFP4 tiers, triaged: which file for your card, what it costs, what breaks first

#ai #inference #opensource #devto

3 min read

ai maya

Oct 20

Can two Mac Studios really run an 816B model? A bandwidth roofline check on Muse Spark Open

#ai #machinelearning #llm #inference

12 min read

EME GUG

Oct 20

Why LLM Inference Engines Are Fast: Continuous Batching, PagedAttention and Prefix Caching Explained with vLLM

#llm #vllm #inference #performance

7 min read

Reno Lu

Oct 20

Strata runs a 125B model on a 12 GB gaming GPU by leaning on system RAM, and the README spells out the cost

#llm #localai #inference #ai

3 min read

ai maya

Oct 19

Ling-3.1-flash and the 7:1 hybrid: how much KV cache does a 560B model need at 1M tokens?

#ai #machinelearning #llm #inference

10 min read

Hacks.gr

Oct 19

Tenstorrent raises $1,5 billion and doubles its value

#tenstorrent #ai #hardware #inference

1 min read

NIA

Oct 12

The Rounding Error That Stops the Machine

#performance #llm #tokenization #inference

5 min read

RC

Oct 8

AI agents designed the chip that runs their inference

#aihardware #fpga #verification #inference

4 min read

Oct 17

Magnitude: The Apache-2.0 Inference Engine That Claims 2x Faster Than llama.cpp

#ai #tools #inference #opensource

5 min read

Sachin Kr. Rajput

Oct 26

A Forward Pass for One Token: Where the FLOPs Go

#llm #machinelearning #inference #performance

12 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.