DEV Community

#dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Proving exactly-once by killing the stream mid-backlog
tantn89 profile

Proving exactly-once by killing the stream mid-backlog

#dataengineering #spark #iceberg #kafka
Comments 3
8 min read
Spark Is Lazy, and That One Fact Explains Most Beginner Confusion
k_hunnurjirao_7ea28e971f profile

Spark Is Lazy, and That One Fact Explains Most Beginner Confusion

#pyspark #dataengineering #beginners #python
1
Comments
5 min read
Your premium preview audit may be measuring a probe cap
dataspryva profile

Your premium preview audit may be measuring a probe cap

#datascience #python #analytics #dataengineering
Comments
7 min read
Un pipeline de clima del Perú con dbt, DuckDB y GitHub Actions, sin AWS
abel_fernandopacompiaor profile

Un pipeline de clima del Perú con dbt, DuckDB y GitHub Actions, sin AWS

#dataengineering #python #dbt #githubactions
Comments
6 min read
CDC Schema Changes: What Breaks and How to Ship Them
pipelinewhiteboard profile

CDC Schema Changes: What Breaks and How to Ship Them

#dataengineering #kafka #snowflake #database
Comments
7 min read
Bridging Python and Rust: Building Type-Safe Data Quality Pipelines with PyO3 and JSON Schema 2020-12 #rust #python #pyo3 #jsonschema
rajgahoi profile

Bridging Python and Rust: Building Type-Safe Data Quality Pipelines with PyO3 and JSON Schema 2020-12 #rust #python #pyo3 #jsonschema

#dataengineering #performance #python #rust
Comments
9 min read
Bridging the Lakehouse: Ingesting Databricks Unity Catalog into Cognee Knowledge Graphs
pranjul_chaurasiya profile

Bridging the Lakehouse: Ingesting Databricks Unity Catalog into Cognee Knowledge Graphs

#databricks #ai #dataengineering #python
1
Comments
4 min read
Don't treat Fabric data ingestion as a one-size-fits-all problem
aniketsoni profile

Don't treat Fabric data ingestion as a one-size-fits-all problem

#microsoftfabric #dataengineering #analytics #azure
1
Comments
5 min read
AWS + Snowflake: Building Modern Data and AI Workflows with Glue, S3, Iceberg, and Cortex
omayrashraf profile

AWS + Snowflake: Building Modern Data and AI Workflows with Glue, S3, Iceberg, and Cortex

#aws #snowflake #dataengineering #cloud
Comments
7 min read
CDC Deletes, Tombstones and the Idempotent MERGE
pipelinewhiteboard profile

CDC Deletes, Tombstones and the Idempotent MERGE

#dataengineering #kafka #snowflake #sql
Comments
7 min read
LLM Text Classification: 4 JSON Schema Invariants for Reliable CRM Tags
jorisrhodes8286 profile

LLM Text Classification: 4 JSON Schema Invariants for Reliable CRM Tags

#ai #llm #jsonschema #dataengineering
Comments
7 min read
Tools for Identifying Network Timeout Causes in Failure Alert APIs — Four Signals
yukikobayashi880 profile

Tools for Identifying Network Timeout Causes in Failure Alert APIs — Four Signals

#observability #distributedsystems #dataengineering #devtools
Comments
5 min read
How to Build a Modern Data Strategy in the Age of AI
revos_ai profile

How to Build a Modern Data Strategy in the Age of AI

#dataengineering #ai #architecture #analytics
1
Comments 1
7 min read
5 Free AI and Data Courses You Can Start Today (Verified Oct 2026)
jathurshan_sownthararasa_ profile

5 Free AI and Data Courses You Can Start Today (Verified Oct 2026)

#freecourses #ai #mcp #dataengineering
Comments
3 min read
Pharaohs of Egypt: a validated dataset from Wikidata, built like a data pipeline
eng_omaressam_7cb0e2a9c profile

Pharaohs of Egypt: a validated dataset from Wikidata, built like a data pipeline

#dataengineering #python #datascience #opendata
Comments
3 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.