psonlinex
  Post Categories
Distributed Systems 1 Data Engineering 5 Observability 2 Distributed Architecture 2 API Gateway 1 Frontend Engineering 1 Backend Architecture 3 Backend Development 1 Cloud Native 1 Architecture Design 1
Building a Type-Safe OLAP Query Layer with Haskell and Solid.js for Rendering Massive Datasets Building a Type-Safe OLAP Query Layer with Haskell and Solid.js for Rendering Massive Datasets
We were facing a thorny problem: the business side needed a highly flexible data exploration interface capable of on-the
2023-10-27 Data Engineering
Data Warehouse Solid.js Haskell OLAP Full-stack
Building a Real-Time CDC Ingestion Gateway from SQL Server to a Data Lake with Go Echo Building a Real-Time CDC Ingestion Gateway from SQL Server to a Data Lake with Go Echo
Our team maintains a core business system with SQL Server as its OLTP database. For a long time, our data analysis needs
2023-10-27 Data Engineering
Go Echo Data Lake & Warehouse SQL Server CDC Debezium
Building a Real-time Feature Store with Automated Quality Monitoring using GitOps and Pulsar Building a Real-time Feature Store with Automated Quality Monitoring using GitOps and Pulsar
The initial motivation for building a machine learning feature store is to solve the classic problem of inconsistency be
2023-10-27 Data Engineering
Feature Store Pulsar Code Review Seaborn Cypress
Orchestrating External Concurrency Control for Delta Lake with Pulumi and ZooKeeper Orchestrating External Concurrency Control for Delta Lake with Pulumi and ZooKeeper
The primary bottleneck our team faced on a core data platform wasn’t compute or storage, but metadata management. Specif
2023-10-27 Data Engineering
Pulumi ZooKeeper Delta Lake IaC Distributed Locking
Building an Exactly-Once MongoDB-to-Snowflake Flink Pipeline with Test-Driven Development Building an Exactly-Once MongoDB-to-Snowflake Flink Pipeline with Test-Driven Development
We faced a thorny engineering problem: a core business system uses MongoDB as its production database, featuring a highl
2023-10-27 Data Engineering
Snowflake CDC MongoDB Apache Flink TDD