Tag: data engineering
-
Pursuit of Canonical Data: Addressing Misconceptions
In a world where data drives numerous businesses, we aim to establish sources of truth (sometimes called canonical data) characterized by correctness and reliability. In my view, companies like Spotify, Amazon, Meta, and Google are fundamentally data companies. While their customer offerings vary, their business objective remains consistent: to connect users with what they seek.… Read more
-
Dataform Explored: Harnessing Its Power and Addressing Opportunities for Improvements
In an earlier post, I spotlighted Dataform’s transformative capabilities, emphasizing its potential to reshape the data transformation and pipelining landscape for teams. Like all sophisticated tools, a deeper examination reveals areas where refinement could enhance the user experience. In this piece, I’ll share the challenges I’ve encountered and suggest improvements to augment Dataform’s effectiveness. Areas… Read more
-
My Dive into MLOps: A Data Engineer’s Perspective
Yesterday, on the recommendation of a colleague/friend, I took my first step into the Machine Learning Engineering for Production (MLOps) Specialization. It wasn’t a random decision but a reflection of where I see the future of data engineering and machine learning converging. Lessons from the Past Throughout my time as a Data Engineer, I’ve tackled… Read more
-
Why I Love My Job as a Data Engineer
I love my job as a Data Engineer. From my start as a Business Intelligence Analyst to my evolution into a Data Engineer, I’ve had the privilege of expanding my skills through exciting projects. I heard it is hard to find a job you love, and I find myself lucky. My career began as a… Read more
-
Google Dataplex: An Introduction and Its Advantages
While exploring Google’s tools recently, I bumped into something called Google Dataplex. As most of us know, data plays a huge role in today’s businesses. Every day, big and small companies are making and using tons of data. But with all this data flowing in and out, handling it becomes a big challenge. That’s where… Read more