News
Entertainment
Science & Technology
Life
Culture & Art
Hobbies
News
Entertainment
Science & Technology
Culture & Art
Hobbies
How To Migrate From SQL Server To Snowflake - Seattle Data Guy
Over the past three years our teams have noticed a pattern. Many companies looking to migrate to the cloud go from SQL Server to Snowflake. There are many reasons this makes sense. One of the reasons and common benefits was that teams found it far easier to manage that SQL Server and in almost every… Read more
What Is BigQuery And How Do You Load Data Into It? - Seattle Data Guy
If you work in data, then you’ve likely used BigQuery and you’ve likely used it without really thinking about how it operates under the hood. On the surface BigQuery is Google Cloud’s fully-managed, serverless data warehouse. It’s the Redshift of GCP except we like it a little more. The question becomes, how does it work?… Read more
How To Set Up Your Data Infrastructure In 2025 – Part 1 - Seattle Data Guy
Planning out your data infrastructure in 2025 can feel wildly different than it did even five years ago. The ecosystem is louder, flashier, and more fragmented. Everyone is talking about AI, chatbots, LLMs, vector databases, and whether your data stack is “AI-ready.” Vendors promise magic, just plug in their tool and watch your insights appear.… Read more
Best Automation Tools In 2025 for Data Pipelines, Integrations, and More - Seattle Data Guy
Since I started working in tech, one goal that kept coming up was workflow automation. Whether automating a report or setting up retraining pipelines for machine learning models, the idea was always the same: do less manual work and get more consistent results. But automation isn’t just for analytics. RevOps teams want to streamline processes… Read more
Alternatives to Talend – How To Migrate Away From Talend For Your Data Pipelines - Seattle Data Guy
Data integration is critical for organizations of all sizes and industries—and one of the leading providers of data integration tools is Talend, which offers the flagship product Talend Studio. In 2023, Talend was acquired by Qlik, combining the two companies’ data integration and analytics tools under one roof. In January 2024, Talend discontinued Talend Open… Read more
What Is PDFMiner And Should You Use It - How To Extract Data From PDFs - Seattle Data Guy
PDF files are one of the most popular file formats today. Because they can preserve the visual layout of documents and are compatible with a wide range of devices and operating systems, PDFs are used for everything from business forms and educational material to creative designs. However, PDF files also present multiple challenges when it… Read more
The Basics of SFTP: Authentication, Encryption, and File Management - Seattle Data Guy
If you’re looking to pass hundreds of GBs of data quickly, you’re likely not going to use a REST API. That’s why every day, companies share data sets of users, patient claims, financial transactions, and more via SFTP. If you’ve been in the industry for a while, you’ve probably come across automated SFTP jobs that… Read more
Alternatives to Azure Document Intelligence Studio: Exploring Powerful Document Analysis Tools - Seattle Data Guy
Document Intelligence Studio is a data extraction tool that can pull unstructured data from diverse documents, including invoices, contracts, bank statements, pay stubs, and health insurance cards. The cloud-based tool from Microsoft Azure comes with several prebuilt models designed to extract data from popular document types. However, you can also use labeled datasets to train… Read more
Preparing Your Data Infrastructure for 2025: Lessons from the Past, Strategies for the Future - Seattle Data Guy
When I broke into the data world, everyone wanted to hire data scientists that would let their companies become more data driven. There were statistics about the exabytes of data that we were creating and the value it would provide. However, a few years into my career, the data world started to make a pivot… Read more
Challenges You Will Face When Parsing PDFs With Python - How To Parse PDFs With Python - Seattle Data Guy
Scraping data from PDFs is a right of passage if you work in data. Someone somewhere always needs help getting invoices parsed, contracts read through, or dozens of other use cases. Most of us will turn to Python and our trusty list of Python libraries and start plugging away. Of course, there are many challenges… Read more
From IC to Data Leader: Key Strategies for Managing and Growing Data Teams - Seattle Data Guy
There are plenty of statistics about the speed at which we are creating data in today’s modern world. On the flip side of all that data creation is a need to manage all of that data and thats where data teams come in. But leading these data teams is challenging and yet many new data… Read more
What is Unstructured Data? A Guide to Storage, Processing, and Analysis - Seattle Data Guy
Much of the data we have used for analysis in traditional enterprises has been structured data. It’s easy for humans to break down, understand, and, in turn, find insights from it. However, much of the data that is being created and will be created comes in some form of unstructured format. However, the digital era… Read more
How Data Teams Drive Business Success by Understanding Core Metrics - Seattle Data Guy
A key responsibility for any data team is to understand the core metrics driving their business. Starting from the top, these metrics often include figures like gross revenue and expenses. However, these high-level metrics can feel too far removed and abstract from the actual business. Many companies, therefore, break down these top-line metrics into more… Read more
What Is AWS DMS And Why You Shouldn't Use It As An ELT - Seattle Data Guy
Recently, I’ve encountered a few projects that used AWS DMS, which is almost like an ELT solution. Whether it was moving data from a local database instance to S3 or some other data storage layer. It was interesting to see AWS DMS used in this manner. But it’s not what DMS was built for. As… Read more
9 Must-Watch Videos for Aspiring Data Leaders: Bridging Tech and Business for Data Team Success - Seattle Data Guy
Leading data teams can be challenging. You’ve got management and non-technical teams constantly reaching out with ad-hoc data requests; you’re likely trying to figure out what tools will work best and not blow the bank. Not to mention, you’ve got to bridge the gap between business and technology. All while trying to grow your data… Read more
How To Automate PDF Data Extraction - 3 Different Methods To Parse PDFs For Analytics - Seattle Data Guy
I.f you work in data, then at some point in your career, you’ll likely need to parse data from a PDF. You might need to parse thousands of PDFs in order to pull out invoice information. Or maybe you need to parse financial filing documents such as 10-Ks. This can seem challenging at first. Afterall,… Read more
How To Modernize Your Data Strategy And Infrastructure For 2025 - Seattle Data Guy
We are still in the early days of data and the value it can add to companies. You’ll read plenty of statistics about how much value data can drive and how far behind companies that aren’t using data are. And as a data consultant, I have helped companies find that value in their data. It… Read more
Real-time Analytics Vs Stream Processing - What Is The Difference? - Seattle Data Guy
One of the holy grails that many data teams seem to chase is real-time data analytics. After all, if you can have real-time analytics, you can make better decisions faster. However, there often is a conflation between real-time data analytics and stream processing. These are two different concepts that are crucial to understanding how to… Read more
Essential Skills for Data Engineers in the Age of AI - Seattle Data Guy
If you work in data, then AI is everywhere at this point. But whether AI is hype or reality doesn’t change the fact that data engineers will play a major role in ensuring that the data sets that are utilized for the growing use cases are usable both by machines and humans. Whether that data… Read more
How To Run A Data Team As A New Head Of Data - Seattle Data Guy
What would you do if you became the head or director of data for a 1,000-person company? Yesterday, you were plugging along as an analyst, and now, suddenly, you have all these new responsibilities. Figuring out where to start is part of the job. You’d probably feel a strong temptation to freak out. Who wouldn’t?… Read more
9 Habits Of Effective Data Managers - Running A Data Team - Seattle Data Guy
Running a successful data team is hard. Data teams are expected to juggle a combination of ad-hoc requests, big bet projects, migrations, etc. All while keeping up with the latest changes in technology. In the past few years I have gotten to work with dozens of teams and see how various directors and managers deal… Read more
How To Data Model - Real Life Examples Of How Companies Model Their Data - Seattle Data Guy
How companies data model varies widely. They might say they use Kimball dimensional modeling. However, when you look in their data warehouse the only part you recognize is the word fact and dim. Over the past near decade, I have worked for and with different companies that have used various methods to capture this data.… Read more
Why Data Analysts And Engineers Make Great Consultants - Seattle Data Guy
Many data engineers and analysts don’t realize how valuable the knowledge they have is. They’ve spent hours upon hours learning SQL, Python, how to properly analyze data, build data warehouses, and understand the differences between eight different ETL solutions. Even what they might think is basic knowledge could be worth $10,000 to $100,000+ for a… Read more
4 ELT Alternatives To Airbyte - How To Ingest Your Data - Seattle Data Guy
Getting data out of source systems and into a data warehouse or data lake is one of the first steps in making it usable by analysts and data scientists. The question is how will your team do that? Will they write custom data connectors, pay for a data connector out of the box or perhaps… Read more
Terms You Should Know If You're Planning To Use Change Data Capture - Seattle Data Guy
If you’ve worked in data long enough, then you’ve likely come across the term change data capture. Often called CDC, change data capture involves tracking and recording changes in a database as they happen, and then transmitting these changes to designated targets. This can be crucial because some pipelines, in particular batch pipelines, don’t capture… Read more
Apache Spark Vs Apache Flink - What Is The Difference? - Seattle Data Guy
As data increased in volume, velocity, and variety, so, in turn, did the need for tools that could help process and manage those larger data sets coming at us at ever faster speeds. As a result, frameworks such as Apache Spark and Apache Flink became popular due to their abilities to handle big data processing… Read more
10 Great Videos To Help You Learn Data Engineering - Seattle Data Guy
How data is structured, managed and processed will continue to grow in importance as the demand for AI and machine learning increase. It’s unavoidable that as businesses demand that their data teams implement AI, they will also realize that data engineers are a crucial piece of the data pipeline. That means, if you’re looking for… Read more
Common Pitfalls of Data Analytics Projects - Seattle Data Guy
Have you ever been part of a data or software project that seems stuck in a loop? Three weeks have passed, and although you arrive at work daily, exhausted, having tackled numerous issues, the project remains stagnant. Why? Then, suddenly, a new engineer or project manager steps in, reorganizes and prioritizes tasks, and just like… Read more
Apache Druid's Architecture - How Druid Processes Data In Real Time At Scale - Seattle Data Guy
Recently, I wrote an article diving into what Druid is and which companies are using it. Now I wanted to do a deeper dive into Apache Druid’s architecture. Apache Druid has several unique features that allow it to be used as a real-time OLAP. Everything from its various nodes and processes that each have unique… Read more
5 Real-Time Data Processing and Analytics Technologies - And Where You Can Implement Them - Seattle Data Guy
No matter your industry, you’ll often need to make split-second business decisions in the digital age. Real-time data can help you do just that. It’s information that’s made available as soon as it’s created, meaning you don’t need to wait around for the insights you need. Real-time data processing can satisfy the ever-increasing demand for… Read more
Alternatives to SSIS(SQL Server Integration Services) - How To Migrate Away From SSIS - Seattle Data Guy
SQL Server Integration Services (SSIS) comes with a lot of functionality useful for extracting, transforming, and loading data. It can also play important roles in application development and other projects. But SSIS is far from the only platform that can provide these services. You might seek alternatives to SSIS because you want a more agile… Read more
Why Your Team Needs To Implement Data Quality For Your AI Strategy - Seattle Data Guy
Companies that range from start-ups to enterprises are looking to implement AI and ML into their data strategy. With that it’s important not to forget about data quality. Regardless of how fancy or sophisticated a company’s AI model might be, poor data quality will break it. It will make the outputs of these models useless… Read more
Data Warehousing Essentials: A Guide To Data Warehousing - Seattle Data Guy
Photo by Tiger Lily Data warehouses and data lakes play a crucial role for many businesses. It gives businesses access to the data from all of their various systems. As well as often integrating data so that end-users can answer business critical questions. But if we take a step back and only focus on the… Read more
Cutting Your Data Stack Costs: How To Approach It And Common Issues - Seattle Data Guy
I once had an engineer tell me that they essentially didn’t want to consider cost as they were building a solution. I was baffled. Don’t get me wrong, yes, when you’re building, you iterate and aim to improve your solutions cost. But from my perspective, I don’t think completely ignoring costs from day one is… Read more
7 Great Embedded Analytics Solutions - Which Embedded Analytics Solutions Should You Use? - Seattle Data Guy
Big data is big business these days. Organizations that hope to get ahead in crowded markets must utilize data from a variety of often highly disparate sources to understand how they’re performing and what customers are saying about them. However, data without the right analysis and reporting tools is just a waste of digital storage… Read more
How To Plan To Data Roadmap For 2024 - Elevating Your Data Strategy - Seattle Data Guy
It’s that time of year again. When data leaders, VPs and Directors need to start planning out their data roadmap. Of course, this brings up an important question, how should you start planning out your data roadmap? Especially if you’re data team has found itself stuck in the data service trap. Simply providing data and… Read more
Finding The Right ETL/ELT Solution - What Is Estuary And Should You Use It? - Seattle Data Guy
Data warehousing would be easy if all data were structured and formatted in the data source. Maybe we wouldn’t even need to build a data warehouse. But as anyone who has worked with data from more than one source knows, that’s rarely the case. Businesses today need to pull data from a plethora of sources,… Read more
Common Pitfalls in Deploying Airflow for Data Teams - Seattle Data Guy
If you’re a data engineer, then you’ve likely at least heard of Airflow. Apache Airflow is one of the most popular open-source workflow orchestration solutions that gets used for data pipelines. This is what spurred me to write the article “Should You Use Airflow” because there are plenty of people who don’t enjoy Airflow or… Read more
Apache Druid: Who's Using It and Why? - Seattle Data Guy
Image Source: Druid The past few decades have increased the need for faster data. Some of the catalysts were the push for better data and decisions to be made around advertising. In fact, Adtech has driven much of the real-time data technologies that we have today. For example, Reddit uses a real-time database to provide… Read more
How To Set Up Your Data Analytics Team For Success - Centralized vs Decentralized vs Federated Data Teams - Seattle Data Guy
Success in the data world hinges on team setup. I’ve delved into onboarding and standards in previous articles, but never into the structure of data teams. Typically, there are three configurations: Centralized, Decentralized, and Federated. Most companies I’ve seen use a mix of these. While the newest tech breakthroughs grab headlines, team organization is the… Read more