The official PASS Blog is where you’ll find the latest blog posts from PASS community members and the PASS Board. Contributors share their thoughts and discuss a wide variety of topics spanning PASS and the data community.

What’s in my Stage? Automating Data Comparison with Biml

When you build an ETL Solution, at some point you will most likely feel the need to compare the data between your source and your staging (or datawarehouse) database. There may be various reasons for them to be out of sync, such as delta loads, aggregations or added business logic, but one day your phone will ring and whoever is on the other end will tell you that the numbers are wrong. While this doesn’t necessarily have to be an issue within your ETL process, it might just as well be the report itself. In many cases, this is a good starting point to look at.

This article focusses on SSIS as your orchestrator, but the same principles could, obviously, also be applied to Azure Data Factory for example. Also, as we want to solve this task as lightweight as possible, we will, demonstrably, use Biml to implement it!

Read more

Re-thinking Types of Execution Plans

A recent discussion on Twitter about the different types of execution plans - specifically estimated vs. actual - got me thinking about how we “define” plans in SQL Server, and I realized we really need one more type of plan. The estimated plan definition needs to be further refined. Rather than two types of plans, I propose we think about plans as one of three types:

  • The estimated plan 
  • The used plan
  • The actual plan

Let me explain…

Read more

How To Start Troubleshooting SQL Server Performance

As a Solutions Engineer for SentryOne, I get to meet SQL Server data professionals daily and talk to them about the challenges that they face. One central theme among all of these conversations is performance troubleshooting. I’d like to take this opportunity to share some thoughts on how to approach a SQL Server performance issue.

When I was formulating this article, I wondered “what do data professionals do FIRST” when faced with a report of an immediate production issue?

Read more

The Importance of Running for the PASS Board of Directors

It's that time of year again; PASS is looking for new leaders to join the Board. You might be asking why you should consider running for a Board seat when you already have a full-time job and a ton of other commitments. I mean, let's face it, attending Board meetings and taking on the responsibilities of a Board member adds to your already busy calendar.

As a sitting Board member, I can tell you that this is time well spent. I have gleaned valuable insights around patience, organizational financial responsibility, and increased knowledge and passion for the community in which we serve.

Read more

Azure Cognitive Services via Power BI

Text Mining and Sentiment Analysis can provide interesting insights, when used to analyze free form text like social media posts, customer reviews, feedback comments and survey responses. Key phrases extracted from the text are useful to identify trends and popular topic/themes. Sentiment scores provide a way to perform quantitative analysis on this text data. Combining these techniques, using visually engaging dashboards will help unlock the value of your text data.

Read more

Test Driving Always Encrypted with Secure Enclaves in SQL Server 2019

In this article, I’ll walk through the steps for setting up your environment so that you can start working with Always Encrypted using secure enclaves. This is a new security feature in SQL Server 2019 (still in preview at press time) that protects sensitive data from highly privileged but unauthorized users (such as DBAs and machine admins), while preserving SQL Server’s ability to perform rich queries against the very same data.

Read more

Azure Databricks from PyCharm IDE

Azure Databricks is a powerful platform for data pipelines using Apache Spark.  It provides the power of Spark’s distributed data processing capabilities with many features that make deploying and maintaining a cluster easier, including integration to other Azure components such as Azure Data Lake Storage and Azure SQL Database.  If you have tried out tutorials for Databricks you likely created a notebook, pasted some Spark code from the example, and the example ran across a Spark cluster as if it were magic.  Notebooks are useful for many things and Azure Databricks even lets you schedule them as jobs.  But, when developing a large project with a team of people that will go through many versions, many developers will prefer to use PyCharm or another IDE (Integrated Development Environment).  Getting to a streamlined process of developing in PyCharm and submitting the code to a Spark cluster for testing can be a challenge and I have been searching for better options for years.

Read more 1 Comments

Is DataOps More Than Just DevOps?

I’ve been drawn in by hype before, but the experience has me casting a critical eye every time there’s a new trendy buzzword going around. DataOps had that effect on me, I raised eyebrow and laughed when I first heard it at the start of 2018. It started appearing on hype lists later in the year. For me, the easy portmanteau of “Data” and “DevOps” (already a conjunction of Development and Operations) was unnecessary. Surely, whether it applies to data, web dev, app dev or any other kind of coding, the idea of DevOps applies the same.

Read more

The NomCom: Helping Guide the Direction of PASS

The PASS election cycle is fast approaching as we head towards PASS Summit, but before we get there we need to have the Nominations Committee (NomCom) in place. The role of the NomCom is to identify suitable candidates for the PASS Board of Directors election, ensuring they meet the criteria set out in the Ideal Candidate Description. The NomCom itself is made up of PASS community members who review applications and interview the potential candidates ahead of the final slate being sent to the Board of Directors for approval.

Read more

Microsoft and The PASS Board

PASS and Microsoft have enjoyed a strong relationship dating back to PASS’ founding in 1999.

As founding members on the Board, our Microsoft counterparts have not only shared insights into the direction of the Microsoft data platform and the data industry with our organization, but they have also connected us into wider networks which have helped us to grow our community and stay ahead of emerging trends.

Read more

RSS
1234
Back to Top
cage-aids
cage-aids
cage-aids
cage-aids