Forum

 
ForumForumDiscussionsDiscussionsSQLBI Methodolo...SQLBI Methodolo...SQLBI Methodology - Staging and Debugging: remarkSQLBI Methodology - Staging and Debugging: remark
Previous Previous
 
Next
 Disabled
New Post
 12/16/2009 3:16 PM
 

I personally enjoyed reading the article and found it very interesting, as I am just starting my professional career in BI.

I have however one small remark regarding a statement about debugging in the Staging chapter:

"Temporary tables are a fundamental part of the debugging of the ETL phase. When we detect an error, we will often see it in the OLAP cube. More often, the user will tell us that a specific number is incorrect when compared with his sources of data. What we need to do is to go back in the ETL pipeline in order to detect where we are computing that specific value and check the code. If we do not have temporary tables, we will not be able to verify the output of each step of the pipeline in order to find the wrong point in our project. We can only look at the boxes and arrows inside the data flow and think to what is happening there. On the other side, temporary tables will let us check the outputs of each step until we find the wrong one. We can correct it and – having the intermediate tables"

Since you are using the Microsoft BI Suite, why not use the data viewers in SSIS? The grid option lets you see the data in a particular flow between two components just as a temporary table would. This saves you the time and effort of creating those temporary tables for debugging and a data viewer is easily added or removed.

New Post
 12/27/2009 9:42 AM
 

Using SSIS Data Viewers requires you to run a SSIS package with BIDS, in debug mode.

When you run the package in production, very often you don't have BIDS, and/or don't have access to BIDS. Moreover, the DBA might not have the skills to open and use BIDS to look at possible causes of a problem in data.

We use SSIS Data Viewers during development of a SSIS package, but when you are in production and the ETL phase consumes 1-2 hours, having an alternative way to look for errors is an invaluable help.

As many others ideas we wrote in the paper, this one come from experience on the field... we started to adopt this when we found that relying only on SSIS Data Viewers were not flexible in some scenarios...

Marco

Previous Previous
 
Next
 Disabled
ForumForumDiscussionsDiscussionsSQLBI Methodolo...SQLBI Methodolo...SQLBI Methodology - Staging and Debugging: remarkSQLBI Methodology - Staging and Debugging: remark