More recently I have been writing little bit more about data migration techniques and best practices. I have to say, it is a point where we all make mistakes. If you don’t map migrate the data properly, whatever the solution you may deliver, will not work. Especially when the current work of the client is dependent on existing data. If your CRM solution is not working, then you will get the blame no matter how good your solution is.
In one of the on going projects we had to create considerable amount of SSIS packages to upload and generate various files. Some times we encounter a scenario where the Control Flow view gets corrupted. All the packages were in Source Control with TFS2010. Couldn’t find the exact reason for this. But found a way to correct the issues. Following is a sample image of the corrupted Control flow of the package.
Because of this issue we can’t do further amendments to the control flow as it is not responding at this stage. At one stage it appears that we had to create the package from the scratch. It is a nightmare to do so for a complex ETL package. I found a workaround to overcome this issue and the source code being checked in to the Source Control helped me a lot. Following are the steps to resolve the problem.
I was using Powershell scripting to create automated deployment scripts for SSIS packages and believe me even though Powershell is a great scripting language it was quite difficult get a script syntactically correct and error free for first time. To overcome this difficulty I used the Powershell ISE (Integrated Scripting Environment). It gives some form of aid to my scripting needs, but I have to say not much. As my self and the team were battling with tight sprint dead lines, these automated scripts writing took considerable amount of time. This was affecting the completion of sprints and one reason for increasing the remaining work at the end of sprints. It was a nightmare to manage the scripts scattered with the team.
When I was reading through my emails I got this email from Tech Republic, of course I’m tech republic fan, regarding this Powershell extension for Visual Studio 2010. So I read it and got the components installed. First you need to install the PowerGUI, otherwise the Visual Studio extension will not work.