The other day I had the task to move a large amount of data from Oracle to SQL Server.
I had chosen to import the data via SSIS, doing a direct copy from one table in oracle to one table in SQL Server.
Everything was fine until I one of my tables crashed, with a data conversion error.
The issue is that the lowest date that SQL Server will accept is “1-Jan-1753”. There was some bad data in Oracle dated year 1049, consequently SQL server did no know how to handle it. How was I going to handle it?
The solution is to use a transformation script. In your data flow task, add a transformation script between your source and destination. When you drop in on the form it pops up a dialogue, leave it default for transform.
On the inputs column choose the date column in question (for the example lets assume its called CreatedDate).
Make sure you change its “UsageType” in the grid from “Read” to “Read-Write”.
Go to the script tab and click “Design Script” enter something like the following code:
Note: This is in vb .net. I think 2008 R2 allows c#, but you should get the idea:
Try Dim dt As DateTime = Row.CreatedDate() Dim dtMin As DateTime = System.DateTime.Parse("1753-01-01 00:00:00") Dim dtMax As DateTime = System.DateTime.Parse("9999-12-31 23:59:59") If (Row.CreatedDate_IsNull = False) Then If (dt < dtMin Or dt > dtMax) Then If (dt < dtMin) Then Row.CreatedDate() = dtMin Else Row.CreatedDate() = dtMax End If End If End If Catch ex As Exception Row.CreatedDate_IsNull = True End Try
This code will find any row that has a date that falls outside the bounds of what SQL server will accept and set it to the Max or Min date values.
Finally some general tips that I learned in attempting to move large amounts of data from Oracle to SQL.
1. Don’t use a linked server connection it is really slow.
2. The Microsoft OLEDB connector for Oracle is pretty fast.
3. When setting up the OLEDB Destination via “Table or View” make sure you choose the “Fast Load” option!! Why you would want it to run slow is beyond me but the “Table or View – fast load” is about 10 to 15 times faster.