Decided to teach myself how to use it, I can now do simple data loads which was part of the point.

However something that has dawned on me is can I dedupe using it.

Question is as follows, I have a database, and I have an excel spreadsheet. The spreadsheet contains accounts that need to be loaded into the database. Some of the accounts are already in the database.

Obviously! column headers in the two do not match by name, but are matchable by content.

So far I have successfully loaded the relevant table and the spreadsheet in to two separate tables in my play database and successfully written SQL that identifies the duplicates. But can I take the data from the database and from the excel spreadsheet and rather than do the load then dedupe, do the dedupe as part of the loading process?

Can anyone offer some pointers as to what I should be looking to do? If it matters on Server 2008 R2

Does any of that make sense?

Sorry, don’t know SSIS very well so this isn’t really the answer you’re looking but… this would quite doable using Qlikview – it’s shithot for ETL.

Free full personal edition available here

Try using a Lookup Transformation in between your source and destinations

Lookup it the values are numeric (ie comparing a customer id), or fuzzy lookup for text values.

Change the output of the lookup to redirect the row to the “No match” instead of fail

I’ve always found SQL Server Central useful, try the Stairway to Integration Services

You may need to register first.

thanks, found a tutorial for fuzzy lookup, done a bit of tweaking and now time to wander off while it does its thing.