Read a csv file that has a JSON column in SSIS?

ibexy

I have the following CSV file that has 4 columns. The last column addresses holds 2 addresses history in a JSON format. I have tried to read it in SSIS but it splits the JSON along with the comma(,) instead of grouping all the addresses under one column.

I am using a flat-file connector for this. Is there any other source component for this type of content? How can I parse this in SSIS so that there are just 4 columns and the addresses appear all under one column?

id,title,name,addresses
J44011,Mr,James,"{""address_line_1"": 45, ""post_code"": ""XY7 10PG""},{""address_line_1"": 15, ""post_code"": ""AB7 1HG""}"
KeithL

You can use a script component to process the JSON into its own detail table.

I created the following dataflow:

enter image description here

Here are the steps to the script component:

On inputs add ID and Address columns:

enter image description here

On inputs and outputs: add a new output and create columns (remember to program the datatypes:

enter image description here

The script:

public class Addresses
{
    public int address_line_1 { get; set; }
    public string post_code { get; set; }
}

public override void Input0_ProcessInputRow(Input0Buffer Row)
{
    //Test if addresses exist, if not leave the Row processing
    if (string.IsNullOrEmpty(Row.addresses)) return;        

    //Fix Json to make it an array of objects
    string json = string.Format("[{0}]", Row.addresses);

    //Load into an array of Addressses
    Addresses[] adds = new System.Web.Script.Serialization.JavaScriptSerializer().Deserialize<Addresses[]>(json);

    //Process the array
    foreach (var a in adds)
    {
        rowsAddressesBuffer.AddRow();
        rowsAddressesBuffer.ID = Row.id;
        rowsAddressesBuffer.Address1 = a.address_line_1;
        rowsAddressesBuffer.PostalCode = a.post_code;
    }
}

Notes:

  1. The class added to store results.
  2. The JSON had to be fixed to create an array of objects.
  3. You need to add a reference to System.Web.Extensions.
  4. This goes to the load. Make sure text qualifier is defined as a double quote (")

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related