How to parse a nested json data file iand insert it into sqlite database using python3

user8720570

I am trying to insert data from the json file into the sqlite database in python.

The json file format is:

   {
"metadata": {
    "start_at": "2016-01-27 20:01:14",
    "end_at": "2016-08-23 20:01:14",
    "act_count": 1
},
"act_data": [{
    "performed_at": "2013-04-24 20:01:14",
    "ticket_id": 531,
    "performer_type": "user",
    "performer_id": 7687,
    "activity": {
        "shipping_address": "N/A",
        "priority": 3,
        "agent_id": 8505,
        "requester": 12559

    }
}, {
    "performed_at": "2012-03-14 20:01:14.340099",
    "ticket_id": 235,
    "performer_type": "user",
    "performer_id": 11829,
    "activity": {
        "note": {
            "id": 22257,
            "type": 1
        }
    }

    }]
}

I want to insert all the columns into the same table.I have already created a table in sqlite database.

I want to populate an additional column "activity_type" as "Note" for the item which has "note" key and others as "ticket".I ran the below code:

import json
import sqlite3

 with open("tickets1.json") as data_file:
    data = json.load(data_file)


 conn = sqlite3.connect('tickets.db')
 c = conn.cursor()

  c.execute(""" CREATE TABLE activitydata1( 
        performed_at VARCHAR(50) NOT NULL,
         ticket_id INTEGER NOT NULL,
         performed_type VARCHAR(10) NOT NULL ,
         performer_id INTEGER NOT NULL,
         acitivity_type VARCHAR(10) NOT NULL,
         note_id INTEGER NULL,
         note_type INTEGER NULL,
        shipping_address   NVARCHAR(50)  NULL,
      shipment_date  NVARCHAR(50)  NULL,
        category  VARCHAR(10)  NULL,
        contacted_customer VARCHAR(10) NULL,
        issue_type VARCHAR(10) NULL,
        source INTEGER NULL,
        status VARCHAR(10) NULL,
        priority INTEGER NULL,
        groups VARCHAR(10) NULL,
        agent_id  INTEGER NULL,
        requester INTEGER NULL,
        product VARCHAR(10) NULL)
  """)

  for record in data["act_data"]:
     for key,val in record["activity"].items():

        if ( key == 'note'):
           activity_type="note"
           c.execute("INSERT INTO activitydata1(performed_at, ticket_id,performed_type,performer_id,acitivity_type,note_id,note_type) VALUES (?,?,?,?,?,?,?)",(record["performed_at"], record["ticket_id"],record["performer_type"],record["performer_id"],activity_type,val["id"],val["type"]))
        else:
           activity_type="ticket"
           c.execute("INSERT INTO activitydata1(performed_at, ticket_id,performed_type,performer_id,acitivity_type,shipping_address,priority,agent_id,requester) VALUES (?,?,?,?,?,?,?,?,?)",(record["performed_at"], record["ticket_id"],record["performer_type"],record["performer_id"],activity_type,record["activity"]["shipping_address"],record["activity"]["priority"],record["activity"]["agent_id"],record["activity"]["requester"]))

conn.commit()
conn.close()

This code inserts four rows of the same ticket and one row of the note. In total 5 rows. The required output is just two rows- One for ticket and other one for note.

Please find the image of the database attached. Please help me correct the code. Please advice some other better approach to handle this problem. Thanks in advance.

Database iamge

DinoCoderSaurus

This for key,val in record["activity"].items(): will execute 4 times because this

"activity": {
        "shipping_address": "N/A",
        "priority": 3,
        "agent_id": 8505,
        "requester": 12559

    }

has four items.

Maybe you just need something like

if "note" in record["activity"]:
    # do the notes thing
else:
    # do the other thing

From the python doc:

x in dictview

Return True if x is in the underlying dictionary’s keys, values or items (in the latter case, x should be a (key, value) tuple).

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

how to insert specific json into sqlite database using python 3

How to parse/extract nested JSON data with Python?

How to parse csv data and insert in database using php codeigniter

How to insert datetime into sqlite3 database using Python?

How to read really LARGE JSON file and insert that file's data into a MYSQL database using node.js?

How to insert data from a list to SQLite3 Database

How to parse a JSON data with JSON object & JSON array from service to a sqlite database in android?

How to Insert JSON data into database

How to insert data from tableWidget into sqlite Database

how to insert xml file data into mongodb database using java?

how to insert .txt file data into database using php?

Insert data into sqlite database

Insert data to database SQLite

Insert data to the sqlite database

How to append data to a nested JSON file in Python

How to parse the json data using python?

Using Codable to parse nested JSON data

How to further parse a nested dictionary in a json file to a dataframe in Python

Insert data into sqlite3 database with API

how do you parse nested json data in python

How to parse data multi row and multi line string and extract data onto JSON file using Python

How to print nested JSON data using Python?

how do you parse nested JSON using python

How to parse a nested json file in typescript (.tsx)?

How to parse nested JSON data structure

how to insert json Data into Sqlite DB in PhoneGap

unable to insert data in database using json

How to insert nested json data in mysql table?

How to parse nested JSON using GSON