Mongoexport blank fields to csv

Chris

I'm exporting a series of mongo collections to csv files. I'm explicitly setting the field names to export by calling a dictionary, if a field does not exist in a given document, I would still like to create an empty field in the csv column

I'm currently getting these errors in my console: too many positional arguments: ['pfrm_uid', 'mark']]

This occurs for all collections. I find it most strange that there are 2 closing brackets in the error.

for item_name in self.exported_fields_Dict.keys():

    fields_to_export = self.exported_fields_Dict[item_name]
    self.item_count_dict[item_name] = self.item_count_dict[item_name] + 1

    os.system("mongoexport --host localhost 
                           --collection {0} 
                           --db scrapy-mongodb-test 
                           --type=csv 
                           --out {1}.csv
                           --fields {2}".format(
                                  item_name,
                                  item_name,
                                  fields_to_export))

Below is the dictionary I pulling the collection name and field names from

// collection name : fields_to_export
exported_fields_Dict = {
       'ent_pfrm': ['pfrm_uid', 'mark'],
       'ent_indv': ['indv_uid', 'fName', 'lName', 'gender', 'DOB'],
       'ent_meet': ['meet_uid', 'name', 'start_date', 'end_date']
    }

Any tips / suggestions appreciated!

thanasisp

mongoexport script expects comma separated fields as a string for the fields argument.

You can test and see that this is not such a string:

> print(exported_fields_Dict['ent_pfrm'])
['pfrm_uid', 'mark']

you can either store strings as your dict values, e.g. "pfrm_uid, mark" or substitute the last argument inside the format method with

",".join( fields_to_export[item_name] )

this prints what you want: pfrm_uid,mark

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related