I have multiple .txt files in a folder (file1, file2, file3, file4, file5,....) and I need to run on a large number of files.
I've created this code to reorder and make some changes ... I would like to apply it on multiple txt files from a original folder(path) and save them separately into another folder (inside the last folder) or another path
Can someone please help?
import pandas as pd
import numpy as np
import os
import glob
import datetime
#from datetime import datetime
df1=pd.read_csv("D:\\Spyder2019\\38A.txt", sep='\s+')
#############################################33#
#to change name of column and add Temp ms column
df1=df1.rename(index=str, columns={"Temps":"Heure", "Force":"Force(N)", "Vitesse":"Vitesse(RPM)", "Puissance":"Puissance(w)","Torque":"Torque(N/m)","Angle":"Angle(deg)"})#to change name Temps to Heure
dtemp=df1['Heure']#to change datetime values into seconds and microseconds in Heure
dtemp=pd.to_datetime(df1.Heure) #change to datetime values float
dtemp1=dtemp.dt.microsecond #dtemp1 object in microseconds with 6 decimals
dtempms=dtemp1 / 1000000 #dtemp1 object into 2 decimals
dtemp2=dtemp.dt.second #dtemp2 object in seconds
df1['Temps_ms']= dtempms + dtemp2 #add second and microseconds to the object
#################################################
# to transform the Heure data into sec and microsec
df1=df1[['Date','NoBille','Heure','Temps_ms','Force(N)','Vitesse(RPM)','Puissance(w)','Torque(N/m)','Angle(deg)']]# reorder the dataframe
io=df1.iat[0,3]
#to transform the Heure data into seconds ans microseconds
df1['Temps(ms)'] = np.where(df1['Temps_ms'] - io <0, df1['Temps_ms'] + 60 - io, df1['Temps_ms'] -io)
df1=df1.drop(columns=['Temps_ms'])#to eliminate column Temps_ms
df1=df1[['Date','NoBille','Heure','Temps(ms)','Force(N)','Vitesse(RPM)','Puissance(w)','Torque(N/m)','Angle(deg)']]# to reorder the final dataframe
##############################################################################
print(df1.head())
df1.to_csv('data1.txt', index = False)
If this script can be applied to all databases that you have in your "input folder" you can pass the DB name as argument to python script:
import sys
#...your other imports...
### take the first argument passed in commmand line as db_name and second as output_file name
db_name = sys.argv[1]
output_filename = sys.argv[2]
df1 = pd.read_csv(db_name, sep='\s+')
#...rest of your script...
df1.to_csv(output_filename, index = False)
and then yuo can easily call the python script over the files in the input directory with an external loop, as ex. in bash:
mkdir <output-dir>
ls <input-dir> | while read file; do
output_filename="<output-dir>/$file.output.txt"
python <your-script-name> $file $output_filename
done
and it will automatically take the files, operate the python script and write them down in output-dir with the specified extension
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments