搜索
搜索

Pandas：使用 read_csv 从 StringIO 读取时为空数据框

Benison Sam 发表于 Dev
12
本尼森山姆

以下是StringIO对象值

DebugPoint csv_log_stream 类型

csv_log_stream.getvalue()

原始输出

'"2022-06-04 12:02:40,248",azure_functions_worker,INFO,"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 5af2b92f-7e82-4515-89c5-846737ba3e60,function Name: ProcessWebSaleExportFilesInRSBlobStorage"\n"2022-06-04 12:02:40,252",azure_functions_worker,INFO,"Received FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 0d487a72-2ded-487e-b269-2ac913e3fcebfunction Name: ReadIntegrationInterfaceConfiguration"\n"2022-06-04 12:02:40,259",azure_functions_worker,INFO,"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 0d487a72-2ded-487e-b269-2ac913e3fceb,function Name: ReadIntegrationInterfaceConfiguration"\n"2022-06-04 12:02:40,261",azure_functions_worker,INFO,"Received FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: c42740c8-21fd-4435-a5cb-7b9f74dc7225function Name: SaveLogsToRSBlobStorage"\n"2022-06-04 12:02:40,265",azure_functions_worker,INFO,"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: c42740c8-21fd-4435-a5cb-7b9f74dc7225,function Name: SaveLogsToRSBlobStorage"\n"2022-06-04 12:02:43,000",azure_functions_worker,INFO,"Received FunctionInvocationRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 5af2b92f-7e82-4515-89c5-846737ba3e60, function name: ProcessWebSaleExportFilesInRSBlobStorage, invocation ID: c42bf678-d155-4859-a71a-b0108645080d, function type: sync, sync threadpool max workers: 1000"\n"2022-06-04 12:02:43,007",root,INFO,Python HTTP trigger :: ProcessWebSaleExportFilesInRSBlobStorage function processed a request.\n"2022-06-04 12:02:43,008",root,INFO,Processing Request object started for the desired parameters.\n"2022-06-04 12:02:43,009",root,INFO,Processing Request object completed for the desired parameters.\n"2022-06-04 12:02:43,010",root,INFO,Processing Request object started for the desired parameters.\n"2022-06-04 12:02:43,011",root,INFO,Processing Request object completed for the desired parameters.\n"2022-06-04 12:02:43,041",azure.core.pipeline.policies.http_logging_policy,INFO,"Request URL: \'https://koxdsrssa.blob.core.windows.net/koxds-export?restype=REDACTED&comp=REDACTED&prefix=REDACTED&st=REDACTED&se=REDACTED&sp=REDACTED&sv=REDACTED&sr=REDACTED&sig=REDACTED\'\nRequest method: \'GET\'\nRequest headers:\n    \'x-ms-version\': \'REDACTED\'\n    \'Accept\': \'application/xml\'\n    \'User-Agent\': \'azsdk-python-storage-blob/12.12.0 Python/3.8.12 (Windows-10-10.0.19044-SP0)\'\n    \'x-ms-date\': \'REDACTED\'\n    \'x-ms-client-request-id\': \'79b647c5-e3ed-11ec-8c08-48a4728e3a8b\'\nNo body was attached to the request"\n"2022-06-04 12:02:43,564",azure.core.pipeline.policies.http_logging_policy,INFO,"Response status: 200\nResponse headers:\n    \'Transfer-Encoding\': \'chunked\'\n    \'Content-Type\': \'application/xml\'\n    \'Server\': \'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0\'\n    \'x-ms-request-id\': \'4a6cab6b-e01e-002d-5ffa-77769c000000\'\n    \'x-ms-client-request-id\': \'79b647c5-e3ed-11ec-8c08-48a4728e3a8b\'\n    \'x-ms-version\': \'REDACTED\'\n    \'Access-Control-Expose-Headers\': \'REDACTED\'\n    \'Access-Control-Allow-Origin\': \'REDACTED\'\n    \'Date\': \'Sat, 04 Jun 2022 10:02:43 GMT\'"\n"2022-06-04 12:02:44,070",azure.core.pipeline.policies.http_logging_policy,INFO,"Request URL: \'https://koxdsrssa.blob.core.windows.net/koxds-export/WebSale/Test/2022_06_03_20_13_23_782-0500_c841f873-9a12-4402-a164-5819cbcddc3e_Test_0.json?st=REDACTED&se=REDACTED&sp=REDACTED&sv=REDACTED&sr=REDACTED&sig=REDACTED\'\nRequest method: \'GET\'\nRequest headers:\n    \'x-ms-range\': \'REDACTED\'\n    \'x-ms-version\': \'REDACTED\'\n    \'Accept\': \'application/xml\'\n    \'User-Agent\': \'azsdk-python-storage-blob/12.12.0 Python/3.8.12 (Windows-10-10.0.19044-SP0)\'\n    \'x-ms-date\': \'REDACTED\'\n    \'x-ms-client-request-id\': \'7a5398fe-e3ed-11ec-a414-48a4728e3a8b\'\nNo body was attached to the request"\n"2022-06-04 12:02:44,226",azure.core.pipeline.policies.http_logging_policy,INFO,"Response status: 206\nResponse headers:\n    \'Content-Length\': \'8337358\'\n    \'Content-Type\': \'application/json\'\n    \'Content-Range\': \'REDACTED\'\n    \'Last-Modified\': \'Sat, 04 Jun 2022 01:14:56 GMT\'\n    \'Accept-Ranges\': \'REDACTED\'\n    \'ETag\': \'""0x8DA45C7A2F73E96""\'\n    \'Server\': \'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0\'\n    \'x-ms-request-id\': \'4a6cad87-e01e-002d-3cfa-77769c000000\'\n    \'x-ms-client-request-id\': \'7a5398fe-e3ed-11ec-a414-48a4728e3a8b\'\n    \'x-ms-version\': \'REDACTED\'\n    \'x-ms-creation-time\': \'REDACTED\'\n    \'x-ms-blob-content-md5\': \'REDACTED\'\n    \'x-ms-lease-status\': \'REDACTED\'\n    \'x-ms-lease-state\': \'REDACTED\'\n    \'x-ms-blob-type\': \'REDACTED\'\n    \'Content-Disposition\': \'REDACTED\'\n    \'x-ms-server-encrypted\': \'REDACTED\'\n    \'Access-Control-Expose-Headers\': \'REDACTED\'\n    \'Access-Control-Allow-Origin\': \'REDACTED\'\n    \'Date\': \'Sat, 04 Jun 2022 10:02:44 GMT\'"\n"2022-06-04 12:09:07,090",root,INFO,Total time taken: 6 minutes and 24 seconds\n'

StringIO从到读取pandas.DataFrame：

df_logs = pd.read_csv(csv_log_stream, header=None)

输出：

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "ProjectDir\\.venv\lib\site-packages\pandas\util\_decorators.py", line 311, in wrapper
    return func(*args, **kwargs)
  File "ProjectDir\\.venv\lib\site-packages\pandas\io\parsers\readers.py", line 680, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "ProjectDir\\.venv\lib\site-packages\pandas\io\parsers\readers.py", line 575, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "ProjectDir\\.venv\lib\site-packages\pandas\io\parsers\readers.py", line 933, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "ProjectDir\\.venv\lib\site-packages\pandas\io\parsers\readers.py", line 1235, in _make_engine
    return mapping[engine](f, **self.options)
  File "ProjectDir\\.venv\lib\site-packages\pandas\io\parsers\c_parser_wrapper.py", line 75, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas\_libs\parsers.pyx", line 551, in pandas._libs.parsers.TextReader.__cinit__
pandas.errors.EmptyDataError: No columns to parse from file

上述尝试从StringIO抛出错误中读取 DataFrame。所以，我做了以下并且得到了空的DataFrame。

df_logs = pd.read_csv(csv_log_stream, names=["Timestamp", "LogName", "LogLevel", "LogMessage"])
print(df_logs)

输出：

Empty DataFrame
Columns: [Timestamp, LogName, LogLevel, LogMessage]
Index: []

我无法理解我做错了什么。我的输入StringIO值似乎是正确的。我错过了什么？！

洛朗

可能是您正在调用输出pd.read_csv的字符串而StringIO.getvalue()不是StringIO对象本身：

import pandas as pd
from io import StringIO

file = StringIO(
    "\"2022-06-04 12:02:40,248\",azure_functions_worker,INFO,\"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 5af2b92f-7e82-4515-89c5-846737ba3e60,function Name: ProcessWebSaleExportFilesInRSBlobStorage\"\n\"2022-06-04 12:02:40,252\",azure_functions_worker,INFO,\"Received FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 0d487a72-2ded-487e-b269-2ac913e3fcebfunction Name: ReadIntegrationInterfaceConfiguration\"\n\"2022-06-04 12:02:40,259\",azure_functions_worker,INFO,\"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 0d487a72-2ded-487e-b269-2ac913e3fceb,function Name: ReadIntegrationInterfaceConfiguration\"\n\"2022-06-04 12:02:40,261\",azure_functions_worker,INFO,\"Received FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: c42740c8-21fd-4435-a5cb-7b9f74dc7225function Name: SaveLogsToRSBlobStorage\"\n\"2022-06-04 12:02:40,265\",azure_functions_worker,INFO,\"Successfully processed FunctionLoadRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: c42740c8-21fd-4435-a5cb-7b9f74dc7225,function Name: SaveLogsToRSBlobStorage\"\n\"2022-06-04 12:02:43,000\",azure_functions_worker,INFO,\"Received FunctionInvocationRequest, request ID: 5bc6ee11-9eaa-4479-902a-3e037ac08503, function ID: 5af2b92f-7e82-4515-89c5-846737ba3e60, function name: ProcessWebSaleExportFilesInRSBlobStorage, invocation ID: c42bf678-d155-4859-a71a-b0108645080d, function type: sync, sync threadpool max workers: 1000\"\n\"2022-06-04 12:02:43,007\",root,INFO,Python HTTP trigger :: ProcessWebSaleExportFilesInRSBlobStorage function processed a request.\n\"2022-06-04 12:02:43,008\",root,INFO,Processing Request object started for the desired parameters.\n\"2022-06-04 12:02:43,009\",root,INFO,Processing Request object completed for the desired parameters.\n\"2022-06-04 12:02:43,010\",root,INFO,Processing Request object started for the desired parameters.\n\"2022-06-04 12:02:43,011\",root,INFO,Processing Request object completed for the desired parameters.\n\"2022-06-04 12:02:43,041\",azure.core.pipeline.policies.http_logging_policy,INFO,\"Request URL: 'https://koxdsrssa.blob.core.windows.net/koxds-export?restype=REDACTED&comp=REDACTED&prefix=REDACTED&st=REDACTED&se=REDACTED&sp=REDACTED&sv=REDACTED&sr=REDACTED&sig=REDACTED'\nRequest method: 'GET'\nRequest headers:\n    'x-ms-version': 'REDACTED'\n    'Accept': 'application/xml'\n    'User-Agent': 'azsdk-python-storage-blob/12.12.0 Python/3.8.12 (Windows-10-10.0.19044-SP0)'\n    'x-ms-date': 'REDACTED'\n    'x-ms-client-request-id': '79b647c5-e3ed-11ec-8c08-48a4728e3a8b'\nNo body was attached to the request\"\n\"2022-06-04 12:02:43,564\",azure.core.pipeline.policies.http_logging_policy,INFO,\"Response status: 200\nResponse headers:\n    'Transfer-Encoding': 'chunked'\n    'Content-Type': 'application/xml'\n    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'\n    'x-ms-request-id': '4a6cab6b-e01e-002d-5ffa-77769c000000'\n    'x-ms-client-request-id': '79b647c5-e3ed-11ec-8c08-48a4728e3a8b'\n    'x-ms-version': 'REDACTED'\n    'Access-Control-Expose-Headers': 'REDACTED'\n    'Access-Control-Allow-Origin': 'REDACTED'\n    'Date': 'Sat, 04 Jun 2022 10:02:43 GMT'\"\n\"2022-06-04 12:02:44,070\",azure.core.pipeline.policies.http_logging_policy,INFO,\"Request URL: 'https://koxdsrssa.blob.core.windows.net/koxds-export/WebSale/Test/2022_06_03_20_13_23_782-0500_c841f873-9a12-4402-a164-5819cbcddc3e_Test_0.json?st=REDACTED&se=REDACTED&sp=REDACTED&sv=REDACTED&sr=REDACTED&sig=REDACTED'\nRequest method: 'GET'\nRequest headers:\n    'x-ms-range': 'REDACTED'\n    'x-ms-version': 'REDACTED'\n    'Accept': 'application/xml'\n    'User-Agent': 'azsdk-python-storage-blob/12.12.0 Python/3.8.12 (Windows-10-10.0.19044-SP0)'\n    'x-ms-date': 'REDACTED'\n    'x-ms-client-request-id': '7a5398fe-e3ed-11ec-a414-48a4728e3a8b'\nNo body was attached to the request\"\n\"2022-06-04 12:02:44,226\",azure.core.pipeline.policies.http_logging_policy,INFO,\"Response status: 206\nResponse headers:\n    'Content-Length': '8337358'\n    'Content-Type': 'application/json'\n    'Content-Range': 'REDACTED'\n    'Last-Modified': 'Sat, 04 Jun 2022 01:14:56 GMT'\n    'Accept-Ranges': 'REDACTED'\n    'ETag': '\"\"0x8DA45C7A2F73E96\"\"'\n    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'\n    'x-ms-request-id': '4a6cad87-e01e-002d-3cfa-77769c000000'\n    'x-ms-client-request-id': '7a5398fe-e3ed-11ec-a414-48a4728e3a8b'\n    'x-ms-version': 'REDACTED'\n    'x-ms-creation-time': 'REDACTED'\n    'x-ms-blob-content-md5': 'REDACTED'\n    'x-ms-lease-status': 'REDACTED'\n    'x-ms-lease-state': 'REDACTED'\n    'x-ms-blob-type': 'REDACTED'\n    'Content-Disposition': 'REDACTED'\n    'x-ms-server-encrypted': 'REDACTED'\n    'Access-Control-Expose-Headers': 'REDACTED'\n    'Access-Control-Allow-Origin': 'REDACTED'\n    'Date': 'Sat, 04 Jun 2022 10:02:44 GMT'\"\n\"2022-06-04 12:09:07,090\",root,INFO,Total time taken: 6 minutes and 24 seconds\n"
)


df = pd.read_csv(
    file,
    names=[
        "Timestamp",
        "LogName",
        "LogLevel",
        "LogMessage",
    ],
)

print(df)
# Output
                  Timestamp                                           LogName  \
0   2022-06-04 12:02:40,248                            azure_functions_worker
1   2022-06-04 12:02:40,252                            azure_functions_worker
2   2022-06-04 12:02:40,259                            azure_functions_worker
3   2022-06-04 12:02:40,261                            azure_functions_worker
4   2022-06-04 12:02:40,265                            azure_functions_worker
5   2022-06-04 12:02:43,000                            azure_functions_worker
6   2022-06-04 12:02:43,007                                              root
7   2022-06-04 12:02:43,008                                              root
8   2022-06-04 12:02:43,009                                              root
9   2022-06-04 12:02:43,010                                              root
10  2022-06-04 12:02:43,011                                              root
11  2022-06-04 12:02:43,041  azure.core.pipeline.policies.http_logging_policy
12  2022-06-04 12:02:43,564  azure.core.pipeline.policies.http_logging_policy
13  2022-06-04 12:02:44,070  azure.core.pipeline.policies.http_logging_policy
14  2022-06-04 12:02:44,226  azure.core.pipeline.policies.http_logging_policy
15  2022-06-04 12:09:07,090                                              root

   LogLevel                                         LogMessage
0      INFO  Successfully processed FunctionLoadRequest, re...
1      INFO  Received FunctionLoadRequest, request ID: 5bc6...
2      INFO  Successfully processed FunctionLoadRequest, re...
3      INFO  Received FunctionLoadRequest, request ID: 5bc6...
4      INFO  Successfully processed FunctionLoadRequest, re...
5      INFO  Received FunctionInvocationRequest, request ID...
6      INFO  Python HTTP trigger :: ProcessWebSaleExportFil...
7      INFO  Processing Request object started for the desi...
8      INFO  Processing Request object completed for the de...
9      INFO  Processing Request object started for the desi...
10     INFO  Processing Request object completed for the de...
11     INFO  Request URL: 'https://koxdsrssa.blob.core.wind...
12     INFO  Response status: 200\nResponse headers:\n    '...
13     INFO  Request URL: 'https://koxdsrssa.blob.core.wind...
14     INFO  Response status: 206\nResponse headers:\n    '...
15     INFO         Total time taken: 6 minutes and 24 seconds

动态访问 StringIO 对象的内容时要小心，这与使用原始字符串不同。

这是一个具有相同“文件”对象的示例：

file.seek(20000)  # Change the stream position to the given byte offset.

df = pd.read_csv(
    file,
    names=[
        "Timestamp",
        "LogName",
        "LogLevel",
        "LogMessage",
    ],
)

print(df)
# Output
Empty DataFrame
Columns: [Timestamp, LogName, LogLevel, LogMessage]
Index: []

然而：

file.seek(0)  # Change the stream position to the beginning of the file

df = pd.read_csv(
    file,
    names=[
        "Timestamp",
        "LogName",
        "LogLevel",
        "LogMessage",
    ],
)

print(df)
# Ouput
   LogLevel                                         LogMessage  
0      INFO  Successfully processed FunctionLoadRequest, re...  
1      INFO  Received FunctionLoadRequest, request ID: 5bc6...  
2      INFO  Successfully processed FunctionLoadRequest, re...  
3      INFO  Received FunctionLoadRequest, request ID: 5bc6...  
4      INFO  Successfully processed FunctionLoadRequest, re...  
5      INFO  Received FunctionInvocationRequest, request ID...  
6      INFO  Python HTTP trigger :: ProcessWebSaleExportFil...  
7      INFO  Processing Request object started for the desi...  
8      INFO  Processing Request object completed for the de...  
9      INFO  Processing Request object started for the desi...  
10     INFO  Processing Request object completed for the de...  
11     INFO  Request URL: 'https://koxdsrssa.blob.core.wind...  
12     INFO  Response status: 200\nResponse headers:\n    '...  
13     INFO  Request URL: 'https://koxdsrssa.blob.core.wind...  
14     INFO  Response status: 206\nResponse headers:\n    '...  
15     INFO         Total time taken: 6 minutes and 24 seconds

本文收集自互联网，转载请注明来源。

如有侵权，请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章

当使用带有 StringIO 而不是文件对象的 pandas read_csv 时，峰值内存使用量要大得多

StringIO和熊猫read_csv

熊猫-使用read_csv读取时间

pandas 是否支持使用 read_csv 读取 `set` 参数

在 Pandas 中使用 read_csv 时忽略双引号 (")

使用 pandas read_csv 时随机打印笑脸？

使用正则表达式 sep 时，Pandas read_csv 将 NaN 读取为“”

无法从 Pandas read_csv 正确读取数据

有没有办法在不创建文件的情况下使用 read_csv ？使用 StringIO

使用熊猫read_csv读取标题时跳过`＃`字符

熊猫-使用read_csv从Generator对象创建数据框

使用.read_csv读取csv文件的所有行

Pandas read_csv()：列包含数据帧但被读取为字符串

如何使用 StringIO 从内存中将数据帧作为 CSV 发送

Python Pandas read_csv 引用问题，无法分离数据

如何使用Pandas read_csv解析Cloud SQL CSV导出空值（“ N”）

在read_csv之后在pandas数据框中选择列时出现关键错误

如果将csv存储为变量，如何使用pandas read_csv（）方法？

使用read_csv读取奇数/数字列

在熊猫中使用read_csv时精度下降

使用熊猫的read_csv时如何设置变量

熊猫：使用read_csv时如何包含双引号？

如何使用带有gzip压缩选项的pandas read_csv读取tar.gz文件？

在Pandas中使用UseCols时，以指定的顺序排列列Read_CSV

使用read_csv将财务数据导入Python Pandas

阻止Pandas read_csv将NA解释为NaN，但保留NaN为空值

Pandas read_csv fillna

如何显示pandas read_csv()函数读取的csv文件名？

熊猫：使用read_csv时如何获取已读取行的状态？

TOP 榜单

  1. 1

    Linux的官方Adobe Flash存储库是否已过时？

  2. 2

    用日期数据透视表和日期顺序查询

  3. 3

    应用发明者仅从列表中选择一个随机项一次

  4. 4

    Java Eclipse中的错误13，如何解决？

  5. 5

    在Windows 7中无法删除文件（2）

  6. 6

    在 Python 2.7 中。如何从文件中读取特定文本并分配给变量

  7. 7

    套接字无法检测到断开连接

  8. 8

    带有错误“ where”条件的查询如何返回结果？

  9. 9

    有什么解决方案可以将android设备用作Cast Receiver？

  10. 10

    Mac OS X更新后的GRUB 2问题

  11. 11

    ggplot：对齐多个分面图-所有大小不同的分面

  12. 12

    验证REST API参数

  13. 13

    如何从视图一次更新多行（ASP.NET - Core）

  14. 14

    尝试反复更改屏幕上按钮的位置 - kotlin android studio

  15. 15

    计算数据帧中每行的NA

  16. 16

    检索角度选择div的当前值

  17. 17

    离子动态工具栏背景色

  18. 18

    UITableView的项目向下滚动后更改颜色，然后快速备份

  19. 19

    VB.net将2条特定行导出到DataGridView

  20. 20

    蓝屏死机没有修复解决方案

  21. 21

    通过 Git 在运行 Jenkins 作业时获取 ClassNotFoundException

热门标签

归档