如何使用beautifulsoup從腳本標籤內部抓取數據？

灰燼

我試圖從網站的腳本中提取一些數據。我能夠從整個頁面獲取數據，但是我被困在如何從腳本標籤內部獲取數據。這是我如何得到湯。

import requests
from bs4 import BeautifulSoup

searchf = input("Enter the product you want: ")

url =f"https://www.daraz.com.np/catalog/?q={searchf}&_keyori=ss&from=input&spm=a2a0e.searchlist.search.go.12ec45adZnn1MP"

r = requests.get(url)
contents = r.content

soup = BeautifulSoup(contents,'html.parser')
print(soup.prettify())

這是我想要抓取的網站，我只想獲取那裡列出的產品的名稱。https://www.daraz.com.np/catalog/?q=cars&_keyori=ss&from=input&spm=a2a0e.searchlist.search.go.3b834ce4JPBUHm

法茲魯

數據也由 javacript 動態加載。轉到網絡選項卡 > xhr/fetch> 標頭，然後您將看到 api url，如果您單擊預覽選項卡，您將看到數據，總項目數為 4080，每個頁麵包含 40 個項目，我使用查詢字符串參數作為與 api url 一起發送的參數。

代碼：

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36'}

api_url = 'https://www.daraz.com.np/catalog/'

params = {
    'keyori': 'ss',
    'ajax': 'true',
    'from': 'input',
    #'page': '2',
    'q': 'cars',
    'spm': 'a2a0e.searchlist.search.go.3b834ce4JPBUHm'
    
    }

for num in range(0, 4080, 40):
    params['totalResults'] = num
    print(num)
    resp = requests.get(api_url, params=params, headers=headers).json()

    items = resp['mods']['listItems']
    for item in items:
        name = item['name']
        print(name)

輸出：

Hot wheels 3 Pack Cars
XimiVogue  Pull-Back-And-Go Alloy Suv Car Toy With Sound (Bmw Mini)
Kids Spider Stunt Flip Car
Funny Diy Electric Thomas Train Truck Set Toy Set Orange
Monster Trucks Inertia Car Toys For Kids
Remote Control Transformer Car
Remote Control Emulation Car
Spiderman  Friction Stunt Car
Remote Control Car
360 Degree Spinning Rolling stunt Farm Truck Super Tipping Car for Kids
Monster Trucks Friction Powered Inertia Cars Toys For Kids (Color May Vary)
Gallant Toy Bus-Set Of 2
Remote Control Transformer Car
19026B Express Train Set - Battery Operated Black Train Toys for Kids Aged 3+ Years
Friction Powered Monster Truck Toy (Multicolor) Pack Of 1
Toy Train
Model Concept R/C Car
Friction Powered 3-Pack Mini Push and Go Car Truck Jam Playset
Makes Your Kids Happy -  Remote Control Vehicle Car For Kids
2.4Ghz 1/18 Rc Rock Crawler Vehicle Buggy Car 4 Wd Shaft Drive High Speed Remote Control 4X4 Monster Off Road Truck
XimiVogue  Pull-Back-And-Go Alloy Suv Car Toy With Sound
Kids Spider Stunt Flip Car Small Size
Black ForMula Remote Control Car For Kids