使用硒进行网页搜集数据

要打印的innerText元素，您必须为其诱导WebDriverWait，visibility_of_element_located()并且可以使用以下两种定位策略之一：

• 使用CSS选择器和get_attribute("innerHTML")：

  • 中国：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul.list-breadcrumb li:nth-child(3) a"))).get_attribute("innerHTML"))
    

  • 2016年超级联赛：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "h1.wrap-section__header__title>a"))).get_attribute("innerHTML"))
    

  • 北京国安：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul.list-details>li:first-child h2.list-details__item__title>a"))).get_attribute("innerHTML"))
    

  • Henan Jianye:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "ul.list-details>li:nth-child(3) h2.list-details__item__title>a"))).get_attribute("innerHTML"))
    

• 使用xpath和text属性：

  • 中国：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//ul[@class='list-breadcrumb']//following::li[3]//a"))).text)
    

  • 2016年超级联赛：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h1[@class='wrap-section__header__title']/a"))).text)
    

  • 北京国安：

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//ul[@class='list-details']//following::li[1]//h2/a"))).text)
    

  • Henan Jianye:

    print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//ul[@class='list-details']//following::li[2]//h2/a"))).text)
    

  • 注意：您必须添加以下导入：

    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    

您可以在如何使用Selenium检索WebElement的文本中找到相关的讨论-Python

其他

链接到有用的文档：

• get_attribute() 方法 Gets the given attribute or property of the element.
• text 属性返回 The text of the element.
• 使用Selenium的文本和innerHTML之间的区别