Python 3.10: I cannot extract text from an element with Selenium 4.8.0

Dimitris02

I'm trying to extract the description from a youtube channel:

driver.get(channel_url+"/about")
try:
    myElem = WebDriverWait(driver, 3).until(lambda d: d.find_element("xpath","//*[@id=\"description\"]"))
    print(myElem.text)
except TimeoutException: print("failed")

This is what the HTML code looks like:

<yt-formatted-string id="description" split-lines="" class="style-scope ytd-channel-about-metadata-renderer">
Text I want to extract
</yt-formatted-string>

I am using a gecko Firefox driver

The .text and get_attribute("textContent") commands both return nothing. The get_attribute("innerHTML") command returns an irrelevent <path></path> element.

I copied the xpath from my browser using Inspect.

Bibhav

Youtube has 3-4 elements having id "description" (I dont know why.")
You can find it if you search with ctrl+f in developers tools or using the method find_all()
So try using full xpath as such:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait

channel_url = "https://www.youtube.com/@MrBeast"
driver = webdriver.Chrome()
driver.get(channel_url+"/about")
try:
    myElem = WebDriverWait(driver, 3).until(lambda d: d.find_element("xpath",'/html/body/ytd-app/div[1]/ytd-page-manager/ytd-browse/ytd-two-column-browse-results-renderer/div[1]/ytd-section-list-renderer/div[2]/ytd-item-section-renderer/div[3]/ytd-channel-about-metadata-renderer/div[1]/div[1]/yt-formatted-string[2]'))
    print(myElem.text)
except TimeoutException: print("failed")
driver.quit()

Or use the proper CSS selector pointing only to the single element:
(Obviously, you can try same way with xpath too)

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait

channel_url = "https://www.youtube.com/@MrBeast"
driver = webdriver.Chrome()
driver.get(channel_url+"/about")
try:
    myElem = WebDriverWait(driver, 3).until(lambda d: d.find_element(By.CSS_SELECTOR,'#description-container > #description'))
    print(myElem.text)
except TimeoutException: print("failed")
driver.quit()

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

I cannot extract the text from an element using ElementTree

Python: Selenium cannot click XPath text element

Selenium Python - Extract Text from Class

Extract text from webpage using Selenium in Python

Selenium: Python cannot find element from elif

Cannot extract text from xml in python

How to extract the text from the element using Selenium through Java

How to extract a table element from html source of a web page using selenium phantomJS in python 3?

Python Selenium - How to extract element based on text inside span tag?

How to extract the text within the element using Selenium WebDriver and Python?

How to locate an element and extract required text with Selenium and Python

Python selenium get text from href element

python selenium get text from element

Printing Text from Element Selenium Python

Cannot find table element from div element in selenium python

Cannot get text of element with Selenium

How i extract text from a model dialog in selenium?

How can I extract the text from a webelement using selenium

python - extract img src address from selenium div element

PYTHON + SELENIUM (CHROME): How can I extract a specific text from my current url and use the extracted text to go to another?

Extract text from class 'bs4.element.Tag' beautifulsoup

How can I extract a specific text in an html code with Selenium and Python

python selenium, i can't find element class or id from a text box

I need a way to get a Random number (0-10) from an array then get the following 2 subsequent #'s. Such as 7,8,9. 3,4,5 etc

Python Selenium text extract list output from for loop

How to extract text from svg using python selenium

How to extract the text from the HTML using Selenium and Python

How to extract the text $7.56 from the webpage using Selenium through Python

Python/Selenium - how to extract text from modal fade content?