我有以下HTML
<p>
<b>Father:</b> Michael Haughton
<br>
<b>Mother:</b> Diane
<br><b>Brother:</b>
Rashad Haughton<br>
<b>Husband:</b> <a href="/people/540/000024468/">R. Kelly</a> (m. 1994, annulled that same year)
<br><b>Boyfriend:</b> <a href="/people/420/000109093/">Damon Dash</a> (Roc-a-Fella co-CEO)<br></p>
我必须将标题和文本分开，例如Mother：Diane ..
所以最后我将得到一个字典列表，如下所示：
[{"label":"Mother","value":"Diane"}]
我正在尝试以下内容，但无法正常工作：
def parse(u):
u = u.rstrip('\n')
r = requests.get(u, headers=headers)
if r.status_code == 200:
html = r.text.strip()
soup = BeautifulSoup(html, 'lxml')
headings = soup.select('table p')
for h in headings:
b = h.find('b')
if b is not None:
print(b.text)
print(h.text + '\n')
print('=================================')
url = 'http://www.nndb.com/people/742/000024670/'
from bs4 import BeautifulSoup
from urllib.request import urlopen
#html = '''<p>
#<b>Father:</b> Michael Haughton
#<br>
#<b>Mother:</b> Diane
#<br><b>Brother:</b>
#Rashad Haughton<br>
#<b>Husband:</b> <a href="/people/540/000024468/">R. Kelly</a> (m. 1994, annulled that same year)
#<br><b>Boyfriend:</b> <a href="/people/420/000109093/">Damon Dash</a> (Roc-a-Fella co-CEO)<br></p>'''
page = urlopen('http://www.nndb.com/people/742/000024670/')
source = page.read()
soup = BeautifulSoup(source)
needed_p = soup.find_all('p')[8]
bs = needed_p.find_all('b')
res = {}
for b in bs:
if b.find_next('a').text:
res[b.text] = b.find_next('a').text.strip().strip('\n')
if b.next_sibling != ' ':
res[b.text] = b.next_sibling.strip().strip('\n')
res
输出：
{'Brother:': 'Rashad Haughton',
'Mother:': 'Diane',
'Husband:': 'R. Kelly',
'Father:': 'Michael Haughton',
'Boyfriend:': 'Damon Dash'}
编辑：有关页面顶部的其他信息：
... (code above) ...
soup = BeautifulSoup(source)
needed_p = soup.find_all('p')[1:4] + [soup.find_all('p')[8]] # here explicitly selecting needed p-tags for further parsing
res = {}
for p in needed_p:
bs = p.find_all('b')
for b in bs:
if b.find_next('a').text:
res[b.text] = b.find_next('a').text.strip().strip('\n')
if b.next_sibling != ' ':
res[b.text] = b.next_sibling.strip().strip('\n')
res
输出：
{'Race or Ethnicity:': 'Black',
'Husband:': 'R. Kelly',
'Died:': '25-Aug',
'Nationality:': 'United States',
'Executive summary:': 'R&B singer, died in plane crash',
'Mother:': 'Diane',
'Birthplace:': 'Brooklyn, NY',
'Born:': '16-Jan',
'Boyfriend:': 'Damon Dash',
'Sexual orientation:': 'Straight',
'Occupation:': 'Singer',
'Cause of death:': 'Accident - Airplane',
'Brother:': 'Rashad Haughton',
'Remains:': 'Interred,',
'Gender:': 'Female',
'Father:': 'Michael Haughton',
'Location of death:': 'Marsh Harbour, Abaco Island, Bahamas'}
对于此页面，您还可以例如通过以下方式抓取高中：
res['High School'] = soup.find_all('p')[9].text.split(':')[1].strip()
本文收集自互联网，转载请注明来源。
如有侵权，请联系 [email protected] 删除。
我来说两句