selenium - Why multiply the text when I am trying web-scraping a table with Python? -
i trying web-scraping table website, , fine, run without error, when open in csv, see there multiple web-scraping: text+table, when need table 1 web-scraping.
the table start 53. row , don't understand that. why code web-scraping text , not table?
my code:
from bs4 import beautifulsoup selenium import webdriver import time import unicodecsv csv filename = r'output.csv' resultcsv = open(filename, "wb") output = csv.writer(resultcsv, delimiter=';', quotechar='"', quoting=csv.quote_nonnumeric, encoding='latin-1') profile = webdriver.firefoxprofile() profile.set_preference("intl.accept_languages", "en-us") driver = webdriver.firefox(firefox_profile=profile) driver.get("https://www.flightradar24.com/data/airports/bud/arrivals") time.sleep(10) html_source = driver.page_source soup = beautifulsoup(html_source, "html.parser") print(soup) table = soup.find('table', { "class" : "table table-condensed table-hover data-table m-n-t-15" }) datatable = [] record in table.find_all('tr'): temp_data = [] data in record.find_all('td'): temp_data.append(data.text.encode('latin-1')) datatable.append(temp_data) output.writerows(datatable) resultcsv.close()
your table contains these lines in tr tags, that's why append line want.
you need filter class of tags expect, in case should work :
for record in table.find_all('tr', class_="hidden-xs hidden-sm ng-scope"): temp_data = [] data in record.find_all("td"): temp_data.append(data.text.encode('latin-1')) datatable.append(temp_data)
Comments
Post a Comment