selenium - Why multiply the text when I am trying web-scraping a table with Python? -


i trying web-scraping table website, , fine, run without error, when open in csv, see there multiple web-scraping: text+table, when need table 1 web-scraping.

enter image description here

the table start 53. row , don't understand that. why code web-scraping text , not table?

my code:

from bs4 import beautifulsoup selenium import webdriver import time import unicodecsv csv  filename = r'output.csv'  resultcsv = open(filename, "wb") output = csv.writer(resultcsv, delimiter=';', quotechar='"',                     quoting=csv.quote_nonnumeric, encoding='latin-1')  profile = webdriver.firefoxprofile() profile.set_preference("intl.accept_languages", "en-us") driver = webdriver.firefox(firefox_profile=profile) driver.get("https://www.flightradar24.com/data/airports/bud/arrivals") time.sleep(10) html_source = driver.page_source soup = beautifulsoup(html_source, "html.parser") print(soup)  table = soup.find('table', { "class" : "table table-condensed table-hover data-table m-n-t-15" })  datatable = [] record in table.find_all('tr'):     temp_data = []     data in record.find_all('td'):         temp_data.append(data.text.encode('latin-1'))     datatable.append(temp_data)  output.writerows(datatable)  resultcsv.close() 

your table contains these lines in tr tags, that's why append line want.

you need filter class of tags expect, in case should work :

for record in table.find_all('tr', class_="hidden-xs hidden-sm ng-scope"):     temp_data = []     data in record.find_all("td"):         temp_data.append(data.text.encode('latin-1'))     datatable.append(temp_data) 

Comments

Popular posts from this blog

html - How to set bootstrap input responsive width? -

javascript - Highchart x and y axes data from json -

javascript - Get js console.log as python variable in QWebView pyqt -