Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code only extracts header but doesn't extract data #11

Open
map06 opened this issue Jan 7, 2024 · 2 comments
Open

Code only extracts header but doesn't extract data #11

map06 opened this issue Jan 7, 2024 · 2 comments

Comments

@map06
Copy link

map06 commented Jan 7, 2024

I need to extract data for weather stations across Asia. I tried installing and running it as said in the README file. However, for some reason, the code is not extracting the data but is only extracting the headers. I don't get any error.

@Karlheinzniebuhr
Copy link
Owner

What are the links you added to the file? Did you make sure there is data available on those stations in the date range you provided ?

@map06
Copy link
Author

map06 commented Jan 7, 2024

For example one of the link is
https://www.wunderground.com/dashboard/pws/IGURUG1

This is my settings,

from datetime import date

# Set Date format like: YYYY, MM, DD
# Note that FIND_FIRST_DATE uses START_DATE as default start date
START_DATE = date(2018, 1, 1)
END_DATE = date(2020, 7, 19)

# set to "metric" or "imperial"
UNIT_SYSTEM = "metric"
# UNIT_SYSTEM = "imperial"

# Automatically find first date where data is logged
FIND_FIRST_DATE = True
                          

When I run:
python3 weather_scraper.py:
It prints:

** Initializing binary search to find the first date with data **
low is 0 - 2018-01-01
high is 2197 - 2024-01-07
mid is 1098 - 2021-01-03
----//----
low is 1099 - 2021-01-04
high is 2197 - 2024-01-07
mid is 1648 - 2022-07-07
----//----
low is 1649 - 2022-07-08
high is 2197 - 2024-01-07
mid is 1923 - 2023-04-08
----//----
low is 1924 - 2023-04-09
high is 2197 - 2024-01-07
mid is 2060 - 2023-08-23
----//----
low is 1924 - 2023-04-09
high is 2059 - 2023-08-22
mid is 1991 - 2023-06-15
----//----
low is 1992 - 2023-06-16
high is 2059 - 2023-08-22
mid is 2025 - 2023-07-19
----//----
low is 2026 - 2023-07-20
high is 2059 - 2023-08-22
mid is 2042 - 2023-08-05
----//----
low is 2026 - 2023-07-20
high is 2041 - 2023-08-04
mid is 2033 - 2023-07-27
----//----
low is 2026 - 2023-07-20
high is 2032 - 2023-07-26
mid is 2029 - 2023-07-23
----//----
First date found! 2023-07-23
https://www.wunderground.com/dashboard/pws/IGURUG1/table/2023-07-23/2023-07-23/daily

When I check IGURUG1.csv
It only has :

Date,Time,Temperature_C,Dew_Point_C,Humidity_%,Wind,Speed_kmh,Gust_kmh,Pressure_hPa,Precip_Rate_mm,Precip_Accum_mm,UV,Solar_w/m2
Date,Time,Temperature_C,Dew_Point_C,Humidity_%,Wind,Speed_kmh,Gust_kmh,Pressure_hPa,Precip_Rate_mm,Precip_Accum_mm,UV,Solar_w/m2
Date,Time,Temperature_C,Dew_Point_C,Humidity_%,Wind,Speed_kmh,Gust_kmh,Pressure_hPa,Precip_Rate_mm,Precip_Accum_mm,UV,Solar_w/m2
Date,Time,Temperature_C,Dew_Point_C,Humidity_%,Wind,Speed_kmh,Gust_kmh,Pressure_hPa,Precip_Rate_mm,Precip_Accum_mm,UV,Solar_w/m2
Date,Time,Temperature_C,Dew_Point_C,Humidity_%,Wind,Speed_kmh,Gust_kmh,Pressure_hPa,Precip_Rate_mm,Precip_Accum_mm,UV,Solar_w/m2

I checked the website. It does have data for specified date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants