I want to create a deep learning model to predict rainfall in the Mediterranean area.

As data source, I will use data from Meteostat (https://dev.meteostat.net/data/bulk/daily).

For now I will be manually downloading the data year by year; later on I will try to automate this in KNIME.

This URL will download a .parquet file with daily data for each year: 

https://data.meteostat.net/daily/2026.parquet

Next step is to read this file into KNIME. This is easily done with the Parquet Reader node, which can read all files in a specified folder.

The data contains a 'station_id' field. The data about these stations is available in the Stations SQLite database, which can also be downloaded from the Meteostat site. I can now join the station_id to the stations table of this database, and so obtain the LAT/LON coordinates of each station.

As I only want data for the Mediterranean area, I use a row filter to limit the latitudes between 50 and 24 Degrees North and 15 Degrees West and 45 Degrees East.

I filter out unneeded columns and leave only:

  • date
  • temp - mean air temperature (Deg. Celsius)
  • tmin - minimum air temperature (Deg. Celsius)
  • tmax - maximum air temperature (Deg. Celsius)
  • rhum - relative humidity (%)
  • prcp - precipitation (mm)
  • snwd - snow depth (cm)
  • wspd - average wind speed (km/h)
  • wpgt - peak wind gust (km/h)
  • pres - average air pressure (hPa)
  • tsun - sunshine duration (min)
  • cldc - cloud cover (okta)

Below is the workflow. It is split up in the following Metanodes, below are screenshots of these, expanded.

  • Pre-process: here we read the data, add the station info, limit by Latitude and Longitude, handle missing values and add the Lag columns for pressure, temperature and relative humidity
  • Model: the Keras nodes making up the Model
  • Partition and Normalize: here we create the Training, Test and Validation sets and normalize the data. It also includes the Network Learner node
  • Evaluate: here we evaluate the results of the model

The complete workflow:

 
The Pre-process metanode:
 
The Model metanode:
 
The Partition and Normalize metanode:
 
The Evaluate metanode:
First attempt

I want to predict heavy rainfall (more than 50 mm over 24 hours). I used the following input features : temp, tmin, tmax, rhum, wspd, wpgt and pres.

The first result is quite disastrous : the model seems to predict heavy rain for every day !

 

Second attempt

In this second attempt I only used the following input features : tmin, tmax, rhum and pres. The result is slightly better, but not a lot.

 

Third attempt 

For the next attempt, I wanted to enhance my input features. I will add to every station row, the temp, tax, tmin, rhum and pres values for the past 5 days and calculate the differences between each 2 days. I will then use these differences as input features. This is done with the Lag Column node in KNIME.

After these 3 attempts, I came to the conclusion that my model is not progressing. What I will try now is, as a first step, examine if I can find a strong correlation between any of my features and the actual precipitation. I will write about this in the next topic.