Announcement of the winners

Position Team number Team member Affiliation
First 20 Valentin Mahler, Kevin Bellinguer, Simon Camal MINES ParisTech
Second 12 Jethro Browell, Rosemary Tawn, Leo May, Ciaran Gilbert University of Strathclyde
Third 17 Sukanta Basu, Bedassa Cheneka, Eric Lacoa Arends, Simon Watson TU Delft

Congratulations to the top three winning teams.

The first team will be awarded with a prize money of 500 €, second team 300 € and third team 200 €. They will also get a chance to present their models during the conference and publish their papers in IEEE Xplore.

Besides, all the 10 active teams will be invited to submit their papers to be published in IEEE Xplore.

Following is the video summarizing the lessons learned from the competition:

Current ranking

Stay tuned while we are in the process of announcing the top three winning teams!

Why?

There has been an upsurge of wind power penetration in the existing power system. This is seen as a step towards a sustainable energy future. However, the intermittent generation from the wind farms poses a challenge to effectively plan the power system and electricity market. Therefore, the increasing importance of better forecasting methods is inevitable. This competition aims to stimulate the igniting minds who wish to be at the frontier of energy transition by proposing effective tools for forecasting wind power generation.

What?

We will provide one year of hourly numerical weather prediction (NWP) and wind power production data at the beginning of the competition. There will be 6 submission rounds, every round releasing 2 month of hourly data. It is up to you to develop methods that use the NWP data to produce forecasts of wind power production. You can either participate individually or as a team. You are not allowed to use any external datasets to train your model or to improve your forecast. The allowed datasets are described under Data overview.

The competition is being organized and hosted by the EEM 20 organizing team at KTH. The organizers thank Greenlytics for archiving the data for the competition. The weather data used for the competition has been acquired from met.no and the organizers acknowledge MET Norway for making it available.

What's in it for me?

The winning team will receive a prize of 500 €. In addition the top three winning teams will get a chance to present their models at EEM 2020 and publish their result in the IEEE conference proceedings. Note, in order to claim the prize, the winners will have to submit their code for review by the competition organizers (this means that the submitted results need to tally when the organizers run your code. So, if you use any random number generator, it is important to fix the seed generator.).

How?

The competition data will be released on 4th March’20 and the competition will take place 5th May’20 to 16th June’20. The first submission deadline is on 5th May’20. There will be one submission each week of a two-month wind power forecast using weather forecast initialised the day before. This means that the competition setting replicated the problem of day ahead wind power forecasting. The last submission is 9th June’20. In total, there will be 6 submissions out of which the best 5 submissions will be considered for evaluating the final competition score of each team.

The deadline for each week is on Tuesdays at 23:59 CET, starting from 5th May’20. Each submission consists of wind power forecasts for the next two months (with hourly timesteps). On each Wednesday during the competition, for example 6th May for the first submission round, we will release the actual wind power production data on a rolling basis to allow participants to update their forecasting models. However, for the first submission only the originally uploaded wind power generation data can be used to train your model. The scoreboard will be released every Friday during the competition, for example 8th May for the first submission round.

Timeline

Illustration of submission and data upload process

The following example illustrates the timeline over the forecasts submissions and the additional uploads of historical wind power generation data. The additional wind power production data is uploaded each Wednesday and covers 2 months of hourly power production (target of previous submission) and NWP (explanatory variables for next submission) data.

Note, the training data is available from 04.03.2020, and can be used to train your model before the competition officially starts.

Data overview

The data provided in this competition consists of three parts:

    1. Gridded weather forecasts over Sweden splitted in daily NetCDF files
    2. Aggregate wind power production in the four different swedish price regions
    3. A record of swedish wind turbines and their location

The gridded weather forecasts are provided as 10 ensembles on a 71×169 projection coordinate grid centered over Sweden and contain the following variables:

 

Variable name
Long name
Unit
Temperature
Surface temperature (T2M)
K
Wind_U
Zonal 10 metre wind (U10M)
m/s
Wind_V
Meridional 10 metre wind (V10M)
m/s
WindGustSpeed
Wind gust
m/s
Pressure
Mean Sea Level Pressure (MSLP)
Pa
RelativeHumidity
Screen level relative humidity (RH2M)
CloudCover
Total cloud cover (TCC)

 

Since every NetCDF file contains hourly values for one day (24 time steps) this means that the shape of the variables in each file is:

(time steps, height levels, ensembles, x_coordinates, y_coordinates) = (24, 1, 10, 169, 71)

The wind power production data is given as aggregates over each price region (SE1, SE2, SE3, SE4) in MWh/h. Since wind power capacity has grown over the course of the 24 months of competition data, we also provide you with a record of wind turbine installations. This record is quality check as good as we could but there might be discrepancies compared to reality. For example, according to the Swedish Wind Power Association there were 4099 wind turbines in Sweden constituting 8984 MW of installed capacity. In the record provided there are only 4004 wind turbines constituting 8640 MW of installed capacity. It is anticipated that the provided wind turbine record could be useful in understanding geographical clusters of wind turbines as well as the growth of installed capacity.

In the illustration below the wind turbines are plotted (left) as well as ten-to-one downsample version of the gridded NWP data (right).

Here is the Python script to access the NWP data:

  • import xarray as xr
  • import s3fs
  • fs_s3 = s3fs.S3FileSystem(anon=True)
  • s3path = ‘greenlytics-public/forecasting-competition/releases/Task0/01/20000101T00Z.nc’
  • remote_file_obj = fs_s3.open(s3path, mode=’rb’)
  • ds = xr.open_dataset(remote_file_obj, engine=’h5netcdf’)

Data Provision

Again, the historical NWP can be downloaded using the Python script provided below:

  • import xarray as xr
  • import s3fs
  • fs_s3 = s3fs.S3FileSystem(anon=True)
  • s3path = ‘greenlytics-public/forecasting-competition/releases/Task0/01/20000101T00Z.nc’
  • remote_file_obj = fs_s3.open(s3path, mode=’rb’)
  • ds = xr.open_dataset(remote_file_obj, engine=’h5netcdf’)

Following is the MATLAB script for accessing the historical NWP data:

% The historical NWP data can be downloaded using the following MATLAB script
    ‘ReadFcn’, @(filename) copyfile(filename, pwd, ‘f’));
read(fds);
% To further read and access the file from your current folder you can use these
% NetCDF NetCDF functions>.
ncinfo(‘20000101T00Z.nc’);
ncdisp(“20000101T00Z.nc”);

More details about the MATLAB code, output and other useful links can be found here.

 

The historical NWP data can also be downloaded by the following R script:

require(curl)
curl_download(url=s3file,destfile = “test.nc“)
In order to obtain the NWP data with AWS CLI, follow the instructions given below:

1. Register on AWS
2. Create an access key (https://www.youtube.com/watch?v=JvtmmS9_tfU)

3. Download and install aws command line interface (CLI) from https://aws.amazon.com/cli/

4. Configure the CLI by running “aws configure”
5. Download files by running “aws s3 cp  s3://greenlytics-public/forecasting-competition/releases/Task0/01/ myfolder/ –recursive”

The wind power data can be downloaded here, and the record of swedish wind turbines and their locations can be downloaded here.

Moreover, the actual wind power production data including January to December 2001​ (forecasted in submissions 1-6) is available here. Each Wednesday this link will be updated to include the actual wind power production for the previous submission.​

Data Provision for the competition

The NWP data for the second submission in the competition are now available. To access the new NWP data, please use the same template-scripts as described above under “Data provision”. However, note that you must replace the link from the template-scripts with:

‘greenlytics-public/forecasting-competition/releases/Task6/11/20011101T00Z.nc’
‘greenlytics-public/forecasting-competition/releases/Task6/11/20011123T00Z.nc’
For all the days in November 2001, and:

‘greenlytics-public/forecasting-competition/releases/Task6/12/20011201T00Z.nc’

‘greenlytics-public/forecasting-competition/releases/Task6/12/20011215T00Z.nc’

For all the days in December 2001.​
Moreover, the actual wind power production data including January to December 2001​ (forecasted in submissions 1-6) is available here.

Evaluation method

The pinball loss function is a metric that is used to evaluate the accuracy of the quantile forecast. The forecast should be given in the form of the quantiles, so the output is in the form of a matrix

TIMESTAMP q10,SE1 q20,SE1 q90,SE4
D1 t1
D1 t2
..
D31 t24

For each time k= 1.336 and percentile i= 10, 20, …, 90 the pinball loss function per area a=1, 2, …, 4 is calculated as:

The pinball loss function has been named after its shape that looks like the trajectory of a ball on a pinball. The function is always positive and the further away from target y_k,a , the bigger the loss ξ(vertical distance between the predicted and estimated value) the larger the value of ρ_k,a,i. The slope is used to reflect the desired imbalance in the quantile forecast.

The pinball loss of quantile τ = i/100.

In order to evaluate overall performance, the score is averaged over all the percentiles. Lower scores indicate better forecasts.

“Averaged pinball loss function per area” for every timestamp k

The loss function for each area is calculated as an average overall timestamps, where T is a number of all timestamps

Then the team score of each submission is calculated as an average over all areas

The final score will be based on the “average loss” over all submissions removed by your one submissions with the highest (worst) „weekly score“.

Registration

Please register your team for the competition via Google form before 23.59 CET on April 30th. The link for the Google form is given here:

After successful registration, your team will receive a unique “team number” used in the submission process, see Submission.

(Please note that you do not need to be registered for the conference in order to be able to participate in this competition. But if you are amongst the top three teams and wish to present your model in the conference then you need to register to the conference.)

Submission

A template for the submission files can be downloaded here. Timestamps and forecasts in the template are just an example of what it should look like. You will put your own timestamp according to the forecast setup.

File submission format:

  • Wind power data should be in [MWh/h] (remember to use a dot as a decimal point)
  • Quantile forecast (0.1, 0.2, …, 0.9 quantiles) per each area

File name:

                       Team_number_sub_submissionnumber.csv

(Please note that submission week number would start from 1 and go on till 6. The first submission is 5th May for example.)

For example, “Team_1_sub_1.csv”.

The submissions should be sent via email to forecast@eem20.eu , with the subject “Submission submission_number. For example, “Submission 1” for the first submission. Note that the submission must be made before 23.59 CET on the respective day.

Note, if your submission deviates from the provided template or in naming of the file – it will not be considered for the competition. Further, remember that no external datasets are allowed.

Contact

If you have any questions regarding the competition, data or submission procedure, please contact us at forecast@eem20.eu.

Frequently asked questions

Here we will post some frequently asked questions together with answers on a rolling basis. (click on the plus icon to see the answer)

Q: We have some missing dates in the NWP training dataset, will they be added later?

There are two dates missing: May 14th and September 26th​ of the year 2000. Unfortunately, the database we use do not have any data for these days so they will not be added later.

Q: Our NWP-grid does not seem to cover all wind turbines, is this correct?

Yes, a few of the wind turbines located in the far north-east corner of Sweden are not covered by the NWP-grid. It is unfortunate but no data is available for that area.

Stack Entry

$120

Per Person

  • 1 Free Entrance
  • Regular Seating
  • Custom Swags
  • Free Wifi

Student Pass

$99

Bulk Order

  • 20x Free Entrance
  • 20x Regular Seating
  • Custom Swags
  • Free Wifi




VIP Pass

$199

VIP's only

  • VIP Entrance
  • VIP Seating
  • Custom Swags
  • Free Wifi