User:Cnapun

Personal Information

• Case Western Reserve University Class of 2019
• Applied Mathematics and Computer Science Major

Weekly Log

Week 0 (30 May - 2 Jun)

• Met Dr. Povinelli 4 times: to discuss possible project topics, to get oriented with the lab, to decide on a project topic, and to establish weekly milestones
• Obtained MU and MSCS account logins and ID card
• Read most of "Learning Deep Architectures for AI"
• Read section on Sequence Modeling (Ch 10) of The Deep Learning book
• Read various other papers

Week 1 (5 Jun - 9 Jun)

• Attended GasDay camp and learned about what GasDay does
• Attended responsible conduct of research training
• Continued to read papers
• Decided to use TensorFlow and Keras for now
• Got Anaconda, TensorFlow, and Keras installed on a couple lab computers
• Learned how to access customer data

Week 2 (12 Jun - 16 Jun)

• Started playing with TensorFlow, mostly using the GEFCom2014-E dataset to allow me to continue working when not at the lab
• Got single layer encoder-decoder architecture to work in TensorFlow

Reading

Here are some of the papers I have read, skimmed, and partially read:

Forecasting with Deep Learning

• "Building Energy Load Forecasting using Deep Neural Networks"
  • Discusses the direct application of LSTMs to load forecasting
• "Training Recurrent Networks by Evolino"
  • Describes the use of genetic algorithms to train RNNs
  • Should offer better performance than Echo State Networks (ESNs)
  • No description of how crossover and mutate operations work; I referred to "Neural Network Weight Selection Using Genetic Algorithms" for a more complete explanation

Hybrid Methods

• • "Hybrid Neural Networks Over Time Series For Trend Forecasting"
  • "Time series forecasting using a hybrid ARIMA and neural network model"
  • "Intelligent Hybrid Wavelet Models for Short-Term Load Forecasting"
  • "Short-Term Load Forecasting: Similar Day-Based Wavelet Neural Networks"

Review Papers

• • "Short-Term Load Forecasting Methods: A Review"
  • "An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting", which gives a good, current overview of methods

LSTMs, Training, and Possible Improvements

• • "Recurrent Neural Network Regularization"
  • "A Theoretically Grounded Application of Dropout in Recurrent Neural Networks"
  • "Multiplicative LSTM for sequence modelling"
  • "Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences"
  • "Semi-supervised Sequence Learning"