The deregulated electricity markets have been in operation m a number of countries since the 1990s. During the deregulation process, vertically integrated power utilities have been reformed into competitive markets, with initial goals to improve the market efficiency, minimize the production cost and reduce the electricity price. Given the benefits that have been achieved by the deregulation, several new challenges are also observed in the market. Due to the fundamental changes of the electric power industry, traditional management and analysis methods cannot deal with these new challenges. Novel electricity market management and analysis methods are therefore needed in the deregulated environment.
Data mining is defined as "the nontrivial extraction of implicit, previously unknown, and potentially useful information from data" and "the science of extracting useful information from large data sets". The modern electricity market produces huge amounts of market data, in which highly useful information can be extracted to facilitate the market management and analysis. In this thesis, I employ data mining together with advanced statistical methods to analyze the data of electricity markets. The data mining and statistical methods are integrated with the market management and analysis techniques to solve several difficult problems in electricity market research.
This research aims at developing novel methods to solve several notoriously difficult problems in the deregulated electricity market. The thesis consists of two main parts. The first part deals with extreme price volatility in the electricity market, and the second part studies power system contingency assessment and prediction in the deregulated market environment.
In the deregulated electricity market, extreme price volatility, which is also known as the price spike, is one of the major challenges not yet solved. Given their significant influences to market participants, price spike forecast together with normal price prediction are highly important in a competitive electricity market for individual market participants as well as the system operator. In the first part of this thesis, a novel framework is proposed to handle the extreme price volatility caused by electricity price spikes. The framework is based on data mining and computational statistics, thus is able to process the large data amount of electricity price signals. In the framework, feature selection techniques are used to identify relevant factors of price spikes. Classification methods are employed to predict the occurrences of price spikes in the future. Based on the results of spike occurrence prediction, regression and time series models are used to forecast the value of the spike. In addition, a Support Vector Machine (SVM) based forecasting model is proposed to estimate the risks involved in price spikes. I also develop a novel approach, namely Bayesian Classifier with Benefit Maximization (BCBM). The BCBM approach integrates the price spike prediction together with decision making of market participants, so as to achieve the maximum decision benefits facing spikes.
In addition to the energy market, which is a complex economical system, the physical power system behind the electricity market is an essential integrated part of the overall market as well. In Chapter 7, the problem of power system contingency assessment and prediction is studied. In the deregulated market, the power system is operating under more stressed condition with much more uncertainties in comparison to the past. Following the recent devastating blackouts in USA, UK and Russia, power system stability analysis and contingency prediction has attracted significant attention from both the academic society and industry. In this thesis, a novel method developed for power system contingency prediction is reported. The proposed method consists of two major stages. In the first stage, a novel type of patterns namely Local Correlation Network Pattern (LCNP) is mined from the structure and system variables of the power system. Correlation rules, which are useful for the network operator to locate potentially instable components, can be further generated from the LCNP. In the second stage, a kernel based classification method is developed to predict the system instability. By testing on a real-world power network (the New England system), I demonstrate that the proposed method is effective in predicting system contingency and thus highly useful for blackout prevention.
In summary, the major contributions of this thesis includes a price spike forecasting framework, a comprehensive empirical study of feature selection m electricity price forecasting, a novel statistical method for estimating the risks of electricity prices, a data mining based approach for making decisions on spikes, and a data mining based approach for power system contingency analysis. This research is finished with 14 publications in major international journals and conferences.