PARAMETER ESTIMATION OF BLACK-SCHOLES MODEL BY REINFORCEMENT LEARNING

In the process of pricing options with the Black-Scholes formula, it is necessary to know the volatility of the underlying asset. By studying the method of estimating the parameters in the Black-Scholes model, we can provide a better volatility estimation when we apply the Black-Scholes option pricing formula.

During the research process, we have established a reinforcement learning model, and after theoretical analysis, we give the setting form of rewards and propose to use the  TD(0)  algorithm to estimate the volatility and the expected return. The algorithm's convergence and the effectiveness of the reinforcement learning method are demonstrated.

In the case study, we give some known Black-Scholes models and sample their  trajectories as research data. Then the  TD(0) algorithm is used to estimate the expected return and the  volatility. The data shows that after setting the appropriate reward function and discount factor, the reinforcement learning method can effectively estimate the volatility and expected return.

We also use the maximum likelihood estimation method to estimate    the volatility and the expected return.  By analyzing the estimation characteristics of these two methods and comparing their estimation results, we find that for the  Black-Scholes model, our method is more advantageous to estimate  the expected return.

[1，BLACK F, SCHOLES M. The Pricing of Options and Corporate Liabilities[J]. The Journal of Political Economy, 1973, 81(3),637–654.3.
[2] BATES, D. Jumps and Stochastic Volatility: Exchange Rate Processes Implicit in Deutsche Mark Options[J]. The Review of Financial Studies, 1996, Vol. 9: No. 1, 69-107.
[3] S.SUTTON R, G.BARTO A. Reinforcement Learning[M]. Westchester Publishing Services, 2018: 25-41.
[4] BORKAR V S. STOCHASTIC APPROXIMATION : A DYNAMICAL SYSTEMS VIEW-POINT[M]. Mumbai: Tata Institute of Fundamental Research, Mumbai, 2008: 1-16.
[5] DAYAN P. The convergence of TD(𝜆) for general 𝜆[J]. Machine Learning, 1992: 8 (3–4):341–362.
[6] SUTTON R S. Learning to Predict by the Methods of Temporal Differences[J]. Machine Learn-ing, February 4. 1988: 3:9-44.
[7] WATKINS, C.J.C.H. Learning from Delayed Rewards[D]. University of Cambridge. EThOSuk.bl.ethos.330022, 1989.
[8] CHRIS W, PETER D. Q-learning[J]. Machine Learning, 1992: 279–292.
[9] LUU R. Convergence of Q-learning: a simple proof[J]. Francisco S. MeloInstitute for Systems and Robotics,Instituto Superior Técnico,Lisboa, PORTUGAL, 2001.
[10] ENGLE, F R. Autoregressive Conditional Heteroskedasticity with Estimates of the Variance of United Kingdom Inflation[J]. Econometrica, 1982: 987–1007.
[11] BOLLERSLEV T. Generalized Autoregressive Conditional Heteroskedasticity[J]. Journal of Econometrics, 1986: 307-327.
[12] KHALED K, SAMIA M. Estimation of the Parameters of the Stochastic Differential Equations Black-Scholes Model Share Price of Gold[J]. Journal of Mathematics and Statistics, 2010, 6(4): 421-424.
[13] GROSS P. Parameter Estimation for Black-Scholes Equation[D]. Advisor: Dr. Jialing Dai Final Report URA, 2006.
[14] XIONG J. An Introduction to Stochastic Filtering Theory[M]. New York: Oxford University Press Inc, 2008: 15-41.
[15] C.HULL J. OPTIONS, FUTURES, AND OTHER DERIVATIVES[M]. University of Toronto,ninth edition, 2014: 321-350.
[16] DOYA K. Reinforcement Learning In Continuous Time and Space[J]. Neural Computation,2000, 12(1): 219-245.
[17] LIN X, WANG M, LAI C. A modification term for Black-Scholes model based on discrepancy calibrated with real market data[J]. Data Science in Finance and Economics, 2021: DSFE, 1(4):313–326.45
[18] WANG H, ZHOU X Y. Continuous-time mean–variance portfolio selection: A reinforcement learning framework[J]. Mathematical Finance, 2020, DOI: 10.1111/mafi.12281.
[19] CHOWDHURY R, MAHDY M, TANISHA NOURIN ALAM E A. Predicting the stock price of frontier markets using machine learning and modified Black–Scholes Option pricing model[J]. Physica A 124444, 2020, A 555.
[20] CHI S. A differential neural network learns stochastic differential equations and the Black-Scholes equation for pricing multi-asset options[D]. Department of Computer Science, Kyung-sung University, Busan 48434, South Korea, 2020.
[21] VAIDYA T, MURGUIA C, PILIOURAS G. Learning agents in Black–Scholes financial markets[J]. Mathematical Finance, 2020, R. Soc. Open Sci. 7: 201188.
[22] HALPERIN I. Q-Learner in the Black-Scholes(-Merton) Worlds[J]. Computational Finance(q-fin.CP); Machine Learning (cs.LG), 2019, Vol. 9.
[23] BAYRAM M, BUYUKOZ G O, PARTAL T. Parameter Estimation in a Black-Scholes Model[J]. Thermal Science, 2018, 22: 117-122.
[24] SHAHRABI J, ADIBI M A, MAHOOTCHI M. A reinforcement learning approach to parameter estimation in dynamic job shop scheduling[J]. Computers Industrial Engineering, 2017, 110:75-82.
[25] TEKA K H. Parameter Estimation Of The Black-Scholes-Merton Model[D]. KANSAS STATE UNIVERSITY Manhattan, Kansas, 2013.
[26] HAIQIU L. Perspective of Expanded Black- Scholes Model and Estimation of Parameter[J].Soft Science, 2005, 110.
[27] NA H S. Black-Scholes Option Pricing Using Three Volatility Models: Moving Average,GARCH(1, 1), and Adaptive GARCH[D]. Rotterdam, Netherlands: Erasmus University, August2003.
[28] FOUQUE J P, PAPANICOLAOU G, SIRCAR K R. Stochastic Volatility Correction to Black-Scholes. NC State University[D]. NC State University, January 2000.
[29] EVEN-DAR E, MANSOUR Y. Learning Rates for Q-learning[J]. Journal of Machine Learning Research, 2017, 110: 75-82.
[30] PARKINSON M. The Extreme Value Method for Estimating the Variance of the Rate of Return [J]. The Journal of Business, February 1980: 61-65.