A new three-term conjugate gradient method for solving nonlinear unconstrained optimization problems

Hussein Saleem Ahmed; Salah Gazi Shareef

doi:10.30762/f_m.v8i2.6320

Authors

Hussein Saleem Ahmed Faculty of Science, University of Zaƙho, Ƙurdistan Region, Iraq
Salah Gazi Shareef Faculty of Science, University of Zaƙho, Ƙurdistan Region, Iraq

DOI:

https://doi.org/10.30762/f_m.v8i2.6320

Keywords:

Three-Term Conjugate Gradient, Conjugate Gradient, Descent Condition, Global Convergence, Unconstrained Optimization

Abstract

This study introduces a new development of the three-term conjugate gradient (TTCG) method. This method is used to solve unconstrained nonlinear optimization problems. The core of this development lies in the formulation of new β parameters, which then form a β-system in the proposed method. These parameters make the search direction more stable, enable it to follow the trajectory of change more smoothly, and accelerate convergence by reducing the number of iterations required. To assess its performance, the developed TTCG method is tested on several standard test functions and compared with the Liu–Storey (LS) method. All numerical calculations are performed in Fortran, while data visualization is implemented using Matplotlib in Python and ggplot2 in R. Based on experiments conducted, the TTCG method with the new β-system tends to reach solutions more quickly and yields more consistent results than the LS method for larger problem dimensions. Overall, the TTCG approach offers a more efficient, stable, and reliable alternative for solving unconstrained nonlinear optimization problems.

Downloads

Download data is not yet available.

References

Agueda, L. S., Flores, A., Zeron, P., & Franco-P, M. (2025). Computationally efficient yet quantitatively accurate scaled MP2 protocols for the prediction of weak interaction energies in complex biological systems. ACS Omega, 10(34), 39292–39308. https://doi.org/10.1021/acsomega.5c07079

Ahadiyah, K., Dewi, A. F., & Romadewanti, S. H. (2024). Application of binary logistic regression analysis to factors that influence participation in 2024 presidential election. Journal Focus Action of Research Mathematic (Factor M), 7(2), 224–238. https://doi.org/10.30762/f_m.v7i2.3703

Alhawarat, A., Salleh, Z., Alolaiyan, H., Hor, H. E., & Ismail, S. (2024). A three-term conjugate gradient descent method with some applications. Journal of Inequalities and Applications, 2024, Article 73. https://doi.org/10.1186/s13660-024-03142-0

Ali, E., & Mahdi, S. (2023). A family of developed hybrid four-term conjugate gradient algorithms for unconstrained optimization with applications in image restoration. Symmetry, 15(6), 1–31. https://doi.org/10.3390/sym15061203

Arora, S., Cohen, N., Golowich, N., & Hu, W. (2018). A convergence analysis of gradient descent for deep linear neural networks. arXiv. https://doi.org/10.48550/arXiv.1810.02281

Biegler, L. T. (2024). Multi-level optimization strategies for large-scale nonlinear process systems. Computers and Chemical Engineering, 185, 1–13. https://doi.org/10.1016/j.compchemeng.2024.108657

Dai, Y. H., & Yuan, Y. (1999). A nonlinear conjugate gradient method with a strong global convergence property. SIAM Journal on Optimization, 10(1), 177–182.

Daniel, J. W. (1967). The conjugate gradient method for linear and nonlinear operator equations. SIAM Journal on Numerical Analysis, 4(1), 10–26.

Dolgopolik, M. V. (2019). A convergence analysis of the method of codifferential descent. Computational Optimization and Applications, 71(3), 879–913. https://doi.org/10.1007/s10589-018-0024-0

Fletcher, R., & Reeves, C. M. (1964). Function minimization by conjugate gradients. The Computer Journal, 7(2), 149–154.

Hestenes, M. R., & Stiefel, E. (1952). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6), 409–436.

Ibrahim, A. L., & Mohammed, M. G. (2022). A new three-term conjugate gradient method for training neural networks with global convergence. Indonesian Journal of Electrical Engineering and Computer Science, 28(1), 547–554. https://doi.org/10.11591/ijeecs.v28.i1.pp547-554

Jameel, M. S., & Abdullah, Z. M. (2019). Three-term conjugate gradient (TTCG) methods. Journal of Multidisciplinary Modeling and Optimization, 2(1), 16–26.

Khatab, H. A., & Shareef, S. G. (2019). A new parameter conjugate gradient method based on three terms unconstrained optimization. General Letters in Mathematics, 7(1), 39–44. https://doi.org/10.31559/glm2019.7.1.5

Khatab, H. A., & Shareef, S. G. (2024). Two new limited-memory preconditioned conjugate gradient algorithms for nonlinear optimization problems. Journal of Intelligent & Fuzzy Systems, 46(2), 5465–5478. https://doi.org/10.3233/JIFS-233081

Liu, Y., & Storey, C. (1991). Efficient generalized conjugate gradient algorithms, part 1: Theory. Journal of Optimization Theory and Applications, 69, 129–137.

Lu, H., & Nov, O. C. (2019). Generalized stochastic Frank-Wolfe algorithm with stochastic “substitute” gradient for structured convex optimization. Mathematical Programming, 187(1), 317–349. https://doi.org/10.1007/s10107-020-01480-7

Mishkin, A. (2020). Interpolation, growth conditions, and stochastic gradient descent [Doctoral dissertation, University of British Columbia].

Nasution, J. Y., & Granita. (2024). Implementation of fuzzy logic using the Tsukamoto method in forecasting the amount of bolu cake production. Journal Focus Action of Research Mathematic (Factor M), 7(1), 123–138. https://doi.org/10.30762/f_m.v7i1.2516

Nazareth, L. (1977). A conjugate direction algorithm without line searches. Journal of Optimization Theory and Applications, 23, 373–387. https://doi.org/10.1007/bf00933447

Neumaier, A., & Kimiaei, M. (2024). Globally linearly convergent nonlinear conjugate gradients without Wolfe line search. Numerical Algorithms, 97(4), 1607–1633. https://doi.org/10.1007/s11075-024-01764-5

Omar, D. H., Shareef, S. G., & Fathi, B. G. (2023). A new three-term conjugate gradient algorithm for solving minimization problems. Science Journal of University of Zakho, 11(4), 544–547. https://doi.org/10.25271/sjuoz.2023.11.4.1152

Polak, E., & Ribiere, G. (1969). Note on the convergence of methods of conjugate directions. Revue Française d'Informatique et de Recherche Opérationnelle, 3(16), 35–43.

Riahi, M. K. (2022). On the convergence rate of Fletcher-Reeves nonlinear conjugate gradient methods satisfying strong Wolfe conditions: Application to parameter identification in problems governed by general dynamics. Mathematical Methods in the Applied Sciences, 45(7), 3644–3664. https://doi.org/10.1002/mma.8009

Rohmah, N., & Mufaidah, I. (2024). Artificial neural network method for forecasting price of purebred chicken egg in East Java. Journal Focus Action of Research Mathematic (Factor M), 7(2), 276–290. https://doi.org/10.30762/f_m.v7i2.3798

Royer, W., O’Neill, M., Wright, S. J., & Nov, O. C. (2020). A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization. Mathematical Programming, 180(1), 451–488. https://doi.org/10.1007/s10107-019-01362-7

Rubio, D. M., & Pokutta, S. (2025). Beyond short steps in Frank-Wolfe algorithms. arXiv. https://doi.org/10.48550/arXiv.2501.18773

Sejourne, C. (2025). Theoretical performance analysis applied to non-convex and fair algorithms using random matrix theory [Doctoral dissertation, Université Grenoble Alpes].

Shareef, A. L., & Gazi, I. S. (2019). Modified conjugate gradient method for training neural networks based on logistic mapping. Journal of University of Duhok, 22(1), 45–51. https://doi.org/10.26682/sjuod.2019.22.1.7

Shareef, S. G., & Ibrahim, A. L. (2016). A new conjugate gradient for unconstrained optimization based on step size of Barzilai and Borwein. Science Journal of University of Zakho, 4(1), 104–114. https://doi.org/10.25271/sjuoz.2016.4.1.311

Sobral, A. (2018). Robust low-rank and sparse decomposition for moving object detection: From matrices to tensors [Doctoral dissertation, Université de La Rochelle].

Sulaiman, I. M., & Mamat, M. (2020). A new conjugate gradient method with descent properties and its application to regression analysis. Journal of Numerical Analysis Industrial and Applied Mathematics (JNAIAM), 14(1-2), 25–39.

Surur, A. M., Saputra, N. N., Laili, U. F., Mohamed, H. B., Anggraini, A., Yulita, N., & Kendari, I. (2023). Differential equations to predict the number of students as a basis for accreditation preparation study program policy. Journal Focus Action of Research Mathematic (Factor M), 6(2), 122–139.

Taher, M. S., & Shareef, S. G. (2023). A combined conjugate gradient quasi-Newton method with modification BFGS formula. International Journal of Analysis and Applications, 21, Article 31. https://doi.org/10.28924/2291-8639-21-2023-31

Thanoon, R. B. (2025). A hybrid three-term conjugate gradient algorithm for solving unconstrained optimization problems. Sinkron: Jurnal dan Penelitian Teknik Informatika, 9(4), 3210–3217. https://doi.org/10.33395/sinkron.v9i4.15339

Yudistira, I., Kuzairi, & Faisol. (2024). Vector autoregressive (VAR) modeling for weather forecasting in Madura. Journal Focus Action of Research Mathematic (Factor M), 7(2), 323–337. https://doi.org/10.30762/f_m.v7i2.3486

Zhao, R., & Freund, R. M. (2023). Analysis of the Frank–Wolfe method for convex composite optimization involving a logarithmically-homogeneous barrier. Mathematical Programming, 199(1), 123–163. https://doi.org/10.1007/s10107-022-01820-9

Zhang, L., Zhou, W., & Li, D. H. (2006). A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence. IMA Journal of Numerical Analysis, 26(4), 629–640.

Zhang, L., Zhou, W., & Li, D. (2007). Some descent three-term conjugate gradient methods and their global convergence. Optimization Methods and Software, 22(4), 697–711.