An Accelerated Iterative Technique: Third Refinement of Gauss–Seidel Algorithm for Linear Systems ^†

Abstract

Obtaining an approximation for the majority of sparse linear systems found in engineering and applied sciences requires efficient iteration approaches. Solving such linear systems using iterative techniques is possible, but the number of iterations is high. To acquire approximate solutions with rapid convergence, the need arises to redesign or make changes to the current approaches. In this study, a modified approach, termed the “third refinement” of the Gauss–Seidel algorithm, for solving linear systems is proposed. The primary objective of this research is to optimize for convergence speed by reducing the number of iterations and the spectral radius. Decomposing the coefficient matrix using a standard splitting strategy and performing an interpolation operation on the resulting simpler matrices led to the development of the proposed method. We investigated and established the convergence of the proposed accelerated technique for some classes of matrices. The efficiency of the proposed technique was examined numerically, and the findings revealed a substantial enhancement over its previous modifications.

1. Introduction

Solving a large linear system is one of the challenges of most modeling problems today. A linear system can be expressed in the format:

$$At=b$$

(1)

where $A\in {ℝ}^{n\times n}$ is a matrix of coefficient, $b\in {ℝ}^n$ is a column of constants and $t$ is an unknown vector. The relation $t=A^{-1}b$ denotes the exact solution of problem (1) in a situation where the matrix of coefficients is not singular. It is common knowledge that direct methods for solving such systems require around $\frac{n^3}{3}$ operations, which makes them unsuitable for large sparse systems. It appears that iterative approaches are the best option, especially when convergence with the requisite precision is attained within $n$ steps [1,2,3,4]. The partitioning of $A$ gives

$$A=D-G-H$$

(2)

in which $D$ is the diagonal component, $-H$ and $-G$ are the strictly upper and bottom triangular constituents of $A$ respectively. A matrix reformulation of conventional stationary iterative methods [5] is employed. Three of these standard iterative approaches are of particular relevance to our current endeavor:

Gauss–Seidel Technique [6,7,8]

$$t^{\left(n+1\right)}={\left(D-G\right)}^{-1}H{t}^{\left(n\right)}+{\left(D-G\right)}^{-1}b$$

(3)

Gauss–Seidel Refinement Technique

$$t^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^2{t}^{\left(n\right)}+\left[I+{\left(D-G\right)}^{-1}H\right]{\left(D-G\right)}^{-1}b$$

(4)

Gauss–Seidel Second Refinement Technique

$$t^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^3{t}^{\left(n\right)}+\left[I+{\left(D-G\right)}^{-1}H+{\left({\left(D-G\right)}^{-1}H\right)}^2\right]{\left(D-G\right)}^{-1}b$$

(5)

Iteration matrices in stationary iterative methods are always the same. From the second step onwards, the computational expenses per iteration are at most $n^2$ because the iteration matrix is only calculated once and then reused (much smaller for sparse matrices). Any iterative method’s convergence speed can be increased by employing the idea of refinement of an iterative process [9,10,11].

Iterative approaches are unquestionably the most effective approach to employ when solving huge sparse linear systems. However, such an approach may require several rounds to converge, which may reduce computer storage and computing performance [12]. In such cases, it is necessary to modify or accelerate existing methods in order to achieve approximate answers with rapid convergence. This motivated the current study to offer an improved technique capable of providing better estimated solutions quickly. A new Gauss–Seidel refinement method is presented in this paper. The pace of convergence and the influence of the proposed refinement technique on certain matrices are investigated. As we will see, the spectral radius of the iteration matrix decreased while the convergence rate increased.

2. Methodology

Considering large linear systems of (1), combination of (1) and (2) process gives the classical first-degree Gauss–Seidel iteration method

$$t^{\left(n+1\right)}={\left(D-G\right)}^{-1}H{t}^{\left(n\right)}+{\left(D-G\right)}^{-1}b$$

(6)

The general Refinement approach is expressed as

$$t^{\left(n+1\right)}={\tilde{t}}^{\left(n+1\right)}+{\left(D-G\right)}^{-1}\left(b-A{\tilde{t}}^{\left(n+1\right)}\right)$$

(7)

Substitution of (6) into ${\tilde{t}}^{\left(n+1\right)}$ in (7), gives the Refinement of Gauss–Seidel [4] as

$$t^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^2{t}^{\left(n\right)}+\left[I+{\left(D-G\right)}^{-1}H\right]{\left(D-G\right)}^{-1}b$$

(8)

Modification of (8) results into (9) [5]

$$t^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^3{t}^{\left(n\right)}+\left[I+{\left(D-G\right)}^{-1}H+{\left({\left(D-G\right)}^{-1}H\right)}^2\right]{\left(D-G\right)}^{-1}b$$

(9)

We remodel (9) to obtain (10)

$$\begin{array}{l}{t}^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^3{t}^{\left(n\right)}+\left[I+{\left(D-G\right)}^{-1}H+{\left({\left(D-G\right)}^{-1}H\right)}^2\right]{\left(D-G\right)}^{-1}b+\\ {\left(D-G\right)}^{-1}\left(e-A\left\{{\left[{\left(D-G\right)}^{-1}H\right]}^3{t}^{\left(n\right)}+\left[\begin{array}{l}I+{\left(D-G\right)}^{-1}H+\\ {\left({\left(D-G\right)}^{-1}H\right)}^2\end{array}\right]{\left(D-G\right)}^{-1}b\right\}\right)\end{array}$$

(10)

Next, (10) is simplified to obtain

$$\begin{array}{l}{t}^{\left(n+1\right)}={\left[{\left(D-G\right)}^{-1}H\right]}^4{t}^{\left(n\right)}+\\ \left[I+{\left(D-G\right)}^{-1}H+{\left({\left(D-G\right)}^{-1}H\right)}^2+{\left({\left(D-G\right)}^{-1}H\right)}^3\right]{\left(D-G\right)}^{-1}b\end{array}$$

(11)

Equation (11) is called Third Refinement of Gauss–Seidel (TRGS) technique. The iteration matrix of TRGS is denoted as ${\left[{\left(D-G\right)}^{-1}H\right]}^4$. The method converges if its spectral radius is less than one, represented as $\rho \left({\left[{\left(D-G\right)}^{-1}H\right]}^4\right)<1$. In addition, the closer the spectral radius is to one, the faster the convergence.

2.1. Convergence of Third-Refinement of Gauss–Seidel (TRGS)

Theorem 1.

If $A$ is strictly diagonally dominant matrix {SDD}, then the third refinement of Gauss–Seidel (TRGS) method converges for any choice of the initial approximation $t^{\left(0\right)}$.

Proof. 

Applying the idea of [4,13], let $T$ be the exact solution of the linear system of the form (1). We know that if $T$ is SDD matrix and ${\tilde{t}}^{\left(n+1\right)}\to T$. The TRGS method can be written as

$$t^{\left(n+1\right)}={\tilde{t}}^{\left(n+1\right)}+{\left(D-G\right)}^{-1}\left(b-A{\tilde{t}}^{\left(n+1\right)}\right)\Rightarrow t^{\left(n+1\right)}-T={\tilde{t}}^{\left(n+1\right)}-T+{\left(D-G\right)}^{-1}\left(b-A{\tilde{t}}^{\left(n+1\right)}\right)$$

Hence, taking the norms of both sides results into

$$\begin{array}{l}\Vert t^{\left(n+1\right)}-T\Vert =\Vert {\tilde{t}}^{\left(n+1\right)}-T+{\left(D-G\right)}^{-1}\left(b-A{\tilde{t}}^{\left(n+1\right)}\right)\Vert \le \Vert t^{\left(n+1\right)}-T\Vert +\Vert {\left(D-G\right)}^{-1}\left(b-A{\tilde{t}}^{\left(n+1\right)}\right)\Vert \\ \Rightarrow \Vert t^{\left(n+1\right)}-T\Vert \le \Vert {\tilde{t}}^{\left(n+1\right)}-T\Vert +\Vert {\left(D-G\right)}^{-1}\Vert \Vert \left(b-A{\tilde{t}}^{\left(n+1\right)}\right)\Vert \to \\ \Vert T-T\Vert +\Vert {\left(D-G\right)}^{-1}\left(n-A{\tilde{t}}^{\left(n+1\right)}\right)\Vert =0+\Vert {\left(D-G\right)}^{-1}\Vert \Vert b-b\Vert =0\end{array}$$

Therefore, ${\tilde{t}}^{\left(n+1\right)}\to T$, implying that TRGS method is convergent.  □

Theorem 2.

If $A$ is an $\mathrm{M}-matrix$, then the third-refinement of Gauss–Seidel (TRGS) technique converges for any preliminary guess $t^{\left(0\right)}$.

Proof. 

We employed a similar procedure to that in works of [5,13]. Therefore, we can show that TRGS converges by using the spectral radius of the iterative matrix. If $A$ is an M-matrix, then the spectral radius of Gauss–Seidel is less than 1. Thus, $\rho \left({\left(D-G\right)}^{-1}H\right)<1\Rightarrow \rho \left[{\left({\left(D-G\right)}^{-1}H\right)}^4\right]={\left[\rho \left({\left(D-G\right)}^{-1}H\right)\right]}^4<1$. Since the spectral radius of TRGS is less than 1, as such TRGS is convergent. □

2.2. Algorithm for Third Refinement of Gauss–Seidel (TRGS) Technique

(i)

Input the coefficients of $A=\left(a_{ij}\right)$, indicate a preliminary estimation $t^{\left(0\right)}$, maximum iteration quantity tolerance $\left(\epsilon \right)$.

(ii)

Obtain the partition matrices $D,G$ and $H$ from $A=\left(a_{ij}\right)$.

(iii)

Create inverse of $\left(D-G\right)$ and obtain $J={\left(D-G\right)}^{-1}H$.

(iv)

Create $K={\left\{{\left(D-G\right)}^{-1}H\right\}}^2,M={\left\{{\left(D-G\right)}^{-1}H\right\}}^{3}andN={\left\{{\left(D-G\right)}^{-1}H\right\}}^4$.

(v)

Establish $Z=\left[I+J+K+M\right]{\left(D-G\right)}^{-1}b$.

(vi)

Iterate $t^{\left(n+1\right)}=N{t}^{\left(n\right)}+Z$ and stop if ${\Vert t^{\left(n+1\right)}-t^{\left(n\right)}\Vert }_{\infty }<\epsilon$.

3. Results and Discussion

In this section, an ideal numerical experiment is observed to test the performance of the proposed technique with respect to its initial refinements.

Applied problem [14]: Consider the linear system of equations;

$$\left(\begin{array}{cccccccc}4.2& 0& -1& -1& 0& 0& -1& -1\\ -1& 4.2& 0& -1& -1& 0& 0& -1\\ -1& -1& 4.2& 0& -1& -1& 0& 0\\ 0& -1& -1& 4.2& 0& -1& -1& 0\\ 0& 0& -1& -1& 4.2& 0& -1& -1\\ -1& 0& 0& -1& -1& 4.2& 0& -1\\ -1& -1& 0& 0& -1& -1& 4.2& 0\\ 0& -1& -1& 0& 0& -1& -1& 4.2\end{array}\right)\left(\begin{array}{c}{t}_1\\ t_2\\ t_3\\ t_4\\ t_5\\ t_6\\ t_7\\ t_8\end{array}\right)=\left(\begin{array}{c}6.20\\ 5.40\\ -9.20\\ 0.00\\ 6.20\\ 1.20\\ -13.4\\ 4.20\end{array}\right)$$

The true solution of the applied problem is $t=\left(1.0,2.0,-1.0,0.0,1.0,1.0,-2.0,1.0\right)$

From

Table 1. Comparison of Spectral radius and Convergence rate for the Applied Problem.

Technique	Iteration Step	Spectral Radius	Execution Time (s)	Convergence Rate
GS	88	0.89530	6.70	0.04803
RGS	44	0.80157	5.53	0.09606
SRGS	30	0.71765	5.00	0.14408
TRGS	22	0.64251	4.10	0.19212

, it can be clearly observed that the proposed method has a greatly minimized spectral radius, with respect to the iteration matrix, and also shows that the convergence rate is very high. Table 1 also shows that the TRGS reduced the number of iterations to one-fourth of GS, half of RGS and a few steps of SRGS. Based on how near their spectral radii are to zero, it is inferred that the TRGS has a faster rate of convergence than the techniques compared $\left(\rho \left(TRGS\right)<\rho \left(SRGS\right)<\rho \left(RGS\right)<\rho \left(GS\right)<1\right)$.

4. Conclusions

In this study, an accelerated iterative technique named “Third-Refinement of Gauss-Seidel (TRGS) technique” is proposed. The TRGS algorithm is very appropriate in solving a large system of linear equations, as it shows a significant improvement in the reduction in iteration step and increase in convergence rate. The analysis from Theorems 1 and 2 verify that the proposed technique is convergent and the efficiency of TRGS is illustrated through the applied problem as shown in

Table 2. The Iterate Solution of Applied Problem.

Technique	$\mathit{n}$	${\mathit{t}}_1^{\left(\mathit{n}\right)}$	${\mathit{t}}_2^{\left(\mathit{n}\right)}$	${\mathit{t}}_3^{\left(\mathit{n}\right)}$	${\mathit{t}}_4^{\left(\mathit{n}\right)}$	${\mathit{t}}_5^{\left(\mathit{n}\right)}$	${\mathit{t}}_6^{\left(\mathit{n}\right)}$	${\mathit{t}}_7^{\left(\mathit{n}\right)}$	${\mathit{t}}_8^{\left(\mathit{n}\right)}$
GS	0	0	0	0	0	0	0	0	0
	1	1.47620	1.63720	−1.44920	0.04476	1.14180	0.91970	−1.95840	0.79746
	2	0.86540	1.96410	−1.02590	−0.02391	0.94982	0.90209	−2.07580	0.94392
	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$
	87	0.99999	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000
	88	1.00000	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000
RGS	0	0	0	0	0	0	0	0	0
	1	0.86540	1.96410	−1.02590	−0.23917	0.949820	0.90209	−2.07580	0.94392
	2	0.94909	1.94710	−1.05170	−0.04878	0.95370	0.95407	−2.04670	0.95306
	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$
	43	0.99999	2.00000	−1.00000	0.00000	0.99999	0.99999	−2.00000	0.99999
	44	1.00000	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000
SRGS	0	0	0	0	0	0	0	0	0
	1	0.95672	1.95870	−1.05540	−0.06439	0.94007	0.94674	−2.04710	0.95308
	2	0.95952	1.96010	−1.03950	−0.03950	0.96145	0.96206	−2.03740	0.96315
	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$
	29	0.99999	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000
	30	1.00000	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000
TRGS	0	0	0	0	0	0	0	0	0
	1	0.94909	1.94710	−1.05170	−0.04878	0.95370	0.95407	−2.04670	0.95306
	2	0.96741	1.96790	−1.03170	−0.03170	0.96917	0.96959	−2.03000	0.97042
	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$	$⋮$
	21	0.99999	2.00000	−1.00000	0.00000	0.99999	0.99999	−2.00000	0.99999
	22	1.00000	2.00000	−1.00000	0.00000	1.00000	1.00000	−2.00000	1.00000

. It can be deduced from our analysis that the proposed technique achieved a qualitative and quantitative shift in solving linear systems of equations and is more efficient than existing refinements of Gauss–Seidel techniques.