Order Of Convergence Of An Iterative Scheme
Let the sequence of iterative values { xn } ¥n = 0   converges to 's'. Also let e n = s-xn and e n+1= s-xn+1 for n > 0 are the errors at nth and (n+1)th iterations respectively. If two positive constants A ¹ 0 and R > 0 exist, and
 lim        n ®¥ |s-xn+1| lim    n ®¥ |en+1 | = =  A |s-xn|R |e n|R
then the sequence is said to converge to 's' with order of convergence R. The number A is called the asymptotic error constant.

This can be derived from Taylor series as follows

Let xi+1= g(xi) define an iterative method, and let 's' and xn  respectively are the exact and approximate solutions of        x = g(x). Then xn= s + e n ,        where en is the error in xn.  Suppose that g is differentiable any number of times, then from Taylor's formula
xn+1 = g(xn) = g(s + en)
 = g(s) + en g'(s) + 1 e n2 g''(s) + ... 2
The exponent of en in the first non-vanishing term after g(s) is called the order of the iteration process defined by g. Since xn+1 - s = en+1 (and g(s) = s) the above equation can now written as
 e n+1 = en g'(s) + 1 e n2 g''(s) + ... 2
that is the error at (n+1)th iteration can be written as a function of error at the previous iteration. In the case of convergence en is small for large n and hence the order is a measure for the speed of convergence. For example if a scheme is second order, that is
 e n+1   = 1 e n2 g''(s) 2
then the number of significant digits are approximately doubled in each step.
Order of Newton's Method: Since for Newton's method
 g(x) = x - f(x) f '(x)

 g'(x)  = f(x) * f ''(x) (f '(x))2
at x = s, g' = 0 since f(s) = 0  and
 g''(x) = 2f ''(x) f '(x)

at x = s, g''(s) need not be zero, hence Newton-Raphson method is of order two. That is for each iteration the scheme converges approximately to two significant digits.
Order of Fixed Point Iteration method : Since the convergence of this scheme depends on the choice of g(x) and the only information available about g'(x) is  |g'(x)| must be lessthan 1  in some interval which brackets the root. Hence g'(x) at x = s may or may not be zero. That is the order of fixed point iterative scheme is only one.
Order of Secant method : The g(x) for secant method is
 xi+1= xi - (xi - xi-1) * f(xi) f(xi) - f(xi-1)
If  xi+1 = s + ei+1,  xi = s + ei  and xi-1 = s + ei-1 then
 s + ei+1 = (s + ei ) - (ei - ei-1) * f(s + ei ) f(s + ei ) - f(s + ei-1)

 ® e i+1 = ei  - (ei - ei-1) * f(s + ei ) f(s + ei ) - f(s + ei-1)

 =   ei  - (ei - e i-1) * (eif '(s) + ( ½)ei 2f ''(s) + . . .) (eif '(s) + ( ½)ei 2f ''(s) + . . .) - (ei-1 f '(s) + (½)e i-12 f ''(s) + . . .)

 =   ei  - (eif '(s) + ( ½)ei 2f ''(s) + . . .) (f '(s) + (½)(e i+ ei-1 )f ''(s) + . . .)

 =   ei - ( ei + 1 ei2 f ''(s) + . . .) (1 + 1 (ei-1 + ei) f ''(s) + . . .)-1 2 f '(s) 2 f '(s)

 e i+1    = 1 ei ei-1 f ''(s) + 0(ei 2ei-1 + e iei-12) 2 f '(s)

 ei+1 =  C eiei-1        where  C = 1 f ''(s) 2 f '(s)
and higher order terms are neglected.
since          ei = Aepi-1

ei-1 = A-1/pe i1/p

=>           ei+1   =    Aei(A -1/pei 1/p)      =     A1-1/pe i1+1/p

=> order of the scheme p = 1 + 1/p

 p = 1 (1 ± 51/2)   = 1.618 2

and A =C p/(p+1)

Solution of Transcendental Equations | Solution of Linear System of Algebraic Equations | Interpolation & Curve Fitting
Numerical Differentiation & Integration | Numerical Solution of Ordinary Differential Equations
Numerical Solution of Partial Differential Equations