Optimal filtering anderson moore free download

Sam Nazari rated it really liked it Jan 18, Chris Schwarz rated it really liked it Sep 24, Renato rated it liked it Mar 18, Michael added it Jun 27, Michael added it Mar 30, Encoder1 added it Aug 27, Anurag is currently reading it Aug 05, Stephen Davis marked it as to-read Jan 10, Xiaoyuan marked it as to-read Sep 07, Alexander marked it as to-read Sep 13, G marked it as to-read Jan 23, Joe Antognini marked it as to-read Jun 24, BookDB marked it as to-read Nov 14, Enrique marked it as to-read Dec 12, Pavel Bulgakov marked it as to-read Jun 15, Ameenah added it Sep 25, Kadiri Saliu added it Oct 02, David marked it as to-read Oct 18, Maverickrit added it Jan 02, There are no discussion topics on this book yet.

Be the first to start one ». About Brian D. Brian D. Books by Brian D. When Dana Schwartz started writing about a 19th-century pandemic ravaging Edinburgh in her latest book, Anatomy: A Love Story, she had no idea Read more Trivia About Optimal Filtering. Topics include filtering, linear systems, and estimation; the discrete-time Kalman filter; time-invariant filters; properties of Kalman filters; computational aspects; and smoothing of discrete-time signals.

Additional subjects encompass applications in nonlinear filtering; innovations representations, spectral factorization, and Wiener and Levinson filtering; parameter identification and adaptive estimation; and colored noise and suboptimal reduced order filters.

Each chapter concludes with references, and four appendixes contain useful supplementary material. Optimal Filtering By: Brian D. Product Description Product Details This graduate-level text augments and extends beyond undergraduate studies of signal processing, particularly in regard to communication systems and digital filtering theory. Reprint of the Prentice-Hall, Inc. We consider now how one might check this property. First, we must pin down a coordinate basis—otherwise the state estimation problem is ill defined.

One way to do this is to assume that F and H are known. A second assumption, which it is almost essential to make, is that [F, H] k completely observable. IS it sufficient? With the complete observability assumption, the answer is yes. Suppose the signal model is of the form 6. IEE, Vol. Theory, Vol. IEEE, to appear. Automatic Contro[, Vol.

AC, No. Control, Vol. First, what is the nature of the errors which can be encountered, and what is their effect on the performance of the filter? Secondly, how may one minimize the computational burden of design? Of course, the two questions are not entirely independent, since, for example, procedures involving a small number of computations may be procedures which offer poor error performance.

In this chapter, we sketch some of the ideas that are useful for dealing with these questions—a complete study would probably run to hundreds of pages, and is therefore out of the question.

The most obvious types of el ror are those in which incorrect values are assumed for the system matrices or noise covariance matrices.

However, many others can be envisaged. Linearization, neglect of certain system modes, neglect of a colored component in the noise, and neglect of biases, whether deliberate or unwitting, will all give rise to errors. Modeling errors aside, round-off errors in computation can also create problems.

Brief mention is made of some techniques for eliminating these problems. Then in the next section, one of these techniques, exponential data weighting, is discussed in a little greater detail. In later sections, methods for streamlining computations and also for avoiding the effects of computational errors are considered.

In particular, a derivation is given of the information filter, and the concept of sequential data processing is developed. Square root filtering algorithms are presented, and simplified suboptimal filters for the high measurement noise case are studied. Finally, alternative algorithms for the time-invariant signal model case are derived. Error Analysis In this subsection, we illustrate how one can analyze, in a particular instance, the effects of a particular class of modeling errors.

In some cases, it can be useful to regard the system input as a time-varying bias term, inserted as a device to take account of linearization, neglect of modes, and the like; in this case, almost certainly the actual value and design value will be different. Finally, Zflk. The calculation is valid for all types of errors.

We assume for convenience that input and measurement noise are independent. At this stage, several points should be made. The important thing here is the procedure for obtaining a result.

The notion of tying together an actual signal model and a designed filter in a single equation set may apply to many situations other than that considered. One major use of the above type of analysis is in sensitivity studies. One can then compute the effect on filter performance of this variation, when the filter is designed using the nominal value. A second major use lies in drawing useful qualitative conclusions, applicable to situations in which errors are described qualitatively but not quantitatively.

Examples are given below. The analysis presented in outline form above is given more fully in [1]. Among other work on errors arising from incorrect modeling, we note [], some of which contain results of simulations; reference [6] also includes equations for sensitivity coefficients.

Why is this obvious? Suppose that the only errors following inequalities holding for all k. One might then imagine that this would lead to a conservative filter design in some sense. This is indeed what we find: the design error covariance.

The usefulness of this resuh a proof of which is called for in the problems is as follows. Suppose one simply does not know accurately the noise covariance of the input or output, but one does know an upper bound. In some sense a WO13tcase design results. A third qualitative result see [1] follows from assuming that errors are possible in P,, Qk, Rk, and the bias term uk, but in no other terms. However, a difficulty arises if ug is unbounded.

Note that 4. The easiest way of deriving these equations is to observe that with 4. In lieu of 4. Block Processing The reverse process to sequential processing is block processing. Such techniques are outside the scope of this text. Main Points of the Section With a diagonal R matrix which may always be secured by a Cholesky decomposition covariance filter formulas for the measurement-update equations exist which amount to using a sequence of scalar updates.

With a diagonal Q matrix, analagous information filter formulas can be found for the time-update step. Cholesky Decomposition. Verify the claims associated with Eqs.

This can happen particularly if at least some of the measurements are very accurate, since then numerical computation using ill-conditioned quantities is involved. As a technique for coping with this difficulty, Potter [3, pp. Update equations for the square root of an inverse covariance matrix were also demonstrated.

Let M be a nonnegative definite symmetric matrix. Sometimes, the notation M12 is used to denote an arbitrary square root of M. We shall shortly present update equations for the square roots in lieu of those for the covariances.

This means that only half as many significant digits are required for square root filter computation as compared with covariance filter computation, if numerical difficulties are to be avoided.

For certain applications with restrictions on processing devices, square root filtering may be essential to retain accuracy. For small r, the square root covariance filter is more efficient than the square root information filter, but for moderate or large r, the reverse is true. The reader may recall one other technique for partially accommodating the first difficulty remedied by square root equations. Thus nonnegativity is not as automatic as with the square root approach.

Covariance Square Root Filter The Potter algorithm [3] was first extended by Bellantoni and Dodge [16] to handle vector measurements, and subsequently by Andrews [17] to handle process noise as well.

Schmidt [18] gave another procedure for handling process noise. Vector measurements can be treated either simultaneously or sequentially; in the latter case a diagonal R matrix simplifies the calculations. Until the work of Morf and Kailath [24], time and measurement updates had been regarded as separate exercises; their work combined the two steps. In this subsection, we shall indicate several of the possible equations covering these ideas for the case of models with uncorrelated input and output noise.

It is readily checked that 5. The reader will perceive an obvious parallel between these equations and those of 5. A similar phenomenon was observed in the normal information filter equations. A derivation is called for in the problems. Again, as with the square root filter, specialization of 5. Review Remarks The last three sections have illustrated choices which can be made in the implementation of filter algorithms: covariance or information filter; square root or not square root; sequential or simultaneous processing of state and covariance data, or covariance data only; symmetry promoting or standard form of covariance and information matrix update.

Yet another choice is available for stationary problems, to be outlined in Sec. There are also further choices available within the square root framework. Main Points of the Section Square root filtering ensures nonnegativity of covariance and information matrices and lowers requirements for computational accuracy, generally at the expense of requiring further calculations. Information and covariance forms are available, with and without sequential processing, and with and without combination of time and measurement update.

Sometimes, it is essential to use square root filtering. Verify the measurement. Potter Algorithm. Q,G; k : Q. See [24]. Derive the following combined measurement- and time-update equations for the square root information filter. For the moment, we shall work with time-invariant, asymptotically stable signal models and filters. Later, we shall note how the ideas extend to the time-varying case. The idea in high measurement noise filtering is the following.

From 6. Tbc gain and performance of the suboptimal filter [Eqs. What we have shown, however, is , ,-, that as R l! What of the time-varying case? Exponential stability of as is expothe signal model is normally needed else Pk can be unbounded , filter. Main Point of the Section In high noise, simplified gain and performance, formulas can be used to calculate the filter 6.

There are, however, other ways of proceeding when the signal model is time invariant and the input and measurement noise are stationary.

We shall describe three different approaches. Chandrasekhar-type Algorithms Methods are described in [29] based on the solution of so-called Chandrasekhar-type equations rather than the usual Riccati-type equation.

The advantages of this approach are that there is a reduction in computational effort at least in the usual case where the state dimension is much greater than the output dimension , and with moderately carefuI programming there is an elimination of the possibility of the covariance matrix becoming nonnegative.

Interestingly, it is possible to compute the filter gain recursively, without simultaneously computing the error covariance. Of course, knowing the steady-state gain, one can easily obtain the steady-state error covariance. The approach described in [29] is now briefly summarized. A Sec. We shall limit our presentation here to one set only, referring the reader to [29] and the problems for other sets.

All the derivations depend on certain observations. First, as shown by 7. One such set is provided by: 7. Thus 7. This is most easily seen by studying the way the number of scalar quantities updated by the Riccati and Chandrasekhar approaches changes as n, the state-variabIe dimension, changes while input and output dimension remain constant. With the Riccati approach it approach, the number varies as n 2, while with the Chandrasekhar varies with n.

For values of n that are not high with respect to input and output dimensions, the Riccati equations can however be better to use. In the remainder of this subsection, we offer a number of miscellaneous comments. Information filter Chandrasekhar-type equations can be developed see [29].

For details, see [24]. This idea allows the introduction of some time variation in R and Q. See the text and Prob. This means that the algorithms provide exact filters for a class of signal models with nonstationary outputs though the outputs are asymptotically stationary. The easiest initialization is 2. The Doubling Algorithm The doubling algorithm is another tool for finding the limiting solution of the Riccati equation 7.

Doubling algorithms have been part of the folklore associated with Riccati equations in linear systems problems for some time.

We are unable to give any original reference containing material close to that presented here; however, more recent references include [3 ], with the latter surveying various approaches to the a! Doubling algorithm. Prove this in two lines! We turn now to a proof of the algorithm. It proceeds via several steps and may be omitted at first reading. For convenience in the proof, we shall assume F is nonsingular.

Now consider the linear equation 7. It is clear that Then one easily obtains the matrix pairs X,, Y, , X4, Y4 ,. X,,, Y,. We therefore need an efficient iteration for 2k.

This flows from a special property of 0 1. Now a further property of symplectic matrices, easily verified, is the following. Proof of the doubling algorithm. The speed of the doubling appears not to be a problem. Then for some square T,, we have with the diagonal entries of A of modulus at least 1. Suppose further that the diagonal entries of A have modulus strictly greater than 1. This will in fact be the case, but we shall omit a proof. Set of the linear equation 7.

It turns out that in problems of interest, S,, is nonsingular, so the method is valid. The theory may however run into difficulties if R is singular and one attempts to use a pseudo-inverse in place of an inverse.

From the numerical point of view, it is unclear that the technique of this subsection will be preferred to those given earlier. Main Points of the Section Via Chandrasekhar-type algorithms, recursive equations are available for the transient filter gain associated with a time-invariant signal model with constant Q, R and arbitrary PO. Particularly for P. The steady state error covariance can also be determined in terms of eigenvectors of a certain 2n X 2n matrix.

Problem 7. Show that the following equations can be used in lieu of 7. These equations appear cumbersome, but, as one might expect, one can separate out the Kalman filter equations for the original signal model and the fixed-point smoothing equations. These latter equations are now extracted directly from the above augmented filter equations using 2. IH,[H; x,,,. The smoother is a linear discrete-time system of dimension equal to that of the filter. Notice also that further manipulations of 2.

The equation also helps provide a rough argument to illustrate the fact that as the signal-to-noise ratio decreases, the improvement due to smoothing disappears. The left side of 2. Rigorous analysis for continuous-time signal smoothing 5. The improvement appears in [19], and Prob. As a rule of thumb, we can say that essentially all the smoothing that it is possible to achieve can be achieved the KaIman jilter. The solution, at least in the time-invariant case, is to set Wk, an exponentially decaying quantity, to be zero from some value of k onwards.

Main Points of the Section A study of fixed-point smoothing points up the fact that improvement due to smoothing is monotonic increasing as more measurement data becomes available.

The time constant of this increase is dependent on the dominant time constant of the Kalman filter. As a rule of thumb, the smoother achieves essentially all the improvement possible from smoothing after two or three times the dominant time constant of the Kalman filter. This maximum improvement from smoothing is dependent on the signal-tonoise ratio and the signal model dynamics and can vary from zero improvement to approaching one hundred percent improvement for some signal models at high signal-to-noise ratios.

For the signal model as in Problem 2. Estimate the value of k for which the improvement due to smoothing is ninety percent of that which is possible.

How does this value of k relate to the closed-loop Kalman filter eigenvalue? These formulas will be used in Sec. It is possible to derive fixed-lag smoothing results directly from the fixed-point smoothing results of the previous section, or alternatively to derive the results using Kalman filter theory on an augmented signal model somewhat along the lines of the derivation of fixed-point smoothing. Since the second approach is perhaps the easier from which to obtain recursive equations, we shall study this approach in detail.

We shall also point out very briefly the relationships between the fixed-point and fixed-lag smoothing equations. We begin our study with a precise statement of the fixed-lag smoothing problem. Discrete-time smoothing problem. The notation we adopt for these quantities is as follows: the basic signal modelfor lags up to N. Kalman it:;? The Fixed-lag Smoothing These equations filter equations as 3.

We refer to Eqs. The fixed-lag smoother as a dynamical system is illustrated in Fig. A single set of state-space equations corresponding to the figure are given in the next section. Properties of the Fixed-lag Smoother The equations for the fixed-lag smoother bear some resemblance to the fixed-point smoothing equations of the previous sections, and the properties we now list for the fixed-lag smoother run parallel to the properties listed in the previous section for the fixed-point smoother.

Of course, it should be noted quantities. Further tion leads to a! Therefore as for fixed-point smoothing, the greater the signal-to-noise ratio, the greater the possible improvement from fixed-lag smoothing. From 3. As a rule of thumb, we can say that essentially all the smoothing that it is possible to achieve can be achieved with N selected to be two or three times the dominant time constant of the Kalman filter.

Though this remark has been made with reference to time-invariant filters, with some modification it also holds for time-varying filters. The fixed-lag smoother inherits the stability properties of the original filter since the only feedback paths in the fixed-lag smoother are those associated with the subsystem comprising the original filter.

The storage requirements and computational effort to implement the fixed-lag smoothing algorithm 3. For the time-varying case, storage is required for at least [Fk-, — K.

Computational reasonably assessed by the number of multiplications involved. For the time,. N requiring but mnN multiplications. A special class of problem for which filtered state estimates yield fixed-lag smoothed estimates is studied in Prob.

Generalizations of this simplification are explored in [30]. Other Approaches to Fixed-interval Smoothing As remarked in the introductory section, there are other approaches fixed-interval smoothing. These approaches involve running a Kalman fil forward in time over the interval [0, M], storing either the state estimates or measurements.

The storage requirements and delay in processing compare unfavoura with the quasi-optimal smoother, unless N and M are comparable, and this instance the fixed-lag smoot. Ch 7 so on. Two ining —N , and ltput ough 4. With j replaced by ,gz,. Notice X,. The initialization is evidently with a filtered estimate.

An alternative derivation of the fixed-lag equations can be obtained in the following way. Because p xO, xl,. Various problem exist see, e. Main Points of the Saction Fixed-interval smoothing can be achieved either optimally or quasi-o timally by direct application of the fixed-lag smoothing equations, which turn are Kalman filtering equations in disguise.

The computations involw in quasi-optimal fixed-interval smoothing may be considerably less than II optimal smoothing for large data sets. LJse 4.

Show that the smoother can be thought of as a backward predictor, i. Automatic Control, Vol. AC-8, No. TUNG, ancl c. Automatic control, Vo]. AC-8, October , pp. Press, Cambridge, Mass. Automatic control, Vol. Franklin Inst. Aulomatic control, Vol, AC, No. Sciences, Vol. Aufomatic Control, Vol. Automatic Control, vo1. G,, and K. Automatic No. Communications, Vol. COM 25, No. ZnJormation Theory. The fact that the filter equation and the performance calculations together with the filter gain calculations are decoupled is particularly advantageous, since the performance calculations and filter gain calculations can be performed offline; and as far as the on-line filter calculations are concerned, the equations involved are no more complicated than the signal model equations.

The filtered estimates and the performance measures are simply the means and covariances of the a posteriori probability density functions, which are gaussian. By comparison, optimal nonlinear filtering is far less precise, and we must work hard to achieve even a little. The most we attempt in this book is to see what happens when we adapt some of the linear algorithms to nonlinear environments. Otherwise the above model is identical to the linear gaussian models of earlier chapters.

The filter equations are applied to achieve quasi-optimal of FM frequency modulation signals in low noise. A special class of extended Kalman filters is defined in Sec. The nonlinear filter algorithms involve a bank of extended Kalman filters, where each extended Kalman filter keeps track of one term in the gaussian sum, The gaussian sum filter equations are applied to achieve quasi-optimal demodulation of FM signals in high noise.

Other nonlinear filtering techniques outside the scope of this text use different means for keeping track of the a posteriori probability distributions than the gaussian sum approach of Sec.

For example, there is the point-mass approach of Bucy and Senne [1], the spline function approach of de Figueiredo and Jan [2], and the Fourier series expansion approach used successfully in [3], to mention just a few of the many references in these fields. I Problem 8. This is the time-update step. This is the measurementupdate step.

The above extended Kalman filter is nothing other than a standard and exact Kalman filter for the signal model 2. When applied to the original signal model 1. Of course in any particular application it may be well worthwhile to explore approximations which would allow decoupling of the filter and filter gain equations.

In the next section, a class of filters is considered of the form of 2. For such filters, there is certainly no coupling to a covariance equation. Quality of approximation. The approximation involved in passing from 1.

Therefore, we would expect that in high signal-to-noise ratio situations, there would be fewer difficulties in using an extended Kalman filter. Another possibility for determining whether in a given situation an extended Kalman filter is or is not working well is to check how white the pseudo-innovations are, for the whiter these are the more nearly optimal is the filter.

Again off-line Monte Carlo simulations can be useful, even if tedious and perilous, or the application of performance bounds such as described in the next section may be useful in certain cases when there exist cone-bounded conditions on the nonlinearities. Selection of a suitable co-ordinate basis. We have already seen that for a certain nonlinear filtering problem—the two-dimensional tracking problem discussed in Chap. This is generally the case in nonlinear filtering, and in [4], an even more significant observation is made.

For some coordinate basis selections, the extended Kalman filter may diverge and be effectively useless, whereas for other selections it may perform well. There are a number of variations on the above extended Kalman filter algorithm, depending on the derivation technique employed and the assumptions involved in the derivation.

Again, there 1 Sec. Any one of these algorithms maybe superior to the standard extended Kalman filter in a particular filtering application,but there are no real guidelines here, and each case has to be studied separately using Monte Carlo simulations.

For the case when cone-bounded non]inearities are involved in an extended Kalman filter, it maywellbe, as shownin [5],that the extended Kalman filter performs better if the non]inearities in the fi[ter are modified by tightening the cone bounds. This modification can be conveniently effected by introducing dither signals prior to the nonlinearities, and compensating for the resulting bias using a fi]tered version of the error caused by the cone bound adjustment.

Gaussian sum aPproach. There are non]inear algorithmswhich involve collections of extended Kalman fi]ters, and thereby become both more powerful and more complex than the algorithmof this section.

In these algorithms, discussed in a later section, the a posteriori density function p. In situations where densitycan be approximated the estimation error is small, the a posterior adequately by one gaussiandensity,and in this ca5ethe gau5siansum filter reduces to the extended Kalman filter of this section.

The following theorem gives some further insight into the quality of the approximations involved in the extended Kalman filter algorithm and is of key importance in demonstratingthe power of the gau5siansum algorithms of later sections. For the signal model through 2. I and 1. In these equations, it is assumed that the relevant probability densities exist, otherwise characteristic functions must be used.

Notice that the approximating quantity involves approximation of Zk — hk xk by the same quantity as in the approximate signal model of 2. Notice also that the approximation error is wrapped up in the quantity ek.

Using 2. The first part of the theorem then follows. Proof of the first claim is requested in Prob. Therefore, for any fixed s, The theorem is now established. As it stands, the theorem is of very limited applicability if 2. First, the a priori density p xO must be gaussian.

Even then, one is only guaranteed that 2. As it turns out, the gaussian sum filter of a later section provides a way round these difficulties. We consider the communications task of the demodulation of frequency- or phase-modulated signals in additive gaussian white noise, with the modulating signal assumed gaussian [8—11].

Of course, higher order state models can be constructed for messages with more sophisticated spectra. Continuous-time signal models such as above have not been discussed in the text to this point, nor is there a need to fully understand them in this example.

After z t is bandpass filtered in an intermediate frequency [IFl filter at the receiver and sampled, a discrete-time signal model for this sampled signal can be employed for purposes of demodulator design. Two methods of sampling are now described, and the measurementupdate equations of the extended Kalman filter derived for each case. A discussion of the time-update equations is not included, as these are given immediately from the equations for the state vector, which are linear.

Application of the extended Kalman filter to a specific problem, as illustrated in the example of FM demodulation, may require ingenuity and appropriate simplifications to achieve a reasonable tradeoff between performance and algorithm complexity.

Simulations may be required to determine under what conditions the filter is working well. Normally it is neeessary that the magnitude of the state error be comparable with the dimension of the region over which the nonlinear system behaves linearly.

Under certain conditions, it is sufficient for the magnitude of the mean square state error to be small. Formulate an iterated extended Kalman filter algorithm as follows. Generally this should be an improved estimate and thus give a more accurate linearization and thereby a more accurate signal model than 2. Is there any guarantee that the iterations will converge and achieve an improved estimate?

Derive extended KaIman fixed-lag smoothing algorithms. Establish the identity 2. Likewise establish the identity 2. The argument prior to the above definition and comparison of 3.

Let us formally state this fact; we leave the straightforward proof to the reader. If the inverse in 3. Though the theory will cover this situation, we shall assume existence of the inverse to keep life simple.

So the innovations model is also immediately computable from the Kalman filter. Again, the filter quantities K. The covariance itself 2.

The Kalman filter corresponding to the covariance 3. The covariance factorization problem is, of course, to pass from a prescribed covariance to a linear system with white noise input with output covariance equal to that prescribed. In this section, we are restricting attention to a finite initial time, but in the next section, we allow an initial time in the infinitely remote past, which in turn allows us to capture some of the classical ideas of spectral factorization see [l —1 9].

Nonstationary covariance factorization is discussed in [5, ], with [5] providing many of the ideas discussed in this section. References [] consider the continuous-time problem, with [30—33] focusing on state-variable methods for tack[ing it. Point 3 above raises an interesting question. Suppose there is prescribed a signal model 3. Equivalently, would the gain of the associated Kalman filter equal the quantity Kk in 3. The answer is yes, provided that Q is nonsingular. This completes Relations bet ween the Innovations the proof.

Model and the Kalman Filter In the course of proving Theorem 3. We also showed that the filter innovations process was identical with the input of the innovations model.

Consider a signal model of the form of 3. This comes about in the following way. Then the model must be of the form of 3. Then VI is known exactly. This proves the theorem. For this reason, one identifies the notion of innovations model and causally invertible model.

Other Types of Innovations Representation representations dealt with have been stateSo far, the innovations variable models. In the remainder of this section, we examine other types of representations—those associated with infinite-dimensional processes, and those associated with ARMA representations.

The causal inverti. Such a representation is essentially defined in Prob. The entries of T define the gkl, and the uniqueness of the factorization corresponds to the uniqueness of the innovations representation. An innovations representation of the process is provided by 3. Initial conditions arez. We obtain the results by setting up an equivalent state-space model to 3. The lemma shows how to connect case m s n. The Kalman filter for the state-variable model of 3. As initial conditions for 3.

Its important properties are as follows.