Cleveland State University
Department of Electrical and
Computer Engineering
EEC 644/744, ESC 794
Optimal Control Systems
Fall 2009
Homework Assignments
Homework problems are taken from the textbook “Optimal Control Theory,” by Donald Kirk.
Homework 1 due August 31
Kirk problems 1.2, 11, 13(a), (b), (i)
Matlab: Simulate the system of Problem 1.2. Plot the two states. Also plot the analytical solutions derived in Problem 1.2. Compare the analytical results with the simulation results. Hand in your source code.
Homework 2 due September 9
Consider the parametric cart control problem.
1. Assume that the control is a constant. Give the control equation, the final position, and the integral of u^2 for the following cases: p/r = 100, p/r = 1, p/r = 0.01, and p/r = 0.001.
2. Repeat problem 1 under the assumption that the control is a linear function of time.
Homework 3 due September 16
Kirk problem 3.13
Matlab: Simulate the nonlinear inverted pendulum system controlled by a discrete-time LQR. You will need to linearize and discretize the system in order to derive the LQR. Use the system parameters that are in Pendulum.m on the course web set. Use an identity matrix for P(0) and Q.
(a) Plot the states between 0 and 6 seconds, and give the magnitude of the largest closed-loop eigenvalue, for R = 0.1, 1, and 10. Explain how the value of R affects the system response and the closed-loop eigenvalues.
(b) For R = 1, plot the elements of the state feedback matrix as a function of time. Use the steady-state state feedback matrix in your controller. How does system performance change compared to the time-varying feedback controller?
Homework 4 due September 23
Problem 1: Kirk problem 3.23. Don’t try to solve analytically, just solve for K numerically for both parts (a) and (b). Plot K for both parts (a) and (b) and compare solutions.
Problem 2: Given xdot = x + u, use the HJB equations to analytically solve for the optimal control when J = x^2(tf) + integral (r u^2) dt. Given x(0) = 5, plot x(t) for a couple of different values of r to show how the value of r affects the state trajectory. Also find the analytical solution to the steady-state controller as a function of r.
Hint: Assume that J* = s(t)[x(t)]^2, where s(t) is a function to be determined. This should eventually lead you to the equation sdot = s^2/r – 2s. This can be written as ds / (s^2/r – 2s) = dt. If you know s(tf), you can integrate both sides and analytically solve for s(t).
Homework 5 due September 30
(1) State the fundamental theorem of the calculus of variations.
(2) Kirk problem 4.2 (Hint: Use proof by contradiction. Show that if h(t) is not zero, then there exists some continuous delta-x that gives a nonzero integral.)
(3) Kirk problem 4.11
(4) Find the extremals of the following functionals.
(a) the integral from a to b of [(dy/dx)^2 / x^3] dx
(b) the integral from a to b of [y^2 + (dy/dx)^2 + 2*y*exp(x)] dx
(5) Plot the solution to the brachistochrone problem for the following final conditions on (x, y): (1, 1), (1,5), and (5, 1). Hint: You can use Matlab’s fzero function to solve for theta at the the final time. Then as x increments from 0 to its final value, you can use fzero at each increment to solve for theta, and given theta, you can solve for y. Hand in your plots and your Matlab code. When you create your plots, use an aspect ratio of 1:1 so that your plot is not skewed; that is, one unit of distance in the x-direction should be equivalent to one unit of distance in the y-direction on your plot.
Homework 6 due October 7
As preparation for your term project, write a project proposal describing some nonlinear system to which you would like to apply optimal control. Give the equations that describe the dynamics of the system. Provide a timeline breaking down the term project into individual tasks. Describe what tasks you need to accomplish, and when they will be done. Write your proposal in a formal way with correct formatting, referencing, etc. Your proposal should be at least four pages. See the following for advice and guidelines related to technical writing:
http://academic.csuohio.edu/simond/courses/writing/
http://academic.csuohio.edu/simond/courses/ReportTemplate.pdf
Doctoral Students: Hand in references and abstracts of five optimal control papers published within the last 10 years that you are considering reviewing for this class.
Homework 7 due October 19
Kirk problems 5.16, 5.17(a), and 5.23. For problem 5.23(d), note that tf and y(tf) are the only constraints at the final time.
Midterm October 21
Homework 8 due October 28
Write a simulation of the system that you chose for your term project. You can use any type of controller that you want. You can also add noise if you want. Hand in a report discussing the simulation and why you might want to improve the results. Formulate a cost function for your system. Derive the necessary conditions (e.g., Euler-Lagrange equations) for the optimal controller. Prepare a 10-minute presentation for class. Your report should be self-contained - that is, I should not need to look back at your previous report (homework 6) in order to understand this report.
Homework 9 due November 4
Kirk problem 6.34 - Hand in your source code, plots showing convergence of gradient descent, and plots of optimal control as a function of time.
Homework 10 due November 16
Solve the Euler-Lagrange equations for your nonlinear optimal control project problem. Use either gradient descent, variation of extremals, or some other method. Your grade will be based on your effort (including the difficulty of your problem), your results, and the quality of your write-up.
Homework 11 due November 23
Kirk problem 5.28.
November 30
Written journal paper review/discussion due from doctoral students (hard copy). Hand in a copy of the paper that you are reviewing and your written review/discussion. Oral presentations of journal paper reviews/discussion will be given in class by doctoral students, 10 minutes per student.
December 2
No written homework due. Term project presentations will be given in class by all students, 10 minutes per student. Focus on progress and results since your previous presentation of October 28.
December 7
Comprehensive final exam.
December 9
Written term project due at www.turnitin.com. Class id = 2985066, enrollment password = optimal.
Department of Electrical and Computer Engineering
Last Revised: November 11, 2009