Cleveland State University

Department of Electrical Engineering and Computer Science


EEC 644/744, Optimal Control Systems


Homework Assignments

-          Homework problems that are labeled "Kirk" are taken from the book Optimal Control Theory, by Donald Kirk

-          The class id is 10248786 and the password is "optimal"


Monday January 22

1.      Kirk Problem 1.3 - use x1 = y and x2 = ydot

2.      Matlab: Numerically simulate the system of Problem 1.3. Plot the two states from the simulation. Also plot the analytical solutions that you derived. Compare the analytical results with the numerical simulation results. Hand in your Matlab code.

3.      In class: Linear systems quiz


Monday January 29

1.      Suppose that Q is a non-symmetric matrix, S is the symmetric part of Q, and x is an arbitrary vector. Prove the following:
x^T * Q * x = x^T * S * x.

2.      Show the instructor a copy of Kirk's book with your name written in the front.

3.      Prove the four properties in the left column of Table 1-1 in Kirk's book.

4.      Find the state transition matrices for the systems of Problem 1-12(e), (f), and (h) in Kirk's book.


Monday February 5

1.      Discretize the system of Kirk Problem 1.3. Compare your continuous-time and discrete-time simulation outputs to verify that you discretized it correctly.

2.      Simulate the discretized system of Kirk Problem 1.3 with a discrete-time LQR with an identity matrix for P(0) and Q.

a.    Plot the states for a reasonable time duration and find the magnitude of the largest closed-loop eigenvalue, for R = 0.1, 1, and 10.

b.   Explain how and why the value of R affects the system response and the closed-loop eigenvalues.

c.    For R = 1, plot the elements of the state feedback matrix as a function of time.

d.   Use the steady-state state feedback matrix (for R = 1) in your controller. How does system performance change relative to time-varying feedback control?

e.    Hand in your Matlab code.


Monday February 12

1.      Kirk problem 3.24. Don't try to solve it analytically - instead solve for the optimal control u numerically for both parts (a) and (b). Plot u for parts (a) and (b) and compare the solutions. Plot the states with x(0) = [1, 1]^T for parts (a) and (b) and compare the solutions.

2.      Repeat the above problem but replace the regulation objective with a tracking objective with r(t) = [sin(t), t/2]^T.


Wednesday February 21

1.      State the fundamental theorem of the calculus of variations.

2.      Find the extremals of the following functionals.

a.       The integral from a to b of [12*x*y + (y')^2] dx

b.      The integral from a to b of [y^2 + (y')^2] dx

3.      The brachistochrone solution provided in class was in the form x(x_f, theta_f, theta) and y(y_f, theta_f, theta). Verify that the provided solution satisfies 1 + (y')^2 + 2*y*y'' = 0. Hint: during this process you will have to calculate dtheta/dx. You can calculate it as 1 / (dx/dtheta).

4.      Plot the brachistochrone solution for the following final conditions on (x, y): (1, 1), (1, 3), and (3, 1). When you create your plots, use an aspect ratio of 1:1 so that your plot is not skewed; that is, one unit in the x-direction should be the same distance as one unit in the y-direction. Hint: You can use Matlab's fzero function to solve for theta at the final time. Then as x increments from 0 to its final value, you can use fzero at each increment to solve for theta, and then solve for y. Hand in your plots and your Matlab code.


Wednesday February 28

1.      Use the Lagrange multiplier technique to find the point in 3D space that is nearest the origin and that satisfies the constraints x1+x2+x3=5, and x1^2+x2^2+x3=9.

2.      Kirk Problem 4-10(b).

3.      A "natural catenary" is the shape and length of a hanging rope without a length constraint; that is, the shape and length of a hanging rope that minimizes potential energy. Consider the boundary conditions y(-1)=y(+1)=0.

a.       Calculate the natural catenary, and also calculate the constrained-length catenary with L=2.05 and L=2.9. Plot all three curves on the same plot.

b.      If ground level is at y=-2, what is the length L for which the hanging rope will just barely touch the ground?

4.      What implicit assumption did we make in our formulation of Dido's problem that makes our solution invalid if L > p*pi?


Wednesday March 7

Write a term project proposal describing a nonlinear system to which you plan to apply the optimal control methods of this class. Give the equations that describe the dynamics of the system. Provide a timeline that organizes the project into individual tasks. Describe what tasks you need to accomplish and when they will be done. Write your proposal in a formal way with correct formatting, referencing, etc. Your proposal should be between 5 and 10 pages. Submit your proposal to the "Proposal" assignment at by 11:59 PM. See the following references for general advice and guidelines related to technical writing.


Monday March 19

1.      Solve Zermelo's problem with V = 1, u(x,y) = -V*y, v(x,y) = 0, x(0) = 3.66, y(0) = -1.86, and x(tf) = y(tf) = 0. Hand in your commented Matlab code and plot your results.

2.      Solve Zermelo's problem with V=1, u(x,y) = 0, v(x,y) = -V*x, x(0) = 1, y(0) = -1, and x(tf) = y(tf) = 0. Hand in your commented Matlab code and plot your results.

3.      Solve the minimum-drag nose shape for a=1 and l=1, 2, and 4. Hand in your commented Matlab code and plot your results.


Monday March 26

1.      Consider the LQR problem for the system y'' = 0.02 * (u - y') with the state vector [y, y']^T, H = 0, Q = diag(1, 0), R = 1e-8, x(0) = [0.2, 0]^T, tf = 0.4.

a.       Plot the states, control, and feedback gains for the time-varying LQR. Calculate the cost. Hand in your Matlab code.

b.      Plot the states, control, and feedback gains for the steady-state LQR. Calculate the cost and compare it with part (a). Hand in your Matlab code.

c.       Repeat the above two problems with tf = 0.2.

d.      Repeat the steady-state LQR calculation with tf = 0.4 using the Hamiltonian matrix approach.

2.      Consider the following system and cost function with fixed final time:
x' = -x + u
J = integral of [exp(u) + (1-x)^2 * exp(-t)]

a.       Write the Hamiltonian.

b.      Write the Euler-Lagrange equations.

c.       Solve for u(t) as a function of p.

d.      Give a two- or three-sentence description of how you could solve this problem numerically.


Monday April 2

Midterm exam


Monday April 9

1.      Kirk problems 6.34 and 6.35 - hand in your source code, plots showing convergence, and plots of optimal control as a function of time.


Monday April 16

1.      Submit a term project progress report to the "Progress Report" assignment at

2.      Come to class prepared to give a 10-minute presentation on the status of your project.


Monday April 23

Consider a series RC circuit with a voltage source that has both a deterministic part and a random part. Define an LQR problem for the system and implement stochastic LQR control. Simulate the system many times and verify that the numerical value for E(J) matches the analytical expression for E(J) that we calculated in class.


Monday April 30

Oral project reports


Wednesday May 2

Oral project reports


Wednesday May 9

Comprehensive final exam, 12:30-2:30 PM


Friday May 11

Submit your term project final report to the "Final Report" assignment at


Professor Simon's Home Page

Department of Electrical Engineering and Computer Science

Cleveland State University

Last Revised: April 23, 2018