Here is a list of good examples to solve in Åström: Introduction to Stochastic Control.

• Chapter 2

  2:3: 4, 5, 6, 9
  2:4: 1, 2, 5
  2:5: 1, 2

• Chapter 3

  3:3: 1, 2, 4
  3:6: 2, 3, 5, 6, 7, 8
  3:8: 1, 2, 3
  3:9: 1, 2
  3:10: 1, 4

• Chapter 4

  4:2: 1, 4, 6
  4:3: 1, 2, 3, 4
  4:5: 1, 2, 3

• Chapter 5

  5:2:1
  5:3:1, 2
  5:4: 1, 2
  5:5: 1, 2, 3

• Chapter 6

  6:3: 1, 2, 3, 4, 5, 6, 7
  6:4:1, 2, 3, 4, 7, 8, 9, 12
  6:5: 1, 3, 4, 5

• Chapter 7

  7:2: 1, 2, 5
  7:3: 1, 2, 3
  7:4: 1, 2, 3, 4, 5, 6
  7:5: 1
  7:6:1, 2, 5, 7, 8

• Chapter 8

  8:3: 1
  8:4: 1, 2, 3
  8:5: 1, 2, 3, 4
  8:6: 1, 2, 3, 5, 6
  8:7: 1, 3, 6