LIBRARY OF THE 
 
 UNIVERSITY OF ILLINOIS 
 
 AT URBANA-CHAMPAIGN 
 
 510 .fc* 
 
 TSKor 
 00.752-757 
 
 cop. 2* 
 
The person charging this material is re- 
 sponsible for its return to the library from 
 which it was withdrawn on or before the 
 Latest Date stamped below. 
 
 Theft, mutilation, and underlining of books 
 are reasons for disciplinary action and may 
 result in dismissal from the University. 
 
 UNIVERSITY OF ILLINOIS LIBRARY AT URBANA-CHAMPAIGN 
 
 QHl 1 3 1978 
 
 JUL12REcij 
 
 MAR 1 5 -jo*, 
 
 '■■ 
 HAY 6 198'' 
 
 L161 — O-1096 
 
Digitized by the Internet Archive 
 in 2013 
 
 http://archive.org/details/stochasticcontro752cutl 
 
0.1* 
 
 UIUCDCS-R-T5-T52 
 
 A STOCHASTIC CONTROL SYSTEM 
 
 by 
 James R. Cutler and David Ficke 
 
 June, 1975 
 
 1975 
 
 UNIVERSITY 01 
 
 DEPARTMENT OF COMPUTER SCIENCE 
 UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN 
 
 URBANA, ILLINOIS 
 
UIUCDCS-R-75-752 
 
 A STOCHASTIC CONTROL SYSTEM 
 
 BY 
 JAMES R. CUTLER 
 AND DAVID FICKE 
 
 June, 1975 
 
 Department of Computer Science 
 University of Illinois 
 Urbana, Illinois 6l801 
 
 This work was supported in part by Contract No. N0O0-1U-67-A-0305-002U. 
 
A STOCHASTIC CONTROL SYSTEM 
 
 James R. Cutler and David Ficke 
 
 Department of Computer Science 
 
 University of Illinois at Urbana-Champaign, 1975 
 
 ABSTRACT 
 
 This paper is concerned with a control system that uses stochastic 
 sequences as the means to reach the steady-state point. The main part of 
 the system is a temperature transducer which outputs a stochastic sequence. 
 The time average of the stochastic sequence is dependent upon the temperature, 
 This sequence is then used to control a heater. 
 
 A description of this system is included in this paper as well as the 
 appropriate theory and results. A system was also constructed. 
 
 11 
 
ACKNOWLEDGMENT 
 
 The authors wish to thank their advisor, Professor W. J. Poppelbaum, 
 for his suggestions and encouragement. They would also like to thank the 
 Electronic Circuits Lab for their help and assistance in the construction 
 of the control system. Lastly, they would like to thank the drafting shop, 
 the printing shop and the typist, Kathy Gee. 
 
 Hi 
 
TABLE OF CONTENTS 
 
 Page 
 
 1. INTRODUCTION 1 
 
 2. GENERAL DESCRIPTION 2 
 
 3. THEORY 5 
 
 k. CONCLUSIONS 11 
 
 LIST OF REFERENCES 12 
 
 APPENDIX 13 
 
 IV 
 
1& 
 
 1. INTRODUCTION 
 
 This project proposed by Professor W. J. Poppelbaum is concerned with 
 a control system in which the control signals are stochastic sequences. A 
 stochastic sequence is a random sequence of O's and l's. The control system 
 that is described is one which controls temperature. The system consists of 
 three parts: the temperature transducer, the temperature controller and the 
 
 heating element. The temperature transducer that is used is described in an 
 
 2 
 earlier paper. Its output is a stochastic sequence in which the time aver- 
 age is dependent upon the temperature. The temperature controller determines 
 whether the measured temperature is higher or lower than the desired tempera- 
 ture and turns on or off the heating element. 
 
2. GENERAL DESCRIPTION 
 
 Figure 1 shows the block diagram of the temperature control system. The 
 
 temperature transducer consists of a reverse-biased collector-to-base junc- 
 
 2 
 tion. When the voltage across this junction exceeds the breakdown voltage, 
 
 shot noise whose variance depends upon the junction temperature occurs. This 
 
 noise and a constant voltage are the inputs to a comparator. The constant 
 
 voltage is called the temperature reference in Figure 1. The output of the 
 
 comparator is obviously a stochastic sequence. The purpose of the temperature 
 
 controller is to determine the mean (or the time average) of the resulting 
 
 stochastic sequence. Depending upon whether this mean is above or below some 
 
 predetermined value, the controller will turn on or off the heating element. 
 
 There are a number of methods that may be used to determine the mean of the 
 
 stochastic sequence; however, the one chosen is a shift register shown in 
 
 Figure 2. The shift register shifts every T seconds - it shifts left if 
 
 Y(kT) = and shifts right if Y(kT) = 1. Notice that the l»s and O's are 
 
 sorted; i.e., the l's are on the left and the O's are on the right. Now 
 
 th 
 if the i bit is observed, there is a relationship between the probability 
 
 that the i bit is equal to 1 and the mean of the stochastic sequence, Y(t). 
 
 This will be shown in the next section. There are two comments about the 
 
 temperature controller: l) the circuitry in implementing it should be kept 
 
 simple and 2) the controller acts as an integrator. 
 
Q 
 li. 
 
 d 
 
 O 
 
 s 
 
 i 
 
 o 
 o: 
 
 
 
 z 
 
 > 
 
 
 
 o 
 
 z 
 
 
 
 o 
 
 UJ 
 
 
 
 1 
 
 
 
 
 
 
 
 
 I 1 
 
 5 S? 
 
 
 
 2 u 
 b 2 
 
 
 UJ 
 
 or 
 
 o: 
 
 < UJ 
 
 
 3 
 
 UJ 
 
 UJ -1 
 
 
 5 
 
 <r 
 
 UJ 
 Q. 
 
 2 
 
 UJ 
 
 u 
 
 X UJ 
 
 
 o 
 en 
 
 z 
 < 
 
 QC 
 
 1- 
 
 t 
 
 
 Q- uj 
 
 
 
 
 TEMPERATU 
 CONTROLL 
 
 
 
 
 
 
 
 
 ll 
 
 
 a 
 
 0) 
 
 -p 
 w 
 !>> 
 CO 
 
 o 
 
 U 
 -P 
 
 o 
 o 
 
 0) 
 
 Ih 
 
 -P 
 o3 
 U 
 
 <D 
 
 Ph 
 
 s 
 
 0) 
 Eri 
 
 <D 
 
 -p 
 
 £h 
 O 
 
 U 
 
 JjO 
 cd 
 
 •H 
 
 Q 
 
 o 
 o 
 
 H 
 
 pq 
 
 UJ Ul 
 
 q: o 
 
 3 Z 
 
 b u 
 
 < cr 
 
 or uj 
 
 uj u. 
 
 Q. UJ 
 
 2 a: 
 uj 
 
 0) 
 bO 
 
 •H 
 
o 
 
 II 
 
 II 
 
 
 ,»-» 
 
 1- 
 
 K 
 
 JC 
 
 4C 
 
 >- 
 
 >- 
 
 
 U. 
 
 u. 
 
 K 
 
 T 
 
 u. 
 
 UJ 
 
 O 
 
 -J 
 
 <T 
 
 h- 
 
 K 
 
 u. 
 
 U. 
 
 
 
 z 
 
 z 
 
 0) 
 
 (/) 
 
 ± 
 
 1 
 
 U ii 
 
 o 2 
 a: 
 
 UJ 
 Q. 
 
 o 
 
 
 
 o 
 
 
 • 
 
 
 0) 
 
 H 
 H 
 
 • 
 
 
 O 
 -P 
 
 • 
 
 It. 
 
 o 
 
 o 
 
 
 11 
 
 <u 
 
 
 
 ■p 
 
 o 
 
 
 • 
 
 cd 
 
 o 
 
 cu 
 
 ft 
 
 
 «^ 
 
 8 
 
 IMBwt 
 
 • ^ 
 
 QJ 
 
 
 
 EH 
 
 
 
 
 ~* 
 
 *- 
 
 
 
 2 
 
 
 
 i 
 
 a 
 
 • 
 
 z 
 
 •H 
 
 CU 
 CO 
 
 • 
 
 
 
 -P 
 
 • 
 
 
 CO 
 •H 
 
 cu 
 
 K 
 
 —■ 
 
 
 £ 
 
 
 •H 
 
 ^-1 
 
 
 ^3 
 
 CO 
 
 f 
 
 OJ 
 
 
 
 cu 
 
 1 
 
 
 £ 
 
 
 
 bO 
 
 •—1 
 
 
 
3 . THEORY 
 
 Figure 3 shows the actual impementation of the temperature control sys- 
 tem. From this diagram calculations will he made. The characteristics of 
 X(t) and Y(t) are discussed in reference 2. 
 
 The calculation of the mean of Z(t) is necessary. The D-type Flip-Flop 
 acts as a sampler - it samples Y(t) every l6.7 ms. Thus, each sample of Y(t) 
 is independent. The position of the leading 1 in the shift register may be 
 modelled by using a Markov chain, ^et tt. = the probability of bit i having 
 the leading 1, p = the probability of Y(t) = 1 and q = 1 - p. Then, 
 
 ^ = q TT + q 1T 2 
 
 *i = P Vl + g Vl for i=2, 3, ..., N-l 
 
 
 "n 
 
 = P Vl + P *N 
 
 Solving, 
 
 TT . 
 
 1 
 
 i-1 
 
 But, also 
 
 N 
 
 
 
 1 
 
 i=l 
 
 TT. = 1 
 
 1 
 
 So that 
 
 
 
 1- E. 1 - * i-1 
 
 tt = 9_ and ir. = V ^ for i = 1,2 , . . . ,N 
 
 1 !_(£)» * !.(£■)» * 
 
 V q. 
 
 From this, it is straightforward to calculate the probability, P, that the 
 L bit of the shift register is equal to 1 (or that Z(t)=l). 
 
 N 1-(E) „ i-1 - 
 
 p= I — Sr & 
 
 i=L l-(^-) N * 
 
 q. 
 
 L+l N 
 
 ■ s - ( «' 
 
-p 
 w 
 
 CO 
 
 H 
 O 
 
 O 
 
 o 
 
 3 
 
 <u 
 
 Ph 
 Eh 
 
 -P 
 
 o 
 
 S) 
 
 cd 
 
 •H 
 
 Q 
 ■P 
 
 •H 
 O 
 
 u 
 
 •H 
 O 
 
 UJ 
 (T 
 
 !S 
 
 or 
 
 UJ 
 
 Q. 
 2 
 UJ UJ 
 
 UJ 
 
 o 
 
 z 
 
 UJ 
 
 <r 
 
 UJ 
 
Obviously, P = E { Z(t)}. 
 
 At this point the characteristics of the transducer and temperature 
 controller are known. It is now necessary to model the characteristics 
 of the heating element. It is assumed that the temperature to be controlled 
 is exponential: i.e., 
 
 T(t) = T u - AT e" pt when Z(t) = 1 
 H 
 
 and T(t) = T + AT e" pt when Z(t) = 
 
 where T(t) = controlled temperature 
 
 T^,Tt = maximum, minimum tempera- 
 ture that the heating element can 
 produce 
 AT = differential temperature and 
 
 AT < T„-T_ . 
 — rl L 
 
 Using this model, it may be shown that the expected temperature, E{T}, 
 of the controlled environment will be: 
 
 E{T} = T T + P(T„ - T T ) where P = E{Z(t)} 
 Several comments can now be made: 
 
 l). The exponential model of the controlled temperature is a very 
 
 good approximation and experiments verify this. 
 2). The expected value of the controlled temperature has a linear 
 
 relationship with the expected value of Z(t). 
 3). Calculations are very involved and complicated and consequently, 
 
 not shown. 
 All the necessary information has been presented to show the steady- 
 state temperature of the control system. One of the necessary results is 
 that this temperature is dependent upon the setting of the reference 
 temperature, A. First, from the previous calculations it is possible to 
 graphically show the steady-state temperature of the control system. 
 
Figure k shows the necessary graphs: there is the response curve of 
 the transducer (two curves are shown but one is for a positive reference, 
 A, and the other is for a negative reference, A) and there is the curve 
 for the response of the temperature controller. The latter curve is 
 
 plotted with two assumptions: l). N is an even integer and 2). 
 
 N 
 L + 1 - 2 • Using these assumptions, 
 
 C p_)N/2 
 P = SL 
 
 1 + (*) N / 2 
 
 N/2 
 
 or E{T} = T T + 3- , — (T - T ) 
 
 L ! + (E.)N/2 U H V 
 
 The equation shown in Figure k is the inverse of the above equation. It is 
 observed that there are two intersections of the response curve(s) and 
 the temperature controller curve. (Actually, there is only one intersection 
 depending upon whether A is positive or negative). Steady-state point 1 
 is such that if there is a perturbation of the temperature, the steady-state 
 value of the temperature will be approached monotonically as shown in 
 Figure 5. Steady-state point 2 is such that if there is a perturbation of 
 the temperature, the steady-state value of the temperature will be approached 
 alternately as shown in Figure 6. With the change of A, the response curve 
 of the transducer shifts to the right or the left. The shape of the curve 
 reamins intact. Therefore, the change of the temperature reference, A, does 
 have an effect on the steady-state temperature. Note that the shifting of 
 the transducer response curve to the left can make the control system unstable, 
 The slopes of the two curves are such that a perturbation will make the 
 system diverge from the steady-state point (unstablility) . Thus, there is 
 a range of values that the temperature reference may take so that the system 
 reamins stable. 
 
 8 
 
I- 
 
 i±j 
 u 
 
 o. 
 
 STEADY-STATE POINT 1 
 
 RESPONSE CURVE FOR 
 THE TRANSDUCER 
 
 TEMPERATURE 
 
 Figure k. Graph of the Steady-State Temperature of the Control System 
 

 TEMPERATURE 
 
 Figure 5. Approach to the Steady-State Point 1 with a 
 Perturbation of the Temperature 
 
 TEMPERATURE 
 
 Figure 6. Approach to the Steady-State Point 2 with a 
 Perturbation of the Temperature 
 
 10 
 
k . CONCLUSIONS 
 The constructed control system as described in the Appendix behaved 
 as expected and performed as well as other types of thermostats (such as 
 a bimetallic strip thermostat). This system also has the advantage that 
 it is wholly electronic and that it uses simple, commonly-used integrated 
 circuits (a comparator and a shift register). The unique quality of this 
 control system is that the control signal is a stochastic sequence . 
 
 11 
 
LIST OF REFERENCES 
 
 1. Poppelbaum, W. J., Proposal for the Office of Naval Research, Department 
 
 of Computer Science, University- of Illinois, June 197^. 
 
 2. Cutler, J. R., "Molecular Stochastics: A Study of Direct Production of 
 
 Stochastic Sequences from Transducers," Department of Computer Science 
 No. 723, University of Illinois, Urbana, Illinois, April 1975. 
 
 12 
 
APPENDIX 
 
 13 
 
>> 
 
 H 
 ft 
 
 CO 
 
 o 
 
 PM 
 QJ 
 ■P 
 
 O 
 
 I 
 
 !h 
 bfl 
 cd 
 
 •H 
 
 Q 
 
 2 
 
 •H 
 O 
 
 •H 
 
 lU 
 
a 
 
 CD 
 -P 
 CO 
 
 >» 
 
 CO 
 
 H 
 O 
 U 
 
 -P 
 
 o 
 o 
 
 0) 
 Sh 
 
 p) 
 
 CD 
 EH 
 
 CD 
 ,C 
 -P 
 
 «H 
 
 o 
 
 bO 
 
 Q 
 
 2 
 o 
 
 •H 
 CJ> 
 
 CM 
 
 I 
 < 
 
 CD 
 
 3 
 
 M 
 •H 
 
 15 
 
SECURITY CLASSIFICATION OF THIS PAGE (When Data Bntarad) 
 
 REPORT DOCUMENTATION PAGE 
 
 READ INSTRUCTIONS 
 BEFORE COMPLETING FORM 
 
 I REPORT NUMBER 
 
 UIUCDCS-R-75-752 
 
 2. OOVT ACCESSION NO 
 
 J. RECIPIENT'S CATALOG NUMBER 
 
 4 TITLE (and Subtltlm) 
 
 A STOCHASTIC CONTROL SYSTEM 
 
 I. TYPE OF REPORT • PERIOD COVERED 
 
 Technical Report 
 
 • PERFORMING ORG REPORT NUMBER 
 
 7. AUTHORf«j 
 
 James R. Cutler and David Ficke 
 
 ■ . CONTRACT OR GRANT NUMBER("»J 
 
 N000-1U-67-A-0305-002U 
 
 9 PERFORMING ORGANIZATION NAME AND ADDRESS 
 
 Department of Computer Science 
 
 University of Illinois at Urbana-Champaign 
 
 Urbana, Illinois 6l801 
 
 10. PROGRAM ELEMENT. PROJECT. TASK 
 AREA * WORK UNIT NUMBERS 
 
 II. CONTROLLING OFFICE NAME AND ADDRESS 
 
 Office of Naval Research 
 219 South Dearborn Street 
 Chicago, Illinois 6060U 
 
 12. REPORT DATE 
 
 June 1975 
 
 IS. NUMBER OF PAGES 
 
 2k 
 
 U MONITORING AGENCY NAME ft ADDRESSCi/ dlllarant from Controlling Olllea) 
 
 IS. SECURITY CLASS. (o( thla report) 
 
 15a. DECLASSIFICATION/ DOWNGRADING 
 SCHEDULE 
 
 16. DISTRIBUTION STATEMENT (oi lhl» Report) 
 
 17. DISTRIBUTION STATEMENT (ol tha abstract antarad In Block 20, II dlllarant from Report) 
 
 !8. SUPPLEMENTARY NOTES 
 
 k 
 
 I s 
 
 s 
 
 19. KEY WORDS (Contlnua on tavaraa alda II nacaaaaiy and Identity by block numbar) 
 
 Stochastic processing 
 Stochastic sequence 
 Stochastic control system 
 
 20 ABSTRACT (Continue on 
 
 tary and Identity by block number; 
 
 This paper is concerned with a control system that uses stochastic se- 
 quences as the means to reach the steady-state point. The main part of the 
 system is a temperature transducer which outputs a stochastic sequence. The 
 time average of the stochastic sequence is dependent upon the temperature. 
 This sequence is then used to control a heater. 
 
 A description of this system is included in this paper as well as the 
 appro pri ate theory and results. A system was also pnnRtn^t.pii . 
 
 DD , JA r 7 3 1473 
 
 EDITION OF I NOV 66 IS OBSOLETE 
 
 S/N 0102-014-6601 | 
 
 SECURITY CLASSIFICATION OP THIS PAOt (Whan Data Bntarad) 
 
BIBLIOGRAPHIC DATA 
 SHEET 
 
 1. Report No. 
 
 UIUCDCS-R-75-752 
 
 2- 
 
 3. Recipient's Accession No. 
 
 4, i i ■ I.- .iinl Sum it !< 
 
 A STOCHASTIC CONTROL SYSTEM 
 
 5- Report Date 
 
 June, 1975 
 
 6. 
 
 7. VjiK.'tls) 
 
 James R. Cutler and David Ficke 
 
 8. Performing Organization Rept. 
 
 No -UIUCDCS-R-75-752 
 
 9 i (•. Mining Urbanization Name and Address 
 
 Department of Computer Science 
 
 University of Illinois at Urbana-Champaign 
 
 Urbana, Illinois 6l801 
 
 10. Project/Task/Work Unit No. 
 
 11. Contract /Grant No. 
 N000-1U-67-A-0305-002U 
 
 ■ soring Organization Name and Address 
 
 Office of Naval Research 
 219 South Dearborn Street 
 Chicago, Illinois 6060U 
 
 13. Type of Report & Period 
 Covered 
 
 Technical Report 
 
 14. 
 
 1 plcmcmary Notes 
 | 
 
 1« lhMMC,« 
 
 This paper is concerned with a control system that uses stochastic sequences as the 
 jneans to reach the steady-state point. The main part of the system is a temperature 
 transducer which outputs a stochastic sequence. The time average of the stochastic 
 [sequence is dependent upon the temperature. This sequence is then used to control a 
 heater. 
 
 A description of this system is included in this paper as well as the appropriate 
 theory and results. A system was also constructed. 
 
 j 
 
 1 
 
 ! 
 
 1 1? K' \ ft'ords and Document Analysis. 17o. Descriptors 
 
 Stochastic processing 
 Stochastic sequence 
 Stochastic control system 
 
 17b. Identifiers .'Open-Ended Terms 
 17c. COSAT1 Field/Group 
 
 18 •vail.tbility Statement 
 
 19. Security Class (This 
 Report) 
 
 UNCLASSIFIED 
 
 21. No. of .Pages 
 2k 
 
 Unlimited distribution 
 
 
 20. Security Class (This 
 Page 
 
 UNCLASSIFIED 
 
 22. Price 
 
 
 JM N TIS- 35 ( 10-70) 
 
 USCOMM-DC 40329-P7I 
 
ocr lo |75 
 
\* 
 
 d* 
 
 **