key: cord-0950806-ur7eu3k6
authors: Dewasme, Laurent; Wouwer, Alain Vande
title: Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking
date: 2021-12-31
journal: IFAC-PapersOnLine
DOI: 10.1016/j.ifacol.2021.10.246
sha: fde0bce3c325acc2c842043ac55a4cbb3bd8a1e8
doc_id: 950806
cord_uid: ur7eu3k6

In this work, the application of a model-free extremum seeking strategy is investigated to achieve the hypothetical control of the covid-19 pandemics by acting on social distancing. The advantage of this procedure is that it does not rely on the accurate knowledge of an epidemiological model and takes realistic constraints into account, such as hospital capacities. The simulation study reveals that the convergence has two time scales, with a fast catch of the transient optimum of the measurable cost function, followed by a slow tracking of this optimum following the original SIR dynamics. Several issues are discussed such as quantization of the sanitary measures.

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020) , for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t). The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020) , for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020) , for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020), for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system: Abstract: In this work, the application of a model-free extremum seeking strategy is investigated to achieve the hypothetical control of the covid-19 pandemics by acting on social distancing. The advantage of this procedure is that it does not rely on the accurate knowledge of an epidemiological model and takes realistic constraints into account, such as hospital capacities. The simulation study reveals that the convergence has two time scales, with a fast catch of the transient optimum of the measurable cost function, followed by a slow tracking of this optimum following the original SIR dynamics. Several issues are discussed such as quantization of the sanitary measures.

Keywords: real-time optimization, extremum seeking, covid-19 oubreak, SIR modeling

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020), for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020), for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020). These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020), for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

Fast transient optimization of social distancing during Covid-19 pandemics using extremum seeking Laurent Dewasme * Alain Vande Wouwer *

Since January 2020, our societies have been deeply impacted by the covid-19 pandemics. In this context, mathematical modeling and numerical simulation of the virus spread as a function of several factors, including social distancing, testing and quarantining, mobility restrictions and vaccination, have been playing a key role in the decision policy of many governments worldwide (McBryde et al., 2020) . These models provide predictions based on historical data and can be used to develop hypothetical control strategies. For instance, Tsay et al. (2020) propose an optimal open-loop control approach and suggest that on-off policies alternating between strict social distancing and relaxing can be effective at flattening the infection curve. Further, Köhler et al. (2020) investigate open-loop optimal control as well as model predictive control (MPC) with on-line adaptation of the social policy constraint, and robust MPC using interval state estimation to take account of uncertainties in the model and measurements. In the same spirit, Péni et al. (2020) develop a MPC control strategy taking account of time-dependent specifications and logical relations between model variables, and multiple predefined discrete levels of governmental interventions (control input quantization). As all the model variables are not accessible to measurements, it is necessary to develop state estimators in order to apply full-state feedback, which poses additional challenges. In (Köhler et al., 2020) , an interval observer is developed whereas an observer for Linear Parameter Varying (LPV) systems is designed in (Péni et al., 2020) .

As stressed in these publications and more globally in (Eker, 2020) , one can of course question the validity of the dynamic prediction models, and the objective of the present study is to investigate the use of model-free optimizing control, and particularly extremum seeking control (ESC; see for instance, Tan et al. (2010) ; Dewasme and Vande Wouwer (2020) , for reviews of ESC developments over the last decades), to infer control measures. The main condition to apply model-free ESC e-mail: laurent.dewasme,alain.vandewouwer@umons.ac.be is the existence of a measurable convex cost function, and a candidate function is proposed in this work.

The paper is organized as follows. The next section presents the epidemiological model used in (Tsay et al., 2020) as an emulator of the population behavior to test our ESC approach. Section 2.2 computes the equilibrium points of the model, and evidences a bifurcation behavior depending on the level of social distancing. In Section 2.3, a measurable cost function is proposed, which will use the concept of barrier functions, and serves as basis for ESC, which is discussed in section 3. The numerical application is detailed in section 4, where the two time scales of the convergence is highligted and the issue of quantization of the measures is introduced. The final section is dedicated to conclusions and research perspectives.

Epimedics modeling is classically based on compartmental population models describing the transitions between susceptible, infected and recovered (SIR) states. Regarding the COVID-19 outbreak, the compartmental SEAIR model of Tsay et al. (2020) is considered, which also accounts for the asymptomatic population A(t) (this class of individuals gathers cases which are not detected due to asymptomatic conditions, or due to the lack of testing) as well as the exposed population E(t).

The model also includes mortality, with a population P(t) who perishes in the epidemy. On a relatively short time span (corresponding to the study of the epidemy), the total population N can however be assumed constant as the death rate is fortunately quite small and other phenomena such as natality are not considered in the model. Depending on the point of view of the analysis, we will therefore consider the actual death rate or neglect it. The dynamics of the compartmental model can be written under the form of an ordinary differential equation system:

where N is the total population and S, E, A, I, R, and P are, respectively, the susceptible, exposed, unreported infected (or asymptomatic/unconfirmed), reported/confirmed infected, recovered and perished populations. The parameters α a and α i are the rates of exposure to the A and I populations, respectively. α a characterizes, in a broad sense, social distancing and α i , quarantining, and can be considered as manipulated (control) inputs from a system and control perspective, as well as the screening/testing rate κ. Constant (at least in first approximation) parameters account for the (inverse of the) latent period of the virus l which is set to 0.5 days −1 , the infectious period of unconfirmed cases ρ set to 0.1 days −1 and the recovery rate β set to 0.025 days −1 . These parameters are representative of the situation in the US in 2020 according to (Tsay et al., 2020) . The reader is referred to this publication for further details about parameter estimation. Of course, other parameter values may be adopted according to the region of the world, the status of the pandemics and the available data sets.

We first neglect the death rate µ and consider a constant total population N with a 5-compartment model, for which we compute two equilibrium points:

Point 1:

which corresponds to the extinction of the infection, and

Point 2:

which corresponds to the stabilization of the epidemy at some level. This latter solution is of course constrained by the positivity of the states.

Stability can be analysed locally considering the eigenvalues of the Jacobian matrix, which reads around an equilibrium point:

For the selected parameter values (shown in table 1), model (1) exhibits two different dynamic behaviors in the selected working range of α a = [0.05 0.4]. Figure 1 shows the evolution of the eigenvalues of (5) with respect to the input α a . It is apparent that the system presents a bifurcation as the eigenvalues switch values around a critical α a,c . Table 2 shows the evolution of the steady-state infected population as a function of α a . When the latter is smaller than α a,c , the state trajectories converge to the equilibrium 1 whereas, when α a is larger than this critical value, the system converges to equilibrium 2. In this case, the imposed social distancing allows stabilizing the pandemics but no longer extincting it. Fig. 1 . Evolution of the Jacobian eigenvalues of (5) with respect to social distancing α a , for a specific parametrization.

Several published studies report on optimal control and model predictive control of Covid-19 outbreak. These approaches require the knowledge of a dynamic model in the form of equations (1) and some provision to account for parameter uncertainties and provide some robustness to the control structure.

However, without effective and fast adaptation of the model parameters when the pandemic dynamics evolves, as it is currently observed with the appearance of mutant strains, the ef- fectiveness of these NMPC formulations may seriously drop. A model-free control strategy driven by a measurable cost function then becomes an appealing alternative allowing key parameter adaptation.

A typical measurable cost function may either aim at minimizing the number of fatalities under sanitary policy constraints (Köhler et al., 2020) or minimizing social distancing, focusing on psychological health (Tsay et al., 2020; Elie et al., 2020) , under constraints such as hospital bed capacity. The latter approach is selected in this work. The proposed optimizing control strategy therefore aims at minimizing the following cost:

where −α a represents social distancing while ψ and φ are respectively a logarithmic barrier on the infected cases and a penalty constraint on the comfort of social distancing:

where η ψ , η φ and ε are design parameters. I re f represents the critical level of infections, corresponding to a number of infected people which might lead to an overflow of intensive care hospitalizations. α a,re f is the penalty reference for social distancing, i.e., a level at which people will start feeling psychologically affected.

In order to avoid possible numerical issues during transient phases, Srinivasan et al. (2008) recommend to approach (7a) by a combined barrier-penalty expression such as:

which is active in the feasible region I(t) − I re f ≥ ε and

which is active in the complementary region and where η P is a new design parameter.

In this way, the cost function can be rewritten as Figure 2 shows the evolution of (10) as a function of α a for a specific design of the constraints reported in Table 1 . The continuous and dashed lines respectively correspond to the steady-state and transient (after 200 days) values of J.

The extremum seeking strategy that will be developed in the next section will have to converge to the minimum of this cost function as fast as possible, resulting in a "catch" (the transient optimum) and "track" (its change over time up to an hypothetical steady-state represented in Figure 2 as a curve with a sharp corner) policy. Obviously, the proposed cost function is convex and presents a unique extremum (minimum) but the derivative at the extremum is discontinuous, which is a situation that was already encountered in (Dewasme et al., 2011) , and has some potential challenge that will be discussed in the next section. 

Extremum seeking (ES) is a real-time optimization (RTO) strategy by direct input adaptation, which aims, by gradient descent (or ascent) of a measurable cost criterion, at reaching an optimum under necessary conditions of optimality (NCO). Let us cast model (1) and cost function (10) under the following generic nonlinear state-space form:

where x ∈ ℜ n is the state vector, u ∈ ℜ r the input vector, y ∈ ℜ m the output vector, C the m × n measurement matrix, J the cost function to be minimized.

In practice, for an ESC algorithm to work, a first assumption relates to the existence of a unique couple of minimizers x * and u * under achievable steady-state conditions, and a second assumption relates to the convexity of the cost function (Ariyur and Krstic, 2003) . These conditions are usually expressed in terms of mathematical conditions on the first and second-order derivatives of the cost function at the optimum, but this formulation is not exploitable here, as the derivative is not continuous at the optimum. In practice, this does not affect seriously the performance of a first-order steepest descent algorithm, which will locate the minimum and then oscillate around it (these oscillations can be damped or avoided as in (Dewasme et al., 2011) , but we will not consider this aspect here).

Even if the SEAIR model (1) of the COVID-19 outbreak is continuous, new measurement data occur with a daily sampling. A discrete perturbation-based ES formulation is therefore welladapted, as shown in Figure 3 . Fig. 3 . Discrete perturbation-based extremum seeking as proposed by Choi et al. (2002) .

The system input is excited by a periodic dither signal and the cost function measurement is high-pass filtered in order to recover the useful information at the dither frequency, rejecting the continuous component of h. The filtered signal, h HP , is then demodulated with the same dither signal, providing the cost criterion gradient estimateξ =∂ h ∂u . Eventually, this gradient is integrated to generate the input signal. The discrete perturbation-based ES of Figure 3 is therefore governed by the following equations:

where f HP is the high-pass filter cut-off frequency, k I the integrator gain, k is the discrete time variable and T s the sampling period.

Stability and convergence analysis are beyond the scope of this paper and could constitute further research work. The reader may however refer to Choi et al. (2002) and Ariyur and Krstic (2003) for additional elements about the stability conditions of the discrete perturbation-based ES as well as DeHaan and Guay (2005) , proposing an accurate analysis of the convergence of a discrete perturbation-based ES strategy under state constraints with barrier and penalty functions defined in (7).

The performance of the proposed ESC application to the SEAIR model (1) with the cost function (10) is assessed in numerical simulation. The ESC parametrization, based on the guidelines Guay, 2005) , is reported in Table 3 .

This table introduces a new parameter δ which aims at attenuating the magnitude of the dither signal a when approaching the optimum neighborhood and therefore evolving as follows (Tan et al., 2009) 

This attenuation will be particularly valuable in our situation (discontinuous derivative at the optimum). The results shown in Figures 4 to 6 demonstrate the fast convergence of the adaptive method. Examining Figure 5 , it can be observed that the convergence time of the input signal α a is approximately 100 days. However, applying daily changes to a sanitary policy is impractical as well as defining rules corresponding to each specific social distancing level. Figure 7 shows a brute force solution where the average input is discretized with a quantum of 0.05, offering a more adequate way to underpin decision making. A more rigorous approach could be the quantized ESC proposed in (Guay and Burns, 2019) , and is left for future research.

Observing the recent evolution of the pandemics (in 2020-21), this result seems quite realistic. Indeed, the sanitary policies applied by the governments have been periodically tighten and relaxed, instinctively searching for an optimum, as the proposed ES strategy does.

It is important to notice that the convergence to the optimum can be observed in two time scales: one corresponding to the convergence of the input where J enters the optimum neighborhood and another one which corresponds to the slow system (1) dynamics. Indeed, Figure 4 shows that I takes more than 3 years to vanish and Figure 6 highlights the remaining small distance between the true steady-state optimum (red star) and the transient optimum in t = 1000 days (blue star), proving that the current method is achieving a fast transient optimization with respect to the SEAIR dynamics. A faster extinction of I could be expected considering other means of action such as quarantining, testing, vaccination, and therapeutic treatment. A multivariable extremum seeking framework could be applied in further work, considering these additional means of action as potential manipulated variables (Dewasme et al., 2021) .

So far the mortality rate was neglected, and one may wonder about the effect of this phenomenon on the proposed ESC scheme. Two numerical simulations considering the mortality rate equal to 0.001 d −1 (relatively low) and 0.05 d −1 (high) are therefore achieved, without changing the ESC parametrization. Figures 8 and 9 show the effect of µ which accelerates the extinction of the infected population and, in turn, the system dynamics. As a benefitial side effect (in terms of control), the ESC reaches faster the steady-state optimum. However, mortality unfortunately entails a decrease of the population. This observation highlights the capability of the ESC strategy to optimize social distancing while limiting the number of infections. 

This paper presents an extremum seeking strategy that aims at determining an optimal level of social distancing so as to limit the effect of the covid-19 pandemics. The advantage of this strategy is that it does not rely on the knowledge of an accurate epidemiological model, but only on the measurement of a cost function related to a critical level of infection among the population and a bearable range of social distancing. The ESC is able to capture the optimum in a fast way and to subsequently follow the dynamic change in the cost function. The resulting policy consists in applying a set of discrete social distancing recommendations. Ongoing research entails multivariable extremum seeking acting on the combination of several measures (quarantining, testing, vaccination), possibly following the methodology recently developed in (Dewasme et al., 2021) . Another direction of research is the application of quantized and saturated ESC as developed in (Guay and Burns, 2019) . Choi, J., Krstić, M., Ariyur, K., and Lee, J. (2002) . Extremum seeking control for discrete-time systems. IEEE Transactions on Automatic Control, 47(2), 318-323. DeHaan, D. and Guay, M. (2005) . Extremum-seeking control of state-constrained nonlinear systems. Automatica, 41, 1567-1574. Dewasme, L., Srinivasan, B., Perrier, M., and Vande Wouwer, A. (2011) . Extremum-seeking algorithm design for fed-batch cultures of microorganisms with overflow metabolism. J. Process Control, 21 (7), 1092-1104. 

Real-time Optimization by Extremum-seeking Control

Model-free extremum seeking control of bioprocesses: A review with a worked example

Maximum-likelihood extremum seeking control of microalgae cultures

Validity and usefulness of covid-19 models

Contact rate epidemic control of covid-19: an equilibrium view

Extremum seeking control for discrete-time with quantized and saturated actuators

Robust and optimal predictive control of the covid-19 outbreak

Role of modelling in covid-19 policy development

Nonlinear model predictive control with logic constraints for covid-19 management

Tracking the necessary conditions of optimality with changing set of active constraints using a barrier-penalty function

Extremum seeking from 1922 to 2010

On global extremum seeking in the presence of local extrema

Modeling, state estimation, and optimal control for the us covid-19 outbreak