Hybrid Modelling for On-Line Penicillin Fermentation Optimisation

Copyright © 2002 IFAC 15th Triennial World Congress, Barcelona, Spain HYBRID MODELLING FOR ON-LINE PENICILLIN FERMENTATION OPTIMISATION M. Ignova, G.C. Paul+, C.A. Kent+, C.R. Thomas+, G.A. Montague, J. Glassey, and A.C. Ward* Department of Chemical and Process Engineering, University of Newcastle, Newcastle upon Tyne, NE1 7RU,UK * Department of Agriculture and Environmental Science, University of Newcastle upon Tyne, Newcastle upon Tyne, NE1 7RU, UK + BBSRC Centre for Biochemical Engineering, School of Chemical Engineering, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK Abstract: This paper describes the procedures that are necessary to arrive at a model that is sufficiently accurate to be used in an on-line penicillin fermentation optimisation scheme. A structured mechanistic model developed previously was available but this model failed to account for the effects of low levels of dissolved oxygen on growth and production. When moving towards optimising the fermentation, dissolved oxygen becomes the parameter limiting process productivity and hence it is important to be able to account for its process influence. Terms predicting dissolved oxygen changes and describing its effect when at low levels were included in the model to compensate for the reduction in growth and production. Even so, the natural variation experienced in the fermentation resulted in process / model mismatch. On line correction of model coefficients via an observer approach provided the accuracy required for optimisation purposes. The improvements in the accuracy of the model predictions are demonstrated. Keywords: Fed-batch fermentation, hybrid modelling, artificial neural networks, penicillin fermentation 1. INTRODUCTION calculate the optimal profiles during the batch and fed- batch mode of operation, a pre-defined function such as Bioprocess engineers have strived for many years to polynomial or more complicated shapes can be used to produce accurate models describing fermentation describe the variations in the manipulated variables. process behaviour. The driving force behind this is that Once parameterised, various optimisation methods can once a representative model is built, it can be used in be applied to find the coefficients of the pre-defined fault diagnosis, performance estimation and prediction, function. Further details of this approach can be found scheduling and optimisation. The work described in this in Rodrigues and Filho (1996) and Montague and Ward paper concentrates on optimisation specifically, with (1994). particular application to a fed-batch penicillin fermentation. At the heart of the optimisation algorithm is a model of the process to be optimised. Although there are many In batch and fed-batch bioprocesses, the maximum subtle variations in model form, in general models performance is achieved by optimising the initial belong to one of two categories: mechanistic or conditions and subsequent profiles of manipulated empirical. Empirical models are generally structured variables such as feed rates, temperature and pH during and do not take account of the microbiological the process operation. However, the industrial use of behaviour from a fundamental understanding models for optimisation purposes has been limited, with perspective. Mechanistic models, on the other hand, a more heuristic procedure often being preferred. attempt to approximate these characteristics in Although this has resulted in considerable mathematical form. Since mechanistic models capture improvements in performance, the application of physical behaviour, they have the potential to be more modern mathematical optimisation theory offers the precise than empirical models. This is particularly the potential of even greater benefits. Classical approaches case when considering extrapolation beyond the regions to solving the optimisation problem exist such as for which the model was constructed. Pontryagin’s Maximum Principle. An overview of the topic is given in Constantinides (1979). Alternatively, to Numerous mechanistic models have been published to the use of the model together with the optimisation date describing the penicillin fermentation (Heijnen et scheme. Space precludes coverage of both modelling al., 1979; Bajpai and Reus, 1981; Nicolai et al., 1991). and optimisation strategies. Fundamental to the success However, when industrial application is considered, it is of the optimiser is the model. This paper therefore generally found that existing model structures fail to concentrates upon obtaining a model of the penicillin take into account important issues. For instance, the fermentation that has sufficient accuracy to be used as variation in environmental conditions in large-scale the basis for optimisation. operation can be difficult to model with any degree of certainty. An alternative approach is to use empirical models. Many forms of empirical model have been 2. EXPERIMENTAL SYSTEM applied to bioprocesses. These range from linear (Linko et al., 1992) to non-linear (Montague et al., 1992; An industrial strain of Penicillium chrysogenum grown Warnes et al., 1996). While success with these in a 5L bioreactor in the Centre for Bioprocess techniques has been reported, they do have major Engineering at Birmingham (as described in Paul et al., drawbacks. Namely, the validity of these models outside 1998), was used for experimental trials. Eight fed-batch the region of the training data is limited. Ideally what is penicillin fermentations were carried out using a range required is a model which possesses the advantages of of fixed and variable feed profiles with the standard on- both the mechanistic and empirical approaches. The and off-line variables monitored. Additionally, product hybrid modelling concept offers this potential. and residual glucose concentrations were made available using an on-line HPLC system, and off-gases As with all modelling techniques there are a number of were analysed using a mass spectrometer. To further options available. One approach is to couple artificial enhance the monitoring, off-line image analysis neural networks (ANNs), an empirical modelling provided information on the physiological state of the technique, with a mechanistic model. Fundamentally, culture. The process information was monitored and the there are two main strategies: serial and parallel. manipulated variables controlled by a SETCIM control Psichogios and Ungar (1992) proposed the serial system (AspenTech, Inc.). An Internet link using approach to model batch and fed-batch fermentation SequeLink Connect Administrative Tool was set up processes. The ANN was used to model variations in the between SETCIM in Birmingham and a monitoring specific growth rate, which was then utilised as a system at Newcastle University. The link allowed all the parameter in the mechanistic model. This approach has SETCIM data to be collected, saved and used in real also been used to derive a model for fed-batch time at Newcastle and information ascertained from the production of a recombinant protein from mammalian model returned to Birmingham. cells (Dors et al., 1995). Su et al. (1992) proposed a parallel hybrid method where the ANNs compensate for the difference between the mechanistic model and the 3. MECHANISTIC MODEL UPDATING real process. Thompson and Kramer (1994) proposed a hybrid model that combines the serial and parallel Paul et al. (1998) developed a comprehensive structured approach. They illustrated the methodology by model describing the behaviour of the penicillin predicting the biomass and product concentration for a fermentation. Verification against experimental results simulated penicillin process. The integration of ANNs demonstrated that the model gave good predictions of with mechanistic models has also been applied to fermentation behaviour but only when the operating improve the prediction accuracy of the state variables of policy was to set carbon feeds at such a level as to mammalian cull cultures by Fu and Barford (1996). De maintain levels of dissolved oxygen (DO2) higher than a Azevedo et al. (1997) compared a hybrid model, critical value. Unfortunately such conditions may not consisting of an ANN and mechanistic model, with the prevail when optimisation of the fermentation is prediction capabilities of the individual models for fed- considered. When optimising the fermentation process it batch yeast fermentation. Their investigation is the DO2 levels that form the constraint that the demonstrated that the hybrid model proved to be more optimiser pushes the process towards. Put simply, high effective. ANNs are not the only model form used to levels of active biomass means more penicillin enhance mechanistic model predictions. For instance, producing cells. However, too high a biomass Schubert et al. (1994) developed a hybrid model of fed- concentration leads to mass transfer limitations and a batch yeast fermentation by integrating a variety of resulting fall in DO2. In this case as DO2 falls firstly modelling techniques such as: ANN, mechanistic model penicillin production becomes inhibited and if it falls and heuristic knowledge using fuzzy rules. Johansen further active biomass is destroyed. Ideal conditions are and Foss (1997) proposed a framework that combines therefore high biomass just avoiding the effects of DO2 different kinds of local empirical and mechanistic limitation. These effects are confirmed by the studies of models into a global model. Henriksen et al. (1997). They studied the influence of the DO2 concentration on penicillin biosynthesis in Whatever optimisation strategy adopted it is essential in steady-state continuous cultures. They showed that at the first instance to have a model for the optimiser to low DO2 concentration, the specific penicillin utilise. It is not the objective of this paper to describe productivity decreases and further lowering the DO2 concentration results in loss of product. However, they rates ( dp dt ) was then plotted against the DO2 data claimed that the penicillin productivity was instantly dp max dt recovered to its maximum value when the DO2 (given as circles in Fig. 1). The parameters Kp and Np concentration was reset to a value above the limited (eqn. 2) were optimised using this information and non- DO2 values. In their case, levels of DO2 critical to linear least squares. The optimised parameters are Kp= biomass were not reached. 7.03 and Np= 3.93. The estimated relation was plotted as a solid line in Fig. 1. Unfortunately, the structured model developed by Paul and Thomas (1998) does not contain terms describing 1 the influence of the DO2 on the biomass and product 0.9 actual formation. From the considerations above it is clear that 0.8 estimated such terms are vital for optimisation purposes. The DO2 0.7 influence on biomass and product concentration can be (dp/dt)/(dpmax/dt) 0.6 incorporated within the mechanistic structured model 0.5 using the following relations (equation 1) (Paul (1999)): 0.4 0.3 1 (1) DOx = Nx 0.2 æ K ö 1 + çç x 0.1 è DO2 0 0 10 20 30 40 50 60 70 80 90 100 DO2 1 (2) DO p = Np Fig. 1: The relation between dp dt and DO2 æ K ö 1 + çç p dp max dt è DO2 measurements The DOx term (eqn. 1) was used to modify the rate 1 expression for branch formation ( rb,o ) and extension 4. HYBRID MODEL DEVELOPMENT 1 The mechanistic model modified to take into account rate ( re,1 ), while the DOp term (eqn. 2) was used to low DO2 levels is not suitable for use in an optimisation 1 modify the rate of product formation ( rp ) in the scheme. DO2 varies primarily as a result of carbon equations given in Paul et al. (1998) in the following feedrate variation. An optimisation scheme requires a manner: model that when feed profiles are specified is able to predict penicillin production levels. DO2 is therefore a model output, although it obviously influences other rb,o = DO x ⋅ rb,1 o (3) states. re,1 = DO x ⋅ r 1 e,1 (4) A serial hybrid model, consisting of a feedforward rp = DO p ⋅ r 1 p (5) artificial neural network (FANN) estimating DO2 values and the structured model including the DO2 influence on the product formation, is proposed in 2. This structure In this application study, only the DO2 influence on can predict the product concentration by taking into the product formation was considered, so the parameter Kx account the DO2 influence. was equal to 0 and Nx to 1. The reason why this is the case comes from a fundamental appreciation of the process. Significant loss in penicillin production occurs Feeds Product Mechanistic model before cell growth becomes affected to any extent by Other inputs (structured) low levels of DO2. The optimum conditions for operation are obviously where penicillin production is DO2 estimate not significantly inhibited. Therefore the region over which the model is required to be accurate does not ANN model stretch into regions where DOx would be significantly different from unity. The parameters Kp and Np were obtained using real experiment data (Fermentation 6 in Fig. 2: Proposed hybrid model structure Fig. 3) and the simulations using the structured model without the DO2 limitation. The production rate dp/dt The FANN was developed off-line using the was calculated from the experimental data, while the experimental data presented in Fig. 3. The inputs to the maximum production rate dpmax/dt was obtained from FANN were the feed, total feed, volume and the batch the model simulations. The ratio between these two age. From the total of eight fermentations (Fig. 3), four  0.04 9 0.035 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8 7 0.03 Product concentration 6 0.025 Feed rate 5 0.02 4 0.015 3 0.01 2 0.005 1 0 0 0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000 Sample Number Sample Number a) b) 110 35 100 1 2 3 4 5 6 7 8 30 1 2 3 4 5 6 7 8 90 Residual glucose concentration 80 25 70 20 60 DO2 50 15 40 30 10 20 5 10 0 0 0 100 200 300 400 500 600 700 800 900 1000 0 100 200 300 400 500 600 700 800 900 1000 Sample Number Sample Number c) d) Fig. 3: Fermentations used for this application study 120 Training data Testing data (-) actual (-.) estimated 100 80 DO2 60 40 20 0 0 100 200 300 400 500 600 700 800 900 1000 Sample Number Fig. 4: Actual and estimated DO2 vs. Sample Number fermentations were used for training (Fermentations 4, product concentration (Pest) is firstly determined. In this 5, 6 and 8) and the remaining four for testing case an on-line HPLC supplied frequent measurements (Fermentations 1, 2, 3 and 7). The batches to be used for of penicillin concentration. The error was used to adjust training / testing were selected in order to get wide µp using PI feedback correction. In general observer model coverage. The optimal topology was 3 nodes in a speed of response is a balance between tracking error single hidden layer and was determined by carrying out and amplification of noise. The observer gains and the multiple training runs. The actual and estimated DO2 sampling interval in this case were obtained empirically measurements for the training and testing data are using the experimental data so that an acceptable presented in Fig. 4. tracking of product concentration was obtained without making changes that were too significant to µp. 5. MODEL / PROCESS MISMATCH CORRECTION Following off-line algorithm adjustments, the algorithm was implemented on-line. Results of the first trial are If the above model were used in a real-time optimisation presented below in Figs. 6 and 7. scheme then any discrepancies between the process and the model would lead to non-optimal behaviour. 0.05 Although reasonably accurate in its present form, 0.04 µp_updated improvements can be made. In particular, during the 0.03 fermentation run the on-line information can be utilised 0.02 - K=0.1 to update the model, and subsequently the updated 0.01 * K=0.02 model can be used to re-optimise the feed in real time. 0 0 20 40 60 80 100 120 The model can be updated using parameter re- Time [h] optimisation of the hybrid model (either the mechanistic or ANN model). However, this is not practical to Fig. 6 - Variations in corrected µp over the fermentation implement in a real-time environment. Apart from the time involved, selection of appropriate data for model Fig. 6 shows that no model correction takes place until building requires manual intervention. As an alternative, twenty five hours into the batch. The reason for this is only a few parameters could be updated and in this case, that no penicillin is produced until after this time. the specific production rate (µp) was chosen. µp has a Following that some severe changes are made in µp direct and significant effect on the penicillin production (denoted by the continuous line in Fig 6). These are as a rate. result of the observer responding to measurement noise. This indicates that a reduced observer gain is required Fig. 5 presents the proposed scheme for the on-line when the model is implemented within the optimisation model correction procedure. It also shows how the scheme. The improvements possible with this reduced model sits within the optimisation algorithm producing gain are indicated by * in the Figure. carbon feed profiles. 8 Other Feeds Process Measurements 6 Fermentation P [g/l] Other inputs Penicillin 4 2 Predicted 0 Penicillin + 0 20 40 60 80 100 120 _ Σ Mechanistic model (structured) Time [h] DO 2estimate µp Fig. 7 - Comparison of model prediction and measured ANN correction model penicillin concentration Hybrid Model Hybrid Model correction Fig. 7 shows that there is very close agreement between measured and model predicted profile. The dashed line in Fig. 7 shows the penicillin prediction resulting from the model without µp correction. The continuous line Optimisation Algorithm indicates the µp updated penicillin prediction with the reduced gain and * indicates the on-line HPLC Fig. 5 - On-line model correction and its place within measurements. the optimisation scheme In the figure, the discrepancy between the hybrid model 6. CONCLUDING COMMENTS prediction and the real data is removed by feedback correction following the observer predictor / corrector In this paper a mechanistic model of penicillin concept. The error between the actual (P) and predicted fermentation was taken as the basis for a hybrid model. Limitations of the mechanistic model were overcome by Henriksen, C. M., Nielsen, J. and Villadsen, J. 1997. making use of an artificial neural network. The resulting Influence of the dissolved oxygen concentration on hybrid model although improved, still lacked the the penicillin biosynthetic pathway in steady-state precision necessary for process optimisation. This cultures of Penicillium chrysogenum. Biotechnology process / model mismatch resulted from natural Progress. 13(6): 776-782. variability. To overcome this, a 'predictor / corrector' Johansen, T. A. and Foss, B. A. 1997. Operating regime method was used to modify a key model parameter based process modeling and identification. Computers which allowed the production of penicillin to be tracked and Chemical Engineering. 21(2): 159-176. with higher accuracy. The on-line model modification Linko, P., RaumanAalto, P., Moller, S., Aarts, R. J. and now provides a model of sufficient accuracy for Kortela, U. 1992. ARMAX modelling and state optimisation purposes. If further experiments to verify estimation of an enzyme fermentation process. the optimisation strategy reveal similar µp trends then a Chemical Engineering Journal and the Biochemical prespecified typical profile would be more appropriate Engineering Journal. 50(3): B45-B49. for model use therefore requiring more limited off-line Ignova M., Paul G.C., Kent C.A., Thomas C.R., correction. Further discussion of the optimisation Montague G.A., Glassey J. and Ward A.C. (2001). approach that this model is embedded within can be ‘On-line optimisation of fed-batch penicillin found in Ignova et al (2001). A final comment to make fermentation’, to be submitted to IASTED MIC2001 concerns model updating; here, an on-line HPLC was Montague, G. A., Morris, A. J. and Tham, M. T. 1992. available to provide frequent measurements of penicillin Enhancing bioprocess operability with generic concentration. Such instruments will not always be software sensors. Journal of Biotechnology. 25: 183- available. In these cases the use of other on-line 201. measurements could be considered for on-line Montague, G. A. and Ward, A. C. 1994. A sub-optimal correction given that observability type assessments are solution to the optimisation of bioreactors using undertaken. chemotaxis algorithm. Process Biochemistry. 29: 489- 496. ACKNOWLEDGEMENTS Nicolai, B. M., Vanimpe, J. F., Vanrolleghem, P. A. and Vandewalle, J. 1991. A modified unstructured The authors of this paper would like to acknowledge the mathematical-model for the penicillin-g fed- batch financial support of the UK Biotechnology and fermentation. Biotechnology Letters. 13(7): 489-494. Biological Sciences Research Council. Paul G.C. (1999). Personal communication Paul, G. C., Syddall, M. T., Kent, C. A. and Thomas, C. REFERENCES R. 1998. A structured model for penicillin production on mixed substrates. Biochemical Engineering Bajpai, R. K. and Reus, M. 1981. Evaluation of feeding Journal. 2(1): 11-21. strategies in carbon-regulated secondary metabolite Psichogios, D. C. and Ungar, L. H. 1992. A hybrid production through mathematical modelling. neural network-1st principles approach to process Biotechnology and Bioengineering. 23: 717-738. modeling. AIChE Journal. 38(10): 1499-1511. Constantinides, A. 1979. Application of rigorous Rodrigues, J. A. and Filho, R. M. 1996. Optimal feed optimisation methods to the control and operation of rates strategies with operating constraints for the fermentation processes. Annals of New York penicillin production process. Chemical Engineering Academy of Science. 326: 193-221. Science. 51(11): 2859-2864. de Azevedo, S. F., Dahm, B. and Oliveira, F. R. 1997. Schubert, J., Simutis, R., Dors, M., Havlik, I. and Hybrid modelling of biochemical processes: A Lubbert, A. 1994. Hybrid modeling of yeast comparison with the conventional approach. production processes - combination of a- priori Computers and Chemical Engineering. 21(SS): S751- knowledge on different levels of sophistication. S756. Chemical Engineering and Technology. 17(1): 10-20. Dors, M., Simutis, R. and Lubbert, A. 1995. Advanced Su, H. T., Bhat, N., Minderman, A. and McAvoy, T. J. supervision of mammalian cell cultures using hybrid 1992. Integrating neural networks with first principles process models. Pre-prints of IFAC International models for dynamic modeling. IFAC Symposium on Conference on Computer Applications in Dynamics and Control of Chemical Reactors, Biotechnology, Garmish-Partenkirchen, Germany, 72- Distillation Columns and Batch Processes, Maryland, 77. USA, 327-332. Fu, P. C. and Barford, J. P. 1996. A hybrid neural- Thompson, M. L. and Kramer, M. A. 1994. Modeling network - first principles approach for modeling of chemical processes using prior knowledge and neural cell-metabolism. Computers and Chemical networks. AIChE Journal. 40(8): 1328-1340. Engineering. 20(6-7): 951-958. Warnes, M. R., Glassey, J., Montague, G. A. and Kara, Heijnen, J. J., Roels, J. A. and Stouthamer, A. H. 1979. B. 1996. On data-based modeling techniques for Application of balancing methods in modeling the fermentation processes. Process Biochemistry. 31(2): penicillin fermentation. Biotechnology and 147-155. Bioengineering. 21: 2175-2201.