Module 6 & 7: Nonlinear Regression & Process Optimization

Nonlinear Regression: a form of regression analysis in which observational data are modeled by a
function which is a nonlinear combination of the model parameters and depends on one or more
independent variables.

Department of Chemical Engineering

Non-linear Regression

xi - Independent variables
ai - Parameters in the non-linear function
yi - Given y values (or dependent variables)
ycalc - Calculated y values

Department of Chemical Engineering

Nonlinear Regression: Motivating Example

Department of Chemical Engineering

Nonlinear Regression: Motivating Example

Estimate k1 and k2.

Department of Chemical Engineering

Nonlinear Regression in MATLAB

X = lsqcurvefit(fun,x0,xdata,ydata)

Input data: xdata,

Observed output: ydata
It starts at x0 and finds coefficients X to best fit the nonlinear function denoted by ‘fun’
(f(xdata) = ydata).

Department of Chemical Engineering

Nonlinear Regression in MATLAB
Example: Consider that the input xdata and observed response data ydata are as given
below.
xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3]
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5]
Find parameters s(1) and s(2) to fit a model of the form y = s(1)exp(s(2)x). Fit the model
using the starting point x0 = [100,-1].

xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3];
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5];
plot(xdata, ydata, ‘ko’)

Department of Chemical Engineering

Nonlinear Regression in MATLAB
Example: Consider that the input xdata and observed response data ydata are as given
below.
xdata = [0.9 1.5 13.8 19.8 24.1 28.2 35.2 60.3 74.6 81.3]
ydata = [455.2 428.6 124.1 67.3 43.2 28.1 13.1 -0.4 -1.3 -1.5]
Find parameters s(1) and s(2) to fit a model of the form y = s(1)exp(s(2)x). Fit the model
using the starting point x0 = [100,-1].

fun = @(s,xdata)s(1)*exp(s(2)*xdata);

s0 = [100,-1];
S = lsqcurvefit(fun,s0,xdata,ydata)

S = 498.8309 -0.1013

Department of Chemical Engineering

Introduction to Process Optimization

Department of Chemical Engineering

Components of Optimization Problem

Department of Chemical Engineering

Some Popular Optimization Problems

Department of Chemical Engineering

Unconstrained Function Minimization in MATLAB

 Optimization solvers
 [X, fval] = fminbnd (myfun, 𝐱𝐋𝐁 , 𝐱 𝐔𝐁 )  Bounded Min. problem
 [X, fval] = fminsearch (myfun, 𝒙𝟎 )  Min. problem
 [X, fval] = fminunc (myfun, 𝒙𝟎 ) Min. problem
myfun – function to be minimized in terms of x
𝐱 𝐋𝐁 - Lower bound on x
𝐱 𝐔𝐁 - Upper bound on x
𝒙𝟎 - Initial guess for x
X – Optimal values of x and fval – Corresponding objective function.

Department of Chemical Engineering

Unconstrained Function Minimization in MATLAB

Example: 𝐦𝐢𝐧 𝒇 𝒙
𝒙
2 2
=100(𝑥2 −𝑥1 ) + (1−𝑥1 )2

fun = @(x)100*(x(2) - x(1)^2)^2 + (1 - x(1))^2;

x0 = [-1.2,1];

[x,fval] = fminsearch(fun,x0)

x = 1.0000 1.0000 and fval = 8.1777e-10

[x,fval] = fminunc(fun,x0)

Department of Chemical Engineering

Unconstrained Function Minimization in MATLAB

[X, fval] = optimizer (@myfun, x0 )

Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5
𝑥

function f = myfun(x)

f = 12*x^5 - 45*x^4 + 40*x^3 + 5;

end
 [X, fval] = fminsearch (@myfun, 1.5)
 [X, fval] = fminunc (@myfun, 1.5)

Department of Chemical Engineering

Unconstrained Function Minimization in MATLAB

[X, fval] = optimizer (@myfun, xLB , xUB )

Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5

𝑥
such that x ∈ (0.5, 2.5)

Optimizer Decision
Variables

myfun
Objective
function
Department of Chemical Engineering
Unconstrained Function Minimization in MATLAB

[X, fval] = optimizer (@myfun, xLB , xUB )

Example min 𝑓 𝑥 = 12𝑥 5 − 45𝑥 4 + 40𝑥 3 + 5
𝑥
such that x ∈ (0.5, 2.5)

function f = myfun(x)

f = 12*x^5 - 45*x^4 + 40*x^3 + 5;

end

 [X, fval] = fminbnd (@ myfun, 0.5, 2.5)

Department of Chemical Engineering

Constrained Function Minimization in MATLAB

Example

Department of Chemical Engineering

Constrained Function Minimization in MATLAB

[X,fval] = fmincon(@FUN,X0,A,B,Aeq,Beq,LB,UB,@NONLCON)

function f = FUN(x)

f = 9.82*x(1)*x(2)+2*x(1);

end

Department of Chemical Engineering

Constrained Function Minimization in MATLAB

function [c, ceq] = NONLCON(x)

% Nonlinear inequality constraints
c(1) = 2500/(pi*x(1)*x(2))-500;

c(2) = 2500/(pi*x(1)*x(2))-
(pi^2*(x(1)^2+x(2)^2))/0.5882;

% Nonlinear equality constraints

ceq = [];
end

Department of Chemical Engineering

Constrained Function Minimization in MATLAB

A = [-1 0; 1 0; 0 -1; 0 1];

B = [-2; 14; -0.2; 0.8];
Aeq = []; LB = [];
Beq = []; UB = [];

[X,fval] = fmincon(@FUN,[0.2,0.2],A,B,Aeq,Beq,LB,UB,@NONLCON)

Department of Chemical Engineering

Constrained Function Minimization in MATLAB
X = fmincon(@FUN,X0,A,B,Aeq,Beq,LB,UB,@NONLCON)

Try this

Min x
2
1  
2
 x 2  11  x1 x 2  7
2

2

x1, x 2

s.t.
x12  x 22  25
 5  x1 , x 2  5

Department of Chemical Engineering

Function Minimization in MATLAB

Department of Chemical Engineering

Function Minimization in MATLAB

Optimization Problem Formulation

Use any initial guess for both k1 and k 2 . Plot concentration vs time with the optimal values of k1 and k 2 .

function f = optim_param(k,t,data)
for i = 1:10
a(i) = exp(-(k(1) + k(2)) * t(i));
b(i) = (k(1) / (k(1) + k(2))) * (1 - exp(-(k(1) + k(2)) * t(i)));
c(i) = (k(2) / (k(1) + k(2))) * (1 - exp(-(k(1) + k(2)) * t(i)));
fcn(i) = (data(i,1) - a(i))^2 + (data(i,2) - b(i))^2 + (data(i,3) - c(i))^2;
end
f = sum(fcn);
end

Department of Chemical Engineering

Function Minimization in MATLAB

clc
function f = clear all
optim_param(k,t,data) data = load('rxn_data.txt');
for i = 1:10 n = 10;
a(i) = exp(-(k(1) + k(2)) * t = 0.1:0.1:1;
t(i)); fun = @(k)optim_param(k,t,data);
b(i) = (k(1) / (k(1) + k(2))) * x0 = [0.2,0.1];
(1 - exp(-(k(1) + k(2)) * t(i))); [X,fval] = fminunc(fun,x0);
c(i) = (k(2) / (k(1) + k(2))) *
(1 - exp(-(k(1) + k(2)) * t(i))); % For plotting
fcn(i) = (data(i,1) - a(i))^2 +
(data(i,2) - b(i))^2 + (data(i,3) - for i = 1:10
c(i))^2; a(i) = exp(-(X(1) + X(2)) * t(i));
end b(i) = (X(1) / (X(1) + X(2))) * (1 - exp(-(X(1) + X(2)) * t(i)));
f = sum(fcn); c(i) = (X(2) / (X(1) + X(2))) * (1 - exp(-(X(1) + X(2)) * t(i)));
end end
plot(t,a,'-*',t,b,'-o',t,c,'-d')

Department of Chemical Engineering

Function Minimization in MATLAB

Department of Chemical Engineering

Data Visualization in MATLAB boxplot(MPG,Origin)
xlabel('Country of Origin’)
 Box plot ylabel('Miles per Gallon (MPG)')

 Create a box plot of the miles per

gallon (MPG) data in MATLAB -
carsmall.
load carsmall
boxplot(MPG)
xlabel('All Vehicles')
ylabel('Miles per Gallon (MPG)')

 For grouped data - Create a box plot

of MPG from the sample data, grouped
by the vehicles' country of origin.
Data Visualization in MATLAB

 Scatter Plot

 Create 1000 normally

distributed random points
each for x and y and plot
them.

x = randn(1000,1);
y = randn(1000,1);
scatter(x,y,'filled')

plot(x,y,'bo')
Data Visualization in MATLAB

 Line Plot

 Create x as a vector of linearly

spaced values between 0 and
2π.
 Use an increment of π/100
between the values.
 y = sin (x). Create a line plot of
the x vs y data.

x = 0:pi/100:2*pi;
y = sin(x);
plot(x,y)
Data Visualization in MATLAB

 Heatmap
 Create a matrix of data.
 Then create a heatmap of the
matrix values.

cdata = [45 60 32; 43 54 76; 32 94 68; 23 95 58];

heatmap(cdata)
Data Visualization in MATLAB

 3-D Data Line Plot

 Define t as a vector of
values between 0 and 10π.
 Define st and ct as vectors of
sine and cosine values of t.
 Then plot st, ct, and t.

t = 0:pi/50:10*pi;
st = sin(t);
ct = cos(t);
plot3(st,ct,t)
Data Visualization in MATLAB plot3(x,y,z,'bo')

 3-D Data Scatter Plot

 Use sphere of radius 16 to

define vectors x, y, and z.
 Create a 3-D scatter plot of x, y
and z.

[X,Y,Z] = sphere(16);
x = [0.5*X(:); 0.75*X(:); X(:)];
y = [0.5*Y(:); 0.75*Y(:); Y(:)];
z = [0.5*Z(:); 0.75*Z(:); Z(:)];
scatter3(x,y,z)
Data Visualization in MATLAB

 High Dimensional Data plot (> 3-D)  Read a 4-D data from MATLAB.

 Show the Pair-

wise plots of
the data

load fisheriris
plotmatrix(meas)
Data Visualization in MATLAB

 High Dimensional Data plot (> 3-D)

 Dimension Reduction Techniques – Reduce High Dimensional Data

to 2 or 3 dimensions such that the data can be visualized.

 Commonly used –
i. Principal Component Analysis (PCA)
ii. t-Stochastic Distributed Neighbors (t-SNE)
Data Visualization in MATLAB

 Principal Component Analysis (PCA) – High Dimensional Data Plot

Transforms a large set of variables into load fisheriris
a smaller one that still contains most of [coeff,score,~,~,estimated] = pca(meas);
the information in the large set. scatter(score(:,1),score(:,2),'filled')

 coeff  Principal component coefficients

 score  Principal component scores  representations of the data
in the principal component space
 estimated  the percentage of the total variance explained by each
principal component.
Data Visualization in MATLAB

 PCA – 4-D Data Plot

Transforms a large set of variables

into a smaller one that still contains
most of the information in the large
set.

load fisheriris
[coeff,score,~,~,estimated] = pca(meas);
scatter(score(:,1),score(:,2),'filled')
Data Visualization in MATLAB
 t-Stochastic Distributed Neighbors (t-SNE) - High Dimensional Data Plot

Transforms a large set of variables into

a smaller one that still contains most of
the information in the large set.

load fisheriris
rng(8)
Y = tsne(meas);
scatter(Y(:,1),Y(:,2),'filled')

Y  two-dimensional embedding of the

high-dimensional data in ‘meas’.
rng  for ensuring reproducibility.
Data Visualization in MATLAB

 t-Stochastic Distributed Neighbors (t-SNE) - High Dimensional Data Plot

Y = tsne(meas,'NumDimensions',3);
scatter3(Y(:,1),Y(:,2),Y(:,3),'filled')

Y  three-dimensional embedding
of the high-dimensional data ‘meas’.