## Constrained Extrema

In practice, we wish to optimize a function considering some existing constraints. In economics and engineering, the constrain may be due to limited funds, materials, or energy. If we wish to find the distance from a point $$P=(x_0,y_0)$$ to a line $$ax+by=c$$, we should find the minimum value of $$d(x,y)=\sqrt{(x-x_0)^2+(y-y_0)^2}$$ while $$(x,y)$$ satisfies $$ax+by=c$$. Suppose $$T(x,y,z)$$ represents the temperature in space, and we want to find the maximum temperature on a surface given by $$g(x,y,z)=0$$. When the constraint (also called the side condition) equation is given, we can solve it for one of the variables, say $$z=\phi(x,y)$$, and replace it in the function $$T$$. Then the problem would reduce to finding the extremum value of the function $$T(x,y,\phi(x,y))$$, which now depends only on two independent variables. We have already applied this method in this example, where we optimized the value of the function on the boundary of a region. To practice how to use this method, let’s consider the following example.

Example 1
We want to make a rectangular box without a top, and of given volume $$V$$. If the least amount of material is to be used, determine the design specifications.

Solution
Let
$$x=$$ length of the box
$$y=$$ width of the box
$$z=$$ height of the box
where $$x,y$$ and $$z$$ are in the interval $$(0,\infty)$$. The specified volume is $V=xyz$ The amount of material for constructing this box is proportional to its surface area $S=xy+2xz+2yz.$ So we want to minimize $$S(x,y,z)$$ subject to $$xyz=V$$.

From $$xyz=V$$ we obtain $$z=\frac{V}{xy}$$ and then we plug it into the formula of $$S$$: $S=xy+2\frac{V}{y}+2\frac{V}{x}.$ Now $$S$$ is expressed as a function of only two variables. To determine the relative extremum, we differentiate and set the partial derivatives equal to zero:
$\frac{\partial S}{\partial x}=y-\frac{2V}{x^2}=0, \quad \frac{\partial S}{\partial y}=x-\frac{2V}{y^2}=0$
or equivalently:
$\left\{\begin{array}{l} x^2y=2V\\ \\ xy^2=2V \end{array}\right.$ If we divide the first question by the second one, we obtain $$x/y=1$$. Therefore: $x^2y=x^2 x=x^3=2V \Rightarrow x=y=(2V)^{1/3}$ From these values of $$x$$ and $$y$$, we get $z=\frac{V}{xy}=\frac{(2V)^{1/3}}{2}$ Using the second derivative test, we can show that these values of $$x,y$$ and $$z$$ give a relative minimum of $$S$$. Because $$S\to \infty$$ as either $$x\to 0+$$, $$y\to 0+$$, $$x\to \infty$$, or $$y\to \infty$$, we can conclude that the relative minimum is also the absolute minimum.

## Lagrange Multipliers

When the constraint is given implicitly by $$g(x,y,z)=c$$, it is not always possible or easy to solve the constraint equation for one of the variables (express $$x, y$$ or $$z$$ as a function of the remaining variables). The problem can be more complicated when there is more than one constraint. In such cases, an alternative procedure is a method called Lagrange multipliers.

To explain this method, let’s start with an example. Suppose we want to find the shortest and longest distance between the point $$(1,-1)$$ and the curve $$C$$

$g(x,y)=\frac{(x-y-2)^2}{18}+\frac{(x+y)^2}{32}=1.$

The distance between a point $$(x,y)$$ and $$(1,-1)$$ is given by $f(x,y)=\sqrt{(x-1)^2+(y+1)^2}.$ So we want to maximize and minimize $$f(x,y)$$ subject to $$g(x,y)=1$$. First let’s sketch the curve $$C$$ and some level curves of $$f$$ (Fig. 1). Figure. 1: The orange lines show contour lines of $f$ with different values. The function $f$ takes on extreme values at four points $A,B,P$, and $Q$.

To extremize $$f(x,y)$$ subject to $$g(x,y)=1$$, we have to find the largest and smallest value of $$k$$ such that the level curve $$f(x,y)=k$$ intersects $$g(x,y)=1$$. Among the level curves that intersect $$g(x,y)=1$$, the minimum value of $$f(x,y)$$ occurs at the points $$P$$ and $$Q$$ where $$f(x,y)$$ has a value of 3 (see Fig. 1). At these points, the constraint curve $$g(x,y)=1$$ and the level curve $$f(x,y)=3$$ just touch each other; in other words, $$f=3$$ and $$g=1$$ have a common tangent line at $$P$$ and a common tangent at $$Q$$.

Note that $$g(x,y)=1$$ is the level curve of $$z=g(x,y)$$. Because at each point $$\overrightarrow{\nabla} f$$ is perpendicular to the level curves of $$f$$ and similarly$$\overrightarrow{\nabla} g$$ is perpendicular to the level curves of $$g$$, a common tangent at $$P$$ means that $$\overrightarrow{\nabla} f(P)$$ and $$\overrightarrow{\nabla} g(P)$$ are parallel. That is, there is a number $$\lambda_1$$ such that
$\overrightarrow{\nabla} f(P)=\lambda_1 \overrightarrow{\nabla} g(P).$
Similarly there is a number $$\lambda_1$$ such that
$\overrightarrow{\nabla} f(Q)=\lambda_1\overrightarrow{\nabla} g(Q).$

We also observe that the maximum value of $$f(x,y)$$ subject to $$g(x,y)=1$$ occurs where the constraint curve and a level curve (here $$f=4$$) touch each other (In Fig. 1, they are denoted by$$A$$ and $$B$$). Thus $\overrightarrow{\nabla} f(A)=\mu_1\overrightarrow{\nabla} g(A),$ and $\overrightarrow{\nabla} f(B)=\mu_2 \overrightarrow{\nabla} g(B),$ for some $$\mu_1$$ and $$\mu_2$$. Therefore, to find the maximum or minimum of $$f(x,y)$$ subject to the constraint $$g(x,y)=1$$ , we look for a point $$\textbf{x}_0$$ such that
$\overrightarrow{\nabla} f(\textbf{x}_0)=\lambda \overrightarrow{\nabla} g(\textbf{x}_0)$
for some $$\lambda$$. This is the method of Lagrange multiplier. But why this is true?

Suppose the constraint curve $$C$$ is parameterized by some functions:1 $x=X(t),\quad y=Y(t)$ If, in the equation of $$f$$, $$x$$ and $$y$$ are replaced by $$X(t)$$ and $$Y(t)$$, then the distance between $$(1,-1)$$ and points on $$C$$ becomes a function of $$t$$ $F(t)=f(x(t),y(t)).$ Therefore, the extreme values of the distance occur where $$F'(t)=0$$. From the chain rule we know
$F'(t)=\frac{dF}{dt}=\frac{\partial f}{\partial x}\frac{dX}{dt}+\frac{\partial f}{\partial y}\frac{dY}{dx}$
We can write the equation $$F'(t)=0$$ as

\begin{align} \frac{\partial R}{\partial x}\frac{dX}{dt}+\frac{\partial R}{\partial y}\frac{dY}{dx}&=0\\ \left(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y}\right)\boldsymbol{\cdot}\left(\frac{dX}{dt},\frac{dY}{dt}\right)&=0\end{align}
This means at the extreme points the gradient vector $$(f_x,f_y)$$ is perpendicular to $$(X'(t),Y'(t))$$. Recall that $$(X'(t),Y'(t))$$ is tangent to the curve $$C$$.

On the other hand $$C$$ is a level curve of $$g$$. Therefore, at each point $$\overrightarrow{\nabla} g$$ is perpendicular to $$C$$. Therefore at the extreme points, $$\overrightarrow{\nabla} g$$ and $$\overrightarrow{\nabla} f$$ are parallel. The simplest version of the method of Lagrange multipliers is as follows.

Theorem 1. Suppose $$U\subseteq \mathbb{R}^n$$ is an open set and $$f:U\to\mathbb{R}$$ and $$g:U\to\mathbb{R}$$ are two continuously differentiable functions2. Let $$S=\left\{\mathbf{x}\in U|\ g(\mathbf{x})=c\right\}$$ be the level set for $$g$$ with value $$c$$. If $$f |S$$ denoting “$$f$$ restricted to $$S$$,” has a relative extremum on $$S$$ at $$\mathbf{x}_0\in U$$, and $$\overrightarrow{\nabla} g(\mathbf{x}_0)\neq\mathbf{0}$$, then there exists a real number $$\lambda$$ such that
$\overrightarrow{\nabla} f(\mathbf{x}_0)=\lambda \overrightarrow{\nabla} g(\mathbf{x}_0).$

Let $$\mathbf{r}:I\subseteq\mathbb{R}\to\mathbb{R}^n$$ be a differentiable curve on the level set $$S$$ such that $$\mathbf{r}(t_0)=\mathbf{x}_0$$, $$\mathbf{r}'(t_0)\neq \mathbf{0}$$, and $$\mathbf{r}(t)\in S$$ for every $$t\in I$$. Then $$g(\mathbf{r}(t))=c$$, so the chain rule gives
$\overrightarrow{\nabla} g(\underbrace{\mathbf{r}(t_0)}_{=\mathbf{x}_0})\boldsymbol{\cdot}\mathbf{r}'(t_0)=0.$
Because $$f|S$$ attains a relative maximum or minimum at $$\mathbf{x}_0$$, the function $$h=f\circ\mathbf{r}:I\to\mathbb{R}$$ attains a relative maximum or minimum at $$t_0$$. Hence $$h'(t_0)=0$$ and according to the chain rule we have:
$h'(t_0)=\overrightarrow{\nabla} f(\mathbf{x}_0)\boldsymbol{\cdot}\mathbf{r}'(t_0)=0.$

Thus the vectors $$\overrightarrow{\nabla} f(\mathbf{x}_0)$$ and $$\overrightarrow{\nabla} g(\mathbf{x}_0)\neq \mathbf{0}$$ are both normal to the nonzero vector $$\mathbf{r}'(t_0)$$ and are therefore parallel; that is, $$\overrightarrow{\nabla} f(\mathbf{x}_0)=\lambda \overrightarrow{\nabla} g(\mathbf{x}_0)$$ for some $$\lambda$$. $$\blacksquare$$

• The number $$\lambda$$ in the above theorem is called a Lagrange multiplier.
• $$\lambda$$ might be zero.

Note that to find the extremum of $$f|S$$, we have $$n+1$$ unknowns ($$n$$ components of $$\mathbf{x}_0$$ and $$\lambda$$) and $$n+1$$ equations:
\label{Eq:LagrangeEQ} \left.\begin{align} \frac{\partial f}{\partial x_1}(\mathbf{x}_0)&=\lambda \frac{\partial g}{\partial x_1}(\mathbf{x}_0)\\ &\vdots\\ \frac{\partial f}{\partial x_n}(\mathbf{x}_0)&=\frac{\partial g}{\partial x_n}(\mathbf{x}_0)\\ g(\mathbf{x}_0)&=c \end{align}\right\} \quad (n+1) \text{ equations} \tag{i}

Example 2
Find the extrema of the function $$f(x,y)=xy$$ subject to the constraint $$\frac{1}{4}x^2+\frac{1}{9}y^2=1$$.

Solution
Let $$g(x,y)=\frac{1}{4}x^2+\frac{1}{9}y^2$$, so $$S$$ consists of all points $$(x,y)$$ such that $$g(x,y)=1$$. We have:
$\overrightarrow{\nabla} f(x,y)=(y,x),\quad \overrightarrow{\nabla} g(x,y)=\left(\frac{1}{2}x,\frac{2}{9}y\right)$

Note that $$\overrightarrow{\nabla} g=\mathbf{0}$$ if and only if $$(x,y)=(0,0)$$. Thus if $$f$$ subject to the constraint has an extremum at $$(x_0,y_0)$$, we must have $$\overrightarrow{\nabla} f(x_0,y_0)=\lambda \overrightarrow{\nabla} g(x_0,y_0)$$. Equations i take on the form:
$y_0=\frac{1}{2}\lambda x_0, \quad x_0=\frac{2}{9}\lambda y_0, \quad \frac{1}{4}x_0^2+\frac{1}{9}y_0^2=1.$
There are different ways to solve the above system of equations. One way is to replace $$y_0$$ from the first equation in the second equation
$x_0=\frac{2}{9}\lambda y_0=\frac{2\lambda}{9}(\frac{\lambda}{2} x_0)=\frac{1}{9} \lambda^2 x_0$
$x_0-x_0\frac{\lambda^2}{9}=x_0(1-\frac{\lambda^2}{9})=0\Rightarrow x=0 \ \text{or}\ \lambda=\pm 3.$
If $$x=0$$, then substituting in $$g(x_0,y_0)=1$$, we obtain $\frac{1}{4} 0^2+\frac{1}{9}y_0^2=1\Rightarrow y_0=\pm 3.$ If $$\lambda=\pm 3$$, it follows from $$y_0=\frac{1}{2}\lambda x_0$$ that $$y_0=\pm \frac{3}{2} x_0$$. Substituting into $$g(x_0,y_0)$$, we get:
$\frac{1}{4}x_0^2+\frac{1}{9}\frac{9}{4}x_0^2=1 \Rightarrow \frac{1}{\sqrt{2}}x_0^2=1\Rightarrow x_0=\pm \sqrt{2}.$
If $$x_0=\sqrt{2}$$, then $$y_0=\pm \frac{3}{2}\sqrt{2}$$, and if $$x_0=-\sqrt{2}$$, then $$y_0=\pm\frac{3}{2}\sqrt{2}$$.

Now we list the points we found and the values of $$f$$ at these points:

$$(0,3)$$ $$(0,-3)$$ $$(\sqrt{2},\frac{3\sqrt{2}}{2})$$ $$(\sqrt{2},-\frac{3\sqrt{2}}{2})$$ $$(\sqrt{2},-\frac{3\sqrt{2}}{2})$$ $$(-\sqrt{2},-\frac{3\sqrt{2}}{2})$$
$$f(x_0,y_0)=x_0y_0$$ $$0$$ $$0$$ $$3$$ $$-3$$ $$-3$$ $$3$$

The function $$f$$ attains its maximum value 3 at $$(\sqrt{2},3\sqrt{2}/2)$$ and $$(-\sqrt{2},-3\sqrt{2}/2)$$, and its minimum value -3 at $$(-\sqrt{2},3\sqrt{2}/2)$$ and $$(-\sqrt{2},3\sqrt{2}/2)$$. Note that at these points the level curve $$g=1$$ is tangent to one of the level curves of $$f$$ (see Fig. 2).  (a) (b) Figure 2. (a) The graph of $$f(x,y)$$; the blue curve shows $$f$$ restricted to the ellipse $$g(x,y)=1$$. (b) The level curves of  $$f$$ and $$g$$; the extrema of $$f$$ subject to the constraint $$g(x,y)=1$$ occur at points where the level curves of $$f$$ are tangent to the level curve $$g(x,y)=1$$.
Let’s go back to the example of making an open box and try to solve it using the method of Lagrange multipliers.

Example 3
We want to make a rectangular box without a top, and of given volume $$V$$. If the least amount of material is to be used, determine the design specifications using Lagrange multipliers.

Solution
Again Let $$x, y$$ and $$z$$ be the length, width, and height of the box, respectively. We want to minimize $S=xy+2xz+2yz$ subject to $$xyz=V$$, where $$V$$ is the given volume.

If we let $$g(x,y,z)=xyz$$, then
$\overrightarrow{\nabla} S=(y+2z, x+2z, 2x+2y),\quad \overrightarrow{\nabla} g=(yz, xz, xy)$
Note that $$\overrightarrow{\nabla} g=\mathbf{0}$$ if and only if $$(x,y,z)=(0,0,0)$$ which is not on the level surface $$g(x,y,z)=V$$. So if $$S$$ has an extremum at $$(x_0,y_0,z_0)$$, we have
$\overrightarrow{\nabla} S(x_0,y_0,z_0)=\lambda \overrightarrow{\nabla} g(x_0,y_0,z_0).$
Hence our equations become
\begin{align} \left\{\begin{array}{l c c} y+2z=\lambda yz & & \text{(i)}\\ \\ x+2z=\lambda xz & & \text{(ii)}\\ \\ 2x+2y=\lambda xy& & \text{(iii)}\\ \\ xyz=V& & \text{(iv)} \end{array}\right.\end{align}
Because $$x,y$$ and $$z$$ are nonzero, if we divide the first equation by $$yz$$, the second equation by $$xz$$ and the third one $$xy$$, we get:
\begin{align} \left\{\begin{array}{l c c} \dfrac{1}{z}+\dfrac{2}{y}=\lambda & & \text{(i^\prime)}\\ \\ \dfrac{1}{z}+\dfrac{2}{x}=\lambda & & \text{(ii^\prime)}\\ \\ \dfrac{2}{y}+\dfrac{2}{x}=\lambda& & \text{(iii^\prime)}\\ \\ xyz=V& & \text{(iv^\prime)} \end{array}\right.\end{align}
Then
$\text{Eq. (i^\prime)},\ \ \text{Eq. (ii^\prime)}\quad \Rightarrow \quad x=y$
$\text{Eq. (ii^\prime)},\ \ \text{Eq (iii^\prime)}, \ \text{and}\ x=y\quad \Rightarrow \quad z=\frac{1}{2} x$
$x=y=2z, \ \ \text{Eq (4)} \Rightarrow \frac{1}{2}x^3=V\quad \Rightarrow\quad x=y=2z=(2V)^{1/3}.$
This is the result we previously obtained in Example 1.

The following example shows an application of Lagrange multipliers in economics and when there are many variables.
Example 4
Assume there are $$n$$ commodities with prices per unit $$p_1,\cdots,p_n$$. Assume we have $$L$$ dollars to spend on these commodities. This means we have the constraint $$p_1 x_1+\cdots p_n x_n=L$$ where $$x_1,\cdots,x_n$$ are the amounts of the commodities. Assume the utility is given by the Cobb-Douglas function $$U=f(x_1,\cdots,x_n)=K x_1^{\alpha_1}\cdots x_n^{\alpha_n}$$ where $$K, \alpha_1,\cdots,\alpha_n$$ are positive constants. Find the maximum utility we can achieve.

Solution
Let $$g(x_1,\cdots,x_n)=p_1 x_1+\cdots+p_n x_n$$. So we seek for the maximum value of $$f(x_1,\cdots,x_n)$$ subject to $$g(x_1,\cdots,x_n)=L$$. Because $$x_i\geq 0$$ (for $$i=1,\cdots,n$$) the constraint is a part of the level set $$g(x_1,\cdots,x_n)=L$$. We note that when the amount of a commodity $$x_i$$ approaches zero, the utility approaches zero. Hence, the utility takes on its maximum value when $$x_i$$’s are positive. Because $$\overrightarrow{\nabla} g(x_1,\cdots,x_n)=(p_1,\cdots,p_n)\neq \mathbf{0}$$, if $$f$$ attains its maximum at a point, we have $$\overrightarrow{\nabla} f(x_1,\cdots,x_n)=\lambda \overrightarrow{\nabla} g(x_1,\cdots,x_n)$$; that is,
$\left\{\begin{array}{l} \alpha_1 K x_1^{\alpha_1-1}x_2^{\alpha_2}\cdots,x_n^{\alpha_n}=\lambda p_1\\ \hspace{1cm}\vdots\\ \alpha_n K x_1^{\alpha_1}x_2^{\alpha_2}\cdots,x_n^{\alpha_n-1}=\lambda p_n \end{array}\right.$
If we multiply both sides of the first equation by $$x_1$$, the second equation by $$x_2$$, $$\cdots$$, and the $$n$$-th equation by $$x_n$$, we get:
$\left\{\begin{array}{l} \alpha_1 f(x_1,\cdots,x_n)=\lambda p_1 x_1\\ \hspace{1cm}\vdots\\ \alpha_n f(x_1,\cdots,x_n)=\lambda p_n x_n \end{array}\right.$
Therefore: $\frac{p_1}{\alpha_1}x_1=\cdots=\frac{p_n}{\alpha_n} x_n$ This result makes sense because it says the amount of commodity $$x_i$$ is proportional to its power and inversely proportional to its price, $$p_i$$.

Let $$\frac{\alpha_i}{p_i} x_i=C$$, then
$p_1 x_1+\cdots+p_n x_n=p_1\frac{\alpha_1}{p_1} C+\cdots+p_n\frac{\alpha_1}{p_n} C=L$
or
$C(\alpha_1+\cdots+\alpha_n)=C\sum_{i=1}^n \alpha_i=L\Rightarrow C=\frac{L}{\sum_{i=1}^n \alpha_i}.$
Consequently,
$x_i=\frac{\alpha_i}{p_i}\frac{L}{\sum_{i=1}^n \alpha_n}.$ Finally we conclude the maximum value of $$f$$ is:
$K\left(\frac{\alpha_i L}{p_i \sum_{i=1}^n \alpha_i}\right)^{\alpha_1} \cdots \left(\frac{\alpha_n L}{p_n \sum_{i=1}^n \alpha_i}\right)^{\alpha_n}=K\left(\frac{L}{\sum_{i=1}^n \alpha_i}\right)^{\sum_{i=1}^n\alpha_i} \left(\frac{\alpha_1}{p_1}\right)^{\alpha_1} \cdots \left(\frac{\alpha_n}{p_n}\right)^{\alpha_n}$

The following example shows when maximizing or minimizing a function subject to a constraint, we should also investigate the points where the gradient of the constraint is zero $$\overrightarrow{\nabla} g(\mathbf{x})=\mathbf{0}$$.

Example 5
Find the nearest point on the curve $$y^3-x^2=0$$ to the point $$(0,-1)$$.

Solution
If we plot the level curve
$$g(x,y)=y^3-x^2=0$$ (Fig. 3), it is clear that $$(0,0)$$ is the nearest point on this curve to the point $$(0,-1)$$, and the distance is 1. If we want to use the method of Lagrange multipliers, we need to minimize the square of distance $$f(x,y)=(x-0)^2+(y-(-1))^2$$ subject to the constraint $$g(x,y)=y^3-x^2=0$$.

Here
$\overrightarrow{\nabla} f(x,y)=(2x,2y+2),\quad \overrightarrow{\nabla} g(x,y)=(-2x,3y^2).$
Equations i take on the form
$\left\{\begin{array}{l} 2x_0=-2\lambda x_0\\ 2y_0+2=3\lambda y_0^2\\ y_0^3-x_0^2=0 \end{array}\right.$
However the solution, $$(x_0,y_0)=(0,0)$$, does not satisfy the second equation. The reason is attributed to the fact that at this point the gradient of $$g$$ is $$\mathbf{0}$$, $$\overrightarrow{\nabla} g(0,0)=(0,0)$$, so we cannot use Theorem 1. In addition, the level curve is not smooth at $$(0,0)$$.

If a function $$f$$ subject to a constraint $$g(\mathbf{x})=c$$ attains its extreme value at a point $$\mathbf{x}_0$$, then $$\mathbf{x}_0$$ is one of the four types of the point:

1. $$\mathbf{x}_0$$ is a point where $$\overrightarrow{\nabla} f(\mathbf{x}_0)=\lambda \overrightarrow{\nabla} g(\mathbf{x}_0)$$,
2. $$\mathbf{x}_0$$ is a point where $$\overrightarrow{\nabla} g(\mathbf{x}_0)=\mathbf{0}$$,
3. $$\mathbf{x}_0$$ is a rough point of $$f$$ or $$g$$ where $$\overrightarrow{\nabla} f(\mathbf{x}_0)$$ or $$\overrightarrow{\nabla} g(\mathbf{x}_0)$$ does not exist, or
4. $$\mathbf{x}_0$$ is on the boundary of the level set $$g(\mathbf{x})=c$$.

## Multiple Constraints

The method of Lagrange multipliers extends to the case of multiple constraints: say to $$f(x,y,z)$$ subject to two constraints
$\label{Eq:Lagrange_2Constraint} g_1(x,y,z)=c_1,\quad \text{and} \quad g_2(x,y,z)=c_2.\tag{ii}$
Geometrically this means we seek the extreme values of $$f(x,y,z)$$ on a curve $$C$$ formed by the intersection of two level surfaces $$g_1(x,y,z)=c_1$$ and $$g_2(x,y,z)=c_2$$ (Fig. 4). Figure 4. Because $$\vec{\nabla}g_1$$ is perpendicular to the surface $$g_1=c_1$$ and $$\vec{\nabla}g_2$$ is perpendicular to the surface $$g_2=c_2$$, both $$\vec{\nabla}g_1$$ and $$\vec{\nabla}g_2$$ are perpendicular to C. At a point where $$f$$ has a maximum or minimum, also $$vec{\nabla}f$$ is perpendicular to C.

Suppose $$f$$ subject to these constraints, $$f|C$$, has an extremum at $$P=(x_0,y_0,z_0)$$, and the functions $$f, g_1$$, and $$g_2$$ have continuous first partial derivatives near $$P$$. Analogous to the case of one constraint, we can argue that $$\overrightarrow{\nabla} f$$ is perpendicular to $$C$$ at $$P$$. Additionally because the gradient is perpendicular to the level surface, both $$\overrightarrow{\nabla} g_1$$ and $$\overrightarrow{\nabla} g_2$$ are perpendicular to $$C$$. If $$\overrightarrow{\nabla} g_1(x_0,y_0,z_0)$$ and $$\overrightarrow{\nabla} g_2(x_0,y_0,z_0)$$ are neither zero vectors nor collinear vectors, then $$\overrightarrow{\nabla} f(x_0,y_0,z_0)$$ lie in a plane spanned by the two vectors $$\overrightarrow{\nabla} g_1(x_0,y_0,z_0)$$ and $$\overrightarrow{\nabla} g_2(x_0,y_0,z_0)$$ and hence can be expressed as a linear combination of these two vectors, say:
\begin{align} \label{Eq:Lagrange_2Multipliers} \bbox[#F2F2F2,5px,border:2px solid black]{\overrightarrow{\nabla} f(x_0,y_0,z_0)=\lambda_1 \overrightarrow{\nabla} g_1(x_0,y_0,z_0)+\lambda_2 \overrightarrow{\nabla} g_2(x_0,y_0,z_0)}\tag{iii}\end{align}
In this method, Equations ii and iii must be solved simultaneously for five unknowns $$x_0,y_0,z_0,\lambda_1$$ and $$\lambda_2$$:
$\left\{\begin{array}{l} \dfrac{\partial f}{\partial x}(x_0,y_0,z_0)=\lambda_1 \dfrac{\partial g_1}{\partial x}(x_0,y_0,z_0)+\lambda_2 \dfrac{\partial g_2}{\partial x}(x_0,y_0,z_0)\\ \\ \dfrac{\partial f}{\partial y}(x_0,y_0,z_0)=\lambda_1 \dfrac{\partial g_1}{\partial y}(x_0,y_0,z_0)+\lambda_2 \dfrac{\partial g_2}{\partial y}(x_0,y_0,z_0)\\ \\ \dfrac{\partial f}{\partial z}(x_0,y_0,z_0)=\lambda_1 \dfrac{\partial g_1}{\partial z}(x_0,y_0,z_0)+\lambda_2 \dfrac{\partial g_2}{\partial z}(x_0,y_0,z_0)\\ \\ g_1(x,y,z)=c_1\\ \\ g_2(x,y,z)=c_2 \end{array}\right.$

Example 6
Find the extreme points of the function $$f(x,y,z)=x+4y-2z$$ on the curve of the intersection of the cylinder $$x^2+y^2=4$$ and the plane $$2y-z=3$$ (Fig. 5).

Solution
Here we are given two constraints:
$g_1(x,y,z)=x^2+y^2=4,\quad g_2(x,y,z)=2y-z=3.$ Note that
$\overrightarrow{\nabla} f=(1,4,-2),\quad \overrightarrow{\nabla} g_1=(2x,2y,0),\quad\text{and}\quad \overrightarrow{\nabla} g_2=(0,2,-1).$
The vector $$\overrightarrow{\nabla} g_1=(0,0,0)$$ only when $$x=0$$ and $$y=0$$, which clearly does not satisfy the constraint $$g_1=4$$. Thus, the two vectors $$\overrightarrow{\nabla} g_1$$ and $$\overrightarrow{\nabla} g_2$$ are clearly not parallel. Therefore, any constrained critical point $$(x_0,y_0,z_0)$$ must satisfy
$\overrightarrow{\nabla} f(x_0,y_0,z_0)=\lambda_1 \overrightarrow{\nabla} g_1(x_0,y_0,z_0)+\lambda_2 \overrightarrow{\nabla} g_2(x_0,y_0,z_0).$
It follows that we must solve the following system of equations for five unknowns
$$x,y,z,\lambda_1$$ and $$\lambda_2$$:
$\left\{\begin{array}{l} 1=2x\lambda_1 +0\\ 4=2y\lambda_1+2\lambda_2\\ -2=0-\lambda_2\\ x^2+y^2=4\\ 2y-z=3 \end{array}\right.$
From the third equation, we know $$\lambda_2=2$$. So the second equation becomes $$2y\lambda_1=0$$. Because from the first equation, $$\lambda_1\neq 0$$, it follows from $$2y\lambda_1=0$$ that $$y=0$$. If we substitute $$y=0$$ in the fourth and fifth equations, we get $$x=\pm 2$$ and $$z=1$$. Therefore, $$f$$ may have extrema at $$(\pm 2,0,1)$$.

The condition $$x^2+y^2=4$$ implies $$-2\leq x,y\leq 2$$. Then it follows from $$2y-z=3$$ that $$-7\leq z\leq 1$$. This means the constraint set, $$S$$, is bounded. Because the constraint set $$S$$ is closed and bounded, and $$f$$ is a continuous function, $$f|S$$ assumes its maximum and minimum ( Theorem 1 in the previous section). Here we have only two potentials, therefore one of them $$(2,0,1)$$ is the maximum point and the other one $$(-2,0,1)$$ is the minimum point.

By similar reasoning, we obtain equations for minimizing or maximizing
$$f(x_1,\cdots,x_n)$$ subject to several constraints
\begin{align} \label{Eq:m-constraints} \left\{\begin{array}{l} g_1(x_1,\cdots,x_n)=c_1\\ g_2(x_1,\cdots,x_n)=c_2\\ \vdots\\ g_m(x_1,\cdots,x_n)=c_m \end{array}\right.\tag{iv}\end{align}
where $$m<n$$. Assume $$f(x_1,\cdots,x_n)$$ has a relative extremum at $$\mathbf{x}_0$$ when the variables are restricted by the constraint equations iv. If $$f, g_1,\cdots,$$ and $$g_m$$ have continuous first partial derivatives at all points near $$\mathbf{x}_0$$ and if each $$\overrightarrow{\nabla} g_i(\mathbf{x}_0)$$ is not a linear combination of the other $$\overrightarrow{\nabla} g_j(\mathbf{x}_0)$$ ($$j\neq i$$), then there exists $$m$$ real numbers $$\lambda_1,\cdots, \lambda_m$$ such that
$\bbox[#F2F2F2,5px,border:2px solid black]{\overrightarrow{\nabla} f(\mathbf{x}_0)=\lambda_1\overrightarrow{\nabla} g_1(\mathbf{x}_0)+\cdots+\lambda_m \overrightarrow{\nabla} g_m(\mathbf{x}_0).}$

1 For instance, in this specific example,
$\begin{cases} x=X(t)=\dfrac{3}{\sqrt{2}}\cos t+2\sqrt{2} \sin t\\ \\ y=Y(t)=-\dfrac{3}{\sqrt{2}}\cos t+2\sqrt{2}\sin t \end{cases}\qquad (0\leq t\leq2\pi)$

2 In other words, $$f$$ and $$g$$ have continuous partial derivatives.