Vector Operations in Linear Algebra

Now that you've learnt some basics of vectors, let's proceed with the different operations you can perform on vectors. Most of these you may remember from previous studies, and the objective of this section is to revise them and get some intuition for where they may be used during analysing data.

Vector Addition

When you add two vectors, you can geometrically add them as shown in the figure below. You may remember the following image taught to you as the "parallelogram rule".

Magnitude And Direction of an Addition

If two vectors $\overrightarrow{a}$ and $\overrightarrow{b}$ are added, and their magnitudes are given by 'a' and 'b' respectively, then the magnitude of their resultant sum is given by:

$||\overrightarrow{a}+\overrightarrow{b}||=\sqrt{a^2+b^2}$

The angle between the two vectors is:

$\theta =\arctan \left(\frac{b}{a}\right)$

where "arctan" is a convetion to write the "tan-inverse" function.

Multiplication with a Scalar

A scalar multiplication, as the name suggests, takes a vector and scales the vector by a numerical factor, keeping the direction exactly the same.

In scaling, magnitude changes by a factor of "c", and direction remains the same.

Vector dot product

The output of a vector dot product is a scalar quantity. For two vectors

$X=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right],Y=\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right]$

The dot product is defined as

$x_1\ast y_1+x_2\ast y_2$

As an example, 

$A=\left[\begin{array}{c}1\\ 2\end{array}\right],B=\left[\begin{array}{c}6\\ 15\end{array}\right]$

The dot product (A . B) is given by

$A.B=1\ast 6+2\ast 15=36$

For the remainder of this course, for denoting dot product, we will use the notation

$dot(A,B)$

This is because the syntax of NumPy writes dot product in this form.

Let's now look at a dot product in terms of magnitude and angle between vectors. If we have two vectors $\overrightarrow{a}$ and $\overrightarrow{b}$ and the angle between them is θ, then the dot product is given by the expression

$dot(A,B)=||\overrightarrow{a}||\ast ||\overrightarrow{b}||\ast cos\left(\theta \right)$

As you can see, the dot product is the magnitude of one vector, multiplied by the projection of the other (magnitude * cos(θ)). This equation gives us an important insight:

$dot(A,B)=dot(B,A)$

In algebra, this sort of a property is called "commutativity", which means the numbers in the operation can "commute" around. Also, the dot product is referred to be "symmetric", as the order does not matter.

Vector cross product

The output of a vector cross product is another vector. For the following two vectors,

$X=\left[\begin{array}{c}{x}_1\\ x_2\end{array}\right],Y=\left[\begin{array}{c}{y}_1\\ y_2\end{array}\right]$

the magnitude of the cross product is:

$x_1\ast y_2-x_2\ast y_1$

The direction of the cross product is perpendicular to the plane defined by the two vectors.

As an example, 

$A=\left[\begin{array}{c}1\\ 2\end{array}\right],B=\left[\begin{array}{c}6\\ 15\end{array}\right]$

The magnitude of the cross product is given by

$||\overrightarrow{A\times B}||=1\ast 15-2\ast 6=3$

For the remainder of this course, for denoting cross product, we will use the notation

$cross(A,B)$

This is because the syntax of NumPy writes cross product in this form.

Also, a cross product is not commutative. The order of a cross product does matter. Specifically, changing the order reverses the sign

$cross(A,B)=-cross(B,A)$

In terms of the magnitude and angle between the vectors, the cross product has the magnitude:

$||cross(A,B)||=||\overrightarrow{a}||\ast ||\overrightarrow{b}||\ast sin\left(\theta \right)$

Let's look at some examples. Python libraries make these calculations extremely simple to execute

In the upcoming section, we will study vector spaces.