Ostralyan

Vectors and Vector Operations

A comprehensive guide to understanding vectors, their properties, and essential operations in linear algebra

What Are Vectors?

A vector is a mathematical object that has both magnitude (size) and direction. Unlike scalars (single numbers like temperature or mass), vectors carry directional information, making them perfect for representing quantities like velocity, force, displacement, and acceleration.

Two Ways to Think About Vectors

1. Geometric Interpretation: Vectors as Arrows

Geometrically, a vector is an arrow in space with:

  • Length: representing the magnitude
  • Direction: showing which way it points
  • No fixed position: vectors can be moved around as long as length and direction stay the same
voriginvdifferent startSame vector, different positionsBoth arrows: same length, same direction= same vector
For example, if you walk 3 miles northeast, that displacement is a vector - it has magnitude (3 miles) and direction (northeast). Whether you start from your house or the library, the displacement vector is the same.

[...]2. Algebraic Interpretation: Vectors as Lists of Numbers

Algebraically, we represent vectors as ordered lists of numbers called components. A 2D vector might be written as v = [3, 4] or v = (3, 4), where:

  • The first number (3) is the x-component
  • The second number (4) is the y-component

In 3D space, we'd have three components: v = [x, y, z].

2D Vector

v
2 × 1
3
4

3D Vector

w
3 × 1
2
-1
3

Converting Between Views

The beauty of linear algebra is connecting these interpretations. The vector [3, 4] represents an arrow that goes 3 units right and 4 units up from the origin. Its magnitude is √(3² + 4²) = 5 units, and it points in a specific direction.

Essential Vector Operations

1. Vector Addition

Geometric View

Place the tail of the second vector at the head of the first. The sum is the vector from the original starting point to the final ending point.

u[2, 3]v[1, -1]u + v[3, 2]Tip-to-Tail MethodSteps:1. Draw u2. Draw v from u's tip3. Sum: start → end

Algebraic View

Add corresponding components:

u⃗=[23]\vec{u} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}andv⃗=[1−1]\vec{v} = \begin{bmatrix} 1 \\ -1 \end{bmatrix}
u⃗+v⃗=[23]+[1−1]=[2+13+(−1)]=[32]\vec{u} + \vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix} + \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 2+1 \\ 3+(-1) \end{bmatrix} = \begin{bmatrix} 3 \\ 2 \end{bmatrix}

Properties

  • Commutative: u⃗+v⃗=v⃗+u⃗\vec{u} + \vec{v} = \vec{v} + \vec{u}
  • Associative: (u⃗+v⃗)+w⃗=u⃗+(v⃗+w⃗)(\vec{u} + \vec{v}) + \vec{w} = \vec{u} + (\vec{v} + \vec{w})
  • Zero vector: v⃗+0⃗=v⃗\vec{v} + \vec{0} = \vec{v}

2. Scalar Multiplication

Definition: Multiplying a vector by a scalar (regular number) scales the vector's magnitude and potentially flips its direction.

Algebraic View: Multiply each component by the scalar.

v⃗=[23]\vec{v} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}and scalarc=−2c = -2
cv⃗=−2[23]=[−4−6]c\vec{v} = -2 \begin{bmatrix} 2 \\ 3 \end{bmatrix} = \begin{bmatrix} -4 \\ -6 \end{bmatrix}

Geometric Effects

  • Positive scalar: stretches/shrinks the vector in the same direction
  • Negative scalar: stretches/shrinks and reverses direction
  • Scalar of 1: no change
  • Scalar of 0: becomes zero vector
v2v-v0.5v-2vScalar Multiplication EffectsKey:2v: doubles length0.5v: halves length-v: reverses direction-2v: reverses & doubles

−3. Vector Subtraction

Geometric View

Vector subtraction u⃗−v⃗\vec{u} - \vec{v} is equivalent to adding u⃗\vec{u} and the negative of v⃗\vec{v}. Geometrically, it gives the vector from the tip of v⃗\vec{v} to the tip of u⃗\vec{u} when both start from the same point.

uvu - vGeometric Subtractionu - v: vector from v to u

Algebraic View

Subtract corresponding components:

u⃗=[34]\vec{u} = \begin{bmatrix} 3 \\ 4 \end{bmatrix}andv⃗=[12]\vec{v} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}
u⃗−v⃗=[34]−[12]=[3−14−2]=[22]\vec{u} - \vec{v} = \begin{bmatrix} 3 \\ 4 \end{bmatrix} - \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 3-1 \\ 4-2 \end{bmatrix} = \begin{bmatrix} 2 \\ 2 \end{bmatrix}
Alternative: u⃗−v⃗=u⃗+(−v⃗)\vec{u} - \vec{v} = \vec{u} + (-\vec{v})
Subtraction is addition with the negative vector.

|v|3. Vector Magnitude (Norm/Length)

Definition: The magnitude of a vector is its length, regardless of direction. It tells you "how much" of the vector there is.

Formula

∣v⃗∣=v12+v22+⋯+vn2=∑i=1nvi2|\vec{v}| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} = \sqrt{\sum_{i=1}^{n} v_i^2}

Example in 2D:

v⃗=[3,4]\vec{v} = [3, 4]
∣v⃗∣=32+42=9+16=25=5|\vec{v}| = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5

Unit Vectors

A unit vector is a vector with magnitude 1. It represents pure direction without magnitude. To create a unit vector from any non-zero vector, divide by its magnitude:

v^=v⃗∣v⃗∣\hat{v} = \frac{\vec{v}}{|\vec{v}|}

Example:

v⃗=[3,4]\vec{v} = [3, 4], ∣v⃗∣=5|\vec{v}| = 5
v^=15[3,4]=[0.6,0.8]\hat{v} = \frac{1}{5}[3, 4] = [0.6, 0.8]

5. The Dot Product (Scalar Product)

Algebraic Formula

The dot product multiplies corresponding components of two vectors and sums them up. This operation combines two vectors into a single scalar value.

u⃗⋅v⃗=u1v1+u2v2+⋯+unvn\vec{u} \cdot \vec{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n

Example:

u⃗=[2,3]\vec{u} = [2, 3] and v⃗=[1,4]\vec{v} = [1, 4]
u⃗⋅v⃗=(2)(1)+(3)(4)=2+12=14\vec{u} \cdot \vec{v} = (2)(1) + (3)(4) = 2 + 12 = 14

Geometric Formula

u⃗⋅v⃗=∣u⃗∣⋅∣v⃗∣⋅cos⁡(θ)\vec{u} \cdot \vec{v} = |\vec{u}| \cdot |\vec{v}| \cdot \cos(\theta)

where θ\theta is the angle between the vectors (in radians or degrees).

uvθAngle Between Vectorsθ = 0°: u·v maximum (parallel)θ = 90°: u·v = 0 (perpendicular)θ = 180°: u·v minimum (opposite)

Key Properties

  • Commutative: u · v = v · u
  • Distributive: u · (v + w) = u · v + u · w
  • If u · v = 0, the vectors are perpendicular (orthogonal)
  • If u · v > 0, the angle between them is acute (< 90°)
  • If u · v < 0, the angle between them is obtuse (> 90°)

→6. Vector Projection

Definition: The projection of vector u⃗\vec{u} onto vector v⃗\vec{v}is the "shadow" that u⃗\vec{u} casts onto v⃗\vec{v}. It tells us how much of u⃗\vec{u}goes in the direction of v⃗\vec{v}.

Scalar Projection (Component)

compv⃗u⃗=u⃗⋅v⃗∣v⃗∣\text{comp}_{\vec{v}}\vec{u} = \frac{\vec{u} \cdot \vec{v}}{|\vec{v}|}

This gives the length of the projection.

Vector Projection

projv⃗u⃗=u⃗⋅v⃗∣v⃗∣2v⃗\text{proj}_{\vec{v}}\vec{u} = \frac{\vec{u} \cdot \vec{v}}{|\vec{v}|^2}\vec{v}

This gives the actual projection vector.

Example:

Project u⃗=[3,4]\vec{u} = [3, 4] onto v⃗=[2,0]\vec{v} = [2, 0]
projv⃗u⃗=3⋅2+4⋅022+02[2,0]=64[2,0]=[3,0]\text{proj}_{\vec{v}}\vec{u} = \frac{3 \cdot 2 + 4 \cdot 0}{2^2 + 0^2}[2, 0] = \frac{6}{4}[2, 0] = [3, 0]

Geometric View

The projection creates a right triangle where the projection is the adjacent side.

vuprojvuProjection of u onto vThe projection is the "shadow" of u on v

7. Cross Product (3D Vectors Only)

Definition: The cross product of two 3D vectors produces a new vector that is perpendicular to both input vectors. Unlike the dot product, the cross product results in a vector, not a scalar.

Formula

u⃗×v⃗=[u2v3−u3v2u3v1−u1v3u1v2−u2v1]\vec{u} \times \vec{v} = \begin{bmatrix} u_2v_3 - u_3v_2 \\ u_3v_1 - u_1v_3 \\ u_1v_2 - u_2v_1 \end{bmatrix}

Determinant Form

u⃗×v⃗=∣i⃗j⃗k⃗u1u2u3v1v2v3∣\vec{u} \times \vec{v} = \begin{vmatrix} \vec{i} & \vec{j} & \vec{k} \\ u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \end{vmatrix}

where i⃗,j⃗,k⃗\vec{i}, \vec{j}, \vec{k} are the standard unit vectors.

Example:

u⃗=[2,3,1]\vec{u} = [2, 3, 1] and v⃗=[1,0,2]\vec{v} = [1, 0, 2]
u⃗×v⃗=[3(2)−1(0)1(1)−2(2)2(0)−3(1)]=[6−3−3]\vec{u} \times \vec{v} = \begin{bmatrix} 3(2) - 1(0) \\ 1(1) - 2(2) \\ 2(0) - 3(1) \end{bmatrix} = \begin{bmatrix} 6 \\ -3 \\ -3 \end{bmatrix}

Geometric View

The cross product creates a vector perpendicular to both input vectors. The direction follows the right-hand rule.

xzyuvu × vCross Product in 3D SpaceRight-Hand Rule:1. Point fingers in direction of u2. Curl fingers toward v3. Thumb points in u × v direction|u × v| = |u| |v| sin(θ) = area of parallelogramResult is perpendicular to both u and v

Key Properties

  • Anti-commutative: u⃗×v⃗=−v⃗×u⃗\vec{u} \times \vec{v} = -\vec{v} \times \vec{u}
  • Magnitude: ∣u⃗×v⃗∣=∣u⃗∣∣v⃗∣sin⁡(θ)|\vec{u} \times \vec{v}| = |\vec{u}||\vec{v}|\sin(\theta)
  • The result is perpendicular to both input vectors
  • If parallel: u⃗×v⃗=0⃗\vec{u} \times \vec{v} = \vec{0}

÷?Why No Vector Division?

The Short Answer: Vector division is not defined because there's no unique way to "undo" vector multiplication. Unlike scalars, vectors have direction, and there are multiple vector operations that could be considered "multiplication."

The Problem

For scalars: a×b=ca \times b = c has a unique solution b=c÷ab = c \div a

For vectors: Given u⃗\vec{u} and v⃗\vec{v}, what wouldu⃗÷v⃗\vec{u} \div \vec{v} mean?

  • Should it undo the dot product? (But that gives a scalar)
  • Should it undo the cross product? (Only works in 3D)
  • Should it find a scalar cc where cv⃗=u⃗c\vec{v} = \vec{u}? (May not exist)

What We Have Instead

  • Scalar division: v⃗÷c=1cv⃗\vec{v} \div c = \frac{1}{c}\vec{v}(divide by a scalar)
  • Unit vectors: v^=v⃗∣v⃗∣\hat{v} = \frac{\vec{v}}{|\vec{v}|}(normalize direction)
  • Projection: Find component of one vector along another
  • Matrix inverse: In linear algebra, we use matrix inverses instead
Key Insight: The absence of vector division isn't a limitation—it reflects the richer structure of vector spaces compared to scalar fields. We have more sophisticated operations that accomplish what we need.