Multilinear maps, tensor products and tensors

Keep several linear inputs separate

A linear map accepts one vector and respects vector addition and scalar multiplication. A bilinear map accepts two vectors and is linear in each input while the other is held fixed. A multilinear map extends this separate linearity to several slots. Tensor products provide a space in which these multilinear rules become ordinary linear rules.

This connects the earlier lessons without replacing them. Vector spaces and bases supply the algebra; matrices supply coordinates; dual functionals distinguish a vector from a rule that measures it. The nonzero rank-one terms σⱼuⱼvⱼ* in the preceding SVD lesson are one important example of tensor constructions, with a dual factor in the map interpretation.

Work throughout this lesson with finite-dimensional real vector spaces. Different factors V and W may have equal dimensions but still be different spaces. A chosen basis permits arrays, but a tensor is an element of a specified tensor-product space, together with its transformation rule—not an arbitrary array carrying a label. Complex and infinite-dimensional settings require additional conventions and, for analytic completions, additional structure.

Bilinear does not mean jointly linear

A map b:V×W→Z is bilinear when b(u+v,w)=b(u,w)+b(v,w), b(au,w)=a b(u,w), and the analogous two identities hold in the second slot. In particular b(0,w)=b(v,0)=0. With k inputs, multilinearity imposes the corresponding condition in every slot separately.

The scalar product of two real numbers is bilinear, and a real inner product is a bilinear scalar-valued map. Matrix multiplication is bilinear in its two matrix arguments. The map (u,v)↦u+v on V×V is generally not bilinear: for fixed v≠0 it does not send u=0 to zero.

Separate linearity is different from linearity on the direct product regarded as one vector space. Scaling both inputs of a bilinear map gives b(au,aw)=a²b(u,w), whereas a linear map on V×W would scale by a. For a trilinear map, scaling all three inputs gives a³. This distinction explains why distributing a tensor product produces cross terms.

For example (u+v)⊗(w+z)=u⊗w+u⊗z+v⊗w+v⊗z. In general it is not u⊗w+v⊗z. The missing cross terms are mathematical information; adding tensor factors coordinate by coordinate is not a valid substitute for multilinear expansion.

Define the tensor product by what it makes linear

A tensor product of V and W consists of a vector space V⊗W and a bilinear map τ:V×W→V⊗W, written τ(v,w)=v⊗w, with this universal property: every bilinear b:V×W→Z has a unique linear map b̃:V⊗W→Z satisfying b̃(v⊗w)=b(v,w). The target Z can be any real vector space.

A pure, simple or decomposable tensor is one of the elements v⊗w. Every tensor is a finite sum of pure tensors, but a sum need not itself be pure. The canonical map τ is bilinear, not an identification of V×W with V⊗W. Its pairs are not unique names for elements: for a≠0, (av)⊗(a⁻¹w)=v⊗w.

One construction starts with the free vector space on all formal pairs [v,w], then takes the quotient by the linearity relations in both slots. The class of [v,w] becomes v⊗w. A bilinear b respects exactly these relations, so extending b to formal sums descends to a well-defined linear map on the quotient. Since pure tensors span it, that extension is unique.

The universal property also proves uniqueness of the construction up to a unique compatible isomorphism. If two candidates have the property, each canonical bilinear map induces a linear map into the other; their composites fix every pure tensor and therefore are identities. This is why different constructions can represent the same tensor product without choosing the same underlying set.

Derive the basis before trusting an array

Let e₁,…,eₘ be a basis of V and f₁,…,fₙ a basis of W. Bilinearity gives v⊗w=Σᵢⱼvⁱwʲ eᵢ⊗fⱼ. Hence the mn products eᵢ⊗fⱼ span V⊗W. They are independent: for each pair (a,b), the bilinear coordinate rule (v,w)↦vᵃwᵇ induces a linear functional extracting exactly that coefficient.

Consequently dim(V⊗W)=mn, and every tensor has a unique coordinate expansion T=ΣᵢⱼTⁱʲeᵢ⊗fⱼ. Its coefficient array is an m-by-n matrix after a row/column convention is chosen. For one pure tensor, this array is the outer product v wᵀ. For a sum, add the outer-product arrays.

The array’s uniqueness is relative to those bases; the written pure-term decomposition is generally not unique. Equal arrays in the same specified tensor basis prove equal tensors. Equal arrays copied into different tensor types or different bases need not represent the same object.

If either factor is the zero vector space, the tensor product is zero. A zero factor gives 0⊗w=v⊗0=0. Conversely, over a field a pure tensor with both factors nonzero is nonzero: choose functionals α,β with α(v)≠0 and β(w)≠0, then the induced functional α⊗β evaluates it to a nonzero product.

Count the fewest pure terms, not the terms on the page

The tensor rank of T∈V⊗W is the least number of pure tensors whose sum is T; define the zero tensor’s rank as 0. For two finite-dimensional factors this equals the matrix rank of its coefficient array. A sum of s outer products has matrix rank at most s, so no decomposition can use fewer than the array rank.

Conversely, if the coefficient matrix has rank r, choose a basis of its column space and write every column in that basis. The rank factorization Tarray=XY expresses it as a sum of r column–row outer products, giving r pure tensors. This proves the equality rather than inferring it from a particular written sum.

For nonzero arrays, purity is equivalent to matrix rank 1; equivalently all 2-by-2 minors vanish. The condition must include every such minor, not merely those using the same row and column indices. The zero array also has all minors zero, but its rank is 0. In dimensions smaller than two, use the rank statement directly.

In R²⊗R², e₁⊗f₁+e₂⊗f₂ has identity coefficient matrix and rank 2, so it is not pure. Yet e₁⊗f₁+e₁⊗f₂=e₁⊗(f₁+f₂) has rank 1. Nonzero summands can also cancel. Array size, number of tensor factors (order), matrix rank and number of terms in one description are different quantities.

For three or more factors, a tensor still means an element of the corresponding product and its rank still counts minimal pure terms, but one matrix rank no longer generally determines that rank. Flattening groups indices into two sets and supplies matrix-rank lower bounds; it does not make all higher-order rank problems ordinary matrix elimination.

A dual slot is a measuring rule, not another vector by default

The dual V* is the space of linear functionals α:V→R. The dual basis ε¹,…,εᵐ is defined by εⁱ(eⱼ)=δⁱⱼ. Thus α=Σᵢαᵢεⁱ and α(v)=Σᵢαᵢvⁱ. A functional takes a vector to a scalar; this role is independent of any inner product.

A tensor v⊗α∈V⊗W* acts naturally on w∈W by w↦α(w)v. Extending linearly gives an isomorphism V⊗W*≅Hom(W,V) in finite dimensions. In bases, v⊗α has coordinate matrix vαᵀ, of rank at most one and of rank one when both factors are nonzero. The dual factor is what turns the second slot into evaluation; an element of V⊗W has no such map interpretation without an additional identification of W with a dual space.

A bilinear form h:V×W→R instead corresponds to an element of V*⊗W*. In finite dimensions it also defines a linear functional on V⊗W by h̃(v⊗w)=h(v,w). The coefficients hᵢⱼ are covariant in both slots, whereas the coefficients Tⁱʲ of V⊗W are contravariant in both. These terms describe how components change when bases change.

For a single V, a tensor of type (p,q) belongs to V⊗⋯⊗V⊗V*⊗⋯⊗V*, with p vector factors and q dual factors in a declared order. It can be evaluated multilinearly on p dual arguments and q vector arguments. The tensor’s type records those roles; conventions about the order of argument lists should always be stated.

Derive three different change-of-basis laws

Use the column convention: new basis vectors of V are the columns of B in its old basis, so xold=Bxnew and xnew=B⁻¹xold. For W use another matrix C. Expanding the same T∈V⊗W in both bases gives Told=B Tnew Cᵀ, hence Tnew=B⁻¹Told C⁻ᵀ. The two inverse factors are essential.

A functional is stored here as a component column α, with evaluation αᵀx. Requiring αoldᵀxold=αnewᵀxnew gives αnew=Bᵀαold. The new dual basis itself uses B⁻ᵀ relative to the old dual basis; the components and basis elements change in inverse ways. This follows from evaluation and does not use a metric.

For a cross-space bilinear form h(x,y)=xᵀHy, substitution gives Hnew=BᵀHold C. For an operator M:V→V, the output must be expressed in the same new vector coordinates as the input, so Mnew=B⁻¹Mold B. A two-vector tensor array, a bilinear-form matrix and an operator matrix therefore obey three different laws.

As an exact example keep B=[[2,1],[1,−1]] from the basis lesson and use C=[[1,1],[0,2]] for W. The tensor (2,1)⊗(1,−1) has old array [[2,−2],[1,−1]], new factor coordinates (1,0) and (3/2,−1/2), and new array [[3/2,−1/2],[0,0]]. Reconstructing with B and Cᵀ returns the old array.

For h with H=[[1,2],[−1,3]], the cross-space array becomes BᵀHC=[[1,15],[2,0]]. Its value on those same two vectors is −6 in either basis. In contrast, M=diag(2,−1) as a V→V operator becomes B⁻¹MB=[[1,1],[2,0]]. The numerical similarity of any entries is not permission to interchange these objects.

Contract matching vector and dual slots

Evaluation V*×V→R is bilinear, so it induces a linear contraction V⊗V*→R with v⊗α↦α(v). Under the operator identification this is trace: for M=Σ vⱼ⊗αⱼ, tr M=Σ αⱼ(vⱼ). The result is independent of how M is written as a sum, because contraction is a well-defined linear map.

For a tensor with several slots, contraction pairs a chosen V factor with a chosen V* factor and sums the corresponding upper and lower component indices. It lowers the type by (1,1). The factors must be appropriate dual partners; having the same number of coordinates does not by itself supply an evaluation.

In particular a V⊗W tensor array with two contravariant indices has no natural trace obtained by summing its diagonal. The invariant pairing with a bilinear form H is ΣᵢⱼhᵢⱼTⁱʲ. Both objects change components: Hnew=BᵀHC and Tnew=B⁻¹TC⁻ᵀ, and their scalar pairing is unchanged.

The Kronecker delta δⁱⱼ represents the identity in V⊗V*. It is not automatically a metric with two lower indices. Index notation is useful when it makes slot types and summations explicit; it becomes misleading when repeated letters are treated as a licence to contract two unrelated vector slots.

Lower or raise an index only after specifying a metric

A real inner product g is a symmetric positive-definite bilinear form, an element of V*⊗V*. It defines an isomorphism ♭:V→V* by v♭(w)=g(v,w). In a basis with Gram matrix G, the covector component column is v♭=Gv. The inverse map ♯ uses G⁻¹. This is an additional geometric structure, not the definition of the dual.

For G=[[2,1],[1,3]] and v=(2,1), lowering gives (5,5), and g(v,v)=15. In the retained new basis B, Gnew=BᵀGB=diag(15,3) and vnew=(1,0). The lowered components are (15,0), which also equal Bᵀ(5,5). Raising with Gnew⁻¹ recovers (1,0).

Even when the old Gram matrix is I in an orthonormal basis, an arbitrary new basis generally has Gnew=BᵀB rather than I. Copying Euclidean component equalities into an oblique basis can therefore change the object. Orthogonal basis changes hide this distinction because B⁻¹=Bᵀ there; the general transformation laws reveal it.

A metric can identify a two-vector tensor with an operator by lowering one slot, but the metric must transform too. It can also define norms and angles. An arbitrary bilinear form need not be positive definite or invertible, so it cannot automatically serve for raising indices or defining a distance.

State which slots can be exchanged

When both factors are the same V, the flip τ:V⊗V→V⊗V sends u⊗v to v⊗u. Over R, T decomposes uniquely into symmetric and antisymmetric parts (T+τT)/2 and (T−τT)/2. In one common basis their coefficient matrices are the familiar symmetric and skew-symmetric parts.

A bilinear form on V×V is symmetric when h(u,v)=h(v,u), and alternating when h(v,v)=0 for every v. Over R, alternation is equivalent to h(u,v)=−h(v,u). These are intrinsic claims about repeated slots of the same space. A cross-space form V×W has no intrinsic slot-exchange symmetry merely because its chosen matrix is square.

The exterior square Λ²V is the quotient of V⊗V imposing v⊗v=0 for every v. Write u∧v for the resulting class of u⊗v. Bilinearity then yields u∧v=−v∧u. Alternating bilinear maps on V×V factor uniquely through Λ²V, just as general bilinear maps factor through V⊗V.

If dim V=n, the products eᵢ∧eⱼ for i<j form a basis of Λ²V, so its dimension is n(n−1)/2. In two dimensions u∧v=(u¹v²−u²v¹)e₁∧e₂. This coefficient is a signed coordinate area; identifying its magnitude with physical area requires a metric and an area convention.

The exterior quotient and the antisymmetric subspace are related but are not literally the same construction. With the quotient map q, one convenient identification over R is j(u∧v)=(u⊗v−v⊗u)/2, for which q∘j is the identity. The unnormalised difference has quotient 2u∧v. Always state the normalization before moving between the two descriptions.

Keep type, units and geometry attached to components

Tensor products can be iterated: (U⊗V)⊗W and U⊗(V⊗W) have a canonical associativity isomorphism determined by pure tensors. There is also a canonical factor-swap isomorphism V⊗W→W⊗V. These are structure-preserving identifications, not a claim that ordered factors have become the same underlying spaces or that multiplication is coordinatewise.

A multilinear map of k finite-dimensional vector arguments is determined by its values on basis tuples, and the universal property converts it to a linear map on their k-fold tensor product. A coordinate array then has k indices with sizes set by the factor dimensions. Tensor order counts factors; each upper or lower index still records its vector or dual role.

In mechanics, a bilinear energy response, a linear constitutive map and a stress-like array can have different input/output types and physical units despite all being stored as matrices in a chosen frame. State the law being represented and any metric identification. A tensor transformation rule preserves that law’s meaning; it does not by itself imply rotational isotropy, conservation, symmetry or positive energy.

For tensor fields on a curved space, the vector and dual slots belong at each point to its tangent and cotangent spaces. A coordinate change uses the Jacobian for vector bases and the inverse relationship for covector components according to the chosen direction of change. This pointwise algebra is the bridge to later multivariable calculus and geometry; general covariant differentiation or transport of tensor fields requires a suitable connection. This does not exclude intrinsic operations such as exterior differentiation of differential forms, or Lie differentiation along a specified vector field.

In computing, an object called a tensor may simply mean an array of numbers. Such arrays are useful storage, but shape alone does not specify a mathematical tensor type, its underlying spaces, a basis-change law or units. Conversely the abstract construction does not prescribe one storage format or a particular numerical decomposition algorithm.

Predict the type before checking the components

Use the exact two-factor examples to compare a pure tensor, an irreducible two-term sum, an alternative description of the same sum, a collapsed sum and several zero cases. Scale by the displayed real scalar and inspect the old and new component arrays. A zero scalar always gives zero; the factor-rescaling identity uses a reciprocal only when that scalar is nonzero.

Check reconstruction Told=B Tnew Cᵀ, the unchanged two-factor rank and the scalar values induced by the displayed bilinear forms. In a separate fixed example, compare vector and dual coordinate changes, an operator similarity and a metric lowering calculation. Their arrays can have the same dimensions while their roles and transformation laws differ.

All controls use two-dimensional rational examples. The preceding universal-property and basis arguments establish the finite-dimensional statements, including higher dimensions; no finite list of examples proves general multilinearity or a higher-order rank formula.

Worked examples

Try it yourself

1. Is (u,v)↦u+v a bilinear map V×V→V for nonzero V?

   Show the answer and reasoning

   No.

   Fixing a nonzero second input leaves a map that does not send the first input zero to zero.

2. If dim V=3 and dim W=4, what is dim(V⊗W)?

   Show the answer and reasoning

   12.

   The 12 products of basis vectors form a basis; the dimensions multiply rather than add.

3. Why is (u+v)⊗(w+z) generally different from u⊗w+v⊗z?

   Show the answer and reasoning

   The former also contains u⊗z and v⊗w.

   Bilinearity distributes separately in both slots.

4. When may one write (av)⊗(a⁻¹w)=v⊗w?

   Show the answer and reasoning

   For a nonzero scalar a.

   At a=0 the reciprocal is undefined; zero scaling still gives the zero tensor by bilinearity.

5. Can a pure tensor over R be zero when both factors are nonzero?

   Show the answer and reasoning

   No.

   Choose dual functionals nonzero on both factors. Their induced functional evaluates the tensor to a nonzero scalar.

6. What is the rank of e₁⊗f₁+e₁⊗f₂?

   Show the answer and reasoning

   1.

   It equals e₁⊗(f₁+f₂), a nonzero pure tensor. The written two-term count is not minimal.

7. Why is e₁⊗f₁+e₂⊗f₂ not pure in two independent two-dimensional factors?

   Show the answer and reasoning

   Its coefficient matrix is I₂, of rank 2.

   An outer product has rank at most 1, while two pure terms already give a rank-two decomposition.

8. For a general nonzero two-factor array, are vanishing principal 2-by-2 minors enough to prove purity?

   Show the answer and reasoning

   No. Every 2-by-2 minor must vanish.

   The 3-by-3 array with entries a₁₂=a₂₃=1 and all others zero has zero principal minors but a nonzero minor using rows 1,2 and columns 2,3.

9. What map does v⊗α∈V⊗W* induce?

   Show the answer and reasoning

   w↦α(w)v.

   The second factor is a functional, supplying the evaluation that a plain W vector does not supply naturally.

10. With old vector coordinates Bxnew and Cynew, how does a V⊗W array change?

    Show the answer and reasoning

    Tnew=B⁻¹Told C⁻ᵀ.

    Expanding both new vector bases gives Told=B Tnew Cᵀ.

11. How does a bilinear-form matrix H on V×W change under those bases?

    Show the answer and reasoning

    Hnew=BᵀHold C.

    Substitute the old vectors into xoldᵀHold yold. Its law differs from the two-vector tensor law.

12. Is summing the diagonal of a V⊗W array an intrinsic trace merely because dim V=dim W?

    Show the answer and reasoning

    No.

    Natural contraction needs paired vector and dual slots or an explicitly supplied additional pairing.

13. Does the dual space require an inner product?

    Show the answer and reasoning

    No.

    Linear functionals define the dual. A metric additionally identifies vectors with particular covectors and permits raising/lowering.

14. Can a square matrix of a cross-space form V×W be declared intrinsically symmetric without identifying the spaces?

    Show the answer and reasoning

    No.

    An exchange of slots requires both inputs to belong to the same space, or a specified identification.

15. For a five-dimensional real V, what is dim Λ²V?

    Show the answer and reasoning

    10.

    There is one basis element eᵢ∧eⱼ for each pair i<j, giving 5·4/2.

16. With q the exterior quotient, what is q(u⊗v−v⊗u)?

    Show the answer and reasoning

    2u∧v.

    The quotient makes v∧u=−u∧v. The normalized representative for q∘j=I divides the difference by 2.

17. Does the equality between two-factor tensor rank and matrix rank automatically extend to tensors with three factors?

    Show the answer and reasoning

    No.

    Flattening yields useful matrix ranks and lower bounds, but one flattening need not determine minimum pure-term rank.

What to carry forward

• Bilinearity is linearity in each slot separately. Scaling or adding both slots introduces products and cross terms.
• The tensor product turns every bilinear map into a unique linear map through its universal property.
• Basis products form a basis, so dimensions multiply; component arrays are unique only after bases and tensor types are specified.
• For two finite factors, tensor rank equals matrix rank and counts the fewest pure terms. Tensor order and a written term count are different.
• A dual slot supplies evaluation. V⊗W*, V⊗W and V*⊗W* have different natural meanings and transformation laws.
• Derive component changes from a declared basis convention. Vector components, covector components, bilinear forms and operators do not all change the same way.
• Natural contraction pairs vector and dual slots. A diagonal sum of an arbitrary two-vector array is not an intrinsic trace.
• A metric is extra structure that permits vector–covector identification, raising/lowering and geometric lengths. It transforms too.
• Symmetry and alternation require appropriate repeated slots. Exterior powers are alternating quotients, with stated factors when identified with antisymmetric subspaces.
• Tensor type, units, bases and field dependence preserve scientific meaning; an array shape alone does not provide them.