Symbolic linear algebra gradients/matrix calculus

Can Mathematica generate symbolic expressions for gradients?

For example, if $x_1$ and $x_2$ are two points, could I get Mathematica to generate expressions similar to the following?

$\frac{\partial \left|x_1 - x_2\right|}{\partial x_1} = \frac{x_1 - x_2}{\left|x_1 - x_2\right|}$

$\frac{\partial \left|x_1 - x_2\right|}{\partial x_2} = \frac{x_2 - x_1}{\left|x_1 - x_2\right|}$

My experience with Mathematica is limited. I know how to get derivatives w.r.t. scalars.

Elsewhere on this site I found this question and from the answers it looks like Mathematica recently acquired the ability to do some amount of symbolic linear algebra.

Answer

Everything you want in the question can be done by defining the derivative of the Norm:

Derivative[1][Norm][z_] := z/Norm[z]


D[Norm[x - y], {x}]
(* ==> (x - y)/Norm[x - y] *)

Simplify[D[Norm[x - y], {y}]]
(* ==> (-x + y)/Norm[x - y] *)

Here, the syntax I used for the derivatives is such that it would remain valid if x or y were replaced by vectors (i.e., Lists).