MediaPipe and the 468 landmarks behind every Facet score

Every Facet score starts the same way. Your photo is processed by MediaPipe Face Mesh, an open-source machine vision model from Google Research that maps 468 landmark points across the face at sub-millimeter precision relative to inter-pupillary distance.

Those 468 points are the substrate. Every clinical parameter Facet calculates, canthal tilt, midface ratio, brow peak position, alar-to-face-width, vermilion border, FWHR, is computed from coordinates between specific landmarks. The math runs in a deterministic TypeScript engine, not in a model.

Why landmarks and not a vision model

A model that takes a photo and outputs a score is a black box. The score moves and you cannot tell why. Bias from training data is invisible. The output cannot be defended in a clinical conversation.

Landmark-based scoring is the opposite. The landmarks are reproducible. The math is auditable. If your canthal tilt is reported at 7.3 degrees, that is the angle between the medial and lateral canthal landmarks of one eye and a horizontal reference, computed once and shown to you. There is no model in the loop deciding what the right answer should be.

What the 468 points cover

• Outer face contour (jawline, hairline, ear-adjacent landmarks).
• Brow line, upper and lower lash lines, medial and lateral canthi.
• Nose: bridge, alae, tip, columella, nasolabial junction.
• Lips: vermilion border, cupid's bow, oral commissures.
• Internal mid-face: cheek prominence, infraorbital landmarks, philtrum.

What MediaPipe cannot do

Landmarks describe shape. They do not describe colour, texture, or anything beneath the skin. For those, Facet uses additional computer-vision passes on the pixels inside specific landmark-bounded regions.

• Skin homogeneity uses Fink chromophore variance computed on the cheek region defined by landmarks.
• Pore density uses a Laplacian-of-Gaussian detector on the same region.
• Erythema uses redness analysis on a different landmark-bounded patch.

Some clinical parameters require a profile photo to compute properly. Gonial angle, Goode nasal index, nasolabial angle, and cervicomental angle all need a true side view. The current Facet pipeline accepts profile photos optionally and flags those parameters as deferred when only a frontal photo is supplied.

Where Claude comes in

After every number is computed deterministically, Claude is invoked for one job: writing qualitative observations and recommendation protocols in plain English. Claude never decides scores. A defensive function called stripClaudeScores removes any score field Claude tries to return, by design. Claude is a writer in this system, not a scorer.

Why this matters

Two years from now, large vision models will be cheaper and faster, and a lot of face-scoring tools will be wrappers around them. The reason to build deterministically anyway is that scoring you cannot defend is worth less than no scoring at all. If a user asks why their nasal width ratio is 0.31, the answer has to be a number, a paper, and a coordinate, not "the model returned that."