To fix distorted faces in Stable Diffusion, give the face more pixels: frame it closer or use Hires. fix, then inpaint only the face with a small margin so it is redrawn at full resolution. ADetailer (AUTOMATIC1111) and FaceDetailer (ComfyUI) do that automatically. Face restoration with CodeFormer or GFPGAN is a quick last pass, but it can trade likeness for a cleaner face.
How to fix distorted faces: the short answer
Most distorted faces in Stable Diffusion are small faces. When a face covers only a few dozen pixels, the model has too little room for two matching eyes, a nose, and a mouth. Fix it by giving the face more pixels: frame the person closer, generate larger with Hires. fix, or keep the image and inpaint only the face with Only masked so it is redrawn at full resolution and pasted back.
Automatic tools do the same repair for you: ADetailer in AUTOMATIC1111 and the Impact Pack's FaceDetailer in ComfyUI detect each face, mask it, and inpaint it. Face restoration (CodeFormer or GFPGAN) is a faster last pass that rebuilds a face with a separate model, at some cost to likeness.
Why Stable Diffusion distorts faces
The model's makers list it as a known limitation: the Stable Diffusion v1.5 and SDXL model cards both say ‘Faces and people in general may not be generated properly.’ A different checkpoint can do better, but plan for a repair step.
Size makes it worse. Stable Diffusion works in a compressed latent space: the v1.5 model card says the autoencoder shrinks each side of the image by a factor of 8. A face 64 pixels wide in a 512×512 image is only 8 latent pixels across, which leaves little room for fine features such as two matching eyes. The same face at 256 pixels wide gets 32 latent pixels. This is one reason close-ups tend to hold up better than faces in group shots and full-body images.
Resolution matters too. Stable Diffusion v1.5 was fine-tuned at 512×512, so generate the first pass near the size the model was trained for and add pixels afterwards, for example with Hires. fix.
Prevent distorted faces before you generate
Frame for the face. Ask for a portrait, a head-and-shoulders shot, or a medium shot rather than a full-body or crowd scene when the face matters; fewer people means each face gets more of the image. A clear viewing angle (facing the camera or a simple three-quarter view) is easier than an extreme profile or a face partly hidden by hands or hair.
Use a two-pass render for larger images. In AUTOMATIC1111, Hires. fix renders the image at a lower resolution, upscales it, and adds detail in a second pass at the higher resolution, so faces get their detail added with more pixels available.
Generate several seeds and keep the one with the best face before repairing anything; Stable Diffusion seeds explained covers keeping that composition. Terms such as ‘bad anatomy’ or ‘deformed’ in the negative prompt can lower the odds of a broken face, but they do not fix the pixel problem above.
PhoneDiffusion generates on-device on iPhone or iPad, with optional cloud models. Download on the App Store →
How to fix a face with inpainting
Send the image to img2img in AUTOMATIC1111, open the Inpaint tab, and paint over the face with a little of the hair and neck around it. Set Inpaint area to Only masked: the wiki explains that only the masked region is then resized to the target resolution and pasted back, so a small face is redrawn at the full target resolution, such as 512×512, instead of at its size in the picture. Only masked padding, pixels (32 by default) sets how much of the surroundings the model sees, which helps the new face match the head angle and light.
Keep Masked content on original so the redraw starts from the existing face and keeps its pose. Denoising strength (0.75 by default in img2img) sets how much changes: lower values keep the person's likeness and expression, higher values redraw more (to change the expression on purpose, see how to prompt emotion and facial expression). Describe the face in the prompt, for example ‘a woman's face, soft daylight, natural skin texture, looking at the camera’, generate a few attempts, and keep the best. Mask blur (4 by default) softens the seam.
In ComfyUI, paint the mask in the MaskEditor and use VAE Encode (for Inpainting) for a full redraw, or VAE Encode with Set Latent Noise Mask and a lower denoise to keep the original face as a starting point. In Diffusers, the inpainting pipeline's padding_mask_crop crops and upscales the masked area the same way as Only masked. The bad hands guide covers these inpainting settings in more detail.
Automatic face fixing: ADetailer and FaceDetailer
ADetailer is an AUTOMATIC1111 extension that finds faces after each generation and inpaints them automatically. face_yolov8n.pt and face_yolov8s.pt detect both 2D and realistic faces; the mediapipe_face models detect realistic faces. Each of its inpainting options matches one on the Inpaint tab, so the same rules apply: a moderate denoising strength keeps the likeness, a higher one redraws more. The maintainer notes it was developed and tested with SD 1.5 on the latest AUTOMATIC1111.
In ComfyUI, the Impact Pack's FaceDetailer node combines a face detector with a Detailer node that improves each face it finds. The YOLO face detectors load through UltralyticsDetectorProvider, which comes from the separate Impact Subpack.
Face restoration: CodeFormer and GFPGAN
Face restoration uses a separate model to rebuild faces after the image is generated. AUTOMATIC1111 supports GFPGAN and CodeFormer: there is a Restore faces option for new generations and a separate tab that restores faces in any picture, with a slider for how visible the effect is. Since version 1.6.0, Restore faces lives in Settings; the ‘Options in main UI’ setting puts it back next to the prompt.
CodeFormer's fidelity weight trades quality against likeness: its authors say smaller values tend to give a higher-quality result and larger values a result closer to the input. AUTOMATIC1111's CodeFormer weight defaults to 0.5, where 0 is the maximum effect and 1 the minimum. Compare with and without restoration at full size, and raise the weight if the restored face no longer looks like the person.
PhoneDiffusion availability and article scope
PhoneDiffusion runs curated Core ML model packs (SD 1.5, SD 2.1, and SDXL) on supported iPhone and iPad devices with iOS 17 or later; local generation is private and works offline once a pack is downloaded. It also offers optional cloud models, such as FLUX and Seedream, on paid monthly or yearly plans; those are text-to-image only, and the text prompt is sent online to generate the image.
To repair a face in the app, open the image and choose Inpaint, tap Select area, and paint the face with the Brush (Brush size and Edge softness adjust the stroke); then describe the replacement and set Replacement strength. Only the painted area changes. If you give a result a thumbs-down and choose ‘Faces, hands, or shapes look wrong’, the app suggests one person and a clear viewing angle with the face closer in the frame, another variation (more steps alone does not reliably fix anatomy), and repairing a small area with Inpaint. Several packs include ‘bad anatomy’ or ‘deformed’ in a built-in negative prompt that the app applies for you.
Upscaling in the app enlarges an image but does not repair a distorted face, and the app's own tips warn that an upscaler can change faces. PhoneDiffusion has no automatic face detection, no face restoration such as CodeFormer or GFPGAN, and no Hires. fix; the AUTOMATIC1111, ComfyUI, Diffusers, ADetailer, and Impact Pack steps above describe desktop and server tools.
Field notes
- Distorted faces are a known Stable Diffusion limitation and are worst when the face is small in the frame.
- Give the face more pixels: frame closer, use Hires. fix, or inpaint only the face so it is redrawn at full resolution.
- ADetailer and the Impact Pack's FaceDetailer detect and inpaint faces automatically.
- CodeFormer and GFPGAN restore faces quickly but can smooth skin or change the likeness; compare before keeping the result.
Try it on your phone
- Generate a few seeds with the face larger in the frame and keep the best image.
- Mask the face with a little of the hair and neck around it.
- Inpaint only the masked area at a moderate strength, describing the face, its angle, and the light.
- Generate several attempts and keep the one that matches the person and the scene.
- Use face restoration only as a final, optional pass, and compare it with the inpainted version at full size.
Sources and verification
Primary and maintainer documentation checked for this guide. Product-specific details were reviewed against the current PhoneDiffusion scope.
- Stable Diffusion v1.5 model card: limitationsModel card limitation: ‘Faces and people in general may not be generated properly’, alongside weak compositionality.
- Stability AI: SDXL base 1.0 model card, limitationsModel card limitation: ‘Faces and people in general may not be generated properly’ also applies to SDXL.
- Stable Diffusion v1.5 model card: training detailsModel card: the autoencoder downsamples by a factor of 8 (H × W × 3 images become H/8 × W/8 × 4 latents), and v1.5 was fine-tuned at 512×512.
- AUTOMATIC1111 Stable Diffusion web UI: Features wiki: face restoration and Hires. fixProject wiki: face restoration with GFPGAN or CodeFormer, a separate tab that restores faces in any picture with a visibility slider, and Hires. fix, which renders at a lower resolution, upscales, and adds detail at the higher resolution.
- AUTOMATIC1111 Stable Diffusion web UI: Features wiki: inpainting, negative prompt, Hires. fixProject wiki: Masked content options, ‘Only masked’ inpainting that resizes just the masked region and pastes it back, dedicated inpainting checkpoints, negative prompts, and Hires. fix.
- AUTOMATIC1111 Stable Diffusion web UI: modules/ui.pySource for the inpaint controls and defaults: Mask blur 4, Masked content ‘original’, Inpaint area ‘Whole picture’ or ‘Only masked’, Only masked padding 32 pixels, and img2img Denoising strength 0.75.
- AUTOMATIC1111 Stable Diffusion web UI: v1.6.0 release notesRelease notes: Restore faces moved out of the main UI into Settings; the ‘Options in main UI’ setting puts it back.
- AUTOMATIC1111 Stable Diffusion web UI: shared_options.py (face restoration settings)Source: Restore faces, Face restoration model (CodeFormer by default), and CodeFormer weight (default 0.5; 0 = maximum effect, 1 = minimum effect).
- CodeFormer: READMEAuthors' repository: fidelity weight w in [0, 1]; smaller w tends to give higher quality, larger w higher fidelity to the input. Licensed under the NTU S-Lab License 1.0.
- ADetailer: README: face detection modelsExtension maintainer's documentation: face_yolov8n.pt and face_yolov8s.pt detect 2D and realistic faces, the mediapipe_face models detect realistic faces, and each inpainting option matches one on the Inpaint tab.
- ComfyUI Impact Pack: READMECustom-node documentation: FaceDetailer combines a face detector with a Detailer node that improves each face; the UltralyticsDetectorProvider detector comes from the separate Impact Subpack.
- ComfyUI: inpainting tutorialOfficial walkthrough: paint the mask with Open in MaskEditor, use VAE Encode (for Inpainting) with grow_mask_by for a soft transition, and prefer a dedicated inpainting checkpoint.
- ComfyUI: nodes.py (VAE Encode for Inpainting, Set Latent Noise Mask, KSampler)Source: VAE Encode (for Inpainting) defaults grow_mask_by to 6 and fills masked pixels with neutral gray before encoding; Set Latent Noise Mask limits sampling to a mask; KSampler denoise keeps more structure at lower values.
- Hugging Face Diffusers: InpaintingMaintainer guide: the mask's white pixels are repainted from the prompt and black pixels are kept; strength sets how far the masked area moves from the original.