update docs

CHANGELOG.md

@@ -1,3 +1,23 @@
+# 1.2.0 :
+
+This version changes quite a few things.
+
++ The upscaled inswapper options are now moved to each face unit. This makes it possible to fine-tune the settings for each face.
+
++ Upscaled inswapper configuration in sd now concerns default values in each unit's interface.
+
++ Pre- and post-inpainting is now possible for each face. Here too, default options are set in the main sd settings.
+
++ Codeformer is no longer the default in post-processing. Don't be surprised if you get bad results by default. You can set it to default in the application's global settings
+
+Bug fixes :
+
++ The problem of saving the grid should be solved.
++ The downscaling problem for inpainting should be solved.
++ Change model download logic and add checksum. This should prevent some bugs.
+
+In terms of the API, it is now possible to create a remote checkpoint and use it in units. See the example in client_api or the tests in the tests directory.
+
 # 1.1.2 :
 
 + Switch face checkpoint format from pkl to safetensors

docs/documentation.markdown

@@ -25,6 +25,8 @@ Here are the main options for configuring a unit:
 
 **You must always have at least one reference face OR a checkpoint. If both are selected, the checkpoint will be used and the reference ignored.**
 
+#### Similarity
+
 Always check for errors in the SD console. In particular, the absence of a reference face or a checkpoint can trigger errors.
 
 + **Comparison of faces** with the obtained swapped face: The swapped face can be compared to the original face using a distance function. The higher this value (from 1 to 0), the more similar the faces are. This calculation is performed if you activate **"Compute Similarity"** or **"Check Similarity"**. If you check the latter, you will have the opportunity to filter the output images with:
@@ -35,7 +37,43 @@ Always check for errors in the SD console. In particular, the absence of a refer
     + **Same gender:** the gender of the source face will be determined and only faces of the same gender will be considered.
     + **Sort by size:** faces will be sorted from largest to smallest.
 
-#### Post-processing
+#### Pre-Inpainting :
+
+This part is applied BEFORE face swapping and only on matching faces.
+
+The inpainting part works in the same way as adetailer. It sends each face to img2img for transformation. This is useful for transforming the face before swapping. For example, using a Lora model before swapping.
+
+You can use a specific model for the replacement, different from the model used for the generation.
+
+For inpainting to be active, denoising must be greater than 0 and the Inpainting When option must be set to:
+
+#### Post-Processing & Advanced Masks Options : (upscaled inswapper)
+
+By default, these settings are disabled, but you can use the global settings to modify the default behavior. These options are called "Default Upscaled swapper..."
+
+The 'Upscaled Inswapper' is an option in SD FaceSwapLab which allows for upscaling of each face using an upscaller prior to its integration into the image. This is achieved by modifying a small segment of the InsightFace code.
+
+The purpose of this feature is to enhance the quality of the face in the final image. While this process might slightly increase the processing time, it can deliver improved results. In certain cases, this could even eliminate the need for additional tools such as Codeformer or GFPGAN in postprocessing. See the processing order section to understand when and how it is used.
+
+![](/assets/images/upscaled_settings.png)
+
+The upscaled inswapper is disabled by default. It can be enabled in the sd options. Understanding the various steps helps explain why results may be unsatisfactory and how to address this issue.
+
++ **upscaler** : LDSR if None. The LDSR option generally gives the best results but at the expense of a lot of computational time. You should test other models to form an opinion. The 003_realSR_BSRGAN_DFOWMFC_s64w8_SwinIR-L_x4_GAN model seems to give good results in a reasonable amount of time. It's not possible to disable upscaling, but it is possible to choose LANCZOS for speed if Codeformer is enabled in the upscaled inswapper. The result is generally satisfactory.
++ **restorer** : The face restorer to be used if necessary. Codeformer generally gives good results.
++ **sharpening** can provide more natural results, but it may also add artifacts. The same goes for **color correction**. By default, these options are set to False.
++ **improved mask:** The segmentation mask for the upscaled swapper is designed to avoid the square mask and prevent degradation of the non-face parts of the image. It is based on the Codeformer implementation. If "Use improved segmented mask (use pastenet to mask only the face)" and "upscaled inswapper" are checked in the settings, the mask will only cover the face, and will not be squared. However, depending on the image, this might introduce different types of problems such as artifacts on the border of the face.
++ **erosion factor:** it is possible to adjust the mask erosion parameters using the erosion settings. The higher this setting is, the more the mask is reduced.
+
+#### Post-Inpainting :
+
+This part is applied AFTER face swapping and only on matching faces.
+
+This is useful for adding details to faces. The stronger the denoising, the more likely you are to lose the resemblance of the face. Some samplers (DPM variants for instance) seem to better preserve this resemblance than others.
+
+## Global Post-processing
+
+By default, these settings are disabled, but you can use the global settings to modify the default behavior. These options are called default "UI Default global post processing..."
 
 The post-processing window looks very much like what you might find in the extra options, except for the inpainting part. The process takes place after all units have swapped faces.
 
@@ -82,21 +120,7 @@ The checkpoint can then be used in the main interface (use refresh button)
 ![](/assets/images/checkpoints_use.png)
 
 
-## Upscaled-inswapper
-
-The 'Upscaled Inswapper' is an option in SD FaceSwapLab which allows for upscaling of each face using an upscaller prior to its integration into the image. This is achieved by modifying a small segment of the InsightFace code.
-
-The purpose of this feature is to enhance the quality of the face in the final image. While this process might slightly increase the processing time, it can deliver improved results. In certain cases, this could even eliminate the need for additional tools such as Codeformer or GFPGAN in postprocessing. See the processing order section to understand when and how it is used.
-
-![](/assets/images/upscaled_settings.png)
 
-The upscaled inswapper is disabled by default. It can be enabled in the sd options. Understanding the various steps helps explain why results may be unsatisfactory and how to address this issue.
-
-+ **upscaler** : LDSR if None. The LDSR option generally gives the best results but at the expense of a lot of computational time. You should test other models to form an opinion. The 003_realSR_BSRGAN_DFOWMFC_s64w8_SwinIR-L_x4_GAN model seems to give good results in a reasonable amount of time. It's not possible to disable upscaling, but it is possible to choose LANCZOS for speed if Codeformer is enabled in the upscaled inswapper. The result is generally satisfactory.
-+ **restorer** : The face restorer to be used if necessary. Codeformer generally gives good results.
-+ **sharpening** can provide more natural results, but it may also add artifacts. The same goes for **color correction**. By default, these options are set to False.
-+ **improved mask:** The segmentation mask for the upscaled swapper is designed to avoid the square mask and prevent degradation of the non-face parts of the image. It is based on the Codeformer implementation. If "Use improved segmented mask (use pastenet to mask only the face)" and "upscaled inswapper" are checked in the settings, the mask will only cover the face, and will not be squared. However, depending on the image, this might introduce different types of problems such as artifacts on the border of the face.
-+ **fthresh and erosion factor:** it is possible to adjust the mask erosion parameters using the fthresh and erosion settings. The higher these settings are (particularly erosion), the more the mask is reduced.
 
 ## Processing order:
 
@@ -123,42 +147,20 @@ The extension is activated after all other extensions have been processed.  Duri
 
 ![](/assets/images/step4.png)
 
-## Settings
 
-Here are the parameters that can be configured in sd settings and their default values
+## API
 
-### General Settings :
+A specific API is available.  To understand how it works you can have a look at the example file in `client_utils`. You can also view the application's tests in the `tests` directory.
 
- Name | Description | Default Value
----|---|---
- faceswaplab_model | Insightface model to use| models[0] if len(models) > 0 else "None"
- faceswaplab_keep_original | keep original image before swapping. It true, will show original image | False
- faceswaplab_units_count | How many faces units to use(requires restart) | 3
- faceswaplab_detection_threshold | Detection threshold to use to detect face, if low will detect non human face as face | 0.5
+The API is documented in the FaceSwapLab tags in the http://localhost:7860/docs docs.
 
-### Default Settings :
+You don't have to use the api_utils.py file and pydantic types, but it can save time.
 
-These parameters are used to configure the default settings displayed in post-processing.
 
- Name | Description | Default Value
- faceswaplab_pp_default_face_restorer | UI Default post processing face restorer (requires restart) | None
- faceswaplab_pp_default_face_restorer_visibility | UI Default post processing face restorer visibility (requires restart) | 1
- faceswaplab_pp_default_face_restorer_weight | UI Default post processing face restorer weight (requires restart) | 1
- faceswaplab_pp_default_upscaler | UI Default post processing upscaler (requires restart) | None
- faceswaplab_pp_default_upscaler_visibility | UI Default post processing upscaler visibility(requires restart) | 1
+## Settings
 
-### Upscaled inswapper Settings :
+You can change the program's default behavior in your webui's global settings (FaceSwapLab section in settings). This is particularly useful if you want to have default options for inpainting or for post-processsing, for example.
 
-These parameters are used to control the upscaled inswapper, see above.
+The interface must be restarted to take the changes into account. Sometimes you have to reboot the entire webui server.
 
- Name | Description | Default Value
- faceswaplab_upscaled_swapper | Upscaled swapper. Applied only to the swapped faces. Apply transformations before merging with the original image | False
- faceswaplab_upscaled_swapper_upscaler | Upscaled swapper upscaler (Recommended : LDSR but slow) | None
- faceswaplab_upscaled_swapper_sharpen | Upscaled swapper sharpen | False
- faceswaplab_upscaled_swapper_fixcolor | Upscaled swapper color correction | False
- faceswaplab_upscaled_improved_mask | Use improved segmented mask (use pastenet to mask only the face) | True
- faceswaplab_upscaled_swapper_face_restorer | Upscaled swapper face restorer | None
- faceswaplab_upscaled_swapper_face_restorer_visibility | Upscaled swapper face restorer visibility | 1
- faceswaplab_upscaled_swapper_face_restorer_weight | Upscaled swapper face restorer weight (codeformer) | 1
- faceswaplab_upscaled_swapper_fthresh | Upscaled swapper fthresh (diff sensitivity) 10 = default behaviour. Low impact | 10
- faceswaplab_upscaled_swapper_erosion | Upscaled swapper mask erosion factor, 1 = default behaviour. The larger it is, the more blur is applied around the face. Too large and the facial change is no longer visible | 1
+There may be display bugs on some radio buttons that may not display the value (Codeformer might look disabled for instance). Check the logs to ensure that the transformation has been applied.

install.py

@@ -2,39 +2,9 @@
 import os
 import pkg_resources
 import sys
-from tqdm import tqdm
-import urllib.request
 
-req_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "requirements.txt")
-
-models_dir = os.path.abspath("models/faceswaplab")
-faces_dir = os.path.abspath(os.path.join("models", "faceswaplab", "faces"))
-model_url = "https://huggingface.co/henryruhs/roop/resolve/main/inswapper_128.onnx"
-model_name = os.path.basename(model_url)
-model_path = os.path.join(models_dir, model_name)
-
-
-def download(url: str, path: str) -> None:
-    request = urllib.request.urlopen(url)
-    total = int(request.headers.get("Content-Length", 0))
-    with tqdm(
-        total=total, desc="Downloading", unit="B", unit_scale=True, unit_divisor=1024
-    ) as progress:
-        urllib.request.urlretrieve(
-            url,
-            path,
-            reporthook=lambda count, block_size, total_size: progress.update(
-                block_size
-            ),
-        )
-
-
-os.makedirs(models_dir, exist_ok=True)
-os.makedirs(faces_dir, exist_ok=True)
-
-if not os.path.exists(model_path):
-    download(model_url, model_path)
 
+req_file = os.path.join(os.path.dirname(os.path.realpath(__file__)), "requirements.txt")
 
 print("Checking faceswaplab requirements")
 with open(req_file) as file:

scripts/configure.py

@@ -0,0 +1,79 @@
+import os
+from tqdm import tqdm
+import urllib.request
+from scripts.faceswaplab_utils.faceswaplab_logging import logger
+from scripts.faceswaplab_swapping.swapper import is_sha1_matching
+from scripts.faceswaplab_utils.models_utils import get_models
+from scripts.faceswaplab_globals import *
+from packaging import version
+import pkg_resources
+
+ALREADY_DONE = False
+
+
+def check_configuration() -> None:
+    global ALREADY_DONE
+
+    if ALREADY_DONE:
+        return
+
+    logger.info(f"FaceSwapLab {VERSION_FLAG} Config :")
+
+    # This has been moved here due to pb with sdnext in install.py not doing what a1111 is doing.
+    models_dir = MODELS_DIR
+    faces_dir = FACES_DIR
+
+    model_url = "https://huggingface.co/henryruhs/roop/resolve/main/inswapper_128.onnx"
+    model_name = os.path.basename(model_url)
+    model_path = os.path.join(models_dir, model_name)
+
+    def download(url: str, path: str) -> None:
+        request = urllib.request.urlopen(url)
+        total = int(request.headers.get("Content-Length", 0))
+        with tqdm(
+            total=total,
+            desc="Downloading inswapper model",
+            unit="B",
+            unit_scale=True,
+            unit_divisor=1024,
+        ) as progress:
+            urllib.request.urlretrieve(
+                url,
+                path,
+                reporthook=lambda count, block_size, total_size: progress.update(
+                    block_size
+                ),
+            )
+
+    os.makedirs(models_dir, exist_ok=True)
+    os.makedirs(faces_dir, exist_ok=True)
+
+    if not is_sha1_matching(model_path, EXPECTED_INSWAPPER_SHA1):
+        logger.error(
+            "Suspicious sha1 for model %s, check the model is valid or has been downloaded adequately. Should be %s",
+            model_path,
+            EXPECTED_INSWAPPER_SHA1,
+        )
+
+    gradio_version = pkg_resources.get_distribution("gradio").version
+
+    if version.parse(gradio_version) < version.parse("3.32.0"):
+        logger.warning(
+            "Errors may occur with gradio versions lower than 3.32.0. Your version : %s",
+            gradio_version,
+        )
+
+    if not os.path.exists(model_path):
+        download(model_url, model_path)
+
+    def print_infos() -> None:
+        logger.info("FaceSwapLab config :")
+        logger.info("+ MODEL DIR : %s", models_dir)
+        models = get_models()
+        logger.info("+ MODELS: %s", models)
+        logger.info("+ FACES DIR : %s", faces_dir)
+        logger.info("+ ANALYZER DIR : %s", ANALYZER_DIR)
+
+    print_infos()
+
+    ALREADY_DONE = True

scripts/faceswaplab.py

@@ -2,6 +2,7 @@
 import traceback
 
 from scripts import faceswaplab_globals
+from scripts.configure import check_configuration
 from scripts.faceswaplab_api import faceswaplab_api
 from scripts.faceswaplab_postprocessing import upscaling
 from scripts.faceswaplab_settings import faceswaplab_settings
@@ -65,8 +66,8 @@
 
 class FaceSwapScript(scripts.Script):
     def __init__(self) -> None:
-        logger.info(f"FaceSwapLab {VERSION_FLAG}")
         super().__init__()
+        check_configuration()
 
     @property
     def units_count(self) -> int:

scripts/faceswaplab_globals.py

@@ -4,6 +4,8 @@
 MODELS_DIR = os.path.abspath(os.path.join("models", "faceswaplab"))
 ANALYZER_DIR = os.path.abspath(os.path.join(MODELS_DIR, "analysers"))
 FACE_PARSER_DIR = os.path.abspath(os.path.join(MODELS_DIR, "parser"))
+FACES_DIR = os.path.abspath(os.path.join(MODELS_DIR, "faces"))
+
 REFERENCE_PATH = os.path.join(
     scripts.basedir(), "extensions", "sd-webui-faceswaplab", "references"
 )
@@ -13,3 +15,4 @@
 
 # The NSFW score threshold. If any part of the image has a score greater than this threshold, the image will be considered NSFW.
 NSFW_SCORE_THRESHOLD: float = 0.7
+EXPECTED_INSWAPPER_SHA1 = "17a64851eaefd55ea597ee41e5c18409754244c5"

scripts/faceswaplab_swapping/swapper.py

@@ -2,6 +2,7 @@
 import os
 from dataclasses import dataclass
 from pprint import pformat
+import traceback
 from typing import Any, Dict, Generator, List, Set, Tuple, Optional
 import tempfile
 from tqdm import tqdm
@@ -271,7 +272,9 @@ def getAnalysisModel() -> insightface.app.FaceAnalysis:
         logger.info("Load analysis model, will take some time. (> 30s)")
         # Initialize the analysis model with the specified name and providers
 
-        with tqdm(total=1, desc="Loading analysis model", unit="model") as pbar:
+        with tqdm(
+            total=1, desc="Loading analysis model (first time is slow)", unit="model"
+        ) as pbar:
             with capture_stdout() as captured:
                 model = insightface.app.FaceAnalysis(
                     name="buffalo_l",
@@ -291,14 +294,21 @@ def getAnalysisModel() -> insightface.app.FaceAnalysis:
 
 def is_sha1_matching(file_path: str, expected_sha1: str) -> bool:
     sha1_hash = hashlib.sha1(usedforsecurity=False)
-
-    with open(file_path, "rb") as file:
-        for byte_block in iter(lambda: file.read(4096), b""):
-            sha1_hash.update(byte_block)
-        if sha1_hash.hexdigest() == expected_sha1:
-            return True
-        else:
-            return False
+    try:
+        with open(file_path, "rb") as file:
+            for byte_block in iter(lambda: file.read(4096), b""):
+                sha1_hash.update(byte_block)
+            if sha1_hash.hexdigest() == expected_sha1:
+                return True
+            else:
+                return False
+    except Exception as e:
+        logger.error(
+            "Failed to check model hash, check the model is valid or has been downloaded adequately : %e",
+            e,
+        )
+        traceback.print_exc()
+        return False
 
 
 @lru_cache(maxsize=1)
@@ -334,8 +344,6 @@ def getFaceSwapModel(model_path: str) -> upscaled_inswapper.UpscaledINSwapper:
         logger.error(
             "Loading of swapping model failed, please check the requirements (On Windows, download and install Visual Studio. During the install, make sure to include the Python and C++ packages.)"
         )
-        import traceback
-
         traceback.print_exc()
         raise FaceModelException("Loading of swapping model failed")
 
@@ -379,6 +387,8 @@ def get_faces(
         # Sort the detected faces based on their x-coordinate of the bounding box
         return sorted(face, key=lambda x: x.bbox[0])
     except Exception as e:
+        logger.error("Failed to get faces %s", e)
+        traceback.print_exc()
         return []
 
 

scripts/faceswaplab_swapping/upscaled_inswapper.py

@@ -193,7 +193,7 @@ def compute_diff(bgr_fake: CV2ImgU8, aimg: CV2ImgU8) -> CV2ImgU8:
 
                 if options:
                     logger.info("*" * 80)
-                    logger.info(f"Upscaled inswapper")
+                    logger.info(f"Inswapper")
 
                     if options.upscaler_name:
                         # Upscale original image