Skip to content

Latest commit

 

History

History
156 lines (116 loc) · 5.25 KB

README.md

File metadata and controls

156 lines (116 loc) · 5.25 KB
Raw

PresentaPulse

PresentaPulse leverages the power of LivePortrait and Real-ESRGAN to create ultra-realistic animated portraits. Developed by Tarek Tarabichi from 2TInteractive, this tool integrates advanced AI models to enhance image quality and create stunning animations.

Features

  • Realistic Animations: Generate ultra-realistic animations from still images.
  • Image Enhancement: Use Real-ESRGAN to upscale and enhance image quality.
  • Interactive UI: User-friendly interface powered by Gradio.
  • Multiprocessing: Optimized for performance with multiprocessing support.

image

What to expect (Magic!)

original Original with Expression Applied LivePortrait Face Markers is magical

How to use:

  • Choose your image
  • Choose your Facial Expression (driving video)
  • Kick-Starts Multi-Threading Operation on Windows Operating System,
  • downscales the original Facial Expression (driving video),
  • Enhances frames using Real-Esrgan,
  • Enhances Generated Video,
  • Re-Assemble Video for Download

Installation

Prerequisites

  • Python 3.8 or higher
  • Pip
  • Virtualenv

Clone the Repository

git clone https://github.com/LebToki/PresentaPulse.git
cd PresentaPulse

Create and Activate Virtual Environment

python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

Install Dependencies

pip install -r requirements.txt

Usage

Run the Application

python app.py

Once the application is running, you can access the web interface using the local URL provided in the terminal.

Recommended Models

Worth mentioning:

  • You need to mind that I am hardcoding my directory path for now! So, you are recommended to change that path to work with your own system deployment path until I get to a mature version.
  • I suggest you clone Real-ESRGAN in the root folder of the project and build it within to avoid messing up with the codebase and compatibility.
  • Make sure your Torch has CUDA support.
  • Make sure your Onnxruntime has CUDA support.
  • Experiment with different expressions (Pexels has a few solid good ones).
  • Avoid facial expressions with tons of camera movements, pans, and tilts.
  • Have fun and experiment.
  • I would appreciate it if you follow me, star the repo, and check the other AI-related repos too :)

Pretrained Models and Configuration: create this directory and to download the Pretrained models

D:\tools\LivePortrait\pretrained_weights

you may need to run git lfs install first

git clone https://huggingface.co/KwaiVGI/liveportrait pretrained_weights

alternatively you may try to download the archive from Google Drive (https://drive.google.com/drive/folders/1UtKgzKjFAOmZkhNK-OYT0caJ_w2XAnib)

Go to Hugging Face and look for these additional models

download them to the pretrained_weights folder too

- realesr-animevideov3.pth
- RealESRGAN_x4plus.pth
- RealESRGAN_x4plus_anime_6B.pth
- RealESRNet_x4plus.pth

Directory Structure of /pretrained_weights/

realesr-animevideov3.pth
RealESRGAN_x4plus.pth
RealESRGAN_x4plus_anime_6B.pth
RealESRNet_x4plus.pth

├───insightface
│   └───models
│       └───buffalo_l
│               2d106det.onnx
│               det_10g.onnx
│               
└───liveportrait
    │   landmark.onnx
    │   
    ├───base_models
    │       appearance_feature_extractor.pth
    │       motion_extractor.pth
    │       spade_generator.pth
    │       warping_module.pth
    │       
    └───retargeting_models
            stitching_retargeting_module.pth

Acknowledgements

Let'us all give kudos to the original creator of LivePortrait and every other library we are using, including:

Contributing

I welcome contributions! Please fork the repository and create a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact

For more information, please visit 2TInteractive (https://2tinteractive.com) or contact Tarek Tarabichi.

TO DO

  • The Fun-part is still being worked on: Taking this whole thing into Real-ESRGAN and applying the highest quality possible using RealESRGAN_x4plus_anime_6B.pth or RealESRGAN_x4plus.pth.
  • The Wishlist is growing:
    • Possibly integrate multiface with object and zones detection and targeting.
    • Possibly working around the square generation limitation (1:1).
    • Possibly trying to integrate MimicMotion which will require updating the workflow to include the use of MimicMotion for generating motion sequences before or after the Real-ESRGAN enhancement. The only issue is its high demands on PC resources which could be an issue (for now at least).