Lama-cleaner

New Model: InstructPix2Pix

- `--model instruct_pix2pix`
- github: https://github.com/timothybrooks/instruct-pix2pix
- paper: https://arxiv.org/abs/2211.09800

InstructPix2Pix model can work without mask.


| original | result | Prompt | Guidance Scale | Image Guidance Scale | Seed |
| :----------------------------------------------- | :------------------------------------------------------------: | -------------------------------------------: | -------------: | -------------------: | --------: |
|| | turn the duck blue, in the style of van gogh | 9 | 1.6 | 108227038 |
| || make her hair black | 7 | 1.5 | 749559112 |
| |  | make it summer | 7 | 1.5 | 195504576 |


Share result by GIF
Click the `GIF` button in the toolbar, you can download a result GIF, it's fun!

<img width="630" alt="image" src="https://user-images.githubusercontent.com/3998421/217264163-c3c912bf-1007-4189-b8b2-21e72bad8507.png">


Windows Installer v2

https://lama-cleaner-docs.vercel.app/install/windows_1click_installer

Remove the `tasks.py` file, and add a [gradio](https://github.com/gradio-app/gradio) based web config page to help the user set command line args. Click the `Save configurations` button will save an `installer_config.json` file in the same directory and the `win_start.bat` script will load this JSON file when starting the server.

<img width="1583" alt="web_config_page" src="https://user-images.githubusercontent.com/3998421/214059654-a3b59ffc-d792-445f-9627-5ea51bf7e383.png">

New command line args
`--no-gui-auto-close`: when starting as a desktop app, the backend server will auto close, if the app is closed, but some users may have bug like https://github.com/Sanster/lama-cleaner/issues/188 . If it happens to you, this arg can prevent the backend auto close after the GUI window is closed.

docker-0.33.0
New GPU docker image, update Cuda and pytorch version, add xformers. With `--enable-xformers`, SD model get 30%~40% performance improvement

- cuda: 11.7
- pytorch: 1.13.1
- minimum nvidia driver: 515.43.04+
- xformers: 0.0.16rc425


bash
docker run --gpus all -p 8080:8080 \
-v /path/to/cache_dir:/root/.cache/ \
--rm cwq1913/lama-cleaner:gpu-0.33.0 \
lama-cleaner --device=cuda --port=8080 --host=0.0.0.0 --enable-xformers

Optimize GPU Memory

Correctly implement `--cpu-offload`, test result for 512x512 image:

Stable Diffusion Model
- fp16: 2GB
- `--no-half`: 3GB

Paint by Example Model
- fp16: 2.6GB
- --no-half: 4.5GB



Document: https://lama-cleaner-docs.vercel.app/memory

- fix image EXIF rotation bug: https://github.com/Sanster/lama-cleaner/issues/185

Lama Cleaner Docs

Docs website: https://lama-cleaner-docs.vercel.app/

<img width="1447" alt="image" src="https://user-images.githubusercontent.com/3998421/212474996-6c6a7d5c-33a9-4d56-a207-d5f951a80591.png">

New command line arg

To solve the most frequently asked question: how to change the model directory

`--model-dir`: set model download directory

Happy new year 🎆 ! I just recovered from COVID and it's been really hard to be sick, hope everyone is doing well.

File Manager

https://user-images.githubusercontent.com/3998421/211202244-57093aef-7d76-4ce6-91e0-c63cb0211755.mov

I realize that some users will use this tool to process lots of photos and I thought a FileManager would be helpful. Users can search images by name, and browser images in both the image directory and the output directory. To enable the FileManager, set two command line args when starting Lama Cleaner.

- `--input`: a directory contains images you want to clean up
- `--output-dir`: The directory where you want to save the resulting image when clicking the download button on the web page, the resulting image will download to the output directory. **A folder will be created in this directory to store the thumbnails**

A folder icon will appear on the left-top corner, click this icon to open FileManager, or use shortcut `f`.

<img width="180" alt="image" src="https://user-images.githubusercontent.com/3998421/211227320-29073a0b-98a2-42c0-879f-c2b36a82e223.png">


SD/PaintByExample Image Resize

https://user-images.githubusercontent.com/3998421/211202340-4ac928ff-23fe-49a2-9b4b-dea31c16bd99.mov

Resize the image before inpainting, then scale up the inpainting area to fill the origin area, the area outside the mask will not lose quality. This option can be used to save vRAM.

cpu_offload

new command line arg: `--cpu-offload` for sd/paint_by_example model, offloads all models to CPU, reducing vRAM usage.

According to [diffusers's document](https://huggingface.co/docs/diffusers/optimization/fp16#offloading-to-cpu-with-accelerate-for-memory-savings), vRAM can be reduced to 2GB, but that doesn't seem to be the case [in my tests](https://github.com/huggingface/diffusers/issues/1934), you can try it yourself.