v4.2.7 includes gallery improvements and some major features focused on upscaling.
Upscaling
We've added a dedicated upscaling tab, support for custom upscaling models, and some new nodes.
Thanks to RyanJDick (backend implementation), chainchompa (frontend) and maryhipp (frontend) for working on this!
Dedicated Upscaling Tab
The new upscaling tab provides a simple and powerful UI to Invoke's `MultiDiffusion` implementation. This builds on the workflow released in v4.2.6, allowing for memory-efficient upscaling to huge output image sizes.
<video src="https://github.com/user-attachments/assets/574edaaa-0ad7-445f-bab2-adedef5ca474"></video>
We're pretty happy with the results!
<img width="744" alt="image" src="https://github.com/user-attachments/assets/a83c4cb5-a8bd-4457-ada9-3d92969eae3c">
> _4x scale, `4x_NMKD-Siax_200k` upscale model, `Deliberate_v5` SD1.5 model, `KDPM 2 scheduler 30 steps`, all other settings default_
Requirements
You need 3 models installed to use this feature:
- An **upscale model** for the first pass upscale
- A **main SD model** (SD1.5 or SDXL) for the image-to-image
- A **tile ControlNet model** of the same model architecture as your main SD model
If you are missing any of these, you'll see a warning directing you to the model manager to install them. You can search the starter models for `upscale`, `main`, and `tile` to get you started.
Tips
- The main SD model architecture has the biggest impact on VRAM usage. For example, SD1.5 2k needs just under 4GB, while SDXL 2k needs just under 9GB. VRAM usage increases a small amount as output size increases - SD1.5 8k needs ~4.5GB while SDXL 8k needs ~10.5GB.
- The upscale and main SD model choices matter. Choose models best suited to your input image or desired output characteristics.
- Some schedulers work better than others. `KDPM 2` is a good choice.
- LoRAs - like a detail-adding LoRA - can make a big impact.
- Higher `Creativity` values give the SD model more leeway in creating new details. This parameter controls denoising start and end percentages.
- Higher `Structure` values tell the SD model to stick closer to the input image's structure. This parameter controls the tile ControlNet.
Custom Upscaling Models
You can now install and use custom upscaling models in Invoke. The excellent [`spandrel`](https://github.com/chaiNNer-org/spandrel/) library handles loading and running the models.
<video src="https://github.com/user-attachments/assets/7d6b99c3-f7f1-486c-af06-efb24bd34efd"></video>
`spandrel` can do a lot more than upscaling - it supports a wide range of "image to image" models. This includes single-image super resolution like ESRGAN (upscalers) but also things like GFPGAN (face restoration) and DeJPEG (cleans up JPEG compression artifacts).
A complete list of supported architectures can be found [here](https://github.com/chaiNNer-org/spandrel/?tab=readme-ov-file#model-architecture-support).
> Note: We have _not_ enabled the restrictively-licensed architectures, which are denoted with a `+` symbol in the list.
Installing Models
We've added a few popular upscaling models to the Starter Models tab in the Model Manager - search for "upscale" to find them.
<img width="708" alt="image" src="https://github.com/user-attachments/assets/f54eba24-76b8-49b3-af63-7cb927fafcf1">
You can install models found online via the Model Manager, just like any other model. [OpenModelDB](https://openmodeldb.info/) is a popular place to get these models. For most of them, you can copy the model's download link and paste in into the Model Manager to install.
Nodes
Two nodes have been added to support processing images with `spandrel` - be that upscaling or any of the other tasks these models support.
<img width="697" alt="image" src="https://github.com/user-attachments/assets/04a5de1c-84b9-46ae-88b7-1cc601166021">
- `Image-to-Image` - Runs the selected model without any extra processing.
- `Image-to-Image (Autoscale)` - Runs the selected model repeatedly until the desired scale is reached. This node is intended for upscaling models specifically, providing some useful extra functionality:
- If the model overshoots the target scale, the final image will be downscaled to the target scale with Lanczos resampling.
- As a convenience, the output image width and height can be fit to a multiple of 8, as is required for SD. This will only resize down, and may change the aspect ratio slightly.
- If the model doesn't actually upscale the image, the `scale` parameter will be ignored.
Gallery Improvements
Thanks to maryhipp and chainchompa for continued iteration on the gallery!
- Cleaner boards UI.
- Improved boards and image search UI.
- Fixed issues where board counts don't update when images are moved between boards.
- Added a "Jump" button to allow you to skip pages of the gallery
<video src=https://github.com/user-attachments/assets/b834cc36-995a-464e-af3f-68cf3b38818f></video>
Other Changes
- Enhancement: When installing starter models, the description is carried over. Thanks lstein!
- Enhancement: Updated translations.
- Fix: Model unpatching when running on CPU, causing bad/no outputs.
- Fix: Occasional visible seams on images with smooth textures, like skies. `MultiDiffusion` tiling now uses gradient blending to mitigate this issue.
- Fix: Model names overflow the model selection drop-downs.
- Internal: Backend SD pipeline refactor (WIP). This will allow contributors to add functionality to Invoke more easily. This will be behind a feature flag until the refactor is complete and tested. Thanks to StAlKeR7779 for leading the effort, with major contributions from dunkeroni and RyanJDick.
Installation and Updating
To install or update to v4.2.7post1, [download the installer](https://github.com/invoke-ai/InvokeAI/releases/download/v4.2.7post1/InvokeAI-installer-v4.2.7post1.zip) and follow the [installation instructions](https://invoke-ai.github.io/InvokeAI/installation/010_INSTALL_AUTOMATED/).
To update, select the same installation location. Your user data (images, models, etc) will be retained.
Missing models after updating from v3 to v4
See [this FAQ](https://invoke-ai.github.io/InvokeAI/help/FAQ/#missing-models-after-updating-to-v4).
Error during installation `ModuleNotFoundError: No module named 'controlnet_aux'`
See [this FAQ](https://invoke-ai.github.io/InvokeAI/help/FAQ/#modulenotfounderror-no-module-named-controlnet_aux)
What's Changed
* fix(backend): revert non-blocking device transfer by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6624
* chore: bump version to 4.2.6post1 by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6625
* Add support for Spandrel Image-to-Image models (e.g. ESRGAN, Real-ESRGAN, Swin-IR, DAT, etc.) by RyanJDick in https://github.com/invoke-ai/InvokeAI/pull/6556
* Add tiling to SpandrelImageToImageInvocation by RyanJDick in https://github.com/invoke-ai/InvokeAI/pull/6594
* Add Spandrel upscale starter models by RyanJDick in https://github.com/invoke-ai/InvokeAI/pull/6605
* Fix model unpatching process when running on CPU by lstein in https://github.com/invoke-ai/InvokeAI/pull/6631
* Add gradient blending to tile seams in MultiDiffusion by RyanJDick in https://github.com/invoke-ai/InvokeAI/pull/6635
* Base of modular backend by StAlKeR7779 in https://github.com/invoke-ai/InvokeAI/pull/6606
* add sdxl tile to starter models by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6634
* Fix function call that we forgot to update in 6606 by RyanJDick in https://github.com/invoke-ai/InvokeAI/pull/6636
* Bump fastapi-events dependency by ebr in https://github.com/invoke-ai/InvokeAI/pull/6644
* fix: update uncategorized board totals when deleting and moving images by chainchompa in https://github.com/invoke-ai/InvokeAI/pull/6646
* feat(ui): upscaling tab by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6633
* Math Updates by hipsterusername in https://github.com/invoke-ai/InvokeAI/pull/6648
* Modular backend - add rescale cfg by StAlKeR7779 in https://github.com/invoke-ai/InvokeAI/pull/6640
* Modular backend - add FreeU by StAlKeR7779 in https://github.com/invoke-ai/InvokeAI/pull/6641
* Modular backend - add ControlNet by StAlKeR7779 in https://github.com/invoke-ai/InvokeAI/pull/6642
* feat(ui): add upsells for pro edition to settings menu by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6650
* Update Simple Upscale Button to work with spandrel models by chainchompa in https://github.com/invoke-ai/InvokeAI/pull/6649
* Simple upscale bugfixes by chainchompa in https://github.com/invoke-ai/InvokeAI/pull/6655
* fix(ui): settings menu layout by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6654
* [MM2] Use typed ModelRecordChanges for model_install() rather than untyped dict by lstein in https://github.com/invoke-ai/InvokeAI/pull/6645
* fix(ui): restore pnpm-lock.yaml by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6659
* feat(ui): upscaling & ad-hoc post-processing misc by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6658
* fix(ui): model select overflowing when model names are too long by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6660
* feat(ui): more gallery UX updates by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6652
* fix(ui): few cases where board totals don't updated when moving by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6665
* ui: translations update from weblate by weblate in https://github.com/invoke-ai/InvokeAI/pull/6653
* chore: bump version v4.2.7rc1 by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6670
* feat(ui): add jump to option for gallery pagination by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6668
* feat(ui): info popovers on by default by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6673
* fix(ui): copied edges must have new ids set by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6676
* fix(backend): add refiner to single-file `load_classes` by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6675
* fix(ui): model manager goes blank by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6679
* feat(ui): upscale folllow-ups by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6671
* feat(ui): update qr code cnet starter model by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6680
* feat(app): update default workflows by psychedelicious in https://github.com/invoke-ai/InvokeAI/pull/6681
* bump version to 4.2.7 by maryhipp in https://github.com/invoke-ai/InvokeAI/pull/6687
* Update docker image to use pnpm version 8 by brandonrising in https://github.com/invoke-ai/InvokeAI/pull/6689
**Full Changelog**: https://github.com/invoke-ai/InvokeAI/compare/v4.2.6...v4.2.7post1