https://youtu.be/Xa9Sg-j62xY

Artstation: https://www.artstation.com/artwork/eR41JJ

Quickstart: https://github.com/bghira/SimpleTuner/blob/main/documentation/quickstart/LTXVIDEO.md

Release notes: https://github.com/bghira/SimpleTuner/releases/tag/v1.3.0

Abstract

Style transfer involves transferring the style from a reference image to the content of a target image. Recent advancements in LoRA-based (Low-Rank Adaptation) methods have shown promise in effectively capturing the style of a single image. However, these approaches still face significant challenges such as content inconsistency, style misalignment, and content leakage. In this paper, we comprehensively analyze the limitations of the standard diffusion parameterization, which learns to predict noise, in the context of style transfer. To address these issues, we introduce ConsisLoRA, a LoRA-based method that enhances both content and style consistency by optimizing the LoRA weights to predict the original image rather than noise. We also propose a two-step training strategy that decouples the learning of content and style from the reference image. To effectively capture both the global structure and local details of the content image, we introduce a stepwise loss transition strategy. Additionally, we present an inference guidance method that enables continuous control over content and style strengths during inference. Through both qualitative and quantitative evaluations, our method demonstrates significant improvements in content and style consistency while effectively reducing content leakage.

Paper: https://arxiv.org/abs/2503.10614

Code: https://github.com/000linlin/ConsisLoRA (coming soon)

Project Page: https://consislora.github.io/

FP8 Model: https://huggingface.co/Kijai/HunyuanVideo_comfy/tree/main

ComfyUI nodes (updated wrapper): https://github.com/kijai/ComfyUI-HunyuanVideoWrapper

Example ComfyUI workflow: https://github.com/kijai/ComfyUI-HunyuanVideoWrapper/blob/main/example_workflows/hyvideo_i2v_example_01.json

Discord Server: https://discord.com/invite/7tsKMCbNFC

Model Description

TensorArt-TurboX-SD3.5Large is a highly optimized variant of Stable Diffusion 3.5 Large, achieving 6x faster generation speed with minimal quality loss. It surpasses the official Stable Diffusion 3.5 Large Turbo in image detail, diversity, richness, and realism.

Repo: https://github.com/nygaard91/Pixel-Perfect-AI-Art-Converter

AI-generated pixel art aren’t truly pixel perfect, too oversized for game assets, or otherwise unsuitable for professional use—this tool provides a practical, manual approach that can better meet your needs.

Abstract

Recent advancements in text-to-image generative systems have been largely driven by diffusion models. However, single-stage text-to-image diffusion models still face challenges, in terms of computational efficiency and the refinement of image details. To tackle the issue, we propose CogView3, an innovative cascaded framework that enhances the performance of text-to-image diffusion. CogView3 is the first model implementing relay diffusion in the realm of text-to-image generation, executing the task by first creating low-resolution images and subsequently applying relay-based super-resolution. This methodology not only results in competitive text-to-image outputs but also greatly reduces both training and inference costs. Our experimental results demonstrate that CogView3 outperforms SDXL, the current state-of-the-art open-source text-to-image diffusion model, by 77.0% in human evaluations, all while requiring only about 1/2 of the inference time. The distilled variant of CogView3 achieves comparable performance while only utilizing 1/10 of the inference time by SDXL.

Paper: https://arxiv.org/abs/2403.05121

Code: https://github.com/THUDM/CogView4

Demo: https://huggingface.co/spaces/THUDM-HF-SPACE/CogView4

Model Weights: https://huggingface.co/THUDM/CogView4-6B/tree/main

Abstract

In 3D modeling, designers often use an existing 3D model as a reference to create new ones. This practice has inspired the development of Phidias, a novel generative model that uses diffusion for reference-augmented 3D generation. Given an image, our method leverages a retrieved or user-provided 3D reference model to guide the generation process, thereby enhancing the generation quality, generalization ability, and controllability. Our model integrates three key components: 1) meta-ControlNet that dynamically modulates the conditioning strength, 2) dynamic reference routing that mitigates misalignment between the input image and 3D reference, and 3) self-reference augmentations that enable self-supervised training with a progressive curriculum. Collectively, these designs result in a clear improvement over existing methods. Phidias establishes a unified framework for 3D generation using text, image, and 3D conditions with versatile applications.

Paper: https://arxiv.org/abs/2409.11406

Code: https://github.com/3DTopia/Phidias-Diffusion

Models: https://huggingface.co/ZhenweiWang/Phidias-Diffusion/tree/main

Project Page: https://rag-3d.github.io/

Abstract

Multi-layer image generation is a fundamental task that enables users to isolate, select, and edit specific image layers, thereby revolutionizing interactions with generative models. In this paper, we introduce the Anonymous Region Transformer (ART), which facilitates the direct generation of variable multi-layer transparent images based on a global text prompt and an anonymous region layout. Inspired by Schema theory suggests that knowledge is organized in frameworks (schemas) that enable people to interpret and learn from new information by linking it to prior knowledge.}, this anonymous region layout allows the generative model to autonomously determine which set of visual tokens should align with which text tokens, which is in contrast to the previously dominant semantic layout for the image generation task. In addition, the layer-wise region crop mechanism, which only selects the visual tokens belonging to each anonymous region, significantly reduces attention computation costs and enables the efficient generation of images with numerous distinct layers (e.g., 50+). When compared to the full attention approach, our method is over 12 times faster and exhibits fewer layer conflicts. Furthermore, we propose a high-quality multi-layer transparent image autoencoder that supports the direct encoding and decoding of the transparency of variable multi-layer images in a joint manner. By enabling precise control and scalable layer generation, ART establishes a new paradigm for interactive content creation.

Paper: https://arxiv.org/abs/2502.18364

Code: https://github.com/microsoft/art-msra

Demo: http://20.65.136.27:8060/

Project Page: https://art-msra.github.io/

Abstract

Diffusion models (DMs) have become the leading choice for generative tasks across diverse domains. However, their reliance on multiple sequential forward passes significantly limits real-time performance. Previous acceleration methods have primarily focused on reducing the number of sampling steps or reusing intermediate results, failing to leverage variations across spatial regions within the image due to the constraints of convolutional U-Net structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in handling variable number of tokens, we introduce RAS, a novel, training-free sampling strategy that dynamically assigns different sampling ratios to regions within an image based on the focus of the DiT model. Our key observation is that during each sampling step, the model concentrates on semantically meaningful regions, and these areas of focus exhibit strong continuity across consecutive steps. Leveraging this insight, RAS updates only the regions currently in focus, while other regions are updated using cached noise from the previous step. The model's focus is determined based on the output from the preceding step, capitalizing on the temporal consistency we observed. We evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up to 2.36x and 2.51x, respectively, with minimal degradation in generation quality. Additionally, a user study reveals that RAS delivers comparable qualities under human evaluation while achieving a 1.6x speedup. Our approach makes a significant step towards more efficient diffusion transformers, enhancing their potential for real-time applications.

Paper: https://arxiv.org/abs/2502.10389

Paper Summary: https://www.aimodels.fyi/papers/arxiv/region-adaptive-sampling-diffusion-transformers

Code: https://github.com/microsoft/RAS

Project Page: https://microsoft.github.io/RAS/

"This feature is launching first for Gold-tier Civitai Subscribers as we refine and improve its functionality" - so they're most likely planning to expand it to all users at some point.

Pretty disappointed with the direction CiviatAI is going.