Generating blended images together with fg and bg is helpful for structural understanding in our very recent tests. Note that different from paper, this model file includes an additional "blended lora", and it actually can generate three images together (fg, bg, and blended image). It includes two rank-256 loras (foreground lora and background lora), and an attention sharing module to share attention between multiple diffusion processes on par. layer_sd15_joint.safetensors This model file allows for generating all layers together with SD1.5.It will change the latent distribution of the model to a "transparent latent space" that can be decoded by the special VAE pipeline. ![]() layer_sd15_transparent_attn.safetensors This is a rank-256 LoRA to turn a SD1.5 into a transparent image generator.layer_sd15_vae_transparent_decoder.safetensors Same as above VAE decoder, but fine-tuned for SD1.5.layer_sd15_vae_transparent_encoder.safetensors Same as above VAE encoder, but fine-tuned for SD1.5.I have made sure that the reduced parameters does not influence result quality. The model architecture is also more lightweight than the paper version to reduce VRAM requirement. vae_transparent_decoder.safetensors This is an image decoder that takes SD VAE outputs and latent image as inputs, and outputs a real PNG image.The released model is more light weighted, requires much less vram, and does not influence result quality in my tests. Note that in the paper we used a relatively heavy model with exactly same amount of parameters as the SD VAE. The offset can be added to latent images to help the diffusion of transparency. vae_transparent_encoder.safetensors This is an image encoder to extract a latent offset from pixel space.layer_xl_bgble2fg.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on backgrounds and blended compositions, and generates foregrounds.layer_xl_bg2ble.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on backgrounds, and generates blended compositions.layer_xl_fgble2bg.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on foregrounds and blended compositions, and generates backgrounds.layer_xl_fg2ble.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on foregrounds, and generates blended compositions.Also, this model may introduce a strong style influence to the base model. This layer_xl_transparent_conv.safetensors is still included for some special use cases that needs special prompt understanding. However, in practice, I find the layer_xl_transparent_attn.safetensors will lead to better results. Because we excluded the offset training of any q,k,v layers, the prompt understanding of SDXL should be perfectly preserved. These offsets can be merged to any XL model to change the latent distribution to transparent images. This safetensors file includes an offset of all conv layers (and actually, all layers that are not q,k,v of any attention layers). layer_xl_transparent_conv.safetensors This is an alternative model to turn your SDXL into a transparent image generator. ![]()
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |