myBywaysthe blog of C.Y. Wong2024-03-28T00:00:00+00:00C.Y. Wonghttps://www.mybyways.com/blogLLM Prompt Generation with Ollama in ComfyUIhttps://www.mybyways.com/blog/llm-prompt-generation-with-ollama-in-comfyui2024-03-28T00:00:00+00:002024-03-28T00:00:00+00:00
In my last post, I described running Mistral, a Large Language Model, locally using Ollama. To accompany that piece, I created a prompt and manually used AI to generate an image. Today, I’ll wire up a ComfyUI workflow to Ollama to do this seamlessly, thanks to ComfyUI-IF_AI_tools.
Read more...
]]>
A game with Mistral 7B using Ollamahttps://www.mybyways.com/blog/a-game-with-mistral-7b-using-ollama2024-03-20T00:00:00+00:002024-03-20T00:00:00+00:00
I keep posting about Stable Diffusion, but I do experiment with Large Language Models too! I do not have much to contribute in this regard, instead, here is the transcript of a game I played with the open source Mistral 7B model via Ollama.
Read more...
]]>
Stable Video Diffusionhttps://www.mybyways.com/blog/stable-video-diffusion2024-03-10T00:00:00+00:002024-03-10T00:00:00+00:00
More and more AI generated images are shared as short video clips. So, here a quick test of Stable Video Diffusion - which was released back in November last year. Don’t know why I didn’t post this when I posted about AnimateDiff and the Hotshot Motion model around the same time.
Read more...
]]>
TripoSR image-to-3D-objecthttps://www.mybyways.com/blog/triposr-image-to-3d-object2024-03-09T00:00:00+00:002024-03-09T00:00:00+00:00
Do you want to convert a 2D image into a 3D model auto-magically? On 5 March 2024, Stability AI and Tripo AI released TripoSR: Fast 3D Object Generation from Single Images that does exactly that!
Read more...
]]>
Differential Diffusion for in-paintinghttps://www.mybyways.com/blog/differential-diffusion-for-in-painting2024-03-05T00:00:00+00:002024-03-05T00:00:00+00:00Differential Diffusion is the newest method (framework) of in-painting without an in-painting model. Instead, all that is needed is a mask (map) where the lighter the area, the greater the re-painting applied.
Read more...
]]>
Generate transparent images with Layer Diffusionhttps://www.mybyways.com/blog/generate-transparent-images-with-layer-diffusion2024-03-03T00:00:00+00:002024-03-03T00:00:00+00:00
Ever wished you could generate Stable Diffusion XL images with transparent backgrounds? Well, your wish has been answered by the smart people behind the Transparent Image Layer Diffusion using Latent Transparency paper. They have made their code and models available, and what do you know, Chenlei Hu has ported it to ComfyUI!
Read more...
]]>
SDXL-based 4-step models comparedhttps://www.mybyways.com/blog/sdxl-based-4-step-models-compared2024-03-02T00:00:00+00:002024-03-02T00:00:00+00:00
With the advent of techniques like Adversarial Diffusion Distillation and Latent Consistency models, A.I. image synthesis based on Stable Diffusion XL has been getting faster and faster. Here is just quick comparison of a few models at 4-steps, some of which are fine-tuned and trained for realism.
Read more...
]]>
Consistent portraits revisisted: InstantIDhttps://www.mybyways.com/blog/consistent-portraits-revisisted-instantid2024-02-25T00:00:00+00:002024-02-25T00:00:00+00:00
Not long ago, in a attempt to obtain Consistent portraits using IP-Adapters for SDXL, I shared a comparison between IP-Adapter-Plus-Face and IP-Adapter-FaceID. Today I’ll look at InstantID.
Read more...
]]>
New Stable Cascase Checkpoints for ComfyUIhttps://www.mybyways.com/blog/new-stable-cascase-checkpoints-for-comfyui2024-02-24T00:00:00+00:002024-02-24T00:00:00+00:00
An update to my previous post on Stable Cascade with ComfyUI - instead of requiring four separate model files, we now only need two checkpoints, and the ComfyUI workflow is now very straightfoward!
Read more...
]]>
Stable Cascade with ComfyUIhttps://www.mybyways.com/blog/stable-cascade-with-comfyui2024-02-18T00:00:00+00:002024-02-18T00:00:00+00:00
On 12 Feb 2024, Stability.ai released Stable Cascade “research preview” (non-commercial license), and over the weekend, ComfyUI was updated to support this new model! Time to give it a go!
Read more...
]]>
Convert animated WebP to WebM movie with Pythonhttps://www.mybyways.com/blog/convert-animated-webp-to-webm-movie-with-python2024-01-21T00:00:00+00:002024-01-21T00:00:00+00:00
In my last post I mentioned converting an animated WebP image format into a WebM movie format. This post expands on how I did it.
Read more...
]]>
Comparing face IP-Adapters for SDXLhttps://www.mybyways.com/blog/comparing-face-ip-adapters-for-sdxl2024-01-14T00:00:00+00:002024-01-14T00:00:00+00:00
As a follow up to my last post regarding Consistent portraits using IP-Adapters for SDXL, this is a short comparison of the two face IP-Adapters for SDXL by h94 / xiaohu: namely, ip-adapter-plus-face_sdxl_vit-h.bin and ip-adapter-faceid_sdxl.bin.
Read more...
]]>
Consistent portraits using IP-Adapters for SDXLhttps://www.mybyways.com/blog/consistent-portraits-using-ip-adapters-for-sdxl2024-01-07T00:00:00+00:002024-01-07T00:00:00+00:00
Getting consistent character portraits generated by SDXL has been a challenge... until now! ComfyUI IPAdapter Plus (dated 30 Dec 2023) now supports both IP-Adapter and IP-Adapter-FaceID (released 4 Jan 2024)!
Read more...
]]>
Extracting images from PDF fileshttps://www.mybyways.com/blog/extracting-images-from-pdf-files2024-01-06T00:00:00+00:002024-01-06T00:00:00+00:00
Happy new year!
Ever needed to extract images from PDFs and found both on-line and off-line tools lacking? Well, I certainly have, and here I present my Python code to extract JPGs/PNGs from PDFs, using PyMuPDF.
Read more...
]]>
Go even faster with SDXL-Turbo!https://www.mybyways.com/blog/go-even-faster-with-sdxl-turbo2023-11-29T00:00:00+00:002023-11-29T00:00:00+00:00
In the span of a couple of weeks, we got Crazy fast image generation with LCM LoRA for SDXL, which led me to ask if I could get Faster Stable Diffusion on M-series macs?. A few hours ago, Stability.ai gave us their response in the form of SDXL-Turbo... and now we go even faster!
Read more...
]]>
Faster Stable Diffusion on M-series macs?https://www.mybyways.com/blog/faster-stable-diffusion-on-mseries-macs2023-11-27T00:00:00+00:002023-11-27T00:00:00+00:00
All my recent Stable Diffusion XL experiments have been on my Windows PC instead of my M2 mac, because it has a faster Nvidia 2060 GPU with more memory. But today, I’m curious to see how much faster diffusion has gotten on a M-series mac (M2 specifically).
Read more...
]]>
QR Code Monster for SDXLhttps://www.mybyways.com/blog/qr-code-monster-for-sdxl2023-11-14T00:00:00+00:002023-11-14T00:00:00+00:00
Time to try another ControlNet for Stable Diffusion XL - QR Code Monster v1 in ComfyUI. This ControlNet can influence SDXL such that the generated image “hides” a scan-able QR code, which at first glance, looks like a photo!
Read more...
]]>
Generating animations / videos with SDXL LCMhttps://www.mybyways.com/blog/generating-animations-videos-with-sdxl-lcm2023-11-13T00:00:00+00:002023-11-13T00:00:00+00:00
I never tried generating video clips or animations with SDXL before, simply because my GPU is not powerful enough. But after testing out the LCM LoRA for SDXL yesterday, I thought I’d try the SDXL LCM LoRA with Hotshot-XL, which is something akin to AnimateDiff.
Read more...
]]>
Crazy fast image generation with LCM LoRA for SDXLhttps://www.mybyways.com/blog/crazy-fast-image-generation-with-lcm-lora-for-sdxl2023-11-12T00:00:00+00:002023-11-12T00:00:00+00:00
Stable Diffusion keeps improving at an astounding pace! This time, it’s the idea of distilling a model into a Latent Consistency Model (LCM) for very, very fast image generation with a quality trade-off. On 24 Oct 2023, the distilled Segmind Stable Diffusion 1B (SSD-1B) model was released, followed by a better implementation in the form of Latent Consistency LoRAs for SDXL and SDD-1B released on 9 Nov 2023.
Read more...
]]>
Fix Web Background Videos That Do Not Auto-play in Safari on iOShttps://www.mybyways.com/blog/fix-web-background-videos-that-do-not-auto-play-in-safari-on-ios2023-09-10T00:00:00+00:002023-09-10T00:00:00+00:00
I recently encountered a situation where Safari would not play background videos in web pages! I initially assumed the page was broken... until I realized mine was the only iPhone with this behaviour, yet I am on the latest iPhone and iOS version. I finally figured what was different with my setup, and how to solve this issue.
Read more...
]]>