Introduction and Explanation of Custom DIY ControlNet:
In the Custom DIY feature, users can freely choose which ControlNet to use. ControlNet serves to add constraints to the generated video.
In Custom DIY, we offer a total of 5 types of ControlNet. Each ControlNet provides a strength control ranging from 0 to 1, with a precision of 0.01. By combining different ControlNets and their strengths, we can achieve satisfactory video effects. Below, we will introduce the function of each ControlNet one by one.
-
Pose
Function Explanation:
Because we generally want to maintain consistency with the character poses and movements in the original video during video generation, it is recommended to enable Pose in most cases and set its strength to 0.85 to 1. Additionally, our Pose feature includes a “pose exclude face” switch. When enabled, it turns off facial detection, which is necessary for effects like plants dancing where facial details need to be ignored.
Usage Recommendation:
Since we usually want the generated video to match the poses and movements of the characters in the original video, it is recommended to enable Pose in most cases and set the intensity to 0.85～1.
-
Depth
Function Explanation:
Depth can extract the depth relationships of characters, objects, and people from the original video and impose these constraints on the generated video.
Usage Recommendation:
It is recommended to enable Depth when character actions overlap with the body, such as when two arms cross each other. Depth can accurately depict front and back relationships. Additionally, if there are buildings in the background, Depth can be enabled. It is recommended to start with 0.5 intensity, giving the AI some room to perform.
-
Tile
Function Explanation:
Tile ignores some details in the original video and then regenerates these details, resulting in a new video. Therefore, using Tile can restore non-detailed parts of the original image to some extent.
Usage Recommendation:
When you want to partially restore the original video, you can use Tile. 0.5～0.7 intensity is recommended.
-
LineArt
Function Explanation:
LineArt can identify key lines in the original video and impose these constraints on the new video.
Usage Recommendation:
If you need to refer to the lines in the original image, you can use LineArt. Additionally, if there are texts in the original video that you want to retain, you can use LineArt. 0.5～0.7 intensity is recommended. It is not recommended to set LineArt intensity to 0.85～1, as it may overly constrain the image and limit the AI's ability to create stylized outputs. Note: Do not use LineArt and SoftEdge simultaneously.
-
SoftEdge
Function Explanation:
Similar to LineArt, SoftEdge extracts lines from the original video but with a difference. SoftEdge can only recognize the general outline details of the image, with softer lines. This provides more room for the AI to perform. Compared to LineArt, SoftEdge imposes weaker constraints on the identified lines, giving the AI a rough outline to draw.
Usage Recommendation:
When we only want to reference the general outline of the original video and allow the AI more creative space in generating the video, we can use SoftEdge.
-
QRCode
Function Description:
QRCode focuses the style and effects more on the characters.
Usage Recommendation:
When using Effect or some reference images, you can use QRCode. It will make your style and effects more centered on the characters.
Simple Style Conversion Configuration Instructions:
Additionally, in our simple style conversion, most styles use Lineart with 0.5, Tile with 0.5 intensity and Pose with 0.85 intensity ControlNet constraints.
So, when you cannot achieve satisfactory results using the simple options, it is recommended to use Custom DIY for customized control. Once you fully understand the function of each ControlNet, you will find that selecting different ControlNets for different videos can yield excellent results. For example, if you think the stability of your video is insufficient, you can increase the strength of Tile and LineArt.