In VMagic’s Custom DIY, we offer two types of Draw Modes:
- Refer Video to Redraw
- Origin Video Turn Video Style
Here, we will briefly outline these two modes, with more comprehensive documentation to follow.
Refer Video to Redraw
This mode involves redrawing the original video on a blank sheet of paper.
The specific content to be referenced depends on the chosen ControlNet (CN). The functions of each ControlNet have been introduced in a previous article, which you can refer to for more details.
This mode is suitable for scenarios where you want to reference certain aspects of the original video. A typical example is the recent trend of dancing plant videos, where the plants only mimic the poses and movements from the original video, making this mode ideal for such cases.
In this mode, we provide character reference images and background reference images. Users can upload the corresponding reference images for drawing.
Prompts are crucial in this mode because the AI needs to know what to draw anew. If the ControlNet strength is not high and no character or background reference images are uploaded, a detailed prompt describing the video content is required.
Origin Video Turn Video Style
This mode involves processing the original video. Simply put, the input video is Image 1, and the drawing will be done on Image 2. This mode naturally retains some information from the input video.
This mode is suitable for scenarios where you want to preserve the content of the original video, such as converting dance styles or movie styles.
In this mode, prompts can be used to specify changes, such as hair color or facial expressions. However, be careful not to conflict with ControlNet settings. For example, when specifying hair color, do not set the tile ControlNet strength too high (since the tile will reference the original video color). Similarly, when specifying facial expressions, do not set the lineArt strength too high (since lineArt will reference the original video lines).
Lastly, the differences between the two modes are mainly in how they handle ControlNet with the same settings. There is no strict boundary between them. The degree to which the original video is retained depends not only on the draw mode but also significantly on the choice and strength of the ControlNet.