FreeStyle: Style and Content Dual Reference Image Generation via Community LoRA Mining

FreeStyle is an open source image generation system published in June 2026 by researchers at Fudan University, StepFun, Westlake University, and the University of Hong Kong. It takes two reference images alongside a text prompt: one image supplies the visual style, and a second supplies the content subject. The model generates a new image that adopts the style of the first reference while depicting the subject of the second.

The system is released under the Apache 2.0 license, permitting commercial use. Model weights and code are publicly available.

Dual Reference Generation

Standard image generation models accept a text prompt. Style transfer models typically accept one reference image. FreeStyle accepts two. A style reference defines the visual language: the color palette, brushwork, rendering quality, or photographic treatment the output should match. A content reference defines what the image should depict: a character, an object, a scene composition.

Style Leakage and Content Leakage: The Core Problem

Style leakage occurs when a model meant to apply a style reference also pulls content elements from that reference into the output. A style reference showing a painting of a forest can cause the output to include trees the user never asked for, simply because the model treats style and content as one signal rather than two. Content leakage is the reverse: elements of the content reference bleed into the style, producing outputs where the rendering quality fluctuates inconsistently across the image.

Both failure modes appear in models that use a single conditioning pathway for both inputs. FreeStyle's architecture maintains separate encoders for the style and content references, then combines their signals at a stage where the model has already established independent representations for each. That separation is the technical mechanism behind the reported improvement in style consistency.

Mining the Community's LoRA Library

The distinguishing technical choice in FreeStyle is how it represents styles and content categories. Instead of training on a fixed proprietary dataset, the system mines the open source community's existing LoRA weight collections. Each LoRA functions as a clustering center for a style or a content concept. The model learns to use these weights as anchors, allowing it to compose style and content conditioning from the vocabulary the community has already built and published.

How the LoRA Mining Pipeline Works

LoRA mining in FreeStyle is not a retrieval step that happens at inference time. It is a training technique. During training, the model was exposed to a large collection of community LoRA weights and learned to use their internal representations as style and content anchors. At inference time, the user provides reference images, not LoRA identifiers. The model maps those reference images onto the learned representations it acquired from the LoRA library.

The practical result is that FreeStyle can generalize to visual styles it has seen in LoRA form during training, even when a user provides a photograph or artwork as the style reference rather than a LoRA weight file. The community LoRA library functioned as a labeled catalog of styles during training, and that knowledge is encoded in the model weights that are now publicly available on HuggingFace.

The FreeStyle_Bench Evaluation Dataset

The research team released FreeStyle_Bench alongside the model weights, providing a structured evaluation dataset for comparing future methods against the same conditions. Having a dedicated benchmark matters for the research community because it replaces ad hoc comparisons with a common test set that any group can use.

FreeStyle_Bench covers multiple style categories and content subject types, allowing evaluation across both style preservation quality and content fidelity independently. The dataset is available under the same Apache 2.0 terms as the model weights, meaning other research groups can publish results on FreeStyle_Bench without licensing restrictions.

Performance and How to Access It

The paper, published as arXiv preprint 2606.20506, reports that FreeStyle outperforms baseline methods on style preservation metrics in ablation studies across both the style-only (SRef) and dual reference (SRef + CRef) task settings. The FreeStyle_Bench dataset, also publicly released, provides a structured evaluation set for comparing future methods against the same conditions.

The system supports two operational modes. Style reference only (SRef) applies a source style to a text prompted generation. Combined style and content reference (CRef + SRef) adds the second conditioning signal for subject control. Weights are available through HuggingFace under the Blue2Giant organization, with separate checkpoints for each mode. The corresponding authors are Gang Yu (Westlake University) and Chi Zhang (University of Hong Kong).

The Two Modes in Practice

The SRef mode is the simpler entry point. It takes a style reference image and a text prompt, and produces output that matches the style while depicting what the prompt describes. For concept art workflows, this allows an art director to establish a visual grammar from one reference image and generate multiple scenes in that grammar without assembling a content reference for each one.

The CRef + SRef mode adds a second reference for subject control. It is the appropriate mode when the output must depict a specific character, object, or face rather than a generic subject. A cinematographer establishing a look for a specific character across multiple shot types would use CRef + SRef, holding the content reference constant while varying style references to test different rendering approaches for the same subject.

Hardware and Runtime Requirements

FreeStyle runs on standard consumer hardware. The model weights total approximately 4GB, which fits within the VRAM budget of an NVIDIA RTX 3080 or equivalent card. Inference time for a single image at standard resolution runs under 30 seconds on that hardware configuration.

The HuggingFace repository includes a Gradio interface for local testing, meaning users who want to evaluate the model before integrating it into a pipeline can run it through a browser on their own machine. The code repository also documents the inference pipeline in enough detail to integrate FreeStyle into custom workflows without modification of the model weights.

The Research Team's Institutional Context

FreeStyle comes from a four institution collaboration. Fudan University led the project, with StepFun contributing industrial AI research resources, Westlake University providing one of the corresponding authors, and the University of Hong Kong contributing a second. That kind of collaboration is common in Chinese AI research, where university and industry researchers routinely co-author papers.

StepFun's involvement is notable because the company operates as an AI research lab with a focus on generative models, meaning FreeStyle had access to the kind of compute and data resources typically available to well-funded industrial labs rather than academic groups alone. The resulting model reflects that resource access. The LoRA mining approach requires exposure to a large, diverse collection of community weights during training.

Where FreeStyle Sits in the 2026 Open Source Image Pipeline

FreeStyle addresses a specific gap in the open source image generation workflow. Models like FLUX and SDXL generate high quality images from text prompts but do not provide a dedicated pathway for holding both style and content constant across multiple generations. IP-Adapter and ControlNet add reference conditioning to diffusion models but were not designed with the LoRA mining approach that gives FreeStyle its style vocabulary.

For production teams building visual consistency workflows, FreeStyle fills the dual reference role that has not had a clean open source solution. It complements rather than replaces base generation models, and the Apache 2.0 license means studios can integrate it into proprietary pipelines without licensing negotiations.

What the Apache 2.0 License Enables

The Apache 2.0 license on FreeStyle is a specific choice with practical consequences for production use. It permits commercial use without restriction, modification and redistribution with attribution, and integration into proprietary software without requiring the integrating software to be open sourced. A studio that builds FreeStyle into a proprietary visual development pipeline is not required to release its pipeline code or to share any royalty with the original authors.

That permissiveness is deliberate on the research team's part. Open commercial licensing drives adoption, which produces citations and follow-up work from the research community, which strengthens the lab's research standing. The commercial production community benefits from tools it can use without legal risk. The research community benefits from real-world usage data that informs the next generation of work.

Comparison With Existing Style Methods

The open source alternatives to FreeStyle for style-content generation include IP-Adapter, ControlNet based approaches, and the style conditioning features built into SDXL and FLUX based checkpoints. IP-Adapter applies a reference image as a conditioning signal but does not separate style and content into independent pathways, which produces the content leakage that FreeStyle is designed to avoid. ControlNet adds structural conditioning but targets spatial layout rather than visual style.

FreeStyle does not replace any of these tools. It addresses a specific limitation they share: the inability to simultaneously hold a style reference and a content reference in separate conditioning channels. For production teams that already use IP-Adapter or ControlNet for other tasks, FreeStyle adds a capability rather than competing with existing tools in their workflow.

The Ablation Studies and What They Show

The paper's ablation studies test each component of the FreeStyle architecture by removing or replacing it and measuring the change in output quality. Removing the separate content conditioning pathway produces style leakage. Removing the LoRA mining component and replacing it with generic CLIP text conditioning produces weaker style adherence. The combination of both components together produces the results the paper reports as state-of-the-art on FreeStyle_Bench.

Ablation results tell researchers and practitioners what is actually doing the work in a model. For FreeStyle, the ablation results confirm that both the separate conditioning pathways and the LoRA mining approach contribute independently to the output quality. A team that wants to replicate or build on FreeStyle's results needs both components, not just one. The public availability of both the model weights and the evaluation dataset on FreeStyle_Bench means any group can verify the ablation claims independently.

Accessing the Weights

Model weights are available on HuggingFace under the Blue2Giant organization, with separate checkpoints for the SRef mode and the CRef + SRef mode. The GitHub repository includes inference scripts and documentation for both modes, along with the Gradio local demo interface. The total download size for both checkpoints is under 10GB, which fits within standard storage budgets for a production team evaluating the model.

The HuggingFace Space for FreeStyle provides a browser-based interface for running inference without a local GPU. That entry point is appropriate for a first evaluation of whether the model handles the specific visual styles a production needs. Local deployment makes sense once a team has confirmed FreeStyle produces usable output for their workflow requirements.

Production Applications

For filmmakers, FreeStyle addresses a specific problem in the visual development phase, specifically generating reference images that faithfully match an established visual style while depicting new subjects. A cinematographer can supply a frame from an established visual grammar as the style reference and a character concept as the content reference, generating consistent production imagery without manual compositing.

The same workflow applies to concept art and production design. An art director can iterate rapidly across a scene's visual treatment while holding the character reference constant, or hold the visual treatment constant while testing new subjects. For mood board work, the model can generate multiple framings in a consistent style from a single style reference image. Filmmakers working with AI FILMS Studio's image workspace and building on open source image generation models like HiDream O1 can add FreeStyle's style and content conditioning as a complementary local tool in their pipeline.

The LoRA mining approach also connects to FLUX.2 and the broader ecosystem of open image generation models. As the community produces new LoRAs trained on specific visual treatments and director styles, those LoRAs become usable as FreeStyle style references immediately.

---

Sources

arXiv: FreeStyle: Free Control of Style Content Generation from Community LoRA Mining GitHub: Blue2Giant/FreeStyle HuggingFace: Blue2Giant/FreeStyle_Checkpoint Project Page: blue2giant.github.io/FreeStyle