MemComposer is a system that generates B-roll sequences from a user's video collection given a natural language directive and a target duration. It first builds an entity-centric memory from the raw footage offline, indexing characters, settings, objects, and style. Then it uses this memory to plan the sequence and retrieve conditioning frames for each shot. Finally, it iteratively generates and critiques shots to maintain identity, setting, and sequence consistency.
In a user preference study, MemComposer outperformed an ungrounded text-to-video planner, winning 60% of prompt-adherence comparisons and 92.8% of visual-alignment comparisons. Against retrieval-only sequences assembled from captured footage, MemComposer won 94.5% of prompt-adherence comparisons, though retrieval-only was preferred for visual alignment in 58.2% of comparisons. This shows that generating missing shots with grounding in the collection's visual world is valuable, especially for adhering to the user's directive.