components manager

Author: stevhliu

docs/source/en/_toctree.yml

@@ -129,7 +129,7 @@
   - local: modular_diffusers/modular_pipeline
     title: ModularPipeline
   - local: modular_diffusers/components_manager
-    title: Components Manager
+    title: ComponentsManager
   - local: modular_diffusers/guiders
     title: Guiders
 

docs/source/en/api/modular_diffusers/pipeline_components.md

@@ -10,4 +10,8 @@
 
 ## ComponentsManager
 
-[[autodoc]] diffusers.modular_pipelines.components_manager.ComponentsManager
+[[autodoc]] diffusers.modular_pipelines.components_manager.ComponentsManager
+
+## InsertableDict
+
+[[autodoc]] diffusers.modular_pipelines.modular_pipeline_utils.InsertableDict

docs/source/en/modular_diffusers/auto_pipeline_blocks.md

@@ -16,7 +16,7 @@ specific language governing permissions and limitations under the License.
 
 This guide shows how to create [`~modular_pipelines.AutoPipelineBlocks`].
 
-Create three [``~modular_pipelines.PipelineBlocks`] for text-to-image, image-to-image, and inpainting. These represent the different workflows available in the pipeline.
+Create three [`~modular_pipelines.PipelineBlock`] for text-to-image, image-to-image, and inpainting. These represent the different workflows available in the pipeline.
 
 <hfoptions id="auto">
 <hfoption id="text-to-image">
@@ -155,14 +155,14 @@ class AutoImageBlocks(AutoPipelineBlocks):
 
 It is **very** important to include a `description` to avoid any confusion over how to run a block and what inputs are required. While [`~modular_pipelines.AutoPipelineBlocks`] are convenient, it's conditional logic may be difficult to figure out if it isn't properly explained.
 
-Create an instance of `AutoImageBlocks` and use [`~modular_pipelines.ModularPipeline.init_pipeline`] to convert it to a pipeline.
+Create an instance of `AutoImageBlocks` and use [`~modular_pipelines.ModularPipelineBlocks.init_pipeline`] to convert it to a pipeline.
 
 ```py
 auto_blocks = AutoImageBlocks()
 auto_pipeline = auto_blocks.init_pipeline()
 ```
 
-For more complex compositions, nested [`~modular_pipelines.AutoPipelineBlocks`] blocks when they're used as sub-blocks in larger pipelines, use the [`~modular_pipelines.PipelineBlocks.get_execution_blocks`] method to extract the a block that is actually run based on your input.
+For more complex compositions, nested [`~modular_pipelines.AutoPipelineBlocks`] blocks when they're used as sub-blocks in larger pipelines, use the [`~modular_pipelines.SequentialPipelineBlocks.get_execution_blocks`] method to extract the a block that is actually run based on your input.
 
 ```py
 auto_blocks.get_execution_blocks("mask")

docs/source/en/modular_diffusers/components_manager.md

@@ -12,7 +12,7 @@ specific language governing permissions and limitations under the License.
 
 # LoopSequentialPipelineBlocks
 
-[`~modular_pipelines.LoopSequentialPipelineBlocks`] are a multi-block type that composes other [`~modular_pipelines.PipelineBlocks`] together in a loop. Data flows circularly, using `intermediate_inputs` and `intermediate_outputs`, and each block is run iteratively. This is typically used to create a denoising loop which is iterative by default.
+[`~modular_pipelines.LoopSequentialPipelineBlocks`] are a multi-block type that composes other [`~modular_pipelines.PipelineBlock`] together in a loop. Data flows circularly, using `intermediate_inputs` and `intermediate_outputs`, and each block is run iteratively. This is typically used to create a denoising loop which is iterative by default.
 
 This guide shows you how to create [`~modular_pipelines.LoopSequentialPipelineBlocks`].
 
@@ -80,13 +80,13 @@ class LoopBlock(PipelineBlock):
 
 ## LoopSequentialPipelineBlocks
 
-Use the [~modular_pipelines.LoopSequentialPipelineBlocks.from_blocks_dict`] method to add the loop block to the loop wrapper to create [~modular_pipelines.LoopSequentialPipelineBlocks`].
+Use the [`~modular_pipelines.LoopSequentialPipelineBlocks.from_blocks_dict`] method to add the loop block to the loop wrapper to create [`~modular_pipelines.LoopSequentialPipelineBlocks`].
 
 ```py
 loop = LoopWrapper.from_blocks_dict({"block1": LoopBlock})
 ```
 
-Add more loop blocks to run within each iteration with [~modular_pipelines.LoopSequentialPipelineBlocks.from_blocks_dict`]. This allows you to modify the blocks without changing the loop logic itself.
+Add more loop blocks to run within each iteration with [`~modular_pipelines.LoopSequentialPipelineBlocks.from_blocks_dict`]. This allows you to modify the blocks without changing the loop logic itself.
 
 ```py
 loop = LoopWrapper.from_blocks_dict({"block1": LoopBlock(), "block2": LoopBlock})

docs/source/en/modular_diffusers/loop_sequential_pipeline_blocks.md

@@ -72,7 +72,7 @@ def __call__(self, components, state):
 
 ## State interaction
 
-[`~modular_pipelines.PipelineState`] and [`BlockState`] interaction is defined by a block's `inputs`, `intermediate_inputs`, and `intermediate_outputs`.
+[`~modular_pipelines.PipelineState`] and [`~modular_pipelines.BlockState`] interaction is defined by a block's `inputs`, `intermediate_inputs`, and `intermediate_outputs`.
 
 - `inputs`, a block can modify an input - like `block_state.image` - but the change is local to the [`~modular_pipelines.BlockState`] and won't affect the original image in [`~modular_pipelines.PipelineState`].
 - `intermediate_inputs`, is often values created from a previous block. When a block modifies `intermediate_inputs` - like `batch_size` - this change is reflected in both the [`~modular_pipelines.BlockState`] and [`~modular_pipelines.PipelineState`]. Any subsequent blocks are also affected.

docs/source/en/modular_diffusers/modular_diffusers_states.md

@@ -12,7 +12,7 @@ specific language governing permissions and limitations under the License.
 
 # ModularPipeline
 
-[`ModularPipeline`] converts [`PipelineBlock`]'s into an executable pipeline that loads models and performs the computation steps defined in a block. It is the main interface for users to run a pipeline and it is very similar to the [`DiffusionPipeline`] API.
+[`ModularPipeline`] converts [`~modular_pipelines.PipelineBlock`]'s into an executable pipeline that loads models and performs the computation steps defined in a block. It is the main interface for users to run a pipeline and it is very similar to the [`DiffusionPipeline`] API.
 
 The main difference is to include an expected `output` argument in the pipeline.
 
@@ -92,6 +92,37 @@ image.save("moduar_inpaint_out.png")
 
 This guide will show you how to create a [`ModularPipeline`] and manage the components in it.
 
+## Adding blocks
+
+Blocks are [`InsertableDict`] objects that can be inserted at specific positions, providing a flexible way to mix-and-match blocks.
+
+Use [`~modular_pipelines.modular_pipeline_utils.InsertableDict.insert`] on either the block class or `sub_blocks` attribute to add a block.
+
+```py
+# BLOCKS is dict of block classes, you need to add class to it
+BLOCKS.insert("block_name", BlockClass, index)
+# sub_blocks attribute contains instance, add a block instance to the  attribute
+t2i_blocks.sub_blocks.insert("block_name", block_instance, index)
+```
+
+Use [`~modular_pipelines.modular_pipeline_utils.InsertableDict.pop`] on either the block class or `sub_blocks` attribute to remove a block.
+
+```py
+# remove a block class from preset
+BLOCKS.pop("text_encoder")
+# split out a block instance on its own
+text_encoder_block = t2i_blocks.sub_blocks.pop("text_encoder")
+```
+
+Swap blocks by setting the existing block to the new block.
+
+```py
+# Replace block class in preset
+BLOCKS["prepare_latents"] = CustomPrepareLatents
+# Replace in sub_blocks attribute using an block instance
+t2i_blocks.sub_blocks["prepare_latents"] = CustomPrepareLatents()
+```
+
 ## Creating a pipeline
 
 There are two ways to create a [`ModularPipeline`]. Assemble and create a pipeline from [`PipelineBlocks`] or load an existing pipeline with [`~ModularPipeline.from_pretrained`].
@@ -104,7 +135,7 @@ You should also initialize a [`ComponentsManager`] to handle device placement an
 <hfoptions id="create">
 <hfoption id="PipelineBlocks">
 
-Use the [`ModularPipelineBlocks.init_pipeline`] method to create a [`ModularPipeline`] from the component and configuration specifications. This method loads the *specifications* from a `modular_model_index.json` file, but it doesn't load the *models* yet.
+Use the [`~ModularPipelineBlocks.init_pipeline`] method to create a [`ModularPipeline`] from the component and configuration specifications. This method loads the *specifications* from a `modular_model_index.json` file, but it doesn't load the *models* yet.
 
 ```py
 from diffusers import ComponentsManager

docs/source/en/modular_diffusers/modular_pipeline.md

@@ -35,6 +35,6 @@ The Modular Diffusers docs are organized as shown below.
 
 - [States](./modular_diffusers_states) explains how data is shared and communicated between pipeline blocks and [`ModularPipeline`].
 - [PipelineBlock](./pipeline_block) is the most basic unit of a [`ModularPipeline`] and this guide shows you how to create one.
-- [SequentialPipelineBlocks](./sequential_pipeline_blocks) is a type of block that chains multiple blocks so they run one after another, passing data along the chain. This guide shows you how to create [`SequentialPipelineBlocks`] and how they connect and work together.
-- [LoopSequentialPipelineBlocks](./loop_sequential_pipeline_blocks) is a type of block that runs a series of blocks in a loop. This guide shows you how to create [`LoopSequentialPipelineBlocks`].
-- [AutoPipelineBlocks](./auto_pipeline_blocks) is a type of block that automatically chooses which blocks to run based on the input. This guide shows you how to create [`AutoPipelineBlocks`].
+- [SequentialPipelineBlocks](./sequential_pipeline_blocks) is a type of block that chains multiple blocks so they run one after another, passing data along the chain. This guide shows you how to create [`~modular_pipelines.SequentialPipelineBlocks`] and how they connect and work together.
+- [LoopSequentialPipelineBlocks](./loop_sequential_pipeline_blocks) is a type of block that runs a series of blocks in a loop. This guide shows you how to create [`~modular_pipelines.LoopSequentialPipelineBlocks`].
+- [AutoPipelineBlocks](./auto_pipeline_blocks) is a type of block that automatically chooses which blocks to run based on the input. This guide shows you how to create [`~modular_pipelines.AutoPipelineBlocks`].

docs/source/en/modular_diffusers/overview.md

@@ -12,11 +12,11 @@ specific language governing permissions and limitations under the License.
 
 # SequentialPipelineBlocks
 
-[`~modular_pipelines.SequentialPipelineBlocks`] are a multi-block type that composes other [`~modular_pipelines.PipelineBlocks`] together in a sequence. Data flows linearly from one block to the next using `intermediate_inputs` and `intermediate_outputs`. Each block in [`~modular_pipelines.SequentialPipelineBlocks`] usually represents a step in the pipeline, and by combining them, you gradually build a pipeline.
+[`~modular_pipelines.SequentialPipelineBlocks`] are a multi-block type that composes other [`~modular_pipelines.PipelineBlock`] together in a sequence. Data flows linearly from one block to the next using `intermediate_inputs` and `intermediate_outputs`. Each block in [`~modular_pipelines.SequentialPipelineBlocks`] usually represents a step in the pipeline, and by combining them, you gradually build a pipeline.
 
 This guide shows you how to connect two blocks into a [`~modular_pipelines.SequentialPipelineBlocks`].
 
-Create two [`~modular_pipelines.PipelineBlocks`]. The first block, `InputBlock`, outputs a `batch_size` value and the second block, `ImageEncoderBlock` uses `batch_size` as `intermediate_inputs`.
+Create two [`~modular_pipelines.PipelineBlock`]. The first block, `InputBlock`, outputs a `batch_size` value and the second block, `ImageEncoderBlock` uses `batch_size` as `intermediate_inputs`.
 
 <hfoptions id="sequential">
 <hfoption id="InputBlock">