✨ Add Recursive Compile

[MFA-X-AI]

Description

This PR introduces a recursive compile feature for workflows that contain workflow components (i.e., a workflow within a workflow). Previously, users had to manually navigate through each nested workflow component and compile them one by one, which is be tedious especially for deep workflows. With this update, recursive compilation is now automated.

Key Updates:

1. Backend / Jupyter Server Backend:

   • A new recursive compile function is added. This function parses the .xircuits file, detects workflow components using the condition if extras.get('type') == "xircuits_workflow", and recursively compiles them.
   • This function extends the existing compile function to handle nested workflows seamlessly.
   • A new backend endpoint is created to handle the recursive compilation request.

2. Frontend (JupyterLab):

   • All compile actions in JupyterLab now default to the new recursive compile endpoint.
   • The user experience remains the same, except for updated compile icons in the toolbar.

3. CLI (Command Line Interface):
   • Users calling xircuits compile workflow.xircuits from the CLI will invoke the recursive compile function by default.
   • An optional --no-recursive flag is added for users who prefer to compile only the top-level workflow without its components.

Pull Request Type

Please check the box that most accurately describes the type of changes in this PR:

• Xircuits Core (JupyterLab-related changes)
• Xircuits Canvas (Custom RD-related changes)
• Xircuits Component Library
• Xircuits Project Template
• Testing Automation
• Documentation
• Others (Xircuits CLI)

Type of Change

Select the appropriate change type:

• New feature (non-breaking change that adds functionality)
• Bug fix (non-breaking change that fixes an issue)
• Breaking change (a feature or fix that might affect existing functionality)
• This change requires a documentation update

Tests

Please follow these steps to verify the changes:

1. Set up a workflow containing nested workflow components:

   • Use a workflow component from the control flow library or create a new workflow with embedded workflows.
   • Compile the new workflow and verify that it works as expected.

2. Front-end Tests:

   • Verify that the recursive compile works in the JupyterLab frontend. Compile the workflow with components and check that all the nested workflows are compiled successfully.
   • Run the workflow to ensure the output is the same as before.
   • Check the updated compile icons in the toolbar.

3. CLI Tests:

   • Run the xircuits compile workflow.xircuits command and confirm that the recursive compile works.
   • Test the --no-recursive flag to ensure only the top-level workflow is compiled.

Tested on:

Please indicate the platforms where these changes were tested:

• Windows
• Linux Ubuntu
• CentOS
• macOS
• Others (Please Specify)

[github-actions]

👈 Launch a binder notebook on branch XpressAI/xircuits/fahreza/recursive-compile