Skip Initial Bootstrap Without Demonstrations in Random Search Optimizer

[Nasreddine]

When using the bootstrap with the random search optimizer, the first round always executes without demonstrations. This results in an initial output of 0 in my case.

Is there a way to skip this initial phase and start directly with demonstrations to avoid unnecessary use of the LLM.

[okhat]

Thanks @Nasreddine ! That first call is there to ensure that running the optimizer doesn't lower performance compared to your baseline. (Btw if the first round is scoring 0.0, I would consider doing some more fundamental change to the system or program, since it might struggle to bootstrap. There are many ways around that.)

[Nasreddine]

I have a specific case where my program cannot bootstrap without demos. I’m trying to produce valid XML and use assertions to ensure that the generated XML aligns with the schema. Unfortunately, no LLM was able to recognize the schema, so there’s no need for this initial phase. Therefore, I suggested making this phase optional (using a parameter).

P.S. In the latest releases, assertions were removed (code commented), which affected my program. As you’re preparing for a new version, where can I find information about these changes?

[okhat]

Yes, assertions v1 will be gone in 2.6, and will be replaced with something new that represents v2 of assertions when 2.6.0 comes out.

It will be easier to use and much more powerful. Stay tuned within a week from now.