Support multiple initial programs

[0x0f0f0f]

Closes #55

[0x0f0f0f]

We should discuss the positioning of arguments

The MR currently gives the openevolve-run.py CLI this usage:

usage: openevolve-run.py [-h] [--config CONFIG] [--output OUTPUT] [--iterations ITERATIONS] [--target-score TARGET_SCORE]
                         [--log-level {DEBUG,INFO,WARNING,ERROR,CRITICAL}] [--checkpoint CHECKPOINT] [--api-base API_BASE] [--primary-model PRIMARY_MODEL]
                         [--secondary-model SECONDARY_MODEL] [--initial-programs-dir INITIAL_PROGRAMS_DIR]
                         evaluation_file [initial_program]

Before, initial_program was the first positional argument. I've swapped them, and also added [--initial-programs-dir INITIAL_PROGRAMS_DIR].

It would probably best, if instead of passing the directory, the first argument stayed the evaluator, and then we require one or more initial programs as positional arguments, so users can do something like python /path/to/openevolve-run.py evaluator.py initial_program1.py initial_program2.py other_programs_dir/*.py

[0x0f0f0f]

Updated in latest version

[codelion]

How about we keep the existing ordered args for backwards compatibility, in addition we can add a --initial-programs argument that can take either a directory or a list of paths.

[0x0f0f0f]

This was what I attempted in the previous commit, and the logic for args parsing was not looking very good. With the current state, we can definitely do python openevolve-run.py evaluator.py prog1.py prog2.py ./a/b/c/*.py and all other bash goodies. Having positional arguments be initial_program.py evaluator.py and then --initial_programs dir/*.py then causes some issues. One has to always provide one initial program, and cannot omit the first positional argument, and the behavior/ux of defining both initial_program argument and --initial_programs together is not really clear.

I would go for the breaking change if possible! (It also keeps things very simple on OpenEvolve side, no directory listing, etc...)

[codelion]

Do you have an example where multiple initial programs would be required? If the idea is to evolve like an entire codebase at once, then just having a folder path instead of file path as first argument should be sufficient. We can then use the config.yaml to control what file types or other things are in scope of evolution v.s. out of scope similar to what was suggested in #111

[codelion]

If it is about just assigning different initial programs to different islands, we can do it via the config.yaml We can have one initial program.py as we do and in teh config.yaml with the island configs we can also add path to other initial programs which can be initialized to the islands. Will need to ensure that the config has sufficient islands so it may be better to define them at the same place in the config itself.

[0x0f0f0f]

Will need to ensure that the config has sufficient islands

This is done in this PR. After initializing config, if number of islands is < number of programs then it throws. Very simple check

[0x0f0f0f]

It's done when loading the programs

[0x0f0f0f]

I also thought about that, but it might also introduce additional logic. I don't really like the idea, because having inputs defined in a config feels like violating the UNIX philosophy

              | Config           
              v                  
        +----------------+       
Input(s)|                | Output
 ------>|   Program      +------>
        |                |       
        +----------------+       

[xyltt]

Do you have an example where multiple initial programs would be required? If the idea is to evolve like an entire codebase at once, then just having a folder path instead of file path as first argument should be sufficient. We can then use the config.yaml to control what file types or other things are in scope of evolution v.s. out of scope similar to what was suggested in #111

The current code does not support inputting the codebase path. Is there a plan to support evolving the entire codebase—similar to the "API" mentioned in AlphaEvolve?

[AMindToThink]

Hi @0x0f0f0f @codelion, is there any update on whether this feature will be added to OpenEvolve? Starting with multiple insightful human-designed programs would likely result in better results in many situations, including my use-case.