Inference

[SahPet]

Great code - worked with some modifications.

1. Will you be adding any inference code?

2. The readme says the roles should be:

"[
{"conversations": [ { "from": "human", "value": "<image>\nWhat skill set might someone need to perform such a frisbee trick?" }, { "from": "gpt", "value": "To perfo...

However, the default in the llava_conversation_lib.py code seems to be:

default_conversation = conv_vicuna_v1

... which has this setup with roles=("USER", "ASSISTANT") and not "human" and "gpt":

conv_vicuna_v1 = Conversation(
    system="A chat between a curious user and an artificial intelligence assistant. "
    "The assistant gives helpful, detailed, and polite answers to the user's questions.",
    roles=("USER", "ASSISTANT"),
    version="v1",
    messages=(),
    offset=0,
    sep_style=SeparatorStyle.TWO,
    sep=" ",
    sep2="</s>",
)

Not sure how much that matters, but should there be another default with "human" and "gpt" or perhaps should the readme say "USER", "ASSISTANT", or will the LLM understand this anyway?