You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey everyone, thank you for the awesome work at creating these examples.
I was following the llama 2 deployment tutorial and I would like to suggest a few improvements for an easier flow:
Indicate where should we run this example.
At the download section it is indicated to perform it at a Notebook session, however the rclone tool is not available at that enviroment. I managed to download it and set it up using conda install rclone and using this documentation.
During the image build/push section we are required to use Docker. However Docker is also not available at the notebook environment. I was not able to install it at the notebook environment and ended up doing it locally.
As a newcomer, changing between environments gave me the impression I was performing those operations wrong. A hint here would be greatly appreciated.
Indicate where to add policies. Another newcomer issue, but it is unclear where should we add the policies. The given documentation also exemplifies types of policies, but not where should we add them. I think that a direction or link would help here.
During the build section the initial part is common to the TGI and vLLM builds. People going for the vLLM build might miss this necessary introductory part. Perhaps different section name?
Hopefully thats helps others when starting to use OCI through this example. Thanks!
The text was updated successfully, but these errors were encountered:
Hey everyone, thank you for the awesome work at creating these examples.
I was following the llama 2 deployment tutorial and I would like to suggest a few improvements for an easier flow:
conda install rclone
and using this documentation.As a newcomer, changing between environments gave me the impression I was performing those operations wrong. A hint here would be greatly appreciated.
Indicate where to add policies. Another newcomer issue, but it is unclear where should we add the policies. The given documentation also exemplifies types of policies, but not where should we add them. I think that a direction or link would help here.
During the build section the initial part is common to the TGI and vLLM builds. People going for the vLLM build might miss this necessary introductory part. Perhaps different section name?
Hopefully thats helps others when starting to use OCI through this example. Thanks!
The text was updated successfully, but these errors were encountered: