article: Optimizing GPU Utilization in Containerized Environments #38
Conversation
varshith257
force-pushed
the
add-llm
branch
3 times, most recently
from
28abb6b to
d4d430d
Compare
|
@nkkko Just opening it as a draft as I have worked on almost everything except a few and need to test this through daytona env and manually it is working. It is blocked by daytonaio/daytona#940 Once this issue is fixed, I will push my code to the repository of this project by testing it thoroughly and documenting the challenges encountered. I have been actively following up in Slack with @Tpuljak on this. |
articles/20240820_optimizing-gpu-utilization-in-containerized-environments.md
Show resolved
Hide resolved
|
@varshith257 this PR fails DCO checks. Equally please give an update. |
|
@mojafa Sure! Will update with the same |
|
@varshith257 any updates? |
|
@varshith257 your PR fails DCO checks |
varshith257
force-pushed
the
add-llm
branch
2 times, most recently
from
6c61e3c to
e96dd84
Compare
Signed-off-by: Vamshi Maskuri <[email protected]>
There was a problem hiding this comment.
@varshith257 Please create a sample project that addresses the issue and use the article to guide us through how you made your project run.
Add screenshots and be very detailed.
Avoid AI generated content.
|
|
||
| ## Introduction | ||
|
|
||
| With the rapid adoption of deep learning and large language models (LLMs), the need for efficient GPU utilization in containerized environments has become more pressing than ever. Containers provide a lightweight, portable solution for deploying applications, but when it comes to GPU-intensive tasks like LLM inference or fine-tuning, careful setup and configuration are crucial to achieving optimal performance. |
There was a problem hiding this comment.
@varshith257 this sounds really AI generated
|
|
||
| # | ||
|
|
||
| Let's adjust the draft to align it more closely with the suggested format: |
There was a problem hiding this comment.
ChatGPT...😅
Please review this whole article again.
|
|
||
| Example configuration: | ||
|
|
||
| ```plaintext |
There was a problem hiding this comment.
no need for plaintext, just add ascreenshot of this step
| Install the NVIDIA Container Toolkit to enable GPU support in Docker: | ||
|
|
||
| ```bash | ||
| sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit |
There was a problem hiding this comment.
I'm using MacOs, I don't think this will work locally or even in devcontainer.
| Daytona manages the initialization and configuration of the workspace, displaying progress information in the terminal. Once completed, the workspace will be ready for use. Example output during workspace creation: | ||
|
|
||
| ```plaintext | ||
| WORKSPACE | ✓ Request submitted |
There was a problem hiding this comment.
add as a screenshot please
|
|
||
| ### Create a Dockerfile for nanoGPT | ||
|
|
||
| Next, create a Dockerfile tailored to run nanoGPT with GPU support. Here’s an example Dockerfile: |
There was a problem hiding this comment.
I suggest you create a project, push to github, do daytona create your-repo-url, and share screenshots or steps to replicate yout project.
you can easily define the dependencies or packages devconatiner.json file.
|
you can check out this PR did there's step by step: #117 |
Pull Request Template
Closes #8
/claim #8
Writer's Checklist
Follow Writing Structure
code elementswhere appropriate.Fact-Check
Assets
/assetsfolder.YYYYMMDD_title_of_the_article_IMG_NAME_NO.png.Interlinking
CTRL+Fto search for relevant keywords on:Glossary/Definitions
Review and Edit