Improvement: Increasing Token Usage Efficiency (In Progress)

[kc0tlh]

Background
Large language models (LLMs) decode text through tokens—frequent character sequences within text/code. Under the hood Bolt.new is powered mostly by Anthropic's Sonnet 3.5 AI model, so using Bolt consumes tokens that we must purchase from Anthropic.

Our goal is for Bolt to use as few tokens as possible to accomplish each task, and here's why: 1) AI model tokens are one of our largest expenses and if less are used, we save money, 2) so that users can get more done with Bolt and become fans/advocates, and 3) ultimately so we can attract more users and continue investing in improving the platform!

When users interact with Bolt, tokens are consumed in 3 primary ways: chat messages between the user and the LLM, the LLM writing code, and the LLM reading the existing code to capture any changes made by the user.

There are numerous product changes that we are working on to increase token usage efficiency, and in the meantime there are many tips and tricks you can implement in your workflow to be more token efficient:

Upcoming Improvements
Optimizing token usage is a high priority for our team, and we are actively exploring several R&D initiatives aimed at improving token usage efficiency automatically behind the scenes. In the meantime, we will be shipping multiple features that improve the user experience in the near term including controlling which files it is able to modify via locking and targeting (shipped) and improving the automated debugging feature (shipped). These improvements, paired with the tips below should help you manage your tokens more efficiently. Subscribe to this issue to be notified when those new features land.

While we work on these improvements, here are some strategies you can use to maximize token usage efficiency today:

• Avoid Repeated Automated Error "Fix" Attempts
  Continuously clicking the automatic "fix" button can lead to unnecessary token consumption. After each attempt, review the result and refine your next request if needed. There are programming challenges that the AI cannot solve automatically, so it is a good idea to do some research and intervene manually if it fails to fix automatically.

• Add Error Handling To Your Project
  If you find yourself stuck in an error loop, a useful strategy is to prompt the AI to enhance error handling and implement detailed logging throughout the problematic area. The AI excels at inserting robust error logs, even at a granular level, such as between functions or key steps. These logs provide valuable feedback that the AI can use to better understand the root cause of the issue. The additional logging provides more precise information when the error occurs again. With this detailed feedback, the AI can make more accurate adjustments to fix the issue. Credit to @Frankg40 for this one!

• Leverage the Rollback Functionality
  Use the rollback feature to revert your project to a previous state without consuming tokens. This is essentially and undo button that can take you back to any prior state of your project, This can save time and tokens if something goes wrong with your project. Keep in mind that there is no "redo" function though, so be sure you want to revert before using this feature because it is final: all changes made after the rollback point will be permanently removed.

• Crawl, Walk, Run
  Make sure the basics of your app are scaffolded before describing the details of more advanced functionality for your site.

• Use Specific and Focused Prompts
  When prompting the AI, be clear and specific. Direct the model to focus on certain files or functions rather than the entire codebase, which can improve token usage efficiency. This approach is not a magic fix, but anecdotally we've seen evidence that it helps. Some specific prompting strategies that other users have reported as helpful are below, and a ton more can be found in the comment thread below:

If you have specific technologies you want to use (IE Astro, Tailwind, ShadCN), say that in your initial prompt

Mention Specific Code Segments or Classes: When possible, refer to specific divs, classes, or functions to guide Bolt to the exact place where you want the changes made. You can do this manually in chat or by highlighting the relevant code in your project and using the "Ask Bolt" functionality.

Use the Prompt Enhancer function: The better the prompt, the higher quality of the output — bolt․new can help you improve your prompts automatically with the prompt enhancement feature!

Be Specific About What Should Remain Unchanged: Mention explicitly that no modifications should occur to other parts of the site.

Batch multiple simple to explain instructions into one message. For example you can ask Bolt to change the color scheme, add mobile responsiveness, and restart the dev server safely all in one message.

• Understand Project Size Impact
  As your project grows, more tokens are required to keep the AI in sync with your code. Larger projects (and longer chat conversations) demand more resources for the AI to stay aware of the context, so it's important to be mindful of how project size impacts token usage.

• Advanced Strategy: Reset the AI Context Window
  If the AI seems stuck or unresponsive to commands, consider refreshing the Bolt.new chat page in your browser. This resets the LLM’s context window, clears out prior chat messages, and reloads your code in a fresh chat session. This will clear the chat, so you will need to remind the AI of any context not already captured in the code, but it can help the AI regain focus when it is overwhelmed due to the context window being full.

We appreciate your patience during this beta period and look forward to updating this thread as we ship new functionality and improvements to increase token usage efficiency!

[kc0tlh]

Checkout our latest X thread outlining some of the upcomming dev priorities related to error fixing and token efficiency, specifically:

breaking error loops
controlling what the AI can & cannot edit
a "require approval before AI changes code" mode
& reducing avg token usage per message

All landing over this week & next, ETAs for each included in thread!

[gboross]

Hello,

How can I connect two separate WebContainers running frontend and backend services on StackBlitz?

As a follow-up to my previous question - if I run the frontend and backend in separate WebContainers:

How can I configure CORS and proxy settings to allow communication between them?
What's the correct way to reference the backend URL from the frontend when both are running in different StackBlitz WebContainers?
Is there a way to share environment variables or configuration between the containers?
I'm using React (Vite) for frontend and Python Flask for backend. Any examples or best practices would be greatly appreciated!

[dukuo]

Is there any update on this? I'm blocked on a project, and I've actually deleted a bunch of files to continue with the most critical for MVP, but I can't even reduce the prompt nor the feature set of my app to continue using Bolt.

That being said, I'm super invested in what you are doing, and I wanted to take the time to say thank you for creating this incredible leap in development speed!

[jennyjanetech]

I created a collapsible menu + few troubleshooting it cost me 500,000 tokens for that simple FEature! . you should review how Bolt consumes token

[Frankg40]

@kc0tlh I have a pretty good approach to this one:

Avoid Repeated Automated Error "Fix" Attempts

Continuously clicking the automatic "fix" button can lead to unnecessary token consumption. After each attempt, review the result and refine your next request if needed. There are programming challenges that the AI cannot solve automatically, so it is a good idea to do some research and intervene manually if it fails to fix automatically.

Tell it to add more error handling and robust logging in every step of the problem area. Often enough, the AI does not get enough feedback from the problem that it doesn't know what to fix. What it IS really good at, is adding error logging. It will add error logs in between every single function if it has to. Eventually when the same error triggers again, it will give the AI much more accurate reasoning as to why the error is happening, then it knows what to fix, and does.

This is working pretty much all the time for me when I get stuck in an error loop.

[Behnod]

Hello everyone,

I’m facing an issue. When I create an app and develop it, then try to open it on Stackblitz, the project does not appear as developed in Bolt.new. Instead, only the default Next.js template files are displayed on Stackblitz.

In summary, the chat history and the developed project are not being saved on Bolt.new. Everything is removed after reloading the page. Additionally, all the tokens I’ve used are gone once the project is cleared after reloading the Chrome browser page.

I would greatly appreciate it if someone could provide a solution to fix this critical issue.

Thank you very much to the entire community.

[prisda]

Aider.chat is another useful AI assistant that can assist you in coding. It scans the entire folder structure and extracts only the code structure. It then sends this information to the AI to get an overview of what needs to be changed. Subsequently, it instructs the AI to focus only on those files. This approach prevents us from overwhelming the AI with unrelated content, allowing it to remain focused on specific tasks while still having knowledge of the entire programming structure.

Here’s the link to the AI assistant: https://aider.chat

It may be a good idea to look into this to incorporate it

[kc0tlh]

@Frankg40 WOW, that is a really good idea! Actually kind of blowing my mind how much sense that makes. I've added it above and to the Help Center here!

[endocytosis]

Update:

Diff-based file editing is now on production as a feature preview! You can enable this by going to Settings > Feature Previews and toggling Diffs to on. We appreciate your continued feedback. Please let us know if you encounter any issues. You will know feature previews are enabled if you see the flask icon in the bottom right of the prompt text box as shown below.