Amazon Bedrock & xAI support, cache breakpoint and more

Release note for v1.29.0

An exciting update! Let's dive in.

TL,DR:

• Official supports for Amazon Bedrock & xAI

• Set custom cache breakpoint when using Anthropic models

• Other quality-of-life improvements:

  • Convert temporary chat to normal chat

  • Added support for o1 models on Azure

  • Added supports for more file types

  • Better scroll-to-bottom behavior

  • Improved drag-and-drop support

Official supports for Amazon Bedrock & xAI

Amazon Bedrock & xAI are officially supported in BoltAI. Go to Settings > Models to add a new AI service.

For Amazon Bedrock, make sure you select the right region. Some models are only available on a certain region. All models are fetched directly from AWS. If you don't see a model, go to AWS Console and check if you have requested access.

Amazon Bedrock in BoltAI

Depends on the model on Amazon Bedrock you're using, you can use other features such as Document Analysis, AI plugins and Image Analysis...

xAI models supports both AI plugins and Document Analysis. To use Image Analysis, make sure to use the grok-vision-beta.

Document Analysis with Anthropic Haiku on Amazon Bedrock

Cache breakpoints

When using Document Analysis with Anthropic Claude, you have the option to set custom cache breakpoints. You can learn more about Anthropic Prompt Caching here.

To mark a message as a cache breakpoint, click the message action button (elipsis button), select "Mark as a cache breakpoint". The message will be highlighted.

Note that you can set up to 4 cache breakpoints and the cached prefixes automatically expired after 5 minutes of inactivity.

You can verify by viewing the token usage after each message. It should show the cache write & cache read tokens.

Cache Read Tokens

Other quality-of-life improvements

Convert a temporary chat to a normal chat

It's really simple. Click the menu item Conversation > Convert to Normal Chat.

Convert to Normal Chat

Added support for o1 models on Azure

You can set up o1 models on Azure now. Make sure to set the model to o1 or o1-mini so BoltAI could work correctly.

Document Analysis supports more file types now

You can attach more file types to BoltAI now: .jsx, .js, .html, .css...

Make sure to select text file as BoltAI doesn't work with binary files yet.

Better scroll experience

I've improved the scroll-to-bottom experience. It doesn't overlap with the chat input field, and it show/hide nicely as you scroll.

During streaming message, you can scroll up to cancel auto scroll-to-bottom, and scroll back down to resume it. I'm really happy with it. Give it a try!

Improved Drag and Drop experience

There is an issue where you can drag and drop a file to the plus button, but you cannot drop a file to a chat bubble in the message list. In this version, I've fixed it and you should be able to drag and drop files into the chats without issues.

Other bug fixes and improvements

• You can unset System Instruction now

• Fixed the issue where control+n / control+p for non-standard keyboard layout

• Fixed the issue where app icon is reverted after update

• Fixed issue where cannot set up Advanced Voice Mode without OpenAI API key

• Fixed issue with Web Search plugin for some Anthropic users

A small update on the iOS app and BoltAI v2

I've been using the iOS app myself but it doesn't support Cloud Sync yet, so I decided to not release it. I will wrap it up and release the first beta version soon, likely next week.

After working on BoltAI for almost 2 years, it becomes clear to me on what would be the core features of BoltAI, and what's not. Which areas work well, and which issues need to be fixed. I attempted to add Cloud Sync and other improvements into the current version, but it's proved to be harder than building a new app with better data handling.

So I've been working on the new major version, BoltAI v2. Despite its name, it's really an improvement of BoltAI v1 with Cloud Sync and cross-platform support.

Can't wait to share all of these with you.

And that's all for now. See you in the next update 👋