Private LLM - Local AI Chat

Works well. The display supports markdown which is great. The Zephyr model is a good default. A few quirks make it 4/5 stars. There’s a strange animation when selecting text out if the chat. The shortcuts are so close to being useful. I want to be able to ask a question and have the response read by Siri while I’m driving. It almost works, but for some reason it always says “something went wrong” while in CarPlay. Finally, it might be nice to be able to save conversations somehow. A history would be a good way, or maybe just a way to export the whole chat to a file.

Fantastic app, I like the selection of models and they run great on the M4 chip of the iPhone 16 Max and iPad Pro. System prompt is also great for some tweaks to responses. The most simple and important feature (aside from model memory) that Private LLM needs is to be able to edit the responses of the AI. I enjoy the chatbot's responses but hate that it'll pull context from previous responses and loop entire phrases regardless if I regenerate its response or edit mine. Sometimes I don't like how the model formatted its response and want to change it without clearing the convo and retrying multiple times. The second feature that would be a quality of life is to be able to save conversations within the app. At the very least, export them as a whole without needing to screenshot, grab text, paste, correct, and format it elsewhere. I learned early on that copying text blocks at a time can cause the app to reload its model and even clear the conversation. A bot refresh would also be nice to clear older message context during a longer conversation to keep the model fast and responsive, but that can coincide with model memory to more easily pull specific facts you want it to remember in the first place without having to create a queue of explanations to pick up where you left off at. Useful if you switch models for different purposes like organizing, summarizing, q&a vs. generation, editing, brainstorming, vs. chatting.

UPDATE: app is much faster now. Still waiting on a prompt library which would be awesome, but the speed and quality is increasing seemingly by the day. There was a small period of time where it didn’t work on the 13 mini but that has been fixed. I’m happy with the amount of work the devs are putting in and I’m hopeful for more features in the future. MLC chat is comparable in speed and quality but this app is looking to widen the gap by adding more features in the (hopefully near) future. Old review: In comparison to MLC chat (free and open source) this app is lacking. MLC chat offers the ability to install additional models, and the default is on par or better than this apps responses (like gpt 3 level, lots of incorrect information and it kinda goes wild when explaining things). Additionally, performance is abysmal comparatively for better quality responses on MLC chat. It’s a great idea but lacks the ability to fine tune. Being able to change the temperature and other settings for the model would be great. Would also be cool if we could create a starting prompt (or a library to choose from) for the AI so it can be used for things like role playing or DND. Kinda bummed that it’s $5 since I was expecting a lot more than MLC AIs app for the pricing. Instead I’ll be uninstalling and checking back later for new features and possible performance increases. If you’re reading this and thinking about buying it, you should probably check out MLC chat for now and just wait for more features and performance updates.