JevMade hello@JevMade.com
← Back to guides

JevMade field notes / Deployment guide

Host a private AI tool that chooses from set options

This guide explains how to set up a private AI assistant that picks from multiple-choice options instead of writing text. It covers testing the software and keeping large files saved on cloud computers.

Original by Eric ZhangIntegrations

Listen to this guide

JevMade’s plain-English explanation

0:00 /

AI narration

Credits

“Deploy a Jev-compatible SGLang server on Modal” by Eric Zhang. Read the original source.

This expanded guide is an AI-narrated adaptation prepared by JevMade. It expands the source’s essential ideas, examples and caveats in JevMade’s own words and is not a word-for-word reading. The synthetic voice does not imitate the author or imply their endorsement.

Our summary

This guide shows how to build a private AI server that uses the same communication format as Jev, an AI tool that chooses from options rather than writing an answer. Someone would use this to automatically sort incoming messages without relying on outside services.

The setup places the heavy AI software on a cloud computing platform while keeping the management tools on your computer. When you ask a question, the software calculates the mathematical probability of each multiple-choice option instead of generating a long text response.

This project requires a paid cloud account and powerful computer chips to run. The software's confidence scores are just mathematical estimates, not proof that an answer is right. It is useful for people who want to host their own private classification software.

Key takeaways

  1. Keep the heavy AI software on the cloud while running management tools on your own computer.
  2. Test the software with a temporary command to ensure it works before making it publicly available.
  3. Save the large AI files on the cloud so they do not download every single time.

This setup requires paid cloud computing resources and login credentials. The speed, cost, and quality of the answers depend entirely on the chosen AI software and cloud settings.

GitHub README · Source reviewed

Read the original guide Opens the author’s site in a new tab.