How to Set Up a Self-Hosted AI System From Scratch
Prerequisites
Before you begin, you need a Linux server meeting the hardware requirements, SSH access to the server with root or sudo privileges, API keys from at least one AI model provider such as Anthropic for Claude or OpenAI for GPT, and your initial training data such as documents, FAQs, or knowledge base content you want the AI to learn from. You do not need specialized AI knowledge. The setup process uses standard Linux tools and packages.
Step-by-Step Setup
Choose your hosting approach: a cloud instance like AWS EC2 or DigitalOcean, a VPS, or a physical server. Install a supported Linux distribution such as Amazon Linux, Ubuntu 22.04, or Debian 12. Ensure the server has a stable internet connection for cloud model API calls and configure your firewall to allow only the ports you need.
Install the required software dependencies. This includes Elixir and Erlang/OTP for process management, PHP for application logic, Python for ML libraries and embedding generation, and standard database and web server packages. Each component installs through the standard package manager for your Linux distribution.
Deploy the AI platform code to your server. This includes the agent orchestration system, the knowledge management layer, the memory bank, the governance engine, and the monitoring dashboard. The deployment process places files in the correct directories, sets permissions, and creates the initial database structures.
Add your API keys for the cloud AI models you want to use. Configure which models are used for which types of tasks. For example, you might assign Claude for complex reasoning, GPT-4.1-mini for routine operations, and Gemini for research tasks. The system supports multiple providers simultaneously and can fall back to alternative models if one is unavailable.
Upload your initial training data. This might include product documentation, customer FAQs, process guides, company policies, and domain expertise documents. The system processes these documents into searchable vector embeddings stored on local disk. You can add more knowledge at any time as your content grows.
Set up the rules your AI must follow. Define what data the AI can access, what actions require human approval, what topics it should escalate rather than handle, and what communication boundaries apply. See How to Set Rules That AI Must Always Follow for guidance on writing effective rules.
Launch the AI platform. The Elixir process manager starts the agent supervisor, which initializes your configured agents. Each agent begins operating according to its schedule and goals. The monitoring dashboard becomes available for you to observe system activity in real time.
After Setup
Once your system is running, monitor its behavior for the first few days. Review the decisions it makes, check that governance rules are being followed, and verify that knowledge retrieval is working correctly. Adjust configurations based on what you observe. Add more knowledge base content as you identify gaps. Tighten or relax governance thresholds based on the AI's performance. For ongoing maintenance guidance, see How to Monitor and Maintain a Self-Hosted AI System.
Get your self-hosted AI system running with guided setup and configuration support.
Contact Our Team