88 lines
2.8 KiB
Markdown
88 lines
2.8 KiB
Markdown
---
|
|
tags:
|
|
- iso27DIY
|
|
- architecture
|
|
- LLMgenerated
|
|
---
|
|
# iso27DYI architecture with LLM
|
|
|
|
For connecting WeWeb (frontend) and Supabase (backend) with LLM functionality, you have several technology options:
|
|
- **OpenAI API** or **Anthropic API** directly from your middleware
|
|
- **Supabase JavaScript client** for database operations
|
|
- **WeWeb API calls** to your middleware endpoints
|
|
|
|
Since WeWeb works well with REST APIs and Supabase has excellent API support, you can build a middleware layer:
|
|
|
|
1. **Node.js/Express API** - Create API endpoints that:
|
|
- Receive requests from WeWeb
|
|
- Query Supabase for data
|
|
- Call LLM APIs (OpenAI, Anthropic, etc.)
|
|
- Return processed responses to WeWeb
|
|
2. **Supabase Edge Functions** - Use Supabase's serverless functions to:
|
|
- Handle LLM processing directly within your Supabase environment
|
|
- Access your database without additional API calls
|
|
- Deploy TypeScript/JavaScript functions that WeWeb can call
|
|
3. **Serverless Functions** (Vercel, Netlify, or AWS Lambda):
|
|
- Deploy functions that orchestrate between WeWeb, Supabase, and LLMs
|
|
- Good for scaling and keeping costs predictable
|
|
|
|
|
|
The **Supabase Edge Functions** approach is often the most elegant since it keeps everything in your existing Supabase ecosystem and supports TypeScript natively.
|
|
|
|
``` typescript
|
|
// Edge function that:
|
|
// 1. Receives user input from WeWeb
|
|
// 2. Queries Supabase database for context
|
|
// 3. Calls OpenAI/Anthropic API directly
|
|
// 4. Returns response to WeWeb
|
|
|
|
```
|
|
|
|
**For your slot-filling + content generation use case:**
|
|
|
|
- Direct API calls to LLMs work perfectly
|
|
- Supabase handles your data needs
|
|
- WeWeb handles the UI interactions
|
|
|
|
## Self-hosting the LLM
|
|
For self-hosting an LLM with an easy API, here are the simplest options:
|
|
|
|
**Ollama (Easiest)**
|
|
|
|
- Install Ollama on your server
|
|
- Pull models like `llama3`, `mistral`, `codellama`
|
|
- Built-in REST API at `http://localhost:11434`
|
|
- Simple JSON requests, OpenAI-compatible endpoints
|
|
- Works great for development and moderate production use
|
|
|
|
**vLLM (Best for Production)**
|
|
|
|
- High-performance serving with OpenAI-compatible API
|
|
- Excellent throughput and batching
|
|
- Simple command: `vllm serve model-name --api-key your-key`
|
|
- API available at `http://localhost:8000/v1/`
|
|
|
|
**Text Generation Inference (TGI)**
|
|
|
|
- Hugging Face's production-ready solution
|
|
- Docker container deployment
|
|
- OpenAI-compatible API
|
|
- Good performance optimization
|
|
|
|
**For your Supabase setup:**
|
|
|
|
1. Deploy Ollama/vLLM on a server (or even locally for development)
|
|
2. From your Supabase Edge Functions, make HTTP requests to your LLM API
|
|
3. No need to change your WeWeb → Supabase → LLM flow
|
|
|
|
**Quick start with Ollama:**
|
|
|
|
```bash
|
|
# Install and run
|
|
curl -fsSL https://ollama.ai/install.sh | sh
|
|
ollama serve
|
|
ollama pull llama3
|
|
```
|
|
|
|
Then your Edge Functions can call `POST http://your-server:11434/api/generate`
|
|
|