iso27diy-corp/AuditGlue/System alternative/iso27DYI architecture with LLM.md

tags
iso27DIY architecture LLMgenerated

iso27DYI architecture with LLM

For connecting WeWeb (frontend) and Supabase (backend) with LLM functionality, you have several technology options:

• OpenAI API or Anthropic API directly from your middleware
• Supabase JavaScript client for database operations
• WeWeb API calls to your middleware endpoints

Since WeWeb works well with REST APIs and Supabase has excellent API support, you can build a middleware layer:

1. Node.js/Express API - Create API endpoints that:
   • Receive requests from WeWeb
   • Query Supabase for data
   • Call LLM APIs (OpenAI, Anthropic, etc.)
   • Return processed responses to WeWeb
2. Supabase Edge Functions - Use Supabase's serverless functions to:
   • Handle LLM processing directly within your Supabase environment
   • Access your database without additional API calls
   • Deploy TypeScript/JavaScript functions that WeWeb can call
3. Serverless Functions (Vercel, Netlify, or AWS Lambda):
   • Deploy functions that orchestrate between WeWeb, Supabase, and LLMs
   • Good for scaling and keeping costs predictable

The Supabase Edge Functions approach is often the most elegant since it keeps everything in your existing Supabase ecosystem and supports TypeScript natively.

// Edge function that:
// 1. Receives user input from WeWeb
// 2. Queries Supabase database for context
// 3. Calls OpenAI/Anthropic API directly
// 4. Returns response to WeWeb

For your slot-filling + content generation use case:

• Direct API calls to LLMs work perfectly
• Supabase handles your data needs
• WeWeb handles the UI interactions

Self-hosting the LLM

For self-hosting an LLM with an easy API, here are the simplest options:

Ollama (Easiest)

• Install Ollama on your server
• Pull models like llama3, mistral, codellama
• Built-in REST API at http://localhost:11434
• Simple JSON requests, OpenAI-compatible endpoints
• Works great for development and moderate production use

vLLM (Best for Production)

• High-performance serving with OpenAI-compatible API
• Excellent throughput and batching
• Simple command: vllm serve model-name --api-key your-key
• API available at http://localhost:8000/v1/

Text Generation Inference (TGI)

• Hugging Face's production-ready solution
• Docker container deployment
• OpenAI-compatible API
• Good performance optimization

For your Supabase setup:

1. Deploy Ollama/vLLM on a server (or even locally for development)
2. From your Supabase Edge Functions, make HTTP requests to your LLM API
3. No need to change your WeWeb → Supabase → LLM flow

Quick start with Ollama:

# Install and run
curl -fsSL https://ollama.ai/install.sh | sh
ollama serve
ollama pull llama3

Then your Edge Functions can call POST http://your-server:11434/api/generate