Cleaning up the Sparks folder
This commit is contained in:
parent
eb610a79b6
commit
96cd8fea7b
78 changed files with 149 additions and 181 deletions
|
|
@ -11,4 +11,4 @@ Additional resources and support are available on the iso27diy.com website.
|
|||
* No need for external consultants or expensive software
|
||||
|
||||
|
||||
See also [ISO27DIY benefits](../Sparks/ISO27DIY%20benefits.md)
|
||||
See also [ISO27DIY benefits](../Sparks/iso27diy/ISO27DIY%20benefits.md)
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
tags:
|
||||
- llm
|
||||
- mcp
|
||||
---
|
||||
Perplexity, 13 juni 2025
|
||||
|
||||
Prompt:`Is there a GGUF model that is particularly good at creating abstracts for documents in PDF format and assigning them to one of a number of predefined categories?
|
||||
|
||||
### Top GGUF Models for Summarization and Categorization
|
||||
|
||||
- **Llama-Chat-Summary-3.2-3B-GGUF**
|
||||
|
||||
A fine-tuned Llama 3.2 model optimized for context-aware summarization of long texts, documents, and conversations. It preserves critical points and creates concise summaries, making it ideal for abstracting lengthy reports or articles[^3_1].
|
||||
|
||||
- **Gemma 7B GGUF**
|
||||
|
||||
A lightweight, efficient model designed for summarization, question answering, and reasoning. It supports long context lengths (up to 8192 tokens) and can generate accurate summaries suitable for document abstraction[^3_3][^3_6].
|
||||
|
||||
- **Phi 3.5 Mini Instruct GGUF**
|
||||
|
||||
Supports very long context lengths (up to 128K tokens), enabling summarization of large documents. Its multilingual and reasoning capabilities make it a strong candidate for document summarization and classification tasks[^3_4].
|
||||
|
||||
- **CausalLM-7B-GGUF**
|
||||
|
||||
A versatile model capable of text summarization and content generation, which can be fine-tuned or prompted for categorization tasks as well[^3_5].
|
||||
|
||||
### How to Use for PDF Documents
|
||||
|
||||
- Extract text from PDFs using tools like `pdfplumber` or `PyMuPDF`.
|
||||
- Feed extracted text chunks into these GGUF models for summarization.
|
||||
- Use prompt templates or fine-tuning to classify summaries into your predefined categories.
|
||||
|
||||
### Summary
|
||||
|
||||
| Model Name | Size | Key Strengths | Context Length | Notes |
|
||||
| :------------------------- | :--- | :-------------------------------- | :------------- | :--------------------------- |
|
||||
| Llama-Chat-Summary-3.2-3B | 3.2B | Context-aware summarization | Moderate | Fine-tuned for summarization |
|
||||
| Gemma 7B GGUF | 7B | Summarization, reasoning | 8192 tokens | Lightweight, efficient |
|
||||
| Phi 3.5 Mini Instruct GGUF | 3.8B | Long document summarization | 128K tokens | Handles very long texts |
|
||||
| CausalLM-7B-GGUF | 7B | Summarization, content generation | Moderate | Versatile, fine-tunable |
|
||||
|
||||
|
||||
These GGUF models are currently among the best for summarization tasks and can be adapted for categorization with proper prompt design or fine-tuning. The Llama-Chat-Summary-3.2-3B-GGUF model is particularly focused on generating concise, context-aware abstracts[^3_1]. For very long documents, Phi 3.5 Mini Instruct GGUF’s extended context window is advantageous[^3_4].
|
||||
|
||||
If you want a ready-to-use model, start with Llama-Chat-Summary-3.2-3B-GGUF or Gemma 7B GGUF and implement classification via prompting or additional fine-tuning.
|
||||
|
||||
[^3_1]: https://huggingface.co/Triangle104/Llama-Chat-Summary-3.2-3B-Q4_K_M-GGUF
|
||||
[^3_2]: https://www.reddit.com/r/LocalLLaMA/comments/1891o5m/whats_the_best_llm_for_summarization_of_long/
|
||||
[^3_3]: https://huggingface.co/LoneStriker/gemma-7b-GGUF
|
||||
[^3_4]: https://dataloop.ai/library/model/quantfactory_phi-35-mini-instruct-gguf/
|
||||
[^3_5]: https://www.aimodels.fyi/models/huggingFace/causallm-7b-gguf-thebloke
|
||||
[^3_6]: https://dataloop.ai/library/model/litellms_gemma-7b-gguf/
|
||||
[^3_7]: https://www.ibm.com/think/topics/gguf-versus-ggml
|
||||
[^3_8]: https://docs.vllm.ai/en/v0.9.0/features/quantization/gguf.html
|
||||
314
Corpus/Various/Instruct an LLM on available tools.md
Normal file
314
Corpus/Various/Instruct an LLM on available tools.md
Normal file
|
|
@ -0,0 +1,314 @@
|
|||
---
|
||||
tags:
|
||||
- project/iso27DIY
|
||||
- dev
|
||||
- llm
|
||||
- agent
|
||||
---
|
||||
|
||||
[orq](https://orq.ai/blog/llm-tools)
|
||||
|
||||
[
|
||||
|
||||
15 Best LLM Tools for AI Product Development \[2025 Guide\]
|
||||
|
||||
](https://orq.ai/blog/llm-tools)[mercity.ai](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)
|
||||
|
||||
[
|
||||
|
||||
Comprehensive Guide to Integrating Tools and APIs with Language...
|
||||
|
||||
](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)[DataCamp](https://www.datacamp.com/blog/llmops-tools)
|
||||
|
||||
[
|
||||
|
||||
Top 15 LLMOps Tools for Building AI Applications in 2025 - DataCamp
|
||||
|
||||
](https://www.datacamp.com/blog/llmops-tools)[softwareag](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)
|
||||
|
||||
[
|
||||
|
||||
LLM Tool Usage & API Integration - Software AG
|
||||
|
||||
](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)
|
||||
|
||||
To instruct an LLM about available tools and how to call them, you typically follow a structured approach:
|
||||
|
||||
**1\. Define Available Tools**
|
||||
|
||||
- Provide the LLM with a list of tools, each including:
|
||||
- A name (identifier)
|
||||
- A description (explains when to use it)
|
||||
- A schema (usually JSON) describing input parameters and their types [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**2\. Inform the LLM via Prompting**
|
||||
|
||||
- When sending a user query, include the tool definitions in the prompt or system message. This allows the LLM to understand what tools are available and how to use them [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**3\. Tool Calling Process**
|
||||
|
||||
- The LLM receives the user prompt and tool list.
|
||||
- It decides if a tool is needed, selects the appropriate one, and formats a tool call (often as a structured JSON object with arguments) [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
- Your application receives this tool call, executes the corresponding function or API, and returns the result to the LLM.
|
||||
- The LLM integrates the tool’s output into its next response, continuing the conversation naturally [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**4\. Training or Fine-Tuning (Optional)**
|
||||
|
||||
- For more advanced usage, you can fine-tune the LLM with example data showing how to use each tool, or use models already trained for tool use (like some recent open-weight models) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**Summary Table**
|
||||
|
||||
| Step | Action |
|
||||
| --- | --- |
|
||||
| Define tools | Name, description, input schema |
|
||||
| Prompt LLM | Include tool definitions in prompt/system message |
|
||||
| Tool call & execution | LLM outputs tool call; app executes it and returns result |
|
||||
| Integrate response | LLM uses tool output in conversation |
|
||||
| (Optional) Fine-tuning | Train LLM on tool usage examples for better performance |
|
||||
|
||||
This approach enables LLMs to use external tools flexibly and accurately within applications [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools).
|
||||
|
||||
I’m using Anything LLM. How can I enable a model to run shell scripts?[GitHub](https://github.com/Mintplex-Labs/anything-llm/issues/3451)
|
||||
|
||||
[
|
||||
|
||||
\[FEAT\]: Shell/Script invocation block for flow builder #3451 - GitHub
|
||||
|
||||
](https://github.com/Mintplex-Labs/anything-llm/issues/3451)[reddit.com](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)
|
||||
|
||||
[
|
||||
|
||||
Can I integrate my local LLM to enable it to run system commands...
|
||||
|
||||
](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)[GitHub](https://github.com/regnull/how.sh)
|
||||
|
||||
[
|
||||
|
||||
how.sh - Create and execute shell commands using LLM - GitHub
|
||||
|
||||
](https://github.com/regnull/how.sh)[anythingllm](https://anythingllm.com/)
|
||||
|
||||
[
|
||||
|
||||
AnythingLLM | The all-in-one AI application for everyone
|
||||
|
||||
](https://anythingllm.com/)
|
||||
|
||||
To enable shell script execution in AnythingLLM, you can implement custom agent skills through these methods:
|
||||
|
||||
**1\. Custom Agent Skill Development**
|
||||
|
||||
- Create a `plugin.json` file defining your shell script entrypoint:
|
||||
```
|
||||
json
|
||||
{
|
||||
"name": "shell_runner",
|
||||
"description": "Execute shell commands",
|
||||
"entrypoint": "./handler.js",
|
||||
"permissions": ["shell"]
|
||||
}
|
||||
```
|
||||
- Develop a handler file (e.g., `handler.js`) using Node.js' `child_process` module [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux):
|
||||
```
|
||||
javascript
|
||||
const { exec } = require('child_process');
|
||||
module.exports = async ({ command }) => {
|
||||
return new Promise((resolve, reject) => {
|
||||
exec(command, (error, stdout, stderr) => {
|
||||
if (error) reject(error);
|
||||
resolve(stdout || stderr);
|
||||
});
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
**2\. Security Considerations**
|
||||
|
||||
- Restrict allowed commands through allow-listing in your handler
|
||||
- Implement user permission tiers for script execution
|
||||
- Use environment sandboxing for dangerous operations [6](https://news.ycombinator.com/item?id=35172486)
|
||||
|
||||
**3\. Integration Approaches**
|
||||
|
||||
- **LangChain Agent Method** (shown in tutorial 5):
|
||||
```
|
||||
python
|
||||
from langchain.agents import initialize_agent
|
||||
from langchain.tools import ShellTool
|
||||
shell_tool = ShellTool()
|
||||
agent = initialize_agent([shell_tool], llm, agent="zero-shot-react-description")
|
||||
```
|
||||
- **Ollama Local Model Integration** [3](https://github.com/regnull/how.sh):
|
||||
```
|
||||
bash
|
||||
ollama pull deepseek-coder-v2
|
||||
llm -m ollama/deepseek-coder-v2 "convert all .txt files to .md" > convert.sh
|
||||
```
|
||||
|
||||
**4\. Execution Flow**
|
||||
|
||||
1. User requests shell operation through chat interface
|
||||
2. LLM processes request and generates structured tool call
|
||||
3. Handler executes the command via Node.js `exec()` [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux)
|
||||
4. Output is returned to LLM for response formatting
|
||||
|
||||
**Example Use Case** (from GitHub issue [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451)):
|
||||
|
||||
```
|
||||
bash
|
||||
# plugin.json entry
|
||||
"entrypoint": "python -m browser_automation.py"
|
||||
```
|
||||
|
||||
**Limitations & Risks**
|
||||
|
||||
- Requires careful input sanitization to prevent injection attacks
|
||||
- Windows support limited due to shell differences 5
|
||||
- Execution environment must match user's system capabilities [2](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/) [6](https://news.ycombinator.com/item?id=35172486)
|
||||
|
||||
For production use, consider implementing:
|
||||
|
||||
- Command validation middleware
|
||||
- Output sanitization filters
|
||||
- Session-based execution isolation
|
||||
- Activity logging for audit trails [6](https://news.ycombinator.com/item?id=35172486) [7](https://docs.anythingllm.com/installation-desktop/linux)
|
||||
|
||||
What are the best methods to integrate local executables with Anything LLM for automation[semaphoreci](https://semaphoreci.com/blog/local-llm)
|
||||
|
||||
[
|
||||
|
||||
6 Ways to Run LLMs Locally (also how to use HuggingFace)
|
||||
|
||||
](https://semaphoreci.com/blog/local-llm)[Stream](https://getstream.io/blog/best-local-llm-tools/)
|
||||
|
||||
[
|
||||
|
||||
The 6 Best LLM Tools To Run Models Locally - GetStream.io
|
||||
|
||||
](https://getstream.io/blog/best-local-llm-tools/)[apidog](https://apidog.com/blog/top-llm-local-tools)
|
||||
|
||||
[
|
||||
|
||||
The 5 Best LLM Tools To Run Models Locally - Apidog
|
||||
|
||||
](https://apidog.com/blog/top-llm-local-tools)[anythingllm](https://anythingllm.com/)
|
||||
|
||||
[
|
||||
|
||||
AnythingLLM | The all-in-one AI application for everyone
|
||||
|
||||
](https://anythingllm.com/)
|
||||
|
||||
To integrate local executables with AnythingLLM for automation, use these methods:
|
||||
|
||||
## 1\. Custom Plugin Development
|
||||
|
||||
Create executable-wrapping plugins using AnythingLLM's extensible architecture:
|
||||
|
||||
```
|
||||
javascript
|
||||
// plugin.json
|
||||
{
|
||||
"name": "pdf_converter",
|
||||
"description": "Converts DOCX to PDF using LibreOffice",
|
||||
"entrypoint": "./convert.js",
|
||||
"permissions": ["file_system"]
|
||||
}
|
||||
|
||||
// convert.js
|
||||
const { execSync } = require('child_process');
|
||||
|
||||
module.exports = async ({ inputPath }) => {
|
||||
try {
|
||||
execSync(\`soffice --convert-to pdf "${inputPath}" --headless\`);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return { error: error.message };
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
|
||||
- Direct CLI execution via Node.js `child_process` [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)
|
||||
- File system access controls through permission tiers
|
||||
- Error handling integration with LLM feedback loops
|
||||
|
||||
---
|
||||
|
||||
## 2\. LangChain Agent Integration
|
||||
|
||||
Leverage LangChain's tool ecosystem for executable orchestration:
|
||||
|
||||
```
|
||||
python
|
||||
from langchain.agents import initialize_agent
|
||||
from langchain.tools import Tool
|
||||
|
||||
def run_executable(command):
|
||||
import subprocess
|
||||
result = subprocess.run(command, shell=True, capture_output=True)
|
||||
return result.stdout.decode()
|
||||
|
||||
agent = initialize_agent(
|
||||
tools=[Tool(
|
||||
name="LocalExec",
|
||||
func=run_executable,
|
||||
description="Runs local system commands"
|
||||
)],
|
||||
llm=anything_llm_instance,
|
||||
agent="structured-chat-zero-shot-react-description"
|
||||
)
|
||||
```
|
||||
|
||||
**Advantages**:
|
||||
|
||||
- Native integration with AnythingLLM's chat interface [4](https://anythingllm.com/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe)
|
||||
- Automatic command validation through LangChain's schema system
|
||||
- Multi-step workflow automation capabilities
|
||||
|
||||
---
|
||||
|
||||
## 3\. Llamafile Binary Bundling
|
||||
|
||||
Package executables with LLMs using Mozilla's Llamafile:
|
||||
|
||||
```
|
||||
bash
|
||||
# Combine executable and LLM into single binary
|
||||
llamafile-convert --exec my_script.py --model mistral-7b.gguf
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
|
||||
- Self-contained deployment without dependency conflicts [2](https://getstream.io/blog/best-local-llm-tools/) [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)
|
||||
- Cross-platform support (Windows/Linux/macOS)
|
||||
- Direct memory sharing between executable and LLM [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)
|
||||
|
||||
---
|
||||
|
||||
## Security Implementation
|
||||
|
||||
| Measure | Implementation Example |
|
||||
| --- | --- |
|
||||
| Input Sanitization | Regex whitelist for allowed CLI parameters |
|
||||
| Permission Tiers | User roles with executable access levels |
|
||||
| Sandboxing | Docker containers for risky operations |
|
||||
| Audit Logging | JSON logs of all executed commands |
|
||||
|
||||
---
|
||||
|
||||
## Use Case Workflow
|
||||
|
||||
1. User requests "Convert quarterly\_report.docx to PDF"
|
||||
2. LLM triggers `pdf_converter` plugin via structured JSON [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches)
|
||||
3. Handler executes LibreOffice CLI in Docker sandbox
|
||||
4. Output PDF path returned to LLM for user sharing
|
||||
|
||||
**Best Practices**:
|
||||
|
||||
- Use absolute paths for local executables
|
||||
- Implement timeout guards for long-running processes
|
||||
- Test with `--dry-run` flags before production deployment [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)
|
||||
|
||||
For complex automation, combine methods 2 and 3 using Llamafile-packaged tools controlled through LangChain agents [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe).
|
||||
|
|
@ -0,0 +1,246 @@
|
|||
---
|
||||
tags:
|
||||
- llm
|
||||
- dev
|
||||
---
|
||||
|
||||
# Integrating Ollama Embeddings with AnythingLLM and LM Studio – Orchestrating a Specialized Local AI Stack
|
||||
|
||||
https://gemini.google.com/app/44c7cb4ab371623a
|
||||
|
||||
## 1. Executive Summary: Orchestrating Your Local AI Stack with Specialized Embedding
|
||||
|
||||
The objective of achieving a highly customized and locally hosted Artificial Intelligence (AI) environment, leveraging specific models for distinct tasks, is increasingly common among technically proficient users. This report details the integration of Ollama for serving a dedicated embedding model, LM Studio for hosting local Large Language Models (LLMs) for chat and generation, and AnythingLLM as the central Retrieval Augmented Generation (RAG) and interaction platform. This modular approach offers significant flexibility in model selection, resource management, and overall control over the AI workflow.
|
||||
|
||||
The desire to employ Ollama for a specific embedding model, while concurrently utilizing LM Studio for primary LLM functionalities, indicates a sophisticated approach to building a local AI stack. This is not merely about enabling _an_ embedding model, but rather integrating _a particular one_ through a preferred tool, Ollama. This specificity suggests requirements related to model availability, unique performance characteristics of a chosen embedding model, or established familiarity with Ollama for this purpose. The inherent flexibility of AnythingLLM in connecting to various LLM and embedding providers is central to enabling such a tailored configuration, allowing each component to play to its strengths within a cohesive system. This report will guide the configuration and explain the interplay of these powerful tools.
|
||||
|
||||
## 2. Core Components & Their Roles in Your Setup
|
||||
|
||||
Understanding the individual roles of AnythingLLM, Ollama, and LM Studio is crucial before detailing their integration. Each component serves a distinct purpose in the proposed architecture.
|
||||
|
||||
### 2.1. AnythingLLM: The Central Orchestrator for RAG
|
||||
|
||||
AnythingLLM functions as a comprehensive, private AI application designed to facilitate RAG, manage AI Agents, and enable interaction with a variety of LLMs and embedding models. Its primary role in this configuration is to act as the user interface and the orchestration layer that manages documents, interacts with the vector database, and routes requests to the appropriate model servers—Ollama for embeddings and LM Studio for chat completions.
|
||||
|
||||
A critical aspect of AnythingLLM's architecture is that embedding models are configured system-wide. This means that the choice to use Ollama as the embedding provider will apply to all documents and workspaces within that specific AnythingLLM instance. Unlike LLMs, which can be selected on a per-workspace basis, the embedding configuration is global. This has significant implications: if a user previously relied on a different embedding model (e.g., a built-in model or one served by LM Studio), switching to Ollama will necessitate the re-embedding of all existing documents to ensure compatibility and accurate retrieval.
|
||||
|
||||
### 2.2. Ollama: Your Dedicated Engine for Custom Embedding Models
|
||||
|
||||
Ollama is an open-source tool designed for running various AI models, including both LLMs and embedding models, locally on either CPU or GPU resources. In this setup, its designated function is to serve the specific embedding model chosen by the user. Ollama excels at pulling and managing a wide array of models, including those from the Hugging Face Hub or models in GGUF format, directly via its command-line interface.1 Common embedding models like `nomic-embed-text` or `mxbai-embed-large` can be easily run, or more specialized ones like `sunzhiyuan/suntray-embedding`.
|
||||
|
||||
Once an embedding model is pulled and Ollama is running (typically initiated with `ollama serve` or by launching the Ollama application), it exposes an API endpoint, commonly `http://127.0.0.1:11434` 2, which AnythingLLM will use to request text embeddings. Ollama's directness in model management ensures that the user can deploy precisely the embedding model required for their RAG pipeline, offering fine-grained control over the vectorization process. This aligns perfectly with the objective of using a _specific_ embedding model.
|
||||
|
||||
### 2.3. LM Studio: Powering Local Language Models (and its own embedding capabilities)
|
||||
|
||||
LM Studio is a desktop application known for its user-friendly interface that simplifies the discovery, download, configuration, and execution of local LLMs, frequently sourced from Hugging Face in formats like GGUF. It features a local inference server that can provide API endpoints for both chat completions and, notably, text embeddings.3 LM Studio also offers valuable features for resource management, such as CPU offloading and multi-GPU support, which can be beneficial when running large models.
|
||||
|
||||
While LM Studio possesses the capability to serve embedding models, the specific requirement here is to utilize Ollama for this task. This implies a deliberate choice, perhaps driven by the availability of a particular embedding model through Ollama, a desire to isolate resources (dedicating LM Studio primarily to the more computationally intensive chat LLM), or specific features offered by Ollama for embedding tasks. Therefore, within this documented setup, LM Studio's primary role will be to serve the chat/generative LLM to AnythingLLM. Its embedding server functionality will be acknowledged but not be the focus for the embedding pipeline, respecting the user's preference for Ollama in this capacity.
|
||||
|
||||
## 3. Implementing Ollama for Specific Embeddings in AnythingLLM
|
||||
|
||||
The following steps outline the process for preparing an embedding model in Ollama and configuring AnythingLLM to use it.
|
||||
|
||||
### 3.1. Step 1: Preparing Your Embedding Model in Ollama
|
||||
|
||||
Before AnythingLLM can use an embedding model from Ollama, the model must be downloaded and Ollama must be serving it.
|
||||
|
||||
1. **Pull the Embedding Model:** Use the Ollama command-line interface (CLI) to download the desired embedding model. The command is `ollama pull <model-name>`. Replace `<model-name>` with the identifier of the specific embedding model. For example:
|
||||
|
||||
- `ollama pull nomic-embed-text`
|
||||
- `ollama pull mxbai-embed-large`
|
||||
- Or any other specific embedding model available through Ollama's library or importable as a GGUF.
|
||||
2. **Ensure Ollama is Serving:** The Ollama application must be running, or the Ollama service started (e.g., via `ollama serve` in the terminal on some systems).3 This makes the Ollama API, including the embedding endpoint, accessible on the local machine, typically at `http://127.0.0.1:11434`.
|
||||
|
||||
|
||||
It is paramount to select a model that is genuinely designed for generating embeddings (i.e., text-to-vector transformations) rather than a model intended for chat or text generation. Ollama's API might list both types of models if both have been pulled; therefore, careful selection of an actual embedder is crucial.2 Using a chat model like `llama2` for embedding tasks will lead to incorrect and ineffective RAG performance.
|
||||
|
||||
### 3.2. Step 2: Configuring AnythingLLM to Utilize Ollama Embeddings
|
||||
|
||||
Once Ollama is serving the chosen embedding model, configure AnythingLLM to use it:
|
||||
|
||||
1. **Navigate to Embedder Settings:** In the AnythingLLM application, access the settings or configuration section for embedding models.
|
||||
2. **Select Ollama:** Choose "Ollama" from the list of available embedding providers.
|
||||
3. **Enter API Base URL:** Input the base URL for the Ollama API. For a standard local installation, this is `http://127.0.0.1:11434`.2
|
||||
4. **Specify Embedding Model Name:** In the designated field (often labeled "Embedding Model Name" or similar), enter the _exact_ name of the model that was pulled and is being served by Ollama (e.g., `nomic-embed-text`, `mxbai-embed-large`).2 This name must precisely match the identifier used by Ollama.
|
||||
|
||||
AnythingLLM can leverage any HuggingFace or GGUF embedding model if served through a compatible local API like Ollama or LocalAI. The "Embedding Model Name" field in AnythingLLM is not for model discovery; it's a specific identifier that tells AnythingLLM which already-active model to request from the Ollama server. If this name doesn't match a model that Ollama is currently serving, the embedding process will fail. This underscores the importance of correctly completing Step 3.1 before configuring AnythingLLM.
|
||||
|
||||
### 3.3. Critical Note: The Impact of Changing Embedding Providers in AnythingLLM
|
||||
|
||||
Changing the embedding model provider in AnythingLLM is a significant operation with direct consequences for existing data. As stated in the AnythingLLM documentation, "Once you select your embedding model provider and begin uploading and embedding documents it is best to not change it. While you can change embedders, doing so will mean you will have to delete your uploaded documents and re-embed them so the new embedder can re-embed them". This can also lead to "broken queries" if not handled correctly.
|
||||
|
||||
The necessity for re-embedding arises because different embedding models generate numerically different and incompatible vector representations for the same text. These vectors are mathematical representations of semantic meaning, and their structure is unique to the model that created them. If documents were embedded with Model A, their vectors reside in a vector space defined by Model A. If the system is then switched to use Model B for embeddings, new queries will be embedded using Model B, resulting in query vectors in a different vector space. Comparing vectors from Model B's space against those from Model A's space in the vector database (such as LanceDB, which AnythingLLM uses for local storage) will yield meaningless similarity scores and thus irrelevant retrieval results.
|
||||
|
||||
Therefore, switching to the Ollama-served embedding model requires that all previously ingested documents in AnythingLLM be deleted and then re-uploaded (or re-processed if the source files are still accessible to AnythingLLM) to generate new embeddings using the Ollama model. This process can be time-consuming and computationally intensive, particularly for users with large document collections, and is a crucial operational factor to consider before making the change.
|
||||
|
||||
## 4. Integrating LM Studio for Chat/Generative Models
|
||||
|
||||
With Ollama configured for embeddings, the next step is to ensure LM Studio is set up to provide the chat/generative LLM capabilities to AnythingLLM.
|
||||
|
||||
### 4.1. Preparing Your Chat Model in LM Studio
|
||||
|
||||
1. **Load Chat Model:** Within the LM Studio application, search for and download (if not already present) the desired chat model. This will typically be a GGUF-formatted model suitable for conversational tasks (e.g., variants of Llama, Mistral, Qwen).3 Once downloaded, load the model into LM Studio's active memory.
|
||||
2. **Start Local Inference Server:** In LM Studio, navigate to the "Local Server" tab. Select the loaded model and start the server. This action creates an HTTP server that exposes API endpoints compatible with OpenAI's API specifications, including `/chat/completions`. The server typically runs on `http://localhost:1234`, with the API versioned endpoint being `http://localhost:1234/v1`.3
|
||||
|
||||
For both Ollama (serving embeddings) and LM Studio (serving the chat LLM) to be simultaneously usable by AnythingLLM, both their respective servers must be running concurrently. This implies a combined load on system resources (CPU, GPU, RAM), which needs to be considered for smooth operation.
|
||||
|
||||
### 4.2. Configuring AnythingLLM to Use LM Studio as the LLM Provider
|
||||
|
||||
Unlike the system-wide setting for embedding providers, LLM providers in AnythingLLM can often be configured on a per-workspace basis. This allows for flexibility if different workspaces need to interact with different chat models.
|
||||
|
||||
1. **Navigate to LLM Settings:** In AnythingLLM, either in the general settings or within a specific workspace's configuration, find the LLM provider options.
|
||||
2. **Select LM Studio:** Choose "LM Studio" from the list of available LLM providers.3
|
||||
3. **Enter Server Base URL:** Input the base URL for the LM Studio inference server, which is typically `http://localhost:1234/v1`.3
|
||||
4. **Model Detection:** Once connected, AnythingLLM should automatically detect and display the model currently loaded and served by LM Studio.3 Select this model for the workspace.
|
||||
|
||||
The ability of AnythingLLM to independently configure the source for embeddings (Ollama) and the source for chat completions (LM Studio) is what makes this specialized, hybrid setup feasible.
|
||||
|
||||
### 4.3. Contextualizing LM Studio's Native Embedding Server Functionality
|
||||
|
||||
It is important to acknowledge that LM Studio's local inference server also supports an `/embeddings` endpoint, meaning it can generate embeddings from models loaded within it.3 However, given the user's explicit intent to use a _specific embedding model through Ollama_, this native embedding capability of LM Studio, while available, is not the primary focus for the embedding task in this particular configuration.
|
||||
|
||||
This clarification serves two purposes: it avoids potential confusion about why Ollama is being introduced if LM Studio could theoretically handle embeddings, and it reinforces the user's control in choosing specialized tools for specific parts of their AI pipeline. This preempts the question, "Why not use LM Studio for embeddings as well?" by centering the solution on the user's stated preference for Ollama for their chosen embedding model.
|
||||
|
||||
## 5. Synergizing the System: Ollama Embeddings with LM Studio LLMs via AnythingLLM
|
||||
|
||||
With Ollama serving embeddings and LM Studio serving the chat LLM, AnythingLLM orchestrates their functions to deliver a RAG experience.
|
||||
|
||||
### 5.1. Defining the Data Flow and Interaction Points
|
||||
|
||||
The interaction between these components follows distinct paths for document ingestion and querying:
|
||||
|
||||
**Document Ingestion Workflow:**
|
||||
|
||||
1. A user uploads a document into an AnythingLLM workspace.
|
||||
2. AnythingLLM processes the document, extracting its text content.
|
||||
3. For each relevant chunk of text, AnythingLLM sends an API request to the Ollama server (e.g., at `http://127.0.0.1:11434`) with the text, specifying the configured embedding model.
|
||||
4. Ollama's embedding model processes the text and returns the corresponding vector embedding.
|
||||
5. AnythingLLM receives this vector and stores it, along with the original text chunk and metadata, in its vector database (e.g., LanceDB).
|
||||
|
||||
**Querying/Chatting (RAG) Workflow:**
|
||||
|
||||
1. A user submits a query or prompt within an AnythingLLM workspace chat interface.
|
||||
2. AnythingLLM sends the user's query text to the Ollama server for embedding, using the same embedding model configured for the system.
|
||||
3. Ollama returns the query vector.
|
||||
4. AnythingLLM uses this query vector to perform a similarity search against the vectors stored in its vector database, retrieving the most relevant document chunks.
|
||||
5. AnythingLLM constructs an augmented prompt by combining the user's original query with the content of these retrieved document chunks.
|
||||
6. This augmented prompt is then sent via API request to the LM Studio server (e.g., at `http://localhost:1234/v1`), targeting the configured chat LLM.
|
||||
7. The chat LLM in LM Studio processes the augmented prompt and generates a response.
|
||||
8. LM Studio sends this generated response back to AnythingLLM.
|
||||
9. AnythingLLM displays the response to the user in the chat interface.
|
||||
|
||||
Throughout these processes, the Ollama and LM Studio servers operate independently, each handling requests specific to their configured models. AnythingLLM acts as the crucial middleware, managing the data flow, interacting with the vector store, and coordinating the calls to the specialized model servers.
|
||||
|
||||
### 5.2. Ensuring Coherent Operation Across the Stack
|
||||
|
||||
For the entire system to function correctly, several conditions must be met:
|
||||
|
||||
- **Server Availability:** Both the Ollama server (with the embedding model loaded) and the LM Studio server (with the chat model loaded) must be running and accessible on their respective network ports.
|
||||
- **Correct API Configuration:** The API endpoints and model names configured in AnythingLLM for both Ollama and LM Studio must accurately reflect the actual server addresses and model identifiers.
|
||||
- **Network Accessibility:** While typically operating on `localhost`, any local firewall or security software must not block AnythingLLM from making HTTP requests to the ports used by Ollama (e.g., 11434) and LM Studio (e.g., 1234). Connection issues can sometimes arise, and debugging tools or logs from each application may be necessary to resolve them.
|
||||
|
||||
The following table provides a consolidated overview of the key configuration points within AnythingLLM for this setup:
|
||||
|
||||
**Table 1: Component Configuration Overview in AnythingLLM**
|
||||
|
||||
| | | | | |
|
||||
|---|---|---|---|---|
|
||||
|**AnythingLLM Role**|**Target Tool**|**Configuration in AnythingLLM (Provider Type)**|**API Endpoint / Key Identifier**|**Model Name (if applicable)**|
|
||||
|Embedding Provider (System-Wide)|Ollama|Ollama|`http://127.0.0.1:11434`|`<User's Chosen Ollama Embedding Model Name>`|
|
||||
|LLM Provider (Workspace)|LM Studio|LM Studio|`http://localhost:1234/v1`|(Auto-detected from LM Studio)|
|
||||
|
||||
This table serves as a quick reference to verify that each component is correctly linked within the AnythingLLM environment.
|
||||
|
||||
## 6. Advanced Considerations and Best Practices
|
||||
|
||||
Operating a multi-component local AI stack involves several advanced considerations to ensure optimal performance and reliability.
|
||||
|
||||
### 6.1. Resource Allocation for Ollama and LM Studio
|
||||
|
||||
Running both Ollama (serving an embedding model) and LM Studio (serving a potentially large chat LLM) simultaneously places considerable demand on system resources, particularly CPU, system RAM, and GPU VRAM. Embedding models are generally less resource-intensive than large chat models, but their combined footprint can be substantial.
|
||||
|
||||
- **GPU VRAM:** Both Ollama and LM Studio can utilize GPU acceleration if available. Large LLMs are particularly VRAM-hungry. If the total VRAM required by both models exceeds the available dedicated GPU memory, performance can degrade significantly as the system resorts to using shared GPU memory or system RAM for model layers, which is much slower.
|
||||
- **LM Studio GPU Controls:** LM Studio provides advanced GPU management features, especially for NVIDIA CUDA setups, such as enabling/disabling specific GPUs, setting priority orders for GPU allocation, and limiting model offload to dedicated GPU memory. These can be crucial for managing resources in multi-GPU systems or for ensuring that a primary GPU is prioritized for the more demanding chat LLM.
|
||||
- **Model Quantization:** Using quantized versions of models (e.g., GGUF with lower bit precision) can significantly reduce their VRAM and RAM footprint, making it more feasible to run multiple models concurrently on systems with limited resources.
|
||||
- **System RAM and CPU:** Sufficient system RAM is also vital, especially if GPU VRAM is limited and parts of the models are offloaded to RAM. CPU performance will affect the overall responsiveness of the applications and the speed of CPU-bound tasks during model inference.
|
||||
|
||||
Users should monitor their system resource usage (VRAM, RAM, CPU/GPU utilization) when both servers are active to identify potential bottlenecks. Adjusting model choices (e.g., smaller or more heavily quantized versions) or leveraging LM Studio's GPU controls may be necessary to achieve a stable and performant setup.
|
||||
|
||||
### 6.2. Model Selection Strategies for Embeddings vs. Chat
|
||||
|
||||
The success of a RAG system hinges on the quality of both its retrieval and generation components.
|
||||
|
||||
- **Embedding Model:** The choice of embedding model (served by Ollama in this case) directly impacts retrieval quality. Models like `nomic-embed-text` or `mxbai-embed-large` are popular choices designed to capture semantic similarity effectively. The user's decision to use a _specific_ embedding model suggests they may have already identified one that suits their domain or data characteristics. The key is that this model should excel at producing vector representations where semantically similar text passages are close together in the vector space.
|
||||
- **Chat Model:** The chat model (served by LM Studio) is responsible for synthesizing an answer based on the user's query and the retrieved context. Its capabilities in instruction following, reasoning, and coherent text generation are paramount.
|
||||
|
||||
Even the most advanced chat LLM cannot produce relevant answers if the RAG pipeline provides it with irrelevant or poorly retrieved context. Conversely, excellent retrieved context can be squandered by a chat model that fails to utilize it effectively. The modular setup allows for independent optimization of these two critical model choices.
|
||||
|
||||
### 6.3. Troubleshooting Common Integration Challenges
|
||||
|
||||
Integrating multiple local AI tools can sometimes lead to issues. A systematic approach to troubleshooting is beneficial:
|
||||
|
||||
- **Connection Issues:**
|
||||
- **Symptom:** AnythingLLM reports errors connecting to Ollama or LM Studio.
|
||||
- **Checks:**
|
||||
- Verify that both Ollama and LM Studio applications/servers are running.
|
||||
- Confirm that the API base URLs entered in AnythingLLM settings (`http://127.0.0.1:11434` for Ollama, `http://localhost:1234/v1` for LM Studio) are correct and that the servers are listening on these ports.
|
||||
- Check for local firewall rules that might be blocking these connections (less common for localhost but possible).
|
||||
- Consult Ollama's connection debugging guides if issues persist with its server.
|
||||
- **Model Not Found / Incorrect Model Behavior:**
|
||||
- **Symptom:** Embeddings fail, or the chat model in AnythingLLM doesn't match the one loaded in LM Studio.
|
||||
- **Checks:**
|
||||
- For Ollama embeddings, ensure the "Embedding Model Name" in AnythingLLM _exactly_ matches the name of the model pulled and served by Ollama.
|
||||
- For LM Studio, ensure the desired chat model is loaded and selected in the LM Studio server interface. AnythingLLM usually auto-detects this.
|
||||
- **Poor RAG Performance:**
|
||||
- **Symptom:** Answers are irrelevant, nonsensical, or do not use the provided documents. This was a reported issue by a user of AnythingLLM in one instance, highlighting the need for careful configuration.
|
||||
- **Checks:**
|
||||
- **Embedding Quality:** Is the chosen embedding model appropriate for the document types and query styles?
|
||||
- **Re-embedding:** If the embedder was changed, were all documents correctly deleted and re-embedded?
|
||||
- **Chunking Strategy:** AnythingLLM's document processing settings (chunk size, overlap) can affect retrieval.
|
||||
- **Sufficient Context:** Is RAG retrieving enough relevant context for the LLM? Some tools offer ways to inspect retrieved chunks.
|
||||
- **LLM Limitations:** The chat LLM itself might have limitations in understanding or synthesizing information from the provided context.
|
||||
- **AnythingLLM Settings:** Explore workspace settings in AnythingLLM, such as "Vector database > Search Preference > Accuracy Optimized," which might improve RAG results by enabling reranking. Document pinning can also force full-text comprehension if context window permits.
|
||||
- **Resource Exhaustion:**
|
||||
- **Symptom:** System becomes very slow, applications crash, out-of-memory errors.
|
||||
- **Checks:**
|
||||
- Monitor GPU VRAM, system RAM, and CPU usage.
|
||||
- Consider using smaller or more quantized models.
|
||||
- Utilize LM Studio's GPU offloading settings to manage VRAM usage.
|
||||
- Ensure no other resource-intensive applications are running unnecessarily.
|
||||
|
||||
The following table offers a comparative perspective on using Ollama versus LM Studio as an embedding provider, which can help validate the current choice or inform future adjustments if the requirement for a _specific_ Ollama-only model changes.
|
||||
|
||||
**Table 2: Comparative Overview of Ollama and LM Studio as Embedding Providers**
|
||||
|
||||
| | | |
|
||||
|---|---|---|
|
||||
|**Feature/Consideration**|**Ollama (for Embeddings)**|**LM Studio (for Embeddings)**|
|
||||
|**Model Access & Management**|||
|
||||
|Access to HuggingFace GGUF Embedders|Good via `Modelfile` or direct pull if compatible|Strong via built-in HuggingFace browser|
|
||||
|User Interface for Model Management|Primarily CLI-centric|GUI-centric, user-friendly|
|
||||
|`Modelfile` for Custom Configurations|Strong capability for custom model definitions|Less direct, relies on pre-configured models or manual GGUF edits|
|
||||
|**Server & API**|||
|
||||
|Resource Footprint (Server for Embeddings)|Potentially lighter if only serving embeddings|Is a fuller desktop application, potentially heavier idle|
|
||||
|API Endpoint Simplicity|Native API available; Can be OpenAI compatible|Aims for OpenAI compatibility for its server 3|
|
||||
|**Ease of Use & Integration**|||
|
||||
|Setup for Dedicated Embedding Server|Straightforward for CLI users|Integrated within the broader LM Studio application|
|
||||
|Integration with AnythingLLM|Supported as a distinct embedder|Supported as a distinct embedder|
|
||||
|**Software Philosophy**|||
|
||||
|Open Source|Yes (MIT License)|No (Proprietary Freeware)|
|
||||
|
||||
This comparison highlights that both tools are capable, but their approaches and strengths differ. Ollama offers flexibility and open-source transparency, ideal for users comfortable with the CLI and potentially needing custom `Modelfile` configurations. LM Studio provides a more integrated, GUI-driven experience for model discovery and management. The choice to use Ollama for a specific embedding model is well-supported if that model is more readily available or configurable through Ollama, or if the user prefers its CLI-driven workflow for that component.
|
||||
|
||||
## 7. Conclusion: Achieving a Flexible and Powerful Local AI Workflow
|
||||
|
||||
The integration of Ollama for specialized embedding model hosting, LM Studio for serving robust local chat LLMs, and AnythingLLM as the unifying RAG platform enables the creation of a highly flexible, powerful, and private local AI workflow. This modular architecture allows users to select best-of-breed components for each part of the RAG pipeline, tailoring the system to specific needs and preferences.
|
||||
|
||||
By configuring AnythingLLM to use Ollama as its system-wide embedding provider and LM Studio as its workspace-level LLM provider, users gain fine-grained control over model selection. This setup respects the user's intent to utilize a specific embedding model via Ollama while leveraging LM Studio's user-friendly interface and serving capabilities for chat models. The key benefits of this approach include:
|
||||
|
||||
- **Precise Model Control:** The ability to choose specific embedding models through Ollama and diverse chat LLMs through LM Studio.
|
||||
- **Leveraging Tool Strengths:** Utilizing Ollama's efficient model serving and LM Studio's comprehensive model management and user interface.
|
||||
- **Data Privacy:** Maintaining a fully local stack ensures that sensitive data processed during document ingestion and chat interactions remains on the user's hardware.
|
||||
- **Customization:** The modularity allows for independent updating or swapping of components as new models or tools emerge.
|
||||
|
||||
Successful implementation requires careful attention to the configuration of each tool, particularly the API endpoints and model names in AnythingLLM, and an understanding of the resource implications of running multiple model servers concurrently. The critical step of re-embedding documents when changing the embedding provider must also be factored into any migration plan.
|
||||
|
||||
Ultimately, this integrated system empowers users to build a sophisticated local AI environment that is tailored to their exact requirements, offering a level of control and customization that is often not possible with monolithic or cloud-based solutions. Experimentation with different embedding and chat models within this established framework can further refine the system's performance and capabilities for various RAG tasks.
|
||||
237
Corpus/Various/JSON validation for Postgres.md
Normal file
237
Corpus/Various/JSON validation for Postgres.md
Normal file
|
|
@ -0,0 +1,237 @@
|
|||
---
|
||||
tags:
|
||||
- json
|
||||
- supabase
|
||||
---
|
||||
# JSON Schema validation for Postgres
|
||||
|
||||
When using the JSON (or JSONB) datatype, the data needs to be validated to assure database integrity.
|
||||
|
||||
pg_jsonschema is a PostgreSQL extension for SupaBase that can validate `json` and `jsonb` data types against a JSON Schema. The extension offers two functions:
|
||||
|
||||
|
||||
```java
|
||||
-- Validates a json *instance* against a JSON Schema *schema*
|
||||
json_matches_schema(schema json, instance json) returns bool
|
||||
|
||||
-- Validates a jsonb *instance* against a JSON Schema *schema*
|
||||
jsonb_matches_schema(schema json, instance jsonb) returns bool
|
||||
```
|
||||
|
||||
JSON Schema is a way to define what valid JSON should look like for a particular use case:
|
||||
|
||||
- What properties an object should have
|
||||
- What data types are expected
|
||||
- Which fields are required vs optional
|
||||
- Validation constraints (like minimum/maximum values, string patterns, etc.)
|
||||
- Default values and descriptions
|
||||
|
||||
A JSON Schema is itself a JSON document. Here's a simple example of a JSON schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Person's full name"
|
||||
},
|
||||
"age": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 150
|
||||
},
|
||||
"email": {
|
||||
"type": "string",
|
||||
"format": "email"
|
||||
}
|
||||
},
|
||||
"required": ["name", "email"]
|
||||
}
|
||||
```
|
||||
|
||||
You can check input against a schema in SQL like this:
|
||||
|
||||
```sql
|
||||
create table some_table(
|
||||
id serial primary key, -- db-column `id` column is an auto-incrementing primary key
|
||||
metadata json not null, -- db-column `metadata` must contain a JSON value and cannot be null
|
||||
check ( -- table-level check constraint to match the JSON in `metadata` to the schema
|
||||
json_matches_schema(
|
||||
schema :='{
|
||||
"type": "object", -- we require an object ...
|
||||
"properties": {
|
||||
"foo": { -- with a single string property `"foo"` ...
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": ["foo"], -- property `"foo"` is required ...
|
||||
"additionalProperties": false -- and no additional properties are allowed
|
||||
}',
|
||||
instance := metadata -- the value of the `metadata` column is passed ...
|
||||
-- as the `instance` argument to the `json_matches_schema` function, for each row
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
|
||||
-- Now we can attempt to insert a row into `some_table`,
|
||||
-- with the `metadata` value provided as `<SQL input>`
|
||||
|
||||
insert into some_table(metadata)
|
||||
values
|
||||
(<SQL input>);
|
||||
|
||||
-- <SQL input> needs to be replaced with an actual JSON value, e.g. '{"foo": "bar"}'.
|
||||
-- The insert will only succeed if the contents of `metadata` matches the schema in the check constraint.
|
||||
```
|
||||
|
||||
## Validating for a set of allowed values
|
||||
|
||||
Use the `enum` keyword to validate that a value must be one of a specific set of allowed values.
|
||||
|
||||
**String values:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "string",
|
||||
"enum": ["red", "green", "blue"]
|
||||
}
|
||||
```
|
||||
|
||||
**Mixed data types:**
|
||||
|
||||
```json
|
||||
{
|
||||
"enum": ["active", "inactive", null, 42]
|
||||
}
|
||||
```
|
||||
|
||||
**In an object property:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"status": {
|
||||
"type": "string",
|
||||
"enum": ["pending", "approved", "rejected"]
|
||||
},
|
||||
"priority": {
|
||||
"type": "integer",
|
||||
"enum": [1, 2, 3, 4, 5]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Validating for a data range
|
||||
|
||||
**Inclusive bounds (default):**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 10
|
||||
}
|
||||
```
|
||||
|
||||
This allows values from 1 to 10, including 1 and 10.
|
||||
|
||||
**Exclusive bounds:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "number",
|
||||
"exclusiveMinimum": 0,
|
||||
"exclusiveMaximum": 100
|
||||
}
|
||||
```
|
||||
|
||||
This allows values greater than 0 and less than 100, but not 0 or 100 themselves.
|
||||
|
||||
**Mixed Bounds:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"exclusiveMaximum": 1
|
||||
}
|
||||
```
|
||||
|
||||
This allows values from 0 (inclusive) to 1 (exclusive), so 0 ≤ value < 1.
|
||||
|
||||
**One-Sided Ranges:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "integer",
|
||||
"minimum": 18
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "number",
|
||||
"maximum": 3.14159
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
**In Object Properties:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"age": {
|
||||
"type": "integer",
|
||||
"minimum": 0,
|
||||
"maximum": 150
|
||||
},
|
||||
"temperature": {
|
||||
"type": "number",
|
||||
"minimum": -273.15,
|
||||
"maximum": 1000.0
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
**Regex Validation:**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "string",
|
||||
"pattern": "^[a-zA-Z0-9]+$"
|
||||
}
|
||||
```
|
||||
|
||||
**Date Validation:**
|
||||
|
||||
JSON Schema supports the ISO 8601 date format:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "string",
|
||||
"format": "date"
|
||||
}
|
||||
```
|
||||
|
||||
`"date"` validates dates like: `2023-12-25`
|
||||
`"date-time"` validates like: `2023-12-25T10:30:00Z` or `2023-12-25T10:30:00.123Z`
|
||||
`"time"` validates like: `10:30:00` or `10:30:00.123`
|
||||
|
||||
Using the ISO 8601 date format is recommended for interoperability.
|
||||
Custom date patterns can be validated with Regex.
|
||||
|
||||
Ranges can be validated using the `"minimum"` and `"maximum"` keywords like before.
|
||||
|
||||
## Documentation**
|
||||
- [pg_jsonschema](https://github.com/supabase/pg_jsonschema)
|
||||
- [JSON Schema](https://json-schema.org/)
|
||||
54
Corpus/Various/Key Areas of Rasa Syntax.md
Normal file
54
Corpus/Various/Key Areas of Rasa Syntax.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
tags:
|
||||
- llm
|
||||
- agent
|
||||
- dev
|
||||
---
|
||||
|
||||
# Key Areas of Rasa Syntax:
|
||||
|
||||
According to Gemini, 4 juli 2025
|
||||
|
||||
- **YAML Configuration:**
|
||||
|
||||
Rasa uses YAML files (e.g., `config.yml`, `domain.yml`, `endpoints.yml`) to define the core components of your chatbot, including the NLU pipeline, domain, and endpoint configurations.
|
||||
|
||||
- **Training Data:**
|
||||
|
||||
Training data, typically in Markdown format, is used to train the NLU model. It includes intents, entities, and examples of user input.
|
||||
|
||||
- **Stories:**
|
||||
|
||||
Stories, also in Markdown, define the dialogue flows of your chatbot, mapping user inputs to bot responses and actions.
|
||||
|
||||
- **Actions:**
|
||||
|
||||
Custom actions, written in Python, define the logic that your chatbot executes in response to user input or specific events.
|
||||
|
||||
- **Rules:**
|
||||
|
||||
Rules are a simplified way to define dialogue flows, often used for simple scenarios or to handle specific edge cases.
|
||||
|
||||
- **Forms:**
|
||||
|
||||
Forms are used to collect multiple pieces of information from the user in a structured way.
|
||||
|
||||
- **Responses:**
|
||||
|
||||
Responses define the text, buttons, or other elements that your chatbot sends back to the user.
|
||||
|
||||
- **Flows:**
|
||||
|
||||
Flows are used in Rasa Pro to define the overall structure of your conversation, including steps, conditions, and actions.
|
||||
|
||||
- **Conditions:**
|
||||
|
||||
Conditions are used to control the flow of conversation based on slot values, user input, or other factors.
|
||||
|
||||
- **Command Line Interface (CLI):**
|
||||
|
||||
Rasa provides a CLI for various tasks, including training models, running servers, and inspecting assistants.
|
||||
|
||||
- **Session Management:**
|
||||
|
||||
Rasa handles session management using session configuration, expiration times, and slot carryover.
|
||||
64
Corpus/Various/Kilo Code development workflow.md
Normal file
64
Corpus/Various/Kilo Code development workflow.md
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
tags:
|
||||
- dev
|
||||
- Kilo
|
||||
---
|
||||
If you want to build an app from scratch with Kilo Code, here’s how to use the different modes and instruct them:
|
||||
|
||||
## Where to Start
|
||||
|
||||
- **Start in Orchestrator Mode** if your project is complex or involves multiple steps (like building a full app). Orchestrator Mode will help break down your big goal into smaller, specialized subtasks and assign them to the right modes automatically[1][2].
|
||||
|
||||
## When to Use Each Mode
|
||||
|
||||
- **Orchestrator Mode:** Use this to manage the overall workflow. It coordinates tasks, delegates subtasks (like planning, coding, debugging), and keeps your main conversation focused on high-level progress[1][2].
|
||||
- **Architect Mode:** Use when you need to design your system, plan features, or make architectural decisions. Architect Mode helps you create plans, diagrams, and documentation, but won’t modify your codebase directly[1][3][4].
|
||||
- **Coder (Code) Mode:** Use for hands-on implementation—writing code, building features, and making changes to your files. This mode has full access to your project and is ideal for actual development work[3][5][4].
|
||||
- **Debugger Mode:** Use when you need to find and fix bugs. Debugger Mode specializes in troubleshooting, running diagnostics, and resolving errors in your codebase[3][5][4].
|
||||
|
||||
## How to Instruct Each Mode
|
||||
|
||||
- **Switch Modes:** You can switch modes via the dropdown menu, slash commands (e.g., `/architect`, `/code`), or keyboard shortcuts (e.g., ⌘ + . on Mac, Ctrl + . on Windows/Linux)[3].
|
||||
- **Direct Instructions:** When in a mode, just type your instructions or questions in the chat input. For example:
|
||||
- In Architect Mode: “Design a scalable backend for a note-taking app.”
|
||||
- In Code Mode: “Implement the user authentication feature.”
|
||||
- In Debug Mode: “Fix the bug causing login failures.”
|
||||
- **Orchestrator Mode:** Describe your overall goal (e.g., “Build a note organizer app with search and tagging”). Orchestrator will suggest how to break it down and automatically switch to the right modes for each subtask[1][2].
|
||||
|
||||
## Typical Workflow
|
||||
|
||||
1. **Start in Orchestrator Mode**: Describe your app idea and main requirements.
|
||||
2. **Architect Mode**: Kilo will switch to Architect Mode to help plan and design the app.
|
||||
3. **Code Mode**: After planning, Kilo will move to Code Mode to implement the plan.
|
||||
4. **Debug Mode**: Once features are built, switch to Debug Mode to test and fix any issues.
|
||||
5. **Repeat as Needed**: You can manually switch modes or let Orchestrator handle transitions for you[1][3][2].
|
||||
|
||||
This approach lets you focus on your goals while Kilo Code uses the best tools for each stage of development.
|
||||
|
||||
Sources
|
||||
[1] Orchestrator Mode is here! https://www.youtube.com/watch?v=20MmJNeOODo
|
||||
[2] Orchestrator Mode: Coordinate Complex Workflows | Kilo Code Docs https://kilocode.ai/docs/basic-usage/orchestrator-mode
|
||||
[3] Using Modes | Kilo Code Docs https://kilocode.ai/docs/basic-usage/using-modes
|
||||
[4] Kilo Code + VS Code is the Ultimate AI Coding Agent For Beginners https://www.youtube.com/watch?v=o7aQhb-PK08
|
||||
[5] Architect, Code, Debug, Ask: Kilo Code's AI Modes Explained https://www.youtube.com/watch?v=cS4vQfX528w
|
||||
[6] Architect, Code, Debug, Ask: Kilo Code's AI Modes Explained https://www.linkedin.com/posts/kilo-code_architect-code-debug-ask-kilo-codes-activity-7321248999125012480-fAJl
|
||||
[7] Kilo Code - Introducing Orchestrator Mode https://www.linkedin.com/posts/kilo-code_how-kilo-codes-orchestrator-mode-changed-activity-7325537004715196416-eNKD
|
||||
[8] How Kilo Code's Orchestrator Mode Changed My ... https://substack.com/home/post/p-162706015
|
||||
[9] Kilo Code 4.19.1: Orchestrator Mode is here! https://blog.kilocode.ai/p/kilo-code-4191-orchestrator-mode
|
||||
[10] Kilo in 100 Seconds https://www.youtube.com/watch?v=MfuCYNaPWTQ
|
||||
[11] Kilo Code 4.19.1: Orchestrator Mode is here! : r/kilocode https://www.reddit.com/r/kilocode/comments/1kbhiba/kilo_code_4191_orchestrator_mode_is_here/
|
||||
[12] Kilo Code AI Agent (Cline / Roo features combined) https://marketplace.visualstudio.com/items?itemName=kilocode.Kilo-Code
|
||||
[13] Kilo Code on Instagram: "Orchestrator Mode is now available https://www.instagram.com/reel/DJIRxYBKstn/
|
||||
[14] Kilo Code - Open source AI agent VS Code extension https://kilocode.ai
|
||||
[15] I deploy an app created using kilocode - Reddit https://www.reddit.com/r/kilocode/comments/1l33ysy/i_deploy_an_app_created_using_kilocode/
|
||||
[16] Build Your Own Text Editor https://viewsourcecode.org/snaptoken/kilo/
|
||||
[17] Kilo Code, The AI Coding Genius That Outshines Cline & ... https://apidog.com/blog/kilo-code/
|
||||
[18] Kilo: A text editor in less than 1000 LOC with syntax highlight and ... https://news.ycombinator.com/item?id=44034459
|
||||
[19] Does Kilo Code have an auto model selection like Perplexity? - Reddit https://www.reddit.com/r/kilocode/comments/1knpit0/does_kilo_code_have_an_auto_model_selection_like/
|
||||
[20] r/kilocode - Reddit https://www.reddit.com/r/kilocode/
|
||||
[21] give it your prompt and watch it… | Kilo Code https://www.linkedin.com/posts/kilo-code_orchestrator-mode-is-now-available-give-activity-7323802816861376512-sILR
|
||||
[22] KILO CODE BUILD ANY WEB APP - YouTube https://www.youtube.com/watch?v=ZD_HdBaTvQk
|
||||
[23] Kilo Code Documentation | Kilo Code Docs https://kilocode.ai/docs/
|
||||
[24] Kilo Code - Coding agents - TAAFT https://theresanaiforthat.com/ai/kilo-code/
|
||||
[25] Best VSCode Coding Setup? Kilo Code + Sonnet 3.7 (With FREE ... https://www.youtube.com/watch?v=_UbmP08SxsY
|
||||
[26] Debug with GitHub Copilot - Visual Studio Code https://code.visualstudio.com/docs/copilot/guides/debug-with-copilot
|
||||
70
Corpus/Various/Knowledge Graph Databases.md
Normal file
70
Corpus/Various/Knowledge Graph Databases.md
Normal file
|
|
@ -0,0 +1,70 @@
|
|||
---
|
||||
tags:
|
||||
- graphDB
|
||||
- dev
|
||||
- dbms
|
||||
---
|
||||
|
||||
# Knowledge Graph Databases
|
||||
|
||||
https://gemini.google.com/app/ad92194f61ec257a
|
||||
|
||||
Building a Proof of Concept (POC) for a Knowledge Graph in Information Security and Compliance on a Mac Mini M2, with an eye towards LLM integration, requires a robust, well-maintained, and performant open-source property graph database.
|
||||
|
||||
Here are the top contenders, focusing on your specific criteria:
|
||||
|
||||
### Top Choices for Your POC
|
||||
|
||||
1. **Neo4j Community Edition**
|
||||
|
||||
- **Pros:**
|
||||
- **Gold Standard:** Neo4j is arguably the most widely recognized and mature property graph database. It has a massive community, extensive documentation, and a wealth of learning resources (GraphAcademy).
|
||||
- **Cypher Query Language:** Cypher is a highly intuitive and powerful declarative query language specifically designed for property graphs. It's excellent for expressing complex patterns and traversals, which is ideal for security and compliance use cases (e.g., "find all assets vulnerable to a specific threat, mitigated by a weak control, and subject to GDPR").
|
||||
- **M2 Compatibility:** Neo4j Desktop (which includes the Community Edition) runs natively on macOS, including Apple Silicon (M1/M2/M3), making installation and local development very smooth. You can also run it via Docker.
|
||||
- **Strong LLM Integration Ecosystem:** Neo4j has heavily invested in LLM integrations. They have a dedicated "GraphRAG" initiative, LangChain integrations, and many tutorials showing how to use LLMs for knowledge graph construction (extracting entities and relationships from unstructured text) and for generating natural language queries from human questions. This ecosystem is probably the most mature for LLM interaction.
|
||||
- **Visualization:** Neo4j Browser provides excellent built-in visualization, which is crucial for understanding complex security relationships in a knowledge graph.
|
||||
- **Cons:**
|
||||
- **Licensing:** The Community Edition is open source under GPLv3, but some advanced features (like clustering for high availability/scalability) are only in the Enterprise Edition (commercial license). For a POC, this isn't usually an issue.
|
||||
|
||||
2. **Memgraph Community Edition**
|
||||
|
||||
- **Pros:**
|
||||
- **High Performance (In-Memory):** Memgraph is an in-memory graph database, making it exceptionally fast for queries and real-time analytics. This could be a significant advantage for dynamic security analysis or incident response simulations.
|
||||
- **OpenCypher Support:** It supports OpenCypher, making it compatible with Neo4j's powerful query language. This allows for easy migration if you later decide to switch or leverage existing Cypher knowledge.
|
||||
- **M2 Compatibility:** Runs well on macOS (including M2), often installed via `brew`.
|
||||
- **Good LLM Integration:** Memgraph has actively promoted its use in GraphRAG applications and offers integrations with tools like LangChain for LLM interaction, similar to Neo4j.
|
||||
- **MAGE (Memgraph Advanced Graph Extensions):** Provides a library of graph algorithms that can be very useful for advanced security analytics (e.g., finding central assets, identifying attack paths).
|
||||
- **Cons:**
|
||||
- **In-Memory (for POC):** While fast, being primarily in-memory means you'll need to be mindful of the Mac Mini's RAM limitations for very large datasets in a long-running POC. However, for a typical POC scope, this is often manageable. Persistent storage is supported, but the core strength is in-memory.
|
||||
- **Smaller Community:** While growing, its community is smaller than Neo4j's.
|
||||
|
||||
3. **Apache AGE (PostgreSQL Extension)**
|
||||
|
||||
- **Pros:**
|
||||
- **Leverages PostgreSQL:** If you're already familiar with PostgreSQL or want to keep your graph data alongside relational data, AGE is a compelling choice. It offers the ACID compliance and robustness of Postgres.
|
||||
- **Cypher-like Querying:** It provides a Cypher-like query language (called `age_graph_query`) within SQL, allowing you to run graph queries directly on your Postgres database.
|
||||
- **M2 Compatibility:** As a PostgreSQL extension, it runs wherever PostgreSQL runs, including natively on Mac Mini M2.
|
||||
- **Apache Project:** Being an Apache project means it's community-driven and open-source with a clear governance model.
|
||||
- **Cons:**
|
||||
- **Less Native Graph Performance:** While good, it's an extension to a relational database, so its raw graph traversal performance might not match native graph databases like Neo4j or Memgraph for extremely deep or complex traversals.
|
||||
- **LLM Integration Maturity:** While you can certainly integrate it with LLMs via custom code (e.g., using LangChain and a PostgreSQL connector), the specific "GraphRAG" tools and tutorials might be less direct compared to Neo4j's dedicated ecosystem. You'd be building more of that integration yourself.
|
||||
|
||||
### Key Considerations for Your POC
|
||||
|
||||
- **Ease of Setup/Local Development on Mac Mini M2:** All options listed are generally easy to install via Homebrew or Docker on Apple Silicon.
|
||||
- **Query Language:** Both Neo4j and Memgraph use (or support) Cypher, which is excellent for security knowledge graphs. Apache AGE uses a Cypher-like syntax.
|
||||
- **LLM Integration (MPC Server):**
|
||||
- **Data Extraction:** LLMs can be used to extract entities (assets, vulnerabilities, controls, regulations) and relationships from unstructured text (e.g., security reports, compliance documents, threat intelligence). Your chosen graph database will then store this structured data.
|
||||
- **Retrieval Augmented Generation (RAG):** This is where the "MPC server" comes in. Your LLM application will query the knowledge graph to retrieve relevant context (e.g., "what controls mitigate this vulnerability for these assets?") and feed that context to the LLM to generate more accurate and grounded responses (e.g., "Here's how to address CVE-X affecting your web server, according to NIST CSF control AC-3").
|
||||
- **Text-to-Graph/Graph-to-Text:** LLMs can also help translate natural language questions into graph queries (e.g., "show me my compliance posture for PCI DSS" -> Cypher query) and translate graph query results back into human-readable text.
|
||||
- **Vector Search:** Many modern graph databases are integrating vector capabilities. This allows you to store embeddings (numerical representations) of nodes and relationships, enabling semantic search and similarity matching, which is very useful for LLM applications.
|
||||
|
||||
**Recommendation for POC:**
|
||||
|
||||
For a POC on a Mac Mini M2 with a strong focus on LLM integration for InfoSec/Compliance, **Neo4j Community Edition** is often the safest and most feature-rich choice.
|
||||
|
||||
- Its mature ecosystem, dedicated GraphRAG tools, and intuitive Cypher language will significantly accelerate your development and experimentation with LLMs.
|
||||
- The community support and wealth of online resources mean you'll likely find solutions to problems quickly.
|
||||
- The visualization capabilities are excellent for demonstrating the value of the knowledge graph to stakeholders.
|
||||
|
||||
If you hit performance bottlenecks with Neo4j (which is unlikely for a typical POC unless your dataset is truly massive), or if you have a strong existing PostgreSQL ecosystem you want to leverage, then **Memgraph** (for raw speed) or **Apache AGE** (for PostgreSQL integration) are excellent alternatives.
|
||||
60
Corpus/Various/LLM Prompt types.md
Normal file
60
Corpus/Various/LLM Prompt types.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
tags:
|
||||
- llm/prompting
|
||||
- agent
|
||||
---
|
||||
|
||||
System prompts are instructions given to a large language model (LLM) that define how it should behave, respond, and interact with users. They act as a foundational set of guidelines that shape the model’s personality, tone, capabilities, and constraints before any user conversation begins.
|
||||
|
||||
Think of system prompts as the “rules of engagement” for an AI assistant. They typically include:
|
||||
|
||||
**Core behavioral guidelines** - How the AI should communicate (formal vs casual, helpful vs neutral), what personality traits to exhibit, and how to structure responses.
|
||||
|
||||
**Capability definitions** - What tasks the model can and cannot perform, what tools it has access to, and how to use those tools appropriately.
|
||||
|
||||
**Safety and ethical constraints** - Boundaries around harmful content, privacy considerations, and responsible AI use. This includes guidelines about not generating illegal content, protecting user privacy, and avoiding bias.
|
||||
|
||||
**Contextual information** - Background knowledge about the AI’s purpose, current date/time, and any specific domain expertise it should demonstrate.
|
||||
|
||||
**Response formatting** - Instructions about how to structure answers, when to use bullet points vs paragraphs, how to cite sources, and other stylistic preferences.
|
||||
|
||||
**Tool instructions**
|
||||
System prompts can specify when and how the AI can use tools, such as accessing external APIs or running code.
|
||||
|
||||
|
||||
For example, a system prompt might instruct an AI to “be helpful and informative while maintaining a friendly tone” and “always cite sources when providing factual information” and “refuse requests that could cause harm.”
|
||||
|
||||
The system prompt essentially programs the AI’s default behavior before it ever sees a user’s question, making it a crucial component in creating consistent, safe, and useful AI interactions. Users typically don’t see the system prompt directly, but its effects are evident in how the AI responds and behaves throughout the conversation.
|
||||
|
||||
## Other types of prompts
|
||||
Yes, there are several different types of prompts used with LLMs beyond system prompts:
|
||||
|
||||
**User prompts** are the most common type - these are the actual questions, requests, or instructions that users send to the AI during a conversation. Every message you send me is a user prompt.
|
||||
|
||||
**Assistant prompts** refer to the AI’s previous responses in a conversation, which become part of the context for subsequent exchanges. The AI considers its own prior responses when generating new ones.
|
||||
|
||||
**Few-shot prompts** provide examples of the desired input-output behavior before asking the AI to perform a task. For instance, showing the AI a few examples of how to format data before asking it to format new data in the same way.
|
||||
|
||||
**Zero-shot prompts** ask the AI to perform a task without any examples, relying entirely on the model’s training to understand what’s needed.
|
||||
|
||||
**Chain-of-thought prompts** explicitly ask the AI to show its reasoning process step-by-step, often using phrases like “think step by step” or “explain your reasoning.”
|
||||
|
||||
**Role-playing prompts** ask the AI to adopt a specific persona or expertise, like “act as a financial advisor” or “respond as if you’re a medieval historian.”
|
||||
|
||||
**Template or structured prompts** use specific formats with placeholders, often used in applications where consistent output formatting is important.
|
||||
|
||||
**Multi-modal prompts** combine text with other inputs like images, audio, or documents when the AI supports those capabilities.
|
||||
|
||||
**Contextual prompts** include relevant background information or documents to help the AI understand the specific context for a task.
|
||||
|
||||
The key distinction is that system prompts set the overall behavioral framework, while these other prompt types are used during actual interactions to guide specific responses or task performance.
|
||||
|
||||
## Examples
|
||||
4. Examples and Repositories:
|
||||
|
||||
- `danielrosehill/System-Prompt-Library`: A curated collection of system prompts for diverse AI applications.
|
||||
- `jujumilk3/leaked-system-prompts`: A repository containing leaked system prompts from various AI tools.
|
||||
- `ncwilson78/System-Prompt-Library`: A library of system prompts, including those for multi-agent systems.
|
||||
- `x1xhlol/system-prompts-and-models-of-ai-tools`: A repository with system prompts for Cursor, Devin, Junie, and other AI tools.
|
||||
- `LouisShark/chatgpt_system_prompt`: A collection of system prompts for ChatGPT and custom GPTs.
|
||||
- `asgeirtj/system_prompts_leaks`: A collection of extracted system prompts from chatbots like ChatGPT, Claude & Gemini.
|
||||
105
Corpus/Various/MCP Servers for Markdown.md
Normal file
105
Corpus/Various/MCP Servers for Markdown.md
Normal file
|
|
@ -0,0 +1,105 @@
|
|||
---
|
||||
tags:
|
||||
- llm
|
||||
- mcp
|
||||
- markdown
|
||||
---
|
||||
|
||||
Perplexity, 13 juni 2025
|
||||
|
||||
Prompt: `looking for an mcp server that can interact with markdown files on my MacOS filesystem`
|
||||
|
||||
## Recommended MCP Servers for Markdown Files on macOS
|
||||
|
||||
|
||||
- **library-mcp**
|
||||
|
||||
- **Purpose:** Specifically designed for interacting with Markdown knowledge bases (folders and subfolders containing `.md` files).
|
||||
|
||||
- **Features:** Query recent content by tag or date, summarize posts, and operate across multiple Markdown folders.
|
||||
|
||||
- **Platform:** Intended for local use, including macOS.
|
||||
|
||||
- **How to Use:** Run the server locally and point it to your Markdown directories[^1_6].
|
||||
|
||||
- **Filesystem MCP / File-system-mcp-server**
|
||||
|
||||
- **Purpose:** Provides a structured API for file and directory operations, including reading, writing, and searching Markdown files.
|
||||
|
||||
- **Features:** Read, write, search, and manage files and directories. Can be configured to use Python and runs on macOS.
|
||||
|
||||
- **Platform:** Cross-platform, with macOS support.
|
||||
|
||||
- **How to Use:** Add configuration to your MCP client (e.g., Cursor, Claude Desktop) and specify the path to the server script[^1_8][^1_9].
|
||||
|
||||
- **Markdownify MCP**
|
||||
|
||||
- **Purpose:** Converts various file types (including Markdown) and web content to Markdown, and can retrieve existing Markdown files.
|
||||
|
||||
- **Features:** Retrieve, convert, and process Markdown files.
|
||||
|
||||
- **Platform:** Runs on Node.js, compatible with macOS.
|
||||
|
||||
- **How to Use:** Configure server settings in your application and set the `MD_SHARE_DIR` environment variable to restrict directory access[^1_1].
|
||||
|
||||
## Quick Comparison
|
||||
|
||||
| Server Name | Main Use Case | macOS Support | File Operations | Markdown-Specific Features |
|
||||
| :-------------- | :------------------------ | :------------ | :-------------- | :------------------------- |
|
||||
| library-mcp | Knowledge base management | Yes | Yes | Query by tag/date |
|
||||
| Filesystem MCP | General file operations | Yes | Yes | Read/write/search files |
|
||||
| Markdownify MCP | File conversion/retrieval | Yes | Yes | Retrieve/convert Markdown |
|
||||
|
||||
|
||||
## How to Get Started
|
||||
|
||||
- **For library-mcp:** Clone the repository and run it locally, pointing it to your Markdown folders[^1_6].
|
||||
|
||||
- **For Filesystem MCP:** Add the server configuration to your MCP client and specify the path to the Python script[^1_9].
|
||||
|
||||
- **For Markdownify MCP:** Add the server configuration to your app and set environment variables as needed[^1_1].
|
||||
|
||||
|
||||
|
||||
All three options are suitable for macOS and can be integrated into your workflow for interacting with Markdown files on your filesystem.
|
||||
|
||||
|
||||
|
||||
<div style="text-align: center">⁂</div>
|
||||
|
||||
|
||||
|
||||
[^1_1]: https://github.com/zcaceres/markdownify-mcp
|
||||
|
||||
|
||||
|
||||
[^1_2]: https://mcp.so
|
||||
|
||||
|
||||
|
||||
[^1_3]: https://github.com/punkpeye/awesome-mcp-servers
|
||||
|
||||
|
||||
|
||||
[^1_4]: https://www.reddit.com/r/bioinformatics/comments/11s5d02/good_free_application_for_reading_md_files_on_mac/
|
||||
|
||||
|
||||
|
||||
[^1_5]: https://huggingface.co/blog/lynn-mikami/markitdown-mcp
|
||||
|
||||
|
||||
|
||||
[^1_6]: https://github.com/lethain/library-mcp
|
||||
|
||||
|
||||
|
||||
[^1_7]: https://www.claudemcp.com/servers/markitdown-mcp
|
||||
|
||||
|
||||
|
||||
[^1_8]: https://www.firecrawl.dev/blog/best-mcp-servers-for-cursor
|
||||
|
||||
|
||||
|
||||
[^1_9]: https://langdb.ai/app/mcp-servers/file-system-mcp-server-0ba7198c-f015-4282-8a8c-e80fc44de59f
|
||||
|
||||
79
Corpus/Various/MCP server for Obsidian Vault.md
Normal file
79
Corpus/Various/MCP server for Obsidian Vault.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
|
||||
---
|
||||
tags:
|
||||
- llm
|
||||
- mcp
|
||||
- obsidian
|
||||
---
|
||||
|
||||
Perplexity, 13 juni 2025
|
||||
|
||||
Prompt: `is there an MCP server that is particularly apt to interacting with my Obsidian vault? One that can work with the YAML metadata and internal links
|
||||
|
||||
Several MCP (Model Context Protocol) servers are specifically designed to interact with Obsidian vaults, supporting advanced features like YAML frontmatter (metadata) handling and internal link management.
|
||||
## Recommended MCP Servers for Obsidian
|
||||
|
||||
**1. cyanheads/obsidian-mcp-server**
|
||||
|
||||
- **YAML Metadata Support**: This server offers robust property management, including YAML frontmatter parsing, intelligent merging, and custom field support. This means it can read, modify, and manage metadata fields in your notes seamlessly[^1_2][^1_4][^1_7].
|
||||
|
||||
- **Internal Link Handling**: It supports comprehensive search capabilities, including full-text search, glob patterns, and frontmatter field queries. These features are well-suited for navigating and managing internal links within your vault[^1_2][^1_4][^1_7].
|
||||
|
||||
- **Security \& Performance**: Includes API key authentication, rate limiting, and SSL options for secure and efficient operation[^1_2][^1_4][^1_7].
|
||||
|
||||
- **Requirements**: Needs the Obsidian Local REST API plugin enabled and Node.js installed[^1_2][^1_7].
|
||||
|
||||
|
||||
**2. natestrong/obsidian-mcp**
|
||||
|
||||
- **YAML and Tag Management**: This server provides tools for reading, creating, searching, and managing notes, with explicit support for tag management in frontmatter (YAML)[^1_6].
|
||||
|
||||
- **Link Analysis**: Offers note insights, including link analysis, which helps in understanding and traversing internal links in your vault[^1_6].
|
||||
|
||||
- **Smart Search**: Allows searching by content, tags, or modification date, making it apt for metadata-driven workflows[^1_6].
|
||||
|
||||
- **Requirements**: Requires the Local REST API plugin, Python 3.10+, and optionally Node.js for inspection tools[^1_6].
|
||||
|
||||
|
||||
**3. jacksteamdev/obsidian-tools**
|
||||
|
||||
- **Semantic Search \& Templates**: Enables semantic searches and template execution, working with your vault’s structure and metadata[^1_5].
|
||||
|
||||
- **Automated Setup**: Installation is streamlined via the plugin’s settings, making it convenient for users who prefer minimal manual configuration[^1_5].
|
||||
|
||||
- **Security**: Publishes binaries with cryptographic provenance for added trust[^1_5].
|
||||
|
||||
|
||||
## Features Comparison Table
|
||||
|
||||
| Server Name | YAML Metadata Support | Internal Link Handling | Search/Query Features | Requirements |
|
||||
| :---------------------------- | :-------------------- | :--------------------- | :--------------------------- | :---------------------------------- |
|
||||
| cyanheads/obsidian-mcp-server | Yes | Yes | Full-text, glob, frontmatter | Node.js, Local REST API plugin |
|
||||
| natestrong/obsidian-mcp | Yes | Yes | Content, tags, link analysis | Python 3.10+, Local REST API plugin |
|
||||
| jacksteamdev/obsidian-tools | Yes | Yes | Semantic, template-based | Plugin install, Local REST API |
|
||||
|
||||
|
||||
|
||||
## Key Points
|
||||
|
||||
- All these servers require the Obsidian Local REST API plugin to enable external access to your vault[^1_2][^1_4][^1_6][^1_7].
|
||||
- They provide comprehensive support for YAML metadata and internal links, making them well-suited for advanced Obsidian workflows.
|
||||
- Security features such as API key authentication and rate limiting are standard.
|
||||
|
||||
|
||||
**Conclusion:**
|
||||
|
||||
The cyanheads/obsidian-mcp-server is particularly robust for YAML metadata and internal link management, but both natestrong/obsidian-mcp and jacksteamdev/obsidian-tools are also strong choices, depending on your environment and preferred setup style[^1_2][^1_4][^1_5][^1_6][^1_7].
|
||||
|
||||
---
|
||||
|
||||
[^1_1]: https://apidog.com/blog/obsidian-mcp-server/
|
||||
[^1_2]: https://github.com/cyanheads/obsidian-mcp-server/blob/main/README.md
|
||||
[^1_3]: https://mcpmarket.com/server/obsidian-5
|
||||
[^1_4]: https://github.com/cyanheads/obsidian-mcp-server
|
||||
[^1_5]: https://playbooks.com/mcp/jacksteamdev-obsidian-tools
|
||||
[^1_6]: https://github.com/natestrong/obsidian-mcp
|
||||
[^1_7]: https://ubos.tech/mcp/obsidian-mcp-server-2/
|
||||
[^1_8]: https://github.com/MarkusPfundstein/mcp-obsidian
|
||||
[^1_9]: https://forum.obsidian.md/t/obsidian-mcp-servers-experiences-and-recommendations/99936
|
||||
[^1_10]: https://github.com/StevenStavrakis/obsidian-mcp
|
||||
45
Corpus/Various/MCP server for creating abstracts.md
Normal file
45
Corpus/Various/MCP server for creating abstracts.md
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
tags:
|
||||
- llm
|
||||
- mcp
|
||||
---
|
||||
Perplexity, 13 juni 2025
|
||||
|
||||
Prompt: `Is there an MCP server that is particularly good at creating abstracts for documents in PDF format and assigning them to one of a number of predefined categories?
|
||||
|
||||
Currently, there is no widely known MCP server explicitly specialized in creating abstracts (summaries) of PDF documents and categorizing them into predefined classes out-of-the-box. However, the Model Context Protocol (MCP) framework is designed to enable developers to build such functionality by connecting AI models to external data sources and tools securely and flexibly[^2_1][^2_3].
|
||||
|
||||
### How to Approach This with MCP
|
||||
- **PDF Processing:** You can build or use an MCP server that extracts text from PDFs (using libraries like PyPDF2 or pdfplumber on macOS) and exposes the extracted content as a resource to the AI model[^2_6].
|
||||
- **Abstract Creation:** The AI model (e.g., Claude 3.5 or GPT) connected via MCP can then generate abstracts or summaries from the extracted text by calling appropriate prompt templates or tools defined in the MCP server[^2_3][^2_6].
|
||||
- **Categorization:** The server or client can implement prompt templates or model-controlled tools that classify the summarized content into your predefined categories, leveraging the AI’s natural language understanding capabilities[^2_3][^2_6].
|
||||
|
||||
### Existing MCP Servers and Tools
|
||||
|
||||
- MCP servers for general file and document handling exist (e.g., filesystem MCP servers), but none are explicitly focused on PDF summarization + classification as a turnkey solution[^2_2].
|
||||
- You would likely need to develop a custom MCP server or extend an existing one to:
|
||||
- Extract PDF text
|
||||
- Provide summarization and classification prompts/tools
|
||||
- Return abstracts and category assignments to the client app
|
||||
|
||||
### Summary
|
||||
|
||||
The MCP ecosystem is designed to enable exactly this kind of integration, but no prebuilt MCP server specifically optimized for PDF abstracting and categorization is publicly documented yet. You can build one by combining PDF text extraction with AI prompt templates for summarization and classification, exposed via MCP’s standardized API[^2_1][^2_3][^2_6].
|
||||
|
||||
If you want a ready-made solution, consider checking MCP community repositories for new servers or building a custom MCP server tailored to your PDF workflow.
|
||||
|
||||
---
|
||||
[^2_1]: https://www.anthropic.com/news/model-context-protocol
|
||||
[^2_2]: https://github.com/modelcontextprotocol/servers
|
||||
[^2_3]: https://www.philschmid.de/mcp-introduction
|
||||
[^2_4]: https://arxiv.org/pdf/2503.23278.pdf
|
||||
[^2_5]: https://experienceleague.adobe.com/en/docs/workfront-fusion/using/references/apps-and-their-modules/tools-and-transformers/model-context-protocol-mcp-connector
|
||||
[^2_6]: https://modelcontextprotocol.io/docs/concepts/resources
|
||||
[^2_7]: https://www.siddharthbharath.com/ultimate-guide-to-model-context-protocol-part-1-what-is-mcp/
|
||||
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
Loading…
Add table
Add a link
Reference in a new issue