Cleaning up the Sparks folder
This commit is contained in:
parent
eb610a79b6
commit
96cd8fea7b
78 changed files with 149 additions and 181 deletions
|
|
@ -1,314 +0,0 @@
|
|||
---
|
||||
tags:
|
||||
- project/iso27DIY
|
||||
- dev
|
||||
- llm
|
||||
- agent
|
||||
---
|
||||
|
||||
[orq](https://orq.ai/blog/llm-tools)
|
||||
|
||||
[
|
||||
|
||||
15 Best LLM Tools for AI Product Development \[2025 Guide\]
|
||||
|
||||
](https://orq.ai/blog/llm-tools)[mercity.ai](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)
|
||||
|
||||
[
|
||||
|
||||
Comprehensive Guide to Integrating Tools and APIs with Language...
|
||||
|
||||
](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)[DataCamp](https://www.datacamp.com/blog/llmops-tools)
|
||||
|
||||
[
|
||||
|
||||
Top 15 LLMOps Tools for Building AI Applications in 2025 - DataCamp
|
||||
|
||||
](https://www.datacamp.com/blog/llmops-tools)[softwareag](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)
|
||||
|
||||
[
|
||||
|
||||
LLM Tool Usage & API Integration - Software AG
|
||||
|
||||
](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)
|
||||
|
||||
To instruct an LLM about available tools and how to call them, you typically follow a structured approach:
|
||||
|
||||
**1\. Define Available Tools**
|
||||
|
||||
- Provide the LLM with a list of tools, each including:
|
||||
- A name (identifier)
|
||||
- A description (explains when to use it)
|
||||
- A schema (usually JSON) describing input parameters and their types [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**2\. Inform the LLM via Prompting**
|
||||
|
||||
- When sending a user query, include the tool definitions in the prompt or system message. This allows the LLM to understand what tools are available and how to use them [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**3\. Tool Calling Process**
|
||||
|
||||
- The LLM receives the user prompt and tool list.
|
||||
- It decides if a tool is needed, selects the appropriate one, and formats a tool call (often as a structured JSON object with arguments) [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
- Your application receives this tool call, executes the corresponding function or API, and returns the result to the LLM.
|
||||
- The LLM integrates the tool’s output into its next response, continuing the conversation naturally [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**4\. Training or Fine-Tuning (Optional)**
|
||||
|
||||
- For more advanced usage, you can fine-tune the LLM with example data showing how to use each tool, or use models already trained for tool use (like some recent open-weight models) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
|
||||
|
||||
**Summary Table**
|
||||
|
||||
| Step | Action |
|
||||
| --- | --- |
|
||||
| Define tools | Name, description, input schema |
|
||||
| Prompt LLM | Include tool definitions in prompt/system message |
|
||||
| Tool call & execution | LLM outputs tool call; app executes it and returns result |
|
||||
| Integrate response | LLM uses tool output in conversation |
|
||||
| (Optional) Fine-tuning | Train LLM on tool usage examples for better performance |
|
||||
|
||||
This approach enables LLMs to use external tools flexibly and accurately within applications [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools).
|
||||
|
||||
I’m using Anything LLM. How can I enable a model to run shell scripts?[GitHub](https://github.com/Mintplex-Labs/anything-llm/issues/3451)
|
||||
|
||||
[
|
||||
|
||||
\[FEAT\]: Shell/Script invocation block for flow builder #3451 - GitHub
|
||||
|
||||
](https://github.com/Mintplex-Labs/anything-llm/issues/3451)[reddit.com](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)
|
||||
|
||||
[
|
||||
|
||||
Can I integrate my local LLM to enable it to run system commands...
|
||||
|
||||
](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)[GitHub](https://github.com/regnull/how.sh)
|
||||
|
||||
[
|
||||
|
||||
how.sh - Create and execute shell commands using LLM - GitHub
|
||||
|
||||
](https://github.com/regnull/how.sh)[anythingllm](https://anythingllm.com/)
|
||||
|
||||
[
|
||||
|
||||
AnythingLLM | The all-in-one AI application for everyone
|
||||
|
||||
](https://anythingllm.com/)
|
||||
|
||||
To enable shell script execution in AnythingLLM, you can implement custom agent skills through these methods:
|
||||
|
||||
**1\. Custom Agent Skill Development**
|
||||
|
||||
- Create a `plugin.json` file defining your shell script entrypoint:
|
||||
```
|
||||
json
|
||||
{
|
||||
"name": "shell_runner",
|
||||
"description": "Execute shell commands",
|
||||
"entrypoint": "./handler.js",
|
||||
"permissions": ["shell"]
|
||||
}
|
||||
```
|
||||
- Develop a handler file (e.g., `handler.js`) using Node.js' `child_process` module [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux):
|
||||
```
|
||||
javascript
|
||||
const { exec } = require('child_process');
|
||||
module.exports = async ({ command }) => {
|
||||
return new Promise((resolve, reject) => {
|
||||
exec(command, (error, stdout, stderr) => {
|
||||
if (error) reject(error);
|
||||
resolve(stdout || stderr);
|
||||
});
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
**2\. Security Considerations**
|
||||
|
||||
- Restrict allowed commands through allow-listing in your handler
|
||||
- Implement user permission tiers for script execution
|
||||
- Use environment sandboxing for dangerous operations [6](https://news.ycombinator.com/item?id=35172486)
|
||||
|
||||
**3\. Integration Approaches**
|
||||
|
||||
- **LangChain Agent Method** (shown in tutorial 5):
|
||||
```
|
||||
python
|
||||
from langchain.agents import initialize_agent
|
||||
from langchain.tools import ShellTool
|
||||
shell_tool = ShellTool()
|
||||
agent = initialize_agent([shell_tool], llm, agent="zero-shot-react-description")
|
||||
```
|
||||
- **Ollama Local Model Integration** [3](https://github.com/regnull/how.sh):
|
||||
```
|
||||
bash
|
||||
ollama pull deepseek-coder-v2
|
||||
llm -m ollama/deepseek-coder-v2 "convert all .txt files to .md" > convert.sh
|
||||
```
|
||||
|
||||
**4\. Execution Flow**
|
||||
|
||||
1. User requests shell operation through chat interface
|
||||
2. LLM processes request and generates structured tool call
|
||||
3. Handler executes the command via Node.js `exec()` [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux)
|
||||
4. Output is returned to LLM for response formatting
|
||||
|
||||
**Example Use Case** (from GitHub issue [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451)):
|
||||
|
||||
```
|
||||
bash
|
||||
# plugin.json entry
|
||||
"entrypoint": "python -m browser_automation.py"
|
||||
```
|
||||
|
||||
**Limitations & Risks**
|
||||
|
||||
- Requires careful input sanitization to prevent injection attacks
|
||||
- Windows support limited due to shell differences 5
|
||||
- Execution environment must match user's system capabilities [2](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/) [6](https://news.ycombinator.com/item?id=35172486)
|
||||
|
||||
For production use, consider implementing:
|
||||
|
||||
- Command validation middleware
|
||||
- Output sanitization filters
|
||||
- Session-based execution isolation
|
||||
- Activity logging for audit trails [6](https://news.ycombinator.com/item?id=35172486) [7](https://docs.anythingllm.com/installation-desktop/linux)
|
||||
|
||||
What are the best methods to integrate local executables with Anything LLM for automation[semaphoreci](https://semaphoreci.com/blog/local-llm)
|
||||
|
||||
[
|
||||
|
||||
6 Ways to Run LLMs Locally (also how to use HuggingFace)
|
||||
|
||||
](https://semaphoreci.com/blog/local-llm)[Stream](https://getstream.io/blog/best-local-llm-tools/)
|
||||
|
||||
[
|
||||
|
||||
The 6 Best LLM Tools To Run Models Locally - GetStream.io
|
||||
|
||||
](https://getstream.io/blog/best-local-llm-tools/)[apidog](https://apidog.com/blog/top-llm-local-tools)
|
||||
|
||||
[
|
||||
|
||||
The 5 Best LLM Tools To Run Models Locally - Apidog
|
||||
|
||||
](https://apidog.com/blog/top-llm-local-tools)[anythingllm](https://anythingllm.com/)
|
||||
|
||||
[
|
||||
|
||||
AnythingLLM | The all-in-one AI application for everyone
|
||||
|
||||
](https://anythingllm.com/)
|
||||
|
||||
To integrate local executables with AnythingLLM for automation, use these methods:
|
||||
|
||||
## 1\. Custom Plugin Development
|
||||
|
||||
Create executable-wrapping plugins using AnythingLLM's extensible architecture:
|
||||
|
||||
```
|
||||
javascript
|
||||
// plugin.json
|
||||
{
|
||||
"name": "pdf_converter",
|
||||
"description": "Converts DOCX to PDF using LibreOffice",
|
||||
"entrypoint": "./convert.js",
|
||||
"permissions": ["file_system"]
|
||||
}
|
||||
|
||||
// convert.js
|
||||
const { execSync } = require('child_process');
|
||||
|
||||
module.exports = async ({ inputPath }) => {
|
||||
try {
|
||||
execSync(\`soffice --convert-to pdf "${inputPath}" --headless\`);
|
||||
return { success: true };
|
||||
} catch (error) {
|
||||
return { error: error.message };
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Key Features**:
|
||||
|
||||
- Direct CLI execution via Node.js `child_process` [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)
|
||||
- File system access controls through permission tiers
|
||||
- Error handling integration with LLM feedback loops
|
||||
|
||||
---
|
||||
|
||||
## 2\. LangChain Agent Integration
|
||||
|
||||
Leverage LangChain's tool ecosystem for executable orchestration:
|
||||
|
||||
```
|
||||
python
|
||||
from langchain.agents import initialize_agent
|
||||
from langchain.tools import Tool
|
||||
|
||||
def run_executable(command):
|
||||
import subprocess
|
||||
result = subprocess.run(command, shell=True, capture_output=True)
|
||||
return result.stdout.decode()
|
||||
|
||||
agent = initialize_agent(
|
||||
tools=[Tool(
|
||||
name="LocalExec",
|
||||
func=run_executable,
|
||||
description="Runs local system commands"
|
||||
)],
|
||||
llm=anything_llm_instance,
|
||||
agent="structured-chat-zero-shot-react-description"
|
||||
)
|
||||
```
|
||||
|
||||
**Advantages**:
|
||||
|
||||
- Native integration with AnythingLLM's chat interface [4](https://anythingllm.com/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe)
|
||||
- Automatic command validation through LangChain's schema system
|
||||
- Multi-step workflow automation capabilities
|
||||
|
||||
---
|
||||
|
||||
## 3\. Llamafile Binary Bundling
|
||||
|
||||
Package executables with LLMs using Mozilla's Llamafile:
|
||||
|
||||
```
|
||||
bash
|
||||
# Combine executable and LLM into single binary
|
||||
llamafile-convert --exec my_script.py --model mistral-7b.gguf
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
|
||||
- Self-contained deployment without dependency conflicts [2](https://getstream.io/blog/best-local-llm-tools/) [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)
|
||||
- Cross-platform support (Windows/Linux/macOS)
|
||||
- Direct memory sharing between executable and LLM [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)
|
||||
|
||||
---
|
||||
|
||||
## Security Implementation
|
||||
|
||||
| Measure | Implementation Example |
|
||||
| --- | --- |
|
||||
| Input Sanitization | Regex whitelist for allowed CLI parameters |
|
||||
| Permission Tiers | User roles with executable access levels |
|
||||
| Sandboxing | Docker containers for risky operations |
|
||||
| Audit Logging | JSON logs of all executed commands |
|
||||
|
||||
---
|
||||
|
||||
## Use Case Workflow
|
||||
|
||||
1. User requests "Convert quarterly\_report.docx to PDF"
|
||||
2. LLM triggers `pdf_converter` plugin via structured JSON [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches)
|
||||
3. Handler executes LibreOffice CLI in Docker sandbox
|
||||
4. Output PDF path returned to LLM for user sharing
|
||||
|
||||
**Best Practices**:
|
||||
|
||||
- Use absolute paths for local executables
|
||||
- Implement timeout guards for long-running processes
|
||||
- Test with `--dry-run` flags before production deployment [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)
|
||||
|
||||
For complex automation, combine methods 2 and 3 using Llamafile-packaged tools controlled through LangChain agents [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe).
|
||||
Loading…
Add table
Add a link
Reference in a new issue