iso27diy-corp/Corpus/🎇 Sparks/Instruct an LLM on available tools.md

---
tags:
  - project/iso27DIY
  - dev
  - llm
  - agent
---

[orq](https://orq.ai/blog/llm-tools)

[

15 Best LLM Tools for AI Product Development \[2025 Guide\]

](https://orq.ai/blog/llm-tools)[mercity.ai](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)

[

Comprehensive Guide to Integrating Tools and APIs with Language...

](https://www.mercity.ai/blog-post/guide-to-integrating-tools-and-apis-with-language-models)[DataCamp](https://www.datacamp.com/blog/llmops-tools)

[

Top 15 LLMOps Tools for Building AI Applications in 2025 - DataCamp

](https://www.datacamp.com/blog/llmops-tools)[softwareag](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)

[

LLM Tool Usage & API Integration - Software AG

](https://www.softwareag.com/en_corporate/blog/llm-tool-usage.html)

To instruct an LLM about available tools and how to call them, you typically follow a structured approach:

**1\. Define Available Tools**

- Provide the LLM with a list of tools, each including:
	- A name (identifier)
	- A description (explains when to use it)
	- A schema (usually JSON) describing input parameters and their types [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).

**2\. Inform the LLM via Prompting**

- When sending a user query, include the tool definitions in the prompt or system message. This allows the LLM to understand what tools are available and how to use them [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).

**3\. Tool Calling Process**

- The LLM receives the user prompt and tool list.
- It decides if a tool is needed, selects the appropriate one, and formats a tool call (often as a structured JSON object with arguments) [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).
- Your application receives this tool call, executes the corresponding function or API, and returns the result to the LLM.
- The LLM integrates the tool’s output into its next response, continuing the conversation naturally [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).

**4\. Training or Fine-Tuning (Optional)**

- For more advanced usage, you can fine-tune the LLM with example data showing how to use each tool, or use models already trained for tool use (like some recent open-weight models) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/).

**Summary Table**

| Step | Action |
| --- | --- |
| Define tools | Name, description, input schema |
| Prompt LLM | Include tool definitions in prompt/system message |
| Tool call & execution | LLM outputs tool call; app executes it and returns result |
| Integrate response | LLM uses tool output in conversation |
| (Optional) Fine-tuning | Train LLM on tool usage examples for better performance |

This approach enables LLMs to use external tools flexibly and accurately within applications [8](https://portkey.ai/blog/what-is-llm-tool-calling) [9](https://docs.anyscale.com/llms/serving/guides/tool_calling/) [6](https://cameronrwolfe.substack.com/p/teaching-language-models-to-use-tools).

I’m using Anything LLM. How can I enable a model to run shell scripts?[GitHub](https://github.com/Mintplex-Labs/anything-llm/issues/3451)

[

\[FEAT\]: Shell/Script invocation block for flow builder #3451 - GitHub

](https://github.com/Mintplex-Labs/anything-llm/issues/3451)[reddit.com](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)

[

Can I integrate my local LLM to enable it to run system commands...

](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/)[GitHub](https://github.com/regnull/how.sh)

[

how.sh - Create and execute shell commands using LLM - GitHub

](https://github.com/regnull/how.sh)[anythingllm](https://anythingllm.com/)

[

AnythingLLM | The all-in-one AI application for everyone

](https://anythingllm.com/)

To enable shell script execution in AnythingLLM, you can implement custom agent skills through these methods:

**1\. Custom Agent Skill Development**

- Create a `plugin.json` file defining your shell script entrypoint:
	```
	json
	{
	  "name": "shell_runner",
	  "description": "Execute shell commands",
	  "entrypoint": "./handler.js",
	  "permissions": ["shell"]
	}
	```
- Develop a handler file (e.g., `handler.js`) using Node.js' `child_process` module [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux):
	```
	javascript
	const { exec } = require('child_process');
	module.exports = async ({ command }) => {
	  return new Promise((resolve, reject) => {
	    exec(command, (error, stdout, stderr) => {
	      if (error) reject(error);
	      resolve(stdout || stderr);
	    });
	  });
	};
	```

**2\. Security Considerations**

- Restrict allowed commands through allow-listing in your handler
- Implement user permission tiers for script execution
- Use environment sandboxing for dangerous operations [6](https://news.ycombinator.com/item?id=35172486)

**3\. Integration Approaches**

- **LangChain Agent Method** (shown in tutorial 5):
	```
	python
	from langchain.agents import initialize_agent
	from langchain.tools import ShellTool
	shell_tool = ShellTool()
	agent = initialize_agent([shell_tool], llm, agent="zero-shot-react-description")
	```
- **Ollama Local Model Integration** [3](https://github.com/regnull/how.sh):
	```
	bash
	ollama pull deepseek-coder-v2
	llm -m ollama/deepseek-coder-v2 "convert all .txt files to .md" > convert.sh
	```

**4\. Execution Flow**

1. User requests shell operation through chat interface
2. LLM processes request and generates structured tool call
3. Handler executes the command via Node.js `exec()` [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451) [7](https://docs.anythingllm.com/installation-desktop/linux)
4. Output is returned to LLM for response formatting

**Example Use Case** (from GitHub issue [1](https://github.com/Mintplex-Labs/anything-llm/issues/3451)):

```
bash
# plugin.json entry
"entrypoint": "python -m browser_automation.py"
```

**Limitations & Risks**

- Requires careful input sanitization to prevent injection attacks
- Windows support limited due to shell differences 5
- Execution environment must match user's system capabilities [2](https://www.reddit.com/r/LocalLLaMA/comments/14cuhmm/can_i_integrate_my_local_llm_to_enable_it_to_run/) [6](https://news.ycombinator.com/item?id=35172486)

For production use, consider implementing:

- Command validation middleware
- Output sanitization filters
- Session-based execution isolation
- Activity logging for audit trails [6](https://news.ycombinator.com/item?id=35172486) [7](https://docs.anythingllm.com/installation-desktop/linux)

What are the best methods to integrate local executables with Anything LLM for automation[semaphoreci](https://semaphoreci.com/blog/local-llm)

[

6 Ways to Run LLMs Locally (also how to use HuggingFace)

](https://semaphoreci.com/blog/local-llm)[Stream](https://getstream.io/blog/best-local-llm-tools/)

[

The 6 Best LLM Tools To Run Models Locally - GetStream.io

](https://getstream.io/blog/best-local-llm-tools/)[apidog](https://apidog.com/blog/top-llm-local-tools)

[

The 5 Best LLM Tools To Run Models Locally - Apidog

](https://apidog.com/blog/top-llm-local-tools)[anythingllm](https://anythingllm.com/)

[

AnythingLLM | The all-in-one AI application for everyone

](https://anythingllm.com/)

To integrate local executables with AnythingLLM for automation, use these methods:

## 1\. Custom Plugin Development

Create executable-wrapping plugins using AnythingLLM's extensible architecture:

```
javascript
// plugin.json
{
  "name": "pdf_converter",
  "description": "Converts DOCX to PDF using LibreOffice",
  "entrypoint": "./convert.js",
  "permissions": ["file_system"]
}

// convert.js
const { execSync } = require('child_process');

module.exports = async ({ inputPath }) => {
  try {
    execSync(\`soffice --convert-to pdf "${inputPath}" --headless\`);
    return { success: true };
  } catch (error) {
    return { error: error.message };
  }
};
```

**Key Features**:

- Direct CLI execution via Node.js `child_process` [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)
- File system access controls through permission tiers
- Error handling integration with LLM feedback loops

---

## 2\. LangChain Agent Integration

Leverage LangChain's tool ecosystem for executable orchestration:

```
python
from langchain.agents import initialize_agent
from langchain.tools import Tool

def run_executable(command):
    import subprocess
    result = subprocess.run(command, shell=True, capture_output=True)
    return result.stdout.decode()

agent = initialize_agent(
    tools=[Tool(
        name="LocalExec",
        func=run_executable,
        description="Runs local system commands"
    )],
    llm=anything_llm_instance,
    agent="structured-chat-zero-shot-react-description"
)
```

**Advantages**:

- Native integration with AnythingLLM's chat interface [4](https://anythingllm.com/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe)
- Automatic command validation through LangChain's schema system
- Multi-step workflow automation capabilities

---

## 3\. Llamafile Binary Bundling

Package executables with LLMs using Mozilla's Llamafile:

```
bash
# Combine executable and LLM into single binary
llamafile-convert --exec my_script.py --model mistral-7b.gguf
```

**Benefits**:

- Self-contained deployment without dependency conflicts [2](https://getstream.io/blog/best-local-llm-tools/) [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)
- Cross-platform support (Windows/Linux/macOS)
- Direct memory sharing between executable and LLM [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/)

---

## Security Implementation

| Measure | Implementation Example |
| --- | --- |
| Input Sanitization | Regex whitelist for allowed CLI parameters |
| Permission Tiers | User roles with executable access levels |
| Sandboxing | Docker containers for risky operations |
| Audit Logging | JSON logs of all executed commands |

---

## Use Case Workflow

1. User requests "Convert quarterly\_report.docx to PDF"
2. LLM triggers `pdf_converter` plugin via structured JSON [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches)
3. Handler executes LibreOffice CLI in Docker sandbox
4. Output PDF path returned to LLM for user sharing

**Best Practices**:

- Use absolute paths for local executables
- Implement timeout guards for long-running processes
- Test with `--dry-run` flags before production deployment [5](https://deepgram.com/learn/local-llm-chatbot-that-can-run-code-searches) [8](https://www.reddit.com/r/LocalLLaMA/comments/1ckw7en/what_software_do_you_use_to_interact_with_local/)

For complex automation, combine methods 2 and 3 using Llamafile-packaged tools controlled through LangChain agents [6](https://blog.lamatic.ai/guides/how-to-run-llm-locally/) [7](https://www.aifire.co/p/top-8-local-llm-tools-run-ai-models-offline-and-keep-your-data-safe).