Skip to content

Using External Text Generation Tools

For advanced users, connecting DREAMIO to an external text generation tool can offer more customization, control, and access to a wider variety of models. This method involves running a separate program on your computer to serve the AI model, and then connecting DREAMIO to it.

The most popular tools for this are Ollama, LM Studio, and koboldcpp.

Setting up the External Tools

First, you'll need to install and configure one of the following applications.

Ollama

Ollama is known for its simplicity and ease of use.

  • Installation:
    1. Download the Ollama installer for Windows from the official website.
    2. Run the downloaded .exe file and follow the installation instructions. Ollama will run in the background.
  • Downloading a Model:
    1. Open the Command Prompt on your Windows PC.
    2. To download a model, use the command ollama run <model_name>. For example, to download the gemma3:12b model, you would type ollama run gemma3:12b and press Enter. A list of available models can be found on the Ollama website.
  • Running the Model:
    • Once a model is downloaded, it will be running automatically. You can interact with it directly in the command prompt to ensure it's working.

LM Studio

LM Studio provides a user-friendly graphical interface for managing and running local models.

  • Installation:
    1. Download the LM Studio installer from the official website.
    2. Run the installer and follow the setup instructions.
  • Downloading a Model:
    1. Open LM Studio. On the main screen, you can search for models from Hugging Face.
    2. Find a model you'd like to try (look for GGUF formats) and click the "Download" button.
  • Running the Model:
    1. Go to the "Local Server" tab (it may be represented by an icon that looks like <>).
    2. At the top, select the model you downloaded.
    3. Click the "Start Server" button.

koboldcpp

koboldcpp is a lightweight and powerful tool that's popular in the AI storytelling community.

  • Installation:
    1. Go to the koboldcpp GitHub releases page.
    2. Download the latest koboldcpp.exe file.
  • Running a Model:
    1. You don't need to install anything. Just double-click the koboldcpp.exe file.
    2. A window will pop up allowing you to browse for and select the GGUF model file you downloaded from Hugging Face.
    3. After selecting your model, koboldcpp will launch a server.

Using an OpenAI-Compatible API

In addition to the local tools mentioned above, DREAMIO can connect to any text generation service that offers an OpenAI-compatible API. This allows for great flexibility, as you can use cloud-based providers or other local tools that follow the OpenAI standard.

Some examples of services you can connect to include:

  • Cloud-based Services: Together.ai, Anyscale, OpenRouter, and others that provide an OpenAI-compatible endpoint.
  • Local Tools: Any local inference server (like FastChat or vLLM) that exposes an OpenAI-compatible endpoint.

Configuring DREAMIO for an External Tool

Once you have your chosen external tool running and a model loaded, follow these steps in DREAMIO:

  1. Go to the Text Generation Settings.
  2. Select the "Custom" provider option.
  3. You will need to enter a Base URL and an Endpoint. Here are the default values for each tool:
    • For LM Studio:
      • Base URL: http://localhost:1234
      • Endpoint: /completions (Recommended)
    • For koboldcpp:
      • Base URL: http://localhost:5001
      • Endpoint: /completions (Recommended)
    • For Ollama:
      • Base URL: http://localhost:11434
      • Endpoint: Ollama
    • For other OpenAI-compatible endpoints BaseURL should be obtained through their documentation.
  4. (Optional) Paste the model name into the Model field.
  5. After entering the correct information, you should be good to go. The game will now communicate with the external tool to generate text.