Updated on 2025-01-29

I stopped using LM Studio and switched over Ollama. You can read the shiny and brand new post here: Ollama + Continue: Open-Source is Always Better!

The realm of technology is constantly in the pursuit of innovation. Among the latest advancements, LM (Language Model) tools have revolutionized programming, making tasks seemingly effortless. But why shell out for them when these capabilities can be accessed for free? Discover LM Studio and Continue, which offer a local solution with unparalleled advantages.

The Advantages of Local LM Models

Embracing a local LM model comes with a host of benefits:

• Cost-free: No subscription fees; use it at no extra cost.
• Complete Offline Access: Work seamlessly even without an internet connection.
• Full Prompt Control: Customize prompts to tailor results to your exact needs.

Of course, it’s essential to weigh the drawbacks alongside the perks:

• Hardware Requirements: Consider the computing power needed to run these models effectively.
• Model Variability: Not all models are created in the same way; some may lack the sophistication of their proprietary counterparts.

LM Studio Configuration

Configuring LM Studio is a breeze.
Begin by downloading a preferred build from the official LM Studio website lmstudio.ai.
After installation, launch the application manually to access the Home screen.

Utilize the search bar at the center of the Home screen to locate desired models.
To make it all easier, Meta’s Code LLama model is recommended.

Searching for Code LLama will yield various results, as LM Studio can search and download available models from platforms like Hugging Face.

Each model offers different quantizations to compress its size.
Always verify model details on Hugging Face before selecting based on your hardware.
In this case, we’ll use the Code LLama 13B model.

Once downloaded, navigate to the AI Chat section of LM Studio to load the model. Additionally, LM Studio can be configured to use it as a server.

Choosing the right model is crucial, considering hardware capabilities. Opt for a model that runs smoothly on your system to avoid performance issues.

As you can see from my screenshot, it’s possible to configure LM Studio to offload operations to the GPU, though optimal settings may vary. Experiment with different configurations to find the most efficient setup for your hardware. Remember that LM Studio will run on 4 CPU cores by default.

After configuring parameters, start the Local Server via the “Local Server” section of LM Studio. Specify a port and other parameters, but note that changing the default port may cause issues with the Visual Studio Code extension we’ll use later.

Visual Studio Code Configuration

With a local model ready and waiting, it’s time to connect using the open-source Continue extension in Visual Studio Code.

After installation, configure the extension to use your LM Studio server by editing the config.json file. This file is typically found at:

• Windows: C:/Users/{user}/.continue/config.json
• Linux/Mac: ~/.continue/config.json

Add your local model to the models array, ensuring to include title, provider, and model parameters.
Set allowAnonymousTelemetry to false.

Save the file, and you’re ready to generate code locally in Visual Studio Code using your LM Studio model!

For a comprehensive understanding of the extension’s features, refer to the official project wiki here.

Conclusion

Wrapping up our dive into LM Studio and Continue, we’ve found a whole new way to rock coding without breaking the bank. These local models give you all the power without any subscription fees. And with easy setup in Visual Studio Code, you’re ready to roll.

So, let’s get coding!

Contacts

For questions or suggestions, contact: noc@balzabu.io.