Today, we’re excited to announce the launch of our SQLMesh Visual Studio Code extension. It is the direct result of listening to the SQLMesh community, watching you in your element, and collaborating with you!

Your IDE

Tobiko strives to provide the best tools for data professionals. In the past, the team built a custom UI to interact with SQLMesh projects. In many ways, it’s great: it’s simple and tailored to SQLMesh. While functional, it’s been underutilized by the community for one simple reason: it pulls you out of your IDE. If you’ve been an engineer for even a brief moment, you’ve perfected your IDE. It’s your happy place. You have your keybindings, extensions, and muscle memory, and our approach requires you to leave all that. The reality is that you can’t afford to do that, and so you don’t.

To avoid this, Alex Butler, one of our SELECT TOP community members from Harness, built a SQLMesh language server and created a VS Code extension. This was before my time at Tobiko, but such a proactive contribution from the community was a clear signal of a strong preference among developers for tools that seamlessly fit into their existing workflows, minimizing context switching and maximizing productivity.

As a co-founder of Quary, we’d originally built the company with this understanding in mind, and it’s what clicked when we met Toby and the rest of the Tobiko team. SQLMesh is a magical transformation tool and with Quary’s product adapted correctly, we could meet customers better where they need us. By joining forces, we’re building an unmatched experience. 

Meet the SQLMesh VS Code Extension and Language Server

The SQLMesh Visual Studio Code extension is designed to bring the power of SQLMesh directly into the editor, where you spend all your time. It streamlines the development, understanding, and management of SQLMesh projects by providing essential functionalities at developers' fingertips. Underneath, powering these features, is a SQLMesh language server.

Our primary goal is to make editing and understanding SQLMesh projects easier and more intuitive. By integrating key SQLMesh operations and insights into VS Code, the extension reduces the need to switch between different tools and delivers a more focused and efficient development experience.

And true to Tobiko’s ethos, the extension and language server are fully open-source. It’s in our SQLMesh https://github.com/tobikoData/sqlmesh repository! In addition to publishing to the VSCode marketplace, we’re also committed to publishing the extension to OpenVSX, so if you’re using Windsurf or any of the other VSCode derivatives, the extension should work too.

Key Features

The SQLMesh VS Code extension comes packed with features. If you’re reading this after publishing, there’s probably a bunch more we have added. Most features should be familiar to you and require no extra thinking: linting, formatting, the ability to go to the definition of a model. They’ll keep you in your IDE and your flow. Others are very visual like model lineage and the ability to render a model.

Lineage at Your Fingertips

Understanding data lineage is crucial for debugging, impact analysis, and maintaining complex data projects. Our extension introduces an interactive view of model lineage directly in VS Code. You can access a dedicated Lineage tab in the panel to visualize the upstream and downstream dependencies of their SQLMesh models. This immediate visual context helps you quickly grasp data flows and relationships without leaving the editor. You can even investigate the lineage of individual columns.

Rendering a model

Macros, Jinja, blueprints…these contraptions are fabulous for reusing code, but painful to debug. Code is excruciatingly pedantic: case and character sensitive. Writing code that generates other code is hard, so we’ve added a render function in SQLMesh to help you visualize the output. Now you can render your models side by side in your IDE. 

Language server goodies

Our extension significantly enhances the SQL model editing process with several intelligent features:

  • Auto-completion: When you start typing a model name or SQL keywords, they’ll appear as you type to provide suggestions. 
  • Hover Information & Go to Definition: Hovering over a model name in the code now displays a tooltip with a summary of the model. Clicking on the model name will take you to the file containing that model's definition.
  • Inline Linter Diagnostics: When you enable the SQLMesh linter, the language server will now report issues directly within the editor. This applies to both built-in SQLMesh linter rules and any custom rules defined by the team. You’ll now see these issues in real time as you’re writing models, rather than when you are running a plan, or only in the linter afterwards. My personal favourite bit is the fact that diagnostic includes a link to the rule code. This takes you straight to the code that defines the rule, making it easy to find the rule definition responsible for linting error.
  • Integrated Formatting: The language server now also registers a formatter that will keep all your SQL code neatly formatted. Whether you like to format on save or manually trigger it - we give you the flexibility.

These editor enhancements are subtle but they’ll help keep you in the zone and hopefully less error-prone. While we aren’t launching any configurations for other IDEs, the language server can easily be used to bring the above features to another development environment.

Authentication

Last but not least, to help our Tobiko Cloud enterprise plan users, we’ve also included tcloud into our extension. You can now login, logout and manage your cloud session, right from the extension. We’ll even ask you to sign when you need to.

Getting Started with the SQLMesh VS Code Extension

Setting up the SQLMesh VS Code extension requires only a few straightforward steps: install the extension in your IDE, make sure you have the lsp extra dependencies with pip install sqlmesh[lsp], and check that VSCode is using the proper Python version. Python environment setup is important, as it's a common area where users might encounter issues with IDE extensions that depend on external tools. You can find detailed instructions in our documentation.

We're All Ears!

The SQLMesh Visual Studio Code extension is open-source and will be continually improved. There is ample opportunity for you to influence the extension's development. Please install the extension, explore its features, and share your experiences. Feedback is invaluable for identifying areas for improvement, prioritizing new features, and ensuring our extension meets your real-world needs as data practitioners. In fact, a few of you have already been using the extension, and your input has already proven crucial to pushing the product forward.

Report any problems encountered, suggest enhancements, or simply share your thoughts by creating GitHub issues in the official repository: https://github.com/tobikodata/sqlmesh/issues 

As users, you can help make the SQLMesh VS Code extension an even more powerful tool for the community - and if you really want to go nuts, feel free to contribute actual code!

Growing the VS Code extension with our open source community

The launch of the SQLMesh VS Code extension marks an exciting new chapter for SQLMesh users. It represents a significant enhancement to the developer experience, bringing data transformation capabilities directly into the preferred environment of most data professionals we’ve worked with. As always, we’re committed to listening to the community and evolving SQLMesh to meet the practical needs of our open source users.

We’ve got many things on our roadmap:

  • Column level go to definition and hovering information
  • Rendering models
  • Evaluating models and CTEs
  • Diff views

With those next steps already in motion, we’ve also thought hard about the future of our related product, the SQLMesh UI. While we aren’t deprecating it immediately, we are shifting our focus and plan to phase it out once more of our roadmap has been realized. As part of that transition, we also plan to extract the data catalog out of the UI, and turn it into a static docs site that you can generate and host yourself. 

In the meantime, install the new VS Code extension today, and let us know how you get along!