· 5 min read

Classloader isolation 101

Separation of concerns for your Gradle plugins

Separation of concerns for your Gradle plugins

If you’re building an Android or Kotlin app, you must surely be using a bunch Gradle plugins. Plugins are reusable pieces of code that define how your build behaves.

Plugins are generally a good thing. Just like regular libraries, they make it easier to develop build logic without having to reinvent the wheel! This is good!

Unlike regular Android or Kotlin libraries, there’s a bit of magic involved. As I wrote back in 2019, there are a lot of different ways to load your plugins.

To simplify the complex classloader hierarchy, you are probably using apply(false) or another trick to ensure your plugins are loaded in a single, central classloader.

But even with the apply(false) trick and a central classpath, you may still bump into two issues:

  • Dependency conflicts.
    • This is your usual dependency hell problem, a widespread and long-standing issue of any software stack, just made slightly worse by plugins having some illusion of isolation and relying on unstable dependencies more easily.
  • Unnecessary build invalidation.
    • This one is a bit more subtle and more specific to Gradle. Because you have a single classpath for all your plugins where all the work is executed, all the build needs to be invalidated whenever you change something in this classpath, which slows down your debugging loop and CI.

This is where classloader isolation helps.

Isolation to the rescue

By isolating your plugin work in a separate classloader, you can avoid both these issues.

Note that it’s about isolating the plugin work, not the plugin itself.

At the end of the day, your build needs to interact with your plugin’s API and therefore needs to access it. But your build logic doesn’t really need to know anything about antlr, kotlinpoet or kotlin-compiler-embeddable.

By moving your work to a separate classloader, you avoid both the aforementioned issues.

  • You do not risk dependency conflicts anymore.
  • You can update your work without invalidating your whole build.

As a bonus, you may even be able to update your build without invalidating your work (but that’s another story).

How do we achieve this? I’ll share two different solutions, one more traditional, which you will come across most often. Another one I have been working on, which I believe has interesting properties.

Let’s dive in.

Classloader isolation using Workers

In Gradle 5, Gradle introduced the Worker API.

The Worker API was introduced in order to allow:

I’ll defer to the official documentation for the details, but the gist of it is this:

interface CookingParameters : WorkParameters {
  val recipe: RegularFileProperty
  val ingredients: RegularFileProperty
  val outputFile: RegularFileProperty
}

abstract class CookingAction : WorkAction<CookingParameters> {
  override fun execute() {
    val recipe = parameters.recipe.get().asFile
    val ingredients = parameters.ingredients.get().asFile

    val meal = cook(recipe, ingredients)
    outputFile.get().asFile.writeText(meal)
  }
}

abstract class CookingTask : DefaultTask() {
  @get:InputFile
  abstract val recipe: RegularFileProperty

  @get:InputFile
  abstract val ingredients: RegularFileProperty

  @get:OutputFile
  abstract val outputFile: RegularFileProperty

  @get:InputFiles
  abstract val myClasspath: ConfigurableFileCollection

  @get:Inject
  abstract fun getWorkerExecutor(): WorkerExecutor

  @TaskAction
  fun cook() {
    val workQueue = getWorkerExecutor().classLoaderIsolation {
      classpath.from(myClasspath)
    }

    workQueue.submit(MyWorkAction::class.java) {
      parameters.recipe.set(recipe)
      parameters.ingredients.set(ingredients)
      parameters.outputFile.set(outputFile)
    }
  }
}

This works pretty well and is what you will most often while browsing the internet. It also has a few drawbacks:

  • there is a bunch of passing around the same input/outputs from the Task to the WorkAction.
  • the isolation is not 100% effective. Some of your main build classpath is still present in the WorkAction classloader. The embedded version of kotlin-stdlib too, meaning you can’t use a newer version of Kotlin in your plugin work.
  • managing the different classloaders properly requires some non-trivial compileOnly and Configuration manipulations in your build scripts (not shown here).
  • it may leak sometimes.

After writing Workers manually for a couple of years and forgetting to pass some inputs around, or bumping into classloader leaks, I decided to investigate what could be done to make this process easier.

Classloader isolation using Gratatouille

Gratatouille is a framework to build Gradle plugins in Kotlin. It uses KSP to generate tasks and workers from pure Kotlin functions.

In addition to that, it provides a classloader isolation mode by creating brand new classloaders for your plugin work and caching them in a shared BuildService.

To do this, Gratatouille requires that you publish two different artifacts for your plugin:

  • my-plugin-wiring contains the Gradle plugin and wiring and should have very few dependencies.
  • my-plugin-tasks contains the work, and may have any number of dependencies, including kotlin-stdlib.

Define your work using a @GTask function:

@GTask
internal fun cook(
    recipe: GInputFile,
    ingredients: GInputFile,
    outputFile: GOutputFile
) { 
  outputFile.writeText(cook(recipe, ingredients))
}

Use the com.gradleup.gratatouille.tasks plugin to generate the Gradle tasks and workers:

// my-plugin-tasks/build.gradle.kts
plugins {
  id("com.gradleup.gratatouille.tasks")
}

dependencies {
  // Add dependencies needed to do your task work
  implementation("org.jetbrains.kotlin:kotlin-compiler-embeddable:2.2.0")
  // do **not** add gradleApi() here
}

gratatouille {
  // Enable code generation
  codeGeneration {
    // Enables classloader isolation
    classLoaderIsolation()
  }
}

In a separate module, you can now depend on your work using the gratatouille configuration:

// my-plugin-wiring/build.gradle.kts
plugins {
    id("com.gradleup.gratatouille.wiring")
}

dependencies {
  gratatouille(project(":my-plugin-tasks"))
  
  // Add the version of Gradle you want to compile against 
  compileOnly("dev.gradleplugins:gradle-api:8.0")
}

You can now use your generated tasks from your plugin code:

abstract class MyPlugin : Plugin<Project> {
  override fun apply(project: Project) {
    project.registerCookTask(
      recipe = TODO(),
      ingredients = TODO()
    ) 
  }
}

With Gratatouille, you can:

  • avoid the boilerplate coming with the Worker API.
  • use the version of kotlin-stdlib you’d like in your plugin work.
  • write plugin work as regular Gradle projects.

There is a bit of initial setup work, but in the long run, you have the guarantee that all your plugin code is run isolated from the rest of your build and won’t trigger unnecessary invalidations.

Closing thoughts

Whether you’re using the Worker API or Gratatouille, I would recommend isolating your plugin work.

The more plugins isolate their work, the less likely you are to run into dependency conflicts.

Bundled with central classpath management, this could ultimately lead to true plugin isolation and safer and faster build systems for everyone.

Photo from Rob Mowe

    Share:
    Back to Blog