User-Managed notebooks in Vertex AI are virtual workspaces for data exploration. But they lack automatic shutdown after being idle for specific amount of time.

Vertex AI User-Managed notebooks auto shutdown from Managed notebooks

Unlike User-Managed notebooks, Managed notebooks have automatic shutdown. Managed notebooks however are meant for production workloads and are twice as expensive.

In some cases higher computation price might be insignificant compared to developer salaries. If you are using GPUs or otherwise expensive instances, auto shutdown is definitely worth you time.

Even though User-Managed notebooks do not have auto shutdown out of the box, there is a workaround. You can easily implement automatic shutdown for User Managed notebooks by a Managed notebook!

Implement auto shutdown for User-Managed notebooks

Create the cheapest Managed notebook to host your auto shutdown script for User-Managed notebooks.

Install the client library in the terminal of the notebook instance:

pip install --user google-cloud-notebooks

At the moment of writing the Google Cloud notebook Python API docs can be found here .

Here is a sample Python code for the User-Managed notebook auto shutdown:

from google.cloud import notebooks_v1

project_id="ypur-project-id"
location="europe-north1-a"

client = notebooks_v1.NotebookServiceClient()

def get_all_instances(project, location):

    request = notebooks_v1.ListInstancesRequest(
        parent=f"projects/{project_id}/locations/{location}"
    )
    page_result = client.list_instances(request=request)
    instances = list(page_result)

    #The following line would print data for all instances
    #print(instances)

    return instances

def stop_instance(instance_name):
    
    print(f"Stopping instance {instance_name}")

    request = notebooks_v1.StopInstanceRequest(
        name=instance_name
    )

    operation = client.stop_instance(request=request)

    print("Waiting for operation to complete...")

    response = operation.result()

    print(response.state)
    
   
all_instances = get_all_instances(project_id, location)
first_instance = all_instances[0]

#Double check instance name while testing to prevent accidents :)
notebook_instance_name = "your-notebook-name"
full_notebook_instance_name = f"projects/{project_id}/locations/{location}/instances/{notebook_instance_name}"
if first_instance.name == full_notebook_instance_name:
    stop_instance(first_instance.name)

Note that this method stops the User-Managed notebook even if you have an execution running! More about this in the following chapter.

Tune the script to accommodate your needs.

Finally click the schedule icon to execute the script for example every afternoon. This will use the Vertex AI executor rather than the Managed notebook instance.

You can shutdown the Managed-notebook instance as you need it only to edit the Python script above.

It is ofcourse possible to execute the same Python script from virtual machine, your laptop or anywhere else.

Disable auto shutdown for specific instances

The problem is that also currently running notebook instances are shutdown. The client library had no method to check if a Jupyter notebook cell is running.

There is no perfect way to achieve this.

One approach is to set a specific GCP resource label for those User-Managed notebook instances that shut not be shutdown. Then you need an if statement to check for the resource label in the above script.

The resource label can be set permanently from Google Cloud console. Or you can set it programmatically in the long running User-Managed notebook when the execution starts and remove the label after the execution has finished.