Vertex AI is a bare-bones analytics environment in Google Cloud. As simple tasks as changing the Python version requires multiple steps.
Creating a new Python environment in Vertex AI
Here is how to change Python version in Vertex AI by creating a new Python environment:
Launch a new terminal in Vertex AI:
Run the lines one by one. This way you can react to command line prompts and possible warnings:
#Deactivate the existing Python environment conda deactivate #Create new Python environment using Python version 3.11 conda create -n python311 python=3.11 -y #Active the new environment conda activate python311 #This enables pip to install to conda environment conda install pip #Verify that you use conda pip to install which -a pip #Install Python packages from requirements.txt file - Make sure you are in the same folder pip install --user -r requirements.txt #Install ipykernel if it was not requirements.txt pip install ipykernel #Install the kernel so it becomes available in notebooks ipython kernel install --name python311 --display-name "Python 3.11" --user
Open Vertex AI notebook with new Python version
After this you can use the new Python version and related packages in two ways. Either launch a new notebook or switch the kernel to an existing one.
The new Python kernel should appear in the launcher available for new notebooks.
Verify the Python and package versions in the notebooks.
You can see the Python version
3.11 on top right corner of the notebook. There you can switch the kernel for existing notebooks. Refresh the browser page if the new Python version is not visible.
Problem with Vertex AI Python environments
Vertex AI notebook platform offers only one Python environment by default.
What if you need Python
3.11 in one project instead the default
You need to create a new environment!
When to create a new Python environment in Vertex AI?
Preferably each analytics project would have their own Python virtual environment. An environment has specific Python and package versions.
Kernel is different concept than Python virtual environment. But for basic Vertex AI usage can use consider to have one kernel and environment per one project. In this regard, it would be smart to name the kernel or environment by project rather than Python version.
In the most straightforward approach each project has their own
git repository. Exact Python package version can be defined in the
Why not to have one environment for all projects?
At some point you start requiring different package versions depending on the task you work on. This happens when you project start to have plenty of code.
Keeping projects small enough is key to success in data science.