Troubleshoot AI Lab¶
This section helps you identify and solve some issues when using AI Lab.
“Close without saving” error when opening a Jupyter Notebook document¶
A close without saving error occurs when you try to open a notebook document in JupyterLab.
Possible Causes¶
Cause 1: git merge conflicts add non-json lines to the document.
Cause 2: the document is edited in other editors.
Solution¶
To fix Cause 1 or Cause 2:
Right select the notebook document in JupyterLab from the File Browser, and select Open With > Editor.
Delete the non-json lines in the document and save.
“File Load Error for {filename}.ipynb Unhandled error” when opening a Jupyter Notebook document¶
A File Load Error for {filename}.ipynb Unhandled error occurs when you try to open a Jupyter Notebook document in JupyterLab. {Filename} refers to the name of the document that you try to open.
Cause¶
This is a history database issue.
Solutions¶
You can fix this issue by either Solution 1 or Solution 2:
Solution 1: remove the nbsignatures files by the following steps:
Open a terminal in Notebook.
Enter the following command and restart the Notebook instance:
rm ~/.local/share/jupyter/nbsignatures.db
Solution 2: remove the history.sqlite file by the following steps:
Locate the history.sqlite file at
~/.ipython/profile_default/history.sqliteRemove or rename the file, and restart the Notebook instance.
“‘_xsrf’ argument missing from post” error when saving a document in Notebook¶
An '_xsrf' argument missing from post error occurs when you try to save a document in Notebook.
Cause¶
This issue might occur due to the long running time of Notebook.
Solutions¶
Solution 1: refresh the notebook.
Solution 2: open a non-running existing Notebook on the same kernel, then go back to the document and save again.
Fail to find the model code files in AI pipeline¶
After you run the eap-notebook push command in Notebook to upload model code files to the internal storage, you cannot find the corresponding files when configuring NotebookEx/PythonEX/ShellEx operators in AI Pipelines.
Possible Causes¶
Cause 1: you might have selected the incorrect directory for the workspace parameter of NotebookEx/PythonEX/ShellEx operators.
Cause 2: the model code files are not uploaded successfully.
Solutions¶
For cause 1: Double-check the workspace parameter of NotebookEx/PythonEX/ShellEx operators to ensure the value is the directory of the uploaded model files.
For cause 2: Upload the model code files again. If you see
Object 'filename' already exists. Overwrite? [Y/n], enterYto overwrite the file. If you want to overwrite all model code files, use the following command to upload the model code files:eap-notebook push -p -y
For more information, see Manage Model Code.
Notebook kernel restarts automatically¶
When you edit a document in JupyterLab Notebook, Notebook kernel restarts automatically with the following pop-up caution:
The kernel for {filename}.ipynb appears to have died. It will restart automatically.
Possible Causes¶
Cause 1: CPU or memory is full.
Cause 2: Python package is out of date. You can select the debug icon to check the version of Python package.
Solutions¶
Check the memory usage at the bottom of the Notebook interface when the pop-up caution appears.
If the resources are full, increase the Notebook instance resources in Instance Details page. For more information, see Manage Notebook Instances.
If the resources are not full, update the Python package by
conda installorpip installcommand. For more information on package management, see Package Management.
“TypeError: can’t pickle _thread.lock objects” error when using “log_model” on a model created via Mlflow Model Wrapper¶
A TypeError: can't pickle _thread.lock objects error occurs when you use the log_model function on a model that is created via Mlflow Model Wrapper.
Cause¶
You use logger functions in the sub-functions of Mlflow Model Wrapper, which is not supported in Mlflow 1.31.1 or Cloudpickle 1.6.0/2.0.0.
Solutions¶
Remove the logger functions Mlflow Model Wrapper.